Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/tahmid-saj/fin-predict

Finance prediction app to view historical and current market data, weekly predictions on BTC / stocks / forex, and get support from AI driven services. Developed with Spring Boot, Maven, Thymeleaf, Django, AWS, GCP, Docker, MySQL.
https://github.com/tahmid-saj/fin-predict

aws bitcoin bootstrap django forecasting-models gcp java maven ml mysql spring-boot thymeleaf

Last synced: 2 days ago
JSON representation

Finance prediction app to view historical and current market data, weekly predictions on BTC / stocks / forex, and get support from AI driven services. Developed with Spring Boot, Maven, Thymeleaf, Django, AWS, GCP, Docker, MySQL.

Awesome Lists containing this project

README 

        
# [financeprediction.io](http://financeprediction.io/)





Finance prediction app to view historical and current market data, weekly predictions on BTC / stocks / forex, and get support from AI driven services. Developed with Spring Boot, Maven, Thymeleaf, Django, AWS, GCP, Docker, MySQL.








The structure of the codebase is as follows:



```

main/

├── java/

│   └── com/

│       └── ts/

│           └── finpredict/

│               ├── assets/

│               ├── FinPredict/

│               │   ├── config/

│               │   │   └── PredictorEntityConfig.java

│               │   ├── controller/

│               │   │   ├── about/

│               │   │   ├── advice/

│               │   │   │   └── AdviceWorker.java

│               │   │   ├── chatbot/

│               │   │   │   └── ChatbotWorker.java

│               │   │   ├── FinPredictController.java

│               │   │   ├── marketdata/

│               │   │   │   └── MarketDataWorker.java

│               │   │   ├── predictor/

│               │   │   │   └── PredictorWorker.java

│               │   │   └── requests/

│               │   │       ├── AdviceRequests.java

│               │   │       ├── ChatbotRequests.java

│               │   │       ├── MarketDataRequests.java

│               │   │       └── PredictorRequests.java

│               │   ├── FinPredictApplication.java

│               │   └── model/

│               │       ├── dao/

│               │       │   ├── PredictorDailyDAO.java

│               │       │   ├── PredictorDailyDAOImpl.java

│               │       │   ├── PredictorWeeklyDAO.java

│               │       │   └── PredictorWeeklyDAOImpl.java

│               │       ├── entity/

│               │       │   ├── Advice.java

│               │       │   ├── Chatbot.java

│               │       │   ├── MarketData.java

│               │       │   ├── Predictor.java

│               │       │   ├── PredictorDailyEntity.java

│               │       │   └── PredictorWeeklyEntity.java

│               │       └── service/

│               │           ├── PredictorDailyService.java

│               │           ├── PredictorDailyServiceImpl.java

│               │           ├── PredictorWeeklyService.java

│               │           └── PredictorWeeklyServiceImpl.java

│               └── util/

│                   ├── errors/

│                   │   └── shared/

│                   │       ├── PageErrorResponse.java

│                   │       ├── PageExceptionHandler.java

│                   │       └── PageNotFoundException.java

│                   ├── helpers/

│                   └── sql/

│                       ├── predictor-current-day-ddl.sql

│                       └── predictor-current-week-ddl.sql

└── resources/

    ├── application.properties

    ├── static/

    │   ├── css/

    │   │   ├── about.css

    │   │   ├── advice.css

    │   │   ├── market.css

    │   │   └── predictor.css

    │   ├── pages/

    │   │   └── index.html

    │   └── shared/

    │       └── index.css

    └── templates/

        ├── about/

        │   └── about.html

        ├── advice/

        │   └── advice.html

        ├── marketdata/

        │   └── marketdata.html

        └── predictor/

            └── predictor.html

```



  image







[Figure 1: High level view](https://whimsical.com/fin-predict-9UPaQyGHkRt5NYyrn1LJh5)








### The application consists of the following main components:



1. __Client__: Springboot frontend web application which sends requests to the __Finance Predict API__. Currently the frontend is deployed using Docker on GCP App Engine.

2. __Finance Predict API__: Finance prediction API developed using django and deployed to AWS EC2. API serves requests for finance prediction, market data queries and chatbot queries.

3. __Services__: The following services are used:

* OpenAI: Services chatbot requests

* polygon.io: Provides market data for requests from the __Finance Predict API__

5. __Machine learning__: ML models for BTC and stock predictions developed using TensorFlow / PyTorch. [ml-job-scheduler](https://github.com/tahmid-saj/ml-job-scheduler) handles the automated job runs from preprocessing, training, predictions, postprocessing, etc. The logs of the job runs are stored in S3 and the data is stored in MongoDB.

6. __Databases__: ML prediction data is stored within MongoDB.

7. __Data engineering__: Performs manual data migration using an external ETL / ELT API developed in Go, [etl-elt-api](https://github.com/tahmid-saj/etl-elt-api)

8. __Security__: AWS security services (AWS Inspector and GuardDuty) which monitors the security of APIs. Later, data and logs from the services are queried and viewed using an external security tool.

9. __Monitoring__: Monitoring service which collects data and logs from APIs using CloudWatch, then stores them in S3 to be viewed as a dashboard via an external monitoring tool. 

10. __Notifications__: Receives various data and logs in S3 buckets and later sends emails (on issues or failures) using SQS and Lambda.



### Setting up the development environment:



1. __Cloning the repository__: You would first need to clone this repository on the host you want to set up your development environment:

```shell

git clone https://github.com/tahmid-saj/fin-predict.git

``` 

2. __Installing dependencies__: The maven dependencies are provided in the __pom.xml__ file. They wil only need to be installed using a valid IDE such as IntelliJ IDEA or NetBeans.

3. __API__: Client requests to go a the __Finance Predict API__ which provides the following:

* Finance prediction

* Historic market data

* Chatbot responses

4. __Services__: API keys are used from the following services in the __Finance Predict API__:

* OpenAI: Services chatbot requests

* polygon.io: Provides market data for requests from the __Finance Predict API__

5. __Databases__: A MongoDB cluster and collections are created, and a connection is established with the __Finance Predict API__.

6. __Data engineering__: The set up for the development environment for the data engineering tool can be found [here](https://github.com/tahmid-saj/etl-elt-api).

7. __Machine learning__: The set up for the development environment for the ML job scheduler can be found [here](https://github.com/tahmid-saj/ml-job-scheduler).

8. __AWS__: Setting up the AWS services is an optional step as this is on a development environment. However, the same services could be used to create the tools mentioned in the high level view.

9. __Running the client__: The client can be run using an IDE or Docker using:

* To package your application with Maven run:

```shell

mvn clean package

```

* Test the jar file:

```shell

java -jar target/.jar

```

* Build the docker image:

```shell

docker build -t  .

```

* Tag the docker image:

```shell

docker tag  /

```

* Push the docker image to Docker Hub:

```shell

docker push /:

```

After the image is pushed, it can be used in GCP App Engine or with a docker container in the development environment.