{"id":14978792,"url":"https://github.com/marwandebbiche/post-tuto-deployment","last_synced_at":"2025-04-05T12:07:21.338Z","repository":{"id":122111860,"uuid":"214425904","full_name":"MarwanDebbiche/post-tuto-deployment","owner":"MarwanDebbiche","description":"Build and deploy a machine learning app from scratch  🚀","archived":false,"fork":false,"pushed_at":"2023-05-22T22:32:59.000Z","size":62199,"stargazers_count":391,"open_issues_count":6,"forks_count":102,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-04-05T12:07:10.589Z","etag":null,"topics":["api","aws","character-level-cnn","deployment","docker","machine-learning","pytorch","scraping","scrapy","selenium"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MarwanDebbiche.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-11T12:04:56.000Z","updated_at":"2025-03-28T23:20:03.000Z","dependencies_parsed_at":null,"dependency_job_id":"3729f2e2-d5f3-4366-9fe0-96b93c95ff01","html_url":"https://github.com/MarwanDebbiche/post-tuto-deployment","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarwanDebbiche%2Fpost-tuto-deployment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarwanDebbiche%2Fpost-tuto-deployment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarwanDebbiche%2Fpost-tuto-deployment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarwanDebbiche%2Fpost-tuto-deployment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MarwanDebbiche","download_url":"https://codeload.github.com/MarwanDebbiche/post-tuto-deployment/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247332609,"owners_count":20921853,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","aws","character-level-cnn","deployment","docker","machine-learning","pytorch","scraping","scrapy","selenium"],"created_at":"2024-09-24T13:58:24.844Z","updated_at":"2025-04-05T12:07:21.318Z","avatar_url":"https://github.com/MarwanDebbiche.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# End 2 End Machine Learning : From Data Collection to Deployment 🚀 \n\nIn this job, I collaborated with \u003ca href=\"https://github.com/ahmedbesbes\"\u003eAhmed BESBES\u003c/a\u003e\n\nMedium post \u003ca href=\"https://medium.com/@ahmedbesbes/end-to-end-machine-learning-from-data-collection-to-deployment-ce74f51ca203\"\u003ehere\u003c/a\u003e.\n\nYou may also read about it \u003ca href=\"https://ahmedbesbes.com/end-to-end-ml.html\"\u003ehere\u003c/a\u003e and \u003ca href=\"https://marwandebbiche.com/posts/e2e-ml/\"\u003ehere\u003c/a\u003e.\n\nIn this post, we'll go through the necessary steps to build and deploy a machine learning application. This starts from data collection to deployment; and the journey, you'll see, is exciting and fun. 😀\n\nBefore we begin, let's have a look at [the app](https://www.reviews.ai2prod.com/) we'll build:\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"./blog-post/assets/app.gif\"  style=\"margin:15px\"\u003e\n\u003c/p\u003e\n\nAs you see, this web app allows a user to evaluate random brands by writing reviews. While writing, the user will see the sentiment score of his input updating in real-time, alongside a proposed 1 to 5 rating.\n\nThe user can then change the rating in case the suggested one does not reflect his views, and submit.\n\nYou can think of this as a crowd sourcing app of brand reviews, with a sentiment analysis model that suggests ratings that the user can tweak and adapt afterwards.\n\nTo build this application, we'll follow these steps:\n\n- Collecting and scraping customer reviews data using `Selenium` and `Scrapy`\n- Training a deep learning sentiment classifier on this data using `PyTorch`\n- Building an interactive web app using `Dash`\n- Setting a `REST API` and a `Postgres` database\n- Dockerizing the app using `Docker Compose`\n- Deploying to `AWS`\n\n\u003chr\u003e\n\n## Project architecture \n\n### Run the app locally\n\n\nTo run this project locally using `Docker Compose` `run`: \n\n```\ndocker-compose build\ndocker-compose up\n```\nYou can then access the dash app at [http://localhost:8050](http://localhost:8050)\n\n### Development\n\nIf you want to contribute to this project and run each service independently:\n\n#### Launch API\n\nIn order to launch the API, you will first need to run a local `postgres` db using `Docker`:\n\n```\ndocker run --name postgres -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=password -e POSTGRES_DB=postgres -p 5432:5432 -d postgres\n```\n\nThen you'll have to type the following commands:\n\n```shell\ncd src/api/\npython app.py\n```\n\n#### Launch Dash app\n\nIn order to run the `dash` server to visualize the output:\n\n```shell\ncd src/dash/\npython app.py\n```\n\n\n### How to contribute 😁\n\nFeel free to contribute! Report any bugs in the [issue section](https://github.com/MarwanDebbiche/post-tuto-deployment/issues).\n\nHere are the few things we noticed, and wanted to add.\n\n- [ ] Add server-side pagination for Admin Page and `GET /api/reviews` route.\n- [ ] Protect admin page with authentication.\n- [ ] Either use [Kubernetes](https://kubernetes.io) or [Amazon ECS](https://aws.amazon.com/ecs) to deploy the app on a cluster of containers, instead of on one single EC2 instance.\n- [ ] Use continuous deployment with [Travis CI](https://travis-ci.org)\n- [ ] Use a managed service such as [RDD](https://aws.amazon.com/rds/) for the database\n\n\n### Licence\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarwandebbiche%2Fpost-tuto-deployment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarwandebbiche%2Fpost-tuto-deployment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarwandebbiche%2Fpost-tuto-deployment/lists"}