{"id":19500024,"url":"https://github.com/anuprshetty/fynd_e2e_solution","last_synced_at":"2026-04-13T19:31:40.628Z","repository":{"id":177135508,"uuid":"525251475","full_name":"anuprshetty/fynd_e2e_solution","owner":"anuprshetty","description":"Fynd onboarding demo project.","archived":false,"fork":false,"pushed_at":"2024-05-10T11:24:14.000Z","size":126,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-25T21:22:34.744Z","etag":null,"topics":["asynchronous-programming","docker","eventloop","fastapi-framework","kafka","marshmallow","mongodb","python","redis","sanic-framework","solr-search"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/anuprshetty.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-16T06:07:32.000Z","updated_at":"2024-05-10T15:06:28.000Z","dependencies_parsed_at":null,"dependency_job_id":"e5fdf45b-9317-4042-a9bf-b8778ff44051","html_url":"https://github.com/anuprshetty/fynd_e2e_solution","commit_stats":null,"previous_names":["anuprshetty/fynd_e2e_solution"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/anuprshetty/fynd_e2e_solution","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anuprshetty%2Ffynd_e2e_solution","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anuprshetty%2Ffynd_e2e_solution/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anuprshetty%2Ffynd_e2e_solution/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anuprshetty%2Ffynd_e2e_solution/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/anuprshetty","download_url":"https://codeload.github.com/anuprshetty/fynd_e2e_solution/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anuprshetty%2Ffynd_e2e_solution/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31768635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-13T15:25:13.801Z","status":"ssl_error","status_checked_at":"2026-04-13T15:25:09.162Z","response_time":93,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asynchronous-programming","docker","eventloop","fastapi-framework","kafka","marshmallow","mongodb","python","redis","sanic-framework","solr-search"],"created_at":"2024-11-10T22:07:15.137Z","updated_at":"2026-04-13T19:31:40.604Z","avatar_url":"https://github.com/anuprshetty.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fynd End To End Solution\n\nFynd onboarding demo project.\n\n## FastAPI\n\n- [documentation](https://fastapi.tiangolo.com)\n- [source code](https://github.com/tiangolo/fastapi)\n- FastAPI stands on the shoulders of giants:\n  - [Starlette for the web parts](https://www.starlette.io/) - Starlette is a lightweight ASGI framework/toolkit, which is ideal for building async web services in Python.\n  - [Pydantic for the data parts](https://pydantic-docs.helpmanual.io/) - pydantic enforces type hints at runtime, and provides user-friendly errors when data is invalid.\n- uvicorn --\u003e ASGI server (Asynchronous Server Gateway Interface)\n\n- commands:\n  - pip install fastapi\n  - pip install \"uvicorn[standard]\"\n  - uvicorn app:app --reload\n\n## Sanic\n\n- Next generation Python web server/framework | Build fast. Run fast.\n- To provide a simple way to get up and running a highly performant HTTP server that is easy to build, to expand, and ultimately to scale.\n- [documentation](https://sanic.dev/en/guide/)\n- [source code](https://github.com/sanic-org/sanic)\n\n- commands:\n  - pip install sanic\n  - sanic server.app --port \u003cport_number\u003e\n\n## marshmallow\n\n- A Python Object Serialization and Deserialization Library.\n- marshmallow is an ORM/ODM/framework-agnostic library for converting complex datatypes, such as objects, to and from native Python datatypes.\n- An ORM maps between an Object Model and a Relational Database. An ODM maps between an Object Model and a Document Database. MySQL is not an ORM, it's a Relational Database, more specifically, a SQL Database. MongoDB is not an ODM, it's a Document Database.\n- xyz is 'Framework agnostic' simply means that xyz does not depend on any framework. It is a great and much required idea that focuses on building libraries/components which are not dependent on any specific framework for their implementation, rather to develop a generic stuff to cater everyone.\n- [documentation](https://marshmallow.readthedocs.io/en/stable/)\n\n- commands:\n  - pip install marshmallow\n\n## MongoDB\n\n- [documentation](https://www.mongodb.com/docs/)\n- [installation](https://www.mongodb.com/docs/manual/tutorial/install-mongodb-on-os-x/)\n- [basics tutorial](https://www.youtube.com/watch?v=ofme2o29ngU)\n- [mongodb aggregation framework](https://www.youtube.com/watch?v=A3jvoE0jGdE\u0026list=PLWkguCWKqN9OwcbdYm4nUIXnA2IoXX0LI)\n- [mongodb GUI](https://robomongo.org/)\n- upsert --\u003e The term upsert is a portmanteau – a combination of the words “update” and “insert.” In the context of relational databases, an upsert is a database operation that will update an existing row if a specified value already exists in a table, and insert a new row if the specified value doesn't already exist.\n\n- commands:\n  - mongosh\n  - cls\n  - exit\n  - show dbs\n  - db\n  - use \u003cdb_name\u003e\n  - db.dropDatabase()\n  - show collections\n  - db.collection_name.insertOne({name: \"Anup\"})\n  - db.collection_name.insertMany([{name: \"Anup\"}, {name: \"Ravi\", age: 25}])\n  - db.collection_name.find()\n  - db.collection_name.find().limit(2)\n  - db.collection_name.find().sort({name: -1, age: 1}).limit(2)\n  - db.collection_name.find().skip(1).limit(2)\n  - db.collection_name.find({name: \"Anup\"})\n  - db.collection_name.findOne({name: \"Anup\"})\n  - db.collection_name.countDocuments({name: \"Anup\"})\n  - db.collection_name.find({name: \"Anup\"}, {name: 1, age: 1, \\_id: 0})\n  - db.collection_name.find({name: {$ne: \"Anup\"}})\n  - db.collection_name.updateOne({\\_id: ObjectId(\"6303d1a6ee47664d92572f88\")}, {$set: {age: 27}})\n  - db.collection_name.updateMany({age: 26}, {$set: {age: 27}})\n  - db.collection_name.replaceOne({age: 26}, {name: \"Ravi\"})\n  - db.collection_name.deleteOne({age: 26})\n\n## Apache Kafka\n\n- [documentation](https://kafka.apache.org/documentation/)\n- [configuring Kafka for access across networks](https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/)\n- [kafdrop - kafka web ui](https://github.com/obsidiandynamics/kafdrop)\n- [kafka-shell](https://betterdatascience.com/master-the-kafka-shell-in-5-minutes-topics-producers-and-consumers-explained/)\n- [kafka-python](https://timber.io/blog/hello-world-in-kafka-using-python/)\n- [kafka-fastapi](https://www.youtube.com/watch?v=l5NOe3jTEso)\n- [kafka-sanic](https://github.com/naruvimama/falcon)\n- Apache Kafka is a distributed event store and stream-processing platform. A high-throughput distributed messaging system. Tool for building real time data pipelines. streaming/queueing/messaging system.\n- It helps in decoupling of data streams and systems.\n- Use cases:\n  - Messaging System\n  - Activity Tracking\n  - Gather metrics from many locations\n  - Application logs gathering\n  - Stream processing (with Kafka streams API or Spark)\n  - Decoupling of system dependencies\n  - It provides connectors to import and export bulk data from databases and other systems.\n  - Integration with Spark, Flink, Storm, Hadoop, and many more Big Data Technologies.\n- Terminologies:\n  - Producer --\u003e An application that sends messages to Kafka.\n  - Message --\u003e Small to medium-sized piece of data. For Kafka, message is just an array of bytes.\n  - Consumer --\u003e An application that reads data from Kafka.\n  - Broker --\u003e Kafka Server\n  - cluster --\u003e A group of computers sharing workload for a common purpose. Kafka is a distributed system. So Kafka cluster has group of computers each executing one instance of Kafka broker.\n  - Topic --\u003e A topic is a unique name for Kafka stream. Kafka stores a stream of records in categories called topics. Each record consists of a key, value and a timestamp.\n  - Partition:\n    - Broker will store the data for a topic. This data can be huge. It may be larger than the storage capacity of a single computer. In this case, broker has a challenge in storing that data.\n    - One of the obvious solution is to break the data into two or more parts (partitions) and distribute them into multiple computers.\n    - When we create a topic, we need to tell the number of partitions to create. And Kafka broker will create that many partitions for the topic. every partition sits on a single machine. You can not further break that partition again.\n  - offset --\u003e A sequence id given to messages as they arrive in a partition. These numbers once assigned, they never change. They are immutable. offset starts from 0. Offsets are local to the partition.\n  - Global unique identifier of a message = Topic name + Partition number + Offset\n  - consumer group --\u003e A group of consumers acting as a single logical unit.\n- References:\n  - [integrating Apache Kafka With Python Asyncio Web Applications](https://www.confluent.io/blog/kafka-python-asyncio-integration/)\n  - [confluent kafka in docker shell](https://developer.confluent.io/quickstart/kafka-docker/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanuprshetty%2Ffynd_e2e_solution","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanuprshetty%2Ffynd_e2e_solution","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanuprshetty%2Ffynd_e2e_solution/lists"}