{"id":22428504,"url":"https://github.com/kwame-mintah/python-fastapi-logging-application","last_synced_at":"2026-05-11T05:52:05.178Z","repository":{"id":247674498,"uuid":"794676141","full_name":"kwame-mintah/python-fastapi-logging-application","owner":"kwame-mintah","description":"Demonstrate an external application receiving logs, from another system.","archived":false,"fork":false,"pushed_at":"2024-07-09T21:18:54.000Z","size":113,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-27T06:45:12.575Z","etag":null,"topics":["docker","fastapi","logging","pydantic-v2","python312"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kwame-mintah.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-05-01T18:04:16.000Z","updated_at":"2024-07-09T21:20:15.000Z","dependencies_parsed_at":"2024-07-10T02:35:06.199Z","dependency_job_id":"504bb1fa-3d3e-4ccd-a860-d8021f41bc8b","html_url":"https://github.com/kwame-mintah/python-fastapi-logging-application","commit_stats":null,"previous_names":["kwame-mintah/python-fastapi-logging-application"],"tags_count":6,"template":false,"template_full_name":"kwame-mintah/python-fastapi-bigger-applications-template","purl":"pkg:github/kwame-mintah/python-fastapi-logging-application","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwame-mintah%2Fpython-fastapi-logging-application","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwame-mintah%2Fpython-fastapi-logging-application/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwame-mintah%2Fpython-fastapi-logging-application/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwame-mintah%2Fpython-fastapi-logging-application/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kwame-mintah","download_url":"https://codeload.github.com/kwame-mintah/python-fastapi-logging-application/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwame-mintah%2Fpython-fastapi-logging-application/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32883469,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-10T13:40:02.631Z","status":"online","status_checked_at":"2026-05-11T02:00:05.975Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","fastapi","logging","pydantic-v2","python312"],"created_at":"2024-12-05T20:15:02.386Z","updated_at":"2026-05-11T05:52:05.161Z","avatar_url":"https://github.com/kwame-mintah.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Python FastAPI Logging Application\n\n![python](https://img.shields.io/badge/python-3.12.3-informational)\n![fastapi-0.110.3-informational](https://img.shields.io/badge/fastapi-0.109.0-informational)\n![semver](https://img.shields.io/badge/semver-2.0.0-blue)\n\u003ca href=\"https://github.com/psf/black\"\u003e\u003cimg alt=\"Code style: black\" src=\"https://img.shields.io/badge/code%20style-black-000000.svg\"\u003e\u003c/a\u003e\n[![🧪 Run unit tests](https://github.com/kwame-mintah/python-fastapi-logging-application/actions/workflows/run-unit-tests.yml/badge.svg)](https://github.com/kwame-mintah/python-fastapi-logging-application/actions/workflows/run-unit-tests.yml)\n\nThis project aims to demonstrate an external application receiving logs, from another system. Logs received will be stored\nwithin the applications' database. Storing and retrieval of log events will be exposed via endpoints using the\n[FastAPI](https://fastapi.tiangolo.com/) framework.\n\n## Development\n\n- [Python](https://www.python.org/downloads/release/python-3123/)\n- [Docker](https://docs.docker.com/compose/install/)\n- [Pre-commit](https://pre-commit.com/#install)\n\n## API Specification\n\nThe application exposes `/v1/events/*` endpoints, which accommodates two types of log events: user and system. Both of which\ncan be consumed and/or returned in the same API request. The expected log event structure for these events is the following:\n\n- User:\n\n  ```json\n  {\n    \"type\": \"user\",\n    \"timestamp\": \"2023-10-01T13:45:00.000Z\",\n    \"event_id\": \"u_001\",\n    \"event\": {\n      \"username\": \"my_user\",\n      \"email\": \"my_user@email.com\",\n      \"operation\": \"write\"\n    }\n  }\n  ```\n\n- System:\n  ```json\n  {\n    \"type\": \"system\",\n    \"timestamp\": \"2023-10-01T13:45:00.000Z\",\n    \"event_id\": \"s_123\",\n    \"event\": {\n      \"system_id\": \"id_123\",\n      \"location\": \"europe\",\n      \"operation\": \"read\"\n    }\n  }\n  ```\n\n[Pydantic](https://docs.pydantic.dev/2.7/) is used heavily within this project, for schema validation and serialisation.\nAll endpoints have been annotated with their models and through FastAPI swagger [documentation](https://fastapi.tiangolo.com/features/#automatic-docs)\n(`/docs`), end users are able to easily identify limitations, expected responses and errors for each endpoint.\n\nThrough Pydantic and FastAPI, API constraints have been addressed, for example ensuring the number of records inserted or returned\ncannot surpass 1000 and should return a [400 BAD request](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400).\nAs mentioned earlier, API specification and constraints can be found on `/docs` endpoint, it is **recommended** to read\nthrough the swagger documentation and try out each endpoint.\n\n## Roadmap\n\nMore work needs to be completed for the final version of the application. Below are additional things required for a\nclearer vision of what is envisioned of this application.\n\n### Implementing MongoDB\n\nAs of version [v1.1.0](https://github.com/kwame-mintah/python-fastapi-logging-application/commit/cd423356da7193c37fae5ac7eb6dcf2a31634cb1), [`pickle`](https://docs.python.org/3.12/library/pickle.html)\nwas introduced to the codebase as a placeholder for storing and querying log events on runtime. Ideally, [MongoDB](https://www.mongodb.com/docs/upcoming/release-notes/7.3/) will be\nused as a datastore for log events. This is because log events received are in `JSON` format and MongoDB stores data as [`BSON`](https://www.mongodb.com/json-and-bson)\nas both are closely related, storing will be much easier without having to do many additional steps.\n\nThe following tasks will need to be completed to accomplish this:\n\n1. Select an appropriate Object Document Mapper (ODM), most likely [Motor](https://www.mongodb.com/docs/drivers/motor/).\n2. Configure credentials for [connecting](https://www.mongodb.com/docs/drivers/motor/#connect-to-a-mongodb-server-on-your-local-machine)\n   to MongoDB using environment variables and store values in a suitable [pass](https://www.passwordstore.org/) store\n   or secrets manager etc.\n3. Create a new database for storing log events. A new Pydantic model will be created to represent the Database model e.g.\n   `EventLogsCollection` re-using the existing model `EventLog` encapsulated as a list.\n4. Update existing endpoints and services to use `CRUD` operations against MongoDB.\n\n\u003e [!NOTE]\n\u003e Due to `pickle` being used, duplicate `event_id` can be inserted, so using the `/v1/events/{event_id}` will\n\u003e only return one result and not the duplicate log events. This is the intended behaviour in the final version,\n\u003e as the MongoDB `_id` for the document will use the `event_id` when being saved into the database.\n\n### Deploying to Kubernetes using Helm charts\n\nA `Dockerfile` is included to start the FastAPI server within a docker container. The next step is to use\nthe docker image created as part of deployment to a [Kubernetes](https://kubernetes.io/) cluster using [Helm](https://helm.sh/).\nMost major cloud providers e.g. Google, AWS, Azure etc. provide a Kubernetes service, deploying to any of the providers\nwill demonstrate a scalable and highly available FastAPI application.\n\nThe following tasks will need to be completed to accomplish this:\n\n1. A new continuous integration (CI) / continuous delivery (CD) pipeline that will build and push new docker images to\n   a remote repository, so it can be pulled down later for deployments to Kubernetes.\n2. Configure a Kubernetes cluster within the chosen cloud provider using [Terraform](https://www.terraform.io/).\n3. Create necessary `.yaml` configuration files to specify docker image, networking, environment variables etc.\n4. Deploy to the Kubernetes namespace using helm.\n\n### API Improvements\n\nThe current query parameters supported as [v1.0.0](https://github.com/kwame-mintah/python-fastapi-logging-application/commit/640dd23fa0ec3fdfff4026c1f075314707db7547)\nfor the `/v1/endpoints/all` endpoint is `?size=`. Although there is a size limit of 1000, does not mean that there is a\nmaximum of 1000 log event stored within the database. Providing the ability to Paginate the log events returned, allows\nend users to potentially scroll through all log events without being an expense operation against the database.\n\nThe following tasks will need to be completed to accomplish this:\n\n1. Create a new Pydantic model to represent the Paginated response e.g. `PaginatedEventResponse`. Detailing, the size,\n   from, total etc.\n2. Update existing endpoint `response_model` to reflect the new response.\n3. Implement MongoDB [`MotorCursor.skip()`](https://motor.readthedocs.io/en/stable/api-tornado/cursors.html#motor.motor_tornado.MotorCursor.skip)\n   to include how many documents to fetch, offset etc.\n\n## Prerequisite\n\nYou will need to create a virtual environment, please follow the official [Python guide](https://docs.python.org/3.12/library/venv.html) how on doing this.\n\n## Project structure\n\nThe project file structure follows the same blueprint [documented](https://fastapi.tiangolo.com/tutorial/bigger-applications/#an-example-file-structure) by FastAPI.\n\n```markdown\n.\n├── app/\n│ ├── __init__.py\n│ ├── main.py\n│ ├── dependencies.py\n│ ├── data/\n│ ├── exceptions/\n│ ├── models/\n│ ├── routers/\n│ └── services/\n└── tests/\n├── integration/\n└── unit/\n```\n\n## Usage\n\nAll commands snippets are run within the root directory of the project. After cloning the repository, please change directories.\n\n1. Install python packages used for the service:\n\n   ```console\n   pip install -r requirements.txt\n   ```\n\n2. Run the FastAPI server, which will start on port 8000:\n\n   ```console\n   python app/main.py\n   ```\n\n   Endpoint documentation is available on: http://127.0.0.1:8000/docs\n\n## Running with Docker\n\nRunning the `docker-compose.yml` file, will build a new image. Named and tagged as: `python-fastapi-logging-application-fastapi:latest`\nusing the [`Dockerfile`](https://docs.docker.com/develop/develop-images/guidelines/), this docker image will be used for\nthe `fastapi` service docker container [^1]:\n\n```console\ndocker-compose up --build -d\n```\n\nEndpoint documentation is available on: http://localhost:8080/docs\n\nTo shut down the docker containers:\n\n```console\ndocker-compose down --remove-orphans\n```\n\n\u003e [!NOTE]\n\u003e The command provided ensures that a new image is built ([`--build`](https://docs.docker.com/reference/cli/docker/compose/build/)),\n\u003e each time to facilitate potential code changes in between container runs. If not provided, the same docker image will\n\u003e be re-used and not reflect code changes. Additionally, no logs will be output in your console due to ([`-d`](https://docs.docker.com/reference/cli/docker/compose/up/#options))\n\u003e instead FastAPI logs can be found within your Docker for desktop application.\n\n## Tests\n\nUnit tests are located in `/tests/unit` directory, run unit tests using:\n\n```console\npytest tests/unit\n```\n\nAdditionally, a coverage report can be generated using [`pytest-cov`](https://pypi.org/project/pytest-cov/):\n\n```console\npytest --cov=app tests/unit --cov-report=html:coverage_report\n```\n\nWill generate a coverage HTML file, in the `/coverage_report/` directory, simply open the `index.html` in your chosen web browser [^2].\n\nIntegration tests are located in `/tests/integration` directory, run integration using:\n\n```console\npytest tests/integration\n```\n\n## Contributing\n\nGit hook scripts are very helpful for identifying simple issues before pushing any changes.\nHooks will run on every commit automatically pointing out issues in the code e.g. trailing whitespace.\n\nTo help with the maintenance of these hooks, [pre-commit](https://pre-commit.com/) is used, along with [pre-commit-hooks](https://pre-commit.com/#adding-pre-commit-plugins-to-your-project).\n\nPlease following [these instructions](https://pre-commit.com/#install) to install `pre-commit` locally and ensure that you have run\n`pre-commit install` to install the hooks for this project.\n\nAdditionally, once installed, the hooks can be updated to the latest available version with `pre-commit autoupdate`.\n\n## GitHub Actions (CI/CD)\n\nGitHub project has three workflow set up, found in `.github/workflows/`:\n\n- '🧹 Run linter' (`run-linter.yml`): To run [Flake8](https://flake8.pycqa.org/en/latest/) and check Python code system and comply with various style guides.\n- '🧪 Run unit tests' (`run-unit-tests.yml`): To run unit tests within a continuous integration (CI) environment, using [`pytest`](https://docs.pytest.org/en/8.2.x/).\n- '🚧 Bump version' (`run-version-bump.yml`): To create a new GitHub tag based on [semantic versioning](https://semver.org/) using [commitizen](https://commitizen-tools.github.io/commitizen/).\n\n[^1]: A [`platform`](https://docs.docker.com/compose/compose-file/build/#platforms) has been specified to ensure the host machine, uses the correct platform during docker image build.\n[^2]: The GitHub workflow ['🧪 Run unit tests'](https://github.com/kwame-mintah/python-fastapi-logging-application/actions/workflows/run-unit-tests.yml) provides a brief overview of coverage for each file, when viewing the step 'Run tests with pytest with coverage'.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwame-mintah%2Fpython-fastapi-logging-application","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkwame-mintah%2Fpython-fastapi-logging-application","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwame-mintah%2Fpython-fastapi-logging-application/lists"}