{"id":21130670,"url":"https://github.com/george-mountain/centralized-log-management","last_synced_at":"2026-04-11T21:44:44.954Z","repository":{"id":245978510,"uuid":"819702422","full_name":"george-mountain/Centralized-Log-Management","owner":"george-mountain","description":"Centralized Log Management - Fastapi, Elastic Search, Logstash, Kibana and Docker.","archived":false,"fork":false,"pushed_at":"2024-06-25T04:18:55.000Z","size":28,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-21T06:11:33.000Z","etag":null,"topics":["docker","elasticsearch","elasticstack","fastapi","kibana","kibana-dashboard","kibana-visualization","logstash","loguru"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/george-mountain.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-25T03:26:30.000Z","updated_at":"2024-06-25T04:52:30.000Z","dependencies_parsed_at":"2024-06-25T06:45:57.164Z","dependency_job_id":null,"html_url":"https://github.com/george-mountain/Centralized-Log-Management","commit_stats":null,"previous_names":["george-mountain/centralized-log-management"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george-mountain%2FCentralized-Log-Management","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george-mountain%2FCentralized-Log-Management/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george-mountain%2FCentralized-Log-Management/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/george-mountain%2FCentralized-Log-Management/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/george-mountain","download_url":"https://codeload.github.com/george-mountain/Centralized-Log-Management/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243573495,"owners_count":20312883,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","elasticsearch","elasticstack","fastapi","kibana","kibana-dashboard","kibana-visualization","logstash","loguru"],"created_at":"2024-11-20T05:36:45.692Z","updated_at":"2026-04-11T21:44:39.929Z","avatar_url":"https://github.com/george-mountain.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Centralized Log Management System\n\n\u003cimg width=\"1259\" alt=\"elastic_stack_dashboard\" src=\"https://github.com/george-mountain/Centralized-Log-Management/assets/19597087/7a2e9686-0b14-4867-bd4d-a67dbab4b289\"\u003e\n\n\nThis repository showcases how to centralize and manage logs from microservice applications using the Elastic Stack (Logstash, Elasticsearch, and Kibana) and Loguru for logging within a FastAPI app.\n\n## Project Overview\n\nThis project is designed for a school management system comprising multiple microservices:\n\n- **Students App**\n- **Library Management App**\n- **Faculty App**\n\nEach of these microservices can operate independently but are currently integrated using a common API gateway. We use Loguru for logging and the Elastic Stack for centralized log management.\n\n## Key Components\n\n### FastAPI App\nThe FastAPI app serves as the core of the school management system, integrating the various microservices.\n\n### Elastic Stack\n- **Logstash:** Collects and processes logs from the FastAPI app.\n- **Elasticsearch:** Stores and indexes logs for efficient searching and analysis.\n- **Kibana:** Provides a web interface for visualizing and exploring logs.\n\n### Docker\nAll applications, including the Elastic Stack, are dockerized for easy deployment and management.\n\n## Prerequisites\n\nTo run this project, you need to have Docker installed on your machine.\n\n## Getting Started\n\n### 1. Fork or clone the Repository\n\n```sh\ngit clone https://github.com/george-mountain/Centralized-Log-Management.git\ncd centralized-log-management\n```\n\n### 2. Create a `.env` File\n\nCreate a `.env` file in the root directory of the project and configure it as shown in the `env-sample` file provided in the repository.\n\n### 3. Update the `.env` File\n\nYour `.env` file should look like this:\n\n```env\nELASTIC_PASSWORD=YOURELASTICPASSWORDHERE #e.g elastic1234\nELASTICSEARCH_USERNAME=YOURELASTICUSERNAMEHERE #e.g elastic\nELASTICSEARCH_PASSWORD=YOURELASTICSEARCHPASSWORDHERE #e.g elastic1234\n```\n\n### 4. Run the Application\n\nUse Docker Compose to start all services:\n\n```sh\ndocker-compose up --build \n```\n\nThis command will start the FastAPI app, Logstash, Elasticsearch, and Kibana.\n\n## Configuration\n\n### Docker Compose\n\nThe `docker-compose.yml` file is configured to:\n\n- Start the Elasticsearch container with security enabled.\n- Start the Logstash container to collect logs from the FastAPI app.\n- Start the Kibana container to visualize logs.\n- Start the FastAPI app for the school management system.\n\n### Logstash Configuration\n\nThe Logstash configuration file `logstash.conf` uses environment variables for credentials, ensuring sensitive information is not hard-coded.\n\n### Environment Variables\n\nEnvironment variables for credentials are stored in the `.env` file, ensuring they are securely managed and can be easily changed without modifying the codebase.\n\n## Accessing Services\n\n### FastAPI App\nThe FastAPI app will be available at `http://localhost:8000`.\n\nThe fastapi endpoints are:\n\nhttp://localhost:8000/docs#/ --- This is the central application endpoint documentation\n\nhttp://localhost:8000/school-management/student/docs#/  --- This is the student app endpoints documentation\n\nhttp://localhost:8000/school-management/library/docs#/  --- This is the library app endpoints documentation\n\nhttp://localhost:8000/school-management/faculty/docs#/  --- This is the faculty app endpoints documentation.\n\n\n### Kibana\nKibana will be available at `http://localhost:5601`. Use the credentials configured in your `.env` file to log in.\nNote: When you access the above url, you will be promted to login. Use the username and password which you configured\non the .env file for the elastic stack.\n\n### Elasticsearch\nElasticsearch will be available at `http://localhost:9200`. Use the credentials configured in your `.env` file to access it.\n\n\nFor any questions or issues, please open an issue in the repository\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeorge-mountain%2Fcentralized-log-management","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgeorge-mountain%2Fcentralized-log-management","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgeorge-mountain%2Fcentralized-log-management/lists"}