{"id":29353727,"url":"https://github.com/jojees/project-genesis","last_synced_at":"2026-04-04T20:32:52.916Z","repository":{"id":303415921,"uuid":"1015302284","full_name":"jojees/project-genesis","owner":"jojees","description":"Project Genesis is a comprehensive, hands-on learning initiative designed to build and manage a tangible, multi-service application within a modern DevOps ecosystem. This project serves as a real-world sandbox, demonstrating best practices across various disciplines, including DevOps, Site Reliability Engineering (SRE), DevSecOps, and FinDevOps.","archived":false,"fork":false,"pushed_at":"2025-08-04T15:42:17.000Z","size":473,"stargazers_count":0,"open_issues_count":16,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-04T15:46:50.137Z","etag":null,"topics":["cicd","devops","docker","gitops","grafana","high-availability","kubernetes","microservices-architecture","observability","postgres","prometheus","rabbitmq","redis","sre"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jojees.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-07T09:51:33.000Z","updated_at":"2025-07-24T11:38:21.000Z","dependencies_parsed_at":"2025-07-07T14:51:54.014Z","dependency_job_id":"ee5817c1-e184-4fce-bff3-a0f4660acbae","html_url":"https://github.com/jojees/project-genesis","commit_stats":null,"previous_names":["jojees/project-genesis"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jojees/project-genesis","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jojees%2Fproject-genesis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jojees%2Fproject-genesis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jojees%2Fproject-genesis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jojees%2Fproject-genesis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jojees","download_url":"https://codeload.github.com/jojees/project-genesis/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jojees%2Fproject-genesis/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31413269,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T20:09:54.854Z","status":"ssl_error","status_checked_at":"2026-04-04T20:09:44.350Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cicd","devops","docker","gitops","grafana","high-availability","kubernetes","microservices-architecture","observability","postgres","prometheus","rabbitmq","redis","sre"],"created_at":"2025-07-09T02:09:03.419Z","updated_at":"2026-04-04T20:32:52.890Z","avatar_url":"https://github.com/jojees.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Project Genesis\n\n[![License](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)\n[![codecov](https://codecov.io/github/jojees/project-genesis/graph/badge.svg?token=E6244R8XGA)](https://codecov.io/github/jojees/project-genesis)\n[![Code Scanning status](https://github.com/jojees/project-genesis/actions/workflows/build-python-services.yml/badge.svg)](https://github.com/jojees/project-genesis/security/code-scanning)\n[![CI Status](https://github.com/jojees/project-genesis/actions/workflows/build-python-services.yml/badge.svg)](https://github.com/jojees/project-genesis/actions/workflows/build-python-services.yml)\n[![CD Status](https://github.com/jojees/project-genesis/actions/workflows/deploy-services.yml/badge.svg)](https://github.com/jojees/project-genesis/actions/workflows/deploy-services.yml)\n\n---\n\n## Table of Contents\n\n* [Introduction](#introduction)\n* [Project Overview \u0026 Learning Objectives](#project-overview--learning-objectives)\n* [Application Architecture: The `AuditFlow Platform`](#application-architecture-the-auditflow-platform)\n* [Technology Stack](#technology-stack)\n* [Directory Structure](#directory-structure)\n* [Getting Started](#getting-started)\n    * [Prerequisites](#prerequisites)\n    * [Infrastructure Provisioning](#infrastructure-provisioning)\n    * [Application Deployment](#application-deployment)\n* [DevOps Pillars \u0026 Their Integration](#devops-pillars--their-integration)\n* [Contributing](#contributing)\n* [License](#license)\n* [Contact](#contact)\n\n---\n\n## Introduction\n\n**Project Genesis** is a comprehensive, hands-on learning initiative designed to build and manage a tangible, multi-service application within a modern **DevOps ecosystem**. This project serves as a real-world sandbox, demonstrating best practices across various disciplines, including **DevOps, Site Reliability Engineering (SRE), DevSecOps, and FinDevOps**.\n\nAt its core, Project Genesis leverages a **Kubernetes (K3s)** cluster to orchestrate the **AuditFlow Platform**. The project provides a holistic view of the software delivery lifecycle, from infrastructure provisioning to application deployment and ongoing operations, all managed as code.\n\n---\n\n## Project Overview \u0026 Learning Objectives\n\nThis project aims to provide a practical foundation for aspiring and current DevOps practitioners. By working through Project Genesis, you will gain hands-on experience in:\n\n* **Designing and deploying the AuditFlow Platform** in a Kubernetes environment.\n* **Implementing Infrastructure as Code (IaC)** for consistent and automated infrastructure provisioning.\n* **Building robust CI/CD pipelines** for continuous integration and delivery.\n* **Managing Kubernetes resources** effectively using Helm and Kustomize.\n* **Integrating testing and code quality** into the development workflow.\n* **Establishing observability** for monitoring application health and performance.\n* **Understanding and applying DevSecOps principles**, specifically gaining hands-on experience with **Static Application Security Testing (SAST)**, **Software Composition Analysis (SCA)** and integrating security analysis directly into CI/CD pipelines.\n* **Exploring FinDevOps concepts** for cost-effective cloud-native operations.\n\n---\n\n## Application Architecture: The `AuditFlow Platform`\n\nThe `AuditFlow Platform` is a conceptual multi-service application designed to generate, process, and display audit events. It typically consists of:\n\n* **`audit_event_generator`**: A service responsible for creating synthetic audit events.\n* **`audit-log-analysis`**: Consumes events, performs analysis, and stores results.\n* **`event-audit-dashboard`**: Provides a user interface to visualize audit events and alerts.\n* **`notification-service`**: Handles alerts and notifications based on analysis.\n* **`postgres`**: A relational database for persistent data storage.\n* **`rabbitmq`**: A message broker for inter-service communication.\n* **`redis`**: A key-value store for caching or temporary data.\n\nThese services communicate primarily via **RabbitMQ**, enabling an event-driven architecture that showcases real-world microservice interaction patterns. For further details about the application architecture, please refer to the [Architecture Documentation](docs/architecture.md).\n\n---\n\n## Technology Stack\n\nProject Genesis utilizes a diverse set of industry-standard tools and technologies:\n\n* **Containerization:** [Docker](https://www.docker.com/)\n* **Source Code Management:** [GitHub](https://github.com/)\n* **CI/CD Automation:** [GitHub Actions](https://github.com/features/actions)\n* **Container Registry:** [Docker Hub](https://hub.docker.com/u/jojees)\n* **Kubernetes Distribution:** [K3s](https://k3s.io/) (Lightweight Kubernetes)\n* **Programming Language:** [Python](https://www.python.org/)\n* **Kubernetes Package Manager:** [Helm](https://helm.sh/)\n* **Testing Framework:** [Pytest](https://docs.pytest.org/en/stable/) (for Python application testing)\n* **Code Coverage:** [Coverage.py](https://coverage.readthedocs.io/en/latest/) (integrated with Pytest)\n\u003c!--* **Kubernetes Configuration Customization:** [Kustomize](https://kustomize.io/)\n* **Monitoring \u0026 Alerting:**\n    * [Prometheus](https://prometheus.io/): For metrics collection and time-series data.\n    * [Grafana](https://grafana.com/): For data visualization and dashboarding.\n    * [Terraform](https://www.terraform.io/): For provisioning and managing infrastructure resources.\n    --\u003e\n* **Infrastructure as Code (IaC):**\n    * [Ansible](https://www.ansible.com/): For configuration management and K3s cluster setup. \n---\n\n## Directory Structure\n\nThis project is a dynamic and evolving learning initiative. Its status and ongoing development can be tracked via the [GitHub Project board](https://github.com/jojees/project-genesis/projects) and [GitHub Issues](https://github.com/jojees/project-genesis/issues).\n\nThe project is meticulously organized to separate application code from infrastructure, documentation, and CI/CD configurations. This structure promotes clarity, maintainability, and scalability.\n```\n.\n├── .github/                      # GitHub Actions workflows for CI/CD\n├── docs/                         # Project documentation (architecture, pillars, setup guides)\n├── infra/                        # Infrastructure as Code (Terraform for provisioning, Ansible for configuration)\n│   ├── terraform/                # Terraform configurations for base infrastructure\n│   └── ansible/                  # Ansible playbooks and roles for configuration management (e.g., K3s installation)\n├── k8s/                          # Kubernetes manifests and Helm charts\n│   ├── base/                     # Base Kubernetes YAMLs for individual services (can be integrated into Helm)\n│   ├── charts/                   # Helm charts for the entire application and individual services\n│   └── overlays/                 # Kustomize overlays for environment-specific configurations\n├── monitoring/                   # Configurations for Prometheus, Grafana, and alert rules\n├── reports/                      # Generated reports (e.g., test coverage XML)\n├── scripts/                      # Helper scripts for deployment, setup, etc.\n└── src/                          # Application source code (the 'AuditFlow Platform' microservices)\n    ├── audit_event_generator/\n    ├── audit-log-analysis/\n    ├── event-audit-dashboard/\n    └── notification-service/\n        └── tests/                # Unit and integration tests for each service\n├── .gitignore                    # Files and directories to ignore in Git\n├── .pytest.ini                   # Global Pytest configuration\n├── CODE_OF_CONDUCT.md\n├── CONTRIBUTING.md\n├── LICENSE\n├── README.md                     # This file\n└── TODO.md                       # Project tasks and notes\n```\nFor a more detailed breakdown of the directory contents, refer to the `docs/` directory.\n\n**🎥 Want to see Project Genesis in action and follow its development?** Check out our video tutorials and development logs on our [YouTube Channel](https://www.youtube.com/@JojeesDevOpsStudio)!\n\n---\n\n\u003c!-- ## Getting Started\n\nTo get **Project Genesis** up and running, follow these high-level steps. Detailed instructions can be found in the `docs/` directory.\n\n### Prerequisites\n\n* Git\n* Docker\n* Python 3.9+ and Poetry (for application development)\n* Terraform CLI\n* Ansible\n* kubectl\n* Helm CLI\n\n### Infrastructure Provisioning\n\n1.  **Review Terraform Configurations:** Navigate to `infra/terraform/` and review the `.tf` files. Adjust variables in `infra/terraform/environments/dev/terraform.tfvars` as needed for your target environment (e.g., IP addresses for nodes or cloud provider credentials).\n2.  **Provision Infrastructure:**\n    ```bash\n    cd infra/terraform\n    terraform init\n    terraform plan\n    terraform apply -auto-approve\n    ```\n3.  **Configure K3s Cluster:** Use Ansible to install and configure K3s on your provisioned nodes. Update `infra/ansible/inventory/hosts.ini` with your node IPs.\n    ```bash\n    cd infra/ansible\n    ansible-playbook -i inventory/hosts.ini playbooks/setup-k3s.yaml\n    ```\n    *Ensure your SSH keys are correctly set up for Ansible to connect to your nodes.*\n\n### Application Deployment\n\n1.  **Build and Push Docker Images:** Your GitHub Actions workflows (e.g., `build-and-push-*.yaml` under `.github/workflows/`) will automatically build and push Docker images for the `AuditFlow Platform` services to Docker Hub upon code changes. Ensure you've configured Docker Hub credentials as GitHub Secrets.\n2.  **Deploy with Helm:** Once images are available, deploy the `AuditFlow Platform` using the main Helm chart.\n    ```bash\n    cd k8s/charts/events-app # Note: This directory name may change if you rename your main chart\n    helm dependency update # If using subcharts\n    helm install auditflow-platform . --namespace auditflow-platform --create-namespace -f values.yaml\n    ```\n    *Refer to `k8s/charts/events-app/values.yaml` for configuration options or create environment-specific `values-*.yaml` files.*\n\n--- --\u003e\n\n## DevOps Pillars \u0026 Their Integration\n\n**Project Genesis** is structured to explicitly demonstrate key DevOps principles:\n\n* **DevOps:** Full automation from code commit to deployment using a hybrid runner strategy, fostering collaboration and rapid feedback cycles for the `AuditFlow Platform`.\n* **Site Reliability Engineering (SRE):** Focus on observability (Prometheus, Grafana), defining SLOs/SLAs, and building resilient, self-healing systems for the `AuditFlow Platform`.\n* **DevSecOps:** Integration of security best practices throughout the SDLC, including **automated Static Application Security Testing (SAST) with Bandit for Python services**, **Software Composition Analysis (SCA) and vulnerability scanning of Docker images with Trivy**, and secure Kubernetes configurations for the `AuditFlow Platform`.\n* **FinDevOps:** Awareness and implementation of cost-efficient strategies in cloud-native environments, such as resource optimization, effective scaling, and cost monitoring of the `AuditFlow Platform` and its underlying infrastructure.\n\n### Security Overview (DevSecOps)\n\nProject Genesis incorporates several security best practices:\n\n* **Static Application Security Testing (SAST):** Python services are scanned using [Bandit](https://bandit.readthedocs.io/) during the CI/CD pipeline. Findings are automatically uploaded to GitHub Code Scanning.\n    * **Scan Tool:** Bandit\n    * **Output Format:** SARIF\n    * **Integration Point:** GitHub Actions (`.github/workflows/build-single-service.yml`)\n    * **View Results:** Check the [Code scanning alerts](https://github.com/jojees/project-genesis/security/code-scanning) in the GitHub repository's Security tab.\n* **Software Composition Analysis (SCA) \u0026 Vulnerability Scanning:**\n    * **Goal:** Identify known vulnerabilities in third-party libraries/dependencies and in Docker images (OS packages, installed software).\n    * **Purpose:** Ensure the security of your supply chain by scanning for components with known CVEs.\n    * **Scan Tool:** [Trivy](https://aquasecurity.github.io/trivy/)\n    * **Scan Types:**\n        * **Filesystem Scan:** Scans Python project dependencies (e.g., `requirements.txt`, `pyproject.toml`) for vulnerabilities.\n        * **Image Scan:** Scans built Docker images for vulnerabilities in OS packages, programming language dependencies, and configuration issues.\n    * **Output Format:** SARIF\n    * **Integration Point:** GitHub Actions (`.github/workflows/build-single-service.yml`)\n    * **View Results:** Check the [Code scanning alerts](https://github.com/jojees/project-genesis/security/code-scanning) in the GitHub repository's Security tab.\n* **Secure Configuration:** Kubernetes security best practices are enforced through Infrastructure as Code. The **self-hosted runner deployment and its lifecycle are managed by Ansible,* ensuring a consistent and secure setup without manual intervention.\n\n---\n\n\u003c!-- ## Contributing\n\nContributions are welcome! If you'd like to contribute to Project Genesis, please refer to our [CONTRIBUTING.md](CONTRIBUTING.md) guide for details on our code of conduct, development process, and submission guidelines.\n\n--- --\u003e\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n---\n\n## Contact\n\n* **Author:** Joji Vithayathil Johny\n* **Email:** joji@jojees.net\n* **GitHub Repository:** [https://github.com/jojees/project-genesis](https://github.com/jojees/project-genesis)\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjojees%2Fproject-genesis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjojees%2Fproject-genesis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjojees%2Fproject-genesis/lists"}