https://github.com/jwalsh/observex-demo

ObserveX demo of an Internal Developer Platform (IDP)
https://github.com/jwalsh/observex-demo

devops distributed-systems distributed-tracing guile-scheme idp internal-developer-platform observability platform-engineering simulation sre

Last synced: 3 months ago
JSON representation

ObserveX demo of an Internal Developer Platform (IDP)

• Host: GitHub
• URL: https://github.com/jwalsh/observex-demo
• Owner: jwalsh
• Created: 2024-07-30T18:21:23.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-12-24T00:37:14.000Z (over 1 year ago)
• Last Synced: 2025-10-31T00:40:26.631Z (7 months ago)
• Topics: devops, distributed-systems, distributed-tracing, guile-scheme, idp, internal-developer-platform, observability, platform-engineering, simulation, sre
• Language: Scheme
• Homepage: https://wal.sh/research/idp
• Size: 896 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 7
• Metadata Files:
  • Readme: README.org

Awesome Lists containing this project

README

          
#+TITLE: observex-demo

#+LAST_MODIFIED: 2024-08-06 06:46:43

#+TITLE: Internal Developer Platform (IDP) Demo

#+AUTHOR: Platform Engineering Team

* Overview

This repository contains a comprehensive demo of an Internal Developer Platform (IDP) designed to streamline development processes, improve collaboration, and enhance productivity across multiple teams. The IDP incorporates modern architectural principles, engineering practices, and operational needs while fostering a conducive environment for tech culture and career development.

* Observability Demo

The IDP demo highlights the following observability aspects:

  + Comprehensive Monitoring: Implementation of monitoring tools to track system performance, latency, and errors.

  + Logging and Tracing: Integration of logging and tracing mechanisms to provide insights into system behavior and facilitate debugging.

  + Metrics and Alerting: Configuration of metrics and alerting systems to ensure proactive issue detection and notification.

  + Visualization: Utilization of visualization tools to provide a unified view of system performance and facilitate data-driven decision-making.

* Simulator

[[./simulator.png]]

* Key Features

  - Open-source and cloud-native architecture

  - Containerization and Kubernetes orchestration

  - Microservices architecture with service mesh

  - Comprehensive observability and monitoring

  - Local-first development approach

  - Feature flag management

  - Secrets management

  - GitOps practices

  - Multi-team support with shared components

* Repository Structure

  - =internal-developer-platform/=:

    - =data-layer/=: Data layer components

    - =demo/=: Demonstration scenarios and scripts

    - =docs/=: Documentation

    - =frontend/=: Frontend components

    - =idp-checklist-article.md=: IDP checklist article

    - =idp-innersource-metrics.md=: IDP innersource metrics

    - =idp-maturity-checklist-2024.org=: IDP maturity checklist

    - =idp-plan-1y.md=: IDP plan

    - =platform/=: Platform components

      - =api-gateway/=: API gateway

      - =change-management/=: Change management

      - =cicd-api-doctor/=: CI/CD API doctor

      - =engineering-standards/=: Engineering standards

      - =feature-flags/=: Feature flags

      - =gitops/=: GitOps

      - =innersource-distributed-tracing/=: Innersource distributed tracing

      - =innersource-shared-models/=: Innersource shared models

      - =observability/=: Observability

      - =secrets-management/=: Secrets management

      - =service-mesh/=: Service mesh

      - =shared-infrastructure/=: Shared infrastructure

      - =shared-models/=: Shared models

      - =ui-analytics/=: UI analytics

    - =teams/=: Team-specific components

      - =customer-experience/=: Customer experience team

      - =order-fulfillment/=: Order fulfillment team

      - =product-catalog/=: Product catalog team

* Teams

1. Order Fulfillment

2. Product Catalog

3. Customer Experience

* Architecture Diagram

[[./output/architecture-diagram.png]]

* Getting Started

1. Clone this repository

2. Review the IDP maturity checklist in =idp-maturity-checklist-2024.org=

3. Explore team-specific directories and their respective IDP checklists

4. Check out the =demo/= directory for specific usage scenarios

* Local Development

The IDP emphasizes a local-first approach:

  - Use LocalStack for AWS service emulation

  - Leverage Telepresence for local microservices development

  - Implement offline-capable features where possible

* Continuous Improvement

Regular audits and assessments using the IDP Maturity Checklist ensure the platform's ongoing evolution and alignment with organizational needs.

* Risks and Challenges

When implementing and maintaining an Internal Developer Platform (IDP), organizations may face the following risks, concerns, and pain points:

1. Adoption Resistance: Teams and individuals may resist changing their established workflows and adopting new IDP practices.

2. Incentive Misalignment: Lack of clear incentives or motivation for teams to use and contribute to the IDP.

3. Not-Invented-Here Syndrome: Reluctance to use or contribute to platform components developed by other teams.

4. Discoverability Issues: Difficulty in finding and leveraging existing IDP components and services across the organization.

5. Quality and Security Concerns: Worries about maintaining code quality, security, and ongoing maintenance when integrating external contributions.

6. Metrics and Evaluation Misalignment: Challenges in aligning IDP usage and contribution with existing performance metrics and evaluations.

7. Cultural Barriers: Silos between teams, lack of trust, and unwillingness to share code or knowledge.

8. Resource Constraints: Time pressures and competing priorities that make teams reluctant to invest in IDP adoption and contribution.

9. Process Ambiguity: Lack of clear guidelines and processes for using and contributing to the IDP.

10. Long-term Sustainability: Difficulty in maintaining momentum and scaling the IDP initiative across the entire organization.

11. Inconsistent Implementation: Confusion and inconsistency in how the IDP is defined and implemented across different teams.

12. Intellectual Property Concerns: Uncertainties about appropriate code sharing and IP protection within the organization.

13. External Collaboration Challenges: Issues with integrating contractors or external developers into the IDP ecosystem.

14. Control and Governance: Reluctance to distribute decision-making and control over platform components to a broader community.

15. Strategic Alignment: Lack of executive support or clear alignment between IDP initiatives and overall organizational strategy.

* Contributing

We welcome contributions! Please review our contribution guidelines before submitting pull requests.

* License

This project is licensed under the MIT License - see the LICENSE file for details.