https://github.com/apache/incubator-devlake-playground

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
https://github.com/apache/incubator-devlake-playground

dashboard-friendly data data-analysis data-engineering data-integration data-transfers devops domain-layer dora etl hacktoberfest integration jira open-source python user-friendly

Last synced: 3 days ago
JSON representation

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.

• Host: GitHub
• URL: https://github.com/apache/incubator-devlake-playground
• Owner: apache
• License: apache-2.0
• Created: 2024-03-01T10:51:15.000Z (7 months ago)
• Default Branch: main
• Last Pushed: 2024-09-18T02:39:53.000Z (15 days ago)
• Last Synced: 2024-09-25T15:51:06.513Z (8 days ago)
• Topics: dashboard-friendly, data, data-analysis, data-engineering, data-integration, data-transfers, devops, domain-layer, dora, etl, hacktoberfest, integration, jira, open-source, python, user-friendly
• Language: Python
• Homepage: https://devlake.apache.org/
• Size: 275 KB
• Stars: 5
• Watchers: 18
• Forks: 1
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        


# DevLake Jupyter Playground

[DevLake](https://devlake.apache.org/) offers an abundance of data for exploration.

This playground contains a basic set-up to interact with the data using [Jupyter Notebooks](https://jupyter.org/) and [Pandas](https://pandas.pydata.org/).

# How to play

## Prerequisites

- [Python >= 3.12](https://www.python.org/downloads/)

- [Poetry](https://python-poetry.org/docs/#installation)

- Access to a DevLake database

- Optional: An IDE or plugin that supports running Jupyter Notebooks directly (e.g. [Visual Studio Code](https://code.visualstudio.com/))

## Usage

1. Have a local clone of this repository.

2. Run `poetry install` in the root directory.

3. Either:

    - navigate to the `notebooks` directory and run the jupyter server `poetry run jupyter notebook` 

    - navigate to one of the notebook files (`.ipynb`) in the `notebooks` directory from your IDE directly

4. Make sure the notebook uses the virtual environment created by poetry.

5. Configure your database URL in the notebook code.

6. Run the notebook.

7. Start exploring the data in your own notebooks!

## Create your own Jupyter Notebook

A good starting point for creating a new notebook is `template.ipynb`.

It contains the basic steps you need to go from query to output.

To define a query, use the [Domain Layer Schema](https://devlake.apache.org/docs/DataModels/DevLakeDomainLayerSchema#schema-diagram) to get an overview of the available tables and fields.

Use [Pandas](https://pandas.pydata.org/) api to organize, transform, and analyze the query results.

## Predefined notebooks and utilities

A notebook might offer a valuable perspective on the data not available within the capabilities of a Grafana dashboard.

In this case, it's worthwhile to contribute this notebook to the community as a predefined notebook, e.g., `process_analysis.ipynb` (it depends on [graphviz](https://graphviz.org/) for its visualization).

The same goes for utility methods with, for example, predefined Pandas data transformations offering an interesting view on the data.

## Contributing

Please check the [contributing guidelines](https://github.com/apache/incubator-devlake/blob/main/README.md#-how-to-contribute).