Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/toddbirchard/jira-database-etl
:mens: 💾 Script to import issues from a JIRA instance into a database.
https://github.com/toddbirchard/jira-database-etl
etl etl-pipeline flask flask-sqlalchemy jira-rest-api pandas python3
Last synced: about 9 hours ago
JSON representation
:mens: 💾 Script to import issues from a JIRA instance into a database.
- Host: GitHub
- URL: https://github.com/toddbirchard/jira-database-etl
- Owner: toddbirchard
- License: bsd-3-clause
- Created: 2019-03-11T14:09:26.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2022-12-08T10:53:54.000Z (almost 2 years ago)
- Last Synced: 2024-05-12T00:44:38.268Z (6 months ago)
- Topics: etl, etl-pipeline, flask, flask-sqlalchemy, jira-rest-api, pandas, python3
- Language: Python
- Homepage: https://hackersandslackers.com/jira-to-sql-etl/
- Size: 1010 KB
- Stars: 53
- Watchers: 5
- Forks: 20
- Open Issues: 20
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# JIRA-to-Database Import
Extracts issues from a JIRA instance via the JIRA REST API, transforms the data, and loads data to a database.
To derive epic-based information from tickets, the script creates an `JiraEpic` table as well as a `JiraIssue` table. The `JiraIssue` table is joined with the former table to easily perform analysis on aggregated epic data.
Accompanying tutorial can be found here: https://hackersandslackers.com/jira-to-sql-etl/
## Installation
**Installation via `requirements.txt`**:
```shell
$ git clone https://github.com/toddbirchard/jira-database-etl.git
$ cd jira-database-etl
$ python3 -m venv myenv
$ source myenv/bin/activate
$ pip3 install -r requirements.txt
$ flask run
```
**Installation via [Pipenv](https://pipenv-fork.readthedocs.io/en/latest/)**:
```shell
$ git clone https://github.com/toddbirchard/jira-database-etl.git
$ cd jira-database-etl
$ pipenv shell
$ pipenv update
$ flask run
```
## Configuration
The following environment variables are needed to run this script:
* `SQLALCHEMY_DATABASE_URI`: A URI for the database intended to store these tables (ie: _mysql+pymysql://[USER]:[PASSWORD]@d[DATABASE_HOST]:[PORT]/[DATABASE_NAME]_)
* `SQLALCHEMY_EPIC_TABLE`: Name of database table to store epics.
* `SQLALCHEMY_JIRA_TABLE`: Name of database table to store JIRA issues.
* `JIRA_ENDPOINT`: Your JIRA Cloud API endpoint for JQL searching (such as *https://mydomain.atlassian.net/rest/api/3/search*)
* `JIRA_USERNAME`: Your JIRA username.
* `JIRA_API_KEY`: An API key associated with the JIRA user.
* `JIRA_ISSUES_JQL`: JQL to get JIRA issues.
* `JIRA_ISSUES_FIELDS`: Specific fields to retrieve from the JIRA query.
* `JIRA_EPICS_JQL`: JQL to get JIRA epics.
* `JIRA_EPICS_FIELDS`: Specific fields to retrieve from the epics query.
### Troubleshooting
Make sure your database contains tables named `JiraEpic` and `JiraIssue` prior to running this script (columns/schema don't matter, these will be overridden).
-----
This project and all publically-visible repositories are free of charge. If you've found this project to be helpful, a [small donation](https://www.buymeacoffee.com/hackersslackers) would be greatly appreciated to keep us in business. All proceeds go towards coffee, and all coffee goes towards improving these projects.