https://github.com/kenriortega/issues_tracker_pipeline

https://github.com/kenriortega/issues_tracker_pipeline

Last synced: 17 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/kenriortega/issues_tracker_pipeline
• Owner: kenriortega
• Created: 2022-09-24T14:42:15.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2023-07-15T14:58:11.000Z (almost 2 years ago)
• Last Synced: 2025-03-29T23:41:10.307Z (about 1 month ago)
• Language: Python
• Size: 596 KB
• Stars: 4
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Issues Tracker OSS project

### Data engineer oss project

![](assets/issues_tracker_oss.jpg)

![](assets/issues-tracker-2022-10-14T15-03-36.448Z.jpg)

### Tech

- apache kafka

- ClickHouse

- Superset

- python

- airflow

- k8s

### Resource 

- [https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/operators.html#howto-operator-kubernetespodoperator](https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/operators.html#howto-operator-kubernetespodoperator)

- [https://splunktool.com/failed-to-extract-xcom-from-airflow-pod-kubernetes-pod-operator](https://splunktool.com/failed-to-extract-xcom-from-airflow-pod-kubernetes-pod-operator)

- [https://www.youtube.com/watch?v=XyvTmuAJ4b4&t=1s](https://www.youtube.com/watch?v=XyvTmuAJ4b4&t=1s)

> Tips

```bash

python3 -c "namespace='';import atexit,subprocess,json,requests,sys;proxy_process = subprocess.Popen(['kubectl', 'proxy']);atexit.register(proxy_process.kill);p = subprocess.Popen(['kubectl', 'get', 'namespace', namespace, '-o', 'json'], stdout=subprocess.PIPE);p.wait();data = json.load(p.stdout);data['spec']['finalizers'] = [];requests.put('http://127.0.0.1:8001/api/v1/namespaces/{}/finalize'.format(namespace), json=data).raise_for_status()"

```