https://github.com/alexye-mapleleafs/ceba_process
Canada Emergency Business Account (CEBA) Process Automation in GCP
https://github.com/alexye-mapleleafs/ceba_process
airflow-dags bigquery docker-container docker-image google-cloud-platform google-cloud-storage python3
Last synced: 2 months ago
JSON representation
Canada Emergency Business Account (CEBA) Process Automation in GCP
- Host: GitHub
- URL: https://github.com/alexye-mapleleafs/ceba_process
- Owner: AlexYe-MapleLeafs
- Created: 2025-08-15T12:12:09.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2025-08-15T12:42:18.000Z (10 months ago)
- Last Synced: 2025-09-10T17:10:49.714Z (9 months ago)
- Topics: airflow-dags, bigquery, docker-container, docker-image, google-cloud-platform, google-cloud-storage, python3
- Language: Python
- Homepage:
- Size: 26.4 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# CEBA_Process
Canada Emergency Business Account (CEBA) Process Automation in GCP
# CEBA_Process
**Automated CEBA Process — Google Cloud Platform**
This repository contains a simplified, demonstration version of a real-world GCP project.
It illustrates the structure, configurations, and code required to automate CEBA process using **Google Cloud Platform** services.
---
## Repository Structure
| File/Folder | Purpose |
| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`ceba_automation_v1.py`** | Python scripts containing SQL queries for extracting data, python functions to transform data and eventually load processed data into **BigQuery** target tables.\_ |
| **`accp.yaml`** | Pipeline configuration file defining build instructions for the container image, including image name, build context, and other ACCP pipeline parameters.
_Referenced during pipeline initialization._ |
| **`Dockerfile`** | Image build specification defining the base image, system dependencies, and Python packages required to execute the scripts.
_Referenced in `accp.yaml`._ |
| **`requirements.txt`** | List of Python dependencies to be installed in the image environment.
_Referenced in the `Dockerfile`._ |
| **`ceba_dag.py`** | **Apache Airflow DAG** for orchestrating and scheduling the end-to-end process. References the container image built from the above files and defines execution logic and scheduling parameters. |
---
## Workflow Overview
1. **Code and Configuration** — Python scripts, configuration files, and dependencies are stored in this repository.
2. **Image Build** — The ACCP pipeline uses `accp.yaml`, `Dockerfile` and `requirement.txt` to build a container image with all required dependencies.
3. **Data Processing** — The container runs Python scripts to fetch, transform, and load data into BigQuery.
4. **Orchestration** — Airflow triggers the container execution according to the schedule defined in `ceba_dag.py`.
---
