{"id":18926103,"url":"https://github.com/ittovate/demo-batch-processing","last_synced_at":"2026-04-12T18:40:15.305Z","repository":{"id":257800607,"uuid":"860127729","full_name":"ittovate/demo-batch-processing","owner":"ittovate","description":"This project demonstrates a batch processing application built using Spring Boot and Spring Batch. It showcases how to read data from both Kafka and a Database, then write the processed data into a CSV file.","archived":false,"fork":false,"pushed_at":"2024-09-30T14:43:45.000Z","size":61,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-12-31T18:30:31.784Z","etag":null,"topics":["kafka","maven","postgres","spring","zookeeper"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ittovate.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-19T21:48:17.000Z","updated_at":"2024-09-30T09:25:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"2befe33d-75cc-4e5c-a312-7b26a823da89","html_url":"https://github.com/ittovate/demo-batch-processing","commit_stats":null,"previous_names":["ittovate/batch-processing","ittovate/demo-batch-processing"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ittovate%2Fdemo-batch-processing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ittovate%2Fdemo-batch-processing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ittovate%2Fdemo-batch-processing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ittovate%2Fdemo-batch-processing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ittovate","download_url":"https://codeload.github.com/ittovate/demo-batch-processing/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239921837,"owners_count":19718844,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kafka","maven","postgres","spring","zookeeper"],"created_at":"2024-11-08T11:14:34.303Z","updated_at":"2026-03-15T00:30:18.135Z","avatar_url":"https://github.com/ittovate.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Batch Processing Demo\n\nThis is a batch brocessing demo used for processing chunks of data from a kafka broker or a relational database and writing them in chunks in a CSV file and can be restarted if any error occurs inside the batch processing\n\n\n# Prerequisites: \n- Java 21\n- Maven\n- Kafka\n- Zookeeper\n- DB (Any database that is supported inside JDBC)\n- Docker (Optionally but recommended to have to just use the instances without having to install it yourself)\n\nNOTE: there is a `docker-compose.yaml` file for both kafka ,zookeeper and postgres that are used in this demo you can just use `docker-compose up -d` then you would have Kafka, Zookeeper and a PostgreSQL database isntances running\n\n# Running\n1. Clone the repository \n    ```\n    git clone https://github.com/ittovate/batch-processing.git\n    ```\n2. Navigate to the project directory\n    ```\n    cd batch-processing\n    ```\n3. Run the following command to start Kafka,Zookeeper and PostgreSQL instances \n   ```\n   docker-compose up -d\n   ```\n   This command will start new instances but if you already ran it before you could use\n   ```\n   docker-compose start\n   ```\n   Note : Make sure that the instances are up and running using \n   ````\n   docker ps\n   ````\n   If there is an instance not running correctly run it again using\n   ```\n   docker start {spring_batch_postgres_db} OR {spring-batch-zookeeper-demo} OR {spring-batch-kafka-demo}\n   ```\n   run ``docker start`` then append the name of either of the above 3 names if any of them is not shown when using ``docker ps``\n4. Build and run the project\n    ```\n    mvn clean install spring-boot:run\n    ```\n### For more information on how to use this demo you can refer to [Usage](./USAGE.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fittovate%2Fdemo-batch-processing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fittovate%2Fdemo-batch-processing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fittovate%2Fdemo-batch-processing/lists"}