{"id":21170185,"url":"https://github.com/subhashchandra-birajdar/springbatch_csv_to_db","last_synced_at":"2026-04-16T04:02:57.932Z","repository":{"id":254406810,"uuid":"846431192","full_name":"Subhashchandra-Birajdar/SpringBatch_CSV_to_DB","owner":"Subhashchandra-Birajdar","description":"Spring Boot Batch Processing: Dumping CSV Data into Database","archived":false,"fork":false,"pushed_at":"2024-08-23T08:11:47.000Z","size":144,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-04T18:40:41.394Z","etag":null,"topics":["chunked-uploads","chunks","csvreader","csvwriter","exception-handling","filtering","postgresql","spring-batch","spring-batch-jobs","spring-boot","spring-data-jpa","stepbuilder","task-scheduler","transformation"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Subhashchandra-Birajdar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-23T07:35:33.000Z","updated_at":"2024-08-23T08:11:50.000Z","dependencies_parsed_at":"2024-08-23T09:24:37.828Z","dependency_job_id":"1ae3d896-100e-450e-8c1c-6b8e01020ab9","html_url":"https://github.com/Subhashchandra-Birajdar/SpringBatch_CSV_to_DB","commit_stats":null,"previous_names":["subhashchandra-birajdar/springbatch_csv_to_db"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Subhashchandra-Birajdar/SpringBatch_CSV_to_DB","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Subhashchandra-Birajdar%2FSpringBatch_CSV_to_DB","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Subhashchandra-Birajdar%2FSpringBatch_CSV_to_DB/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Subhashchandra-Birajdar%2FSpringBatch_CSV_to_DB/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Subhashchandra-Birajdar%2FSpringBatch_CSV_to_DB/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Subhashchandra-Birajdar","download_url":"https://codeload.github.com/Subhashchandra-Birajdar/SpringBatch_CSV_to_DB/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Subhashchandra-Birajdar%2FSpringBatch_CSV_to_DB/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31870516,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"online","status_checked_at":"2026-04-16T02:00:06.042Z","response_time":69,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chunked-uploads","chunks","csvreader","csvwriter","exception-handling","filtering","postgresql","spring-batch","spring-batch-jobs","spring-boot","spring-data-jpa","stepbuilder","task-scheduler","transformation"],"created_at":"2024-11-20T15:56:50.878Z","updated_at":"2026-04-16T04:02:57.917Z","avatar_url":"https://github.com/Subhashchandra-Birajdar.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Spring Boot Batch Processing: Dumping CSV Data into Database\n\nThis Spring Boot application facilitates batch processing to import data from a CSV file into a database. The process involves reading data from a CSV file, processing it, and persisting it into the database.\n\n## Features\n\n### CSV Data Import\n\n- **Source Data:** The application reads data from a CSV file named `customers.csv`.\n- **File Location:** The CSV file is located in the `src/main/resources` directory of the project.\n- **Data Structure:** The CSV file contains customer information such as ID, first name, last name, email, gender, contact number, country, and date of birth.\n  \n### Chunk Processing\n\n- **Chunk Size:** Data processing is performed in chunks of 10 records.\n- **Efficiency:** Chunk processing enhances the application's efficiency, particularly when dealing with large datasets.\n- **Transaction Management:** Each chunk of data is processed within a transaction to ensure data consistency.\n\n### Parallel Processing\n\n- **Task Executor:** The application utilizes a customized task executor to enable parallel processing of chunks.\n- **Concurrency:** The task executor is configured with a concurrency limit of 10 threads.\n- **Improved Performance:** Parallel processing enhances throughput and performance, allowing for faster data processing.\n\n### Database Persistence\n\n- **Data Persistence:** Processed data is persisted into a database using Spring Data JPA and Hibernate.\n- **Repository:** The `CustomerRepository` interface, extending `JpaRepository`, is used to interact with the database and perform CRUD operations on customer entities.\n- **Automatic Schema Generation:** Hibernate automatically generates database schema based on the entity mappings defined in the application.\n\n### Spring Data JPA\n\n- **JPA Repository:** Spring Data JPA provides a powerful repository abstraction layer on top of JPA, simplifying database access and reducing boilerplate code.\n- **Entity-Repository Association:** Each entity in the application is associated with a corresponding repository interface, allowing for easy database operations.\n\n## Prerequisites\n\n- **Java Development Kit (JDK):** Ensure that JDK 8 or higher is installed on your system.\n- **Apache Maven:** Maven is required to build and run the Spring Boot application.\n- **Database:** Install and configure MySQL or another compatible database server. Ensure that the database server is running locally.\n\n## Usage\n\n1. **Clone Repository:** Clone the repository to your local machine using the following command:\n\n   ```bash\n   git clone \u003chttps://github.com/Paras2322/SpringBatch-CSV-to-DB.git\u003e\n   ```\n\n2. **Navigate to Project Directory:** Change to the project directory using the following command:\n\n   ```bash\n   cd spring-boot-batch-processing\n   ```\n\n3. **Place CSV File:** Ensure that the `customers.csv` file containing the data to be imported is placed in the `src/main/resources` directory of the project.\n\n4. **Run Application:** Execute the application using Maven with the following command:\n\n   ```bash\n   mvn spring-boot:run\n   ```\n4. **Test :** Through Postman Run\n   ![postman client](https://github.com/user-attachments/assets/a4d171ad-8317-4cc0-850a-9c669c1f9b02)\n\n5. **Monitor Progress:** Monitor the console logs for batch processing status and any errors encountered during execution.\n\n## Customized Task Executor\n\nIn this application, the task executor is customized to allow parallel processing of chunks. The following configuration is used to define the task executor bean:\n\n```java\n@Bean\npublic TaskExecutor taskExecutor(){\n    SimpleAsyncTaskExecutor asyncTaskExecutor = new SimpleAsyncTaskExecutor();\n    asyncTaskExecutor.setConcurrencyLimit(10);\n    return asyncTaskExecutor;\n}\n```\n\n## Contributions\n\nContributions to this project are welcome! If you have any suggestions, enhancements, or bug fixes, feel free to open an issue or submit a pull request.\n\n## Additional Information\n\n- **Official Documentation:** Refer to the [official documentation](https://spring.io/projects/spring-batch) for more information on Spring Batch framework.\n- **Customization:** Customize batch processing logic, chunk size, and task executor configuration as per your specific requirements.\n- **Maintenance:** Regularly monitor batch job execution and database performance for optimal operation of the application.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsubhashchandra-birajdar%2Fspringbatch_csv_to_db","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsubhashchandra-birajdar%2Fspringbatch_csv_to_db","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsubhashchandra-birajdar%2Fspringbatch_csv_to_db/lists"}