Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/sboussekeyt/spring-batch-rs
A toolkit for building enterprise-grade batch applications
https://github.com/sboussekeyt/spring-batch-rs
batch csv json mongodb mysql postgres rust sqlite
Last synced: 3 days ago
JSON representation
A toolkit for building enterprise-grade batch applications
- Host: GitHub
- URL: https://github.com/sboussekeyt/spring-batch-rs
- Owner: sboussekeyt
- License: apache-2.0
- Created: 2023-04-08T06:44:07.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-11-12T12:07:56.000Z (3 days ago)
- Last Synced: 2024-11-12T13:17:25.630Z (3 days ago)
- Topics: batch, csv, json, mongodb, mysql, postgres, rust, sqlite
- Language: Rust
- Homepage:
- Size: 313 KB
- Stars: 6
- Watchers: 1
- Forks: 1
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- License: LICENSE-APACHE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
Spring-Batch for Rust
🐞 A toolkit for building enterprise-grade batch applications
[](https://crates.io/crates/spring-batch-rs)
[](https://docs.rs/spring-batch-rs)
[](https://github.com/sboussekeyt/spring-batch-rs/actions/workflows/test.yml)
[](https://discord.gg/9FNhawNsG6)
[](https://codecov.io/gh/sboussekeyt/spring-batch-rs)
# Spring-Batch for Rust
Spring Batch for Rust, offers a robust and flexible framework for the development of batch processing applications, addressing the challenges of handling large-scale data processing tasks efficiently and reliably. It provides developers a comprehensive toolkit for building enterprise-grade batch applications.
## Features
| **Feature** | **Description** |
|---------------|---------------------------------------------------------------|
| mongodb | Enable reader and writer for Mongodb database |
| rdbc-postgres | Enable rdbc reader and writer for Postgres database |
| rdbc-mysql | Enable rdbc reader and writer for Mysql and MariaDb databases |
| rdbc-sqlite | Enable rdbc reader and writer for Sqlite database |
| json | Enable json reader and writer |
| csv | Enable csv reader and writer |
| fake | Enable fake reader. Useful for generate fake dataset |
| logger | Enable logger writer. Useful for debugging |
## Roadmap
+ XML reader and writer
+ Filter items
+ Kafka reader and writer
+ Pulsar reader and writer
+ Retry/Skip policies
+ Save execution data in database
## Getting Started
Make sure you activated the suitable features crate on Cargo.toml:
```toml
[dependencies]
spring-batch-rs = { version = "", features = [""] }
```
Then, on your main.rs:
```rust,no_run
fn main() -> Result<(), BatchError> {
let csv = "year,make,model,description
1948,Porsche,356,Luxury sports car
1995,Peugeot,205,City car
2021,Mazda,CX-30,SUV Compact
1967,Ford,Mustang fastback 1967,American car";
let reader = CsvItemReaderBuilder::new()
.delimiter(b',')
.has_headers(true)
.from_reader(csv.as_bytes());
let processor = UpperCaseProcessor::default();
let writer = JsonItemWriterBuilder::new().from_path(temp_dir().join("cars.json"));
let step: Step = StepBuilder::new()
.reader(&reader) // set csv reader
.writer(&writer) // set json writer
.processor(&processor) // set upper case processor
.chunk(2) // set commit interval
.skip_limit(2) // set fault tolerance
.build();
let job = JobBuilder::new().start(Box::new(&step)).build();
let result = job.run();
assert!(result.is_ok());
assert!(step.get_status() == StepStatus::Success);
Ok(())
}
```
## Examples
+ [Generate CSV file from JSON file with processor](https://github.com/sboussekeyt/spring-batch-rs/blob/main/examples/generate_csv_file_from_json_file_with_processor.rs)
+ [Generate JSON file from CSV string with fault tolerance](https://github.com/sboussekeyt/spring-batch-rs/blob/main/examples/generate_json_file_from_csv_string_with_fault_tolerance.rs)
+ [Generate JSON file from fake persons](https://github.com/sboussekeyt/spring-batch-rs/blob/main/examples/generate_json_file_from_fake_persons.rs)
+ [Generate CSV file without headers from fake persons](https://github.com/sboussekeyt/spring-batch-rs/blob/main/examples/generate_csv_file_without_headers_from_fake_persons.rs)
+ [Insert records into Mysql database](https://github.com/sboussekeyt/spring-batch-rs/blob/main/examples/insert_records_into_mysql_database.rs)
+ [Log records from Postgres database](https://github.com/sboussekeyt/spring-batch-rs/blob/main/examples/log_records_from_postgres_database.rs)
+ [Read records from MongoDb database](https://github.com/sboussekeyt/spring-batch-rs/blob/main/examples/read_records_from_mongodb_database.rs)
+ [Write records to MongoDb database](https://github.com/sboussekeyt/spring-batch-rs/blob/main/examples/write_records_to_mongodb_database.rs)
## License
Licensed under either of
- Apache License, Version 2.0
([LICENSE-APACHE](LICENSE-APACHE) or )
- MIT license
([LICENSE-MIT](LICENSE-MIT) or )
at your option.
## Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in the work by you, as defined in the Apache-2.0 license, shall be
dual licensed as above, without any additional terms or conditions