{"id":18457934,"url":"https://github.com/alokkusingh/home-etl-service","last_synced_at":"2025-04-23T12:18:48.083Z","repository":{"id":95088313,"uuid":"581614012","full_name":"alokkusingh/home-etl-service","owner":"alokkusingh","description":"Home Stash ETL Service","archived":false,"fork":false,"pushed_at":"2025-03-30T05:06:12.000Z","size":1418,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-23T12:18:30.526Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alokkusingh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-23T18:08:09.000Z","updated_at":"2025-03-30T05:06:15.000Z","dependencies_parsed_at":"2023-03-05T07:00:12.993Z","dependency_job_id":"fff25151-1365-4cce-9dbc-76cefba8b493","html_url":"https://github.com/alokkusingh/home-etl-service","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alokkusingh%2Fhome-etl-service","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alokkusingh%2Fhome-etl-service/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alokkusingh%2Fhome-etl-service/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alokkusingh%2Fhome-etl-service/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alokkusingh","download_url":"https://codeload.github.com/alokkusingh/home-etl-service/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250430601,"owners_count":21429324,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T08:16:05.378Z","updated_at":"2025-04-23T12:18:48.056Z","avatar_url":"https://github.com/alokkusingh.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Home ETL Service \nHome Stack ETL Service\n\n## Functionality\n- Reads downloaded bank PDF statements (with password/without password protection) or imported CSV files from bank site\n- Parses it (based on plugable parsing logic)\n- Process the records (transaction categorization, amount extraction)\n- Writes to H2/MySql DB\n- Finally export to csv format order by transaction date to be imported to Excel or Google Sheet\n\n### Supported Bank Statemetnts\n1. Citi Bank Saving Account \n2. Kotak Mahindra Bank Saving Account \n3. HDFC Bank\n\n### How to run\n````\njava -jar target/home-etl-service-1.0.0-SNAPSHOT.jar --file.path.base.dir=/home/alok/data/git/BankStatements\n````\n\n### Build\n#### Set JAVA_HOME (in case mvn run through terminal)\n```shell\nexport JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-21.jdk/Contents/Home\nexport PATH=$JAVA_HOME/bin:$PATH\n```\n#### Generate Proto file\n```shell\nprotoc --java_out=src/main/java src/main/resources/proto/expenseCategorizer.proto\n```\n#### Generate gRPC Stub\n```shell\nprotoc --plugin=protoc-gen-grpc-java=protoc-gen-grpc-java-1.66.0-osx-aarch_64.exe --grpc-java_out=src/main/java src/main/resources/proto/expenseCategorizer.proto\n```\n1. Maven Package\n   ```shell\n   mvn clean package -X\n   ```\n2. Docker Build, Push \u0026 Run\n   ```shell\n   docker build -t alokkusingh/home-etl-service:latest -t alokkusingh/home-etl-service:2.0.0 --build-arg JAR_FILE=target/home-etl-service-1.0.0-SNAPSHOT.jar .\n   ```\n   ```shell\n   docker push alokkusingh/home-etl-service:latest\n   ```\n   ```shell\n   docker push alokkusingh/home-etl-service:2.0.0\n   ```\n   ```shell\n   docker run -d -v /home/alok/data/git/BankStatements:/Users/aloksingh/BankStatements:rw,Z -p 8081:8081 --rm --name home-etl-service alokkusingh/home-etl-service\n   ```\n   \n### Manual commands\n```shell\ndocker run -it --entrypoint /bin/bash -v /home/alok/data/git/BankStatements:/Users/aloksingh/BankStatements:rw,Z -p 8081:8081 --rm --name home-etl-service alokkusingh/home-etl-service\n```\n```shell\njava -Djava.security.egd=file:/dev/urandom -Dspring.profiles.active=prod -Dspring.datasource.url=jdbc:mysql://192.168.1.200:32306/home-stack -Dspring.datasource.hikari.minimum-idle=5 \\\n-Dspring.datasource.hikari.connection-timeout=20000 -Dspring.datasource.hikari.maximum-pool-size=10 -Dspring.datasource.hikari.idle-timeout=10000 \\\n-Dpring.datasource.hikari.max-lifetime=1000 -Dspring.datasource.hikari.auto-commit=true -jar /opt/app.jar\n```\n```shell\ndocker run -v /home/alok/data/git/BankStatements:/Users/aloksingh/BankStatements:rw,Z -p 8081:8081 --rm --name home-etl-service alokkusingh/home-etl-service \\\n--java.security.egd=file:/dev/urandom --spring.profiles.active=prod --spring.datasource.url=jdbc:mysql://192.168.1.200:32306/home-stack --spring.datasource.hikari.minimum-idle=5 \\\n--spring.datasource.hikari.connection-timeout=20000 --spring.datasource.hikari.maximum-pool-size=10 --spring.datasource.hikari.idle-timeout=10000 \\\n--pring.datasource.hikari.max-lifetime=1000 --spring.datasource.hikari.auto-commit=true\n```\n```shell\ncurl --location 'http://localhost:8081/home/etl/gsheet/refresh/expense' --header 'Accept: text/event-stream;charset=UTF-8'\n```\n```shell\ncurl --location 'http://localhost:8081/home/etl/gsheet/refresh/expense'\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falokkusingh%2Fhome-etl-service","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falokkusingh%2Fhome-etl-service","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falokkusingh%2Fhome-etl-service/lists"}