{"id":23081604,"url":"https://github.com/bluecolor/lauda","last_synced_at":"2026-05-02T04:39:30.559Z","repository":{"id":92200087,"uuid":"260969138","full_name":"bluecolor/lauda","owner":"bluecolor","description":"Cross database data transfer tool","archived":false,"fork":false,"pushed_at":"2020-07-01T21:34:24.000Z","size":80938,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-03T14:16:10.716Z","etag":null,"topics":["data","database","etl","extract","jdbc","load"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bluecolor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-03T16:11:00.000Z","updated_at":"2020-07-01T21:34:26.000Z","dependencies_parsed_at":"2023-06-07T23:15:37.430Z","dependency_job_id":null,"html_url":"https://github.com/bluecolor/lauda","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/bluecolor/lauda","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluecolor%2Flauda","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluecolor%2Flauda/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluecolor%2Flauda/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluecolor%2Flauda/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bluecolor","download_url":"https://codeload.github.com/bluecolor/lauda/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluecolor%2Flauda/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32523428,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-02T01:12:54.858Z","status":"online","status_checked_at":"2026-05-02T02:00:05.923Z","response_time":132,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","database","etl","extract","jdbc","load"],"created_at":"2024-12-16T13:54:05.775Z","updated_at":"2026-05-02T04:39:30.533Z","avatar_url":"https://github.com/bluecolor.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Lauda\n\nAn ETL'ish tool that ransfers data between any JDBC compliant databases.\n\nLauda has its own simple repository in given database, that can be configured and loaded either from command line\nor directly in database. Keeping extract-load definitions in database, makes it easier for database people to\nmanage transfers using a database client. And for the same reason lauda has a simple database model and no `orm` in source.\n\nLauda has three concepts for defining an ETL process;\n\n- `connection`: Connection definition to source and target databases. These definitions are\nkept in `connections` table in repository. A Connection has the following attributes.\n\n  - `name`: Unique name of the connection. Case sensitive.\n  - `url`: JDBC url for the connection. Example: `jdbc:oracle:thin:@localhost:1521:orcl`\n  - `username`: user name for connection.\n  - `password`: passowrd for connection.\n  - `class_name`: Class name of the JDBC driver. Example: `oracle.jdbc.driver.OracleDriver`\n\n- `mapping`: Mapping definition of the extract and load targets. These data is kept in `mappings` table\nin repository. A mapping has the following attributes.\n  - `name`: Unique name of the mapping. Case sensitive.\n  - `source_connection`: Name of the source connection. Lookup to `connections`\n  - `target_connection`: Name of the target connection. Lookup to `connections`\n  - `source_table`: Name of the source table. Give database/schema prefix if table is another database/schema than the connection. For example the value can be `my_schema.table_name` or just `table_name`\n  - `target_table`: Name of the target table. Give database/schema prefix if table is another database/schema than the connection. For example the value can be `my_schema.table_name` or just `table_name`\n  - `source_hint`: Optional source query hint. Example `/*+ parallel(8) */`\n  - `target_hint`: Optional target statement hint. Example `/*+ nologging */`\n  - `filter`: Filter expression for the source. Example `column_a in ('X', 'Y', 'Z') and column_b=2`\n  - `batch_size`: Batch select and insert size. Defaults to 1000\n  - `drop_create`: Drop the target table before insert. Defaults to `0`, `false`\n\n  Since some databases do not support boolean, in repository tables, boolean options like `drop_create` are managed by integers. For `false` give the value `0` for `true` give any positve integer.\n  In repository data files(see command line options and data file loading) you can use `true` or `false`.\n\n- `column mapping`: Column mapping pairs between source and target tables. This mappings are stored in `columns`\ntable in repository. Has the following attributes;\n\n  - `mapping`: Name of the mapping, this pair belongs. Lookup to `mappings`.\n  - `source`: Name of the source column or an expression. Example: `COL_A` or `to_char(COL_DATE, 'yyymmdd')`\n  - `target`: Name of the target column that source will be inserted.\n\n\n### Installation\n- Make sure you have java 1.8 or greater\n- Download archive file from [dist](/dist)\n- Extract `lauda-[version].tar.gz` to desired location.\n- Put the required `jdbc` drivers to `lib` folder\n\n\n### Building from source\n- Make sure you have [sbt](https://www.scala-sbt.org/)\n- Clone the repository\n- run `dist.sh`\n\n### Configuration\n- Use `config.yml` for configuratin\n\n### Command line arguments\n```\nUsage: lauda [-hV] [--columns=\u003ccolumns\u003e]\n             [--source-connection=\u003csourceConnection\u003e]\n             [--source-table=\u003csourceTable\u003e]\n             [--target-connection=\u003ctargetConnection\u003e]\n             [--target-table=\u003ctargetTable\u003e] \u003ccommand\u003e\nLoads data between databases https://github.com/bluecolor/lauda\n      \u003ccommand\u003e             Command to exeucte\n                            See https://github.\n                              com/bluecolor/lauda#command-line-arguments\n      --columns=\u003ccolumns\u003e   Comma seperated list of columns. Optional\n  -h, --help                Show this help message and exit.\n      --source-connection=\u003csourceConnection\u003e\n                            Source connection name\n      --source-table=\u003csourceTable\u003e\n                            Source table name. can be schema.table_name or just\n                              table_name\n      --target-connection=\u003ctargetConnection\u003e\n                            Target connection name\n      --target-table=\u003ctargetTable\u003e\n                            Target table name. can be schema.table_name or just\n                              table_name\n  -V, --version             Print version information and exit.\n```\n\n`command` command can be one of the following and can take parameters;\n\n  - `repository.up`: Initialize repository. Uses `config.yml`\n  - `repository.down`: Drops the repsitory. Uses `config.yml`\n  - `repository.import`: Import repository data with parameter\n  - `repository.print.connections`: Print connections\n  - `repository.print.mappings`: Print mappings\n  - `repository.print.columns`: Print source and target columns of given mapping with\n  - `mapping.delete`: Delete a mapping by name with parameter\n  - `mapping.exists`: Check if mapping exists with parameter\n  - `mapping.run`: Run the mapping given with\n  - `mapping.create`: Create target table in given mapping with\n  - `connection.delete`: Delete connection by name\n  - `connection.test`: Test jdbc connection by name\n\n**Examples**\n\n  Create repository, using the parameters in `config.yml`\n  ```sh\n  ./lauda.sh repository.up\n  ```\n\n  Reset/drop repository, using the parameters in `config.yml`\n  ```sh\n  ./lauda.sh repository.down\n  ```\n\n  Import mapping and connection definitions. [See example data file](/examples/repository-seed-01.yml).\n  ```sh\n  ./lauda.sh repository.import examples/repository-seed.yml\n  ```\n\n  Print available connections\n  ```sh\n  ./lauda.sh repository.print.connections\n  ```\n\n  Print defined mappings\n  ```sh\n  ./lauda.sh repository.print.mappings\n  ```\n\n  Print the source and target columns of the given mapping\n  ```sh\n  ./lauda.sh repository.print.columns mapping_name\n  ```\n\n  Delete a mapping definition(column mappings also)\n  ```sh\n  ./lauda.sh mapping.delete mapping_name\n  ```\n  ```sh\n  ./lauda.sh mapping.delete \"Mapping name\"\n  ```\n\n  Check if mapping with given name already exists\n  ```sh\n  ./lauda.sh mapping.exists mapping_name\n  ```\n\n  Run mapping\n  ```sh\n  ./lauda.sh mapping.run mapping_name\n  ```\n\n  Create target table in mapping. Does not run mapping, only created target.\n  ```sh\n  ./lauda.sh mapping.create mapping_name\n  ```\n\n  Delete given connection definition\n  ```sh\n  ./lauda.sh connection.delete connection_name\n  ```\n\n  ```sh\n  ./lauda.sh connection.delete \"Connection name\"\n  ```\n\n  Test the connection\n  ```sh\n  ./lauda.sh connection.test connection_name\n  ```\n\n  Generate mapping\n  ```sh\n    ./lauda.sh mapping.generate table_4 \\\n      --source-connection oracle_1 \\\n      --target-connection postgre_2 \\\n      --source-table source_table_3 \\\n      --target-table target_table_3\n  ```\n\n\n\n### Loading definitions\n  An ETL definiton can be given directly using a database client in repository or using\n  command line option. With a database client you can execute a script like [this one](/examples/repository-seed-01.sql)\n  For command line, use the `repsitory.import` command and give data file like\n  [this one](/examples/repository-seed-01.yml). See [examples](/examples) for different options.\n\n  Example command line usage;\n  `./lauda.sh repository.import /path/to/data-file.yml`\n  `./lauda.sh repository.import ./data-file.yml`\n\n\n\n### Oracle to Oracle transfers\nFor data transfers between two oracle databases you can use [elo](https://github.com/bluecolor/elo)\nwhich uses oracle dblink's and is faster.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbluecolor%2Flauda","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbluecolor%2Flauda","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbluecolor%2Flauda/lists"}