https://github.com/thbar/tinytl

Lightweight ETL in Ruby (experimental fork of activewarehouse-etl)
https://github.com/thbar/tinytl

Last synced: about 1 month ago
JSON representation

Lightweight ETL in Ruby (experimental fork of activewarehouse-etl)

• Host: GitHub
• URL: https://github.com/thbar/tinytl
• Owner: thbar
• Created: 2011-06-04T20:15:07.000Z (over 13 years ago)
• Default Branch: master
• Last Pushed: 2011-06-06T12:48:18.000Z (over 13 years ago)
• Last Synced: 2024-10-14T02:50:05.400Z (2 months ago)
• Homepage:
• Size: 89.8 KB
• Stars: 3
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
An experimental fork of activewarehouse-etl (work in progress).

[![](http://travis-ci.org/thbar/tinytl.png)](http://travis-ci.org/#!/thbar/tinytl)

Mostly a "free thinking" experiment. There is no certainty I will keep working on this.

## Ideal goals

* should be compatible with

  * MRI 1.9.2

  * MRI 1.8.7

  * JRuby 1.6.2

  * Windows

* should be well-tested to make it easy to add/change/remove features

* should have tests that are dead-easy to run

* should have a core that is independent of ActiveRecord

* should have plugins for database-specific code

* should be documented from the code

* should have a command line binary

* should support bundler

* should largely use the syntax and transforms coming from aw-etl

* should support pre-post-load screens

* should run its tests in production

  * the rational behind this is that the etl can't be tested against all the databases and specific configurations (think: collations, encodings, os, bulk load behaviour), so at least one run should be done in production to check everything works as expected

* should support user-defined profiling

* could support multi-core

* could support multi-agent

## Random thoughts

* not sure to use Cucumber or not

* having a well-bundled "upsert on unique keys" destination would be nice