{"id":23383797,"url":"https://github.com/mdwheele/shovel","last_synced_at":"2026-05-08T13:32:58.526Z","repository":{"id":71079340,"uuid":"84272714","full_name":"mdwheele/shovel","owner":"mdwheele","description":"A tool to shovel data from one place to another.","archived":false,"fork":false,"pushed_at":"2017-03-10T20:58:38.000Z","size":87,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-08T09:58:09.304Z","etag":null,"topics":["mariadb","mysql","ops","oracle","php"],"latest_commit_sha":null,"homepage":null,"language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mdwheele.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-03-08T03:10:57.000Z","updated_at":"2017-03-08T06:12:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"a21cc819-7993-4e08-b6d3-4f01b3a37a55","html_url":"https://github.com/mdwheele/shovel","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mdwheele/shovel","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdwheele%2Fshovel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdwheele%2Fshovel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdwheele%2Fshovel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdwheele%2Fshovel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mdwheele","download_url":"https://codeload.github.com/mdwheele/shovel/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mdwheele%2Fshovel/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32782608,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-08T08:22:46.396Z","status":"ssl_error","status_checked_at":"2026-05-08T08:22:45.650Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mariadb","mysql","ops","oracle","php"],"created_at":"2024-12-21T22:30:01.995Z","updated_at":"2026-05-08T13:32:58.482Z","avatar_url":"https://github.com/mdwheele.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Shovel\n\n[![Build Status](https://travis-ci.org/mdwheele/shovel.svg?branch=master)](https://travis-ci.org/mdwheele/shovel)\n\n**Yeah, don't use this...**\n\nA tool to shovel data from one place to another.\n\nShovel is written in PHP, because that's what you do. It's a command-line utility that tries to make it braindead simple to move data from one table in one database server to... another table... in another database server. \n\nStory time:\n\n\u003e Let's say you're a baller and you shelled out dat moolah for Oracle 12c, 11a, 11g, 4G, LTE, wahtever... and now some freaking plebs don't wanna pay up but they want your data. So they use this tool to pipe all that Oracle crap over to MariaDB (until it's bought by Oracle or whatever, you know).\n\nStory time is over, fools.\n\n## Usage \n\nWell... you're going to make these YAML files becuase that's what you do. They're going to look super special too and say a lot. Something like this:\n\n```yaml\n# official-to-trendy.yaml\nsrc:\n  driver: oracle\n  host: something.official.as.fk\n  database: xe\n  username: ohlawd\n  password: ohl$wd?\n  \ndest:\n  driver: mariadb\n  host: something.trendy.as.fk\n  database: ohlawd\n  username: mustache\n  password: mu$tac$e!\n\ntables:\n  - PS_NC_HELLO_WORLD\n  - PS_NC_HELLO_CONTINENT\n  - PS_NC_HELLO_COUNTRY\n\ntransforms:\n  - SomeClassNameInPipeline\n```\n\nSo once we describe the pull we want to do (by specifying source and destination servers, tables to mirror and any transformations to apply), we can run the pull like this:\n\n```bash\n$ bin/shovel dig official-to-trendy.yaml\n```\n\nDigging does the following to be super helpful to us because it's a cool buddy:\n\n- If tables do not exist in the destination database, they will be created with a schema matching the source.\n- If the number of rows in the source are below destination beyond some threshold, complaints are filed.\n- If all is well, build some fancy progress meter shit.\n- Pick a chunk of rows in source table, insert into destination.\n- Slack shit.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmdwheele%2Fshovel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmdwheele%2Fshovel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmdwheele%2Fshovel/lists"}