{"id":19592083,"url":"https://github.com/catalyst/moodle-tool_etl","last_synced_at":"2025-04-27T14:33:37.894Z","repository":{"id":40434626,"uuid":"111480360","full_name":"catalyst/moodle-tool_etl","owner":"catalyst","description":"A general purpose ETL tool","archived":false,"fork":false,"pushed_at":"2023-11-28T18:14:55.000Z","size":621,"stargazers_count":6,"open_issues_count":22,"forks_count":10,"subscribers_count":28,"default_branch":"master","last_synced_at":"2025-04-05T00:51:14.186Z","etag":null,"topics":["etl","moodle","moodle-plugin"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/catalyst.png","metadata":{"files":{"readme":"README.md","changelog":"history.php","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-21T00:59:39.000Z","updated_at":"2024-02-29T10:29:31.000Z","dependencies_parsed_at":"2024-11-11T08:35:01.320Z","dependency_job_id":"5e855557-6614-4fd2-83d4-72e7c1c6f253","html_url":"https://github.com/catalyst/moodle-tool_etl","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catalyst%2Fmoodle-tool_etl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catalyst%2Fmoodle-tool_etl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catalyst%2Fmoodle-tool_etl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catalyst%2Fmoodle-tool_etl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/catalyst","download_url":"https://codeload.github.com/catalyst/moodle-tool_etl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251154689,"owners_count":21544543,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["etl","moodle","moodle-plugin"],"created_at":"2024-11-11T08:33:02.672Z","updated_at":"2025-04-27T14:33:35.878Z","avatar_url":"https://github.com/catalyst.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ca href=\"https://github.com/catalyst/moodle-tool_etl/actions?query=workflow%3A%22Run+all+tests%22\"\u003e\n\u003cimg src=\"https://github.com/catalyst/moodle-tool_etl/workflows/Run%20all%20tests/badge.svg\"\u003e\n\u003c/a\u003e\n\n# Extract, transform, load (ETL)\nETL is a Moodle admin tool that allows to extract, transform and then load any data from a source system to a target system.\n\nFor example it is often used to slightly massage data before using it with Totara HR sync.\n\n* [Branches](#branches)\n* [Installation](#installation)\n* [Configuration](#configuration)\n* [Contributing and Support](#contributing-and-support)\n\n\n## Branches\n\n| Moodle version     | Branch  | PHP        |\n|--------------------| ------- | ---------- |\n| Moodle 3.3 to 3.8  | master  | 7.1 - 7.2+ |\n| Moodle 3.9 to 3.10 | master  | 7.2 - 7.4+ |\n| Totara 12+         | master  | 5.6 - 7.0+ |\n\nNOTE: Moodle 3.11 is not supported. This is going to be the end of life version of the plugin.\n\nNOTE: There is a fork of this under the MOODLE_35 branch which moves to a sub plugin architecture, and has various improvements:\n\nhttps://github.com/catalyst/moodle-tool_etl/tree/MOODLE_35\n\nAt some point that branch needs to be untangled and parts of that merged into master with a proper upgrade script. Until then please consider than branch as experimental.\n\n\n## Installation\nTo install this plugin in your Moodle.\n1. Get the code and copy/install it to: `\u003cmoodledir\u003e/admin/tool/etl`\n2. Run the upgrade using Moodle admin interface or command line, like `sudo -u www-data php admin/cli/upgrade.php`\n\n**Note:** the user may be different to www-data on your system.\n\n## Configuration\n1. Log into your Moodle site as an administrator.\n2. Navigate to  `Site administration ► Plugins ► Admin tools ► Extract, transform, load (ETL)`.\n3. Create a new task: set Source, Target and Processor and configure each element as required. Also set a schedule for the task.\n4. The tasks will be executed on Moodle cron.\n5. You can see a history of any task execution.\n\n## Existing elements\n\n### Sources\n* **FTP** - remote FTP server, login using username and password.\n* **SFTP** - remote SFTP server, login using username and password.\n* **SFTP with key authentication** - remote SFTP server, login using SSH key.\n* **Server folder** - server local folder.\n* **Database** - SQL query to export.\n* **URL** - remote URL.\n\n### Targets\n* **Moodle sitedata** - a folder inside Moodle sitedata directory.\n* **Server folder** - server local folder.\n* **SFTP with key authentication** - remote SFTP server, login using SSH key.\n\n\n### Processors\n* **Default processor** - doesn't do any transformation, simply pass extracted data from a source to a target.\n* **Lowercase processor** - transforms csv fields to lowercase \n\n## Developer notes\nTo create a new source, target or processor you'd just need to create a new class and extend a relevant base class and implement related interface. See existing elements as an example.\n\n# Crafted by Catalyst IT\n\nThis plugin was developed by Catalyst IT Australia:\n\nhttps://www.catalyst-au.net/\n\n# Contributing and Support\n\nIssues, and pull requests using github are welcome and encouraged!\n\nhttps://github.com/catalyst/moodle-tool_etl/issues\n\nIf you would like commercial support or would like to sponsor additional improvements\nto this plugin please contact us:\n\nhttps://www.catalyst-au.net/contact-us\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcatalyst%2Fmoodle-tool_etl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcatalyst%2Fmoodle-tool_etl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcatalyst%2Fmoodle-tool_etl/lists"}