{"id":22079771,"url":"https://github.com/learninglocker/v1-to-v2-migrator","last_synced_at":"2025-07-24T14:31:43.016Z","repository":{"id":52146706,"uuid":"81332426","full_name":"LearningLocker/v1-to-v2-migrator","owner":"LearningLocker","description":null,"archived":false,"fork":false,"pushed_at":"2021-05-06T20:33:16.000Z","size":127,"stargazers_count":4,"open_issues_count":6,"forks_count":6,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-01T11:02:36.019Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LearningLocker.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-02-08T13:25:15.000Z","updated_at":"2023-05-07T18:33:09.000Z","dependencies_parsed_at":"2022-09-15T05:00:57.544Z","dependency_job_id":null,"html_url":"https://github.com/LearningLocker/v1-to-v2-migrator","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/LearningLocker/v1-to-v2-migrator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LearningLocker%2Fv1-to-v2-migrator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LearningLocker%2Fv1-to-v2-migrator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LearningLocker%2Fv1-to-v2-migrator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LearningLocker%2Fv1-to-v2-migrator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LearningLocker","download_url":"https://codeload.github.com/LearningLocker/v1-to-v2-migrator/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LearningLocker%2Fv1-to-v2-migrator/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266855772,"owners_count":23995545,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-24T02:00:09.469Z","response_time":99,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-30T23:11:14.418Z","updated_at":"2025-07-24T14:31:42.677Z","avatar_url":"https://github.com/LearningLocker.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Learning Locker v1 to v2 Migrator\n\u003e Migrates v1 statements, clients, and stores to a v2 instance\n\n## Migrating from v1 to v2\n\nThis tool will take a Learning Locker v1 database and migrate the LRS, Clients and xAPI records (Statements, Attachments, Documents) into a v2 database format. \n\nPlease note that you will need to ensure your v1 instance is upgraded to **at least v1.12.0 or later**.\n\nUsers, Reports and Exports are not migrated as part of this process. You will need to install and create your v2 Organisation before running this migration (please see documentation) - this also sets up your initial admin user who will be able to invite subsequent users.\n\nAll client credentials attached to migrated LRS's are copied during this migration. \n\nOnce the migration has succesffully been run, only the URL of Learning Locker will need to be changed on any Activity Provider currently sending in statements.\n\nA timestamp is appended to the config after the initial run; subsequent runs of the migration script will then also take any statements that had been inserted into v1 since this date, allowing for any \"missed\" statements to be retrieved and migrated whilst switching over the Learning Locker URL. This process can be repeated as many times as required.\n\n\n## Backups\n\n**Backups of your v1 files and database should also be taken prior to using this script.** We cannot be held responsible for any loss, damage or accidental termination of your data.\n\n**For that reason it is highly recomended that the source and target are different Mongo databases, if not different hosts.** File storage should also be separated where possible.\n\n\n## Basic usage\n\nThe script works in 4 broad phases:\n\n* Exporting required database records from your \"source\" v1 LRS using `mongodump`\n* Restoring to a \"local\" Mongo instance using `mongorestore`, where migrations will be perfomed on the data\n* Exporting and restoring data to a \"target\" database using `mongodump` and `mongorestore`\n* Copying and migrating local file storage to a \"target\" folder - note that this stage does not have a \"local\" step, and will migrate your files directly from \"source\" to \"target\"\n\n\n## Requirements\n\n- A fully installed and configured v2 instance\n\n\nAn instance with:\n- Node 6.*\n- A running Mongo instance accesible at `127.0.0.1:27017` (no user credentials)\n- Access to both the source and target databases\n\n\n## Installation\n\n1. Clone the repository `git clone git@github.com:LearningLocker/v1-to-v2-migrator.git`.\n1. Install dependencies `npm install`.\n1. Change your permissions `sudo chmod 775 ./bin/migrater`.\n\n\n## Configuration\n\nCopy the config/example.json into a new file. The config file is broken into 3 areas:\n\n\n### Source\n\nThe source points to the existing v1 database and files (attachement, documents). \n\nParam | Description | v1 parameter\n--- | --- | ---\n`source.lrs` | The `_id` of LRS you wish to migrate.\u003cbr\u003e\u003cbr\u003eIt is possible to filter to a single LRS if required . This is useful if you require migrating different LRSs to different v2 instances, or even different Organisations within the same v2 instance.\u003cbr\u003e\u003cbr\u003eBy default, if left blank, all LRSs will be migrated. | The lrs's `_id`\n`source.database` | **This section contains information about your source v1 database** | `app/config/.../database.php`\n`source.database.hosts` | The host(s) of you source Mongo instance | `host`\n`source.database.name` | Database name | `database`\n`source.database.user` | User with read access to database | `username`\n`source.database.password` | Password | `password\n`source.database.authenticationDatabase` | Mongo authentication database (if required) | `options.authenticationDatabase`\n`source.database.ssl` | Boolean - use SSL for transfer | `options.ssl`\n`source.documentStorage.storageType` | Either `local` or `s3`\u003cbr\u003e\u003cbr\u003eDetermines where the script will look to find v1 Attachments and Documents | `FS_REPO`\n`source.documentStorage.local.storageDir` | The absolute path to the v1 storage directory | `FS_LOCAL_ENDPOINT`\n`source.documentStorage.s3` | **This section contains information about your AWS S3 v1 storage** | \n`source.documentStorage.s3.bucketName` | The S3 bucket | 'FS_S3V3_BUCKET'\n`source.documentStorage.s3.subFolder` | The subFolder within S3 | 'FS_S3V3_PREFIX'\n`source.documentStorage.s3.region` | The S3 region | `FS_S3V3_REGION`\n`source.documentStorage.s3.accessKeyId` | AWS access key ID | `FS_S3V3_ACCESS_KEY`\n`source.documentStorage.s3.secretAccessKey` | AWS secret access key | `FS_S3V3_SECRET`\n\n\n### Local\n\nParam | Description\n--- | ---\n`local.database` | Database name on the local `127.0.0.1:27017` instance.\u003cbr\u003e\u003cbr\u003eData is restored, migrated and exported from this database and will be cleared down with subsequent runs of the scripts.\n`local.sourceDumpLocation` | Absolute or relative directory to export source mongodump to\n`local.targetDumpLocation` | Absolute or relative directory to export migrated mongodump, prior to target restoration (mongorestore)\n\n\n### Target\n\nParam | Description\n--- | ---\n`target.orgId` | The `_id` of the Organisation where all data will be inserted to.\u003cbr\u003e\u003cbr\u003eThis can be found in your newly setup v2 database, or simply by looking at the URL of an organisation when you have accessed it in the v2 GUI.\n`target.database` | **This section contains information about your target v1 database**\n`target.database.hosts` | The host(s) of you target Mongo instance\n`target.database.name` | Database name\n`target.database.user` | User with read access to database\n`target.database.password` | User password\n`target.database.authenticationDatabase` | Mongo authentication database (if required)\n`target.database.ssl` | Boolean - use SSL for transfer\n`target.documentStorage.storageType` | Either `local` or `s3`\u003cbr\u003e\u003cbr\u003eDetermines where the script will push Attachments and Documents for v2\n`target.documentStorage.local.storageDir` | The absolute path to the target storage directory\n`target.documentStorage.s3` | **This section contains information about your AWS S3 v2 storage**\n`target.documentStorage.s3.bucketName` | The S3 bucket \n`target.documentStorage.s3.subFolder` | The subFolder within S3\n`target.documentStorage.s3.region` | The S3 region\n`target.documentStorage.s3.accessKeyId` | AWS access key ID\n`target.documentStorage.s3.secretAccessKey` | AWS secret access key\n\n\n## Command\n\nTo run the process, execute the following:\n\n```\n./bin/migrater \u003cconfig_file_location\u003e [start_step [end_step]]\n```\n`config_file_location`: A relative or absolute path to your config file\n\n`start`: The first \"step\" you wish to run (defaults to 1)\n\n`end`: The last \"step\" you wish to run (defaults to 8)\n\n\n### Example\n```\n./bin/migrater config/ht2.json 1 5\n```\n\n\n### Steps\n\n1. `writeTimestamp` - Write a current timestamp to the config\n1. `clearLocalData` - Clear down existing \"local\" database and database mongodump/mongorestores\n1. `dumpSourceData` - mongodump all matching data from the \"source\"\n1. `restoreLocalData` - restore the dumped data to \"local\"\n1. `migrateLocalData` - locally migrate from v1 to v2\n1. `migrateDocumentStorage` - Migrate state documents (files) directly from \"source\" to \"target\" (**no local step provided here**)\n1. `migrateAttachmentStorage` - Migrate statement attachment files directly from \"source\" to \"target\" (**no local step provided here**)\n1. `dumpLocalData` - Dump the locally migrated data\n1. `restoreTargetData` - Restore migrated data into target\n\n\n## Full process:\n\n1. Copy the `config/example.json` file into a new config file\n1. Create an organisation in your V2 instance - enter the organisation's `_id` into the config\n1. (optional) Find the LRS `_id` in the V1 instance for use in the config file, or leave blank to copy all LRSs\n1. Run `node ./bin/migrater config/YOURCONFIG.json` in this directory.\n\n\n## Final steps\n\nLearning Locker's workers are setup to provide extra processing to statements as they are consumed through the LRS.  Your newly migrated data will have not have had these processes applied. You may run the following to provide that functionality.\n\nInstead, you will need to run these commands on your v2 instance at the root of where v2 is installed\n\nUpdate the store count on each LRS:\n```\nnode cli/dist/server updateStatementCount \n```\n\n#### Query Builder Caches:\n\nPopulate the Query Builder:\n```\nnode cli/dist/server batchJobs -j querybuildercache\n```\n\nPersona processing:\n\nCreate \"personas\" for each agent:\n```\nnode cli/dist/server batchJobs -j personas -b 1000\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flearninglocker%2Fv1-to-v2-migrator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flearninglocker%2Fv1-to-v2-migrator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flearninglocker%2Fv1-to-v2-migrator/lists"}