{"id":19571651,"url":"https://github.com/serlo/lenabi-migration-algorithm","last_synced_at":"2025-02-26T10:43:10.834Z","repository":{"id":152143595,"uuid":"598511675","full_name":"serlo/lenabi-migration-algorithm","owner":"serlo","description":"Ein Migrationsalgorithmus für das Speicher- und Austauschformat ist implementiert. (Dieser ermöglicht, dass parallel neue Lerninhalte für den Editor entwickelt werden können und Lerninhalte im Speicher- und Austauschformat standardisiert werden können.) ","archived":false,"fork":false,"pushed_at":"2023-03-03T20:54:46.000Z","size":11630,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-01-09T03:14:18.417Z","etag":null,"topics":["documentation","poc","serlo"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/serlo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-07T09:06:47.000Z","updated_at":"2023-07-12T15:28:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"883fb31c-329f-4113-9d43-a49d9637eabc","html_url":"https://github.com/serlo/lenabi-migration-algorithm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serlo%2Flenabi-migration-algorithm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serlo%2Flenabi-migration-algorithm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serlo%2Flenabi-migration-algorithm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serlo%2Flenabi-migration-algorithm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/serlo","download_url":"https://codeload.github.com/serlo/lenabi-migration-algorithm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240840032,"owners_count":19866164,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["documentation","poc","serlo"],"created_at":"2024-11-11T06:19:40.799Z","updated_at":"2025-02-26T10:43:10.809Z","avatar_url":"https://github.com/serlo.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"https://github.com/serlo/lenabi-migration-algorithm/blob/main/assets/serlo.svg?raw=true\" alt=\"Serlo Logo\" title=\"Serlo\" align=\"right\" height=\"75\" /\u003e\n\n# Migration Algorithm for Serlo Editor Document Format\n\n## Introduction\n\nNo software is set in stone, and there will always be a need to add or change\nfeatures. However, these changes should not render existing content\ninaccessible. As a relatively young project, the Serlo Editor is poised for\nsignificant development in the near future, so we must be prepared for changes\nin the format of the stored content.\n\n## Overview\n\nTo maintain backward compatibility, the Serlo Editor will employ a built-in\nmigration algorithm. Each document contains a field that defines its current\nformat version, represented by an integer starting from 1. Whenever a format\nchange occurs, the version number is incremented by 1. Moreover, a migration\nfunction will be supplied that converts a document from version `n` to a valid\ndocument of version `n + 1`.\n\n![grafik](https://user-images.githubusercontent.com/13507950/217872727-9c725bb5-b45e-402e-b1de-63a1bff5b30f.png)\n\nBy employing this approach, we can guarantee that the latest version of the\neditor will be able to read all documents created using the same or earlier\nversions of the format. This way, we can ensure that the existing content\nremains accessible even as the software evolves and improves over time.\nAdditionally, this approach also makes it easier to update to the latest version\nof the editor without having to worry about compatibility issues.\n\n## Implementation\n\nThis section outlines a proposed implementation strategy.\n\n### Document Format and version number\n\nAt the top level, a document is encapsulated in a JSON object with three\nproperties:\n\n- `type`: This property identifies the document as one created by the Serlo\n  Editor and has a fixed string value of `\"https://serlo.org/editor\"`.\n- `version`: An integer, as described previously.\n- `content`: This property contains a plugin that is compatible with the Serlo\n  editor. Each plugin has a type and a state, where the state may contain nested\n  plugins (see\n  [description of the Serlo content format](https://github.com/serlo/documentation/wiki/Content-format)).\n\nThis is a minimal example for such a document:\n\n```json\n{\n  \"type\": \"https://serlo.org/editor\",\n  \"version\": 1,\n  \"content\": {\n    \"plugin\": \"article\",\n    \"state\": {}\n  }\n}\n```\n\nThe file\n[document_v1.json](https://github.com/serlo/lenabi-migration-algorithm/blob/49599f2d33d087200b9006b3d45e61d610b13e32/document_v1.json)\ncontains a full example of a Serlo Editor document that implements the proposed\nversion management scheme.\n\n### Migrations\n\nFor every version except the current one, the file\n[migrations.js](https://github.com/serlo/lenabi-migration-algorithm/blob/main/migrations.js)\ndefines in a registry `migrations` a function that transforms the content of a\ndocument from that version to the next version. The basic structure is as\nfollows:\n\n```js\n//migrations.js\n\nconst migrations = {\n  1: function (content) {\n    // transform content from version 1 to version 2\n    return content\n  },\n  2: function (content) {\n    // transform content from version 2 to version 3\n    return content\n  },\n  // ... and more migrations\n}\n```\n\nThose functions can be used to migrate a document `document` from its current\nversion to `targetVersion` by applying the necessary migrations in sequence:\n\n```js\nfunction applyMigrations({ document, targetVersion = getCurrentVersion() }) {\n  for (let v = document.version; v \u003c targetVersion; v++) {\n    document = {\n      ...document,\n      content: migrations[v](document.content),\n      version: v + 1,\n    }\n  }\n\n  return document\n}\n```\n\nThis repository showcases some examples of potential migrations and their\nimplementation (Note that the examples only showcase the possibilities of\nmigrations and do not represent features that are planned to be implemented):\n\n- `v1 -\u003e v2`: A new \"metadata\" property is added to the state of the image\n  plugin. It is given a default value of `{author: null, license: null}` to\n  indicate the absence of any special metadata.\n- `v2 -\u003e v3`: The multimedia plugin is made more generic and renamed to\n  `sidebyside`. The `illustrating` and `width` properties are removed and\n  `explanation` and `multimedia` are renamed to `left` and `right`.\n- `v3 -\u003e v4`: A new property \"caption\" is added to the state of the sidebyside\n  plugin. The default value of the caption is set to the empty string `\"\"`.\n\nYou can refer to\n[migrations.js](https://github.com/serlo/lenabi-migration-algorithm/blob/main/migrations.js)\nfor more details on how these features are implemented.\n\n### Running tests / examples\n\nThe file [`migrations.test.js`](./migrations.test.js) showcases small examples\nfor the above migrations as well as how the migrations can be applied in\nsequence in two cases. You can run the tests with `yarn test`.\n\n### Running Migrations\n\nThis repository also includes a small script that executes the mutations on the\ndocument `document_v1.json`. If you have node.js installed, you can run it with\nfollowing command (or directly with `yarn run:migrations`):\n\n```sh\nnode index.js\n```\n\nIf everything worked out, you can see this output:\n\n```\nDocument migrated from version 1 to version 2\nDocument migrated from version 2 to version 3\nDocument migrated from version 3 to version 4\n```\n\nYou can see the result of the migrations in the files `document_v2.json` to\n`document_v4.json`. For demonstrating, the repository also contains the diffs of\nthe files.\n\n## Comparison to the migration algorithm of H5P\n\nThe algorithm specified in this document is compatible to the\n[migration algorithm of H5P](https://h5p.org/documentation/developers/content-upgrade).\nAlso H5P uses a sequence of migrations plugins. The main difference is, that in\nH5P the version of the content matches the version of the source code itself.\nThey use the major and minor version to specify each migration step in the\nregistry:\n\n```js\n{\n  [majorVersion]: {\n    [minorVersion]: {\n      contentUpgrade: function (parameters, finished) {}\n    }\n  }\n}\n```\n\nOur implementation has some advantages:\n\n- H5P applies a versioning to each of its plugins while we would apply only one\n  global version number for all Serlo editor plugins. Thus we can incorporate\n  migrations where one plugin is replaced by another (for example when it is\n  completely rewritten) or when the content format totally changes (for example\n  when we change the core framework of the editor). For those use cases the H5P\n  migration algorithm cannot be used.\n- We can add multiple migration steps in one update of the editor while for H5P\n  all migrations in a version update need to be concatenated.\n- Our migration algorithm is slightly simpler to implement.\n- Our migration algorithm can be easily mapped to the H5P algorithm and thus be\n  used in case we want to export our Content in the H5P file format.\n\nIn contrast in H5P it is easier to see which Plugin version is needed to render\na certain content. However both algorithm share the same basic ideas.\n\n## Limitations\n\nIn order to ensure that the system operates smoothly, it is important to make\nall changes convertible without manual intervention. This can be achieved by\nselecting appropriate default values for new fields.\n\nTo minimize the risk of breaking existing documents, migrations should not be\nperformed directly on the database. Instead, documents should only be updated\nafter a new edit is made.\n\nA migration can also only transformation which are already stored. Take for\nexample a table plugin where in a newer version an optional header got\nintroduced. Here we cannot provide a general migration algorithm since we do\ncannot decide automatically whether a already defined table has a header or not.\nHowever we can use and combine some solutions in this case:\n\n- We can implement a good guess for the transformation (e.g if the design of the\n  first row differs from the rest then the chance is high that it is a table).\n- We can still ship the old table plugin which we only use for rendering tables\n  in the old format.\n- We can show to authors that a transformation is needed when they edit the\n  document again.\n\n## License\n\n\u003cimg src=\"https://github.com/serlo/lenabi-migration-algorithm/raw/main/assets/cc-by-sa.svg\" alt=\"Logo CC-BY-SA 4.0 license\" title=\"CC-BY-SA 4.0\" align=\"right\" width=\"150\" /\u003e\n\nThis specification is licensed under the\n[CC-BY-SA 4.0 license](https://creativecommons.org/licenses/by-sa/4.0/). Please\nprovide [Serlo Education e.V.](https://serlo.org) together with the link to this\nrepository as the source attribution.\n\n## Funding\n\n\u003cimg src=\"https://github.com/serlo/lenabi-migration-algorithm/blob/main/assets/eu.png?raw=true\" alt=\"Logo European Commission\" title=\"European Comission\" align=\"right\" height=\"150\" /\u003e\n\u003cimg src=\"https://github.com/serlo/lenabi-migration-algorithm/blob/main/assets/bmbf.png?raw=true\" alt=\"Logo BMBF\" title=\"BMBF\" align=\"right\" height=\"150\" /\u003e\n\nThe project covered by this report has been financed by public means granted by\nthe German Ministry for Education and Research under the funding code LENABI2.\nThe responsibility for the content of this publication lies solely with the\nauthor.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fserlo%2Flenabi-migration-algorithm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fserlo%2Flenabi-migration-algorithm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fserlo%2Flenabi-migration-algorithm/lists"}