{"id":19777466,"url":"https://github.com/ilib-js/message-accumulator","last_synced_at":"2025-09-17T23:31:37.142Z","repository":{"id":57295188,"uuid":"162833107","full_name":"iLib-js/message-accumulator","owner":"iLib-js","description":"A package to help transform localizable messages in a variety of syntaxes into a form that translators can easily translate without knowing anything about that syntax.","archived":false,"fork":false,"pushed_at":"2024-11-08T16:46:28.000Z","size":86,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-11-12T05:30:38.459Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iLib-js.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-12-22T18:12:18.000Z","updated_at":"2024-02-01T16:54:37.000Z","dependencies_parsed_at":"2024-06-21T04:18:07.663Z","dependency_job_id":"a2481d55-853a-476b-a74d-016e6f69c41c","html_url":"https://github.com/iLib-js/message-accumulator","commit_stats":{"total_commits":51,"total_committers":3,"mean_commits":17.0,"dds":0.2941176470588235,"last_synced_commit":"c9b3fc4cfe4d5e6590c623a90769f37e440bcbf3"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iLib-js%2Fmessage-accumulator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iLib-js%2Fmessage-accumulator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iLib-js%2Fmessage-accumulator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iLib-js%2Fmessage-accumulator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iLib-js","download_url":"https://codeload.github.com/iLib-js/message-accumulator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233430256,"owners_count":18675058,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T05:25:05.735Z","updated_at":"2025-09-17T23:31:31.748Z","avatar_url":"https://github.com/iLib-js.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\u003e :warning: **Deprecation Notice** :warning:\n\u003e This repository has been deprecated. Please use the corresponding package from the [iLib-js monorepo](https://github.com/iLib-js/ilib-mono) instead.\n\n# message-accumulator\n\nA package to help transform localizable messages in a variety of\nsyntaxes into a form that\ntranslators can easily translate without knowing anything about\nthat syntax, and after translation, back again into a form that\nprograms can easily use to recompose localized messages in the\noriginal syntax.\n\nIn HTML or JSX, for example, whole translatable messages are hard to identify.\nIn HTML, some tags are commonly found inside of whole\nsentences, and some are not. What forms a whole translatable\nmessage?\n\nConsider this snippet:\n\n```\n\u003cdiv\u003e\n\t\u003cspan class=\"body\"\u003e\n\t    There are \u003ca href=\"http://url\" title=\"localizable title\"\u003e50 files\u003c/a\u003e in the \u003cspan class=\"copyright\"\u003eSimple Markdown\u003c/span\u003e system.\n\t\u003c/span\u003e\n\u003c/div\u003e\n```\n\nIn this case, the outer \"div\" and \"span\" tags are not part of any localizable\nsnippet of text. The entire string \"There are\n\u0026lt;a href=\"http://url\" title=\"localizable title\"\u0026gt;50\nfiles\u0026lt;/a\u0026gt; in the \u0026lt;span class=\"copyright\"\u0026gt;Simple Markdown\u0026lt;/span\u0026gt; system.\"\nshould be localized as a single sentence because it would not\nmake any sense to localize the parts \"There are \",\n\"50 files\", \" in the \", \"Simple Markdown\", and \" system.\" separately.\nThey are not simply phrases that you can\ntranslate out-of-context, and then re-concatenate and have\nany hope that it will make logical sense in many other languages. Human\nlanguage is more complicated than that! In order for translators to do\na good job, they need the entire sentence.\n\nThe only problem is that translators\nare not so good with programming language syntax and tend to do things\nlike occasionally translating HTML tag names or attribute values. For example,\nthey may see a series of CSS class names that forms a short phrase in English,\nand decide that they should be translated. The situation is even worse in\nJSX because the names of tags are not a fixed list like HTML.\nComponents can have any name\nand even translators who are familiar with HTML tags are confused as to\nwhat is translatable and what is not. In our example above, we even have\nanother added complication\nthat the value of the \"title\" attribute of the \"a\" tag is actual\nlocalizable text but the value of the other attributes are not, which\nis even more confusing to the translators. Again, they are not programmers.\n\nBut that is okay. They are amazing linguists, and this library helps to hide\nthese complications from them. This library hides the contents of such tags\nfrom the translators and lets them translate with minimal syntax\ngetting in the way. The sentence above would be easier for translators\nto translate if it were something like this:\n\n```\nThere are \u003cc0\u003e50 files\u003c/c0\u003e in the \u003cc1\u003eSimple Markdown\u003c/c1\u003e system.\n\nwhere:\n\nc0 = \u003ca href=\"http://url\" title=\"localizable title\"\u003e\nc1 = \u003cspan class=\"copyright\"\u003e\n```\n\nTranslators can learn this simple XML syntax quickly, and don't need to know the\nintracacies of any programming or markup language. They\ncan focus on the linguistic part of the translation\nand only have to make sure that the corresponding portion of translation\nis surrounded by the XML-tags \"c0\" and \"c1\". (The \"c\" stands for\nthe word \"component\" -- XML tags have to start with a letter.)\n\nTranslating this type of message has many advantages:\n\n* The contents of the tags is hidden from the\n  translators, so they cannot mess it up by translating things that should\n  not be translated and by leaving out brackets or quotation characters.\n* It prevents code injection attacks. A nefarious person working at the translation\n  agency would not be able to insert some malicious javascript code in the\n  middle of a translation hidden inside of some HTML tags because source string\n  does not contain HTML.\n* The contents of the tags can change frequently without affecting the\n  content of the source string, and therefore the translation. A designer\n  can add a new CSS class if they desire, and the programmer can change\n  the href attribute of a link tag without causing a\n  retranslation of the string. The new CSS class and url will be\n  recomposed into the translated string later.\n\nNow in many human languages, grammar is different than in English, so it is\nentirely possible that the order of the components turns out different for\na translated string than in English. Also,\nthe nesting of those components may change. We need to allow the translators\nthe freedom to do what is right for the grammar of their target language.\nThat means we need to be able to decompose a translated string back\ninto a tree of syntax nodes that can easily be transformed back into\nthe source programming language again, whether that is HTML, JSX, or\neven Markdown. This is accomplished by reapplying the mapping between\nthe components and the original tag text to the appropriate parts\nof the translation.\n\nConsider this translation of our example above into German:\n\n```\nIn den \u003cc1\u003eSimple Markdown\u003c/c1\u003e System, gibt es \u003cc0\u003e50 Dateien\u003c/c1\u003e.\n```\n\nNote that the order of the components is indeed reversed from English -- c1\ncomes before c0. Ideally, we would like to decompose this into this tree:\n\n```\nroot\n  \"In den \"\n  c1\n    \"Simple Markdown\"\n  \"System, gibt es\"\n  c0\n    \"50 Dateien\"\n  \".\"\n```\n\nFrom there you can easily reapply the mapping `c1 = \u003cspan class=\"copyright\"\u003e`\nand `c0 = \u003ca href=\"http://url\" title=\"localizable title\"\u003e`, plus the appropriate\nclose tags of course, to reconstruct the HTML into nicely translated HTML:\n\n```\nIn den \u003cspan class=\"copyright\"\u003eSimple Markdown\u003c/span\u003e System, gibt es \u003ca href=\"http://url\" title=\"localizable title\"\u003e50 Dateien\u003c/a \u003e.\n```\n\nIn many cases, the caller of\nthis message accumulator class will have an abstract syntax tree (AST) in memory\nwhich is the result of parsing the original English source file with a\nstandard parser. In this case,\n\"c0\" and \"c1\" would map to particular nodes in that tree instead of to snippets\nof text containing the HTML tags.\nThe caller's AST can be modified for the translation by reusing existing AST nodes\nin the place of the components.\n\nThe goal of the message-accumulator class is to help the caller accumulate\na localizable unit (a \"message\") while traversing the AST of the source file,\nas well as to be able to decompose a translated string back into a tree\nthat can be easily transformed into AST nodes again.\n\nUsage\n-----\n\n### Extracting source strings from your source file\n\nThe MessageAccumulator class has four main methods to accumulate a string:\n\n- addText() - Add new text to the current context of the string\n- push() - Start a new context, such as text within an HTML tag\n- pop() - End the current context and return to the previous one\n- getString() - Retrieve the translatable string in this accumulator, where\n  the contexts are hidden with the \"c\" XML tags\n\nStep 1. Use your parser to generate an abstract syntax tree (AST) that represents\nthe file.\n\nStep 2. Walk the AST, accumulating text as appropriate using addText and pushing\ncontexts for any nodes that do not mark a break in the text. For example, if\nyour HTML parser has some text followed by the \"b\" tag, then that \"b\" tag should\nnot cause a break in the text. The code should push a new context and continue\nto accumulate more text.\n\nStep 3. At some point, the parser will eventually come to a node\nin the AST that marks a break in the translatable message. (Or it will come\nto the end of the file!) For example in HTML, you might encounter a \u0026lt;div\u0026gt;\ntag. When this happens, the current value of the message accumulator is the\ntranslatable string. The code can retrieve the string using the getString\nmethod, and this string can be sent into the localization process. Typical\nthe code will then create a new MessageAccumulator instance for the next\npiece of text.\n\n### Localizing your source file\n\nAt some point, the translations of all the strings will be done, and the\nlocalized file can be reconstructed.\n\nTo do this, the code starts with the source file and the set of translations\nfrom your localization system, in the form of resource files or a translation\nserver.\n\nStep 1. The source file is reparsed and re-walked as above, but this time, you keep\ntrack of nodes in the AST by pushing them into your contexts. This decorates\nthe nodes in the accumulator with the AST nodes. Doing this creates\na mapping between contexts and the AST nodes that they represent. The push()\nmethod takes a parameter that is your AST node.\n\nStep 2. As the code walks the nodes and hit some node that causes the end of\nthe translatable text, it can then apply the translation. The result of\ngetString() gives the translatable source, and the translation of that is\nlooked up in the translation system. Then, the code will create a new MessageAccumulator\nfrom that translated string plus the current MessageAccumulator containing\nthe source string. This will apply the mapping from context to AST node\nappropriately to the translated MessageAccumulator.\n\nStep 3. Walk the new translated MessageAccumulator again, converting the\nMessageAccumulator nodes into AST nodes. Then, replace the AST nodes with\nthese new ones.\n\nStep 4. When all of the text is translated, convert the AST back into text\nagain to reconstruct your translated file.\n\n## License\n\nCopyright © 2018-2024, JEDLSoft\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\n# Release Notes\n\n## v3.0.0\n\n- convert all unit tests from nodeunit to jest\n- export the es6 code as real es6 modules and old transpiled javascript\n  for older packages to use (breaking change)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Filib-js%2Fmessage-accumulator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Filib-js%2Fmessage-accumulator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Filib-js%2Fmessage-accumulator/lists"}