Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/i18next/i18next-parser

Parse your code to extract translation keys/values and manage your catalog files
https://github.com/i18next/i18next-parser

i18n internationalization parser translation

Last synced: about 6 hours ago
JSON representation

Parse your code to extract translation keys/values and manage your catalog files

Awesome Lists containing this project

README 

        
# i18next Parser [![Build Status](https://travis-ci.org/i18next/i18next-parser.svg?branch=master)](https://travis-ci.org/i18next/i18next-parser) [![codecov](https://codecov.io/gh/i18next/i18next-parser/branch/master/graph/badge.svg?token=CJ74Vps41L)](https://codecov.io/gh/i18next/i18next-parser)



[![NPM](https://nodei.co/npm/i18next-parser.png?downloads=true&stars=true)](https://www.npmjs.com/package/i18next-parser)



When translating an application, maintaining the translation catalog by hand is painful. This package parses your code and automates this process.



Finally, if you want to make this process even less painful, I invite you to check [Locize](https://locize.com/). They are a sponsor of this project. Actually, if you use this package and like it, supporting me on [Patreon](https://www.patreon.com/karelledru) would mean a great deal!





  

    Become a Patreon

  




## Features



- Choose your weapon: A CLI, a standalone parser or a stream transform

- 5 built in lexers: Javascript, JSX, HTML, Handlebars, and TypeScript+tsx

- Handles VueJS 3.0 (.vue) files

- Creates one catalog file per locale and per namespace

- Backs up the old keys your code doesn't use anymore in `namespace_old.json` catalog

- Restores keys from the `_old` file if the one in the translation file is empty

- Parses comments for static keys to support dynamic key translations.

- Supports i18next features:

  - **Context**: keys of the form `key_context`

  - **Plural**: keys of the form `key_zero`, `key_one`, `key_two`, `key_few`, `key_many` and `key_other` as described [here](https://www.i18next.com/translation-function/plurals)



## Versions



You can find information about major releases on the [dedicated page](https://github.com/i18next/i18next-parser/releases). The [migration documentation](docs/migration.md) will help you figure out the breaking changes between versions.



- `9.x` is tested on Node 18, 20 and 22.

- `8.x` is tested on Node 16, 18 and 20.

- `7.x` is tested on Node 14, 16 and 18.

- `6.x` is tested on Node 14 and 16.



## Usage



### CLI



You can use the CLI with the package installed locally but if you want to use it from anywhere, you better install it globally:



```

yarn global add i18next-parser

npm install -g i18next-parser

i18next 'app/**/*.{js,hbs}' 'lib/**/*.{js,hbs}' [-oc]

```



Multiple globbing patterns are supported to specify complex file selections. You can learn how to write globs [here](https://github.com/isaacs/node-glob). Note that glob must be wrapped with single quotes when passed as arguments.



**IMPORTANT NOTE**: If you pass the globs as CLI argument, they must be relative to where you run the command (aka relative to `process.cwd()`). If you pass the globs via the `input` option of the config file, they must be relative to the config file.



- **-c, --config **: Path to the config file (default: i18next-parser.config.{js,mjs,json,ts,yaml,yml}).

- **-o, --output **: Path to the output directory (default: locales/$LOCALE/$NAMESPACE.json).

- **-s, --silent**: Disable logging to stdout.

- **--fail-on-warnings**: Exit with an exit code of 1 on warnings

- **--fail-on-update**: Exit with an exit code of 1 when translations are updated (for CI purpose)



### Gulp



Save the package to your devDependencies:



```

yarn add -D i18next-parser

npm install --save-dev i18next-parser

```



[Gulp](http://gulpjs.com/) defines itself as the streaming build system. Put simply, it is like Grunt, but performant and elegant.



```javascript

import { gulp as i18nextParser } from 'i18next-parser'



gulp.task('i18next', function () {

  gulp

    .src('app/**')

    .pipe(

      new i18nextParser({

        locales: ['en', 'de'],

        output: 'locales/$LOCALE/$NAMESPACE.json',

      })

    )

    .pipe(gulp.dest('./'))

})

```



**IMPORTANT**: `output` is required to know where to read the catalog from. You might think that `gulp.dest()` is enough though it does not inform the transform where to read the existing catalog from.



### Broccoli



Save the package to your devDependencies:



```

yarn add -D i18next-parser

npm install --save-dev i18next-parser

```



[Broccoli.js](https://github.com/broccolijs/broccoli) defines itself as a fast, reliable asset pipeline, supporting constant-time rebuilds and compact build definitions.



```javascript

import Funnel from 'broccoli-funnel'

import { broccoli as i18nextParser } from 'i18next-parser'



const appRoot = 'broccoli'



let i18n = new Funnel(appRoot, {

  files: ['handlebars.hbs', 'javascript.js'],

  annotation: 'i18next-parser',

})



i18n = new i18nextParser([i18n], {

  output: 'broccoli/locales/$LOCALE/$NAMESPACE.json',

})



export default i18n

```



> **Note**: You may need to configure Broccoli to place temporary files (option: tmpdir) within the current working

> directory as I18next-parser does not traverse down beyond that.



## Options



Using a config file gives you fine-grained control over how i18next-parser treats your files. Here's an example config showing all config options with their defaults.



```js

// i18next-parser.config.js



export default {

  contextSeparator: '_',

  // Key separator used in your translation keys



  createOldCatalogs: true,

  // Save the \_old files



  defaultNamespace: 'translation',

  // Default namespace used in your i18next config



  defaultValue: '',

  // Default value to give to keys with no value

  // You may also specify a function accepting the locale, namespace, key, and value as arguments



  indentation: 2,

  // Indentation of the catalog files



  keepRemoved: false,

  // Keep keys from the catalog that are no longer in code

  // You may either specify a boolean to keep or discard all removed keys.

  // You may also specify an array of patterns: the keys from the catalog that are no long in the code but match one of the patterns will be kept.

  // The patterns are applied to the full key including the namespace, the parent keys and the separators.



  keySeparator: '.',

  // Key separator used in your translation keys

  // If you want to use plain english keys, separators such as `.` and `:` will conflict. You might want to set `keySeparator: false` and `namespaceSeparator: false`. That way, `t('Status: Loading...')` will not think that there are a namespace and three separator dots for instance.



  // see below for more details

  lexers: {

    hbs: ['HandlebarsLexer'],

    handlebars: ['HandlebarsLexer'],



    htm: ['HTMLLexer'],

    html: ['HTMLLexer'],



    mjs: ['JavascriptLexer'],

    js: ['JavascriptLexer'], // if you're writing jsx inside .js files, change this to JsxLexer

    ts: ['JavascriptLexer'],

    jsx: ['JsxLexer'],

    tsx: ['JsxLexer'],



    default: ['JavascriptLexer'],

  },



  lineEnding: 'auto',

  // Control the line ending. See options at https://github.com/ryanve/eol



  locales: ['en', 'fr'],

  // An array of the locales in your applications



  namespaceSeparator: ':',

  // Namespace separator used in your translation keys

  // If you want to use plain english keys, separators such as `.` and `:` will conflict. You might want to set `keySeparator: false` and `namespaceSeparator: false`. That way, `t('Status: Loading...')` will not think that there are a namespace and three separator dots for instance.



  output: 'locales/$LOCALE/$NAMESPACE.json',

  // Supports $LOCALE and $NAMESPACE injection

  // Supports JSON (.json) and YAML (.yml) file formats

  // Where to write the locale files relative to process.cwd()



  pluralSeparator: '_',

  // Plural separator used in your translation keys

  // If you want to use plain english keys, separators such as `_` might conflict. You might want to set `pluralSeparator` to a different string that does not occur in your keys.

  // If you don't want to generate keys for plurals (for example, in case you are using ICU format), set `pluralSeparator: false`.



  input: undefined,

  // An array of globs that describe where to look for source files

  // relative to the location of the configuration file



  sort: false,

  // Whether or not to sort the catalog. Can also be a [compareFunction](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#parameters)



  verbose: false,

  // Display info about the parsing including some stats



  failOnWarnings: false,

  // Exit with an exit code of 1 on warnings



  failOnUpdate: false,

  // Exit with an exit code of 1 when translations are updated (for CI purpose)



  customValueTemplate: null,

  // If you wish to customize the value output the value as an object, you can set your own format.

  //

  // - ${defaultValue} is the default value you set in your translation function.

  // - ${filePaths} will be expanded to an array that contains the absolute

  //   file paths where the translations originated in, in case e.g., you need

  //   to provide translators with context

  //

  // Any other custom property will be automatically extracted from the 2nd

  // argument of your `t()` function or tOptions in 

  //

  // Example:

  // For `t('my-key', {maxLength: 150, defaultValue: 'Hello'})` in

  // /path/to/your/file.js,

  //

  // Using the following customValueTemplate:

  //

  // customValueTemplate: {

  //   message: "${defaultValue}",

  //   description: "${maxLength}",

  //   paths: "${filePaths}",

  // }

  //

  // Will result in the following item being extracted:

  //

  // "my-key": {

  //   "message": "Hello",

  //   "description": 150,

  //   "paths": ["/path/to/your/file.js"]

  // }



  resetDefaultValueLocale: null,

  // The locale to compare with default values to determine whether a default value has been changed.

  // If this is set and a default value differs from a translation in the specified locale, all entries

  // for that key across locales are reset to the default value, and existing translations are moved to

  // the `_old` file.



  i18nextOptions: null,

  // If you wish to customize options in internally used i18next instance, you can define an object with any

  // configuration property supported by i18next (https://www.i18next.com/overview/configuration-options).

  // { compatibilityJSON: 'v3' } can be used to generate v3 compatible plurals.



  yamlOptions: null,

  // If you wish to customize options for yaml output, you can define an object here.

  // Configuration options are here (https://github.com/nodeca/js-yaml#dump-object---options-).

  // Example:

  // {

  //   lineWidth: -1,

  // }

}

```



### Lexers



The `lexers` option let you configure which Lexer to use for which extension. Here is the default:



Note the presence of a `default` which will catch any extension that is not listed.

There are 4 lexers available: `HandlebarsLexer`, `HTMLLexer`, `JavascriptLexer` and

`JsxLexer`. Each has configurations of its own. Typescript is supported via `JavascriptLexer` and `JsxLexer`.

If you need to change the defaults, you can do it like so:



#### Javascript



The Javascript lexer uses [Typescript compiler](https://github.com/microsoft/TypeScript/wiki/Using-the-Compiler-API) to walk through your code and extract translation functions.



The default configuration is below:



```js

{

  // JavascriptLexer default config (js, mjs)

  js: [{

    lexer: 'JavascriptLexer',

    functions: ['t'], // Array of functions to match

    namespaceFunctions: ['useTranslation', 'withTranslation'], // Array of functions to match for namespace

  }],

}

```



#### Jsx



The JSX lexer builds off of the Javascript lexer and extends it with support for JSX syntax.



Default configuration:



```js

{

  // JsxLexer default config (jsx)

  // JsxLexer can take all the options of the JavascriptLexer plus the following

  jsx: [{

    lexer: 'JsxLexer',

    attr: 'i18nKey', // Attribute for the keys

    componentFunctions: ['Trans'], // Array of components to match

  }],

}

```



If your JSX files have `.js` extension (e.g. create-react-app projects) you should override the default `js` lexer with `JsxLexer` to enable jsx parsing from js files:



```js

{

  js: [{

    lexer: 'JsxLexer'

  }],

}

```



##### Supporting helper functions in Trans tags



If you're working with i18next in Typescript, you might be using a helper function to make sure that objects in  components pass the typechecker: e.g.,



```tsx

const SomeComponent = (props) => (

  

    Visit

    {castAsString({ name: props.name })}'s profile

    {/* Equivalent to, but resolves typechecker errors with */}

    {{ name: props.name }}'s profile

  

)



function castAsString(record: Record): string {

  return record as unknown as string

}

```



In order for the parser to extract variables properly, you can add a list of

such functions to the lexer options:



```

{

  js: [{

    lexer: 'JsxLexer',

    transIdentityFunctionsToIgnore: ['castAsString']

  }],

}

```



#### Ts(x)



Typescript is supported via Javascript and Jsx lexers. If you are using Javascript syntax (e.g. with React), follow the steps in Jsx section, otherwise Javascript section.



#### Handlebars



```js

{

  // HandlebarsLexer default config (hbs, handlebars)

  handlebars: [

    {

      lexer: 'HandlebarsLexer',

      functions: ['t'], // Array of functions to match

    },

  ]

}

```



#### Html



```js

{

  // HTMLLexer default config (htm, html)

  html: [{

    lexer: 'HTMLLexer',

    attr: 'data-i18n' // Attribute for the keys

    optionAttr: 'data-i18n-options' // Attribute for the options

  }]

}

```



#### Custom lexers



You can provide function instead of string as a custom lexer.



```js

import CustomJsLexer from './CustomJsLexer';



// ...

{

  js: [CustomJsLexer],

  jsx: [{

    lexer: CustomJsLexer,

    customOption: true // Custom attribute passed to CustomJsLexer class constructor

  }]

}

// ...

```



### Caveats



While i18next extracts translation keys in runtime, i18next-parser doesn't run the code, so it can't interpolate values in these expressions:



```

t(key)

t('key' + id)

t(`key${id}`)

```



As a workaround you should specify possible static values in comments anywhere in your file:



```

// t('key_1')

// t('key_2')

t(key)



/*

t('key1')

t('key2')

*/

t('key' + id)

```



## Events



The transform emits a `reading` event for each file it parses:



`.pipe( i18next().on('reading', (file) => {}) )`



The transform emits a `error:json` event if the JSON.parse on json files fail:



`.pipe( i18next().on('error:json', (path, error) => {}) )`



The transform emits a `warning` event if the file has a key that is not a string litteral or an option object with a spread operator:



`.pipe( i18next().on('warning', (path, key) => {}) )`



Here is a list of the warnings:



- **Key is not a string literal**: the parser cannot parse variables, only literals. If your code contains something like `t(variable)`, the parser will throw a warning.

- **Found same keys with different values**: if you use different default values for the same key, you'll get this error. For example, having `t('key', {defaultValue: 'foo'})` and `t('key', {defaultValue: 'bar'})`. The parser will select the latest one.

- **Found translation key already mapped to a map or parent of new key already mapped to a string**: happens in this kind of situation: `t('parent', {defaultValue: 'foo'})` and `t('parent.child', {defaultValue: 'bar'})`. `parent` is both a translation and an object for `child`.



## Contribute



Any contribution is welcome. Please [read the guidelines](docs/development.md) first.



Thanks a lot to all the previous [contributors](https://github.com/i18next/i18next-parser/graphs/contributors).



If you use this package and like it, supporting me on [Patreon](https://www.patreon.com/karelledru) is another great way to contribute!





  

    Become a Patreon

  




---



## Gold Sponsors