Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/t2ym/gulp-i18n-preprocess
Preprocess Polymer templates and extract UI strings to JSON for build-time I18N with i18n-behavior
https://github.com/t2ym/gulp-i18n-preprocess
html-template i18n polymer preprocess
Last synced: 18 days ago
JSON representation
Preprocess Polymer templates and extract UI strings to JSON for build-time I18N with i18n-behavior
- Host: GitHub
- URL: https://github.com/t2ym/gulp-i18n-preprocess
- Owner: t2ym
- License: other
- Created: 2016-02-02T12:23:03.000Z (almost 9 years ago)
- Default Branch: master
- Last Pushed: 2017-07-03T06:20:08.000Z (over 7 years ago)
- Last Synced: 2024-10-03T16:49:47.844Z (about 1 month ago)
- Topics: html-template, i18n, polymer, preprocess
- Language: JavaScript
- Size: 111 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
[](https://travis-ci.org/t2ym/gulp-i18n-preprocess)
[](https://coveralls.io/github/t2ym/gulp-i18n-preprocess?branch=master)
[](https://www.npmjs.com/package/gulp-i18n-preprocess)
# gulp-i18n-preprocess
Preprocess Polymer templates and extract UI strings to JSON for build-time I18N with [i18n-behavior](https://github.com/t2ym/i18n-behavior)
## Features
- Preprocess Polymer templates to replace hard-coded UI strings with {{annotated}} variables
- Extract hard-coded UI strings to JSON
- Embed the JSON to the target preprocessed templates as default strings
- Export the JSON as files
- Scan custom element HTMLs for constructing repository for I18N target attributes
- Build-time functionalities in [gulp-i18n-preprocess](https://github.com/t2ym/gulp-i18n-preprocess) are in sync with those in [i18n-behavior](https://github.com/t2ym/i18n-behavior) at run-time
## Install
```
npm install --save-dev gulp-i18n-preprocess
```
[Quick Tour](#quick-tour) with [polymer-starter-kit-i18n](https://github.com/t2ym/polymer-starter-kit-i18n)
## Workflow
Build tasks from source to dist:
### 1. Scan task
- Scan source HTMLs for custom elements
- Construct localizable attributes repository
### 2. Preprocess task
- Preprocess source HTMLs
- Extract UI texts to JSON as default
- Replace them with {{annotations}}
- Embed default texts in HTMLs as JSON
- Externalize default texts to JSON files
- Put them in dist
### 3. (Optional) Import XLIFF task with [xliff-conv](https://github.com/t2ym/xliff-conv)
### 4. Leverage task with [gulp-i18n-leverage](https://github.com/t2ym/gulp-i18n-leverage)
- Update localized JSON files by merging differences in default JSON from the previous build
- Put them in dist
- Merge all the UI texts into bundles object
### 5. Bundles task with `fs.writeFileSync()`
- Generate default bundled JSON file `bundle.json` from the bundles object
- Generate per-locale bundled JSON files `bundle.*.json` from the bundles object
- Put them in dist
### 6. (Optional) Export XLIFF task with [xliff-conv](https://github.com/t2ym/xliff-conv)
### 7. Feedback task
- Update default and localized JSON files in source to commit them later by a developer or a build system
## Usage
### Default options
Sample to show default options:
```javascript
var gulp = require('gulp');
var i18nPreprocess = require('gulp-i18n-preprocess');
gulp.task('preprocess', function () {
// default options values
var options = {
replacingText: false, // does not replace strings with {{annotations}}
jsonSpace: 2, // JSON stringification parameter for formatting
srcPath: 'app', // base source path
force: false, // does not force preprocessing when i18n-behavior.html is not imported
dropHtml: false, // does not drop the preprocessed HTML for output
dropJson: false, // does not drop the extracted JSON files for output
constructAttributesRepository: false, // does not construct localizable attributes repository
attributesRepositoryPath: null // does not specify the path to i18n-attr-repo.html
};
return gulp.src([ 'app/elements/**/*.html' ])
.pipe(i18nPreprocess(options))
.pipe(gulp.dest('dist/elements'));
});
```
### Scan task
#### Note: Target HTMLs must import [i18n-behavior.html](https://github.com/t2ym/i18n-behavior) directly.
#### Input:
- Custom element HTMLs in source
#### Output:
- attributesRepository object in gulpfile.js
```javascript
var gulp = require('gulp');
var i18nPreprocess = require('gulp-i18n-preprocess');
// Global object to store localizable attributes repository
var attributesRepository = {};
// Scan HTMLs and construct localizable attributes repository
gulp.task('scan', function () {
return gulp.src([ 'app/elements/**/*.html' ]) // input custom element HTMLs
.pipe(i18nPreprocess({
constructAttributesRepository: true, // construct attributes repository
attributesRepository: attributesRepository, // output object
srcPath: 'app', // path to source root
attributesRepositoryPath:
'bower_components/i18n-behavior/i18n-attr-repo.html', // path to i18n-attr-repo.html
dropHtml: true // drop HTMLs
}))
.pipe(gulp.dest('dist/elements')); // no outputs; dummy output path
});
```
### Preprocess task
#### Note: Target custom element HTMLs must import [i18n-behavior.html](https://github.com/t2ym/i18n-behavior) directly.
#### Input:
- Custom element HTMLs
- Non-custom-element HTMLs in source
#### Output:
- Preprocessed HTMLs and default JSON files in dist
```javascript
var gulp = require('gulp');
var merge = require('merge-stream');
var i18nPreprocess = require('gulp-i18n-preprocess');
// Global object to store localizable attributes repository
var attributesRepository; // constructed attributes repository
// Other standard pipes such as crisper / minification / uglification are omitted for explanation
gulp.task('preprocess', function () {
var elements = gulp.src([ 'app/elements/**/*.html' ]) // input custom element HTMLs
.pipe(i18nPreprocess({
replacingText: true, // replace UI texts with {{annotations}}
jsonSpace: 2, // JSON format with 2 spaces
srcPath: 'app', // path to source root
attributesRepository: attributesRepository // input attributes repository
}))
.pipe(gulp.dest('dist/elements')); // output preprocessed HTMLs and default JSON files to dist
var html = gulp.src([ 'app/**/*.html', '!app/{elements,test}/**/*.html' ]) // non-custom-element HTMLs
.pipe(i18nPreprocess({
replacingText: true, // replace UI texts with {{annotations}}
jsonSpace: 2, // JSON format with 2 spaces
srcPath: 'app', // path to source root
force: true, // force processing even without direct i18n-behavior.html import
attributesRepository: attributesRepository // input attributes repository
}))
.pipe(gulp.dest('dist'));
return merge(elements, html);
});
```
### Leverage task with [gulp-i18n-leverage](https://github.com/t2ym/gulp-i18n-leverage)
#### Input:
- Current localized JSON files in source
- Current default JSON files in source
- Next default JSON files in dist
#### Output:
- Next localized JSON files in dist
- Bundles object in gulpfile.js
```javascript
var gulp = require('gulp');
var i18nLeverage = require('gulp-i18n-leverage');
var bundles = {};
gulp.task('leverage', function () {
return gulp.src([ 'app/**/locales/*.json' ]) // input localized JSON files in source
.pipe(i18nLeverage({
jsonSpace: 2, // JSON format with 2 spaces
srcPath: 'app', // path to source root
distPath: 'dist', // path to dist root to fetch next default JSON files
bundles: bundles // output bundles object
}))
.pipe(gulp.dest('dist')); // path to output next localized JSON files
});
```
### Bundles task
#### Input:
- Bundles object in gulpfile.js
#### Output:
- Bundles JSON files in dist
```javascript
var gulp = require('gulp');
var fs = require('fs');
var JSONstringify = require('json-stringify-safe');
var bundles; // constructed bundles
gulp.task('bundles', function (callback) {
var DEST_DIR = 'dist';
var localesPath = DEST_DIR + '/locales';
try {
fs.mkdirSync(localesPath);
}
catch (e) {}
for (var lang in bundles) {
bundles[lang].bundle = true;
if (lang) {
fs.writeFileSync(localesPath + '/bundle.' + lang + '.json',
JSONstringify(bundles[lang], null, 2));
}
else {
fs.writeFileSync(DEST_DIR + '/bundle.json',
JSONstringify(bundles[lang], null, 2));
}
}
callback();
});
```
### Feedback task
#### Note: Target custom element HTMLs must import [i18n-behavior.html](https://github.com/t2ym/i18n-behavior) directly.
#### Input:
- Next localized JSON files in dist
- Custom element HTMLs
- Non-custom-element HTMLs
#### Output:
- Overwritten localized JSON files in source
- Overwritten default JSON files in source
Outputs are ready to commit in the repository
```javascript
var gulp = require('gulp');
var merge = require('merge-stream');
var i18nPreprocess = require('gulp-i18n-preprocess');
// Only applicable to development builds; Skip it in production builds
gulp.task('feedback', function () {
// Copy from dist
var locales = gulp.src([ 'dist/**/locales/*.json', '!dist/locales/bundle.*.json'])
.pipe(gulp.dest('app'));
// Regenerate default JSON files
var elementDefault = gulp.src([ 'app/elements/**/*.html' ])
.pipe(i18nPreprocess({
replacingText: false,
jsonSpace: 2,
srcPath: 'app',
dropHtml: true,
attributesRepository: attributesRepository
}))
.pipe(gulp.dest('app/elements'));
// Regenerate default JSON files for non-custom-element HTMLs, i.e., i18n-dom-bind
var appDefault = gulp.src([ 'app/**/*.html', '!app/{elements,test}/**/*.html' ])
.pipe(i18nPreprocess({
replacingText: false,
jsonSpace: 2,
srcPath: 'app',
force: true,
dropHtml: true,
attributesRepository: attributesRepository
}))
.pipe(gulp.dest('app'));
return merge(locales, elementDefault, appDefault);
});
```
### Integrate with Polymer CLI project templates by `polymer-build` library (highly experimental)
#### Notes:
- As of [`polymer-build 0.4.0`](https://github.com/Polymer/polymer-build), `polymer-build` library is pre-release and subject to change.
- As of [`Polymer CLI 0.13.0`](https://github.com/Polymer/polymer-cli), the private API `userTransformers` is deprecated and no longer available.
#### Set up `package.json` and the dependent packages of the following `gulpfile.js`
```sh
npm init # if package.json is missing
npm install --save-dev gulp gulp-debug gulp-grep-contents \
gulp-i18n-add-locales gulp-i18n-leverage gulp-i18n-preprocess \
gulp-if gulp-ignore gulp-match gulp-merge gulp-size gulp-sort gulp-util \
json-stringify-safe strip-bom through2 xliff-conv polymer-build plylog merge-stream
```
#### Gulp Filters:
- scan - Scan HTMLs and construct localizable attributes repository
- basenameSort - Sort source files according to their base names; Bundle files come first.
- dropDefaultJSON - Drop default JSON files to avoid overwriting new ones
- preprocess - Preprocess Polymer templates for I18N
- tmpJSON - Store extracted JSON in the temporary folder .tmp
- importXliff - Import XLIFF into JSON
- leverage - Merge changes in default JSON into localized JSON
- exportXliff - Generate bundles and export XLIFF
- feedback - Update JSON and XLIFF in sources
- debug - Show the list of processed files including untouched ones
- size - Show the total size of the processed files
#### Gulp Tasks:
- `gulp locales --targets="{space separated list of target locales}"`
- `gulp default` - Build with `polymer-build` library for `gulp`
#### [gulpfile.js](https://gist.github.com/t2ym/c37990e422d4a19774ba1d749510c1b8#file-gulpfile-js): Put it in the root folder of the project
```javascript
'use strict';
var gulp = require('gulp');
var gutil = require('gulp-util');
var debug = require('gulp-debug');
var gulpif = require('gulp-if');
var gulpignore = require('gulp-ignore');
var gulpmatch = require('gulp-match');
var sort = require('gulp-sort');
var grepContents = require('gulp-grep-contents');
var size = require('gulp-size');
var merge = require('gulp-merge');
var through = require('through2');
var path = require('path');
var stripBom = require('strip-bom');
var JSONstringify = require('json-stringify-safe');
var i18nPreprocess = require('gulp-i18n-preprocess');
var i18nLeverage = require('gulp-i18n-leverage');
var XliffConv = require('xliff-conv');
var i18nAddLocales = require('gulp-i18n-add-locales');
const logging = require('plylog');
const mergeStream = require('merge-stream');
const isPolymerCLI = global._babelPolyfill;
// Global object to store localizable attributes repository
var attributesRepository = {};
// Bundles object
var prevBundles = {};
var bundles = {};
var title = 'I18N transform';
var tmpDir = '.tmp';
var xliffOptions = {};
// Scan HTMLs and construct localizable attributes repository
var scan = gulpif('*.html', i18nPreprocess({
constructAttributesRepository: true, // construct attributes repository
attributesRepository: attributesRepository, // output object
srcPath: '.', // path to source root
attributesRepositoryPath:
'bower_components/i18n-behavior/i18n-attr-repo.html', // path to i18n-attr-repo.html
dropHtml: false // do not drop HTMLs
}));
var basenameSort = sort({
comparator: function(file1, file2) {
var base1 = path.basename(file1.path).replace(/^bundle[.]/, ' bundle.');
var base2 = path.basename(file2.path).replace(/^bundle[.]/, ' bundle.');
return base1.localeCompare(base2);
}
});
var dropDefaultJSON = gulpignore([ 'src/**/*.json', '!**/locales/*.json' ]);
var preprocess = gulpif('*.html', i18nPreprocess({
replacingText: true, // replace UI texts with {{annotations}}
jsonSpace: 2, // JSON format with 2 spaces
srcPath: '.', // path to source root
attributesRepository: attributesRepository // input attributes repository
}));
var tmpJSON = gulpif([ 'src/**/*.json', '!src/**/locales/*' ], gulp.dest(tmpDir));
var unbundleFiles = [];
var importXliff = through.obj(function (file, enc, callback) {
// bundle files must come earlier
unbundleFiles.push(file);
callback();
}, function (callback) {
var match;
var file;
var bundleFileMap = {};
var xliffConv = new XliffConv(xliffOptions);
while (unbundleFiles.length > 0) {
file = unbundleFiles.shift();
if (path.basename(file.path).match(/^bundle[.]json$/)) {
prevBundles[''] = JSON.parse(stripBom(String(file.contents)));
bundleFileMap[''] = file;
}
else if (match = path.basename(file.path).match(/^bundle[.]([^.\/]*)[.]json$/)) {
prevBundles[match[1]] = JSON.parse(stripBom(String(file.contents)));
bundleFileMap[match[1]] = file;
}
else if (match = path.basename(file.path).match(/^bundle[.]([^.\/]*)[.]xlf$/)) {
xliffConv.parseXliff(String(file.contents), { bundle: prevBundles[match[1]] }, function (output) {
if (bundleFileMap[match[1]]) {
bundleFileMap[match[1]].contents = new Buffer(JSONstringify(output, null, 2));
}
});
}
else if (gulpmatch(file, '**/locales/*.json') &&
(match = path.basename(file.path, '.json').match(/^([^.]*)[.]([^.]*)/))) {
if (prevBundles[match[2]] && prevBundles[match[2]][match[1]]) {
file.contents = new Buffer(JSONstringify(prevBundles[match[2]][match[1]], null, 2));
}
}
this.push(file);
}
callback();
});
var leverage = gulpif([ 'src/**/locales/*.json', '!**/locales/bundle.*.json' ], i18nLeverage({
jsonSpace: 2, // JSON format with 2 spaces
srcPath: '', // path to source root
distPath: '/' + tmpDir, // path to dist root to fetch next default JSON files
bundles: bundles // output bundles object
}));
var bundleFiles = [];
var exportXliff = through.obj(function (file, enc, callback) {
bundleFiles.push(file);
callback();
}, function (callback) {
var file;
var cwd = bundleFiles[0].cwd;
var base = bundleFiles[0].base;
var xliffConv = new XliffConv(xliffOptions);
var srcLanguage = 'en';
var promises = [];
var self = this;
var lang;
while (bundleFiles.length > 0) {
file = bundleFiles.shift();
if (!gulpmatch(file, [ '**/bundle.json', '**/locales/bundle.*.json', '**/xliff/bundle.*.xlf' ])) {
this.push(file);
}
}
for (lang in bundles) {
bundles[lang].bundle = true;
this.push(new gutil.File({
cwd: cwd,
base: base,
path: lang ? path.join(cwd, 'locales', 'bundle.' + lang + '.json')
: path.join(cwd, 'bundle.json'),
contents: new Buffer(JSONstringify(bundles[lang], null, 2))
}));
}
for (lang in bundles) {
if (lang) {
(function (destLanguage) {
promises.push(new Promise(function (resolve, reject) {
xliffConv.parseJSON(bundles, {
srcLanguage: srcLanguage,
destLanguage: destLanguage
}, function (output) {
self.push(new gutil.File({
cwd: cwd,
base: base,
path: path.join(cwd, 'xliff', 'bundle.' + destLanguage + '.xlf'),
contents: new Buffer(output)
}));
resolve();
});
}));
})(lang);
}
}
Promise.all(promises).then(function (outputs) {
callback();
});
});
var feedback = gulpif([ '**/bundle.json', '**/locales/*.json', '**/src/**/*.json', '**/xliff/bundle.*.xlf' ], gulp.dest('.'));
var config = {
// list of target locales to add
locales: gutil.env.targets ? gutil.env.targets.split(/ /) : []
}
// Gulp task to add locales to I18N-ready elements and pages
// Usage: gulp locales --targets="{space separated list of target locales}"
gulp.task('locales', function() {
var elements = gulp.src([ 'src/**/*.html' ], { base: '.' })
.pipe(grepContents(/i18n-behavior.html/))
.pipe(grepContents(/ {
// process source files in the project
let sources = project.sources()
.pipe(project.splitHtml())
// I18N processes
.pipe(scan)
.pipe(basenameSort)
.pipe(dropDefaultJSON)
.pipe(preprocess)
.pipe(tmpJSON)
.pipe(importXliff)
.pipe(leverage)
.pipe(exportXliff)
.pipe(feedback)
.pipe(debug({ title: title }))
.pipe(size({ title: title }))
// add compilers or optimizers here!
.pipe(project.rejoinHtml());
// process dependencies
let dependencies = project.dependencies()
.pipe(project.splitHtml())
// add compilers or optimizers here!
.pipe(project.rejoinHtml());
// merge the source and dependencies streams to we can analyze the project
let allFiles = mergeStream(sources, dependencies)
.pipe(project.analyze);
// fork the stream in case downstream transformers mutate the files
// this fork will vulcanize the project
let bundled = fork(allFiles)
.pipe(project.bundle)
// write to the bundled folder
.pipe(gulp.dest('build/bundled'));
let unbundled = fork(allFiles)
// write to the unbundled folder
.pipe(gulp.dest('build/unbundled'));
return mergeStream(bundled, unbundled);
});
}
```
## API
`i18nPreprocess(options)`
### `options` object
- replacingText: Boolean, default: false - If true, UI texts are replaced with {{annotations}}
- jsonSpace: Number, default: 2 - JSON stringification parameter for formatting
- srcPath: String, default: 'app' - Path to source root
- force: Boolean, default: false - Force preprocessing even if `i18n-behavior.html` or `i18n-element.html` is not imported
- dropHtml: Boolean, default: false - If true, drop the preprocessed HTML for output
- dropJson: Boolean, default: false - If true, drop the extracted JSON files for output
- constructAttributesRepository: Boolean, default: false - If true, construct localizable attributes repository
- attributesRepository: Object, default: {} - Input/Output - attributes respository object
- attributesRepositoryPath: String, default: null - Path to bower_components/i18n-behavior/i18n-attr-repo.html
- targetVersion: Number, default: 0 - Values: 1 for Polymer 1.x, 2 for Polymer 2.x, 0 for automatic detection by `i18n-behavior.html` or `i18n-element.html` import
## Quick Tour
### Quick deployment of [`polymer-starter-kit-i18n`](https://github.com/t2ym/polymer-starter-kit-i18n)
```
npm install -g polymer-cli
npm install -g generator-polymer-init-i18n-starter-kit
mkdir i18n-starter-kit
cd i18n-starter-kit
polymer init i18n-starter-kit
# Add Locales
npm run build locales -- --targets="de es fr ja zh-Hans"
# Build
npm run build
# Translate XLIFF ./xliff/bundle.*.xlf
# Build and Merge Translation
npm run build
# App with Run-time I18N on http://localhost:8080
polymer serve
# App with Build-time I18N on http://localhost:8080
polymer serve build/bundled
```
### Change language
##### 1. Press F12 to open debugger console on the browser
##### 2. Navigate to the elements or DOM tab in the debugger
##### 3. Change `lang` attribute of `html` element from "en" to other locales such as "ja"
```html
```
### Update UI strings
##### 1. Change any UI strings in the following HTMLs
```
i18n-starter-kit/src/*.html
```
##### 2. Merge changes into JSON files
```
cd i18n-starter-kit
npm run build
```
##### 3. Check diffs
```
git diff
```
## License
[BSD-2-Clause](https://github.com/t2ym/gulp-i18n-preprocess/blob/master/LICENSE.md)