Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jonchenn/amp-prototyper
https://github.com/jonchenn/amp-prototyper
Last synced: 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/jonchenn/amp-prototyper
- Owner: jonchenn
- License: apache-2.0
- Created: 2019-01-31T02:46:33.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2022-12-02T23:35:02.000Z (about 2 years ago)
- Last Synced: 2024-11-13T00:32:53.448Z (2 months ago)
- Language: JavaScript
- Size: 681 KB
- Stars: 25
- Watchers: 8
- Forks: 10
- Open Issues: 20
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
- awesome-amp - AMP Prototyper - A prototyping tool that automatically converts a HTML page to AMP for demonstrating performance gains with AMP. (Development and Optimization)
README
# amp-prototyper
This Node.js-based script is a prototyping tool that automatically converts a HTML page to a [Accelerated Mobile Page (AMP)](https://amp.dev) to demonstrate performance gains with AMP. It follows [the general guideline of converting HTML to AMP](https://amp.dev/documentation/guides-and-tutorials/start/converting/).
## TL;DR
- The goal is to showcase the advantages of AMP in page load performance.
- It generates a converted AMP HTML, a screenshot, and AMP validation errors for each step.
- It automatically converts a HTML page to AMP with pre-defined steps. You can customize the steps for specific scenarios.
- Final output contains image comparison and match percentage to original URL. Remove the watermark from the HTML and manually review the output## What is amp-prototyper
The main goal is to minimize the effort of creating a prototype AMP page from a canonical HTML page, such as adding AMP boilerplate, removing custom Javascript, making all CSS inline, etc. The output of this tool includes converted AMP, the screenshot, and AMP validation errors for each conversion step.
This script uses [puppeteer](https://github.com/GoogleChrome/puppeteer) to load and render pages.
## Non-scope
This tool doesn't aim to create production-ready AMP pages from any arbitrary HTML pages. If your goal is to create high-quality AMP pages for production, please follow [the general guideline of converting HTML to AMP](https://amp.dev/documentation/guides-and-tutorials/start/converting/).
## Getting started
To run the script, you may choose one of the following methods:
#### npx
Run the script directly with npx:
```
npx amp-prototyper [URL]
```- `URL` - URL to the page to convert.
#### npm or yarn
Install amp-prototyper package.
```
git clone https://github.com/jonchenn/amp-prototyper.git
cd amp-prototyper
yarn install
```Then, run the script.
```
./amp-prototyper [URL]
```### Usage
```
./amp-prototyper [URL]
```Required arguments:
- `URL` - URL to the page to convert.
### Options
- `--steps=FILE` - Path to the custom steps JS file. If not defined, it will use ./steps/default-steps.js
- `--output=FILE` - Path to the output file.
- `--device=DEVICE_NAME` - Use specific device name for screenshots.
- `--headless=(true|false)` - Whether to show browser.
- `--fullPageScreenshot=(true|false*)` - tWhether to save full-page screenshots.
- `--compareScreenshots=(true|false*)` - Whether to compare original site with converted.
- `--customHost=HOST` - Use a custom host name when updating relative asset URLs.
- `--port=PORT_NUMBER` - Port number to use to compare before and after (defaults to 8080)
- `--verbose` - Display AMP validation errors.### Examples:
```
# Amplify a page and generate results in /output folder.
./amp-prototyper https://thinkwithgoogle.com# Amplify a page and generate results in /output/test folder.
./amp-prototyper https://thinkwithgoogle.com --output=test# Amplify a page with customized steps.
./amp-prototyper https://thinkwithgoogle.com --steps=custom/mysteps.js# Amplify a page and display AMP validation details.
./amp-prototyper https://thinkwithgoogle.com --verbose# Amplify a page and generate screenshots with specific Device.
./amp-prototyper https://thinkwithgoogle.com --device='Pixel 2'# Amplify a page and display browser.
./amp-prototyper https://thinkwithgoogle.com --headless=false# Amplify a page and compare original site with converted.
./amp-prototyper https://thinkwithgoogle.com --compareScreenshots=true# Amplify a page that served from localhost and generate results with correct absolute URLs for assets.
./amp-prototyper https://thinkwithgoogle.com --customHost=https://example.com
```### Test with a sample HTML.
You can also run a sample HTML with following:
```
# Run a localhost web server using http-server.
yarn sample
```This opens up a localhost web server at by default that serves [test/index.html](https://github.com/jonchenn/amp-prototyper/blob/master/test/index.html). This is a quick and simple HTML page to test amp-prototyper. You can run the following to see how amp-prototyper works. (If localhost has trouble connecting to the port, add the --port flag with a different port number)
```
# Amplify the page at localhost and output in sample/ folder.
./amp-prototyper http://127.0.0.1:8080 --output=sample
```Then, check out the `./output/sample`, and you will see a list of output files.
## Output of each step
When you run the script, it follows predefined steps, either default steps at [src/default-steps.js](https://github.com/jonchenn/amp-prototyper/blob/master/src/default-steps.js), or customized steps.
You can amplify a HTML page with default steps:
```
# Amplify a page with default steps.
./amp-prototyper http://127.0.0.1:8080
```Or run amplify a page with customized steps:
```
# Amplify a page with customized steps.
./amp-prototyper http://127.0.0.1:8080 --steps=custom/mysteps.js
```At each step, it executes a set of actions and writes the files below to the output/ folder:
- `output-step-[STEP_ID].html` - the modified HTML.
- `output-step-[STEP_ID].png` - the screenshot after this step.
- `output-step-[STEP_ID]-validation.txt` (only with --verbose) - AMP validation errors from console output.If you don't specify --output, it uses the domain from the given URL as the name of the output folder.
### Verify your output
You can run the following command to run a simple web server locally to serve your output files.
```
yarn output
```Open up the localhost URL with a browser. For example:
Check the image comparsion to see visual differences in the original URL and AMP HTML
Remove the watermark in the AMP HTML and review the code for additional changes/AMP validation error fixes
### Caveats
- Styles from '@import' media queries must be manually added
- HTML containing ShadowDOMs and nested ShadowDOMS are unsupported## Customize steps
### Structure of steps
You can check out the default steps at [src/default-steps.js](https://github.com/jonchenn/amp-prototyper/blob/master/src/default-steps.js).
Each step follows the structure below.
```
{
name: 'Name of the step',
actions: [{
skip: false,
log: 'Log output for this action',
actionType: 'replace',
selector: 'html',
regex: '(.*)',
replace: '$2',
}, {
...
}],
},
```Step properties:
- `name` <[string]> Step name.
- `actions` <[Array]<[Action]()>> actions to execute.
- `skip` <[boolean]> Whether to skip this step.Common properties of an action:
- `actionType` Action type.
- `log` <[string]> Message output of this action.
- `waitAfterLoaded` <[number]> Wait for a specific milliseconds after the page loaded.### Environment Variables
You can also use the following EnvVars in the steps configuration.
- `$URL` <[string]> The URL from the `--url` parameter.
- `$HOST` <[string]> The host derived from the URL.
- `$DOMAIN` <[string]> The domain derived from the URL.For example, you have a step like below:
```
{
name: 'Name of the step',
actions: [{
log: 'Log output for this action',
actionType: 'replace',
selector: 'html',
regex: '(.*)',
replace: '$HOST',
}],
},
```While running the script with `https://example.com`, it replaces """$HOST""" with ".
### Supported actions:
#### setAttribute
Set an attribute to a specific element.
- `selector` <[string]> target element.
- `attribute` <[string]> attribute to ad.
- `value` <[string]> the attribute value.#### removeAttribute
Remove an attribute to a specific element.
- `selector` <[string]> target element.
- `attribute` <[string]> attribute to remove.
#### replaceUse Regex to find and replace in the DOM.
- `selector` <[string]> target element.
- `regex` <[string]> Regex string to match.
- `replace` <[string]> Replace matches with this string.
#### removeDisallowedAttributeRemove an attribute to specific elements based on AMP validation errors.
- `selector` <[string]> target element.
#### replaceBasedOnAmpErrorsUse Regex to find and replace in the DOM based on AMP validation errors.
- `selector` <[string]> target element.
- `ampErrorRegex` <[string]> Regex string to match for AMP validation errors.
- `regex` <[string]> Regex string to match.
- `replace` <[string]> Replace matches with this string.
For example, in a specific step it has the following AMP validation errors.```
line 61, col 4: The attribute 'onclick' may not appear in tag 'button'.
line 70, col 4: The tag 'custom-tag' is disallowed.
```To replace the
in the body based on the AMP validation result, you
can have the following step:```
{
name: 'Convert disallowed tags tobased on AMP validation result.',
actions: [{
log: 'Change tags to',
actionType: 'replaceBasedOnAmpErrors',
selector: 'body',
ampErrorRegex: 'The tag \'([^\']*)\' is disallowed',
regex: '<($1)((.|[\\r\\n])*)$1>',
replace: '',
}],
}
```This step matches the AMP validation result with `ampErrorRegex`. Then it replace the `regex` with the capturing group #1 from `ampErrorRegex`. In this case, the `regex` becomes:
```
<(custom-tag)((.|[\\r\\n])*)
```Finally, it uses the revised `regex` to replace the content with `replace` value.
#### replaceOrInsert
Use Regex to find and replace in the DOM. If not found, insert to the destination element.
- `selector` <[string]> target element.
- `regex` <[string]> Regex string to match.
- `replace` <[string]> Replace matches with this string.
#### insertInsert a string to the bottom of the destination element. E.g. adding a string to the bottom of the .
- `selector` <[string]> target element.
- `value` <[string]> the string to insert.
- `destSelector` <[string]> destination element.
#### moveMove elements to the bottom of the destination element. E.g. moving all
to the bottom of the .
- `selector` <[string]> target element.
- `destSelector` <[string]> destination element.
#### appendAfterAppend a string right after a specific element.
- `selector` <[string]> target element.
- `value` <[string]> the string to append.
#### inlineExternalStylesCollect all external CSS and append a `` tag with inline CSS.
- `selector` <[string]> target element to append the CSS.
- `value` <[string]> the string to append.
- `excludeDomains` <[Array]<[string]>> the array of excluded domains. E.g. ['examples.com'] excludes all CSS loaded from <code>examples.com</code>.
- `minify` <[boolean]> whether to minify CSS.
- `attributes` <[Array]<[string]>> add attributes when appending `<style>` tag.#### removeUnusedStyles
Remove unused CSS using [clean-css](https://github.com/jakubpawlowicz/clean-css) and [purifycss](https://github.com/purifycss/purifycss).
- `selector` <[string]> target element.
- `value` <[string]> the string to append.#### customFunc
Run the action with a custom function. Example:
An action object:
```
{
log: 'Click a button',
actionType: 'customFunc',
customFunc: async (action, sourceDom, page) => {
await page.click('button#summit');
},
}],
},
```In the custom function, there are three arguments:
- `action` <[ActionObject]> the action object itself.
- `sourceDom` <[DOM document]> the raw DOM document object before rendering, as in the View Source in Chrome.
- `page` <[puppeteer Page object]> The page object in puppeteer.### Customize steps
To customize your own steps for specific scenarios, create a .js file like below:
```
module.exports = [
{
name: 'Remove unwanted styles',
actions: [{
log: 'Remove inline styles in body',
actionType: 'replace',
selector: 'body',
regex: '(<!--)?.*<style[^<]*(?:(?!<\/style>)<[^<]*)*<\/style>.*(-->)?',
replace: '',
}, {
log: 'Remove noscript in body',
actionType: 'replace',
selector: 'body',
regex: '(<!--)?.*<noscript[^<]*(?:(?!<\/noscript>)<[^<]*)*<\/noscript>.*(-->)?',
replace: '',
}],
}, {
...
}
];
```Next, run the script with `--steps=/path/to/mysteps.js`
```
# Amplify a page with customized steps.
./amp-prototyper http://127.0.0.1:8080 --steps=/path/to/mysteps.js
```## Use cases with specific conditions
### Sites with Crawler Protection
Since AMP-prototyper is built with [Puppeteer](https://github.com/GoogleChrome/puppeteer) to fetch a page content, some sites may treat this as a crawler or bot, and thus block the access. You may see a different content (e.g. 404 page or timeout) than the regular page content.
The blocking logic usually resides in the web hosting servers. Hence, there's no easy way to overcome. However, you can try the following options:
#### Option 1 - Set headless=false
Some sites may treat puppeteer activities as crawler if it runs in the headless mode. You can try running with headless off, and puppeteer will show the browser window like browsing by a real user.
```
# Amplify a page without headless mode.
./amp-prototyper http://127.0.0.1:8080 --headless=false
```#### Option 2 - Serve the page locally
Another option to bypass crawler-protection is to download the page and serve the page from a local web server. (E.g. [http-server](https://www.npmjs.com/package/http-server))
For example, you can save the index page of [ThinkWithGoogle.com](https://www.thinkwithgoogle.com) and serve it locally. It's recommended to copy the **page source** directly instead of using a browser's Save function to avoid unexpected artifacts by the browser.
Since the host is now `localhost` instead of its original host (www.thinkwithgoogle.com), all relative asset paths would incorrectly end up with localhost. Hence, we need to pass `--customHost` to restore its true remote host for asset URLs.
```
# Assuming that the index page is saved to ./tmp/web/index.htmlnpm i http-server -g
http-server ./tmp/web -p 3000
```In a new terminal:
```
./amp-prototyper http://127.0.0.1:3000 --customHost=https://www.thinkwithgoogle.com
```## Reference
[puppeteer](https://github.com/GoogleChrome/puppeteer)
[clean-css](https://github.com/jakubpawlowicz/clean-css)
[purifycss](https://github.com/purifycss/purifycss)