Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/galaxiat/galaxiat.serve.seo
Node.JS package to serve React app and prerender path (cron)
https://github.com/galaxiat/galaxiat.serve.seo
crawler cron puppeteer seo seo-optimization ssr
Last synced: about 16 hours ago
JSON representation
Node.JS package to serve React app and prerender path (cron)
- Host: GitHub
- URL: https://github.com/galaxiat/galaxiat.serve.seo
- Owner: galaxiat
- License: mit
- Created: 2022-04-26T15:28:41.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2022-05-15T11:35:02.000Z (over 2 years ago)
- Last Synced: 2024-04-25T04:01:55.647Z (7 months ago)
- Topics: crawler, cron, puppeteer, seo, seo-optimization, ssr
- Language: TypeScript
- Homepage: https://galaxiatapp.com
- Size: 175 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## About
galaxiat.serve.seo [Node.js](https://nodejs.org) package allows you to easily cron crawl path that you want to have an html version (for dynamic rendering like React) without have to make SSR when request is passed.
We use it at Galaxiat to do our https://galaxiatapp.com/pub/hash/dev rendering.
## Package support
[galaxiat.serve.seo](https://github.com/galaxiat.serve.seo) Support both static and dynamic route.
Dynamic route can be delivered by a remote json endpoint and static route can be delivered by the `.galaxiat.json` file.
## Installation
**Node.js 16.9.0 or newer is required.**
```sh-session
npm install galaxiat.serve.seo
```
## Example
### Static and remote crawl
---
`.galaxiat.json` OR `.galaxiat.{env}.json`
To set env use the `GALAXIAT_SERVE_ENV` var
`type` : `remote` | `local`
- Remote will use the url provided on the `remote` key to connect a remote chrome instance.
- NB : include the `/playwright` at the end of the url
- For remote usage we recommend the use of Token see the docs of [browserless.io](https://docs.browserless.io/) for more infos
- Local will spawn a chrome headless browser with args on the `args` key.
```json
{
"hostname" : "galaxiatapp.com",
"port" : 3000,
"type" : "remote",
"args" : ["--no-sandbox",
"--disable-setuid-sandbox"],
"remote" : "wss://chrome.shared.svc.galaxiat.fr/playwright?token=MWkH6L4K3knkG3hvsaHrnzA5g6dtfucYk5nD9YVBRRh9ZtdPyDaE",
"target" : "http://localhost:3000",
"public" : "./public",
"crawl" : [
{
"type" : "config",
"url" : "/path",
"file" : "/cache/path.html",
"cron" : "0 * * * * *"
},
{
"type" : "remote",
"json_url" : "https://api.galaxiatapp.com/seo/galaxiat.json",
"cron" : "0 */15 * * * *"
}
],
"crawl_cron" : "* * * * * *",
"crawl_max_num" : 3,
"crawl_queue_num" : 10,
"errors" : {
"https" : false
}
}
```
---
`https://galaxiatapp.com/seo/galaxiat.json`
```json
[
{
"url": "https://galaxiatapp.com/pub/hash/dev",
"file": "/pub/hash/dev.html"
},{
"url": "https://galaxiatapp.com/pub/hash/something",
"file": "/pub/hash/something.html"
}
]
```
## RoadMap
- `V1.X.X` - Single workload implementation
- Per node deployment -> not so good for performance
- Crawling is done on the local node
- `V2.X.X` - Multiple workload implementation
- Multi-node deployment -> better performance
- Crawling is done on remote node
- `V3.X.X` - Advanced Multiple workload implementation
- Multi-node deployment + Cluster cache -> better performance
- Cache is cluster wide instead of a local cache per node
## We have added remote chrome support
see [browserless.io](https://docs.browserless.io/) for more infos
**Local mode is not recommended for production**
## Links
- [Galaxiat](https://galaxiatapp.com/)
## Contributing
Before creating an issue, please ensure that it hasn't already been reported/suggested.
## License
Software is under [MIT](./LICENSE.md) license