Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/t9tio/cloudquery
Turn any website to API by several clicks (serverless and support SPA!)
https://github.com/t9tio/cloudquery
Last synced: 4 days ago
JSON representation
Turn any website to API by several clicks (serverless and support SPA!)
- Host: GitHub
- URL: https://github.com/t9tio/cloudquery
- Owner: t9tio
- License: gpl-3.0
- Created: 2019-01-19T06:29:29.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2023-01-07T04:08:19.000Z (about 2 years ago)
- Last Synced: 2024-10-29T17:49:55.615Z (4 months ago)
- Language: JavaScript
- Homepage: https://cloudquery.t9t.io/
- Size: 1.4 MB
- Stars: 2,164
- Watchers: 33
- Forks: 136
- Open Issues: 31
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
- awesome-list - cloudquery
README
# CloudQuery
[](https://t9t.io/#contact)
Turn any website to API by several clicks.
Try online: https://cloudquery.t9t.io

## API
- Sample Query: [Best 3 posts on HN](https://cloudquery.t9t.io/query?url=https%3A%2F%2Fnews.ycombinator.com%2F&selectors=*:nth-child(3)%20*:nth-child(1)%20%3E%20*:nth-child(3)%20%3E%20*:nth-child(1),*:nth-child(4)%20%3E%20*:nth-child(3)%20%3E%20*:nth-child(1),*:nth-child(7)%20%3E%20*:nth-child(3)%20%3E%20*:nth-child(1))
Query params:
- url: url of the webpage you want to fetch
- selectors: css selectors describing the elements interests youReturns:
- innerText: element innerText
- href: element href## Run locally
```bash
# install dependencies
npm i
# start backend server(for auto restart when code change)
npm run sb
# start frontend server(for auto restart when code change)
npm run sf# visit http://localhost:1234
```## How to deploy your own CloudQuery to AWS lambda
### 1. Configuration
- update `profile`([AWS Credential Profiles](https://up.docs.apex.sh/#aws_credentials)) in `up.json` to use your own aws account
- update `rateLimit` in `config.json` to set your own rate limit (Default: 5 request/hour)### 2. Deploy
```bash
# deploy to aws lambda
up# see the URL of your CloudQuery
up url# deploy production version(to cloudquery.t9t.io)
up deploy production```
# Thanks
- [up](https://github.com/apex/up) for deploying serverless API to AWS with ease
- [serverless-chrome](https://github.com/adieuadieu/serverless-chrome) for running chrome on AWS lambda
- [finder](https://github.com/antonmedv/finder) for making it easy to select elements on webpage