Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jordaneremieff/headless-api
Python API service for headless Chromium
https://github.com/jordaneremieff/headless-api
aws-lambda fastapi headless-chrome mangum python selenium serverless starlette
Last synced: 14 days ago
JSON representation
Python API service for headless Chromium
- Host: GitHub
- URL: https://github.com/jordaneremieff/headless-api
- Owner: jordaneremieff
- License: mit
- Created: 2020-07-19T12:06:34.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2020-07-19T15:46:20.000Z (over 4 years ago)
- Last Synced: 2024-12-06T14:21:21.286Z (27 days ago)
- Topics: aws-lambda, fastapi, headless-chrome, mangum, python, selenium, serverless, starlette
- Language: Python
- Homepage:
- Size: 253 KB
- Stars: 19
- Watchers: 2
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Headless API
Python API service for headless Chromium.
This project consists of a [FastAPI](https://fastapi.tiangolo.com/) application for running [headless Chromium](https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md) browser commands using [Selenium](https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver), and includes an example [Serverless](https://www.serverless.com/) configuration for AWS Lambda & API Gateway deployments that requires [Serverless-Chrome](https://github.com/adieuadieu/serverless-chrome) and [ChromeDriver](https://chromedriver.chromium.org/) binaries for headless Chromium support and [Mangum](https://mangum.io/) to wrap the ASGI application.
It is also possible to use this with a normal Chrome/Chromium binary and ASGI server (such as [Uvicorn](https://www.uvicorn.org/) or [Hypercorn](https://pgjones.gitlab.io/hypercorn/), but my intention is to demonstrate a serverless example.
*Note*: This is a proof-of-concept/example that I created to experiment with FastAPI and serverless. I do not intend to actively maintain it.
## Usage
Live: https://headless-api.eremieff.com/
You can play around with the generated API docs in the testing deployment docs to see how it works.
## Serverless deployment
You'll need to modify the `serverless.yml` file and change occurences of `headless-api` in your project. I've only confirmed this working with 3.7 and have not tested other configurations.
The following binaries must be included in the `src/bin/` directory:
```shell
curl -SL https://chromedriver.storage.googleapis.com/2.37/chromedriver_linux64.zip > chromedriver.zip
unzip chromedriver.zip -d src/bin/
rm chromedriver.zip
```
```shell
mkdir -p bin/
curl -SL https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-37/stable-headless-chromium-amazonlinux-2017-03.zip > headless-chromium.zip
unzip headless-chromium.zip -d src/bin/
rm headless-chromium.zip
```
## Running locally
To run the project locally, you'll need binaries specifc to your operation system and a normal ASGI server for the application. The local application can be run using Uvicorn:
```shell
python -m src.asgi
```
## Known Issues
- It is currently not possible to screenshot all of the content on a page if the page height is greater than the window height size.
- Some sites are likely able to detect the automated request and/or some other issues. For example, GitHub which displays "an unsupported browser" banner in a screenshot (only in the deployment). Not sure if this is related to the binaries or something else.