https://github.com/aeharding/tfr-scraper
Scrape TFRs and store them in MongoDB (with a geospatial index)
https://github.com/aeharding/tfr-scraper
Last synced: 3 months ago
JSON representation
Scrape TFRs and store them in MongoDB (with a geospatial index)
- Host: GitHub
- URL: https://github.com/aeharding/tfr-scraper
- Owner: aeharding
- Created: 2022-08-05T03:13:51.000Z (about 4 years ago)
- Default Branch: main
- Last Pushed: 2026-03-13T23:35:39.000Z (5 months ago)
- Last Synced: 2026-03-14T11:03:13.081Z (5 months ago)
- Language: TypeScript
- Size: 147 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# tfr-scraper
AWS Lambda service that scrapes FAA Temporary Flight Restrictions (TFRs) and stores them in MongoDB for geo-querying.
## Architecture
Two Lambda functions deployed via Serverless Framework:
- **scraper** -- Runs every 15 minutes. Fetches the current TFR list and geometry from the [FAA WFS GeoJSON endpoint](https://tfr.faa.gov/geoserver/TFR/ows?service=WFS&version=1.1.0&request=GetFeature&typeName=TFR:V_TFR_LOC&maxFeatures=300&outputFormat=application/json), then retrieves detailed NOTAM data from the [FAA NMS-API](https://api-nms.aim.faa.gov/nmsapi/v1). Results are stored in MongoDB with a 2dsphere index.
- **query** -- `GET /api/tfr?lat=...&lon=...&radialDistance=...` returns TFRs within the given radius (meters).
## Prerequisites
- Node.js 24+
- [Serverless Framework v3](https://www.serverless.com/framework/docs/getting-started)
- AWS CLI configured with credentials (`aws configure`)
- A MongoDB Atlas cluster
- FAA NMS-API credentials (email [7-AWA-NAIMES@faa.gov](mailto:7-AWA-NAIMES@faa.gov) to request access)
## Setup
### 1. Install dependencies
```bash
yarn install
```
### 2. Create config file
Create a `config.prod.json` in the project root (gitignored):
```json
{
"MONGODB_USER": "your-mongodb-user",
"MONGODB_PASSWORD": "your-mongodb-password",
"MONGODB_HOST": "your-cluster.mongodb.net",
"NMS_API_HOST": "https://api-nms.aim.faa.gov",
"NMS_API_KEY": "your-nms-api-key",
"NMS_API_SECRET": "your-nms-api-secret"
}
```
For other stages, create `config..json` (e.g. `config.dev.json`).
## Deployment
```bash
npx serverless deploy --stage prod
```
To deploy to a different stage:
```bash
npx serverless deploy --stage dev
```
## Invoking manually
Trigger the scraper on-demand:
```bash
npx serverless invoke --function scraper --stage prod
```
Test the query endpoint:
```bash
curl "https://.execute-api.us-east-1.amazonaws.com/api/tfr?lat=38.85&lon=-77.04&radialDistance=50000"
```