https://github.com/mdn/mdn-http-observatory
Backend for HTTP Observatory on MDN
https://github.com/mdn/mdn-http-observatory
http http-headers mdn privacy security security-tools
Last synced: 2 days ago
JSON representation
Backend for HTTP Observatory on MDN
- Host: GitHub
- URL: https://github.com/mdn/mdn-http-observatory
- Owner: mdn
- License: mpl-2.0
- Created: 2024-06-25T12:43:58.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2026-04-14T08:18:53.000Z (3 days ago)
- Last Synced: 2026-04-14T10:11:20.119Z (3 days ago)
- Topics: http, http-headers, mdn, privacy, security, security-tools
- Language: JavaScript
- Homepage: https://developer.mozilla.org/en-US/observatory
- Size: 5.73 MB
- Stars: 123
- Watchers: 3
- Forks: 33
- Open Issues: 27
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
- Security: SECURITY.md
Awesome Lists containing this project
README
# Welcome to Mozilla's HTTP Observatory
[HTTP Observatory](https://developer.mozilla.org/en-US/observatory/) is a service that checks web sites for security-relevant headers. It is hosted by [MDN Web Docs](https://github.com/mdn).
## Getting Started
If you just want to scan a host, please head over to . If you want to
run the code locally or on your premises, continue reading.
### Running a simple scan from the command line
Using npx to install the package, simply run
```sh
npx @mdn/mdn-http-observatory mdn.dev
```
Subpaths like `example.com/path` and port numbers like `example.com:8080/path` are suported.
If you want to install the package first, use npm to install it globally
```sh
npm install --global @mdn/mdn-http-observatory
```
After that, the `mdn-http-observatory-scan` command should be available in your shell. To scan a host, run
```sh
mdn-http-observatory-scan mdn.dev
```
You can pass custom request headers as JSON using the `--headers` option:
```sh
mdn-http-observatory-scan --headers '{"X-Custom": "value"}' mdn.dev
```
**Warning:** Headers will also be sent on unencrypted HTTP requests, even if the host enforces HTTPS. Do not pass sensitive data.
Both methods return a JSON response of the following form:
```json
{
"scan": {
"algorithmVersion": 4,
"grade": "A+",
"error": null,
"score": 105,
"statusCode": 200,
"testsFailed": 0,
"testsPassed": 10,
"testsQuantity": 10,
"responseHeaders": {
...
}
},
"tests": {
"cross-origin-resource-sharing": {
"expectation": "cross-origin-resource-sharing-not-implemented",
"pass": true,
"result": "cross-origin-resource-sharing-not-implemented",
"scoreModifier": 0,
"data": null
},
...
}
}
```
### Running a local API server
This needs a [postgres](https://www.postgresql.org/) database for the API to use as a persistence layer. All scans and results initiated via the API are stored in the database.
#### Configuration
Default configuration is read from a default `config/config.json` file. See [this file](src/config.js) for a list of possible configuration options.
Create a configuration file by copying the [`config/config-example.json`](conf/config-example.json) to `config/config.json`.
Put in your database credentials into `config/config.json`:
```json
{
"database": {
"database": "observatory",
"user": "postgres"
}
}
```
To initialize the database with the proper tables, use this command to migrate. This is a one-time action, but future code changes
might need further database changes, so run this migration every time the code is updated from the repository.
```sh
npm run migrate
```
Finally, start the server by running
```sh
npm start
```
The server is listening on your local interface on port `8080`. You can check the root path by opening in your browser or `curl` the URL. The server should respond with `Welcome to the MDN Observatory!`.
## JSON API
**Note:** We provide these endpoints on our public deployment of HTTP Observatory at
### POST `/api/v2/scan`
For integration in CI pipelines or similar applications, a JSON API endpoint is provided. The request rate is limited to one scan per host per `api.cooldown` (default: One minute) seconds. If exceeded, a cached result will be returned.
#### Query parameters
- `host` hostname (required)
#### Examples
- `POST /api/v2/scan?host=mdn.dev`
- `POST /api/v2/scan?host=google.com`
#### Result
On success, a JSON object is returned, structured like this example response:
```json
{
"id": 77666718,
"details_url": "https://developer.mozilla.org/en-US/observatory/analyze?host=mdn.dev",
"algorithm_version": 4,
"scanned_at": "2024-08-12T08:20:18.926Z",
"error": null,
"grade": "A+",
"score": 105,
"status_code": 200,
"tests_failed": 0,
"tests_passed": 10,
"tests_quantity": 10
}
```
**Note:** For a full set of details about the host, use the provided link in the `details_url` field.
If an error occurred, an object like this is returned:
```json
{
"error": "invalid-hostname-lookup",
"message": "some.invalid.hostname.dev cannot be resolved"
}
```
## Migrating from the public V1 API to the V2 API
### Sunset of the V1 API
The previous iteration of the Observatory JSON API has been deprecated and shut down on October 31, 2024.
### Migrating your application
If you previously used the Observatory API with some automation or a CI context, the switch from the old `/api/v1/analyze` endpoint to the new `/api/v2/scan` endpoint should be painless:
- Replace all API calls to `POST https://http-observatory.security.mozilla.org/api/v1/analyze?host=` with `POST https://observatory-api.mdn.mozilla.net/api/v2/scan?host=`
- Be aware that the complete list of headers has been removed from the response.
- The POST parameters `rescan` and `hidden` in the POST body have been removed.
- Remove all other requests from your application, if any. If you need any additional information about your scan, open the URL from the `detail_url` field of the response in your browser.
- Note that scans are still limited to one every minute per host, otherwise a cached response is returned.
## Contributing
Our project welcomes contributions from any member of our community.
To get started contributing, please see our [Contributor Guide](CONTRIBUTING.md).
By participating in and contributing to our projects and discussions, you acknowledge that you have read and agree to our [Code of Conduct](CODE_OF_CONDUCT.md).
## Communications
If you have any questions, please reach out to us on [Mozilla Developer Network](https://developer.mozilla.org).
## License
This project is licensed under the [Mozilla Public License 2.0](LICENSE).