https://github.com/cisagov/code-gov-update
Update the DHS code.gov JSON
https://github.com/cisagov/code-gov-update
code-gov
Last synced: 5 months ago
JSON representation
Update the DHS code.gov JSON
- Host: GitHub
- URL: https://github.com/cisagov/code-gov-update
- Owner: cisagov
- License: cc0-1.0
- Created: 2019-03-29T16:55:30.000Z (about 7 years ago)
- Default Branch: develop
- Last Pushed: 2026-01-15T03:35:53.000Z (5 months ago)
- Last Synced: 2026-01-15T09:31:01.726Z (5 months ago)
- Topics: code-gov
- Language: Shell
- Size: 1.54 MB
- Stars: 5
- Watchers: 3
- Forks: 4
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# code-gov-update #
[](https://github.com/cisagov/code-gov-update/actions/workflows/build.yml)
[](https://spdx.org/licenses/)
[](https://github.com/cisagov/code-gov-update/actions/workflows/codeql-analysis.yml)
## Docker Image ##
[](https://hub.docker.com/r/cisagov/code-gov-update)
[](https://hub.docker.com/r/cisagov/code-gov-update)
[](https://hub.docker.com/r/cisagov/code-gov-update/tags)
This project contains code for updating the DHS
[code.gov](https://code.gov)
[inventory file](https://www.dhs.gov/code.json).
## How it works ##
The [LLNL/scraper](https://github.com/LLNL/scraper) project is used to
scrape a handful of GitHub organizations that belong to DHS and
produce an updated JSON file per [the code.gov
specification](https://code.gov/about/compliance/inventory-code). If
that file differs from the previously-generated one, it is emailed to
the appropriate address so that it can be used to update the [content
hosted by DHS](https://www.dhs.gov/code.json).
## Running ##
### Running with Docker ###
To run the `cisagov/code-gov-update` image via Docker:
```console
docker run cisagov/code-gov-update:0.3.0-rc.7
```
### Running with Docker Compose ###
1. Create a `compose.yml` file similar to the one below to use [Docker Compose](https://docs.docker.com/compose/).
```yaml
---
name: code-gov-update
services:
update:
environment:
- AWS_CONFIG_FILE=path/to/aws_config
- AWS_PROFILE=default
image: cisagov/code-gov-update:0.3.0-rc.7
init: true
```
1. Start the container and detach:
```console
docker compose up --detach
```
## Using secrets with your container ##
This container also supports passing sensitive values via [Docker
secrets](https://docs.docker.com/engine/swarm/secrets/). Passing sensitive
values like your credentials can be more secure using secrets than using
environment variables. See the
[secrets](#secrets) section below for a table of all supported secret files.
1. To use secrets, create `aws_config` and `scraper.json` files containing the
values you want set:
```ini
[default]
credential_source = Ec2InstanceMetadata
region = us-east-2
role_arn = arn:aws:iam::123456789012:role/AssumeSesSendEmail-CodeGovUpdate
```
Please see the [documentation](https://github.com/LLNL/scraper#config-file-options)
for creating your own `scraper.json` configuration file.
1. Then add the secrets to your `compose.yml` file:
```yaml
---
name: code-gov-update
secrets:
aws_config:
file: ./src/secrets/aws_config
scraper_config:
file: ./src/secrets/scraper.json
services:
update:
environment:
- AWS_CONFIG_FILE=/run/secrets/aws_config
- AWS_PROFILE=default
image: cisagov/code-gov-update:0.3.0-rc.7
init: true
secrets:
- source: aws_config
target: aws_config
- source: scraper_config
target: scraper_config.json
```
## Updating your container ##
### Docker Compose ###
1. Pull the new image from Docker Hub:
```console
docker compose pull
```
1. Recreate the running container by following the [previous instructions](#running-with-docker-compose):
```console
docker compose up --detach
```
### Docker ###
1. Stop the running container:
```console
docker stop
```
1. Pull the new image:
```console
docker pull cisagov/code-gov-update:0.3.0-rc.7
```
1. Recreate and run the container by following the [previous instructions](#running-with-docker).
## Updating Python dependencies ##
This image uses [Pipenv] to manage Python dependencies using a [Pipfile](https://github.com/pypa/pipfile).
Both updating dependencies and changing the [Pipenv] configuration in `src/Pipfile`
will result in a modified `src/Pipfile.lock` file that should be committed to the
repository.
> [!WARNING]
> The `src/Pipfile.lock` as generated will fail `pre-commit` checks due to JSON formatting.
### Updating dependencies ###
If you want to update existing dependencies you would run the following command
in the `src/` subdirectory:
```console
pipenv lock
```
### Modifying dependencies ###
If you want to add or remove dependencies you would update the `src/Pipfile` file
and then update dependencies as you would above.
> [!NOTE]
> You should only specify packages that are direct requirements of
> your Docker configuration. Allow [Pipenv] to manage the dependencies
> of the specified packages.
## Image tags ##
The images of this container are tagged with [semantic
versions](https://semver.org) of the underlying example project that they
containerize. It is recommended that most users use a version tag (e.g.
`:0.3.0-rc.7`).
| Image:tag | Description |
|-----------|-------------|
|`cisagov/code-gov-update:0.3.0-rc.7`| An exact release version. |
|`cisagov/code-gov-update:0.3`| The most recent release matching the major and minor version numbers. |
|`cisagov/code-gov-update:0`| The most recent release matching the major version number. |
|`cisagov/code-gov-update:edge` | The most recent image built from a merge into the `develop` branch of this repository. |
|`cisagov/code-gov-update:nightly` | A nightly build of the `develop` branch of this repository. |
|`cisagov/code-gov-update:latest`| The most recent release image pushed to a container registry. Pulling an image using the `:latest` tag [should be avoided.](https://vsupalov.com/docker-latest-tag/) |
See the [tags tab](https://hub.docker.com/r/cisagov/code-gov-update/tags) on Docker
Hub for a list of all the supported tags.
## Volumes ##
There are no volumes.
## Ports ##
No ports are exposed by this container.
## Environment variables ##
### Required ###
There are no required environment variables.
### Optional ###
There are no optional environment variables.
## Secrets ##
| Filename | Purpose |
|----------|---------|
| `aws_config` | Provides the necessary AWS authentication to send email using SES. |
| `scraper.json` | Provides the configuration to use for LLNL/scraper. |
## Building from source ##
Build the image locally using this git repository as the [build context](https://docs.docker.com/engine/reference/commandline/build/#git-repositories):
```console
docker build \
--tag cisagov/code-gov-update:0.3.0-rc.7 \
https://github.com/cisagov/code-gov-update.git#develop
```
## Cross-platform builds ##
To create images that are compatible with other platforms, you can use the
[`buildx`](https://docs.docker.com/buildx/working-with-buildx/) feature of
Docker:
1. Copy the project to your machine using the `Code` button above
or the command line:
```console
git clone https://github.com/cisagov/code-gov-update.git
cd code-gov-update
```
1. Create the `Dockerfile-x` file with `buildx` platform support:
```console
./buildx-dockerfile.sh
```
1. Build the image using `buildx`:
```console
docker buildx build \
--file Dockerfile-x \
--platform linux/amd64 \
--output type=docker \
--tag cisagov/code-gov-update:0.3.0-rc.7 .
```
## Contributing ##
We welcome contributions! Please see [`CONTRIBUTING.md`](CONTRIBUTING.md) for
details.
## License ##
This project is in the worldwide [public domain](LICENSE).
This project is in the public domain within the United States, and
copyright and related rights in the work worldwide are waived through
the [CC0 1.0 Universal public domain
dedication](https://creativecommons.org/publicdomain/zero/1.0/).
All contributions to this project will be released under the CC0
dedication. By submitting a pull request, you are agreeing to comply
with this waiver of copyright interest.
[Pipenv]: https://pypi.org/project/pipenv/