Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/diabonas/arch-security-tracker-tools

Tools to make working with the Arch Linux Security Tracker easier
https://github.com/diabonas/arch-security-tracker-tools

advisory archlinux cve security tracker

Last synced: 12 months ago
JSON representation

Tools to make working with the Arch Linux Security Tracker easier

Awesome Lists containing this project

README 

          
# Arch Linux Security Tracker Tools



This is a collection of Python scripts to make working with the [Arch Linux

Security Tracker](https://github.com/archlinux/arch-security-tracker) easier.



## Features



* CVE entry extraction from multiple sources (currently

  [Chromium](#chromium), [GitLab](#gitlab), [Mozilla](#mozilla),

  [NVD](#national-vulnerability-database-nvd), [Oracle](#oracle) and [WebKitGTK](#webkitgtk)) into

  a JSON format consumable by the tracker

* Automatic batch addition of the extracted CVE entries to the tracker



## Dependencies



* python >= 3.6

* python-click

* python-lxml

* python-requests



## CVE entry extraction



CVEs can be extracted from multiple sources. All extractors take the CVEs to

be considered as a list of arguments and output the extracted CVE in JSON form.

The JSON format follows the one used by the tracker as part of its API

endpoints, e.g. .



### Chromium



```sh

./trackertools extract chromium URL...

```

extracts CVEs issued for [Chrome](https://chromereleases.googleblog.com/), where

`URL` is the URL of a Chrome release blog post, e.g.

.



The description is of the form "A `type` security issue has been found in the

`component` component of the Chromium browser engine before version

`new_version`.", where `type`, `component` and `new_version` are extracted from

the blog post. The corresponding severity is taken from the blog post as well.

The URL of the blog post and the link to the corresponding Chromium bug report

as specified in the blog post are used as references. The attack vector is

assumed to be "Remote" by default as Chromium is a browser. The type of the

vulnerability is always set to "Unknown" and needs to be filled in by hand by

the user.



### GitLab



```sh

./trackertools extract gitlab CVE...

```

extracts CVE entries assigned by the [GitLab

CNA](https://gitlab.com/gitlab-org/cves), for the GitLab products as well as

some projects hosted on GitLab. These CVEs are usually added to the NVD

database quite quickly as well, but the GitLab entries have more detailed

information regarding the CVSS score quicker.



Description and references are taken verbatim from the NVD CVE entry. Severity

and attack vector are derived from the CVSS v3. The type of the vulnerability

is always set to "Unknown" and needs to be filled by hand by the user.



### Mozilla



```sh

./trackertools extract mozilla CVE... MFSA...

```

extracts CVEs issued by

[Mozilla](https://www.mozilla.org/en-US/security/advisories/), mostly for

Firefox and Thunderbird. `MFSA` is an advisory number issued by Mozilla, e.g.

[`mfsa2021-01`](https://www.mozilla.org/en-US/security/advisories/mfsa2021-01/).

If a MFSA is specified, all CVEs included in this advisory will be extracted.



Description, references and severity are taken verbatim from the Mozilla

advisory. The attack vector is assumed to be "Remote" by default due to the

nature of the Mozilla products. The type of the vulnerability is always set to

"Unknown" and needs to be filled by hand by the user.



### National Vulnerability Database (NVD)



```sh

./trackertools extract nvd CVE...

```

extracts CVE entries from the official [National Vulnerability

Database](https://nvd.nist.gov/). This is mostly included as an example for

working with the JSON format. CVEs obtained from this source often require

manual changes to the description and references before they can be used for

the tracker.



Description and references are taken verbatim from the NVD CVE entry. Severity

and attack vector are derived from the CVSS v3 if present (this usually takes a

few day after the CVE has been published). The type of the vulnerability is

always set to "Unknown" and needs to be filled by hand by the user.



### Oracle



```sh

./trackertools extract oracle URL...

```

extracts CVEs issued by [Oracle](https://www.oracle.com/security-alerts/),

where `URL` is the URL of the *verbose text form* of an Oracle Critical Patch

Update (CPU), e.g.

.



The description is taken verbatim from the adivsory (and is therefore quite low

quality because these texts are autogenerated). Severity and attack vector are

derived from the CVSS v3. The URL of the advisory is used as a reference. The

type of the vulnerability is always set to "Unknown" and needs to be filled in

by hand by the user.



### WebKitGTK



```sh

./trackertools extract webkitgtk URL...

```

extracts CVEs issued for [WebKitGTK](https://webkitgtk.org/security.html),

where `URL` is the URL of a WebKitGTK advisory, e.g.

.



The description is of the form "A security issue has been found in `version`.

`impact`", where `version` and `impact` are extracted from the advisory. The

URL of the advisory is used as a reference. The attack vector is assumed to be

"Remote" by default as WebKitGTK is a browser engine. The type and severity of

the vulnerability are always set to "Unknown" and need to be filled in by hand

by the user.



## CVE upload to the security tracker



```sh

./trackertools add INPUT

```

adds CVEs to the Arch Linux Security Tracker. It reads a JSON file generated by

one of the extractors and tries to create a new CVE for each of the items found in

there. The necessary login credentials can be supplied using the `--username`

and `--password` options, the `TRACKER_USERNAME` and `TRACKER_PASSWORD`

environment variables, or will otherwise be queried on the TTY.



Note that only adding new CVEs is supported at the moment. Trying to add an

already existing CVE will try to merge the data according to the upstream

tracker logic, which will only partially succeed if the data is conflicting.



The URL to the tracker is set as  by default,

but can be changed for debugging purposes by setting the `--tracker-url` option

or `TRACKER_URL` environment variable, e.g. to a tracker instance running

locally:



```sh

TRACKER_URL='http://127.0.0.32:5000' ./trackertools add

```



## Example workflow



1. Download a set of CVEs using one of the extractors to a JSON file, e.g.



    ```sh

    ./trackertools extract mozilla --output=mfsa2021-01.json mfsa2021-01

    ```



2. Edit the file to check the generated data and add missing information like

the vulnerability type:



    ```sh

    $EDITOR mfsa2021-01.json

    ```



3. Upload the CVEs to the tracker:



    ```sh

    ./trackertools add mfsa2021-01.json

    ```



Missing or incorrect information can be edited afterwards using the web

interface of the tracker. Be careful, mass-editing messed up CVE entries has

not been implemented yet...



## Utilities



```sh

./trackertools strip-markdown INPUT

```

takes a JSON list of CVEs and strips all markdown formatting from it, which can

be useful if the upstream advisory makes use of it because the Arch Linux

security tracker does not. The stripped version is written to stdout by

default, but can be redirected to a file using the `--output` option.



## TODO



* Implement more extractors

* Validation of the generated JSON files, at least in `./trackertools add`

* Better error handling

* [SSO support using Keycloak](https://github.com/archlinux/arch-security-tracker/pull/181)

* Batch editing of existing CVEs