https://github.com/misp/misp-airgap
Scripts to deploy and update MISP in air-gapped environments
https://github.com/misp/misp-airgap
air-gapped classified intelligence-community misp threat-intelligence
Last synced: 11 months ago
JSON representation
Scripts to deploy and update MISP in air-gapped environments
- Host: GitHub
- URL: https://github.com/misp/misp-airgap
- Owner: MISP
- License: agpl-3.0
- Created: 2023-11-10T13:12:28.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2025-03-10T08:52:48.000Z (over 1 year ago)
- Last Synced: 2025-05-08T01:34:14.758Z (about 1 year ago)
- Topics: air-gapped, classified, intelligence-community, misp, threat-intelligence
- Language: Shell
- Homepage:
- Size: 308 KB
- Stars: 17
- Watchers: 8
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# MISP airgap
MISP airgap is a project that facilitates the deployment and maintenance of [MISP](https://github.com/MISP/MISP), in air-gapped environments. It utilizes [LXD](https://ubuntu.com/lxd), a popular Linux containerization platform, to create and manage isolated containers for MISP and its associated databases. Additionally, this approach is adaptable for standard networked environments, allowing for the deployment of MISP in LXD in a broader range of operational contexts.
## Key Features
- Automated setup and configuration of MISP in a secure, isolated environment.
- Containerized approach using LXD for easy management and isolation.
- Support for both interactive and non-interactive installation modes.
- Comprehensive validation and security checks, ensuring secure deployment.
- Modular setup allowing for easy updates and maintenance.
## Requirements
Before setting up your environment, ensure that you meet the following prerequisites on your host system:
- **Operating System:**
- [Ubuntu 22.04](https://releases.ubuntu.com/jammy/)
- **Containerization:**
- [LXD 5.19](https://documentation.ubuntu.com/lxd/en/latest/installing/)
- **Additional Software:**
- [jq 1.6](https://jqlang.github.io/jq/download/)
- [yq 4.35.2](https://github.com/mikefarah/yq/#install)
## Hardware Requirements
To run all containers set up by the installation script, the following hardware specifications are recommended:
- **CPU**:
- Minimum: 4 cores
- Recommended for optimal performance: 4 or more cores
- **Memory (RAM)**:
- Minimum: 8 GB
- Recommended: 16 GB or more for better performance
- **Storage**:
- Minimum: 50 GB
- Recommended: 100 GB or more, SSD preferred for better performance
## Installation
First, [install LXD](https://ubuntu.com/lxd/install) on your air-gapped host system. Additionally, install the [additional software](#requirements) needed.
After installation, proceed with the following steps:
1. **Download Images**
You can download the images from the [MISP images page](https://images.misp-project.org/). It is recommended to use the latest version of the images. For a minimal air-gapped setup, you need the following images:
- `MISP`
- `MySQL`
- `Redis`
If you want to use MISP Modules, you also need the `Modules` image.
2. **Verify Signature**
Download the signature file for the images you want to use. You can find the signature files in the same directory as the images. Verify the signature using GPG:
You can find the public key for verifying the images on CIRCL's [PGP key server](https://openpgp.circl.lu/pks/lookup?op=get&search=0xec1862fc82cdaf7aebabc002287b725897d881d2).
Import the MISP-airgap public key:
```bash
gpg --import /path/to/misp-airgap.asc
```
Verify the signature using GPG:
```bash
gpg --verify /path/to/file.sig /path/to/file
```
3. **Transfer images and repo to air-gapped system**:
Transfer the exported images and the whole repo to your air gapped system.
### Interactive Mode
Run the `INSTALL.sh` script with the `--interactive` flag to enter the interactive mode, which guides you through the configuration process:
```bash
bash INSTALL.sh --interactive
```
### Non-Interactive Mode
For a non-interactive setup, use command-line arguments to set configurations:
**Example:**
```bash
bash INSTALL.sh --misp-image --mysql-image --redis-image --no-modules
```
Below is the table summarizing the script flags and variables:
| Variable | Default Value | Flag | Description |
| --------------------- | ------------------------------ | --------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `INTERACTIVE_MODE` | N/A | `-i`, `--interactive` | Activates an interactive installation process. |
| `PROJECT_NAME` | `misp-project-` | `--project ` | Name of the LXD project used to organize and run the containers. |
| `MISP_IMAGE` | `` | `--misp-image ` | The exported image file containing the configuration and setup of the MISP instance. |
| `MISP_CONTAINER` | `misp-` | `--misp-name ` | The name of the container responsible for running the MISP application. |
| `MYSQL_IMAGE` | `` | `--mysql-image ` | The exported image file of a MariaDB instance, containing the necessary configurations. |
| `MYSQL_CONTAINER` | `mysql-` | `--mysql-name ` | The name of the container running the MariaDB database for MISP. |
| `MYSQL_DATABASE` | `misp` | `--mysql-db ` | The name of the database used by the MISP application. |
| `MYSQL_USER` | `misp` | `--mysql-user ` | The database user for MISP to interact with the MariaDB database. |
| `MYSQL_PASSWORD` | `misp` | `--mysql-pwd ` | The password associated with the MISP database user. |
| `MYSQL_ROOT_PASSWORD` | `misp` | `--mysql-root-pwd ` | The root user password for MariaDB. |
| `REDIS_IMAGE` | `` | `--redis-image ` | The exported image file for the Redis instance, including necessary configurations. |
| `REDIS_CONTAINER` | `redis-` | `--redis-name ` | The name of the container running the Redis server for MISP. |
| `MODULES` | `yes` | `--no-modules` | If set, a container with MISP Modules gets set up. |
| `MODULES_IMAGE` | `` | `--modules-image ` | The exported image file of a MISP Modules instance, containing the necessary configurations. |
| `MODULES_CONTAINER` | `modules-` | `--modules-name ` | The name of the container running MISP Modules. |
| `APP_PARTITION` | `` | `--app-partition ` | Dedicated partition for the storage of the MISP container. |
| `DB_PARTITION` | `` | `--db-partition ` | Dedicated partition for the storage of the database container(s). |
| `PROD` | `no` | `-p`, `--production` | If set to true, the MISP application runs in production mode, activating the `islive` option and adjusting settings accordingly. |
>**Note**: It is crucial to **modify all default credentials** when using this installation in a production environment. Specifically, if the PROD variable is set to true, the installer will not accept default values.
After completing these steps, MISP should be up and running. You can add the IP address of the MISP container to your `/etc/hosts` file to be able to access it through the set domain (default: misp.local). The IP address is displayed when the installing process has finished. Alternatively, you can identify the IP addresses of all running containers within the project by executing the command `lxc list`.
## Update
1. **Download Images**
You need to dowload the images for the components you want to update. You can download the images from the [MISP images page](https://images.misp-project.org/). It is recommended to use the latest version of the images.
2. **Verify Signature**
Download the signature file for the images you want to use. You can find the signature files in the same directory as the images. Verify the signature using GPG:
You can find the public key for verifying the images on CIRCL's [PGP key server](https://openpgp.circl.lu/pks/lookup?op=get&search=0xec1862fc82cdaf7aebabc002287b725897d881d2).
Import the MISP-airgap public key:
```bash
gpg --import /path/to/misp-airgap.asc
```
Verify the signature using GPG:
```bash
gpg --verify /path/to/file.sig /path/to/file
```
3. **Transfer images to air-gapped system**:
Transfer the exported images to your air gapped system.
### Interactive Mode
Run the `UPDATE.sh` script with the `--interactive` flag to enter the interactive mode, which guides you through the configuration process:
```bash
bash UPDATE.sh --interactive
```
>Note: It is recommende to use new names for the newly created containers. However if you want to keep the current container names, you can enter the same name for the new container as the current one. Please refer to the [Container Naming Conventions](#container-naming-conventions) section for more information.
### Non-Interactive Mode
For a non-interactive setup, use command-line arguments to set configurations:
**Example:**
```bash
bash UPDATE.sh --current-misp --update-misp -p --misp-image
```
Below is the table summarizing the script flags and variables:
| Variable | Default Value | Flag | Description |
| --------------------- | ------------------------- | --------------------------- | ---------------------------------------------------- |
| `INTERACTIVE_MODE` | N/A | `-i`, `--interactive` | Activates an interactive installation process. |
| `MYSQL_ROOT_PASSWORD` | User Input Required | `-p`, `--passwd ` | Set the MySQL root password. |
| `MISP` | `no` | `--update-misp` | Update MISP container. |
| `MISP_IMAGE` | `` | `--misp-image ` | Specify the MISP image. |
| `CURRENT_MISP` | `` | `--current-misp ` | Specify the current MISP container name (Mandatory). |
| `NEW_MISP` | `misp-` | `--new-misp ` | Specify the new MISP container name. |
| Multiple | N/A | `-a`, `--all` | Apply updates to all components. |
| `MYSQL` | `no` | `--update-mysql` | Update MySQL container. |
| `MYSQL_IMAGE` | `` | `--mysql-image ` | Specify the MySQL image. |
| `NEW_MYSQL` | `mysql-` | `--new-mysql ` | Specify the new MySQL container name. |
| `REDIS` | `no` | `--update-redis` | Update Redis container. |
| `REDIS_IMAGE` | `` | `--redis-image ` | Specify the Redis image. |
| `NEW_REDIS` | `redis-` | `--new-redis ` | Specify the new Redis container name. |
| `MODULES` | `no` | `--update-modules` | Update modules container. |
| `MODULES_IMAGE` | `` | `--modules-image ` | Specify the modules image. |
| `NEW_MODULES` | `modules-` | `--new-modules ` | Specify the new modules container name. |
### Container Naming Conventions
In LXD container names are used to identify containers for various operations including networking. Therefore, it is important to use a consistent naming convention for containers.
As LXD does not allow containers with the same name, the script will automatically rename the old container if the new container name is the same as the old one. In iteractive mode the script will ask you if you are sure you want to rename the container. In non-interactive mode the script will automatically rename the container.
The script will use the following naming convention for renamed containers:
`updated--`
All changes made by the script will be logged in the `naming.log` file in the `logs/` directory.
## Container Management
Installing and updating MISP using the MISP airgap project creates and manages containers using LXD. The following commands can be used to manage the containers created by the installation script. If you are not familiar with LXD, you can find more information in the [LXD documentation](https://documentation.ubuntu.com/lxd/en/latest/).
| Command | Description |
| ------------------------------------------ | ------------------------------------------- |
| `lxc list` | List all containers in the current project. |
| `lxc start ` | Start a container. |
| `lxc stop ` | Stop a container. |
| `lxc project list` | List all projects. |
| `lxc project switch ` | Switch to a different project. |
| `lxc snapshot ` | Create a snapshot of a container. |
| `lxc restore ` | Restore a container to a snapshot. |
## Build
If you want to build the images yourself, you can use the `build.sh` script in the `build/` directory. This is completely optional, as the images are already built and available for download.
**Requirements**:
- jq 1.6
- curl 7.81.0
- gpg (GnuPG) 2.2.27
You can build the images using the build script:
```bash
bash build.sh [OPTIONS]
```
Below is the table summarizing the script options:
| Variable | Default Value | Flag | Description |
| --------------- | ------------- | --------------------------- | ------------------------------------------------ |
| `MISP` | `false` | `--misp` | Create a MISP image. |
| `MYSQL` | `false` | `--mysql` | Create a MySQL image. |
| `REDIS` | `false` | `--redis` | Create a Redis image. |
| `MODULES` | `false` | `--modules` | Create a Modules image. |
| `MISP_IMAGE` | `MISP` | `--misp-name ` | Specify a custom name for the MISP image. |
| `MYSQL_IMAGE` | `MySQL` | `--mysql-name ` | Specify a custom name for the MySQL image. |
| `REDIS_IMAGE` | `Redis` | `--redis-name ` | Specify a custom name for the Redis image. |
| `MODULES_IMAGE` | `Modules` | `--modules-name ` | Specify a custom name for the Modules image. |
| `REDIS_VERSION` | N/A | `--redis-version ` | Specify a Redis version to build. |
| `MYSQL_VERSION` | N/A | `--mysql-version ` | Specify a MySQL version to build. |
| `OUTPUTDIR` | N/A | `-o`, `--outputdir ` | Specify the output directory for created images. |
| `SIGN` | `false` | `-s`, `--sign` | Sign the created images. |
### Signing
When the `-s` or `--sign` flag is used, the `build.sh` script will sign the created images using GPG. To utilize this feature, first configure your signing keys in the `/conf/sign.json` file. You can use the provided template file as a starting point:
```bash
cd ./build/conf
cp sign.json.template sign.json
```
If no key with the specified ID is found in your GPG keyring, the script will automatically generate a new key.
### Running Image Creation with systemd
This section describes how to run the image creation process as a systemd service on a Linux system. The service is designed to periodically check for updates in specified GitHub repositories and execute an image creation process using the `build.sh` script if new updates are found.
**Prerequisites**
- Ubuntu 22.04 system with systemd.
- Python 3 and the requests module installed.
- Access to GitHub repositories (internet connection required).
**Config**
Edit the `tracker.json` configuration file in `build/conf/` to specify the GitHub repositories to track, the build arguments, and the check interval. You can use the provided template file as a starting point:
```bash
cd ./build/conf
cp tracker.json.template tracker.json
```
If your build process requires GPG signing, edit the `sign.json` configuration file in `build/conf/` by copying the template and modifying the default values:
```bash
cd ./build/conf
cp sign.json.template sign.json
```
**Setup**
To setup the service run the `setup.sh` script in the `systemd/` directory. This script performs several tasks:
- Creates a dedicated user for the service.
- Copies necessary files to appropriate locations.
- Sets required permissions.
- Automatically configures and enables the systemd service.
To run the script, execute:
```bash
cd build/systemd
sudo bash setup.sh
```
**Monitoring**
To check the status of the service, use:
```bash
systemctl status updatetracker.service
```
For debugging or monitoring, access the service logs with:
```bash
journalctl -u updatetracker.service
```
**Modifying**
If you need to modify the service (e.g., changing the repositories to track or the check interval), update the `tracker.json` file in `/opt/misp_airgap/build/conf/` and restart the service:
```bash
systemctl restart updatetracker.service
```
Alternatively, you can use the `update.sh` script in the `systemd/` directory to automatically update the service configuration and restart the service:
```bash
sudo bash update.sh
```
This can be helpfull if there are changes to the scripts used by the service such as `build.sh` or `updatetracker.py`.
*Alternative Method Using `update.sh` Script:*
If your modifications include changes to the service's operational scripts (like `build.sh` or `updatetracker.py`), it's recommended to use the `update.sh` script. This script ensures that the service configuration is updated and the service is restarted to reflect the changes.
To use the update.sh script:
```bash
cd /path/to/systemd/
sudo bash update.sh
```
> Note: Using the update.sh script is especially useful for comprehensive updates, as it automates the process of applying configuration changes and restarting the service.
## Maintenance
A few useful commands to manage the LXC images.
* Open a shell on the MISP image:
```bash
lxc exec misp- bash
```
* reset the password of the MISP user:
```bash
sudo -u www-data /var/www/MISP/app/Console/cake user change_pw admin@admin.test Password1234
```