Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/creativecommons/sre-salt-prime

Site Reliability Engineering / DevOps SaltStack configuration files
https://github.com/creativecommons/sre-salt-prime
aws git-crypt saltstack
Last synced: about 3 hours ago
JSON representation
Site Reliability Engineering / DevOps SaltStack configuration files
Host: GitHub
URL: https://github.com/creativecommons/sre-salt-prime
Owner: creativecommons
License: mit
Created: 2018-11-20T21:50:33.000Z (almost 6 years ago)
Default Branch: main
Last Pushed: 2024-07-24T15:00:08.000Z (2 months ago)
Last Synced: 2024-09-18T19:59:13.318Z (6 days ago)
Topics: aws, git-crypt, saltstack
Language: SaltStack
Homepage:
Size: 1.98 MB
Stars: 20
Watchers: 10
Forks: 9
Open Issues: 5
Metadata Files:
- Readme: README.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project

README

        # sre-salt-prime

Site Reliability Engineering / DevOps SaltStack configuration files

## Code of Conduct

[`CODE_OF_CONDUCT.md`][org-coc]:

> The Creative Commons team is committed to fostering a welcoming community.

> This project and all other Creative Commons open source projects are governed

> by our [Code of Conduct][code_of_conduct]. Please report unacceptable

> behavior to [[email protected]](mailto:[email protected])

> per our [reporting guidelines][reporting_guide].

[org-coc]: https://github.com/creativecommons/.github/blob/main/CODE_OF_CONDUCT.md

[code_of_conduct]: https://opensource.creativecommons.org/community/code-of-conduct/

[reporting_guide]: https://opensource.creativecommons.org/community/code-of-conduct/enforcement/

## Contributing

See [`CONTRIBUTING.md`][org-contrib].

[org-contrib]: https://github.com/creativecommons/.github/blob/main/CONTRIBUTING.md

## Development Notes

- **Avoid insecure repository clones:** This repository includes encrypted

  secrets. Do not run `git-crypt unlock` on clones that are not otherwise

  secured (ex. strong login password, disk encryption).

- **Avoid editing the base environment:** The base environment is configured to

  prevent commit and push actions. Please use your development environment and

  pull the changes to base.

- **Sign your commits:**

  - The commit signing option has been adjusted in the repository to facilitate 

   smoother collaboration. This modification is intended to ease the contribution 

   process. However, it is strongly encourged that staff members continue to 

   adhere to best practices by using GPG for all signed commits, ensuring the 

   security and integrity of the project.

    - [About required commit signing - User Documentation][signing]

  - Ensure you are using `RemoteForward` in your SSH configuration to forward

    your GnuPG agent to `salt-prime` (see the example configuration, under

    [Setup](#Setup), below).

  - Ensure you have configured your newly cloned repository to sign commits

    (see the `git config` command, under [Setup](#Setup), below).

[signing]:https://help.github.com/articles/about-required-commit-signing/

### Setup

- **SSH connection information:** example local/laptop `~/.ssh/config`

  configugration:

    ```

    Host bastion-us-east-2

        HostName bastion-us-east-2.creativecommons.org

        User ARTHUR

    Host salt-prime

        HostName 10.22.11.11

        ProxyJump bastion-us-east-2

        RemoteForward /run/user/4242/gnupg/S.gpg-agent /Users/ARTHUR/.gnupg/S.gpg-agent.extra

        User ARTHUR

    Host *

        ServerAliveCountMax 60

        ServerAliveInterval 30

        TCPKeepAlive no

    ```

    - Assumes remote username ARTHUR and remote uid 4242. Replace these values

      in your own local/laptop configuration.

    - ProxyJump allows you to `ssh salt-prime` from your local/laptop.

    - RemoteForward allows you to sign your commits.

- **Setup your development repository** on `salt-prime`:

  1. Clone repository to `/srv` with your username. For example:

        ```shell

        cd /srv

        git clone [email protected]:creativecommons/sre-salt-prime.git ${USER}

        ```

  2. Setup your newly cloned repository.

     1. Configure commit signing:

        ```shell

        cd /srv/${USER}

        git config user.email YOUR_EMAIL

        git config user.signingkey YOUR_GPG_ID

        git config commit.gpgsign true

        ```

     1. Unlock encrypted secrets:

        ```shell

        cd /srv/${USER}

        git-crypt unlock

        ```

  3. Specify the environment when you test changes. For example:

        ```shell

        sudo salt \* state.highstate saltenv=${USER} test=True

        ```

     - use `--state-verbose=True` to see successes

     - use `--state-output=full_id` to see full detail of successes

     - use `--log-level=debug --log-file-level=warning` to see debug messages

       (without logging those debug messages, which may contain secrets, to the

       log file)

### Goals

- Use AWS well, but avoid technologies that create AWS lock-in (ex. Confidant)

- Salt Prime must not contain any exclusive data (use Git)

- Git repository must not contain any unencrypted secrets

- Git repository commits must be signed and applied to the main branch via

  Pull Requests

- A compromised minion must not be able to escalate access

  - SysAdmins must not forward their SSH agent

  - Must not reuse application passwords (ex. Prod and Dev databases must have

    different passwords)

  - Pillar data must be restricted by Minion ID based classification

    - *The only grain which can be safely used is `grains['id']` which contains

      the Minion ID.* ([FAQ Q.21][FAQ21])

[FAQ21]: https://docs.saltstack.com/en/latest/faq.html#is-targeting-using-grain-data-secure

### Decisions

- Amazon Web Services (AWS)

  - Creative Commons is already using it and staff are familiar with it

  - Features allow security (ex. screened subnets, security groups policies)

  - Features allows Infrastructure as Code

  - `us-east-2`

    - cost effective

    - avoid conflict/collision over region limited resources (ex. ElasticIPs)

- Debian 11 (Bullseye), Debian 10 (Buster), and Debian 9 (Stretch)

  - Free/Open Source

  - Debian Stable

  - Creative Commons is already using it and staff are familiar with it

- [git-crypt][gitcrypt] - transparent file encryption in git

  - Free/Open Source

  - Performance: files are decrypted in the checked out repository

  - Security: automatic encryption and directory based filters minimize the

    chance of unencrypted secrets being pushed to GitHub

- SaltStack

  - Free/Open Source

  - Performance

  - Creative Commons is already using it and staff are familiar with it

  - Version: `3006.8`

    - For current targeted minion version, see `minion_target_version` in

      [`pillars/salt/init.sls`](pillars/salt/init.sls)

[gitcrypt]: https://www.agwa.name/projects/git-crypt/

## Host Classification

Minions are added and configured from `salt-prime` with the following Minion ID

schema: **`HST__POD__LOC`** (host/role__pod/group__location). These variables

are used to determine the state and pillar data.

Show top states example command:

```

sudo salt \* pillar.item states saltenv=${USER}

```

See [`docs/Host_Classification.md`](docs/Host_Classification.md) for details.

## Orchestration

See [`docs/Orchestration.md`](docs/Orchestration.md).

## References

### SaltStack

- For the SaltStack version that this repository is developed on, see

  [Decisions](#Decisions), above.

- This repository attempts to tracks the most current release of SaltStack in

  the SaltStack Debian repository: https://repo.saltstack.com/apt/debian/

- For current version of SaltStack in Debian proper, see [Debian -- Package

  Search Results -- salt-master][pkgsearch]

[pkgsearch]: https://packages.debian.org/search?suite=default&section=all&arch=any&searchon=names&keywords=salt-master

####  Best Practices

- [Hardening Salt][hardensalt]

  - *The only grain which can be safely used is `grains['id']` which contains

    the Minion ID.* ([FAQ Q.21][FAQ21])

- [Salt Best Practices][saltbest]

- [Salt Formulas][saltformulas]

[hardensalt]: https://docs.saltstack.com/en/latest/topics/hardening.html

[saltbest]: https://docs.saltstack.com/en/latest/topics/best_practices.html

[saltformulas]: https://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html

#### Frequently Referenced Documentation

- [Salt Module Reference][moduleref]

  - [state modules][statemodules]

- [Requisites and Other Global State Arguments][requisites]

[moduleref]: https://docs.saltstack.com/en/latest/ref/index.html

[statemodules]: https://docs.saltstack.com/en/latest/ref/states/all/index.html

[requisites]: https://docs.saltstack.com/en/latest/ref/states/requisites.html

## Repository Documentation

- [`README.md`](README.md)

- [`bootstrap-aws/README.md`](bootstrap-aws/README.md)

- `docs/`

  - [`Host_Classification.md`](docs/Host_Classification.md)

  - [`Orchestration.md`](docs/Orchestration.md)

  - [`WordPress.md`](docs/WordPress.md)

## Formula Repositories

- [saltstack-formulas/mysql-formula][mysql-formula]: Install the MySQL client

  and/or server

- [saltstack-formulas/php-formula][php-formula]: Formula to set up and

  configure php

[mysql-formula]: https://github.com/saltstack-formulas/mysql-formula

[php-formula]: https://github.com/saltstack-formulas/php-formula

## License

- [`LICENSE`](LICENSE) (Expat/[MIT][mit] License)

[mit]: http://www.opensource.org/licenses/MIT "The MIT License | Open Source Initiative"