{"id":23512613,"url":"https://github.com/linkorb/infra","last_synced_at":"2025-04-19T12:17:59.997Z","repository":{"id":48458658,"uuid":"100495442","full_name":"linkorb/infra","owner":"linkorb","description":"🚀 INFRA: your infrastructure as a GraphQL service","archived":false,"fork":false,"pushed_at":"2020-12-15T09:35:51.000Z","size":181,"stargazers_count":48,"open_issues_count":1,"forks_count":5,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-29T07:41:52.156Z","etag":null,"topics":["ansible","devops","firewall-rules","graphql","infra","infrastructure","monitoring"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/linkorb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-08-16T13:59:17.000Z","updated_at":"2024-12-04T12:20:00.000Z","dependencies_parsed_at":"2022-08-24T14:38:50.087Z","dependency_job_id":null,"html_url":"https://github.com/linkorb/infra","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkorb%2Finfra","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkorb%2Finfra/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkorb%2Finfra/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linkorb%2Finfra/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/linkorb","download_url":"https://codeload.github.com/linkorb/infra/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249692904,"owners_count":21311420,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","devops","firewall-rules","graphql","infra","infrastructure","monitoring"],"created_at":"2024-12-25T13:18:03.685Z","updated_at":"2025-04-19T12:17:59.947Z","avatar_url":"https://github.com/linkorb.png","language":"PHP","readme":"\u003cimg src=\"https://upr.io/pRyK8d.png\" style=\"width: 100%\" /\u003e\n\n## Introduction\n\nInfra allows you to define all objects in your infrastructure in a way that you can load it\ninto a GraphQL service. For example: Hosts, HostGroups, Users, Dns, Monitoring, BackupRules, Deployments, etc, and all of their relationships.\n\nAccessing your infrastructure as a graph allows you to do a couple of cool things:\n\n* Generate multiple inventory definitions from a single source (ansible, puppet, rundeck, etc)\n* Generate configuration files for backup and monitoring services\n* Generate complex firewall scripts\n* Configure objects in context. I.e. configure dns records as part of your Deployment, or monitoring rules as part of your HostGroup\n* Generate dashboard configurations based on infra objects\n* Generate comprehensive infrastructure documentation\n* Create scripts that operate on your infrastructure data in language agnostic ways\n* As your configuration consists of YAML files only, everything is heavily scriptable (for both importing and exporting data from all relevant systems) and managable through version control.\n\n## Resources\n\nYou define your infrastructure as a set of Resources in YAML files. The format is heavily inspired by Kubernetes. Here's an example:\n\n```yaml\n---\nkind: Host\nmetadata:\n  name: my-app-server\n  description: This my application server\nspec:\n  os: Ubuntu 16.04\n  publicIp: 192.168.1.1\n  privateIp: 10.0.1.1\n  hostGroups: app-servers\n---\nkind: HostGroup\nmetadata:\n  name: app-servers\n  description: All my application server\n  labels:\n    color: green\nspec: ~\n```\n\nFor a more comprehensive example, see the `example/` directory in this repository.\n\nYou can define your resources in standalone YAML files, or (as in this example) configure multiple resources in one file using YAML's [multiple document feature](https://en.wikipedia.org/wiki/YAML#Advanced_components)\n\nIf you're familiar with Kubernetes, you'll feel right at home.\n\nEach resource specifies it's type using the `kind` key. A list of available resource types is provided below.\n\nResources contain a `metadata` key that allows you to specify the resource name, description, a set of labels and annotations (more on metadata below).\n\nThe `spec` key configures the resource, and it's available keys depend on the `kind` of resource you're creating.\n\n## Installation\n\n    cd /opt\n    git clone git@github.com:linkorb/infra.git\n    cd infra\n    composer install # See https://getcomposer.org/download/ if you don't have composer installed yet\n\n**NOTE:** It is **strongly** recommended to add `bin/` to your environment's `PATH` variable, so you can easily invoke the `infra` command-line tool from anywhere on your system.\n\nIf you're using Bash, you can do this by adding the following line to your `~/.bashrc` file:\n\n    PATH=/opt/infra/bin:$PATH\n\nDon't forget to open a new bash session for the changes to take effect (close your terminal, or just run `bash` again).\n\n## Configuration\n\nInfra decides where to load it's infrastructure data based on the `INFRA_CONFIG` environment variable. It should contain a path to directory that contains a set of YAML files describing your infrastructure.\n\nYou can define the variable wherever you like. For example in your `~/.bashrc`, or a `.env` file in the root of the infra directory (i.e. `/opt/infra/.env`). A `.env.dist` file is provided.\n\nIf `INFRA_CONFIG` is undefined, it will point to the `example/` directory as a default.\n\n## Example project\n\nThe `example/` directory contains an example infrastructure configuration (You can view `example/README.md` for it's details). To load it, make sure your `INFRA_CONFIG` variable points to the `example/` directory (absolute path required), or simply leave `INFRA_CONFIG` undefined: the example project is loaded by default.\n\nIt is recommended to explore the example project first, before using infra to configure your own infrastructure. The example configuration will contain examples for all relevant resource types, configured in a sensible way to show you all available functionality.\n\nOnce you're ready to define your own infrastructure, simply point `INFRA_CONFIG` to a new directory (ideally a git repository), and start creating configuration files there.\n\n## Usage\n\nYou can get a list of available commands by simply typing `infra list` or just `infra`.\n\n### Exploring using the `get` command.\n\nThe `infra get` command lets you explore your infrastructure configuration. Type `infra get --help` for it's options. All arguments are optional, and the more you specify, the more specific your response will be. Some examples:\n\n* `infra get`: returns a list of available resource types (and their aliases)\n* `infra get hosts`: returns all defined objects of that resource type. \n* `infra get hg`: you can use any of the available resource type aliases as shortcuts: hg, hostGroup, HostGroup, hostGroups, HostGroups are all equivalent here.\n* `infra get Host app0`: returns a resource definition by type and name.\n* `infra get HostGroup db hosts`: returns a calculated property from a resource by type, name and propertyname.\n \n### Running GraphQL queries\n\nThe `infra query` command let's you perform a GraphQL query against your infrastructure configuration.\nIt reads the query from `stdin` and outputs the response (as json) to `stdout`.\n\nFor example:\n\n    infra query \u003c /opt/infra/example/graphql/example.graphql\n\nIntrospection is also enabled, so you can execute the following command to get a full export of all supported types with detailed information about available fields:\n\n    infra query \u003c /opt/infra/example/graphql/introspection.graphql\n\n## Listing hosts (host expansion algorithm)\n\nThe `infra host:list \u003chosts\u003e` allows you to list a set of hosts. The list you can provide is quite flexible. It's using the host expansion algorithm which is used in all places in infra where you can specify a set of hosts. Some of the command-line utilities allow you to specify a list of hosts, but also linking to hosts from `spec` sections in your resource definitions follow this algorithm. This command helps you to test this. Some examples:\n\n* `app0`: (i.e., a host name) returns an list containing only the app0 host\n* `db`: (i.e., a hostgroup name) returns an list containing all hosts in this hostgroup.\n* `app0,db0m`: (i.e., two host name) returns a list containing these specific hosts.\n* `app0,db`: (i.e., a host name and a host group name) returns a list containing hosts in the `db` host group, plus the specific `app0` host.\n* `db0m,db`: (i.e., a host group name and a host name already in that host group) returns a list containing hosts in the `db` host, not listing the host `db0m` twice.\n* `prod-east`: (i.e., a child host group name of `prod`) returns a list in the `prod-east` host group.\n* `prod-west`: (i.e., a child host group name of `prod`) returns a list in the `prod-west` host group.\n* `prod`: (i.e., a parent host group name) returns a list of all hosts linked to the `prod` host group directly, or through one of it's child host groups (`prod-east` and `prod-west`).\n\nAs you can see, you can freely mix and match host names, host group names, and take advantage of hierarchy in host groups.\n\n## Executing commands in bulk over SSH\n\nThe `infra host:exec \u003chosts\u003e \u003ccommand\u003e` command lets you perform a standard shell command one one or more hosts in bulk.\n\nThe first argument is the list of hosts you want to perform the command on (following the *host expansion algorithm* described earlier), and the second argument is the actual command. \n\n## Generating firewall rules\n\nInfra can generate complex firewall configurations based on very simple rule defintions. For example, it allows you to set up a rule that allows access to a specified port from all hosts in group X to all hosts in group Y over public or private IP addresses. It will dynamically generate the required rules based on your infra graph. You can leverage the full power of the host expansion algorithm (described earlier) to specify source and remote target host lists.\n\nRun `infra firewall:show \u003chosts\u003e` (or short-hand notation `infra f:s \u003chosts\u003e`) to let infra generate firewall rules for the specified host.\n\nRunning `infra f:s db0m` will output the generated iptables rules to the console like this:\n\n```\n#### iptables boilerplate removed for brevity ####\n\n# rule=allow-mysql hosts='db' remoteHosts='app'\n-A INPUT -d 10.0.2.1 -s 10.0.1.1 -p tcp --dport 3306 -j ACCEPT -m comment --comment \"host='db1m' remoteHost='app1' rule='allow-mysql'\"\n-A INPUT -d 10.0.2.1 -s 10.0.1.2 -p tcp --dport 3306 -j ACCEPT -m comment --comment \"host='db1m' remoteHost='app2' rule='allow-mysql'\"\n\n# rule=allow-ssh hosts='prod' remoteHosts=''\n-A INPUT -d 10.0.2.1 -s 192.168.99.99 -p tcp --dport 22 -j ACCEPT -m comment --comment \"host='db1m' rule='allow-ssh'\"\n\nCOMMIT\n```\n\nCompare these to the definitions in `example/resources/FirewallRule/`;\n\nNote how you'll find rules defined for both the \"db\" hostgroup (of which db0m is a direct member), and rules for the \"prod\" hostgroup (of which db0m is only indirectly a member through the \"prod\" child hostgroup \"prod-east\").\n\nAdditionally, you'll notice that the db0m server has 2 rules generated for the 2 app servers (app1 and app2). In case you'll add 5 more app servers, infra will automatically generate a rule for each of them (7 in total).\n\n## Installing firewall rules\n\nRun `infra firewall:install \u003chosts\u003e` to install the firewall. Infra will ssh into the host(s) and:\n\n1. backup the current iptable rules using `iptables-save`. The backup is created in /tmp, and the exact filename will be displayed on the console.\n2. Copy the firewall rules to the server\n3. Run `iptables-restore` on the rules file and output the results.\n\n## Scripts (in your language of choice)\n\nIf you'd like to build tools that leverage your infrastructure graph in your language of choice (php, node, python, ruby, bash, etc), scripts are your solution.\n\n### Creating your own script\n\nTo create your own custom script:\n\n1. Add an executable script in the `scripts/` directory of your infrastructure configuration repository\n2. Make sure to `chmod 755` the file, as only executable files are detected by infra\n3. Let your script execute GraphQL queries using the `infra query` command (pass in the query over stdin, getting the response json from stdout)\n4. Use the returned data to apply any logic you see fit: export to common config formats, call a set of APIs, generate documentation, etc.\n\n### SDKs\n\nAn SDK is not required, but may make your life as an infra script developer more convenient. Currently available SDKs:\n\n* https://github.com/linkorb/infra-sdk-php\n\nLet us know if you've created an SDKs for other languages, we'll be glad to link to it from this section.\n\n### Example scripts\n\nExamples are included in the `example/scripts` directory. (examples in more languages welcomed as PRs!)\n\n### Core scripts\n\nA set of \"core\" scripts are included in the `scripts/` directory of this repository. The core scripts are for commonly used applications that will be of use to many infra users. If you've created a reusable script that you'd like to be included in the core scripts, be sure to send a PR!\n\n\n\n## Todo\n\n* [x] Command: Add `infra script` command for listing and executing discovered scripts\n* [ ] Command: Add `infra serve` command that spins up a GraphQL server\n* [ ] Feature: Allow in-line resources in order to define resources as part of their container (i.e. DnsRecord as part of Deployment etc)\n* [ ] Feature: Allow variable expansion in resource configuration (i.e. to access parent container variables)\n* [ ] Feature: Set resource index value (incremental, from 1) and output in resource listings. Allow to fetch resource by index. This should simplify exploring using the `get` command.\n* [x] Feature: Create minimal PHP SDK for scripts to simplify querying\n* [x] Feature: script discovery\n* [ ] Feature: infra manifest file (for base variables, directory references, bulk script commands, etc)\n* [ ] Feature: support wildcards in host expansion algorithm (i.e. by using fnmatch)\n* [x] Refactor: extract sensu commands into scripts\n* [ ] Refactor: extract ansible commands into scripts\n* [ ] Refactor: Loader in external class\n* [ ] Refactor: SSH or other infra-specific methods in external services (so Infra becomes general resource container)\n* [ ] Script: Add script to generate HTML documentation from infra\n* [ ] Script: Add script to fetch host facts\n* [ ] Script: Add grafana dashboard generator \n* [x] Script: Add DNS zone publisher\n* [x] Script: Add Rundeck inventory generator\n* [ ] Feature: Add option to augment loaded resources with labels (i.e. for host facts)\n* [ ] Resource: BackupRule + script for BackupPC, rsync, or similar\n* [ ] Resource: CronJob + conf.d/infra file generator\n* [ ] Resource: DockerEngine, DockerApp. With docker-compose YAML as inline spec\n* [ ] Resource: Secret\n* [ ] Resource: DatabaseCluster or MysqlCluster to setup and monitor replication\n* [ ] Resource: Database or MysqlDatabase\n* [ ] Resource: Ingress or NginxSite or ApacheVhost for ingress configuration\n* [ ] Resource: GitRepository\n* [ ] Resource: Deployment / Ansistrano / Capistrano / DeployerPHP configurations\n* [ ] Resource: File to manage deployment configuration files\n\n## License\n\nMIT. Please refer to the [license file](LICENSE) for details.\n\n## Brought to you by the LinkORB Engineering team\n\n\u003cimg src=\"http://www.linkorb.com/d/meta/tier1/images/linkorbengineering-logo.png\" width=\"200px\" /\u003e\u003cbr /\u003e\nCheck out our other projects at [linkorb.com/engineering](http://www.linkorb.com/engineering).\n\nBtw, we're hiring!\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinkorb%2Finfra","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flinkorb%2Finfra","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinkorb%2Finfra/lists"}