{"id":20314482,"url":"https://github.com/release-engineering/iib","last_synced_at":"2025-04-11T17:17:35.046Z","repository":{"id":37913596,"uuid":"238475032","full_name":"release-engineering/iib","owner":"release-engineering","description":"A REST API to manage operator index container images (and some bundle images)","archived":false,"fork":false,"pushed_at":"2025-04-09T22:41:54.000Z","size":11246,"stargazers_count":16,"open_issues_count":15,"forks_count":24,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-09T23:30:41.911Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://iib.readthedocs.io/en/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/release-engineering.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-05T14:53:02.000Z","updated_at":"2025-04-09T10:15:23.000Z","dependencies_parsed_at":"2023-09-26T22:39:24.324Z","dependency_job_id":"a8b8aebc-94df-485b-baa5-1778cafc3f7b","html_url":"https://github.com/release-engineering/iib","commit_stats":null,"previous_names":[],"tags_count":110,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/release-engineering%2Fiib","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/release-engineering%2Fiib/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/release-engineering%2Fiib/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/release-engineering%2Fiib/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/release-engineering","download_url":"https://codeload.github.com/release-engineering/iib/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248447600,"owners_count":21105140,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T18:15:33.487Z","updated_at":"2025-04-11T17:17:35.023Z","avatar_url":"https://github.com/release-engineering.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# IIB Image Builder (IIB) Service\n\nA REST API to manage operator index container images (and some bundle images).\n\nIIB is a recursive acronym.  IIB stands for IIB Image Builder.\n\nNote: IIB was originally called \"Index Image Build Service\" but the name has been changed since\n  its scope widened.\n\nPodman 1.9.2+ is required by IIB.\n\n## External Documentation\n\n* [API Documentation](http://release-engineering.github.io/iib)\n* [General Documentation](https://iib.readthedocs.io/en/latest/)\n* [Python Module Documentation](https://iib.readthedocs.io/en/latest/module_documentation/index.html)\n\n# Development\n## Coding Standards\n\nThe codebase conforms to the style enforced by `flake8` with the following exceptions:\n\n* The maximum line length allowed is 100 characters instead of 80 characters\n\nIn addition to `flake8`, docstrings are also enforced by the plugin `flake8-docstrings` with\nthe following exemptions:\n\n* D100: Missing docstring in public module\n* D104: Missing docstring in public package\n* D105: Missing docstring in magic method\n\nThe format of the docstrings should be in the\n[reStructuredText](https://docs.python-guide.org/writing/documentation/#restructuredtext-ref) style\nsuch as:\n\n```python\n\"\"\"\nGet the IIB build request from the REST API.\n\n:param int request_id: the ID of the IIB request\n:return: the request\n:rtype: dict\n:raises IIBError: if the HTTP request fails\n\"\"\"\n```\n\nAdditionally, `black` is used to enforce other coding standards with the following exceptions:\n\n* Single quotes are used instead of double quotes\n\nTo verify that your code meets these standards, you may run `tox -e black,flake8`.\n\n## Running the Unit Tests\n\nThe testing environment is managed by [tox](https://tox.readthedocs.io/en/latest/). Simply run\n`tox` and all the linting and unit tests will run.\n\nIf you'd like to run a specific unit test, you can do the following:\n\n```bash\ntox -e py312 tests/test_web/test_api_v1.py::test_add_bundle_invalid_param\n```\n\n## Development Environment\n\n[docker-compose](https://docs.docker.com/compose/) is the supported mechanism for setting up a\ndevelopment environment. This will automatically run the following containers:\n\n* **iib-api** - the IIB REST API. This is accessible at [http://localhost:8080](http://localhost:8080).\n* **iib-worker** - the IIB Celery worker.\n* **rabbitmq** - the RabbitMQ instance for communicating between the API and the worker. The\n  management UI is accessible at [http://localhost:8081](http://localhost:8081). The username is\n  `iib` and the password is `iib`.\n* **db** - the Postgresql database used by the IIB REST API.\n* **registry** - the Docker Registry where the worker pushes its build index images to. This is\n  accessible at docker://localhost:8443.\n* **message-broker** - the Apache ActiveMQ instance for publishing messages for external consumers.\n  The web console is accessible at [http://localhost:8161/admin](http://localhost:8161/admin). The\n  username is `admin` and the password is `admin`. The docker-compose environment is configured for\n  IIB to publish AMQP 1.0 messages to the Apache ActiveMQ instance at the destinations\n  `topic://VirtualTopic.eng.iib.batch.state` and `topic://VirtualTopic.eng.iib.build.state`.\n\nIt's recommended to use the wrapper targets in the `Makefile` for pre-requisites. Simply run `make`\nto view available targets.\n\nThe Flask application will automatically reload if there is a change in the codebase. If invalid\nsyntax is added in the code, the `iib-api` container may shutdown. The Celery worker will\nautomatically restart if there is a change under the `iib/workers` directory.\n\nTo run a built index image from the development registry, you can perform the following:\n\n```bash\npodman login --tls-verify=false -u iib -p iibpassword localhost:8443\npodman pull --tls-verify=false localhost:8443/iib-build:1\n```\n\nIf you are using Docker (a modern version is required), you can perform the following:\n\n```bash\nsudo docker login -u iib -p iibpassword localhost:8443\nsudo docker run localhost:8443/iib-build:1\n```\n\nIf your development environment requires accessing a private container registry, please read\nthe section titled Registry Authentication.\n\nYou may also run the development environment with\n[podman-compose](https://github.com/containers/podman-compose). Use the script from the `devel`\nbranch as it has various fixes and new features required to run IIB. Set the environment variable\n`IIB_COMPOSE_ENGINE` to the path of the `podman-compose` script before running the `make` commands.\n\nSetting the `IIB_COMPOSE_ENGINE` variable will update compose targets to point to a similarly named\nfile inside the `compose-files` directory. Any changes made to the compose files should be submitted to\nall files in the directory.\n\n## Dependency Management\n\nTo manage dependencies, this project uses [pip-tools](https://github.com/jazzband/pip-tools) so that\nthe production dependencies are pinned and the hashes of the dependencies are verified during\ninstallation.\n\nThe unpinned dependencies are recorded in `setup.py`, and to generate the `requirements.txt` file,\nrun `pip-compile --generate-hashes --output-file=requirements.txt`. This is only necessary when\nadding a new package. To upgrade a package, use the `-P` argument of the `pip-compile` command.\n\nTo update `requirements-test.txt`, run\n`pip-compile --generate-hashes requirements-test.in -o requirements-test.txt`.\n\nBoth `requirements.txt` and `requirements-test.txt` can be updated using the tox command\n\n```bash\n$ tox -e pip-compile\n```\n\nYou can also use this to upgrade specific packages and specific package versions\n\n```bash\n$ tox -e pip-compile -- --upgrade-package \u003cpackage-name\u003e\u003c==package-version\u003e\n```\n\nWhen installing the dependencies in a production environment, run\n`pip install --require-hashes -r requirements.txt`. Alternatively, you may use\n`pip-sync requirements.txt`, which will make sure your virtualenv only has the packages listed in\n`requirements.txt`.\n\n\nTo ensure the pinned dependencies are not vulnerable, this project uses\n[safety](https://github.com/pyupio/safety), which runs on every pull-request.\n\n## Database Migrations\n\nWhen the models are modified, the database schema also needs to be updated which includes leveraging\n`alembic` to generate a database migration. In order to simplify this process, a `tox` command can be\nused\n\n```bash\n$ tox -e migrate-db \u003csimple message describing the change.\u003e\n```\n\nBefore running this command, however, you should `SQLALCHEMY_DATABASE_URI` to point to `'sqlite:///'`\nin `iib/web/config.py`. This parameter also needs to be added to the `Config` class specification.\n\n**NOTE:** This tox command was written as a convenience function that might not work in all situations.\nIf alembic cannot auto-create the version file for the migration with `migrate`, you can just create it\nand then manually populate the steps with\n\n```bash\n$ flask db revision -m \u003csimple message describing the change.\u003e\n```\n\n# Configuring IIB deployments\n## Registry Authentication\n\nIIB does not handle authentication with container registries directly. If authentication is needed,\nconfigure the `~/.docker/config.json.template` file for the user running the IIB worker. This path\ncan be customized with the `iib_docker_config_template` configuration.\n\nDuring development, you may choose to add a volume entry of\n`- /root/.docker/config.json:/root/.docker/config.json.template:ro,z` on the workers in\n`docker-compose.yml` so that your host's root user's Docker configuration with authentication is\nused by the workers. This is only needed if you are working with private images.\n\n## Configuring the REST API\n\nTo configure the IIB REST API, create a Python file at `/etc/iib/settings.py`. Any variables set in\nthis configuration file will be applied to the Celery worker when running in production mode\n(default).\n\nThe custom configuration options for the REST API are listed below:\n\n* `IIB_ADDITIONAL_LOGGERS` - a list of Python loggers that should have the same log level that is\n  set for `IIB_LOG_LEVEL`. This defaults to `[]`.\n* `IIB_AWS_S3_BUCKET_NAME` - the name of the AWS S3 bucket used to fetch artifact files like logs\n  and related_bundles from if specified. This defaults to `None` which means IIB will try to get\n  the files locally if `IIB_REQUEST_LOGS_DIR` and `IIB_REQUEST_RELATED_BUNDLES_DIR` are configured.\n  For the REST API, if `IIB_AWS_S3_BUCKET_NAME` is specified, you cannot specify `IIB_REQUEST_LOGS_DIR`\n  and `IIB_REQUEST_RELATED_BUNDLES_DIR`.\n* `IIB_BINARY_IMAGE_CONFIG` - the mapping, `dict(\u003cstr\u003e: dict(\u003cstr\u003e:\u003cstr\u003e))`, of distribution scope\n  to another dictionary mapping ocp_version label to a binary image pull specification.\n  This is useful in setting up customized binary image for different index image images thus\n  reducing complexity for the end user. This defaults to `{}`.\n* `IIB_GRAPH_MODE_INDEX_ALLOW_LIST` - the list of index image pull specs on which using the\n  `graph_update_mode` parameter while submitting an IIB request is permitted. This defaults to `[]`\n  . Please check out the [API Documentation](http://release-engineering.github.io/iib) for more\n  information on how to use \"graph_update_mode\" for Add requests.\n* `IIB_GRAPH_MODE_OPTIONS` - the list of valid options for the `graph_update_mode` parameter in\n  the Add API endpoint. It defaults to `['replaces', 'semver', 'semver-skippatch']` i.e. the list\n  of modes supported by OPM.\n* `IIB_GREENWAVE_CONFIG` - the mapping, `dict(\u003cstr\u003e: dict(\u003cstr\u003e:\u003cstr\u003e))`, of celery task queues to\n  another dictionary of [Greenwave](https://docs.pagure.org/greenwave/) query parameters to their\n  values. This is useful in setting up customized gating for each queue. This defaults to `{}`. Use\n  the task queue name as `None` to configure Greenwave config for the default Celery task queue.\n* `IIB_LOG_FORMAT` - the format of the logs. This defaults to\n  `%(asctime)s %(name)s %(levelname)s %(module)s.%(funcName)s %(message)s`.\n* `IIB_LOG_LEVEL` - the Python log level of the REST API (Flask). This defaults to `INFO`.\n* `IIB_MAX_PER_PAGE` - the maximum number of build requests that can be shown on a single page.\n  This defaults to `20`.\n* `IIB_REQUEST_DATA_DAYS_TO_LIVE` - the amount of days after which per request temmporary data is\n  considered to be expired and may be removed. This defaults to `3`.\n* `IIB_REQUEST_LOGS_DIR` - the directory to load the request specific log files. If `None`, per\n  request log files information will not appear in the API response. This defaults to `None`.\n* `IIB_REQUEST_RELATED_BUNDLES_DIR` - the directory to load the request specific related\n  bundles files. If `None`, per request related bundles files information will not appear in\n  the API response. This defaults to `None`.\n* `IIB_RECURSIVE_REQUEST_RELATED_BUNDLES_DIR` - the directory to load the recursive-related-bundles\n  request specific recursive related bundles files. This is a required config variable if\n  `IIB_AWS_S3_BUCKET_NAME` is not specified.\n* `IIB_USER_TO_QUEUE` - the mapping, `dict(\u003cstr\u003e: \u003cstr\u003e)` or `dict(\u003cstr\u003e: dict(\u003cstr\u003e: \u003cstr\u003e))`, \n  of usernames to celery task queues.\n  This is useful in isolating the workload from certain users. Some celery tasks must execute\n  serially, while others can execute in parallel. Add the prefix `SERIAL:` or `PARALLEL:` to the\n  **username** key in this mapping to create queues based on serial vs parallel tasks. The default\n  queue is used for tasks from users not found in the mapping. This defaults to `{}`.\n* `IIB_WORKER_USERNAMES` - the list of case-sensitve Kerberos principals that are allowed to update\n  build requests using the PATCH API endpoint. This defaults to `[]`.\n* `LOGIN_DISABLED` - determines if authentication is required. This defaults to `False`\n  (i.e. authentication is required).\n* `SQLALCHEMY_DATABASE_URI` - the database URI of the database the REST API connects to. See the\n  [Flask-SQLAlchemy configuration](https://flask-sqlalchemy.palletsprojects.com/en/2.x/config/#configuration-keys)\n  documentation.\n\nThe custom configuration options for AMQP 1.0 messaging are listed below:\n\n* `IIB_MESSAGING_BATCH_STATE_DESTINATION` - the AMQP 1.0 destination to send the batch state change\n  messages. If this is not set, IIB will not send these types of messages. If this is set,\n  `IIB_MESSAGING_URLS` must also be set.\n* `IIB_MESSAGING_BUILD_STATE_DESTINATION` - the AMQP 1.0 destination to send the build request state\n  change messages. If this is not set, IIB will not send these types of messages. If this is set,\n  `IIB_MESSAGING_URLS` must also be set.\n* `IIB_MESSAGING_CA` - the path to a file with the certificate authority that signed the certificate\n  of the AMQP 1.0 message broker. This defaults to `/etc/pki/tls/certs/ca-bundle.crt`.\n* `IIB_MESSAGING_CERT` - the path to the identity certificate used for authentication with the\n  AMQP 1.0 message broker. This defaults to `/etc/iib/messaging.crt`.\n* `IIB_MESSAGING_DURABLE` - determines if the messages are durable and cannot be lost due to an\n  unexpected termination or restart by the AMQP 1.0 broker. If the broker is not capable of\n  guaranteeing this, it may not accept the message. In that case, set this configuration option to\n  `False`. This defaults to `True`.\n* `IIB_MESSAGING_KEY` - the path to the private key of the identity certificate used for\n  authentication with the AMQP 1.0 message broker. This defaults to `/etc/iib/messaging.key`.\n* `IIB_MESSAGING_TIMEOUT` - the number of seconds before a messaging operation times out.\n  Examples of messaging operations include connecting to the broker and sending a message to the\n  broker. In this case, if the timeout is set to `30`, then it could take a maximum of 60 seconds\n  before the operation times out. This is because it can take up to 30 seconds to connect to the\n  broker and also up to 30 seconds for the message to be sent. This defaults to `30`.\n* `IIB_MESSAGING_URLS` - a list of AMQP(S) URLs to use when connecting to the AMQP 1.0 broker. This\n  must be set if messaging is enabled.\n\nIf you wish to configure AWS S3 bucket for storing artifact files, the following **environment variables**\nmust be set along with `IIB_AWS_S3_BUCKET_NAME` config variable:\n\n* `AWS_ACCESS_KEY_ID` - the secret access key ID used to access the AWS S3 bucket.\n* `AWS_SECRET_ACCESS_KEY` - the secret access key used to access the AWS S3 bucket.\n* `AWS_DEFAULT_REGION` - the default region of the AWS S3 bucket\n\nMore info on these environment variables can be found in the [AWS User Guide](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html)\n\n### Opentelemetry Environment Variable\n\nTo integrate IIB with Opentelemetery tracing, we will need the following parameters as\nenvironment variables.\n\n* `IIB_OTEL_TRACING` - the boolean value which indicates if OpenTelemetry tracing is enabled in IIB.\n  This defaults to `False`. If this is set to `True`, `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_SERVICE_NAME`\n  must be set.\n* `OTEL_EXPORTER_OTLP_ENDPOINT` - The endpoint for the OpenTelemetry exporter.\n* `OTEL_SERVICE_NAME` - \"iib-api\" / \"iib-worker\"\n\nFor more info on these environment variables, please refer to [Opentelemetry Guide](https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/)\n\nDev-env supports Opentelemetry tracing by default. It uses Jaeger for collecting and exporting traces.\nWEB UI is accessible on port 16686 - `http://localhost:16686/`\n\n## Configuring the Worker(s)\n\nTo configure an IIB Celery worker, create a Python file at `/etc/iib/celery.py`. The location\ncan be overridden with the `IIB_CELERY_CONFIG` environment variable. This is useful if the worker is\nrunning on the same host as another worker or the REST API.\n\nAny variables set in this configuration file will be applied to the Celery worker when running in\nproduction mode (default).\n\nThe custom configuration options for the Celery workers are listed below:\n\n* `broker_url` - the AMQP(S) URL to connect to RabbitMQ. See the\n  [broker_url](https://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-broker_url)\n  configuration documentation.\n* `iib_api_timeout` - the timeout in seconds for HTTP requests to the REST API. This defaults to\n  `60` seconds.\n* `iib_api_url` - the URL to the IIB REST API (e.g. `https://iib.domain.local/api/v1/`).\n* `iib_aws_s3_bucket_name` - the name of the AWS S3 bucket used to store artifact files like logs\n  and related_bundles if specified. `iib_request_logs_dir` and `iib_request_related_bundles_dir`\n  are required when this variable is specified. This defaults to `None` which means IIB will try to store\n  the files locally if `iib_request_logs_dir` and `iib_request_related_bundles_dir` are configured.\n* `iib_docker_config_template` - the path to the Docker config.json file for IIB to use as a\n  template. IIB will symlink this file to `~/.docker/config.json` at the beginning of every request.\n  Additionally, it will use this file as a base and set the `overwrite_from_index_token` for the\n  registry of the `from_index` container image when applicable. IIB will never directly modify this\n  file though. This defaults to `~/.docker/config.json.template`.\n*  `iib_dogpile_backend` - the configuration for the dogpile.cache backend. The default value is\n   `'dogpile.cache.null'`. In case you want to enable caching, set this to `'dogpile.cache.memcached'`.\n*  `iib_dogpile_expiration_time` - the number of seconds after which the cached item is expired.\n*   `iib_dogpile_arguments` - additional arguments for the dogpile backend.\n* `iib_greenwave_url` - the URL to the Greenwave REST API if gating is desired\n  (e.g. `https://greenwave.domain.local/api/v1.0/`). This defaults to `None`.\n* `iib_grpc_init_wait_time` - time to wait for the index image service to be initialized. This\n  defaults to `3` seconds.\n* `iib_grpc_max_port_tries` - maximum ports to try when initializing the index image service.\n  This defaults to `100` tries.\n* `iib_grpc_start_port` - first port to try when starting the service (subsequent are increments).\n  This defaults to `50051`.\n* `iib_grpc_max_tries` - maximum number of times to try to start the index image service\n  before giving up. This defaults to `5` attempts.\n* `iib_index_image_output_registry` - if set, that value will replace the value from `iib_registry`\n  in the output `index_image` pull specification. This is useful if you'd like users of IIB to\n  pull from a proxy to a registry instead of the registry directly.\n* `iib_image_push_template` - the Python string template of the push destination for the resulting\n  manifest list. The available variables are `registry` and `request_id`. The default value is\n  `{registry}/iib-build:{request_id}`.\n* `iib_log_level` - the Python log level for `iib.workers` logger. This defaults to `INFO`.\n* `iib_max_recursive_related_bundles` - the maximum number of recursive related bundles IIB will\n  recurse through. This is to avoid DOS attacks.\n* `iib_no_ocp_label_allow_list` - list of index images to which we can add bundles \n  without \"com.redhat.openshift.versions\" label\n* `iib_organization_customizations` - this is used to customize aspects of the bundle being\n  regenerated. The format is a dictionary where each key is an organization that requires\n  customizations. Each value is a list of dictionaries with the ``type`` key set to one of the\n  optional values `csv_annotations`, `package_name_suffix`,\n  and `registry_replacements`. The order of the dictionaries in the list will determine the order\n  of customizations applied to the bundle.\n\n  * The `csv_annotations` customization type is a dictionary where the key `annotations` value is\n    a dictionary where each key is the annotation to set on the ClusterServiceVersion files, and\n    the value is a Python template string of the value to be set. IIB only substitutes\n    `{package_name}` in the template string.\n  * The `package_name_suffix` customization type is a dictionary where the key `suffix` value is\n    a string of a suffix to add to the package name of the operator.\n  * The `registry_replacements` customization type is a dictionary where the key `replacements`\n    value is a dictionary where the keys are the old registries to replace and the values\n    are the registries to replace the old registries with. This replaces the registry in all\n    the ClusterServiceVersion files.\n  * The `image_name_from_labels` customization type is a dictionary where the key `template`\n    value is a string which specifies a combination of label names in curly braces which will be\n    substituted with the actual label values from the bundle image.\n  * The `enclose_repo` customization type is a dictionary where the key `enclosure_glue` value\n    is a string which specifies the glue to replace ``/`` (forward slashes) in the pull spec\n    name and repo. The key `namespace` value is also a string which specifies the new namespace\n    for pull specs of the ClusterServiceVersion files.\n  * The `related_bundles` customization type is a dictionary with no additional arguments and\n    it essentially finds all the bundle image pull specifications in the CSV files of the operator\n    bundle image in question. If this customization is not specified in the config for an\n    organization, the `related_bundles` endpoint won't work for regenerate-bundle requests of that\n    organization. If no organization is specified, IIB will try to find `related_bundles` for all\n    regenerate-bundle requests.\n  * The `resolve_image_pullspecs` customization type is a dictionary with no additional arguments\n    and it basically pins all the pull specs in the CSV files of the operator bundle image in\n    question to their digests. If this customization is not specified in the config for an\n    organization, the pinning will not be done. If no organization is specified, IIB will try and\n    pin the pull specs in the CSV files to their digests.\n  * The `perform_bundle_replacements` customization type is a dictionary with no additional\n    arguments. It enables bundle replacements to be passed to the bundle regeneration APIs with\n    the `bundle_replacements` parameter. If the customization type is not set, and bundle\n    replacements specified will be ignored.\n\n  Here is an example that ties this all together:\n\n  ```python\n  iib_organization_customizations = {\n        'company-marketplace': [\n            {'type': 'resolve_image_pullspecs'},\n            {\n                'type': 'csv_annotations',\n                'annotations': {\n                    'marketplace.company.io/remote-workflow': (\n                        'https://marketplace.company.com/en-us/operators/{package_name}/pricing'\n                    ),\n                    'marketplace.company.io/support-workflow': (\n                        'https://marketplace.company.com/en-us/operators/{package_name}/support'\n                    ),\n                },\n            },\n            {'type': 'package_name_suffix', 'suffix': '-cmp'},\n            {\n                'type': 'registry_replacements',\n                'replacements': {\n                    'registry.access.company.com': 'registry.marketplace.company.com/cm',\n                },\n            },\n            {'type': 'image_name_from_labels', 'template': '{name}-{version}-final'},\n            {'type': 'enclose_repo', 'enclosure_glue': '----', 'namespace': \"company-pending\"},\n        ]\n    }\n  ```\n\n* `iib_related_image_registry_replacement` - the mapping `dict(\u003cstr\u003e: dict(\u003cstr\u003e: \u003cstr\u003e))` to specify if the registry of the related image needs to be changed to inspect the related images. The mapping denotes the username and the registries that need to be replaced to inspect the related images.\n* `iib_request_related_bundles_dir` - the directory to write the request specific related bundles\n  file. If `None`, per request related bundles files are not created. This defaults to `None`.\n* `iib_request_logs_dir` - the directory to write the request specific log files. If `None`, per\n  request log files are not created. This defaults to `None`.\n* `iib_request_recursive_related_bundles_dir` - the directory to load the recursive-related-bundles\n  request specific recursive related bundles files. This is a required config variable if\n  `iib_aws_s3_bucket_name` is not specified.\n* `iib_request_logs_format` - the format for the log messages of the request specific log files.\n  This defaults to `%(asctime)s %(name)s %(levelname)s %(module)s.%(funcName)s %(message)s`.\n* `iib_request_logs_level` - the log level for the request specific log files. This defaults to\n  `DEBUG`.\n* `iib_registry` - the container registry to push images to (e.g. `quay.io`).\n* `iib_sac_queues` - list of names of celery queues which should be created as single-active-consumer \n* `iib_skopeo_timeout` - the command timeout for skopeo commands run by IIB. This defaults to\n  `30s` (30 seconds).\n* `iib_total_attempts` - the total number of attempts to make at trying a function relating to the\n  container registry before erroring out. This defaults to `5`. It's also used as the max number of attempts to buildah when receiving HTTP 50X errors.\n* `iib_retry_delay` - the delay in seconds between retry attempts. It's just used for buildah when receiving HTTP 50X errors. This defaults to `5`.\n* `iib_retry_jitter` - the extra seconds to be added on delay between retry attempts. It's just used for buildah when receiving HTTP 50X errors. This defaults to `5`.\n* `iib_retry_multiplier` - the constant in the `2^x * multiplier` formula, where x stands for attempt number. Formula is used to calculate the\n  seconds to be added on delay between retry attempts. It's just used for buildah when receiving HTTP 50X errors. This defaults to `5`.\n* `iib_supported_archs` - the architectures supported by IIB. IIB can build index images for these\n  architectures. The dictionary has mapping of arch aliases with formal names like\n  `{\"arm64\": \"aarch64\"}`\n* `iib_default_opm` - the default opm version command to use. This defaults to\n    `opm`\n* `iib_ocp_opm_mapping` - the dictionary mapping of OCP version to OPM version\nindicating the OPM version to be used for the corresponding OCP version like\n`{\"v4.15\": \"opm-v1.28.0\"}`\n\n\nIf you wish to configure AWS S3 bucket for storing artifact files, the following **environment variables**\nmust be set along with `iib_aws_s3_bucket_name` config variable:\n\n* `AWS_ACCESS_KEY_ID` - the secret access key ID used to access the AWS S3 bucket.\n* `AWS_SECRET_ACCESS_KEY` - the secret access key used to access the AWS S3 bucket.\n* `AWS_DEFAULT_REGION` - the default region of the AWS S3 bucket\n\nMore info on these environment variables can be found in the [AWS User Guide](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html)\n\n### Opentelemetry Environment Variable\n\nTo integrate IIB with Opentelemetery tracing, we will need the following parameters\nas environment variables.\n\n* `IIB_OTEL_TRACING` - the boolean value which indicates if OpenTelemetry tracing is enabled in IIB.\n  This defaults to `False`. If this is set to `True`, `OTEL_EXPORTER_OTLP_ENDPOINT` and `OTEL_SERVICE_NAME`\n  must be set.\n* `OTEL_EXPORTER_OTLP_ENDPOINT` - The endpoint for the OpenTelemetry exporter.\n* `OTEL_SERVICE_NAME` - \"iib-workers\"\n\nFor more info on these environment variables, please refer to [Opentelemetry Guide](https://opentelemetry.io/docs/concepts/sdk-configuration/otlp-exporter-configuration/)\n\n# Additional information on specific IIB functionality\n## Regenerating Bundle Images\n\nIn addition to building operator index images, IIB can also be used to regenerate operator bundle\nimages. This is useful for applying modifications to the manifests embedded in the bundle image.\nIIB uses the [operator-manifest](https://github.com/containerbuildsystem/operator-manifest) library\nto assist in these modifications.\n\nCurrently, IIB will not perform any modifications on a ClusterServiceVersion file if\n[spec.relatedImages](https://access.redhat.com/documentation/en-us/openshift_container_platform/4.3/html-single/operators/index#olm-enabling-operator-for-restricted-network_osdk-generating-csvs)\nis set.\n\nIf it's not set, IIB will pin any container image pull specification and set\n[spec.relatedImages](https://access.redhat.com/documentation/en-us/openshift_container_platform/4.3/html-single/operators/index#olm-enabling-operator-for-restricted-network_osdk-generating-csvs).\nSee the different\n[pull specifications](https://github.com/containerbuildsystem/operator-manifest#pull-specifications)\nto which this process applies to. There are also a variety of customizations that can be made to\nthe bundle. See the `iib_organization_customizations` configuration option for more details.\n\nBundle images regenerated by IIB will have the label `com.redhat.iib.pinned` set to `'true'`.\nIf the bundle image already has this label set to this value, pinning is skipped. Any other\nmodifications, such as registry replacement, will still be applied.\n\n## Messaging\n\nIIB has support to send messages to an AMQP 1.0 broker. If configured to do so, IIB will send\nmessages when a build request state changes and when a batch state changes. Please note that if a\nmessage can't be sent due to an infrastructure issue, the build request will continue as it is not\nconsidered a fatal error.\n\nThe build request state change message body is the JSON representation of the build request in\nthe non-verbose format like in the `/builds` API endpoint. The message has the following keys set in\nthe application properties: `batch`, `id`, `state`, and `user`.\n\nThe batch state change message body is a JSON object with the following keys: `annotations`,\n`batch`, `requests`, `state`, and `user`. The `requests` value is an array of JSON objects with the\nkeys `id`, `organization`, and `request_type`. The message has the following keys set in the\napplication properties: `batch`, `state`, and `user`.\n\n## Gating Bundle Images\n\nIn addition to building operator index images, IIB can also gate your bundle images before adding\nthem to the index image. If a Greenwave configuration is setup for your queue, IIB will query\nGreenwave to check if your bundle image builds have passed the tests in the Greenwave policy you\nhave defined. The IIB request submitted to that queue will succeed only if the policy is satisfied.\n\n# Documentation\nThis package has documentaiton on [Read the Docs](https://iib.readthedocs.io)\n\n## Build the Docs\n\nTo build and serve the docs, run the following commands:\n\n```bash\ntox -e docs\ngoogle-chrome .tox/docs_out/index.html\n```\n\n## Expanding the Docs\n\nTo document a new Python module, find the `rst` file of the corresponding Python package that\ncontains the module. Once found, add a section under \"Submodules\" in alphabetical order such as:\n\n```rst\niib.workers.tasks.build module\n------------------------------\n\n.. automodule:: iib.workers.tasks.build\n   :ignore-module-all:\n   :members:\n   :private-members:\n   :show-inheritance:\n```\n\nSome of the options include:\n\n* `ignore-module-all` - include all members regardless of the definition of `__all__`.\n* `members` - automatically document the members in that Python module.\n* `private-members` - include private functions and methods.\n* `show-inheritance` - show the class inheritance.\n\n# Opentelemetry Instrumentation\n\nIIB is now implemented with Opentelemetry python instrumentation.\nThis is done in order to collect detailed information about the behavior and performance of IIB.\nBy using a combination of automatic and manual instrumentation, it is possible to capture all relevant information about the application's behavior.\n\n## Environment setup\nThe two most important packages to be installed are the `opentelemetry-api` and `opentelemetry-sdk`.\nPlease see the ``requirements.txt`` file for a more detailed list..\n\n```bash\npip install virtualenv\nvirtualenv venv\nsource venv/bin/activate\npip install -r requirements.txt\n```\n\n## Traces\nThe application is instrumented to collect traces using OpenTelemetry.\nThe traces include information about the duration of each request, as well as any errors that may have occurred.\n\n## Instrumentation\n\nThe manual instrumentation decorator is added in `iib.common.tracing`.\nTo add new instrumentation, ensure to get the *trace_provider* from the `iib.common.tracing.TracingWrapper` class\nTo fetch the `trace_provider`:\n\n```python\ntracerWrapper = TracingWrapper()\ntracerWrapper.provider\n```\n\nTo add manual instrumentation to a function:\n```python\n@instrument_tracing(span_nam=\"test_instrumentation\")\ndef test_instrumentation():\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frelease-engineering%2Fiib","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frelease-engineering%2Fiib","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frelease-engineering%2Fiib/lists"}