{"id":18825462,"url":"https://github.com/grycap/toscarizer","last_synced_at":"2026-05-01T18:32:03.887Z","repository":{"id":204451665,"uuid":"711880228","full_name":"grycap/toscarizer","owner":"grycap","description":"TOSCARIZER: Generate TOSCA files for the AI-SPRINT project","archived":false,"fork":false,"pushed_at":"2023-12-11T10:05:07.000Z","size":64863,"stargazers_count":2,"open_issues_count":2,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-02-12T07:10:24.321Z","etag":null,"topics":["ai-sprint","docker","infrastructure-manager","tosca"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/grycap.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-10-30T11:08:52.000Z","updated_at":"2023-12-13T09:50:05.000Z","dependencies_parsed_at":"2023-12-07T08:40:42.462Z","dependency_job_id":null,"html_url":"https://github.com/grycap/toscarizer","commit_stats":null,"previous_names":["grycap/toscarizer"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/grycap/toscarizer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grycap%2Ftoscarizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grycap%2Ftoscarizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grycap%2Ftoscarizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grycap%2Ftoscarizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/grycap","download_url":"https://codeload.github.com/grycap/toscarizer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grycap%2Ftoscarizer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32508900,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-30T13:12:12.517Z","status":"online","status_checked_at":"2026-05-01T02:00:05.856Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-sprint","docker","infrastructure-manager","tosca"],"created_at":"2024-11-08T00:59:37.021Z","updated_at":"2026-05-01T18:32:03.859Z","avatar_url":"https://github.com/grycap.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TOSCARIZER\n\n![TOSCARIZER-logo](toscarizer.png)\n\n## Quick-Start\n\n### Step1: Install TOSCARIZER\n\n```sh\ngit clone https://github.com/grycap/toscarizer\ncd toscarizer\npython3 -m pip install . \n```\n\n#### Docker image\n\nA docker image is available with TOSCARIZER and AI-SPRINT Desing tools:\n`registry.gitlab.polimi.it/ai-sprint/toscarizer/toscarizer`\n\nYou can use it setting the path of you local application directory as in\nthis example:\n\n```sh\ndocker run --rm -v local_app_path:/app \\\n    -ti registry.gitlab.polimi.it/ai-sprint/toscarizer/toscarizer \\\n    toscarizer tosca --application_dir /app --base\n```\n\nIn the case of the `docker` operation (Step3) docker is used to build and\npush the application images so it must be enabled inside the docker container\nto work, as in this example:\n\n```sh\ndocker run --rm -v local_app_path:/app \\\n    -v /var/run/docker.sock:/var/run/docker.sock  \\\n    -v $HOME/.docker/config.json:/root/.docker/config.json \\\n    -ti registry.gitlab.polimi.it/ai-sprint/toscarizer/toscarizer \\\n    toscarizer docker --application_dir /app\n    ...\n```\n\n### Step2: Try --help\n\n```sh\ntoscarizer --help\n```\n\n### Step3: Build and push the container images needed by each component of the application\n\nThis step requires `docker` installed. See how to install it [here](https://docs.docker.com/engine/install/).\nIn case that any of the images will run on an ARM platform, support multiarch\ndocker builds must be installed also. See how to configure it [here](https://docs.docker.com/desktop/multi-arch/).\n\nFirst you need to login to the container registry that you will use in the `docker` operation:\n\n```sh\ndocker login registry.gitlab.polimi.it\n```\n\nAlso in case that any of the steps uses NativeCloudFunctions (AWS Lambda). You need to also\nto login to the ECR repository, using the aws-cli tool (See how to install it\n[here](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html)):\n\n```sh\naws ecr get-login-password --region [region] | docker login --username AWS --password-stdin\n XXXXXXXXXXXX.dkr.ecr.[region].amazonaws.com\n```\n\n```sh\ntoscarizer docker --registry registry.gitlab.polimi.it \\\n                  --registry_folder /ai-sprint \\\n                  --application_dir app\n```\n\nOptionally the `--base_image` parameter can be set to define a different base image\nfor the generated images. The default value is:\n`registry.gitlab.polimi.it/ai-sprint/toscarizer/ai-sprint-base`.\n\nFurthermore in case that any of the steps uses NativeCloudFunctions (AWS Lambda). You need to also\nset an existing ECR repository URL:\n\n```sh\ntoscarizer docker --registry registry.gitlab.polimi.it \\\n                  --registry_folder /ai-sprint \\\n                  --application_dir app\n                  --ecr XXXXXXXXXXXX.dkr.ecr.[region].amazonaws.com/repo_name\n```\n\n### Step4: Generate the corresponding TOSCA YAML files\n\nThe `tosca` command uses the files ``templates\\*.yaml`` to generate the TOSCA\nfile for the OSCAR clusters.\n\nGenerate the TOSCA IM input files for the base case:\n\n```sh\ntoscarizer tosca --application_dir app --base\n```\n\nGenerate the TOSCA IM input files for the optimal case:\n\n```sh\ntoscarizer tosca --application_dir app --optimal\n```\n\nIn all cases if some resources are of type ``PhysicalAlreadyProvisioned`` or\n``NativeCloudFunction`` an extra file with the needed information to connect\nwith this resources (IP and SSH auth data, MinIO service info, InfluxDB info\nAWS S3 bucket, info) is needed. It is expected in the app common_config\ndirectory with name ``physical_nodes.yaml``. See the following example:\nThe first layer corresponds with a ``PhysicalAlreadyProvisioned`` cluster, thatç\nis not already installed with OSCAR, that will be accessed via SSH to install\nthe required software. The second one corresponds with a ``PhysicalAlreadyProvisioned``\ncluster where oscar has been already installed so we can access it directly.\nFinally the last layer corresponds with an ``NativeCloudFunction`` function,\nwhere the user must specify the AWS S3 bucket that will be used to trigger\nthe function.\n\n```yaml\nComputationalLayers: \n   computationalLayer1:\n      number: 1\n      Resources: \n         resource1:\n            name: RaspPi\n            fe_node:\n               public_ip: 8.8.8.1\n               private_ip: 10.0.1.1\n               ssh_user: ubuntu\n               ssh_key: |\n                  -----BEGIN RSA PRIVATE KEY-----\n                  ssh11\n                  -----END RSA PRIVATE KEY-----\n            wns:\n               - private_ip: 10.0.1.1\n                 ssh_user: ubuntu\n                 ssh_key: |\n                  -----BEGIN RSA PRIVATE KEY-----\n                  ssh12\n                  -----END RSA PRIVATE KEY-----\n               - private_ip: 10.0.1.2\n                 ssh_user: ubuntu\n                 ssh_key: |\n                  -----BEGIN RSA PRIVATE KEY-----\n                  ssh13\n                  -----END RSA PRIVATE KEY-----\n            influx:\n               token: some_token\n               endpoint: http://influx.endpoint.com\n   computationalLayer2:\n      number: 2\n      Resources: \n         resource1:\n            name: RaspPi\n            minio:\n               endpoint: https://minio.endpoint.some\n               access_key: user\n               secret_key: pass\n            oscar:\n               name: oscar-test\n   computationalLayer3:\n      number: 3\n      Resources: \n         resource1:\n            name: AWS-FaaS\n            aws:\n               bucket: test1\n               region: us-east-1\n               access_key: ak\n               secret_key: sk\n```\n\nIn the OSCAR configuration a set of valid DNS records are assigned to the nodes to\nenable correct and secure external access to the services. A Route53 managed domain\nis required to make it work. You can set it with the ``--domain`` parameter\n(otherwise the default im.grycap.net will be used):\n\nIt has also the option to prepare the OSCAR clusters to be elastic,\nuse the `--elastic` option setting the maximum number of nodes to set.\nThis option has a limitation: the WNs of the cluster must have the same\nfeatures.\n\n```sh\ntoscarizer tosca --application_dir app --optimal --elastic 10\n```\n\nIn the elastic cases it needs the [IM authentication file](https://imdocs.readthedocs.io/en/latest/client.html#auth-file).\nThe default location is ``app/im/auth.dat`` or you can set another one using\n`--im_auth` option.\n\nThe generated TOSCA will also set the needed recipes to include the AI-SPRINT\nmonitoring system. These recipes require to set the central InfluxDB instance URL\n(`--influxdb_url` option) and a valid API token (`--influxdb_token` option).\n\n### Step5: Deploy TOSCA YAML files\n\nTo deploy the TOSCA files generated for the base case use:\n\n```sh\ntoscarizer deploy --application_dir app --base\n```\n\nTo deploy the TOSCA files generated for the optimal case use:\n\n```sh\ntoscarizer deploy --application_dir app --optimal\n```\n\nIn both cases it assumes that the [IM authentication file](https://imdocs.readthedocs.io/en/latest/client.html#auth-file)\nis located at path ``app/im/auth.dat``. It will use the EGI IM instance (\u003chttps://im.egi.eu/im/\u003e).\nThe auth file must contain not only the InfrastructureManager and the cloud provider\nselected, but also some AWS credentials (EC2 type) to manage the DNS domain names\nused in the OSCAR TOSCA template. In case of using the default domain value\n`im.grycap.net` you should contact the authors to get a set of valid credentials.\nOtherwise you have to add some EC2 credentials able to manage the specified domain in AWS Route53.\n\nBut you can also specify the URL of another IM endpoint, an specific IM\nauthentication file, and even the set of TOSCA files to deploy.\n\n```sh\ntoscarizer deploy --im_url http://someim.com \\\n                  --im_auth auth.dat \\\n                  --tosca_file some_path/tosca1.yaml \\\n                  --tosca_file some_path/tosca1.yaml\n```\n\nDuring deployment process the command will show the IDs of the infrastructures\ndeployed. You can use this ID to track the status of the deployment. You can\naccess the IM-Dashboard associated to the IM instance you have specified in\nthe deploy command ([default one](https://im.egi.eu)), or you can also use the\n[IM-Client](https://github.com/grycap/im-client).\n\nAt the end it will return in the standard output a YAML formatted output with the name\nof each yaml file with the infrastructure ID generated or the error message\nreturned. In case of unconfigured infrastructures it will also retun the contextualization\nlog with all the Ansible tasks performed to the infrastructure enabled to debug the error.\n\n### Step6: Get infrastructures Outputs\n\nTo get the TOSCA outputs of the infrastructures generated for the base case use:\n\n```sh\ntoscarizer outputs --application_dir app --base\n```\n\nTo get the TOSCA outputs of the infrastructures generated for the optimal case use:\n\n```sh\ntoscarizer outputs --application_dir app --optimal\n```\n\n### Step7: Delete infrastructures\n\nTo delete the infrastructures generated for the base case use:\n\n```sh\ntoscarizer delete --application_dir app --base\n```\n\nTo delete the infrastructures generated for the optimal case use:\n\n```sh\ntoscarizer delete --application_dir app --optimal\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrycap%2Ftoscarizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgrycap%2Ftoscarizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrycap%2Ftoscarizer/lists"}