{"id":15678979,"url":"https://github.com/trstringer/sherlock","last_synced_at":"2025-05-07T09:07:48.945Z","repository":{"id":147740266,"uuid":"97267718","full_name":"trstringer/sherlock","owner":"trstringer","description":":cloud: Integration testing sandbox environment provisioning tool for Microsoft Azure","archived":false,"fork":false,"pushed_at":"2017-11-05T13:22:29.000Z","size":72,"stargazers_count":12,"open_issues_count":0,"forks_count":6,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-07T09:07:41.816Z","etag":null,"topics":["azure","azure-functions","devops","integration-testing"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/trstringer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-07-14T19:45:43.000Z","updated_at":"2024-01-03T08:02:15.000Z","dependencies_parsed_at":"2023-05-27T09:15:07.519Z","dependency_job_id":null,"html_url":"https://github.com/trstringer/sherlock","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trstringer%2Fsherlock","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trstringer%2Fsherlock/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trstringer%2Fsherlock/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trstringer%2Fsherlock/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/trstringer","download_url":"https://codeload.github.com/trstringer/sherlock/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252847490,"owners_count":21813453,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure","azure-functions","devops","integration-testing"],"created_at":"2024-10-03T16:25:46.466Z","updated_at":"2025-05-07T09:07:48.927Z","avatar_url":"https://github.com/trstringer.png","language":"JavaScript","readme":"# Sherlock\n\n*Integration testing sandbox provisioning tool for Microsoft Azure*\n\n## Index\n\n- [Overview](#overview)\n- [Setup with Ansible](#setup-with-ansible)\n- [Setup manually](#setup-manually)\n- [Database](#database)\n- [Metadata Service](#metadata-service)\n- [Usage](#usage)\n\n## Overview\n\n**What does Sherlock provide for me?**\n\nSherlock will create one or more resource groups in a subscription, and create a corresponding service principal that has rights *only* in that/those resource group(s). There is also a cleanup process that will routinely run to delete resource groups and service principals from past integration test runs. In essence, this is a turn-key solution that requires no administration overhead for an integration testing environment.\n\n**What is Sherlock built with?**\n\nThis tool is an Azure Function app, with two functions: the first one is a web API that listens for requests to create a sandbox environment (and respond with the necessary connection information). The second Function is a cleanup process that is the cron job to remove sandbox environments in the subscription when they expire.\n\n## Setup with Ansible\n\n[Ansible playbook setup guide](setup/)\n\n## Setup Manually\n\nTo quickly and easily standup Sherlock in your Azure Subscription, I highly recommend you use the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli). The following steps assume that you have the Azure CLI installed and logged in for your subscription.\n\n1. Fork [this repository](https://github.com/tstringer/sherlock) into your GitHub account. I *highly* recommend that you fork this repository into your own GitHub account. I will continue active development on Sherlock and in order to ensure I don't introduce breaking changes into your integration testing environment, you should have a downstream fork so you can pick and choose when you merge updates (fixes, etc.)\n1. Create a resource group for Sherlock: `$ az group create -n sherlock-rg -l eastus`\n1. Create a storage account: `$ az storage account create -g sherlock-rg -n sherlockstor -l eastus --sku Standard_LRS`\n1. Create the Azure Function App: `$ az functionapp create -g sherlock-rg -n sherlockinttest -s sherlockstor -u https://github.com/tstringer/sherlock.git --consumption-plan-location eastus` (you will need to create a unique name for your Function App)\n1. Configure Sherlock (see the [Configuration section](#configuration) below)\n\n### Configuration\n\n1. Set the client ID app setting for Sherlock: `$ az functionapp config appsettings set -g sherlock-rg -n sherlockinttest --settings AZURE_CLIENT_ID=\u003cservice_principal_app_id\u003e` (this is going to be the service principal application ID that you have to prestage in your Azure AD tenant)\n1. Set the client secret for Sherlock: `$ az functionapp config appsettings set -g sherlock-rg -n sherlockinttest --settings AZURE_CLIENT_SECRET=\u003cservice_principal_key\u003e`\n1. Set the subscription ID: `$ az functionapp config appsettings set -g sherlock-rg -n sherlockinttest --settings AZURE_SUBSCRIPTION_ID=\u003csubscription_id\u003e`\n1. Set the tenant ID for your Azure AD: `$ az functionapp config appsettings set -g sherlock-rg -n sherlockinttest --settings AZURE_TENANT_ID=\u003ctenant_id\u003e`\n1. Set the prefix for Sherlock: `$ az functionapp config appsettings set -g sherlock-rg -n sherlockinttest --settings RES_PREFIX=sherlock` (this will be the prefix that is used to name provisioned resource groups and service principals)\n\n## Queue Setup and Configuration\n\nSherlock utilizes queueing for pooling identities. This queue is provided by Azure Storage, and therefore you need to setup the account prior to using Sherlock.\n\n1. Create a general purpose storage account in an Azure subscription\n1. On the Sherlock Function App, set the following environment variables:\n  - SHERLOCK_IDENTITY_STORAGE_ACCOUNT - set this to the Azure storage account\n  - SHERLOCK_IDENTITY_STORAGE_KEY - set this to the storage key\n\n:bulb: Note, you don't have to prestage the queue. The `identity-manager` Function will create it if it doesn't already exist\n\n## Database\n\nStarting in v0.4.0, Sherlock now uses persistent storage for metadata, moving away from resource group tags. The storage is a PostgreSQL database. Set the following Azure Function app setting environment variables to their appropriate value:\n\n- `PG_HOST`: the postgres hostname\n- `PG_DATABASE`: the database name\n- `PG_USER`: the role to connect to postgres\n- `PG_PASSWORD`: the role's password\n\n## Metadata Service\n\nWith the inception of the metadata service (`meta-manager` Azure Function), you need to set the following Azure Function app setting environment variables:\n\n- `META_URL`: the URL to the `meta-manager` Azure Function (can be retrieved from the portal)\n- `META_KEY`: the Azure Function auth key for the `meta-manager` Function\n\n## Usage\n\nOnce you have Sherlock setup and configured (see above), you only need to make a POST request to Sherlock. The request will look like: `https://\u003cfunction_app_name\u003e.azurewebsites.net/api/sandbox-provisioning?code=\u003ckey\u003e`, where `function_app_name` is the name of the Azure Function App you used when you created it above (in my case, I used `sherlockinttest` but you would have a different name).\n\nThe `key` is either the existing Function key that was created with the Azure Function was created, or a newly generated key (it is recommended to create a new key for each user and integration testing framework so that it is a more secure implementation, allowing you to revoke a key without affecting more users/clients). To create a new key you will have to use the Azure Portal. Navigate to the portal, and go to your Azure Function. Click on the **Manage** section for the `sandbox-provisioning` Function. Here you can view existing keys as well as create new keys.\n\n**Request parameters** (all optional)\n\n- *rgcount*: the amount of resource groups that need to be created (default **1**)\n- *region*: the location to create the resource groups in (default **eastus**)\n- *duration*: amount of time (in minutes) that the resource group (and corresponding principal) needs to be preserved for (default **30 minutes**). After this elapsed time, the cleanup Function will delete the resource groups and any resources in the resource group(s)\n- *prefix*: prefix the resource group (appended to `RES_PREFIX` or the default 'sherlock') with a request-level prefix\n\n**Examples**\n\n- Create a single resource group in East US that should live for 30 minutes: `$ curl \"https://\u003cfunction_app_name\u003e.azurewebsites.net/api/sandbox-provisioning?code=\u003ckey\u003e\"`\n- Create two resource groups in West US that should live for 2 hours: `$ curl \"https://\u003cfunction_app_name\u003e.azurewebsites.net/api/sandbox-provisioning?code=\u003ckey\u003e\u0026region=westus\u0026duration=120\u0026rgcount=2\"`\n\n**Response**\n\nSherlock, if successfully run, will respond with the following:\n\n- *resourceGroupNames*: an array of the name(s) of the resource group(s) that were created\n- *clientId*: the ID of the service principal that was created\n- *clientSecret*: the secret/password of the service principal that was created\n- *subscriptionId*: the subscription ID for the current Azure subscription\n- *tenantId*: the tenant ID of the current Azure AD tenant\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrstringer%2Fsherlock","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftrstringer%2Fsherlock","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrstringer%2Fsherlock/lists"}