{"id":13425639,"url":"https://github.com/MakerXStudio/algorand-sandbox-dev","last_synced_at":"2025-03-15T20:30:50.421Z","repository":{"id":43661290,"uuid":"455411644","full_name":"MakerXStudio/algorand-sandbox-dev","owner":"MakerXStudio","description":"Algorand sandbox containers with dev configuration and patched for indexer speed","archived":true,"fork":false,"pushed_at":"2024-01-02T09:25:44.000Z","size":93,"stargazers_count":15,"open_issues_count":0,"forks_count":2,"subscribers_count":17,"default_branch":"main","last_synced_at":"2024-01-25T05:11:17.857Z","etag":null,"topics":["algorand","blockchain","dev-xp","devops","docker","oss","sandbox","web3"],"latest_commit_sha":null,"homepage":"","language":"PowerShell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MakerXStudio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-02-04T03:38:56.000Z","updated_at":"2024-04-15T06:30:37.043Z","dependencies_parsed_at":"2024-04-15T06:30:34.529Z","dependency_job_id":"a3fda716-2164-4c39-bf88-63b8d1808daa","html_url":"https://github.com/MakerXStudio/algorand-sandbox-dev","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MakerXStudio%2Falgorand-sandbox-dev","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MakerXStudio%2Falgorand-sandbox-dev/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MakerXStudio%2Falgorand-sandbox-dev/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MakerXStudio%2Falgorand-sandbox-dev/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MakerXStudio","download_url":"https://codeload.github.com/MakerXStudio/algorand-sandbox-dev/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243790939,"owners_count":20348378,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorand","blockchain","dev-xp","devops","docker","oss","sandbox","web3"],"created_at":"2024-07-31T00:01:15.939Z","updated_at":"2025-03-15T20:30:50.083Z","avatar_url":"https://github.com/MakerXStudio.png","language":"PowerShell","readme":"# algorand-sandbox-dev\n\nThis repository is now deprecated in favour of using official Algorand images via [AlgoKit LocalNet](https://github.com/algorandfoundation/algokit-cli/blob/main/docs/features/localnet.md). This repository will be archived.\n\nThis repo is loosely based on [Algorand's official sandbox](https://github.com/algorand/sandbox), but solves some performance and experience problems which impact local development and CI/CD usage.\n\nThe key problems we resolve are:\n\n* Long build times for CI/CD\n* Slow devnet + indexer performance\n* Ability to use cross-platform (e.g. on Windows)\n\nThis project was developed by [MakerX](https://makerx.com.au/) to help us with our initiatives, but we are happily providing it for the Algorand community to use as Open Source.\n\n[![MakerX logo](logo.png)](https://makerx.com.au/)\n\n## How to use\n\nYou'll need [Docker installed](https://docs.docker.com/get-docker/). The following examples also use `docker compose`, which ships with Docker Desktop on Windows and macOS - for Linux users, see the [installation instructions](https://docs.docker.com/compose/install/).\n\n### Option A: With pre-built images from DockerHub\n\nEither save the following as `docker-compose.yaml` locally, or if you've already got a compose setup, just copy in the\nservices:\n\n```yaml\nversion: '3'\n\nservices:\n  algod:\n    image: makerxau/algorand-sandbox-dev:latest\n    ports:\n      - \"4001:4001\"\n      - \"4002:4002\"\n      - \"9392:9392\"\n\n  indexer:\n    image: makerxau/algorand-indexer-dev:latest\n    ports:\n      - \"8980:8980\"\n    restart: unless-stopped\n    environment:\n      ALGOD_HOST: \"algod\"\n      POSTGRES_HOST: \"indexer-db\"\n      POSTGRES_PORT: \"5432\"\n      POSTGRES_USER: algorand\n      POSTGRES_PASSWORD: algorand\n      POSTGRES_DB: indexer_db\n    depends_on:\n      - indexer-db\n      - algod\n\n  indexer-db:\n    image: \"postgres:13-alpine\"\n    container_name: \"algorand-sandbox-postgres\"\n    ports:\n      - \"5433:5432\"\n    user: postgres\n    environment:\n      POSTGRES_USER: algorand\n      POSTGRES_PASSWORD: algorand\n      POSTGRES_DB: indexer_db\n```\n\nThen run `docker compose up` in the directory of your compose file. This will pull the pre-built images down from DockerHub.\n\nCurrently, the only tags being pushed are `latest`, which are manually triggered builds based on the latest stable \nreleases of both [go-algorand](https://github.com/algorand/go-algorand/) and [indexer](https://github.com/algorand/indexer).\n\nNote that when using the `latest` tag, if a new build gets pushed to DockerHub but you have an existing stack,\n`docker compose` won't check to see if there's a new build available. To update, stop the running stack and:\n\n    docker compose down\n    docker compose pull\n\nThis should fetch the latest images and you can start your stack again.\n\nNote in particular that we need to use `down` here, and not just `stop` (which is what ctrl-c will do if you're not running docker compose with the `-d` flag),\na new build will generate a new network genesis, but the database container will still contain data from the previous network.\nIf you don't remove the container before restarting (e.g. through the `down` command  above), you'll get errors like the following:\n\n    algorand-sandbox-indexer   | {\"error\":\"genesis hash not matching\",\"level\":\"error\",\"msg\":\"importer.EnsureInitialImport() error\",\"time\":\"2022-02-25T01:33:59Z\"}\n    algorand-sandbox-indexer exited with code 1\n\n### Option B: From this repo\n\nIf you want to test out a newer version (e.g. the latest beta version of `go-algorand`), you can check out this repo,\nand run your `docker compose` commands from here, the `BRANCH` args in `docker-compose.yaml` control which git branches\nof their respective repos get checked out and built.\n\nAlso, be careful of the `indexer-db` container persisting after a rebuild, as noted above in Option A, this is even \nmore important if you're building/re-building locally.\n\n## Replicating sandbox.sh functionality\n\nThere are a number of things that you can do with `sandbox.sh` within the [Algorand Sandbox](https://github.com/algorand/sandbox#algorand-sandbox) that we have created cross-platform equivalents of using [PowerShell Core](https://docs.microsoft.com/en-us/powershell/scripting/overview?view=powershell-7.2).\n\n### Status\n\nIn order to check the status of the running environment, copy [status.ps1](./status.ps1) into your project wherever you put your `docker-compose.yml` file and execute it via `./status.ps1`.\n\n### Reset\n\nIn order to reset the environment, copy [reset.ps1](./reset.ps1) into your project wherever you put your `docker-compose.yml` file and execute it via `./reset.ps1`.\n\n### Goal\n\nIn order to run a [goal](https://developer.algorand.org/docs/clis/goal/goal/) command against your algod instance, copy [goal.ps1](./goal.ps1) into your project wherever you put your `docker-compose.yml` file and execute it via `./goal.ps1 {command}` e.g. `./goal.ps1 account list`.\n\n## Getting access to the private key of the faucet account\n\nIf you want to use the sandbox then you need to get the private key of the initial wallet so you can transfer ALGOs out of it to other accounts you create. There are two ways to do this:\n\n### Option 1: Manually via goal\n\n```\n./goal.ps1 account list\n./goal.ps1 account export -a {address_from_online_account_from_above_command}\n```\n\n### Option 2: Automatically via kmd API\n\nNeeding to do this manual step every time you spin up a new development environment or reset your Sandbox is frustrating. Instead, it's useful to have code that uses the Sandbox APIs to automatically retrieve the private key of the default account. The following `getSandboxDefaultAccount` function (in TypeScript, but equivalent will work with algosdk in other languages) will help you achieve that:\n\n```typescript\n// account.ts\nimport algosdk, { Account, Algodv2 } from 'algosdk'\nimport { getKmdClient } from './client'\n\nexport async function isSandbox(client: Algodv2): Promise\u003cboolean\u003e {\n  const params = await client.getTransactionParams().do()\n\n  return params.genesisID === 'devnet-v1' || params.genesisID === 'sandnet-v1'\n}\n\nexport async function isReachSandbox(client: Algodv2): Promise\u003cboolean\u003e {\n  const params = await client.getTransactionParams().do()\n\n  return params.genesisHash === 'lgTq/JY/RNfiTJ6ZcZ/Er8uR775Hk76c7MvHVWjvjCA='\n}\n\nexport function getAccountFromMnemonic(mnemonic: string): Account {\n  return algosdk.mnemonicToSecretKey(mnemonic)\n}\n\nexport async function getSandboxDefaultAccount(client: Algodv2): Promise\u003cAccount\u003e {\n  if (!(await isSandbox(client))) {\n    throw \"Can't get default account from non Sandbox network\"\n  }\n\n  if (await isReachSandbox(client)) {\n    // https://github.com/reach-sh/reach-lang/blob/master/scripts/devnet-algo/algorand_network/FAUCET.mnemonic\n    return getAccountFromMnemonic(\n      'frown slush talent visual weather bounce evil teach tower view fossil trip sauce express moment sea garbage pave monkey exercise soap lawn army above dynamic'\n    )\n  }\n\n  const kmd = getKmdClient()\n  const wallets = await kmd.listWallets()\n\n  // Sandbox starts with a single wallet called 'unencrypted-default-wallet', with heaps of tokens\n  const defaultWalletId = wallets.wallets.filter((w: any) =\u003e w.name === 'unencrypted-default-wallet')[0].id\n\n  const defaultWalletHandle = (await kmd.initWalletHandle(defaultWalletId, '')).wallet_handle_token\n  const defaultKeyIds = (await kmd.listKeys(defaultWalletHandle)).addresses\n\n  // When you create accounts using goal they get added to this wallet so check for an account that's actually a default account\n  let i = 0\n  for (i = 0; i \u003c defaultKeyIds.length; i++) {\n    const key = defaultKeyIds[i]\n    const account = await client.accountInformation(key).do()\n    if (account.status !== 'Offline' \u0026\u0026 account.amount \u003e 1000_000_000) {\n      break\n    }\n  }\n\n  const defaultAccountKey = (await kmd.exportKey(defaultWalletHandle, '', defaultKeyIds[i])).private_key\n\n  const defaultAccountMnemonic = algosdk.secretKeyToMnemonic(defaultAccountKey)\n  return getAccountFromMnemonic(defaultAccountMnemonic)\n}\n```\n\nTo get the `Kmd` instance you can use something like this (tweak it based on where you retrieve your ALGOD token and server from):\n\n```typescript\n// client.ts\nimport {Kmd} from 'algosdk'\n\n// KMD client allows you to export private keys, which is useful to get the default account in a sandbox network\nexport function getKmdClient(): Kmd {\n  // We can only use Kmd on the Sandbox otherwise it's not exposed so this makes some assumptions (e.g. same token and server as algod and port 4002)\n  return new Kmd(process.env.ALGOD_TOKEN!, process.env.ALGOD_SERVER!, '4002')\n}\n```\n\n## Motivation\n\n### 1. ~~DevMode and indexer do not play nicely together~~\n\nThis turned out to be a bug, and [has been fixed upstream as of indexer version 2.10.0](https://github.com/algorand/indexer/pull/920)!\n\n### 2. Executing docker builds as part of CI/CD is slow\nBuilding the official algorand sandbox as part of CI/CD takes quite some time, so we do this as part of a separate pipeline\n(see `azure-pipelines.yml` in this repo for the configuration), and push the results to DockerHub. As outlined above, \nwe've also made these builds public, which saves time locally when updating to new versions.\n\nWe've also optimised the docker images to minimise their size by utilising multi-stage builds, and building for multiple\narchitectures (AMD64 and ARM64 - so M1 Macs are supported).\n\n### 3. Other minor issues fixed\nThere was a potential race condition in the sandbox setup, where the `indexer` might start up before `postgres` comes alive,\nparticularly on first run when the database is being initialised. This is solved in the sandbox by specifying the `indexer`\nservice to have `restart: unless-stopped`, which we've also retained here, but to prevent potentially confusing error\nmessages and/or container restarts, we've made the `indexer` wait on startup until the `postgres` daemon is responding\nto network requests.\n\n## Alternatives\n\n`reachsh/devnet-algo` - on DockerHub, but not always kept up to date, and only supports AMD64 images. Also, no access\nto `kmd`.\n\n## Outstanding Issues\nThere appears to be an issue with `DevMode` enabled, where after ~258 rounds, a Compact Cert is generated, and the \nindexer can't determine the signature type for it:\n\n    algorand-sandbox-indexer   | {\"level\":\"info\",\"msg\":\"adding block 259\",\"time\":\"2022-02-25T04:38:44Z\"}\n    algorand-sandbox-indexer   | {\"error\":\"AddBlock() err: TxWithRetry() err: attemptTx() err: AddBlock() err: AddBlock() err: getSigTypeDelta() err: unable to determine the signature type\",\"level\":\"error\",\"msg\":\"adding block 259 to database failed\",\"time\":\"2022-02-25T04:38:44Z\"}\n\nAt this point, the only way to resolve the issue is to re-create the containers by stopping the stack and running `docker compose down`\nbefore restarting it.\n","funding_links":[],"categories":["Development \u0026 Tools","Development Tools"],"sub_categories":["Deployment \u0026 Environment","Other Development Tools"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMakerXStudio%2Falgorand-sandbox-dev","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FMakerXStudio%2Falgorand-sandbox-dev","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMakerXStudio%2Falgorand-sandbox-dev/lists"}