{"id":34033747,"url":"https://github.com/numerai/numerai-cli","last_synced_at":"2026-04-01T19:18:39.387Z","repository":{"id":34901024,"uuid":"187930883","full_name":"numerai/numerai-cli","owner":"numerai","description":"Automated submission workflows in the cloud.","archived":false,"fork":false,"pushed_at":"2025-12-17T07:55:56.000Z","size":11169,"stargazers_count":95,"open_issues_count":10,"forks_count":29,"subscribers_count":20,"default_branch":"master","last_synced_at":"2026-03-27T15:38:34.397Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/numerai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2019-05-22T00:00:47.000Z","updated_at":"2026-01-27T03:02:07.000Z","dependencies_parsed_at":"2025-01-02T22:27:16.417Z","dependency_job_id":"a338ba1d-8d54-4a4c-b3ee-ba255934e37f","html_url":"https://github.com/numerai/numerai-cli","commit_stats":{"total_commits":160,"total_committers":12,"mean_commits":"13.333333333333334","dds":"0.48750000000000004","last_synced_commit":"032f6222f4ee2b4925fb411124a9739061f2e57f"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/numerai/numerai-cli","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numerai%2Fnumerai-cli","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numerai%2Fnumerai-cli/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numerai%2Fnumerai-cli/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numerai%2Fnumerai-cli/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/numerai","download_url":"https://codeload.github.com/numerai/numerai-cli/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/numerai%2Fnumerai-cli/sbom","scorecard":{"id":698609,"data":{"date":"2025-08-11","repo":{"name":"github.com/numerai/numerai-cli","commit":"e4e16559ac609801acd4598eee417efccff8d3c8"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.7,"checks":[{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":2,"reason":"Found 6/24 approved changesets -- score normalized to 2","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/test-and-deploy.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test-and-deploy.yml:51: update your workflow using https://app.stepsecurity.io/secureworkflow/numerai/numerai-cli/test-and-deploy.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test-and-deploy.yml:52: update your workflow using https://app.stepsecurity.io/secureworkflow/numerai/numerai-cli/test-and-deploy.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test-and-deploy.yml:14: update your workflow using https://app.stepsecurity.io/secureworkflow/numerai/numerai-cli/test-and-deploy.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test-and-deploy.yml:15: update your workflow using https://app.stepsecurity.io/secureworkflow/numerai/numerai-cli/test-and-deploy.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test-and-deploy.yml:26: update your workflow using https://app.stepsecurity.io/secureworkflow/numerai/numerai-cli/test-and-deploy.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test-and-deploy.yml:27: update your workflow using https://app.stepsecurity.io/secureworkflow/numerai/numerai-cli/test-and-deploy.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test-and-deploy.yml:38: update your workflow using https://app.stepsecurity.io/secureworkflow/numerai/numerai-cli/test-and-deploy.yml/master?enable=pin","Warn: containerImage not pinned by hash: numerai/examples/crypto-python3/Dockerfile:2: pin your Docker image by updating python:3.9 to python:3.9@sha256:b93ed1cb6815be1d99908ab629d49680be28024c4183611ded45c425ecf7b907","Warn: containerImage not pinned by hash: numerai/examples/signals-python3/Dockerfile:2: pin your Docker image by updating python:3.9 to python:3.9@sha256:b93ed1cb6815be1d99908ab629d49680be28024c4183611ded45c425ecf7b907","Warn: containerImage not pinned by hash: numerai/examples/tournament-python3/Dockerfile:2: pin your Docker image by updating python:3.9 to python:3.9@sha256:b93ed1cb6815be1d99908ab629d49680be28024c4183611ded45c425ecf7b907","Warn: containerImage not pinned by hash: numerai/examples/tournament-rlang/Dockerfile:2: pin your Docker image by updating r-base:latest to r-base:latest@sha256:c143ac223eff70f437c56f980ed582b4960dbb460a499b1d5119ff461a9f051b","Warn: pipCommand not pinned by hash: numerai/examples/crypto-python3/Dockerfile:26","Warn: pipCommand not pinned by hash: numerai/examples/signals-python3/Dockerfile:26","Warn: pipCommand not pinned by hash: numerai/examples/tournament-python3/Dockerfile:26","Warn: pipCommand not pinned by hash: .github/workflows/test-and-deploy.yml:19","Warn: pipCommand not pinned by hash: .github/workflows/test-and-deploy.yml:31","Warn: pipCommand not pinned by hash: .github/workflows/test-and-deploy.yml:56","Info:   0 out of   7 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   4 containerImage dependencies pinned","Info:   0 out of   6 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Vulnerabilities","score":7,"reason":"3 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2024-231 / GHSA-2586-f3p4-hq84","Warn: Project is vulnerable to: PYSEC-2023-238 / GHSA-5wvp-7f3h-6wmm","Warn: Project is vulnerable to: PYSEC-2024-161"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 21 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-22T04:30:20.326Z","repository_id":34901024,"created_at":"2025-08-22T04:30:20.326Z","updated_at":"2025-08-22T04:30:20.326Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31051803,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-27T16:55:14.406Z","status":"ssl_error","status_checked_at":"2026-03-27T16:55:07.885Z","response_time":164,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-13T19:20:40.677Z","updated_at":"2026-04-01T19:18:39.374Z","avatar_url":"https://github.com/numerai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# numerai-cli\n\n[![PyPI](https://img.shields.io/pypi/v/numerai-cli.svg?color=brightgreen)](https://pypi.org/project/numerai-cli/)\n\nWelcome to the Numerai CLI for the [Numerai Tournament](https://docs.numer.ai/tournament/learn).\nNumerai CLI (aka Compute Heavy) is a tool to help you set up cloud automation for your Numerai predictions.\nThis README is designed to have EVERYTHING you need to setup and maintain a Numerai Compute Heavy node.\n\nThis CLI runs on your local computer to configure a Numerai Prediction Node in the cloud.\nThis solution is architected to cost less than $5/mo on average, but actual costs may vary.\nIt has been tested on MacOS/OSX, Windows 8/10, and Ubuntu 18/20,\nbut should theoretically work anywhere that Docker and Python 3 are available.\n\n## Contents\n\n- [Getting Started](#getting-started)\n  - [List of Commands](#list-of-commands)\n  - [Upgrading](#upgrading)\n    - [Upgrading from 0.1/0.2 to 0.3.0](#upgrading-from-0102-to-030)\n    - [Beyond](#beyond)\n  - [Uninstalling](#uninstalling)\n- [Troubleshooting and Feedback](#troubleshooting-and-feedback)\n  - [Python](#python)\n  - [Docker](#docker)\n    - [MacOS and Windows 10](#macos-and-windows-10)\n    - [Linux](#linux)\n    - [Older PCs: Docker Toolbox](#older-pcs-docker-toolbox)\n  - [Azure](#azure)\n  - [Common Errors](#common-errors)\n- [Billing Alerts](#billing-alerts)\n- [Prediction Node Architecture](#prediction-node-architecture)\n  - [Python Example](#python-example)\n  - [RLang Example](#rlang-example)\n  - [The Dockerfile](#the-dockerfile)\n  - [Cloud Components](#cloud-components)\n- [Special Thanks](#special-thanks)\n\n## Getting Started\n\nTo use this tool you need:\n\n- Numerai API keys\n- Paid cloud provider account\n- Python\n- Docker\n\n1. Sign up a Numerai Account, get your Numerai API Keys, and your first Model:\n\n   1. Sign up at \u003chttps://numer.ai/signup\u003e and log in to your new account\n   2. Go to \u003chttps://numer.ai/account\u003e \u003e \"Your API keys\" section \u003e click \"Add\"\n   3. Name your key and check all boxes under \"Scope this key will have\"\n   4. Enter your password and click \"Confirm\"\n   5. Copy your secret public key and secret key somewhere safe\n\n2. Choose your cloud provider and follow the corresponding guide:\n\n   - [AWS Setup Guide](./docs/aws_setup_guide.md)\n   - [Azure Setup Guide](./docs/azure_setup_guide.md)\n   - [GCP Setup Guide](./docs/gcp_setup_guide.md)\n\n3. Install Docker and Python for your Operating System (if you encounter errors or your\n   OS is not supported, please see [Troubleshooting and Feedback](#troubleshooting-and-feedback)):\n\n   - Mac Terminal (cmd + space, type `terminal`, select `terminal.app`):\n\n     ```shell\n     curl https://raw.githubusercontent.com/numerai/numerai-cli/master/scripts/setup-mac.sh | bash\n     ```\n\n   - Ubuntu 18/20 Terminal (ctrl + alt + t):\n\n     ```shell\n     sudo apt update \u0026\u0026 sudo apt install -y libcurl4 curl \u0026\u0026 sudo curl https://raw.githubusercontent.com/numerai/numerai-cli/master/scripts/setup-ubu.sh | sudo bash\n     ```\n\n   - Windows 10 Command Prompt (windows key, type `cmd`, select Command Prompt):\n\n     ```powershell\n     powershell -command \"$Script = Invoke-WebRequest 'https://raw.githubusercontent.com/numerai/numerai-cli/master/scripts/setup-win10.ps1'; $ScriptBlock = [ScriptBlock]::Create($Script.Content); Invoke-Command criptBlock $ScriptBlock\"\n     ```\n\n4. After the setup script confirms Python and Docker, install `numerai-cli` via:\n\n   ```shell\n   pip3 install --upgrade numerai-cli --user\n   ```\n\n   NOTES:\n\n   - This command will also work to update to new versions of the package in the future.\n   - If you are using python venv then drop the --user option.\n     If you don't know what that is, disregard this note.\n\n5. Run these commands on your personal computer (not a temporary instance)\n   to get an example node running in minutes:\n\n   For AWS run:\n\n   ```shell\n   numerai setup --provider aws\n   ```\n\n   and enter your user's keys when prompted.\n\n   For Azure users run:\n\n   ```shell\n   numerai setup --provider azure\n   ```\n\n   and enter your application's keys when prompted.\n\n   For GCP users run:\n\n   ```shell\n   numerai setup --provider gcp\n   ```\n\n   and enter the complete path to your downloaded key when prompted.\n\n   ```shell\n   numerai node config --example tournament-python3\n   numerai node deploy\n   numerai node test\n   ```\n\n   If you want to use larger instances to generate your predictions first run `numerai list-constants`\n   to list the vCPU/mem presets available, then you can configure a node to use one of the presets via:\n\n   ```shell\n   numerai node config  mem-lg\n   ```\n\n   Your compute node is now setup and ready to run! When you make changes to your code or re-train your model,\n   simply deploy and test your node again:\n\n   ```shell\n   numerai node deploy\n   numerai node test\n   ```\n\n   NOTES:\n\n   - These commands have stored configuration files in `$USER_HOME/.numerai/`. DO NOT LOSE THIS FILE!\n     or else you will have to manually delete every cloud resource by hand.\n   - The example node trains a model in the cloud, which is bad. You should train locally, pickle the\n     trained model, deploy your node, then unpickle your model to do the live predictions\n   - The default example does _not_ make stake changes; please refer to the [numerapi docs](https://numerapi.readthedocs.io/en/latest/api/numerapi.html#module-numerapi.numerapi)\n     for the methods you must call to do this.\n   - You can view resources and logs in the AWS Console (region us-east-1) for your\n     [ECS Cluster](https://console.aws.amazon.com/ecs/home?region=us-east-1#/clusters/numeraiubmission-ecs-cluster/tasks)\n     and [other resources](https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:log-groups)\n\n### List of Commands\n\nUse the `--help` option on any command or sub-command to get a full description of it:\n\n```shell\nnumerai\nnumerai --help\nnumerai [command] --help\nnumerai [command] [sub-command] --help\n```\n\nEach command or sub-command takes its own options, for example if you want to copy the\nnumerai signals example and configure a node for a signals model with large memory\nrequirements you'd use something like this (replacing [MODEL NAME] with the relevant\nsignals model name):\n\n```shell\nnumerai node -m [MODEL NAME] -t 11 config -s mem-lg -e signals-python3\n```\n\nHere, the `node` command takes a model name with `-m` and an argument `-t 11` to specify the tournament number\n(numerai is tournament 8, signals is tournament 11, crypto is tournament 12).\nThe `config` sub-command also takes a `-s` option to specify the size of the node to configure.\n\n### Upgrading\n\nUpgrading numerai-cli will always require you to update the package itself using pip:\n\n```shell\npip install --upgrade numerai-cli --user\nnumerai upgrade\n```\n\n#### 0.1/0.2 to 0.3\n\nCLI 0.3 uses a new configuration format that is incompatible with versions 0.1 and 0.2,\nbut a command to migrate you configuration is provided for you. Run this in the directory\nyou ran `numerai setup` from the previous version:\n\n```shell\nnumerai upgrade\n```\n\n#### 0.3 to 0.4\n\nCLI 0.4 introduces a new provider option (Microsoft Azure) and moves the default aws\nterraform into a subdirectory. You'll need to run `upgrade`:\n\n```shell\nnumerai upgrade\n```\n\nIf you want to use azure, follow the [setup guide for azure](./docs/azure_setup_guide.md)\nthen run:\n\n```shell\nnumerai setup --provider azure\nnumerai node config --provider azure\n```\n\n#### 0.4 to 1.0\n\nCLI 1.0 is a big change for AWS, you'll need to:\n\n- update the permissions for the role you created in the [AWS Setup Guide](./docs/aws_setup_guide.md)\n- update the terraform with `numerai setup`\n- replace your aws nodes by running:\n  - `numerai node  -m \u003cmodel_name\u003e destroy --preserve-node-config`\n  - `numerai node -m \u003cmodel_name\u003e config`\n  - `numerai node -m \u003cmodel_name\u003e deploy`\n\n#### Beyond\n\nSome updates will make changes to configuration files used by Numerai CLI. These will\nrequire you to re-run some commands to upgrade your nodes to the newest versions:\n\n- `numerai setup` will copy over changes to files in the `$HOME/.numerai` directory\n- `numerai node config` will apply those changes to a node\n\n### Uninstalling\n\n```shell\nnumerai uninstall\n```\n\n## Troubleshooting and Feedback\n\nPlease review this entire section and check github issues before asking questions.\nIf you've exhausted this document, then join us on Discord\n\nIf you still cannot find a solution or answer, please join us on\n[Discord](https://discord.gg/numerai)\nand include the following information with your issue/message:\n\n- The commands you ran that caused the error (even previous commands)\n- Version information from running:\n  - `pip3 show numerai-cli`\n  - `python -V`\n  - `docker -v`\n- System Information from running\n  - Mac: `system_profiler SPSoftwareDataType \u0026\u0026 system_profiler SPHardwareDataType`\n  - Linux: `lsb_release -a \u0026\u0026 uname -a`\n  - Windows: `powershell -command \"Get-ComputerInfo\"`\n\n### Python\n\nIf the environment setup script fails to install Python3 for you, report the error to Numerai\nand then install it manually with one of the following options:\n\n- [Download Python3 directly](https://www.python.org/downloads/)\n- install it from [your system's package manager](https://en.wikipedia.org/wiki/List_of_software_package_management_systems)\n- Use [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/) to install and manage python for you\n\n### Docker\n\nIf the environment setup script fails to install Docker for you, report the error to Numerai\nthen read the following to get a working installation on your machine. For PCs, you may need to [activate virtualization in your BIOS](https://superuser.com/questions/1382472/how-do-i-find-and-enable-the-virtualization-setting-on-windows-10) before installing.\n\n#### MacOS and Windows 10\n\nJust install [Docker Desktop](https://www.docker.com/products/docker-desktop) to get it running.\nYou should also increase the RAM allocated to the VM:\n\n1. Open the Docker Desktop GUI then\n2. Click Gear in top right corner\n3. Select Resources \u003e Advanced in left sidebar\n4. Use slider to allocate more memory (leave a few gigs for your OS and background applications, otherwise your computer might crash)\n\n#### Linux\n\nCheck here for instructions for your distribution:\n\u003chttps://docs.docker.com/engine/install/\u003e\n\n#### Older PCs: Docker Toolbox\n\nIf your machine is older and/or doesn't have Hyper-V enabled, then you will have to follow these steps to install docker toolbox on your machine:\n\n1. [Install Oracle VirtualBox](https://www.virtualbox.org/wiki/Downloads) for your Operating System\n2. Restart your computer\n3. [Install Docker Toolbox](https://github.com/docker/toolbox/releases)\n4. Restart your computer\n5. After it's installed, open the \"Docker QuickStart Terminal\" and run the following to increase its RAM:\n\n```shell\ndocker-machine rm default\ndocker-machine create -d virtualbox --virtualbox-cpu-count=2 --virtualbox-memory=4096 --virtualbox-disk-size=50000 default\ndocker-machine restart default\n```\n\nNOTE: your code must live somewhere under your User directory (ie. C:\\Users\\USER_NAME\\ANY_FOLDER). This is a restriction of docker toolbox not sharing paths correctly otherwise.\n\n### Azure\n\nIf you just made your Azure account there's a chance your account provisioning could take some time, potentially up to 24 hours.\n\nWhen configuring your node for the first time the numerai-cli may hang as it tries to provision infrastructure in your account. If running `numerai node config --provider azure` shows no log output for more than 5 minutes, your account is likely in the stuck provisioning state. While we investigate this issue, the best course of action is to wait until the following day and run the command again as there is no way to skip this Azure provisioning step.\n\n### Common Errors\n\n```shell\nError:\nsubprocess.CalledProcessError: Command 'docker run --rm -it -v /home/jason/tmp/.numerai:/opt/plan -w /opt/plan hashicorp/terraform:light init' returned non-zero exit status 127.\n\n\nReason:\nDocker is not installed.\n\nSolutions:\nIf you're certain that docker is installed, make sure that your user can execute docker, ie. try to run `docker ps`.\nIf that's the issue, then depending on your system, you can do the following:\n\n- Windows/OSX\n    - Make sure the Docker Desktop is running and finished booting up.\n      It can take a few minutes to be completely ready.\n      When clicking on the docker tray icon, it should say \"Docker Desktop is Running\".\n    - If you're using Docker Toolbox on Windows, then make sure you've opened the \"Docker Quickstart Terminal\".\n\n- Linux\n    - Run `sudo usermod -aG docker $USER`\n    - Then reboot or logout/login for this to take effect.\n```\n\n```shell\nError:\ndocker: Error response from daemon: Drive has not been shared\n\nSolutions:\n- You need to [share your drive](https://docs.docker.com/docker-for-windows/#shared-drives).\n```\n\n```shell\nError:\nnumerai: command not found\n\nSolutions:\n- osx/linux: Try and run `~/.local/bin/numerai`\n- Windows: `%LOCALAPPDATA%\\Programs\\Python\\Python37-32\\Scripts\\numerai.exe`\n- Alternatively, exit then re-open your terminal/command prompt.\n```\n\n```shell\nError:\nerror calling sts:GetCallerIdentity: InvalidClientTokenId: The security token included in the request is invalid.\n...\nCommand 'docker run -e \"AWS_ACCESS_KEY_ID=...\" -e \"AWS_SECRET_ACCESS_KEY=...\" --rm -it -v /home/jason/tmp/.numerai:/opt/plan -w /opt/planhashicorp/terraform:light apply -auto-approve' returned non-zero exit status 1.\n\nSolutions:\n- Run `numerai configure` to re-write your API Keys\n```\n\n```shell\nError:\nERROR numerapi.base_api: Error received from your webhook server: {\"tasks\":[],\"failures\":[{\"reason\":\"The requested CPU configuration is above your limit\"}]}\n\nSolution:\n1. Go to the [Quota Dashboard for EC2](console.aws.amazon.com/servicequotas/home/services/ec2/quotas)\n2. Search for \"On-Demand\", this will list all instance types and their limits for your account.\n3. Click the bubble next to \"Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) instances\"\n4. click \"Request quota increase\" in the top right\n5. Input a higher value than the currently applied quota and finish the request\n```\n\n- If after AWS increases your quota, you still get this error, try again 1-2 more times\n- You may have to complete a quota request for other types of instances too depending on how resource intensive your setup is\n\n## Billing Alerts\n\nThere's no automated way to setup billing alerts, so you'll need to\n[configure one manually](https://www.cloudberrylab.com/resources/blog/how-to-configure-billing-alerts-for-your-aws-account/).\nWe estimate costs to be less than $5 per month unless your compute takes more than 12 hours a week,\nbut increasing the RAM/CPU will increase your costs.\n\n## Prediction Node Architecture\n\nA Prediction Node represents a cloud-based model that Numerai can trigger for predictions; it is designed to be reliable, resource efficient, and easy to configure and debug. Prediction Nodes use a few important components like a `Dockerfile`, a `Trigger`, a `Container`, and a `Compute Cluster`, all of which can be created using one of the following examples.\n\n### Python Example\n\n```shell\nnumerai-python3\n├── Dockerfile\n├── .dockerignore\n├── predict.py\n├── requirements.txt\n└── train.py\n```\n\n- `Dockerfile`: Used during `numerai node deploy` to build a Docker image that's used to run your code in the cloud. It copies all files in its directory, installs Python requirements for requirements.txt, and runs `python predict.py` by default.\n\n- `.dockerignore`: This file uses regex to match files that should not be included in the Docker image.\n\n- `train.py`: This is an extra entry point specifically for training, it's used when running `numerai node test --local --command \"python train.py\"`\n\n- `requirements.txt`: Defines python packages required to run the code.\n- `predict.py`: Gets run by default locally and in the cloud when running `numerai test` without the `--command|-c` option.\n\n### RLang Example\n\n```shell\nnumerai-rlang\n├── Dockerfile\n├── .dockerignore\n├── install_packages.R\n└── main.R\n```\n\n- `Dockerfile`: Used during `numerai node deploy` to build a Docker image that's used to run your code in the cloud. It copies all files in its directory, installs Rlang requirements from install_packages.R, and runs main.R by default.\n\n- `.dockerignore`: This file uses regex to match files that should not be included in the Docker image.\n\n- `install_packages.R`: Installs dependencies necessary for running the example.\n- `main.R`: Ran by default locally and in the cloud and when running `numerai test` without the `--command|-c` option.\n\n### The Dockerfile\n\nThis is the most important component of deploying a Prediction Node. It is a program (much like a bash script), that packages up your code as an `image`; this image contains everything your code needs to run in the cloud. The most typical case of a Dockerfile is demonstrated in [Numerai Examples](./numerai/examples/), if you're not sure how to use a Dockerfile, first copy an example with `numerai copy-example`, then read the documentation in the Dockerfile to learn the basics.\n\nThese files are very flexible, the default Dockerfile will just copy everything in whatever directory it is in, but this can be customized if you'd like to share code between models. For example, if you have a python project setup like so:\n\n```shell\nnumerai_models\n├── common\n├──── __init__.py\n├──── data.py\n├──── setup.py\n├── model_1\n├──── Dockerfile\n├──── .dockerignore\n├──── predict.py\n├──── requirements.txt\n└──── train.py\n```\n\nWhere `common` is an installable python package you want to use in multiple models, you can add this line to model_1/Dockerfile: `RUN pip install ../common/`. Finally, run `numerai node deploy` from the `numerai_models` directory to install the package in the image, making it available to your model code.\n\nIf you want to learn more about how to customize this file [checkout the Dockerfile reference] (\u003chttps://docs.docker.com/engine/reference/builder/\u003e).\n\n### Cloud Components\n\nThe CLI uses [Terraform](https://www.terraform.io/) to provision cloud resources. Each component and the related cloud resource(s) are listed below. The links will take you to the AWS console where you can monitor any of these resources for a given node; just visit the link and select the resource with the same name as the node you want to monitor (further directions are given for each resource below).\n\n- `Trigger`: A small function that schedules a \"task\" on your `Compute Cluster`. This \"task\" handles pulling the image that was created by the `Dockerfile` and running it as a `Container` on your `Compute Cluster`. This is handled by two resources:\n\n  - **[API Gateway](https://console.aws.amazon.com/apigateway/main/apis)**:\n    Hosts the webhook (HTTP endpoint) that Numerai calls to trigger your nodes.\n    After clicking the link and selecting the resource, use the left sidebar to access metrics and logging.\n  - **[Lambda](https://console.aws.amazon.com/lambda/home#/functions)**:\n    Schedules your compute job when you call your Webhook URL.\n    After clicking the link and selecting the resource, use the \"Monitor\" tab below the \"Function Overview\" section.\n\n- `Container`: The thing that actually contains and runs your code on a computer provisioned by the `Compute Cluster`. The `--size` (or `-s`) flag on the `numerai node config` sets the CPU and Memory limits for a `Container`. This is stored in one place:\n\n  - **[ECR (Elastic Container Repository)](https://console.aws.amazon.com/ecr/repositories)**:\n    Used for storing docker images. This is the location to which `numerai docker deploy` will push your image.\n    There is not much monitoring here, but you can view your images and when they were uploaded.\n\n- `Compute Cluster`: A handler that accepts scheduled \"tasks\" and spins up and down computers to run `Containers`. This is handled by ECS:\n  - **[ECS (Elastic Container Service)](https://console.aws.amazon.com/ecs/home#/clusters)**:\n    This is where your containers will actually run and where you'll want to look if your containers don't seem to be scheduled/running.\n    After clicking the link, you'll be able to scroll and monitor the top-level metrics of each cluster.\n    After selecting a specific cluster, you can use the various tabs to view different components of the cluster (tasks are the runnable jobs\n    that the Lambda schedules, instances are the computers the tasks run on, and metrics will show cluster-wide information)\n\n## Special Thanks\n\n- Thanks to [uuazed](https://github.com/uuazed) for their work on [numerapi](https://github.com/uuazed/numerapi)\n- Thanks to [hellno](https://github.com/hellno) for starting the Signals [ticker map](https://github.com/hellno/numerai-signals-tickermap)\n- Thanks to tit_BTCQASH ([numerai profile](https://numer.ai/tit_btcqash) and [twitter profile](https://twitter.com/tit_BTCQASH)) for debugging the environment setup process on Windows 8\n- Thanks to [eses-wk](https://github.com/eses-wk) for implementing Azure support\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnumerai%2Fnumerai-cli","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnumerai%2Fnumerai-cli","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnumerai%2Fnumerai-cli/lists"}