{"id":19815937,"url":"https://github.com/replicate/cog-safe-push","last_synced_at":"2025-07-19T22:04:40.309Z","repository":{"id":254196794,"uuid":"845678183","full_name":"replicate/cog-safe-push","owner":"replicate","description":"Safely push a Cog model version by making sure it works and is backwards-compatible with previous versions.","archived":false,"fork":false,"pushed_at":"2025-06-10T04:22:56.000Z","size":1742,"stargazers_count":16,"open_issues_count":8,"forks_count":0,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-07-18T10:59:47.516Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://replicate.com/docs/guides/continuous-model-deployment","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/replicate.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-08-21T17:59:44.000Z","updated_at":"2025-06-03T14:02:10.000Z","dependencies_parsed_at":"2024-08-22T01:15:50.632Z","dependency_job_id":"760df43b-160e-41b6-a21b-231b341a09bd","html_url":"https://github.com/replicate/cog-safe-push","commit_stats":{"total_commits":20,"total_committers":3,"mean_commits":6.666666666666667,"dds":0.09999999999999998,"last_synced_commit":"7b811fcb88b0235b9d79cb4d01feef3497c6d946"},"previous_names":["replicate/cog-safe-push"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/replicate/cog-safe-push","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/replicate%2Fcog-safe-push","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/replicate%2Fcog-safe-push/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/replicate%2Fcog-safe-push/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/replicate%2Fcog-safe-push/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/replicate","download_url":"https://codeload.github.com/replicate/cog-safe-push/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/replicate%2Fcog-safe-push/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266026179,"owners_count":23866030,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T10:07:48.249Z","updated_at":"2025-07-19T22:04:40.267Z","avatar_url":"https://github.com/replicate.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cog-safe-push\n\nSafely push a Cog model version by making sure it works and is backwards-compatible with previous versions.\n\n\u003e [!TIP]\n\u003e Check out our [guide to building a CI/CD pipeline for your model](https://replicate.com/docs/guides/continuous-model-deployment), which includes a step-by-step walkthrough of how to use this tool.\n\n## Prerequisites\n\n1. Set the `ANTHROPIC_API_KEY` and `REPLICATE_API_TOKEN` environment variables.\n1. Install Cog and `cog login`\n1. If you're running this from a cloned source, `pip install .` in the `cog-safe-push` directory.\n\n## Installation\n\nThis package is not on PyPI yet, but you can install it directly from GitHub using pip:\n\n```\npip install git+https://github.com/replicate/cog-safe-push.git\n```\n\n## Usage\n\nTo safely push a model to Replicate, run this inside your Cog directory:\n\n```\n$ cog-safe-push --test-hardware=\u003chardware\u003e \u003cusername\u003e/\u003cmodel-name\u003e\n```\n\nThis will:\n1. Lint the predict file with ruff\n1. Create a private test model on Replicate, named `\u003cusername\u003e/\u003cmodel-name\u003e-test` running `\u003chardware\u003e`\n1. Push the local Cog model to the test model on Replicate\n1. Lint the model schema (making sure all inputs have descriptions, etc.)\n1. If there is an existing version on the upstream `\u003cusername\u003e/\u003cmodel-name\u003e` model, it will\n   1. Make sure that the schema in the test version is backwards compatible with the existing upstream version\n   1. Run predictions against both upstream and test versions and make sure the same inputs produce the same (or equivalent) outputs\n1. Fuzz the test model for five minutes by throwing a bunch of different inputs at it and make sure it doesn't throw any errors\n\nBoth the creation of model inputs and comparison of model outputs is handled by Claude.\n\n## Example GitHub Actions workflow\n\nCreate a new workflow file in `.github/workflows/cog-safe-push.yaml` and add the following:\n\n```yaml\nname: Cog Safe Push\n\non:\n  workflow_dispatch:\n    inputs:\n      model:\n        description: 'The name of the model to push in the format owner/model-name'\n        type: string\n\njobs:\n  cog-safe-push:\n    # Tip: Create custom runners in your GitHub organization for faster builds\n    runs-on: ubuntu-latest\n\n    steps:\n    - uses: actions/checkout@v3\n\n    - name: Set up Python\n      uses: actions/setup-python@v4\n      with:\n        python-version: \"3.12\"\n\n    - name: Install Cog\n      run: |\n        sudo curl -o /usr/local/bin/cog -L https://github.com/replicate/cog/releases/latest/download/cog_`uname -s`_`uname -m`\n        sudo chmod +x /usr/local/bin/cog\n\n    - name: cog login\n      run: |\n        echo ${{ secrets.COG_TOKEN }} | cog login --token-stdin\n\n    - name: Install cog-safe-push\n      run: |\n        pip install git+https://github.com/replicate/cog-safe-push.git\n\n    - name: Push selected models\n      env:\n        ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}\n        REPLICATE_API_TOKEN: ${{ secrets.REPLICATE_API_TOKEN }}\n      run: |\n        cog-safe-push ${{ inputs.model }}\n```\n\nAfter pushing this workflow to the main branch, you can run it manually from the Actions tab.\n\n### Full help text\n\n\u003c!-- generate: cog-safe-push --help --\u003e\n\n```text\n# cog-safe-push --help\n\nusage: cog-safe-push [-h] [--config CONFIG] [--help-config] [--test-model TEST_MODEL]\n                     [--no-push] [--test-hardware TEST_HARDWARE] [--no-compare-outputs]\n                     [--predict-timeout PREDICT_TIMEOUT] [--fast-push]\n                     [--test-case TEST_CASES] [--fuzz-fixed-inputs FUZZ_FIXED_INPUTS]\n                     [--fuzz-disabled-inputs FUZZ_DISABLED_INPUTS]\n                     [--fuzz-iterations FUZZ_ITERATIONS] [--fuzz-prompt FUZZ_PROMPT]\n                     [--parallel PARALLEL] [--ignore-schema-compatibility] [-v]\n                     [--push-official-model]\n                     [model]\n\nSafely push a Cog model, with tests\n\npositional arguments:\n  model                 Model in the format \u003cowner\u003e/\u003cmodel-name\u003e\n\noptions:\n  -h, --help            show this help message and exit\n  --config CONFIG       Path to the YAML config file. If --config is not passed, ./cog-\n                        safe-push.yaml will be used, if it exists. Any arguments you pass\n                        in will override fields on the predict configuration stanza.\n  --help-config         Print a default cog-safe-push.yaml config to stdout.\n  --test-model TEST_MODEL\n                        Replicate model to test on, in the format \u003cusername\u003e/\u003cmodel-name\u003e.\n                        If omitted, \u003cmodel\u003e-test will be used. The test model is created\n                        automatically if it doesn't exist already\n  --no-push             Only test the model, don't push it to \u003cmodel\u003e\n  --test-hardware TEST_HARDWARE\n                        Hardware to run the test model on. Only used when creating the\n                        test model, if it doesn't already exist.\n  --no-compare-outputs  Don't make predictions to compare that prediction outputs match\n                        the current version\n  --predict-timeout PREDICT_TIMEOUT\n                        Timeout (in seconds) for predictions. Default: 300\n  --fast-push           Use the --x-fast flag when doing cog push\n  --test-case TEST_CASES\n                        Inputs and expected output that will be used for testing, you can\n                        provide multiple --test-case options for multiple test cases. The\n                        first test case will be used when comparing outputs to the current\n                        version. Each --test-case is semicolon-separated key-value pairs\n                        in the format '\u003ckey1\u003e=\u003cvalue1\u003e;\u003ckey2=value2\u003e[\u003coutput-checker\u003e]'.\n                        \u003coutput-checker\u003e can either be '==\u003cexact-string-or-url\u003e' or\n                        '~=\u003cai-prompt\u003e'. If you use '==\u003cexact-string-or-url\u003e' then the\n                        output of the model must match exactly the string or url you\n                        specify. If you use '~=\u003cai-prompt\u003e' then the AI will verify your\n                        output based on \u003cai-prompt\u003e. If you omit \u003coutput-checker\u003e, it will\n                        just verify that the prediction doesn't throw an error.\n  --fuzz-fixed-inputs FUZZ_FIXED_INPUTS\n                        Inputs that should have fixed values during fuzzing. All other\n                        non-disabled input values will be generated by AI. If no test\n                        cases are specified, these will also be used when comparing\n                        outputs to the current version. Semicolon-separated key-value\n                        pairs in the format '\u003ckey1\u003e=\u003cvalue1\u003e;\u003ckey2=value2\u003e' (etc.)\n  --fuzz-disabled-inputs FUZZ_DISABLED_INPUTS\n                        Don't pass values for these inputs during fuzzing. Semicolon-\n                        separated keys in the format '\u003ckey1\u003e;\u003ckey2\u003e' (etc.). If no test\n                        cases are specified, these will also be disabled when comparing\n                        outputs to the current version.\n  --fuzz-iterations FUZZ_ITERATIONS\n                        Maximum number of iterations to run fuzzing.\n  --fuzz-prompt FUZZ_PROMPT\n                        Additional prompting for the fuzz input generation\n  --parallel PARALLEL   Number of parallel prediction threads.\n  --ignore-schema-compatibility\n                        Ignore schema compatibility checks when pushing the model\n  -v, --verbose         Increase verbosity level (max 3)\n  --push-official-model\n                        Push to the official model defined in the config\n```\n\n### Using a configuration file\n\nYou can use a configuration file instead of passing all arguments on the command line. If you create a file called `cog-safe-push.yaml` in your Cog directory, it will be used. Any command line arguments you pass will override the values in the config file.\n\n\u003c!-- generate: cog-safe-push --help-config --\u003e\n\n```yaml\n# cog-safe-push --help-config\n\nmodel: \u003cmodel\u003e\ntest_model: \u003ctest model, or empty to append '-test' to model\u003e\ntest_hardware: \u003chardware, e.g. cpu\u003e\npredict:\n  compare_outputs: true\n  predict_timeout: 300\n  test_cases:\n  - inputs:\n      \u003cinput1\u003e: \u003cvalue1\u003e\n    exact_string: \u003cexact string match\u003e\n  - inputs:\n      \u003cinput2\u003e: \u003cvalue2\u003e\n    match_url: \u003cmatch output image against url\u003e\n  - inputs:\n      \u003cinput3\u003e: \u003cvalue3\u003e\n    match_prompt: \u003cmatch output using AI prompt, e.g. 'an image of a cat'\u003e\n  - inputs:\n      \u003cinput3\u003e: \u003cvalue3\u003e\n    error_contains: \u003cassert that these inputs throws an error, and that the error\n      message contains a string\u003e\n  fuzz:\n    fixed_inputs: {}\n    disabled_inputs: []\n    iterations: 10\n    prompt: \u003cadditional prompt for the language model when selecting fuzz inputs\u003e\ntrain:\n  destination: \u003cgenerated prediction model, e.g. andreasjansson/test-predict. leave\n    blank to append '-dest' to the test model\u003e\n  destination_hardware: \u003chardware for the created prediction model, e.g. cpu\u003e\n  train_timeout: 300\n  test_cases:\n  - inputs:\n      \u003cinput1\u003e: \u003cvalue1\u003e\n    exact_string: \u003cexact string match\u003e\n  - inputs:\n      \u003cinput2\u003e: \u003cvalue2\u003e\n    match_url: \u003cmatch output image against url\u003e\n  - inputs:\n      \u003cinput3\u003e: \u003cvalue3\u003e\n    match_prompt: \u003cmatch output using AI prompt, e.g. 'an image of a cat'\u003e\n  - inputs:\n      \u003cinput3\u003e: \u003cvalue3\u003e\n    error_contains: \u003cassert that these inputs throws an error, and that the error\n      message contains a string\u003e\n  fuzz:\n    fixed_inputs: {}\n    disabled_inputs: []\n    iterations: 10\n    prompt: \u003cadditional prompt for the language model when selecting fuzz inputs\u003e\ndeployment:\n  owner: \u003cowner\u003e\n  name: \u003cname\u003e\n  hardware: \u003chardware\u003e\nparallel: 4\nfast_push: false\nuse_cog_base_image: true\nignore_schema_compatibility: false\nofficial_model: \u003cofficial model, e.g. user/model\u003e\n\n# values between \u003c and \u003e should be edited\n```\n\n## Deployments\n\nThe tool can automatically create or update deployments for your model on Replicate. To use this feature:\n\n1. Add deployment settings to your `cog.yaml`:\n\n```yaml\ndeployment:\n  name: my-model-deployment\n  owner: your-username  # optional, defaults to model owner\n  hardware: cpu  # or gpu-t4, gpu-a100, etc.\n```\n\n2. When you run `cog-safe-push`, it will:\n   - Create a new deployment if one doesn't exist\n   - Update the existing deployment with the new version if it does exist\n   - Use appropriate instance scaling based on hardware:\n     - CPU: 1-20 instances\n     - GPU: 0-2 instances\n\nThe deployment will be created under the specified owner (or model owner if not specified) and will use the hardware configuration you provide.\n\n## Nota bene\n\n* This is alpha software. If you find a bug, please open an issue!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freplicate%2Fcog-safe-push","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freplicate%2Fcog-safe-push","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freplicate%2Fcog-safe-push/lists"}