{"id":13695214,"url":"https://github.com/replicate/replicate-python","last_synced_at":"2025-05-13T22:10:29.318Z","repository":{"id":37891237,"uuid":"490915757","full_name":"replicate/replicate-python","owner":"replicate","description":"Python client for Replicate","archived":false,"fork":false,"pushed_at":"2025-04-25T16:07:10.000Z","size":6240,"stargazers_count":824,"open_issues_count":47,"forks_count":244,"subscribers_count":38,"default_branch":"main","last_synced_at":"2025-05-08T07:13:01.597Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://replicate.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/replicate.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-05-11T01:26:58.000Z","updated_at":"2025-05-07T08:21:50.000Z","dependencies_parsed_at":"2023-02-16T11:15:26.323Z","dependency_job_id":"6c791298-d2ff-4a1a-a5c8-9dda0983f653","html_url":"https://github.com/replicate/replicate-python","commit_stats":{"total_commits":274,"total_committers":19,"mean_commits":"14.421052631578947","dds":"0.47080291970802923","last_synced_commit":"461ec70f566b0b993a94966ea73aa80a03eb5d60"},"previous_names":[],"tags_count":82,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/replicate%2Freplicate-python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/replicate%2Freplicate-python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/replicate%2Freplicate-python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/replicate%2Freplicate-python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/replicate","download_url":"https://codeload.github.com/replicate/replicate-python/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254036832,"owners_count":22003654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T18:00:19.756Z","updated_at":"2025-05-13T22:10:29.262Z","avatar_url":"https://github.com/replicate.png","language":"Python","funding_links":[],"categories":["Client libraries","Python"],"sub_categories":[],"readme":"# Replicate Python client\n\nThis is a Python client for [Replicate](https://replicate.com). It lets you run models from your Python code or Jupyter notebook, and do various other things on Replicate.\n\n## Breaking Changes in 1.0.0\n\nThe 1.0.0 release contains breaking changes:\n\n- The `replicate.run()` method now returns `FileOutput`s instead of URL strings by default for models that output files. `FileOutput` implements an iterable interface similar to `httpx.Response`, making it easier to work with files efficiently.\n\nTo revert to the previous behavior, you can opt out of `FileOutput` by passing `use_file_output=False` to `replicate.run()`:\n\n```python\noutput = replicate.run(\"acmecorp/acme-model\", use_file_output=False)\n```\n\nIn most cases, updating existing applications to call `output.url` should resolve any issues. But we recommend using the `FileOutput` objects directly as we have further improvements planned to this API and this approach is guaranteed to give the fastest results.\n\n\u003e [!TIP]\n\u003e **👋** Check out an interactive version of this tutorial on [Google Colab](https://colab.research.google.com/drive/1K91q4p-OhL96FHBAVLsv9FlwFdu6Pn3c).\n\u003e\n\u003e [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1K91q4p-OhL96FHBAVLsv9FlwFdu6Pn3c)\n\n## Requirements\n\n- Python 3.8+\n\n## Install\n\n```sh\npip install replicate\n```\n\n## Authenticate\n\nBefore running any Python scripts that use the API, you need to set your Replicate API token in your environment.\n\nGrab your token from [replicate.com/account](https://replicate.com/account) and set it as an environment variable:\n\n```\nexport REPLICATE_API_TOKEN=\u003cyour token\u003e\n```\n\nWe recommend not adding the token directly to your source code, because you don't want to put your credentials in source control. If anyone used your API key, their usage would be charged to your account.\n\n## Run a model\n\nCreate a new Python file and add the following code, replacing the model identifier and input with your own:\n\n```python\n\u003e\u003e\u003e import replicate\n\u003e\u003e\u003e outputs = replicate.run(\n        \"black-forest-labs/flux-schnell\",\n        input={\"prompt\": \"astronaut riding a rocket like a horse\"}\n    )\n[\u003creplicate.helpers.FileOutput object at 0x107179b50\u003e]\n\u003e\u003e\u003e for index, output in enumerate(outputs):\n        with open(f\"output_{index}.webp\", \"wb\") as file:\n            file.write(output.read())\n```\n\n`replicate.run` raises `ModelError` if the prediction fails.\nYou can access the exception's `prediction` property \nto get more information about the failure.\n\n```python\nimport replicate\nfrom replicate.exceptions import ModelError\n\ntry:\n  output = replicate.run(\"stability-ai/stable-diffusion-3\", { \"prompt\": \"An astronaut riding a rainbow unicorn\" })\nexcept ModelError as e\n  if \"(some known issue)\" in e.prediction.logs:\n    pass\n\n  print(\"Failed prediction: \" + e.prediction.id)\n```\n\n\u003e [!NOTE]\n\u003e By default the Replicate client will hold the connection open for up to 60 seconds while waiting\n\u003e for the prediction to complete. This is designed to optimize getting the model output back to the\n\u003e client as quickly as possible.\n\u003e\n\u003e The timeout can be configured by passing `wait=x` to `replicate.run()` where `x` is a timeout\n\u003e in seconds between 1 and 60. To disable the sync mode you can pass `wait=False`.\n\n## AsyncIO support\n\nYou can also use the Replicate client asynchronously by prepending `async_` to the method name. \n\nHere's an example of how to run several predictions concurrently and wait for them all to complete:\n\n```python\nimport asyncio\nimport replicate\n \n# https://replicate.com/stability-ai/sdxl\nmodel_version = \"stability-ai/sdxl:39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b\"\nprompts = [\n    f\"A chariot pulled by a team of {count} rainbow unicorns\"\n    for count in [\"two\", \"four\", \"six\", \"eight\"]\n]\n\nasync with asyncio.TaskGroup() as tg:\n    tasks = [\n        tg.create_task(replicate.async_run(model_version, input={\"prompt\": prompt}))\n        for prompt in prompts\n    ]\n\nresults = await asyncio.gather(*tasks)\nprint(results)\n```\n\nTo run a model that takes a file input you can pass either\na URL to a publicly accessible file on the Internet\nor a handle to a file on your local device.\n\n```python\n\u003e\u003e\u003e output = replicate.run(\n        \"andreasjansson/blip-2:f677695e5e89f8b236e52ecd1d3f01beb44c34606419bcc19345e046d8f786f9\",\n        input={ \"image\": open(\"path/to/mystery.jpg\") }\n    )\n\n\"an astronaut riding a horse\"\n```\n\n## Run a model and stream its output\n\nReplicate’s API supports server-sent event streams (SSEs) for language models. \nUse the `stream` method to consume tokens as they're produced by the model.\n\n```python\nimport replicate\n\nfor event in replicate.stream(\n    \"meta/meta-llama-3-70b-instruct\",\n    input={\n        \"prompt\": \"Please write a haiku about llamas.\",\n    },\n):\n    print(str(event), end=\"\")\n```\n\n\u003e [!TIP]\n\u003e Some models, like [meta/meta-llama-3-70b-instruct](https://replicate.com/meta/meta-llama-3-70b-instruct), \n\u003e don't require a version string. \n\u003e You can always refer to the API documentation on the model page for specifics.\n\nYou can also stream the output of a prediction you create.\nThis is helpful when you want the ID of the prediction separate from its output.\n\n```python\nprediction = replicate.predictions.create(\n    model=\"meta/meta-llama-3-70b-instruct\",\n    input={\"prompt\": \"Please write a haiku about llamas.\"},\n    stream=True,\n)\n\nfor event in prediction.stream():\n    print(str(event), end=\"\")\n```\n\nFor more information, see\n[\"Streaming output\"](https://replicate.com/docs/streaming) in Replicate's docs.\n\n\n## Run a model in the background\n\nYou can start a model and run it in the background using async mode:\n\n```python\n\u003e\u003e\u003e model = replicate.models.get(\"kvfrans/clipdraw\")\n\u003e\u003e\u003e version = model.versions.get(\"5797a99edc939ea0e9242d5e8c9cb3bc7d125b1eac21bda852e5cb79ede2cd9b\")\n\u003e\u003e\u003e prediction = replicate.predictions.create(\n    version=version,\n    input={\"prompt\":\"Watercolor painting of an underwater submarine\"})\n\n\u003e\u003e\u003e prediction\nPrediction(...)\n\n\u003e\u003e\u003e prediction.status\n'starting'\n\n\u003e\u003e\u003e dict(prediction)\n{\"id\": \"...\", \"status\": \"starting\", ...}\n\n\u003e\u003e\u003e prediction.reload()\n\u003e\u003e\u003e prediction.status\n'processing'\n\n\u003e\u003e\u003e print(prediction.logs)\niteration: 0, render:loss: -0.6171875\niteration: 10, render:loss: -0.92236328125\niteration: 20, render:loss: -1.197265625\niteration: 30, render:loss: -1.3994140625\n\n\u003e\u003e\u003e prediction.wait()\n\n\u003e\u003e\u003e prediction.status\n'succeeded'\n\n\u003e\u003e\u003e prediction.output\n\u003creplicate.helpers.FileOutput object at 0x107179b50\u003e\n\n\u003e\u003e\u003e with open(\"output.png\", \"wb\") as file:\n        file.write(prediction.output.read())\n```\n\n## Run a model in the background and get a webhook\n\nYou can run a model and get a webhook when it completes, instead of waiting for it to finish:\n\n```python\nmodel = replicate.models.get(\"ai-forever/kandinsky-2.2\")\nversion = model.versions.get(\"ea1addaab376f4dc227f5368bbd8eff901820fd1cc14ed8cad63b29249e9d463\")\nprediction = replicate.predictions.create(\n    version=version,\n    input={\"prompt\":\"Watercolor painting of an underwater submarine\"},\n    webhook=\"https://example.com/your-webhook\",\n    webhook_events_filter=[\"completed\"]\n)\n```\n\nFor details on receiving webhooks, see [replicate.com/docs/webhooks](https://replicate.com/docs/webhooks).\n\n## Compose models into a pipeline\n\nYou can run a model and feed the output into another model:\n\n```python\nlaionide = replicate.models.get(\"afiaka87/laionide-v4\").versions.get(\"b21cbe271e65c1718f2999b038c18b45e21e4fba961181fbfae9342fc53b9e05\")\nswinir = replicate.models.get(\"jingyunliang/swinir\").versions.get(\"660d922d33153019e8c263a3bba265de882e7f4f70396546b6c9c8f9d47a021a\")\nimage = laionide.predict(prompt=\"avocado armchair\")\nupscaled_image = swinir.predict(image=image)\n```\n\n## Get output from a running model\n\nRun a model and get its output while it's running:\n\n```python\niterator = replicate.run(\n    \"pixray/text2image:5c347a4bfa1d4523a58ae614c2194e15f2ae682b57e3797a5bb468920aa70ebf\",\n    input={\"prompts\": \"san francisco sunset\"}\n)\n\nfor index, image in enumerate(iterator):\n    with open(f\"file_{index}.png\", \"wb\") as file:\n        file.write(image.read())\n```\n\n## Cancel a prediction\n\nYou can cancel a running prediction:\n\n```python\n\u003e\u003e\u003e model = replicate.models.get(\"kvfrans/clipdraw\")\n\u003e\u003e\u003e version = model.versions.get(\"5797a99edc939ea0e9242d5e8c9cb3bc7d125b1eac21bda852e5cb79ede2cd9b\")\n\u003e\u003e\u003e prediction = replicate.predictions.create(\n        version=version,\n        input={\"prompt\":\"Watercolor painting of an underwater submarine\"}\n    )\n\n\u003e\u003e\u003e prediction.status\n'starting'\n\n\u003e\u003e\u003e prediction.cancel()\n\n\u003e\u003e\u003e prediction.reload()\n\u003e\u003e\u003e prediction.status\n'canceled'\n```\n\n## List predictions\n\nYou can list all the predictions you've run:\n\n```python\nreplicate.predictions.list()\n# [\u003cPrediction: 8b0ba5ab4d85\u003e, \u003cPrediction: 494900564e8c\u003e]\n```\n\nLists of predictions are paginated. You can get the next page of predictions by passing the `next` property as an argument to the `list` method:\n\n```python\npage1 = replicate.predictions.list()\n\nif page1.next:\n    page2 = replicate.predictions.list(page1.next)\n```\n\n## Load output files\n\nOutput files are returned as `FileOutput` objects:\n\n```python\nimport replicate\nfrom PIL import Image # pip install pillow\n\noutput = replicate.run(\n    \"stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478\",\n    input={\"prompt\": \"wavy colorful abstract patterns, oceans\"}\n    )\n\n# This has a .read() method that returns the binary data.\nwith open(\"my_output.png\", \"wb\") as file:\n  file.write(output[0].read())\n  \n# It also implements the iterator protocol to stream the data.\nbackground = Image.open(output[0])\n```\n\n### FileOutput\n\nIs a [file-like](https://docs.python.org/3/glossary.html#term-file-object) object returned from the `replicate.run()` method that makes it easier to work with models that output files. It implements `Iterator` and `AsyncIterator` for reading the file data in chunks as well as `read()` and `aread()` to read the entire file into memory.\n\n\u003e [!NOTE]\n\u003e It is worth noting that at this time `read()` and `aread()` do not currently accept a `size` argument to read up to `size` bytes.\n\nLastly, the URL of the underlying data source is available on the `url` attribute though we recommend you use the object as an iterator or use its `read()` or `aread()` methods, as the `url` property may not always return HTTP URLs in future.\n\n```python\nprint(output.url) #=\u003e \"data:image/png;base64,xyz123...\" or \"https://delivery.replicate.com/...\"\n```\n\nTo consume the file directly:\n\n```python\nwith open('output.bin', 'wb') as file:\n    file.write(output.read())\n```\n\nOr for very large files they can be streamed:\n\n```python\nwith open(file_path, 'wb') as file:\n    for chunk in output:\n        file.write(chunk)\n```\n\nEach of these methods has an equivalent `asyncio` API.\n\n```python\nasync with aiofiles.open(filename, 'w') as file:\n    await file.write(await output.aread())\n\nasync with aiofiles.open(filename, 'w') as file:\n    await for chunk in output:\n        await file.write(chunk)\n```\n\nFor streaming responses from common frameworks, all support taking `Iterator` types:\n\n**Django**\n\n```python\n@condition(etag_func=None)\ndef stream_response(request):\n    output = replicate.run(\"black-forest-labs/flux-schnell\", input={...}, use_file_output =True)\n    return HttpResponse(output, content_type='image/webp')\n```\n  \n**FastAPI**\n\n```python\n@app.get(\"/\")\nasync def main():\n    output = replicate.run(\"black-forest-labs/flux-schnell\", input={...}, use_file_output =True)\n    return StreamingResponse(output)\n```\n\n**Flask**\n\n```python\n@app.route('/stream')\ndef streamed_response():\n    output = replicate.run(\"black-forest-labs/flux-schnell\", input={...}, use_file_output =True)\n    return app.response_class(stream_with_context(output))\n```\n\nYou can opt out of `FileOutput` by passing `use_file_output=False` to the `replicate.run()` method.\n\n```python\nconst replicate = replicate.run(\"acmecorp/acme-model\", use_file_output=False);\n```\n\n## List models\n\nYou can list the models you've created:\n\n```python\nreplicate.models.list()\n```\n\nLists of models are paginated. You can get the next page of models by passing the `next` property as an argument to the `list` method, or you can use the `paginate` method to fetch pages automatically.\n\n```python\n# Automatic pagination using `replicate.paginate` (recommended)\nmodels = []\nfor page in replicate.paginate(replicate.models.list):\n    models.extend(page.results)\n    if len(models) \u003e 100:\n        break\n\n# Manual pagination using `next` cursors\npage = replicate.models.list()\nwhile page:\n    models.extend(page.results)\n    if len(models) \u003e 100:\n          break\n    page = replicate.models.list(page.next) if page.next else None\n```\n\nYou can also find collections of featured models on Replicate:\n\n```python\n\u003e\u003e\u003e collections = [collection for page in replicate.paginate(replicate.collections.list) for collection in page]\n\u003e\u003e\u003e collections[0].slug\n\"vision-models\"\n\u003e\u003e\u003e collections[0].description\n\"Multimodal large language models with vision capabilities like object detection and optical character recognition (OCR)\"\n\n\u003e\u003e\u003e replicate.collections.get(\"text-to-image\").models\n[\u003cModel: stability-ai/sdxl\u003e, ...]\n```\n\n## Create a model\n\nYou can create a model for a user or organization\nwith a given name, visibility, and hardware SKU:\n\n```python\nimport replicate\n\nmodel = replicate.models.create(\n    owner=\"your-username\",\n    name=\"my-model\",\n    visibility=\"public\",\n    hardware=\"gpu-a40-large\"\n)\n```\n\nHere's how to list of all the available hardware for running models on Replicate:\n\n```python\n\u003e\u003e\u003e [hw.sku for hw in replicate.hardware.list()]\n['cpu', 'gpu-t4', 'gpu-a40-small', 'gpu-a40-large']\n```\n\n## Fine-tune a model\n\nUse the [training API](https://replicate.com/docs/fine-tuning) to fine-tune models to make them better at a particular task.  To see what **language models** currently support fine-tuning,  check out Replicate's [collection of trainable language models](https://replicate.com/collections/trainable-language-models).\n\nIf you're looking to fine-tune **image models**, check out Replicate's [guide to fine-tuning image models](https://replicate.com/docs/guides/fine-tune-an-image-model).\n\nHere's how to fine-tune a model on Replicate:\n\n```python\ntraining = replicate.trainings.create(\n    model=\"stability-ai/sdxl\",\n    version=\"39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b\",\n    input={\n      \"input_images\": \"https://my-domain/training-images.zip\",\n      \"token_string\": \"TOK\",\n      \"caption_prefix\": \"a photo of TOK\",\n      \"max_train_steps\": 1000,\n      \"use_face_detection_instead\": False\n    },\n    # You need to create a model on Replicate that will be the destination for the trained version.\n    destination=\"your-username/model-name\"\n)\n```\n\n## Customize client behavior\n\nThe `replicate` package exports a default shared client. This client is initialized with an API token set by the `REPLICATE_API_TOKEN` environment variable.\n\nYou can create your own client instance to pass a different API token value, add custom headers to requests, or control the behavior of the underlying [HTTPX client](https://www.python-httpx.org/api/#client):\n\n```python\nimport os\nfrom replicate.client import Client\n\nreplicate = Client(\n    api_token=os.environ[\"SOME_OTHER_REPLICATE_API_TOKEN\"]\n    headers={\n        \"User-Agent\": \"my-app/1.0\"\n    }\n)\n```\n\n\u003e [!WARNING]\n\u003e Never hardcode authentication credentials like API tokens into your code.\n\u003e Instead, pass them as environment variables when running your program.\n\n## Development\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freplicate%2Freplicate-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freplicate%2Freplicate-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freplicate%2Freplicate-python/lists"}