{"id":26119247,"url":"https://github.com/scottroot/transformers-ao","last_synced_at":"2026-04-21T01:32:06.507Z","repository":{"id":241200519,"uuid":"804618945","full_name":"scottroot/transformers-ao","owner":"scottroot","description":null,"archived":false,"fork":false,"pushed_at":"2024-06-23T00:30:50.000Z","size":620,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-12-15T02:15:36.258Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scottroot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-05-22T23:59:26.000Z","updated_at":"2024-06-23T00:30:53.000Z","dependencies_parsed_at":"2024-05-23T00:52:56.897Z","dependency_job_id":"218fbf09-48b8-443e-be78-e0caeb62fde9","html_url":"https://github.com/scottroot/transformers-ao","commit_stats":null,"previous_names":["scottroot/transformers-ao"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/scottroot/transformers-ao","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottroot%2Ftransformers-ao","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottroot%2Ftransformers-ao/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottroot%2Ftransformers-ao/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottroot%2Ftransformers-ao/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scottroot","download_url":"https://codeload.github.com/scottroot/transformers-ao/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottroot%2Ftransformers-ao/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32072953,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-20T21:26:33.338Z","status":"ssl_error","status_checked_at":"2026-04-20T21:26:22.081Z","response_time":94,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-10T12:25:42.836Z","updated_at":"2026-04-21T01:32:06.483Z","avatar_url":"https://github.com/scottroot.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Transformers-AO Logo](web/images/transformers-ao.png)\n# Transformers-AO (wip)\nTransformers-AO brings the functionality of the popular [Hugging Face](https://huggingface.co/)'s Transformers library to Arweave's new cloud computing blockchain \"AO\".  \nThis means the ability to use the latest and your favorite open-source models in a familiar API.\n\n----\n## Components of Transformers-AO\n * WeaveDrive: method to deterministically load model/config data into AO processes\n * Model-Specific functions.  Currently live:\n   * Bert\n\n## Models supported\n✅ SentenceTransformers (bert)  \n⬜ Clip vit-base32 text model  \n✅ Stable Diffusion 1.5  \n⬜ Mistral  \n⬜ Stable Diffusion 2  \n⬜ Stable Diffusion XL  \n⬜ Stable Diffusion Turbo  \n✅ T5  \n⬜ MusicGen (in progress)  \n⬜ ...  \n\n## What is AO?\n\u003e **TL;DR:** AO is a massive, shared-computer where tasks run in their own virtual machines. Requests and results are stored immutably on-chain with Arweave. Highly customizable, it is ideal for creating powerful machine learning pipelines.\n\nAO is like a massive shared computer that anyone can use. Each task run on AO is executed in its own personal virtual machine, separate from anyone else’s.\n\nTo run a program or trigger a function, like a feature-extraction pipeline or generating an image with StableDiffusion, you open an AO instance and send a message listing the target machine (the “Process”), and tags indicating what you want to do. The results are computed, saved on-chain, and handled or returned based on your function's logic.\n\nBecause AO processes are highly customizable and easy to configure using a simple language called Lua, developers can create chained sequences that conditionally run based on the output of the previous result. This is similar to directed acyclic graphs (DAGs), and enables users to build powerful machine learning pipelines, either running synchronously or triggering many processes in parallel.\n\n## How To Use\nThere are many ways to use TFAO both on-demand and as part of an existing AO/Arweave application.\n\n### Primary tools to work with AO:\n * [AOconnect-ts](https://github.com/scottroot/aoconnect): Typescript package providing the basic tools to connect to AO, run tasks and read results\n * [AO Connect (official)](https://github.com/permaweb/ao/connect): Original JavaScript aoconnect npm package - same functionality as the TS version above, though likely to be more up-to-date.\n * [AOS](https://github.com/permaweb/aos): Command-line program to interact with AO within a Lua programming language runtime, send messages and read results.\n\nUse the ao-connect package to call an AO process running the model you want to use\nFor example:\n```JS\nimport { connect } from \"aoconnect\";\n\nconst { dryrun, message, results } = connect();\n\n/*\n  A manifest of active TFAO processes will be published once \n  they are live.  This value is just for demonstration purposes.\n */\nconst tfaoProcess = \"P0BHJWnF4JjQ_NotARealTxId-qGsJdXa7yUDaqN3BOlYFYwAwR0\"\n\n/*\n  Each of the files for the model you're using\n  are the same as you would for Python Transformers or Pytorch, \n  but we reference them by their Arweave transaction id,\n  since they must be stored on-chain.\n  Example TX ids for the model and config data:\n */\nconst miniLM = {\n  model: \"ABCD\",\n  tokenizer: \"EFGH\",\n  config: \"IJKL\"\n}\n\nconst codeToRun = `\n   wd = require(\"weavedrive\")\n   bert = require(\"transformers\").bert\n   \n   local opts = {\n     model = wd.getData(\"${miniLM.model}\"),\n     tokenizer = wd.getData(\"${miniLM.tokenizer}\"),\n     config = wd.getData(\"${miniLM.config}\"),\n     prompt = \"Hello world.\"\n   }\n   \n   return bert.encode_text(opts)\n`;\n\n\n/* \n  You could use dryrun to evaluate your code and run \n  your desired model, which does not save the results to memory.\n */\nconst tempResult = await dryrun({\n  process: tfaoProcess,\n  tags: [{name: 'Action', value: 'Eval'}],\n  data: codeToRun,\n  });\n\n/*\n  You could send a message, which will run your code and store the result, and\n  then read the result.\n */\nconst messageId = await message({\n    process: processId,\n    tags: [{name: 'Action', value: 'Eval'}],\n    signer: createDataItemSigner(wallet),\n    data: codeToRun,\n});\n\nlet { Messages, Spawns, Output, Error } = await result({\n    message: messageId,\n    process: processId,\n});\nconsole.log(JSON.stringify({ Messages, Spawns, Output, Error }));\n```\n\n\n## Disclaimer\nPlease note that this is an independent community project and is not affiliated with or endorsed by HuggingFace or AO.\n\n#### License for Transformers\nThis work, like the original [Transformers library](https://github.com/huggingface/transformers)/[Candle library](https://github.com/huggingface/candle) by HuggingFace, is licensed under the Apache License 2.0. You can find the full text of the license in the [LICENSE](./LICENSE) file. Contributions to this project will also be made available under the same license.\n  \n#### License for AO\n* **Licensed Work:** ao codebase. The Licensed Work is (c) 2024 Forward Research.\n* **Additional Use Grant:** The ao codebase is offered under the BSL 1.1 license for the duration of the testnet period. After the testnet phase is over, the code will be made available under either a new evolutionary forking license or a traditional OSS license (GPLv3/v2, MIT, etc). For more information, please consult this [article on Arweave's medium](https://arweave.medium.com/arweave-is-an-evolutionary-protocol-e072f5e69eaa).\n* **Change Date:** Four years from the date the Licensed Work is published.\n* **Change License:** MPL 2.0.\n  \nFor more information about the original AO library, please see the [Permaweb/ao GitHub](https://github.com/permaweb/ao).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscottroot%2Ftransformers-ao","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscottroot%2Ftransformers-ao","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscottroot%2Ftransformers-ao/lists"}