{"id":21414396,"url":"https://github.com/houraiteahouse/tapioca","last_synced_at":"2026-03-09T17:46:17.348Z","repository":{"id":90519782,"uuid":"155306781","full_name":"HouraiTeahouse/Tapioca","owner":"HouraiTeahouse","description":"Efficient and cost effective continuous delivery for large binary games.","archived":false,"fork":false,"pushed_at":"2020-05-27T12:14:32.000Z","size":146,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2023-10-20T20:14:59.963Z","etag":null,"topics":["deployment","game"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HouraiTeahouse.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-10-30T01:40:17.000Z","updated_at":"2023-10-20T20:15:03.523Z","dependencies_parsed_at":null,"dependency_job_id":"2d5bb88f-b976-4d80-8f7a-b51b5dcc3931","html_url":"https://github.com/HouraiTeahouse/Tapioca","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HouraiTeahouse%2FTapioca","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HouraiTeahouse%2FTapioca/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HouraiTeahouse%2FTapioca/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HouraiTeahouse%2FTapioca/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HouraiTeahouse","download_url":"https://codeload.github.com/HouraiTeahouse/Tapioca/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225945052,"owners_count":17549498,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deployment","game"],"created_at":"2024-11-22T18:29:19.020Z","updated_at":"2026-03-09T17:46:12.320Z","avatar_url":"https://github.com/HouraiTeahouse.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tapioca [![Travis CI](https://api.travis-ci.org/HouraiTeahouse/Tapioca.svg?branch=master)](https://travis-ci.org/HouraiTeahouse/Tapioca)\n\nTapioca is Hourai Teahouse's game deployment system.  Inspired by Valve's\nSteamPipe upload system, Tapioca is built to efficiently and continuously\ndeploy large binary games to players' computers.\n\n## Features\n\n * Continuous deployment of games builds to player.\n * Support for multiple deployment branches. Useful for seperating the mainline.\n * Maximized bandwidth efficiency - downloads only changed portions of games for\n   each patch.\n * Minimizes storage costs by deduplicating redundant data across builds and\n   branches.\n * Leverages HTTP caching - built to maximize usage of content delivery networks\n   (CDNs).\n * Can operate with no user-facing server - minimizes costs on server usage.\n\n## Architecture\n\nThe process of pushing new game builds involves the following processes:\n\n * Downloading build artifacts from a continuous integration enviroment.\n * Chunking up build artifacts into roughly equal sized blocks.\n * Hashing each block to generate a unique block identifier.\n * Creating a manifest to describe binary builds using these blocks.\n * Uploading compressed blocks and manifest to a static file server or object\n   store.\n\nTapioca clients check the game manifests on startup and ensure that the local\ncopy is up to date with the latest build. If a new build is available, only the\nchanged blocks are downloaded and patched into the local copy.\n\nBlocks are uniquely identified by their hash. Tapioca currently uses SHA-512 as\nits hashing algorithm. This has some nice properties:\n\n * This identifier is immutable for a given block, making it reasonable to\n   aggressively cache the blocks in CDNs and other intermediate HTTP caches.\n * Identical blocks always have the same hash. This allows storing only one copy\n   of the block for multiple builds and branches across a project.\n * As any machine is capable of running the hash, clients can independently\n   verify the integrity of their local game files without needing to make any\n   network calls to a server.\n * For the purposes of deduplication, SHA-512 hashes are virtually guaranteed to\n   be unique. As Tapioca stores and operates on 1MB blocks, one would likely\n   need to store more data than the universe can hold before finding a effective\n   block collision.\n * The SHA hash function family will soon see broad hardware acceleration\n   support, making game verification on clients faster. SHA-512 also generally\n   runs faster on 64-bit machines, compared to SHA-256.\n\nBlocks are compressed on upload. By default Tapioca is set to `gzip -9` blocks\nbefore upload to the object store.\n\nTo minimize bandwidth and storage costs, Hourai Teahouse's instance of Tapioca\nutilizes Backblaze B2 as the backing block store, with Cloudflare as the public\nfacing CDN, but practically any AWS S3-style object store or static file server\nworks. B2 to CloudFlare bandwidth is free, so the only monetary costs in this\nconfiguration only involve storage and requests.\n\nFinally, as each client must make at least one HTTP request on every launch of a\ngame, to minimize the payload size of these requests, Tapioca uses compressed\nGoogle ProtoBuffers as a binary message format to deliver game manifest updates.\n\n## Development\n\n```\nTOOD(james7132): Document\n```\n\n## Potential Future Features\n\n * IPFS storage or P2P block downloads to decrease the number of requests made\n   to the backing object store.\n * Support encrypting or signing blocks or entire builds.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhouraiteahouse%2Ftapioca","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhouraiteahouse%2Ftapioca","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhouraiteahouse%2Ftapioca/lists"}