{"id":36892325,"url":"https://github.com/transparency-dev/tessera","last_synced_at":"2026-02-17T17:02:46.381Z","repository":{"id":247607400,"uuid":"813665926","full_name":"transparency-dev/tessera","owner":"transparency-dev","description":"Go library for building tile-based transparency logs (tlogs)","archived":false,"fork":false,"pushed_at":"2026-02-10T15:53:11.000Z","size":4666,"stargazers_count":156,"open_issues_count":25,"forks_count":40,"subscribers_count":7,"default_branch":"main","last_synced_at":"2026-02-10T18:30:34.066Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/transparency-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-06-11T14:06:11.000Z","updated_at":"2026-02-10T15:51:38.000Z","dependencies_parsed_at":"2024-07-22T12:35:57.924Z","dependency_job_id":"048aa04f-b971-4900-9553-771f543899fe","html_url":"https://github.com/transparency-dev/tessera","commit_stats":null,"previous_names":["transparency-dev/trillian-tessera","transparency-dev/tessera"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/transparency-dev/tessera","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transparency-dev%2Ftessera","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transparency-dev%2Ftessera/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transparency-dev%2Ftessera/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transparency-dev%2Ftessera/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/transparency-dev","download_url":"https://codeload.github.com/transparency-dev/tessera/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transparency-dev%2Ftessera/sbom","scorecard":{"id":489558,"data":{"date":"2025-08-19T16:51:51Z","repo":{"name":"github.com/transparency-dev/tessera","commit":"cf19b80c2dcf96d409ae75c4acf012ed02f22eea"},"scorecard":{"version":"v5.2.1","commit":"ab2f6e92482462fe66246d9e32f642855a691dc1"},"score":8.5,"checks":[{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: SECURITY.md:1","Info: Found linked content: SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1","Info: Found text in security policy: SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#security-policy"}},{"name":"Dependency-Update-Tool","score":10,"reason":"update tool detected","details":["Info: detected update tool: Dependabot: .github/dependabot.yml:1"],"documentation":{"short":"Determines if the project uses a dependency update tool.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#dependency-update-tool"}},{"name":"Code-Review","score":10,"reason":"all changesets reviewed","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#code-review"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#binary-artifacts"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":10,"reason":"30 commit(s) and 11 issue activity found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#maintained"}},{"name":"Token-Permissions","score":10,"reason":"GitHub workflow tokens follow principle of least privilege","details":["Warn: jobLevel 'deployments' permission set to 'write': .github/workflows/benchmark-go-main.yml:17","Warn: jobLevel 'contents' permission set to 'write': .github/workflows/benchmark-go-main.yml:19","Info: jobLevel 'packages' permission set to 'read': .github/workflows/codeql.yml:39","Info: jobLevel 'actions' permission set to 'read': .github/workflows/codeql.yml:42","Info: jobLevel 'contents' permission set to 'read': .github/workflows/codeql.yml:43","Info: jobLevel 'contents' permission set to 'read': .github/workflows/scorecard.yml:30","Info: jobLevel 'contents' permission set to 'read': .github/workflows/terragrunt_test.yml:16","Info: topLevel 'contents' permission set to 'read': .github/workflows/aws_integration_test.yml:16","Info: topLevel 'contents' permission set to 'read': .github/workflows/benchmark-go-main.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/benchmark-go-pr.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/benchmark.yml:6","Info: topLevel 'contents' permission set to 'read': .github/workflows/codeql.yml:23","Info: topLevel 'contents' permission set to 'read': .github/workflows/generated_files.yml:12","Info: topLevel 'contents' permission set to 'read': .github/workflows/go_test.yml:6","Info: topLevel 'contents' permission set to 'read': .github/workflows/golangci-lint.yml:8","Info: topLevel 'contents' permission set to 'read': .github/workflows/govulncheck.yml:12","Info: topLevel 'contents' permission set to 'read': .github/workflows/integration_test.yml:6","Info: topLevel 'contents' permission set to 'read': .github/workflows/scorecard.yml:18","Info: topLevel 'contents' permission set to 'read': .github/workflows/terragrunt_test.yml:6"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":10,"reason":"all dependencies are pinned","details":["Info:  23 out of  23 GitHub-owned GitHubAction dependencies pinned","Info:  11 out of  11 third-party GitHubAction dependencies pinned","Info:  10 out of  10 containerImage dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#cii-best-practices"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#signed-releases"}},{"name":"Vulnerabilities","score":8,"reason":"2 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GO-2022-0635","Warn: Project is vulnerable to: GO-2022-0646"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":9,"reason":"SAST tool detected but not run on all commits","details":["Info: SAST configuration detected: CodeQL","Warn: 29 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#sast"}},{"name":"Packaging","score":10,"reason":"packaging workflow detected","details":["Info: Project packages its releases by way of GitHub Actions.: .github/workflows/aws_integration_test.yml:29"],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#packaging"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#license"}},{"name":"Branch-Protection","score":5,"reason":"branch protection is not maximal on development and all release branches","details":["Info: 'allow deletion' disabled on branch 'main'","Info: 'force pushes' disabled on branch 'main'","Info: 'branch protection settings apply to administrators' is required to merge on branch 'main'","Warn: 'stale review dismissal' is disabled on branch 'main'","Warn: required approving review count is 1 on branch 'main'","Info: codeowner review is required on branch 'main'","Warn: 'last push approval' is disabled on branch 'main'","Info: 'up-to-date branches' is required to merge on branch 'main'","Info: status check found to merge onto on branch 'main'","Info: PRs are required in order to make changes on branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#branch-protection"}},{"name":"CI-Tests","score":10,"reason":"30 out of 30 merged PRs checked by a CI test -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project runs tests before pull requests are merged.","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#ci-tests"}},{"name":"Contributors","score":6,"reason":"project has 2 contributing companies or organizations -- score normalized to 6","details":["Info: found contributions from: google, googlers"],"documentation":{"short":"Determines if the project has a set of contributors from multiple organizations (e.g., companies).","url":"https://github.com/ossf/scorecard/blob/ab2f6e92482462fe66246d9e32f642855a691dc1/docs/checks.md#contributors"}}]},"last_synced_at":"2025-08-19T18:42:32.464Z","repository_id":247607400,"created_at":"2025-08-19T18:42:32.464Z","updated_at":"2025-08-19T18:42:32.464Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29550829,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-17T14:33:00.708Z","status":"ssl_error","status_checked_at":"2026-02-17T14:32:58.657Z","response_time":100,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-12T15:38:38.176Z","updated_at":"2026-02-17T17:02:46.372Z","avatar_url":"https://github.com/transparency-dev.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tessera\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/transparency-dev/tessera)](https://goreportcard.com/report/github.com/transparency-dev/tessera)\n[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/transparency-dev/tessera/badge)](https://scorecard.dev/viewer/?uri=github.com/transparency-dev/tessera)\n[![Benchmarks](https://img.shields.io/badge/Benchmarks-blue.svg)](https://transparency-dev.github.io/tessera/dev/bench/)\n[![Slack Status](https://img.shields.io/badge/Slack-Chat-blue.svg)](https://transparency-dev.slack.com/)\n\nTessera is a Go library for building [tile-based transparency logs (tlogs)](https://c2sp.org/tlog-tiles).\nIt is the logical successor to the approach [Trillian v1][] takes in building and operating logs.\n\nThe implementation and its APIs bake-in\n[current best-practices based on the lessons learned](https://transparency.dev/articles/tile-based-logs/)\nover the past decade of building and operating transparency logs in production environments and at scale.\n\nTessera was introduced at the Transparency.Dev summit in October 2024.\nWatch [Introducing Tessera](https://www.youtube.com/watch?v=9j_8FbQ9qSc) for all the details,\nbut here's a summary of the high level goals:\n\n*   [tlog-tiles API][] and storage\n*   Support for both cloud and on-premises infrastructure\n    *   [GCP](./storage/gcp/)\n    *   [AWS](./storage/aws/)\n    *   [MySQL](./storage/mysql/)\n    *   [POSIX](./storage/posix/)\n*   Make it easy to build and deploy new transparency logs on supported infrastructure\n    *   Library instead of microservice architecture\n    *   No additional services to manage\n    *   Lower TCO for operators compared with Trillian v1\n*   Fast sequencing and integration of entries\n*   Optional functionality which can be enabled for those ecosystems/logs which need it (only pay the cost for what you need):\n    *   \"Best-effort\" de-duplication of entries\n    *   Synchronous integration\n*   Broadly similar write-throughput and write-availability, and potentially _far_ higher read-throughput\n    and read-availability compared to Trillian v1 (dependent on underlying infrastructure)\n*   Enable building of arbitrary log personalities, including support for the peculiarities of a\n    [Static CT API][] compliant log.\n\nThe main non-goal is to support transparency logs using anything other than the [tlog-tiles API][].\nWhile it is possible to deploy a custom personality in front of Tessera that adapts the tlog-tiles API\ninto any other API, this strategy will lose a lot of the read scaling that Tessera is designed for.\n\n## Table of Contents\n\n- [Status](#status)\n- [Roadmap](#roadmap)\n- [Concepts](#concepts)\n- [Usage](#usage)\n  - [Getting Started](#getting-started)\n  - [Writing Personalities](#writing-personalities)\n- [Features](#features)\n- [Lifecycles](#lifecycles)\n- [Contributing](#contributing)\n- [License](#license)\n- [Contact](#contact)\n\n## Status\n\nTessera is under active development, and is considered production ready since the\n[Beta release](https://github.com/transparency-dev/tessera/releases/tag/v0.2.0).\nSee the table below for details.\n\n### Storage drivers\n\n| Driver                  | Appender | Migration | Antispam | Garbage Collection | Notes                                         |\n| ----------------------- | :------: | :-------: | :------: | :----------------: | --------------------------------------------- |\n| Amazon Web Services     |    ✅    |     ⚠️    |    ✅    |          ✅        |                                               |\n| Google Cloud Platform   |    ✅    |     ⚠️    |    ✅    |          ✅        |                                               |\n| POSIX filesystem        |    ✅    |     ⚠️    |    ✅    |          ✅        |                                               |\n| MySQL                   |    ⚠️    |     ⚠️    |    ❌    |          N/A       | MySQL will remain in BETA for the time being. |\n\n\n\u003e [!Note]\n\u003e Please get in touch if you are interested in using any of the features or drivers held back in BETA above.\n\nUsers of GCP, AWS, MySQL, and POSIX are welcome to try the relevant [Getting Started](#getting-started) guide.\n\n## Roadmap\n\nProduction ready around mid 2025.\n\n|  #  | Step                                                      | Status |\n| :-: | --------------------------------------------------------- | :----: |\n|  1  | Drivers for GCP, AWS, MySQL, and POSIX                    |   ✅   |\n|  2  | [tlog-tiles API][] support                                |   ✅   |\n|  3  | Example code and terraform scripts for easy onboarding    |   ✅   |\n|  4  | Stable API                                                |   ✅   |\n|  5  | Data migration between releases                           |   ✅   |\n|  6  | Data migration between drivers                            |   ✅   |\n|  7  | Witness support                                           |   ✅   |\n|  8  | Monitoring and metrics                                    |   ✅   |\n|  9  | Production ready                                          |   ✅   |\n|  10 | Mirrored logs (#576)                                      |   ⚠️   |\n|  11 | Preordered logs (#575)                                    |   ❌   |\n|  12 | Trillian v1 to Tessera migration (#577)                   |   ❌   |\n|  N  | Fancy features (to be expanded upon later)                |   ❌   |\n\nThe current API is unlikely to change in any significant way, however the API is subject to minor breaking changes until we tag 1.0.\n\n### What’s happening to Trillian v1?\n\n[Trillian v1][] is still in use in production environments by\nmultiple organisations in multiple ecosystems, and is likely to remain so for the mid-term. \n\nNew ecosystems, or existing ecosystems looking to evolve, should strongly consider planning a\nmigration to Tessera and adopting the patterns it encourages.\n\n\u003e [!Tip]\n\u003e To achieve the full benefits of Tessera, logs must use the [tlog-tiles API][].\n\n## Concepts\n\nThis section introduces concepts and terms that will be used throughout the user guide.\n\n### Sequencing\n\nWhen data is added to a log, it is first stored in memory for some period (this can be controlled via the [batching options](https://pkg.go.dev/github.com/transparency-dev/tessera#WithBatching)).\nIf the process dies in this state, the entry will be lost.\n\nOnce a batch of entries is processed by the sequencer, the new data will transition from a volatile state to one where it is durably assigned an index.\nIf the process dies in this state, the entry will be safe, though it will not be available through the read API of the log until the leaf has been [Integrated](#integration).\nOnce an index number has been issued to a leaf, no other data will ever be issued the same index number.\nAll index numbers are contiguous and start from 0.\n\n\u003e [!IMPORTANT]\n\u003e Within a batch, there is no guarantee about which order index numbers will be assigned.\n\u003e The only way to ensure that sequential calls to `Add` are given sequential indices is by blocking until a sequencing batch is completed.\n\u003e This can be achieved by configuring a batch size of 1, though this will make sequencing expensive!\n\n### Integration\n\nIntegration is a background process that happens when a Tessera lifecycle object has been created.\nThis process takes sequenced entries and merges them into the log.\nOnce this process has been completed, a new entry will:\n - Be available via the read API at the index that was returned from sequencing\n - Have Merkle tree hashes that commit to this data being included in the tree\n\n### Publishing\n\nPublishing is a background process that creates a new Checkpoint for the latest tree.\nThis background process runs periodically (configurable via\n[WithCheckpointInterval](https://pkg.go.dev/github.com/transparency-dev/tessera#AppendOptions.WithCheckpointInterval)\nand\n[WithCheckpointRepublishInterval](https://pkg.go.dev/github.com/transparency-dev/tessera#AppendOptions.WithCheckpointRepublishInterval))\nand performs the following steps:\n  1. Create a new Checkpoint and sign it with the signer provided by [WithCheckpointSigner](https://pkg.go.dev/github.com/transparency-dev/tessera#AppendOptions.WithCheckpointSigner)\n  2. Contact witnesses and collect enough countersignatures to satisfy any witness policy configured by [WithWitnesses](https://pkg.go.dev/github.com/transparency-dev/tessera#AppendOptions.WithWitnesses)\n  3. If the witness policy is satisfied, make this new Checkpoint public available\n\nAn entry is considered published once it is committed to by a published Checkpoint (i.e. a published Checkpoint's size is larger than the entry's assigned index).\nDue to the nature of append-only logs, all Checkpoints issued after this point will also commit to inclusion of this entry.\n\n## Usage\n\n### Getting Started\n\nThe best place to start is the [codelab](./cmd/conformance#codelab). \nThis will walk you through setting up your first log, writing some entries to it via HTTP, and inspecting the contents.\n\nTake a look at the example personalities in the `/cmd/` directory:\n  - [posix](./cmd/conformance/posix/): example of operating a log backed by a local filesystem\n    - This example runs an HTTP web server that takes arbitrary data and adds it to a file-based log.\n  - [mysql](./cmd/conformance/mysql/): example of operating a log that uses MySQL\n    - This example is easiest deployed via `docker compose`, which allows for easy setup and teardown.\n  - [gcp](./cmd/conformance/gcp/): example of operating a log running in GCP.\n    - This example can be deployed via terraform, see the [deployment instructions](./deployment/live/gcp/conformance#manual-deployment).\n  - [aws](./cmd/conformance/aws/): example of operating a log running on AWS.\n    - This example can be deployed via terraform, see the [deployment instructions](./deployment/live/aws/codelab#aws-codelab-deployment).\n  - [posix-oneshot](./cmd/examples/posix-oneshot/): example of a command line tool to add entries to a log stored on the local filesystem\n    - This example is not a long-lived process; running the command integrates entries into the log which lives only as files.\n\nThe `main.go` files for each of these example personalities try to strike a balance when demonstrating features of Tessera between simplicity, and demonstrating best practices.\nPlease raise issues against the repo, or chat to us in [Slack](#contact) if you have ideas for making the examples more accessible!\n\n### Writing Personalities\n\n#### Introduction\n\nTessera is a library written in Go.\nIt is designed to efficiently serve logs that allow read access via the [tlog-tiles API][].\nThe code you write that calls Tessera is referred to as a personality, because it tailors the generic library to your ecosystem.\n\nBefore starting to write your own personality, it is strongly recommended that you have familiarized yourself with the provided personalities referenced in [Getting Started](#getting-started).\nWhen writing your Tessera personality, the first decision you need to make is which of the native drivers to use:\n *   [GCP](./storage/gcp/)\n *   [AWS](./storage/aws/)\n *   [MySQL](./storage/mysql/)\n *   [POSIX](./storage/posix/)\n\nThe easiest drivers to operate and to scale are the cloud implementations: GCP and AWS.\nThese are the recommended choice for the majority of users running in production.\n\nIf you aren't using a cloud provider, then your options are MySQL and POSIX:\n- POSIX is the simplest to get started with as it needs little in the way of extra infrastructure, and\n  if you already serve static files as part of your business/project this could be a good fit.\n- Alternatively, if you are used to operating user-facing applications backed by a RDBMS, then MySQL could\n  be a natural fit.\n\nTo get a sense of the rough performance you can expect from the different backends, take a look at\n[docs/performance.md](/docs/performance.md).\n\n\n#### Setup\n\nOnce you've picked a storage driver, you can start writing your personality!\nYou'll need to import the Tessera library:\n```shell\n# This imports the library at main.\n# This should be set to the latest release version to get a stable release.\ngo get github.com/transparency-dev/tessera@main\n```\n\n#### Constructing the Appender\n\nImport the main `tessera` package, and the driver for the storage backend you want to use:\n```go file=README_test.go region=common_imports\n\t\"github.com/transparency-dev/tessera\"\n\n\t// Choose one!\n\t\"github.com/transparency-dev/tessera/storage/posix\"\n\t// \"github.com/transparency-dev/tessera/storage/aws\"\n\t// \"github.com/transparency-dev/tessera/storage/gcp\"\n\t// \"github.com/transparency-dev/tessera/storage/mysql\"\n\n```\n\nNow you'll need to instantiate the lifecycle object for the native driver you are using.\n\nBy far the most common way to operate logs is in an append-only manner, and the rest of this guide will discuss\nthis mode.\nFor lifecycle states other than Appender mode, take a look at [Lifecycles](#lifecycles) below.\n\nHere's an example of creating an `Appender` for the POSIX driver:\n```go file=README_test.go region=construct_example\n\tdriver, _ := posix.New(ctx, \"/tmp/mylog\")\n\tsigner := createSigner()\n\n\tappender, shutdown, reader, err := tessera.NewAppender(\n\t\tctx, driver, tessera.NewAppendOptions().WithCheckpointSigner(signer))\n```\n\nSee the documentation for each driver implementation to understand the parameters that each takes.\n\nThe final part of configuring Tessera is to set up the addition features that you want to use.\nThese optional libraries can be used to provide common log behaviours.\nSee [Features](#features) after reading the rest of this section for more details.\n\n#### Writing to the Log\n\nNow you should have a Tessera instance configured for your environment with the correct features set up.\nNow the fun part - writing to the log!\n\n```go file=README_test.go region=use_appender_example\n\tappender, shutdown, reader, err := tessera.NewAppender(\n\t\tctx, driver, tessera.NewAppendOptions().WithCheckpointSigner(signer))\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tindex, err := appender.Add(ctx, tessera.NewEntry(data))()\n```\n\nThe `AppendOptions` allow Tessera behaviour to be tuned.\nTake a look at the methods named `With*` on the `AppendOptions` struct in the root package, e.g. [`WithBatching`](https://pkg.go.dev/github.com/transparency-dev/tessera@main#AppendOptions.WithBatching) to see the available options are how they should be used.\n\n\u003e [!Tip]\n\u003e If you know ahead of time how many entries you want to add (e.g. if you're writing an \"off-line\" or batch tool,\n\u003e which will process entries and then exit), you can pass that value as a performance hint to Tessera via the\n\u003e _size_ parameter of `WithBatching`.\n \nWriting to the log follows this flow:\n 1. Call `Add` with a new entry created with the data to be added as a leaf in the log.\n    - This method returns a _future_ of the form `func() (Index, error)`.\n 2. Call this future function, which will block until the data passed into `Add` has been sequenced\n    - On success, an index number is _durably_ assigned and returned\n    - On failure, the error is returned\n    \nOnce an index has been returned, the new data is sequenced, but not necessarily integrated into the log.\n\nAs discussed above in [Integration](#integration), sequenced entries will be _asynchronously_ integrated into the log and be made available via the read API.\nSome personalities may need to block until this has been performed, e.g. because they will provide the requester with an inclusion proof, which requires integration.\nSuch personalities are recommended to use [Synchronous Publication](#synchronous-publication) to perform this blocking.\n\n#### Reading from the Log\n\nData that has been written to the log needs to be made available for clients and verifiers.\nTessera makes the log readable via the [tlog-tiles API][].\nIn the case of AWS and GCP, the data to be served is written to object storage and served directly by the cloud provider.\nThe log operator only needs to ensure that these object storage instances are publicly readable, and set up a URL to point to them.\n\nIn the case of MySQL and POSIX, the log operator will need to take more steps to make the data available.\nPOSIX writes out the files exactly as per the API spec, so the log operator can serve these via an HTTP File Server.\n\nMySQL is the odd implementation in that it requires personality code to handle read traffic.\nSee the example personalities written for MySQL to see how this Go web server should be configured.\n\n## Features\n\n### Antispam\n\nIn some scenarios, particularly where logs are publicly writable such as Certificate Transparency, it's possible for logs to be asked,\nwhether maliciously or accidentally, to add entries they already contain. Generally, this is undesirable, and so Tessera provides an\noptional mechanism to try to detect and ignore duplicate entries on a best-effort basis.\n\nLogs that do not allow public submissions directly to the log may want to operate without this optional antispam measure, instead relying on the\npersonality to never generate duplicates. This can allow for significantly cheaper operation and faster write throughput.\n\nThe antispam mechanism consists of two layers which sit in front of the underlying `Add` implementation of the storage:\n1. The first layer is an `InMemory` cache which keeps track of a configurable number of recently-added entries.\n   If a recently-seen entry is spotted by the same application instance, this layer will short-circuit the addition\n   of the duplicate, and instead return and index previously assigned to this entry. Otherwise the requested entry is\n   passed on to the second layer.\n2. The second layer is a `Persistent` index of a hash of the entry to its assigned position in the log.\n   Similarly to the first layer, this second layer will look for a record in its stored data which matches the incoming\n   entry, and if such a record exists, it will short-circuit the addition of the duplicate entry and return a previous\n   version's assigned position in the log.\n\nThese layes are configured by the `WithAntispam` method of the\n[AppendOptions](https://pkg.go.dev/github.com/transparency-dev/tessera@main#AppendOptions.WithAntispam) and\n[MigrateOptions](https://pkg.go.dev/github.com/transparency-dev/tessera@main#AppendOptions.WithAntispam).\n\n\u003e [!Tip]\n\u003e Persistent antispam is fairly expensive in terms of storage-compute, so should only be used where it is actually necessary.\n\n\u003e [!Note]\n\u003e Tessera's antispam mechanism is _best effort_; there is no guarantee that all duplicate entries will be suppressed.\n\u003e This is a trade-off; fully-atomic \"strong\" de-duplication is _extremely_ expensive in terms of throughput and compute costs, and\n\u003e would limit Tessera to only being able to use transactional type storage backends.\n\n### Witnessing\n\nLogs are required to be append-only data structures.\nThis property can be verified by witnesses, and signatures from witnesses can be provided in the published checkpoint to increase confidence for users of the log.\n\nPersonalities can configure Tessera with options that specify witnesses compatible with the [C2SP Witness Protocol](https://github.com/C2SP/C2SP/blob/main/tlog-witness.md).\nConfiguring the witnesses is done by either using the [`NewWitnessGroupFromPolicy`](https://pkg.go.dev/github.com/transparency-dev/tessera@main#NewWitnessGroupFromPolicy)\nhelper, or programatically creating a top-level [`WitnessGroup`](https://pkg.go.dev/github.com/transparency-dev/tessera@main#WitnessGroup) that contains either\nsub `WitnessGroup`s, or [`Witness`es](https://pkg.go.dev/github.com/transparency-dev/tessera@main#Witness).\n\nEach `Witness` is configured with a URL at which the witness can be reached, and a `Verifier` for the key that it must sign with.\n`WitnessGroup`s are configured with their sub-components, and a number of these components that must be satisfied in order for the group to be satisfied.\n\nThese primitives allow arbitrarily complex witness policies to be specified.\n\nOnce a top-level `WitnessGroup` is configured, it is passed in to the `Appender` lifecycle options using\n[AppendOptions#WithWitnesses](https://pkg.go.dev/github.com/transparency-dev/tessera@main#AppendOptions.WithWitnesses).\nIf this option is not set, no witnessing will be configured.\n\n\u003e [!Note]\n\u003e If the policy cannot be satisfied then no checkpoint will be published.\n\u003e It is up to the log operator to ensure that a satisfiable policy is configured, and that the requested publishing rate is acceptable to the configured witnesses.\n\n### Synchronous Publication\n\nSynchronous Publication is provided by [`tessera.PublicationAwaiter`](https://pkg.go.dev/github.com/transparency-dev/tessera#PublicationAwaiter).\nThis allows applications built with Tessera to block until leaves passed via calls to `Add()` are committed to via a public checkpoint.\n\n\u003e [!Tip]\n\u003e This is useful if e.g. your application needs to return an inclusion proof in response to a request to add an entry to the log.\n\n## Lifecycles\n\n### Appender\n\nThis is the most common lifecycle mode. Appender allows the application to add leaves, which will be assigned positions in the log\ncontiguous to any entries the log has already committed to.\n\nThis mode is instantiated via [`tessera.NewAppender`](https://pkg.go.dev/github.com/transparency-dev/tessera@main#NewAppender), and\nconfigured using the [`tessera.NewAppendOptions`](https://pkg.go.dev/github.com/transparency-dev/tessera@main#NewAppendOptions) struct.\n\nThis is described above in [Constructing the Appender](#constructing-the-appender).\n\nNote that entries are limited to 64KB in size by the [tlog-tiles][] spec, with the exception that when Tessera is configured for use\nwith Static CT via the `WithCTLayout` option, entries are then limited to 256KB.\n\nSee more details in the [Lifecycle Design: Appender](https://github.com/transparency-dev/tessera/blob/main/docs/design/lifecycle.md#appender).\n\n### Migration Target\n\nThis mode is used to migrate a log from one location to another.\n\nThis is instantiated via [`tessera.NewMigrationTarget`](https://pkg.go.dev/github.com/transparency-dev/tessera@main#NewMigrationTarget),\nand configured using the [`tessera.NewMigratonOptions`](https://pkg.go.dev/github.com/transparency-dev/tessera@main#NewMigrationOptions) struct.\n\n\u003e [!Tip]\n\u003e This mode enables the migration of logs between different Tessera storage backends, e.g. you may wish to switch\n\u003e serving infrastructure because:\n\u003e    * You're migrating between/to/from cloud providers for some reason.\n\u003e    * You're \"freezing\" your log, and want to move it to a cheap read-only location.\n\u003e\n\u003e You can also use this mode to migrate a [tlog-tiles][] compliant log _into_ Tessera.\n\nBinaries for migrating _into_ each of the storage implementations can be found at [./cmd/experimental/migrate/](./cmd/experimental/migrate/).\nThese binaries take the URL of a remote tiled log, and copy it into the target location.\nThese binaries ought to be sufficient for most use-cases.\nUsers that need to write their own migration binary should use the provided binaries as a reference codelab.\n\nSee more details in the [Lifecycle Design: Migration](https://github.com/transparency-dev/tessera/blob/main/docs/design/lifecycle.md#migration).\n\n### Freezing a Log\n\nFreezing a log prevents new writes to the log, but still allows read access.\nWe recommend that operators allow all pending [sequenced](#sequencing) entries to be [integrated](#integration), and all integrated entries to be [published](#publishing) via a Checkpoint before proceeding.\nOnce all pending entries are published, the log is now _quiescent_, as described in [Lifecycle Design: Quiescent](https://github.com/transparency-dev/tessera/blob/main/docs/design/lifecycle.md#quiescent).\n\nTo ensure all pending entries are published, keep an instance object for the current lifecycle state in a running process, but disable writes to this at the personality level.\nFor example, a personality that takes HTTP requests from the Internet and calls `Appender.Add` should keep a process running with an `Appender`, but disable any code paths that lead to `Add` being invoked (e.g. by flipping a flag that changes this behaviour).\nThe instantiated `Appender` allows its background processes to keep running, ensuring all entries are sequenced, integrated, and published.\n\nDetermining when this is complete can be done by inspecting the databases or via the OpenTelemetry metrics which instrument this code;\nonce the next-available sequence number and published checkpoint size have converged and remain stable, the log is in a quiescent state.\n\nA quiescent log using GCP, AWS, or POSIX that is now permanently read-only can be made cheaper to operate. The implementations no longer need any running binaries running Tessera code. Any databases created for this log (i.e. the sequencing tables, or antispam) can be deleted. The read-path can be served directly from the storage buckets (for GCP, AWS) or via a standard HTTP file server (for POSIX).\n\nA log using MySQL must continue to run a personality in order to serve the read path, and thus cannot benefit from the same degree of cost savings when frozen.\n\n### Deleting a Log\n\nDeleting a log is generally performed after [Freezing a Log](#freezing-a-log).\n\nDeleting a GCP, AWS, or POSIX log that has already been frozen just requires deleting the storage bucket or files from disk.\n\nDeleting a MySQL log can be done by turning down the personality binaries, and then deleting the database.\n\n### Sharding a Log\n\nA common way to deploy logs is to run multiple logs in parallel, each of which accepts a distinct subset of entries.\nFor example, CT shards logs temporally, based on the expiry date of the certificate.\n\nTessera currently has no special support for sharding logs.\nThe recommended way to instantiate a new shard of a log is simply to create a new log as described above.\nThis requires the full stack to be instantiated, including:\n - any DB instances\n - a personality binary for each log\n\n#589 tracks adding more elegant support for sharing resources for sharded logs.\nPlease upvote that issue if you would like us to prioritize it.\n\n## Contributing\n\nSee [CONTRIBUTING.md](/CONTRIBUTING.md) for details.\n\n## License\n\nThis repo is licensed under the Apache 2.0 license, see [LICENSE](/LICENSE) for details.\n\n## Contact\n\n- Slack: https://transparency-dev.slack.com/ ([invitation](https://transparency.dev/slack/))\n- Mailing list: https://groups.google.com/forum/#!forum/trillian-transparency\n\n## Acknowledgements\n\nTessera builds upon the hard work, experience, and lessons from many _many_ folks involved in\ntransparency ecosystems over the years.\n\n[tlog-tiles API]: https://c2sp.org/tlog-tiles\n[Static CT API]: https://c2sp.org/static-ct-api\n[Trillian v1]: https://github.com/google/trillian\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftransparency-dev%2Ftessera","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftransparency-dev%2Ftessera","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftransparency-dev%2Ftessera/lists"}