{"id":31601700,"url":"https://github.com/intersectmbo/uplc-cape","last_synced_at":"2025-10-06T07:58:55.615Z","repository":{"id":309521061,"uuid":"1020731128","full_name":"IntersectMBO/UPLC-CAPE","owner":"IntersectMBO","description":"Comparative Artifact Performance Evaluation","archived":false,"fork":false,"pushed_at":"2025-09-12T16:42:37.000Z","size":2063,"stargazers_count":2,"open_issues_count":3,"forks_count":3,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-12T18:53:14.205Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Untyped Plutus Core","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/IntersectMBO.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-07-16T10:01:54.000Z","updated_at":"2025-09-12T16:42:42.000Z","dependencies_parsed_at":"2025-08-28T13:26:29.867Z","dependency_job_id":"6eaff6fc-7979-4b18-a95d-9a8e47526eb2","html_url":"https://github.com/IntersectMBO/UPLC-CAPE","commit_stats":null,"previous_names":["intersectmbo/uplc-cape"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/IntersectMBO/UPLC-CAPE","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IntersectMBO%2FUPLC-CAPE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IntersectMBO%2FUPLC-CAPE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IntersectMBO%2FUPLC-CAPE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IntersectMBO%2FUPLC-CAPE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/IntersectMBO","download_url":"https://codeload.github.com/IntersectMBO/UPLC-CAPE/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/IntersectMBO%2FUPLC-CAPE/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278577931,"owners_count":26009701,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-06T02:00:05.630Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-06T07:58:50.255Z","updated_at":"2025-10-06T07:58:55.604Z","avatar_url":"https://github.com/IntersectMBO.png","language":"Untyped Plutus Core","funding_links":[],"categories":[],"sub_categories":[],"readme":"# UPLC-CAPE\n\n_**C**omparative **A**rtifact **P**erformance **E**valuation for UPLC programs_\n\n\u003cimg src=\"uplc-cape-logo.jpg\" alt=\"UPLC-CAPE Logo\" width=\"300\" align=\"left\" style=\"margin-right: 20px;\"\u003e\n\n\u003cbr /\u003e\u003cbr /\u003e\u003cbr /\u003e\n\nA framework for measuring and comparing UPLC programs generated by different Cardano smart contract compilers.\n\n\u003cbr /\u003e\n\n[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) [![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](CONTRIBUTING.md)\n\n\u003cbr clear=\"left\"\u003e\n\n---\n\n## Table of Contents\n\n- [Overview](#overview)\n- [Quick Start](#quick-start)\n  - [Prerequisites](#prerequisites)\n  - [Setup](#setup)\n  - [Your first benchmark](#your-first-benchmark)\n- [Live Performance Reports](#live-performance-reports)\n- [Available benchmark scenarios](#available-benchmark-scenarios)\n- [Usage (CLI)](#usage-cli)\n  - [Core commands](#core-commands)\n- [Creating a Submission](#creating-a-submission)\n- [Metrics Explained](#metrics-explained)\n- [Project Structure](#project-structure)\n- [Resources](#resources)\n- [Version and Tooling Requirements](#version-and-tooling-requirements)\n- [Development](#development)\n- [Documentation (ADRs)](#documentation-adrs)\n- [Contributing](#contributing)\n- [License](#license)\n- [Acknowledgments](#acknowledgments)\n\n---\n\n## Overview\n\nUPLC-CAPE provides a structured, reproducible way for Cardano UPLC compilers authors and users to:\n\n- Benchmark compiler UPLC output against standardized scenarios\n- Compare results across compilers and versions\n- Track optimization progress over time\n- Share results with the community\n\nKey properties:\n\n- Consistent benchmarks and metrics (CPU units, memory units, script size, term size)\n- Reproducible results with versioned scenarios and metadata\n- Automation-ready structure for future tooling\n\n---\n\n## Quick Start\n\n### Prerequisites\n\n- Nix with flakes enabled\n- Git\n\n### Setup\n\n```zsh\n# Clone and enter repository\ngit clone https://github.com/IntersectMBO/UPLC-CAPE.git\ncd UPLC-CAPE\n\n# Enter development environment\nnix develop\n# Or, if using direnv (recommended)\ndirenv allow\n\n# Verify CLI\nscripts/cape.sh --help\n# Or use the cape shim if available in PATH\ncape --help\n```\n\n### Your first benchmark\n\n```zsh\n# List available benchmarks\ncape benchmark list\n\n# View a specific benchmark\ncape benchmark fibonacci\ncape benchmark two-party-escrow\n\n# Generate JSON statistics for all benchmarks\ncape benchmark stats\n\n# Create a submission for your compiler\ncape submission new fibonacci MyCompiler 1.0.0 myhandle\ncape submission new two-party-escrow MyCompiler 1.0.0 myhandle\n```\n\n---\n\n## Live Performance Reports\n\nLatest benchmark reports: [UPLC-CAPE Reports](https://intersectmbo.github.io/UPLC-CAPE/)\n\n---\n\n## Available benchmark scenarios\n\n| Benchmark | Type | Description | Status |\n| --- | --- | --- | --- |\n| [Fibonacci](scenarios/fibonacci.md) | Synthetic | Recursive algorithm performance | Ready |\n| [Factorial](scenarios/factorial.md) | Synthetic | Recursive algorithm performance | Ready |\n| [Two-Party Escrow](scenarios/two-party-escrow.md) | Real-world | Smart contract escrow validator | Ready |\n| Streaming Payments | Real-world | Payment channel implementation | Planned |\n| Simple DAO Voting | Real-world | Governance mechanism | Planned |\n| Time-locked Staking | Real-world | Staking protocol | Planned |\n\n---\n\n## Usage (CLI)\n\nFor the full and up-to-date command reference, see [USAGE.md](USAGE.md).\n\n### Core commands\n\n```zsh\n# Benchmarks\ncape benchmark list              # List all benchmarks\ncape benchmark \u003cname\u003e            # Show benchmark details\ncape benchmark stats             # Generate JSON statistics for all benchmarks\ncape benchmark new \u003cname\u003e        # Create a new benchmark from template\n\n# Submissions\ncape submission list             # List all submissions\ncape submission list \u003cname\u003e      # List submissions for a benchmark\ncape submission new \u003cbenchmark\u003e \u003ccompiler\u003e \u003cversion\u003e \u003chandle\u003e\ncape submission verify           # Verify correctness and validate schemas\ncape submission measure          # Measure UPLC performance\ncape submission aggregate        # Generate CSV performance report\ncape submission report \u003cname\u003e    # Generate HTML report for a benchmark\ncape submission report --all     # Generate HTML reports for all benchmarks\n```\n\n### JSON Statistics\n\nThe `cape benchmark stats` command generates comprehensive JSON data for all benchmarks:\n\n```zsh\n# Output JSON statistics to console\ncape benchmark stats\n\n# Save to file\ncape benchmark stats \u003e stats.json\n\n# Use with jq for filtering\ncape benchmark stats | jq '.benchmarks[] | select(.submission_count \u003e 0)'\n```\n\nThe output includes formatted metrics, best value indicators, and submission metadata, making it ideal for generating custom reports or integrating with external tools.\n\n---\n\n## Creating a Submission\n\n1. Choose a benchmark\n\n   ```zsh\n   cape benchmark list\n   cape benchmark fibonacci\n   ```\n\n1. Create submission structure\n\n   ```zsh\n   cape submission new fibonacci MyCompiler 1.0.0 myhandle\n   # → submissions/fibonacci/MyCompiler_1.0.0_myhandle/\n   ```\n\n1. Add your UPLC program\n\n   - Replace the placeholder UPLC with your fully-applied program (no parameters).\n   - Path:\n     - submissions/fibonacci/MyCompiler_1.0.0_myhandle/fibonacci.uplc\n   - The program should compute the scenario’s required result deterministically within budget.\n\n1. Verify and measure\n\n   Use the unified verification command to ensure your submission is correct and schema-compliant, then measure performance.\n\n   - Verify correctness and JSON schemas (all submissions or a path):\n\n     ```zsh\n     cape submission verify submissions/fibonacci/MyCompiler_1.0.0_myhandle\n     # or, verify everything\n     cape submission verify --all\n     ```\n\n   - Measure and write metrics.json automatically:\n\n     - Measure all .uplc files under a path (e.g., your submission directory):\n\n       ```zsh\n       cape submission measure submissions/fibonacci/MyCompiler_1.0.0_myhandle\n       # or, from inside the submission directory\n       cape submission measure .\n       ```\n\n     - Measure every submission under submissions/:\n\n       ```zsh\n       cape submission measure --all\n       ```\n\n   - What verification does:\n\n     - Evaluates your UPLC program; if it reduces to BuiltinUnit, correctness passes\n     - Otherwise, runs the comprehensive test suite defined in `scenarios/{benchmark}/cape-tests.json`\n     - Validates your `metrics.json` and `metadata.json` against schemas\n\n   - What measure does automatically:\n\n     - Measures CPU units, memory units, script size, and term size for your .uplc file(s)\n     - Generates or updates a `metrics.json` with scenario, measurements, evaluator, and timestamp\n     - Keeps your existing `notes` and `version` if present; otherwise fills sensible defaults\n     - Works for a single file, a directory, or all submissions with `--all`\n     - Produces output that validates against `submissions/TEMPLATE/metrics.schema.json`\n\n   - **Aggregation Strategies**: The `measure` tool now runs multiple test cases per program and provides several aggregation methods for CPU and memory metrics:\n\n     - `maximum`: Peak resource usage across all test cases (useful for identifying worst-case performance)\n     - `sum`: Total computational work across all test cases (useful for overall efficiency comparison)\n     - `minimum`: Best-case resource usage (useful for identifying optimal performance)\n     - `median`: Typical resource usage (useful for understanding normal performance)\n     - `sum_positive`: Total resources for successful test cases only (valid execution cost)\n     - `sum_negative`: Total resources for failed test cases only (error handling cost)\n\n     Higher-level tooling can extract the most relevant aggregation for specific analysis needs.\n\n   - Resulting file example:\n\n     ```json\n     {\n       \"scenario\": \"fibonacci\",\n       \"version\": \"1.0.0\",\n       \"measurements\": {\n         \"cpu_units\": {\n           \"maximum\": 185916,\n           \"sum\": 185916,\n           \"minimum\": 185916,\n           \"median\": 185916,\n           \"sum_positive\": 185916,\n           \"sum_negative\": 0\n         },\n         \"memory_units\": {\n           \"maximum\": 592,\n           \"sum\": 592,\n           \"minimum\": 592,\n           \"median\": 592,\n           \"sum_positive\": 592,\n           \"sum_negative\": 0\n         },\n         \"script_size_bytes\": 1234,\n         \"term_size\": 45\n       },\n       \"evaluations\": [\n         {\n           \"name\": \"fibonacci_25_computation\",\n           \"description\": \"Pre-applied fibonacci(25) should return 75025\",\n           \"cpu_units\": 185916,\n           \"memory_units\": 592,\n           \"execution_result\": \"success\"\n         }\n       ],\n       \"execution_environment\": {\n         \"evaluator\": \"plutus-core-executable-1.52.0.0\"\n       },\n       \"timestamp\": \"2025-01-15T00:00:00Z\",\n       \"notes\": \"Optional notes.\"\n     }\n     ```\n\n1. Provide metadata\n\n   Create `metadata.json` according to `submissions/TEMPLATE/metadata.schema.json` (see also `metadata-template.json`).\n\n   ```json\n   {\n     \"compiler\": {\n       \"name\": \"MyCompiler\",\n       \"version\": \"1.0.0\"\n     },\n     \"compilation_config\": {\n       \"target\": \"uplc\",\n       \"optimization_level\": \"O2\",\n       \"flags\": []\n     },\n     \"contributor\": {\n       \"name\": \"myhandle\"\n     },\n     \"submission\": {\n       \"date\": \"2025-01-15T00:00:00Z\",\n       \"source_available\": false,\n       \"implementation_notes\": \"Brief explanation of approach.\"\n     }\n   }\n   ```\n\n1. Document\n   - Add notes to README.md inside your submission folder (implementation choices, optimizations, caveats).\n\n---\n\n## Metrics Explained\n\n| Metric       | Description                         | Measurement        |\n| ------------ | ----------------------------------- | ------------------ |\n| CPU Units    | Computational cost for execution    | CEK machine steps  |\n| Memory Units | Memory consumption during execution | CEK machine memory |\n| Script Size  | Size of serialized UPLC script      | Bytes              |\n| Term Size    | Size of the UPLC term               | AST nodes          |\n\n---\n\n## Project Structure\n\n```text\nUPLC-CAPE/\n├── scenarios/                    # Benchmark specifications\n│   ├── TEMPLATE/                 # Template for new scenarios\n│   ├── fibonacci.md\n│   ├── factorial.md\n│   └── two-party-escrow.md\n├── submissions/                  # Compiler submissions (per scenario)\n│   ├── TEMPLATE/                 # Templates and schemas\n│   │   ├── metadata.schema.json\n│   │   ├── metadata-template.json\n│   │   ├── metrics.schema.json\n│   │   └── metrics-template.json\n│   ├── fibonacci/\n│   │   └── MyCompiler_1.0.0_handle/\n│   └── two-party-escrow/\n│       └── MyCompiler_1.0.0_handle/\n├── scripts/                      # Project CLI tooling\n│   ├── cape.sh                   # Main CLI\n│   └── cape-subcommands/         # Command implementations\n├── lib/                          # Haskell library code (validators, fixtures, utilities)\n├── measure-app/                  # UPLC program measurement tool\n├── plinth-submissions-app/       # Plinth submission generator\n├── test/                         # Test suites\n├── report/                       # Generated HTML reports and assets\n├── doc/                          # Documentation\n│   ├── domain-model.md\n│   └── adr/\n└── README.md\n```\n\n---\n\n## Resources\n\n- [Log4brains](https://github.com/thomvaill/log4brains)\n- [Architecture Decision Records](http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions)\n\n---\n\n## Version and Tooling Requirements\n\n- Development environment: Nix shell (`nix develop`) with optional direnv (`direnv allow`).\n- GHC: 9.6.6 (provided in Nix shell).\n- Plutus Core target: 1.1.0.\n  - Use `plcVersion110` (for Haskell/PlutusTx code).\n- Package baselines (CHaP):\n  - plutus-core \u003e= 1.52.0.0\n  - plutus-tx \u003e= 1.52.0.0\n  - plutus-ledger-api \u003e= 1.52.0.0\n  - plutus-tx-plugin \u003e= 1.52.0.0\n\n---\n\n## Development\n\nEnter environment:\n\n```zsh\nnix develop\n# or\ndirenv allow\n```\n\nCommon tools:\n\n- cape … (project CLI)\n- cabal build (builds all Haskell components: library, executables, tests)\n- treefmt (format all files)\n- fourmolu (Haskell formatting)\n- adr (Architecture Decision Records)\n- mmdc -i file.mmd (diagram generation, if available)\n\n---\n\n## Documentation (ADRs)\n\nADRs document important design decisions (managed with Log4brains).\n\nHelpful commands:\n\n```zsh\nadr new \"Decision Title\"\nadr preview\nadr build\nadr help\n```\n\n---\n\n## Contributing\n\nWe welcome contributions from compiler authors, benchmark designers, and researchers.\n\n- Add a new benchmark:\n\n  ```zsh\n  cape benchmark new my-new-benchmark\n  # edit scenarios/my-new-benchmark.md\n  ```\n\n- Add a submission:\n\n  ```zsh\n  cape submission new existing-benchmark MyCompiler 1.0.0 myhandle\n  # fill uplc and json files, then open a PR\n  ```\n\nPlease read [CONTRIBUTING.md](CONTRIBUTING.md) before opening a PR.\n\n---\n\n## License\n\nLicensed under the Apache License 2.0. See [LICENSE](LICENSE).\n\n---\n\n## Acknowledgments\n\n- Plutus Core team for infrastructure and reference implementations\n- Compiler authors and community contributors\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fintersectmbo%2Fuplc-cape","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fintersectmbo%2Fuplc-cape","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fintersectmbo%2Fuplc-cape/lists"}