{"id":45591761,"url":"https://github.com/openjobspec/ojs-conformance","last_synced_at":"2026-02-28T17:07:55.529Z","repository":{"id":338056538,"uuid":"1155816708","full_name":"openjobspec/ojs-conformance","owner":"openjobspec","description":"Language-agnostic conformance test suite for validating OJS implementations across all levels.","archived":false,"fork":false,"pushed_at":"2026-02-23T10:03:30.000Z","size":261,"stargazers_count":0,"open_issues_count":3,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-23T19:28:25.119Z","etag":null,"topics":["conformance","go","golang","ojs","openjobspec","specification","testing"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openjobspec.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-11T23:53:47.000Z","updated_at":"2026-02-18T20:13:58.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/openjobspec/ojs-conformance","commit_stats":null,"previous_names":["openjobspec/ojs-conformance"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/openjobspec/ojs-conformance","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openjobspec%2Fojs-conformance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openjobspec%2Fojs-conformance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openjobspec%2Fojs-conformance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openjobspec%2Fojs-conformance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openjobspec","download_url":"https://codeload.github.com/openjobspec/ojs-conformance/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openjobspec%2Fojs-conformance/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29943809,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-28T13:49:17.081Z","status":"ssl_error","status_checked_at":"2026-02-28T13:48:50.396Z","response_time":90,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["conformance","go","golang","ojs","openjobspec","specification","testing"],"created_at":"2026-02-23T12:42:15.195Z","updated_at":"2026-02-28T17:07:55.080Z","avatar_url":"https://github.com/openjobspec.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OJS Conformance Test Suite\n\n[![CI](https://github.com/openjobspec/ojs-conformance/actions/workflows/ci.yml/badge.svg)](https://github.com/openjobspec/ojs-conformance/actions/workflows/ci.yml)\n[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)\n\nLanguage-agnostic conformance tests for [Open Job Spec (OJS)](https://github.com/openjobspec/spec) implementations.\n\n## Overview\n\nThis test suite validates that job queue implementations conform to the OJS specification. Tests are defined as JSON files describing HTTP interactions and expected outcomes, making them runnable against any OJS-conformant server regardless of implementation language.\n\n## Structure\n\n```\nojs-conformance/\n  suites/                          # Test case JSON files\n    level-0-core/                  # Level 0: Core conformance\n      envelope/                    # Job envelope validation\n      events/                      # Lifecycle event emission\n      lifecycle/                   # State machine transitions\n      operations/                  # PUSH/FETCH/ACK/FAIL/CANCEL/INFO/manifest/health\n    level-1-reliable/              # Level 1: Reliable delivery\n      retry/                       # Retry policies, backoff \u0026 error history\n      dead-letter/                 # Dead letter queue\n      timeout/                     # Execution timeout handling\n      visibility/                  # Visibility timeout\n      worker/                      # Worker heartbeat \u0026 signals\n    level-2-scheduled/             # Level 2: Scheduled execution\n      delay/                       # Delayed jobs (scheduled_at)\n      cron/                        # Cron scheduling\n      ttl/                         # Job expiration (expires_at)\n    level-3-workflows/             # Level 3: Orchestration\n      chain/                       # Sequential execution\n      group/                       # Parallel execution\n      batch/                       # Parallel with callbacks\n    level-4-advanced/              # Level 4: Advanced features\n      priority/                    # Priority queues\n      unique/                      # Job deduplication\n      rate-limit/                  # Rate limiting\n      bulk/                        # Batch enqueue\n      queue-ops/                   # Queue pause/resume/stats\n    ext-admin-api/                 # Extension: Admin API\n    ext-backpressure/              # Extension: Backpressure\n    ext-dead-letter/               # Extension: Dead letter (extended)\n    ext-rate-limiting/             # Extension: Rate limiting (extended)\n  runner/                          # Test runner implementations\n    http/                          # Go-based HTTP test runner\n    grpc/                          # Go-based gRPC test runner (scaffold)\n  lib/                             # Shared library code\n```\n\n## Conformance Levels\n\n| Level | Name | Description |\n|-------|------|-------------|\n| 0 | Core | Job envelope, 8-state lifecycle, PUSH/FETCH/ACK/FAIL, error catalog, manifest, health |\n| 1 | Reliable | Retry with backoff, dead letter queue, heartbeat, visibility timeout, execution timeout, error history |\n| 2 | Scheduled | Delayed jobs, cron scheduling, job expiration |\n| 3 | Orchestration | Chain, group, batch workflows |\n| 4 | Advanced | Priority queues, unique jobs, batch enqueue, queue ops |\n\nEach level builds upon the previous. A Level 2 implementation must also pass all Level 0 and Level 1 tests.\n\n## Quick Start\n\n### 1. Start your OJS server\n\nEnsure your implementation is running and accessible via HTTP.\n\n### 2. Build the test runner\n\n```bash\ncd runner/http\ngo build -o ojs-conformance-runner .\n```\n\n### 3. Run tests\n\n```bash\n# Run all tests\n./ojs-conformance-runner -url http://localhost:8080 -suites ../../suites\n\n# Run only Level 0 tests\n./ojs-conformance-runner -url http://localhost:8080 -suites ../../suites -level 0\n\n# Run a specific category\n./ojs-conformance-runner -url http://localhost:8080 -suites ../../suites -category envelope\n\n# Output as JSON\n./ojs-conformance-runner -url http://localhost:8080 -suites ../../suites -output json\n```\n\n## Test Server Requirements\n\nYour OJS implementation must register these standard test handlers:\n\n| Handler | Behavior |\n|---------|----------|\n| `test.echo` | Returns arguments as result |\n| `test.fail_once` | Fails first attempt, succeeds second |\n| `test.fail_twice` | Fails first two attempts, succeeds third |\n| `test.fail_always` | Always fails with retryable error |\n| `test.slow` | Sleeps for configurable duration |\n| `test.timeout` | Exceeds any configured timeout |\n| `test.panic` | Crashes the handler |\n| `test.produce` | Returns a configurable result value |\n| `test.noop` | Succeeds immediately |\n\n## Conformance Report\n\nThe test runner produces a conformance report:\n\n```json\n{\n  \"test_suite_version\": \"1.0\",\n  \"target\": \"http://localhost:8080\",\n  \"run_at\": \"2026-02-12T10:30:00Z\",\n  \"duration_ms\": 45230,\n  \"requested_level\": -1,\n  \"results\": {\n    \"total\": 85,\n    \"passed\": 83,\n    \"failed\": 2,\n    \"skipped\": 0\n  },\n  \"conformant\": false,\n  \"conformant_level\": 1\n}\n```\n\n## Documentation\n\n- **[Test Case Reference](docs/test-case-reference.md)** — Complete reference for the test DSL: every field, matcher, operator, JSONPath syntax, template references, and timing assertions.\n\n## Contributing\n\nSee [CONTRIBUTING.md](../CONTRIBUTING.md) for guidelines on adding tests.\n\n## License\n\nApache License 2.0 - see [LICENSE](LICENSE).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenjobspec%2Fojs-conformance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenjobspec%2Fojs-conformance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenjobspec%2Fojs-conformance/lists"}