{"id":50494177,"url":"https://github.com/sashakolpakov/bayesilisk","last_synced_at":"2026-06-02T05:30:46.841Z","repository":{"id":358187701,"uuid":"1240287580","full_name":"sashakolpakov/bayesilisk","owner":"sashakolpakov","description":"Deterministic local layer for permission, entitlement, route, and data-boundary sitting over Playwright, with Grassmann attention, and LLM-generated scenario-proposal workflows gated by a finite-state verifier.","archived":false,"fork":false,"pushed_at":"2026-05-30T20:12:58.000Z","size":4066,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-30T22:10:39.314Z","etag":null,"topics":["bayesian-data-analysis","bayesian-methods","data-boundaries","deterministic-finite-automaton","entitlement","grassmann-manifold","grassmannian","permission","playwright","route","scenario-generation","scenario-progression"],"latest_commit_sha":null,"homepage":"https://sashakolpakov.github.io/bayesilisk/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sashakolpakov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-16T00:59:35.000Z","updated_at":"2026-05-30T20:13:01.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sashakolpakov/bayesilisk","commit_stats":null,"previous_names":["sashakolpakov/bayesilisk"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sashakolpakov/bayesilisk","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sashakolpakov%2Fbayesilisk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sashakolpakov%2Fbayesilisk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sashakolpakov%2Fbayesilisk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sashakolpakov%2Fbayesilisk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sashakolpakov","download_url":"https://codeload.github.com/sashakolpakov/bayesilisk/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sashakolpakov%2Fbayesilisk/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33808702,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-02T02:00:07.132Z","response_time":109,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayesian-data-analysis","bayesian-methods","data-boundaries","deterministic-finite-automaton","entitlement","grassmann-manifold","grassmannian","permission","playwright","route","scenario-generation","scenario-progression"],"created_at":"2026-06-02T05:30:46.550Z","updated_at":"2026-06-02T05:30:46.831Z","avatar_url":"https://github.com/sashakolpakov.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Bayesilisk\n\n[![CI](https://github.com/sashakolpakov/bayesilisk/actions/workflows/ci.yml/badge.svg)](https://github.com/sashakolpakov/bayesilisk/actions/workflows/ci.yml)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"logo/bayesilisk_logo.png\" alt=\"Bayesilisk logo\" width=\"220\"\u003e\n\u003c/p\u003e\n\n**Beyond E2E Scripts: Using LLM-Proposed Scenarios Without Letting the LLM Be the Oracle.**\n\nBayesilisk is a deterministic local layer for permission, entitlement, route,\nand data-boundary sitting over Playwright, with Grassmann attention, and\nLLM-generated scenario-proposal workflows gated by a finite-state verifier.\n\nBayesilisk is intentionally local-first. It uses static scenario fragments,\ncaller-provided context, optional observation history, optional browser evidence,\nand optional local model proposals. It does not connect to production systems or\ninspect live customer data. It is built for testers and agents that need\nreproducible findings without granting a model authority over the final verdict.\n\n## What It Is\n\nBayesilisk is designed to find \"bad spots\" in authorization and data-boundary\nlogic before those gaps become hard-to-debug application bugs.\n\nIt checks scenarios involving:\n\n- permission and role-route matrices;\n- customer module entitlements;\n- expense approval and receipt evidence;\n- billing export access;\n- HR document access boundaries;\n- support takeover sessions;\n- DMS tenant and process boundaries;\n- travel funding and travel-expense consistency.\n\nThe core verifier is deterministic:\n\n```text\nscenario facts -\u003e invariant checks -\u003e pass/fail -\u003e Bayesian ranking\n```\n\nNo embedding, model output, issue text, or Playwright observation can directly\ndeclare a bug. Those layers can only steer where Bayesilisk looks next.\n\nSee [docs/architecture.md](docs/architecture.md) for the public architecture:\n\n```text\nPlaywright is the sensor.\nGrassmann attention is the router.\nThe scenario proposer model is the proposer.\nBayesilisk is the judge.\n```\n\n## Quick Start\n\nRun the CLI from the repository root:\n\n```sh\npython3 -m bayesilisk --seed 150 --format json\npython3 -m bayesilisk --seed 150 --format markdown --output /tmp/bayesilisk.md\npython3 -m bayesilisk --seed 150 --context /tmp/bayesilisk-context.json --issue-payloads\n```\n\nAfter installation, the same entry points are available as:\n\n```sh\nbayesilisk --seed 150 --format json\nbayesilisk-mcp\n```\n\nRun the test suite:\n\n```sh\npython3 -m pytest\n```\n\nGitHub CI runs deterministic tests and the Sphinx docs build without Ollama,\nhosted models, browser services, or hidden local state:\n\n```sh\npython3 -m pytest -m \"not live_playwright and not live_ollama\"\nsphinx-build -b html docs docs/_build/html\n```\n\nLive browser/model checks are local opt-in tests:\n\n```sh\npython3 -m pytest tests/test_live_integrations.py -m live_playwright -rs\nBAYESILISK_LIVE_OLLAMA=1 python3 -m pytest tests/test_live_integrations.py -m live_ollama -rs\n```\n\n## Reports\n\nReports include:\n\n- seed and tool version;\n- deterministic production-access boundary;\n- scenario fragments and generated sub-scenarios;\n- access patterns;\n- expected invariant and observed result;\n- stable fingerprint and dedupe key;\n- classification and issue readiness;\n- attention score and attention reasons when context is supplied;\n- posterior probability and risk score;\n- suggested issue title and body.\n\nOnly findings with:\n\n```text\nobservedResult = fail\nissueReadiness = ready-for-issue\n```\n\nshould be opened automatically. `probe-only`, `regression-watch`,\n`do-not-open-muted`, and `no-issue-control` findings are intentionally not\nautomatic issue material.\n\n## Proof Artifacts\n\n![Bayesilisk proof loop](docs/assets/bayesilisk-proof-loop.svg)\n\nThe proof loop is deliberately split. Evidence and proposals can route\nattention, but only deterministic verification can produce automatic issue\nmaterial:\n\n```text\nPlaywright evidence + local context\n(browser trace, DOM state, fixture state, app facts)\n        |\n        v\nGrassmann attention\n(rank suspicious contexts; no verdict authority)\n        |\n        v\nCandidate scenario\n(catalog, rule, or model proposed; untrusted)\n        |\n        v\nBayesilisk verification\n(deterministic invariants and controls decide pass/fail)\n        |\n        +--\u003e ready issue payload\n        |    stable fingerprint + evidence summary\n        |\n        +--\u003e reject / watchlist\n             no automatic issue\n```\n\nExample artifacts:\n\n- [example JSON report](docs/examples/example-report.json)\n- [example GitHub issue payloads](docs/examples/example-issue-payloads.json)\n- [Cal.com connector evidence](examples/calcom/)\n\nThe Cal.com example uses the general Bayesilisk core with an app-specific\nconnector that follows the connector docs. It records the Cal.com repository\nURL, exact tested commit, connector source context, generated proposals,\nobserved local execution context, reports, and upstream outcome references. In\nthe clean current run Bayesilisk generated 7 proposals: 6 route mutations from\nexplicit connector rules plus 1 bounded workflow sequence from a\nconnector-declared action graph. All 7 local observations were verified as app\nfindings. One reported finding already has an upstream human-authored fix PR\nwith a human approval review, which is stronger validation than an issue being\nclosed without fix context.\n\nFor coding agents and LLM teams building connectors, use the ingestible contract\nat [examples/connector-agent-contract.json](examples/connector-agent-contract.json).\nIt spells out required source-context fields, observed-evidence fields, allowed\nagent steps, and boundaries that keep app-specific logic out of Bayesilisk core.\nFor reusable workflow motifs, see the typed ABAG example at\n[examples/abag-action-graph-context.json](examples/abag-action-graph-context.json).\n\n### Why This Is Not a Black Box\n\nBayesilisk exposes separate ledgers for `observedByPlaywright`,\n`selectedByGrassmannAttention`, `proposedByModel`, and `verifiedByBayesilisk`.\nOnly `verifiedByBayesilisk` contains deterministic invariant results that can\nfeed issue payloads. Model output remains untrusted candidate input.\n\n### Model Unavailable? Still Works\n\nThe default verifier path requires no model provider. With no Ollama or hosted\nmodel configured, Bayesilisk still composes deterministic scenarios, evaluates\nfinite-state invariants, ranks findings, validates report schemas, and emits\nissue payloads from verified failures.\n\n## Microsoft Playwright Bridge\n\nBayesilisk includes a local workflow pressure demo and an optional Microsoft\nPlaywright probe. Playwright observes concrete browser behavior and writes\nBayesilisk context; Bayesilisk still performs deterministic verification\nafterward.\n\nInstall the optional browser dependency:\n\n```sh\npython3 -m pip install -e '.[playwright]'\npython3 -m playwright install chromium\n```\n\nRun the bundled demo from a repo checkout:\n\n```sh\ncd /path/to/bayesilisk\npython3 -m pip install -e '.[playwright]'\npython3 -m playwright install chromium\n\n# Terminal transcript only; no browser window.\npython3 -m bayesilisk.demo --no-playwright\n\n# Full screen-recordable run with headed Chromium.\npython3 -m bayesilisk.demo --recording\n```\n\nAfter editable install, the console script is also available from the active\nenvironment:\n\n```sh\nbayesilisk-demo\nbayesilisk-demo --recording\nbayesilisk-demo --no-playwright\n```\n\nThe demo accepts a deterministic seed. Changing it changes the sweep order while\nkeeping that run reproducible:\n\n```sh\npython3 -m bayesilisk.demo --seed 150 --recording\npython3 -m bayesilisk.demo --seed 151 --no-playwright\n```\n\nTo run only the lower-level Playwright adapter against the bundled static probe\ntarget and then feed the captured context to Bayesilisk:\n\n```sh\npython3 tools/playwright_probe.py --demo --output /tmp/bayesilisk-playwright-context.json\npython3 -m bayesilisk --seed 150 --context /tmp/bayesilisk-playwright-context.json --format markdown\npython3 -m bayesilisk --seed 150 --context /tmp/bayesilisk-playwright-context.json --issue-payloads\n```\n\n`bayesilisk-demo` serves a synthetic local fixture defined in\n`bayesilisk/demo.py::DEMO_PROBES`. Those rows are not claims about an existing\ncustomer app; they are twelve deliberately brittle product-like workflows across\nTravel, Expenses, Billing, HR, Support, and DMS, with stale state, impossible\nordering, duplicate submission, feature-flag exposure, tenant boundaries, two\ncontrols, and role lanes. Its output shows the chain:\n\n```text\nPlaywright evidence\n  -\u003e Grassmann plane\n  -\u003e generated catalog/attention scenarios\n  -\u003e optional model-style proposal\n  -\u003e deterministic verdict\n  -\u003e issue payload\n```\n\nIt also includes a hard-to-find drill-down showing a route-matrix failure that\nappears only after connecting support takeover state, HR document access, route\npermissions, and module context. The drill-down includes a seeded sweep order,\nso changing `--seed` can make the same buried failure surface earlier or later\nwhile remaining reproducible for that seed. Use\n`bayesilisk-demo --recording` to open headed Chromium, slow the probe clicks, and\nhold the browser long enough to screen-record the local workflow pressure. Use\n`bayesilisk-demo --no-playwright` to see the same local loop without launching a\nbrowser. The transcript explains every finding class: `breakage.easy`,\n`breakage.hard-to-find`, `finding.candidate-breakage`, and\n`control-confirmed`. `breakage.hard-to-find` means the deterministic invariant\nfailed only after context narrowed the search to a cross-role, cross-module,\nstale-state, or unusual workflow path; it does not mean the model guessed the\nverdict.\n\nFor a real app, serve a page that exposes `data-bayesilisk-probe` rows with\nactor, route, invariant, expected status, and actual click behavior, then run:\n\n```sh\npython3 tools/playwright_probe.py --url http://localhost:3000/probe-page \\\n  --output /tmp/bayesilisk-real-context.json\npython3 -m bayesilisk --seed 150 --context /tmp/bayesilisk-real-context.json --format markdown\n```\n\n### Realistic App Integration Demo\n\nThe realistic demo is a small local permission app, not a static table. It has\nusers, tenants, module flags, support takeover state, HR documents, DMS\nreceipts, billing exports, and expense approvals. The page at\n`/internal/bayesilisk-probes` exposes `data-bayesilisk-probe` rows, and each\nbutton calls a local permission handler before writing the observed status back\nto the page. Bayesilisk then consumes the captured context exactly like it would\nfor a caller-provided app.\n\nRun it without launching a browser:\n\n```sh\npython3 -m bayesilisk.realistic_demo --no-playwright\n```\n\nRun the screen-recordable browser flow:\n\n```sh\npython3 -m bayesilisk.realistic_demo --recording\n```\n\nWrite the captured context and inspect it through the normal verifier:\n\n```sh\npython3 -m bayesilisk.realistic_demo \\\n  --context-output /tmp/bayesilisk-realistic-context.json \\\n  --no-playwright\npython3 -m bayesilisk \\\n  --seed 150 \\\n  --context /tmp/bayesilisk-realistic-context.json \\\n  --format markdown\n```\n\nAfter editable install, the console script is:\n\n```sh\nbayesilisk-realistic-demo --recording\n```\n\nTo run it like a real app integration, keep the local app serving in one\nterminal:\n\n```sh\npython3 -m bayesilisk.realistic_demo --serve-only\n```\n\nThen copy the printed `/internal/bayesilisk-probes` URL into the normal\nPlaywright bridge command from a second terminal.\n\n## Grassmann Attention\n\nContextual reports include a bounded Grassmann-style attention layer. It treats\nPlaywright observations, repository facts, issue text, and invariant descriptions\nas local context planes, then scores which planes look bad or under-tested.\n\nBy default this uses a dependency-free anchor-plane proxy. Set\n`BAYESILISK_USE_OLLAMA_EMBEDDINGS=1` to add Ollama `/api/embed` similarities with\n`BAYESILISK_OLLAMA_MODEL`, defaulting to `nomic-embed-text`.\n\nThe same behavior can be controlled explicitly from the CLI:\n\n```sh\npython3 -m bayesilisk --seed 150 --context /tmp/bayesilisk-playwright-context.json \\\n  --enable-embeddings \\\n  --embedding-model nomic-embed-text \\\n  --attention-threshold 0.4 \\\n  --attention-selection-limit 3\n```\n\nAttention scores answer:\n\n```text\nWhere should Bayesilisk look next?\n```\n\nRisk scores answer:\n\n```text\nGiven this deterministic rule result, how important is this finding?\n```\n\nThose are deliberately separate.\n\n## Scenario Proposer Model\n\nSet `BAYESILISK_USE_OLLAMA_SCENARIO_MODEL=1` to let a local scenario proposer\nmodel suggest extra scenario compositions through Ollama `/api/chat`.\nThe provider is selected with `BAYESILISK_SCENARIO_PROVIDER`, defaulting to\n`ollama`. API-key backed providers read keys from `BAYESILISK_SCENARIO_API_KEY`\nor the env var named by `BAYESILISK_SCENARIO_API_KEY_ENV`; reports record only\nwhether a key was configured, never the key itself.\nRuntime config precedence is explicit CLI/MCP arguments, then environment\nvariables, then defaults.\n\nThe preferred local proposer is `gemma4:e2b`:\n\n```sh\nBAYESILISK_USE_OLLAMA_SCENARIO_MODEL=1 \\\nBAYESILISK_OLLAMA_SCENARIO_MODEL=gemma4:e2b \\\npython3 -m bayesilisk --seed 150 --context /tmp/bayesilisk-playwright-context.json --format json\n```\n\nEquivalent CLI controls avoid hidden environment-only behavior:\n\n```sh\npython3 -m bayesilisk --seed 150 --context /tmp/bayesilisk-playwright-context.json \\\n  --enable-scenario-proposer \\\n  --scenario-provider ollama \\\n  --scenario-model gemma4:e2b \\\n  --scenario-proposal-limit 3 \\\n  --ollama-base-url http://localhost:11434\n```\n\nModel output is untrusted. Bayesilisk accepts a proposal only if it uses known\nfragment ids and invariant ids, targets a selected attention plane, and passes\nschema validation. Accepted proposals appear as `generated.model.*` scenarios\nwith `weak-model-proposal:*` provenance for compatibility with the earlier\nreport field name.\n\nEvery JSON report includes `effectiveConfiguration`, recording the effective\nattention/model settings with the Ollama base URL reduced to a safe URL class.\n\n## MCP Server\n\nBayesilisk includes a small stdio MCP tool server:\n\n```sh\nbayesilisk-mcp\n```\n\nFrom a checkout, the module form is equivalent:\n\n```sh\npython3 -m bayesilisk.mcp_server\n```\n\nBy default the server writes only MCP JSON-RPC frames on `stdout` and stays\nquiet on `stderr`. Set `BAYESILISK_MCP_BANNER=1` when running it manually if\nyou want the ASCII startup banner.\n\nVerifier tools:\n\n- `run`;\n- `rank_context`;\n- `issue_payloads`;\n- `propose_probes`.\n\nCodex orchestration tools:\n\n- `interview_connector_need`;\n- `establish_provenance`;\n- `connector_prompt_packet`;\n- `scenario_plan`;\n- `verify_connector_outputs`;\n- `fix_packet`.\n\nThe MCP tools accept the same control names as JSON arguments, including\n`enableEmbeddings`, `embeddingModel`, `enableScenarioProposer`,\n`scenarioModel`, `scenarioProposalLimit`, `attentionThreshold`,\n`attentionSelectionLimit`, and `ollamaBaseUrl`.\n\nAgents should pass current issue lists, open PRs, branch facts, local verifier\nnotes, Playwright observations, and known Bayesilisk fingerprints as context.\nThe MCP server still runs locally and does not mutate GitHub or production\nsystems.\n\n### Codex Setup\n\nInstall Bayesilisk directly from GitHub:\n\n```sh\npython3 -m pip install 'git+https://github.com/sashakolpakov/bayesilisk.git'\n```\n\nOr clone and install editable:\n\n```sh\ngit clone https://github.com/sashakolpakov/bayesilisk.git\ncd bayesilisk\npython3 -m pip install -e .\n```\n\nFrom an existing checkout:\n\n```sh\npython3 -m pip install -e .\n```\n\nThen add Bayesilisk to Codex config:\n\n```toml\n[mcp_servers.bayesilisk]\ncommand = \"bayesilisk-mcp\"\nargs = []\nstartup_timeout_sec = 60\ntool_timeout_sec = 120\n```\n\nFor a project-local config inside a Bayesilisk checkout, use an explicit\ncheckout path. An absolute Python path is safest if Codex does not inherit your\ninteractive shell `PATH`.\n\n```toml\n[mcp_servers.bayesilisk]\ncommand = \"python3\"\nargs = [\"-m\", \"bayesilisk.mcp_server\"]\ncwd = \"/absolute/path/to/bayesilisk\"\nstartup_timeout_sec = 60\ntool_timeout_sec = 120\n```\n\nRestart Codex, then ask:\n\n```text\nUse Bayesilisk to build a connector for this repo. Start by interviewing me\nabout the connector need, then establish provenance, generate a connector prompt\npacket, plan scenarios, and verify connector outputs.\n```\n\nThe intended loop is:\n\n```text\ninterview_connector_need\n  -\u003e establish_provenance\n  -\u003e connector_prompt_packet\n  -\u003e Codex writes connector code in the target app/test repo\n  -\u003e scenario_plan\n  -\u003e connector executes local fixtures\n  -\u003e verify_connector_outputs\n  -\u003e fix_packet\n```\n\n`run` can also call the local scenario proposer model/API when\n`enableScenarioProposer=true`. The model proposes; Bayesilisk validates and\nverifies. Codex remains responsible for app-specific connector execution, issue\ncreation, and code changes, and should act only on verified Bayesilisk output.\n\nThe OpenAI Codex configuration reference documents `mcp_servers.\u003cid\u003e.command`,\n`args`, `cwd`, `startup_timeout_sec`, and `tool_timeout_sec`:\nhttps://developers.openai.com/codex/config-reference\n\n## Documentation\n\nSphinx documentation lives in [docs/](docs/). The GitHub Pages workflow builds it\nwith MyST Markdown support and publishes it from GitHub Actions.\n\nLocal docs build:\n\n```sh\npython3 -m pip install -r docs/requirements.txt\nsphinx-build -b html docs docs/_build/html\n```\n\n## Development Notes\n\nThe test suite includes scenario-matrix coverage:\n\n- every catalog scenario must reference valid fragments and invariants;\n- every invariant must have at least one passing control and one failing\n  bad-spot case in the deterministic catalog;\n- Playwright, Grassmann attention, and model proposals must not override\n  finite-state verifier results.\n\nCurrent public planning issues are tracked in GitHub Issues.\n\n## Boundaries\n\nBayesilisk is a verifier and prioritizer, not an authorization engine. It must\nnot connect to production systems, inspect live customer data, create migrations,\nor emit internal platform claims as customer package claims.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsashakolpakov%2Fbayesilisk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsashakolpakov%2Fbayesilisk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsashakolpakov%2Fbayesilisk/lists"}