{"id":49305333,"url":"https://github.com/fraction12/starglass","last_synced_at":"2026-04-26T09:04:30.454Z","repository":{"id":351873465,"uuid":"1212762243","full_name":"fraction12/starglass","owner":"fraction12","description":"Observation and dispatch layer for agent systems.","archived":false,"fork":false,"pushed_at":"2026-04-16T20:59:38.000Z","size":104,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-16T22:23:10.754Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fraction12.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-16T17:52:23.000Z","updated_at":"2026-04-16T20:59:44.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/fraction12/starglass","commit_stats":null,"previous_names":["fraction12/starglass"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/fraction12/starglass","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fraction12%2Fstarglass","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fraction12%2Fstarglass/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fraction12%2Fstarglass/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fraction12%2Fstarglass/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fraction12","download_url":"https://codeload.github.com/fraction12/starglass/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fraction12%2Fstarglass/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32291347,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-26T08:29:33.829Z","status":"ssl_error","status_checked_at":"2026-04-26T08:29:18.366Z","response_time":129,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-26T09:04:09.526Z","updated_at":"2026-04-26T09:04:30.446Z","avatar_url":"https://github.com/fraction12.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./docs/assets/starglass-readme-banner.png\" alt=\"Starglass banner showing sources flowing through the Starglass runtime to downstream agents and commands\" /\u003e\n\u003c/p\u003e\n\n# Starglass\n\nStarglass is an observation runtime for agent systems.\n\nIt gives AI research and agent infrastructure teams a clean way to watch external state, turn meaningful change into normalized events, and resume safely across long-lived runs, cron jobs, and restarts.\n\nIn practice, Starglass handles the hard, boring parts of observation-heavy systems:\n- poll-cycle execution\n- normalized event envelopes\n- compact checkpointing\n- duplicate suppression\n- dispatch to commands or in-process handlers\n- explicit observation planning and adaptive cadence\n- generic source families for HTTP, feeds, and the filesystem\n\nYou supply the source-specific meaning. Starglass supplies the runtime discipline.\n\nThat source can be a status API, an evaluation artifact, an RSS or Atom feed, a filesystem tree, a queue, a database row, a private internal endpoint, or some cursed research prototype that only made sense at 1:14am. Starglass should not care.\n\n## Why this exists\n\nMost agent systems eventually need the same substrate:\n- observe changing state outside the model\n- detect meaningful deltas rather than raw churn\n- survive restarts without replaying junk\n- preserve a compact resume point\n- hand off normalized events to downstream automation\n\nTeams often rebuild that machinery inside each connector, cron, or worker. The result is usually fragile, provider-specific, and impossible to reason about once the system grows.\n\nStarglass separates the observation problem from the business-logic problem.\n\n## What Starglass is for\n\nStarglass fits best when you are building:\n- agent or research pipelines that monitor external systems for meaningful change\n- evaluation or observability loops that need deterministic event identity\n- long-lived background watchers with restart-safe checkpointing\n- source-agnostic monitoring primitives for internal AI tooling\n- lightweight ingestion layers that should emit stable events, not raw payload archives\n\n## Boundary\n\nStarglass owns:\n- watcher lifecycle\n- poll execution and restart-safe resumption\n- normalized event contracts\n- compact checkpointing and dedupe\n- explicit observation planning and strategy selection\n- bounded adaptive cadence for long-lived watch loops\n- generic HTTP, feed, and filesystem observation primitives\n- dispatch primitives\n\nStarglass does not own:\n- a branded connector catalog\n- provider-specific business policy\n- workflow orchestration\n- deciding what an agent should do next\n- mutating external systems\n- pretending every source should look like a SaaS integration marketplace\n\n## Mental model\n\n1. Define an observation target.\n2. Implement a `SourceAdapter` for that target, or use a built-in adapter.\n3. Let `ObservationRuntime` resolve the observation plan, execute the poll, and persist compact state.\n4. Handle the normalized event envelope downstream in your agent, worker, or CLI.\n\nThe key idea is simple: adapters decide what changed, Starglass decides how to observe and resume it safely.\n\n## Minimal usage\n\n```ts\nimport {\n  CommandDispatchAdapter,\n  FileCheckpointStore,\n  ObservationRuntime,\n  type CheckpointRecord,\n  type ObservationEvent,\n  type ObservationTarget,\n  type SourceAdapter,\n} from 'starglass'\n\ntype BuildTarget = ObservationTarget \u0026 {\n  source: 'buildkite'\n  pipeline: string\n}\n\nclass BuildkiteSourceAdapter implements SourceAdapter\u003cBuildTarget\u003e {\n  readonly source = 'buildkite'\n\n  async poll(target: BuildTarget, checkpoint?: CheckpointRecord) {\n    const events: ObservationEvent[] = []\n\n    return {\n      events,\n      providerCursor: checkpoint?.providerCursor,\n      polledAt: new Date().toISOString(),\n    }\n  }\n}\n\nconst runtime = new ObservationRuntime({\n  sourceAdapter: new BuildkiteSourceAdapter(),\n  checkpointStore: new FileCheckpointStore('./.starglass/checkpoints.json'),\n  dispatchAdapters: [new CommandDispatchAdapter()],\n})\n\nawait runtime.poll({\n  id: 'buildkite:acme/release',\n  source: 'buildkite',\n  subject: 'buildkite:acme/release',\n  pipeline: 'acme/release',\n  dispatch: {\n    kind: 'command',\n    command: 'node',\n    args: ['./handle-event.js'],\n  },\n})\n```\n\nThe command target receives a normalized JSON envelope on stdin.\n\n## Built-in observation families\n\nStarglass includes built-in adapters for common source families:\n- `HttpObservationAdapter` for generic JSON and HTML resources\n- `FeedObservationAdapter` for RSS and Atom resources\n- `FileSystemObservationAdapter` for files and directories\n\nThese all use the same runtime machinery: planning, compact checkpointing, normalized projection diffing, duplicate suppression, and dispatch.\n\n## Strategy-aware observation\n\nStarglass can now resolve an explicit observation plan from declared capabilities.\n\nPriority order:\n1. push\n2. conditional request\n3. cursor\n4. cheap probe then fetch\n5. projection diff\n6. snapshot diff fallback\n\nAdapters can expose capabilities with `capabilities()`, and targets can add `observationCapabilities` hints. The runtime resolves a compact `ObservationPlan` before execution, including the selected strategy, the merged planning inputs, the previous strategy when one existed, and whether the plan is initial, unchanged, upgraded, or degraded.\n\nFor generic HTTP, Starglass starts honestly in projection-diff mode until it learns validators from a prior response, then upgrades to conditional mode on later polls. If stronger resumable state disappears after restart or replanning, the runtime records that degradation explicitly instead of pretending the stronger mode is still active. The checkpoint stays compact, but callers can inspect the current plan through hooks or persisted observation metadata.\n\n## Generic HTTP observation\n\nStarglass includes a built-in `HttpObservationAdapter` for generic JSON and HTML resources, aimed at the common case where you need reliable change detection without building a bespoke connector first.\n\nIt supports:\n- conditional `ETag` and `Last-Modified` reuse once validators are learned\n- minimal `Retry-After` and `Cache-Control: max-age` hint capture in compact checkpoint metadata\n- normalized JSON projection diffing\n- normalized HTML extraction diffing\n- compact checkpoint state with validators, next-poll hints, and fingerprints\n- no raw response body retention by default\n\n### JSON example\n\n```ts\nimport {\n  FileCheckpointStore,\n  HttpObservationAdapter,\n  ObservationRuntime,\n} from 'starglass'\n\nconst runtime = new ObservationRuntime({\n  sourceAdapter: new HttpObservationAdapter(),\n  checkpointStore: new FileCheckpointStore('./.starglass/http-checkpoints.json'),\n  dispatchAdapters: [\n    {\n      supports(target) {\n        return target.kind === 'handler'\n      },\n      async dispatch(envelope) {\n        console.log('meaningful change', envelope.event.payload.projection)\n      },\n    },\n  ],\n})\n\nawait runtime.poll({\n  id: 'http:json:status',\n  source: 'http',\n  subject: 'http:https://status.example.com/api/summary',\n  url: 'https://status.example.com/api/summary',\n  format: 'json',\n  project: (document) =\u003e {\n    const payload = document as {\n      status: { indicator: string; description: string }\n      page: { updated_at: string }\n    }\n\n    return {\n      indicator: payload.status.indicator,\n      description: payload.status.description,\n      updatedAt: payload.page.updated_at,\n    }\n  },\n  dispatch: {\n    kind: 'handler',\n    handler: async (envelope) =\u003e {\n      console.log('observed projection', envelope.event.payload.projection)\n    },\n  },\n})\n```\n\nFor HTML targets, replace `format: 'json'` and `project()` with `format: 'html'` and `extract()`.\n\nYou can keep projection authoring small and generic with the built-in helpers:\n\n```ts\nimport { html, normalize, projectJson } from 'starglass'\n\nconst projectStatus = projectJson.shape({\n  indicator: projectJson.path('status', 'indicator'),\n  description: projectJson.path('status', 'description'),\n  updatedAt: projectJson.path('page', 'updated_at'),\n})\n\nconst stableProjection = normalize.stable({\n  noisyField: undefined,\n  meaningful: projectStatus(document),\n})\n\nconst extractHeadline = html.extract((document) =\u003e ({\n  headline: document.match(/\u003ch1\u003e(.*?)\u003c\\/h1\u003e/)?.[1] ?? null,\n}))\n```\n\nThese helpers stay intentionally narrow: Starglass helps define a stable observed projection, but your business policy still decides what that projection means downstream.\n\nSee `examples/http-observation/README.md` for a focused example.\n\n## Adapter authoring guidance\n\nA good adapter does five things consistently:\n\n1. Chooses one durable `subject` per watched thing.\n2. Derives deterministic event ids from provider identity plus the meaningful state transition.\n3. Uses a provider cursor or compact validators that can safely resume polling after restarts.\n4. Filters raw provider noise down to meaningful normalized events before returning them.\n5. Keeps normalized payloads small, stable, and downstream-friendly.\n\n### Subject identity\n\n`target.subject` is the stable identity of the thing being watched, not the identity of one individual event.\n\nGood subjects usually look like:\n- `linear:team-123:issue:ABC-42`\n- `rss:https://example.com/feed`\n- `example.buildkite:acme/release:main`\n- `http:https://status.example.com/api/summary`\n\nUse a subject that stays stable across polls and process restarts. If downstream systems should think of two observations as \"the same thing\", they should share a subject.\n\n### Event ids\n\n`event.id` should be deterministic. If the same upstream change is observed twice, the adapter should produce the same event id so Starglass can suppress duplicates safely. The built-in HTTP adapter keys event ids to the normalized projection fingerprint, which means a later return to a previously seen state reuses the same id by design. Feed entry events instead key off the stable subject, entry id, and the same meaningful revision signal used for feed change detection, so a meaningful re-emit gets a fresh event id even when the projected payload fingerprint stays the same.\n\nA practical rule is: hash the stable subject, normalized event kind, provider object id, and provider update/version marker.\n\n```ts\nconst id = stableEventId(target.subject, kind, providerObjectId, providerUpdatedAt)\n```\n\nDo not use random ids for provider-backed changes. Random ids defeat duplicate suppression.\n\n### Provider cursor and compact state strategy\n\n`providerCursor` is the adapter-owned resume token. Starglass stores it, but your adapter defines what it means.\n\nGood resumable state is usually one or more of:\n- a monotonic provider timestamp\n- a provider sequence number\n- a page token or opaque checkpoint returned by the provider\n- HTTP validators like `ETag` or `Last-Modified`\n- a compact fingerprint of the last meaningful projection\n- optional next-poll hints derived from generic transport metadata like `Retry-After` or `Cache-Control: max-age`\n\nChoose state that lets the next poll ask \"what changed since the last successful run?\" without storing whole payload archives.\n\n### Meaningful-change filtering\n\nAdapters should do provider-specific filtering before returning events.\n\nExamples:\n- ignore build updates unless the state enters `failed`, `passed`, or another watched terminal state\n- ignore issue updates that only change unread counts or internal metadata\n- ignore feed entries outside the configured category or policy\n- ignore JSON or HTML churn that does not change the normalized observed projection\n\nStarglass handles dispatch, checkpointing, duplicate suppression, and compact strategy state. The adapter should decide which upstream changes are meaningful enough to become normalized events.\n\n### Normalized payload boundaries\n\n`event.payload` should contain the downstream-useful summary of the change, not a full raw provider response dump.\n\nGood payloads usually include:\n- the fields downstream automation actually needs\n- normalized names and shapes\n- a human-readable summary when helpful\n- provider-native ids or URLs only when they help follow-up work\n\nKeep bulky or unstable provider details out of the normalized payload when possible. Put traceable provider identity in `sourceRef`.\n\n### External-style examples\n\nSee:\n- `examples/external-adapter/` for a custom external adapter\n- `examples/http-observation/` for generic HTTP observation\n- `examples/feed-observation/` for generic RSS/Atom observation\n- `examples/filesystem-observation/` for generic filesystem observation\n\n## Long-lived watchers and hooks\n\n```ts\nconst controller = runtime.watch(target, {\n  intervalMs: 30_000,\n  backoffMs: 5_000,\n  maxBackoffMs: 60_000,\n  maxConsecutiveFailures: 5,\n  cadence: {\n    minIntervalMs: 10_000,\n    maxIntervalMs: 120_000,\n    activityMultiplier: 0.5,\n    idleMultiplier: 1.5,\n    maxIdleDelayMs: 90_000,\n  },\n})\n\nawait controller.stop()\n```\n\nStarglass now plans the next attempt inside the watch loop itself. It uses compact observation metadata like `Retry-After`, `Cache-Control: max-age`, recent activity, and idle streaks, but always clamps the result to caller-supplied cadence bounds.\n\nThat execution cadence is separate from observation-plan selection. Plan selection decides how a target will be observed. The watch loop then executes that plan and schedules the next attempt.\n\nYou can also attach structured hooks without coupling Starglass to a logger or metrics backend:\n\n```ts\nconst runtime = new ObservationRuntime({\n  sourceAdapter,\n  checkpointStore,\n  dispatchAdapters,\n  hooks: {\n    onPollStarted: ({ target }) =\u003e console.log('poll started', target.id),\n    onObservationPlanSelected: ({ plan }) =\u003e {\n      console.log('strategy', plan.strategy.mode, plan.strategy.reason, plan.change.kind)\n    },\n    onDispatchSucceeded: ({ event }) =\u003e console.log('dispatch ok', event.id),\n    onDispatchFailed: ({ event, error }) =\u003e console.error('dispatch failed', event.id, error),\n    onCheckpointAdvanced: ({ reason, record }) =\u003e console.log('checkpoint', reason, record.observationTargetId),\n    onWatchBackoff: ({ consecutiveFailures, delayMs }) =\u003e console.warn('backing off', consecutiveFailures, delayMs),\n    onWatchCadencePlanned: ({ reason, delayMs, nextAttemptAt, boundedBy }) =\u003e {\n      console.log('next attempt', reason, delayMs, nextAttemptAt, boundedBy)\n    },\n    onWatchStopped: ({ reason }) =\u003e console.log('watch stopped', reason),\n  },\n})\n```\n\n`onWatchCadencePlanned` exposes structured, logger-agnostic observability so callers can inspect why a target sped up, slowed down, deferred for freshness hints, or stayed pinned to configured bounds.\n\nThe existing `poll()` API remains available for one-shot or job-style usage.\n\n## Release discipline\n\nSee `docs/release.md` for the release checklist and packaging contract.\n\nKey commands:\n- `npm run verify:packaging` checks the tarball file list and performs a clean install/import smoke test\n- `npm run release:check` runs the full local release verification sequence\n\n## Generic feed and filesystem observation\n\nStarglass also includes built-in source families for feeds and the filesystem, using the same planning, checkpointing, dedupe, and projection primitives as HTTP. The goal is not to ship a connector zoo. The goal is to prove the runtime generalizes cleanly across source families that show up constantly in agent and research workflows.\n\n- `FeedObservationAdapter` observes RSS and Atom resources through a small bounded XML tag parser, normalized entry projections, and compact per-entry state that tracks entry content revision separately from projected output.\n- `FileSystemObservationAdapter` observes files or directories through normalized projection diffs and stores compact fingerprints rather than raw archives.\n\nFeed defaults are intentionally conservative: Starglass stores a compact entry-content revision fingerprint separately from the projected payload, so meaningful entry updates still emit when timestamps stay fixed and your projection intentionally omits volatile body fields. The default version signal comes from `updatedAt`, then `publishedAt`, then that compact content fingerprint. Feed event ids are derived from the entry id plus a compact revision signal that combines that version with the entry-content fingerprint, so unchanged-timestamp RSS body edits still get distinct event ids without bloating checkpoint state. You can override `entryVersion` when a feed exposes a better revision marker.\n\nFilesystem observation is projection-oriented, not archival. File targets still read `text`, `json`, or raw `bytes`. Directory targets with `includeContent: true` return UTF-8 text when content decodes cleanly, otherwise they expose base64 content with `contentEncoding: 'base64'` so binary files are represented honestly instead of being silently mangled.\n\nBoth stay source-agnostic. They are meant to be solid generic observation primitives, not thinly disguised product-marketing connectors.\n\n## Status\n\nCurrent release: `v0.2.1`\n\nImplemented today:\n- observation runtime with restart-safe file-backed checkpointing\n- deterministic duplicate suppression and normalized event envelopes\n- command and in-process handler dispatch\n- long-lived watch loops with lifecycle hooks\n- explicit observation planning and bounded adaptive cadence\n- generic HTTP observation for JSON and HTML\n- generic RSS/Atom feed observation\n- generic filesystem observation\n- release verification and trusted publishing support\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffraction12%2Fstarglass","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffraction12%2Fstarglass","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffraction12%2Fstarglass/lists"}