{"id":33902220,"url":"https://github.com/nshkrdotcom/flowstone","last_synced_at":"2026-01-13T20:59:35.102Z","repository":{"id":327164275,"uuid":"1108129780","full_name":"nshkrdotcom/flowstone","owner":"nshkrdotcom","description":"Asset-first data orchestration for Elixir/BEAM. Dagster-inspired with OTP fault tolerance, LiveView dashboard, lineage tracking, checkpoint gates, and distributed execution via Oban.","archived":false,"fork":false,"pushed_at":"2025-12-21T08:39:31.000Z","size":739,"stargazers_count":12,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-21T23:48:01.520Z","etag":null,"topics":["asset-management","beam","chaos-engineering","dag","data-lineage","data-orchestration","data-pipeline","ecto","elixir","fault-tolerance","human-in-the-loop","multi-tenant","nshkr-ai-agents","oban","otp","phoenix-liveview","scheduling","tdd","telemetry","workflow-engine"],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nshkrdotcom.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-02T04:06:22.000Z","updated_at":"2025-12-21T08:39:35.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/nshkrdotcom/flowstone","commit_stats":null,"previous_names":["nshkrdotcom/flowstone"],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/nshkrdotcom/flowstone","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nshkrdotcom%2Fflowstone","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nshkrdotcom%2Fflowstone/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nshkrdotcom%2Fflowstone/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nshkrdotcom%2Fflowstone/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nshkrdotcom","download_url":"https://codeload.github.com/nshkrdotcom/flowstone/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nshkrdotcom%2Fflowstone/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28400347,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-13T14:36:09.778Z","status":"ssl_error","status_checked_at":"2026-01-13T14:35:19.697Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asset-management","beam","chaos-engineering","dag","data-lineage","data-orchestration","data-pipeline","ecto","elixir","fault-tolerance","human-in-the-loop","multi-tenant","nshkr-ai-agents","oban","otp","phoenix-liveview","scheduling","tdd","telemetry","workflow-engine"],"created_at":"2025-12-12T00:14:19.908Z","updated_at":"2026-01-13T20:59:35.095Z","avatar_url":"https://github.com/nshkrdotcom.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/flowstone.svg\" alt=\"FlowStone Logo\" width=\"220\" height=\"248\"\u003e\n\u003c/p\u003e\n\n# FlowStone\n\n[![CI](https://github.com/nshkrdotcom/flowstone/actions/workflows/ci.yml/badge.svg)](https://github.com/nshkrdotcom/flowstone/actions/workflows/ci.yml)\n[![Hex.pm](https://img.shields.io/hexpm/v/flowstone.svg)](https://hex.pm/packages/flowstone)\n[![Hex Docs](https://img.shields.io/badge/hex-docs-blue.svg)](https://hexdocs.pm/flowstone)\n[![License](https://img.shields.io/hexpm/l/flowstone.svg)](https://github.com/nshkrdotcom/flowstone/blob/main/LICENSE)\n\n**Asset-first data orchestration for the BEAM.**\n\nFlowStone is an orchestration library for Elixir that treats *assets* (named data artifacts) as the primary abstraction. It's designed for building reliable, auditable data pipelines where persistence, lineage tracking, and operational visibility matter.\n\n## Quick Start\n\n**v0.5.2** adds maintenance and resilience improvements, building on the HTTP client from v0.5.1:\n\n```elixir\ndefmodule MyApp.Pipeline do\n  use FlowStone.Pipeline\n\n  asset :greeting do\n    execute fn _, _ -\u003e {:ok, \"Hello, World!\"} end\n  end\n\n  asset :numbers do\n    execute fn _, _ -\u003e {:ok, [1, 2, 3, 4, 5]} end\n  end\n\n  asset :doubled do\n    depends_on [:numbers]\n    execute fn _, %{numbers: nums} -\u003e {:ok, Enum.map(nums, \u0026(\u00261 * 2))} end\n  end\n\n  asset :sum do\n    depends_on [:doubled]\n    execute fn _, %{doubled: nums} -\u003e {:ok, Enum.sum(nums)} end\n  end\nend\n\n# Run with automatic dependency resolution\n{:ok, 30} = FlowStone.run(MyApp.Pipeline, :sum)\n\n# Check if a result exists\ntrue = FlowStone.exists?(MyApp.Pipeline, :sum)\n\n# Retrieve cached result\n{:ok, 30} = FlowStone.get(MyApp.Pipeline, :sum)\n```\n\nNo explicit registration, no server configuration needed. FlowStone auto-registers pipelines and uses in-memory storage by default.\n\n## Installation\n\nAdd to your `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:flowstone, \"~\u003e 0.5.2\"}\n  ]\nend\n```\n\n## When to Use FlowStone\n\nFlowStone is the right choice when you need:\n\n| Use Case | Why FlowStone |\n|----------|---------------|\n| **ETL/ELT pipelines** | Partition-aware materialization, dependency tracking |\n| **Data warehouse orchestration** | Scheduled jobs, backfill support, lineage queries |\n| **Audit-sensitive workflows** | Comprehensive audit logging, approval gates |\n| **Durable job execution** | Oban-backed persistence, jobs survive restarts |\n| **Multi-format data storage** | Pluggable I/O managers (PostgreSQL, S3, Parquet) |\n\n### Ideal Workloads\n\n- Daily/hourly batch processing with date-partitioned data\n- Data transformation pipelines with explicit dependencies\n- Workflows requiring human approval checkpoints\n- Systems where \"what produced this data?\" matters (lineage)\n- Long-running jobs that must not be lost on node failure\n\n## When to Use Something Else\n\nFlowStone is **not** the best fit for:\n\n| Use Case | Better Alternative |\n|----------|-------------------|\n| **Real-time distributed compute** | [Handoff](https://github.com/polvalente/handoff) - native BEAM distribution, resource-aware scheduling |\n| **High-throughput stream processing** | [Broadway](https://github.com/dashbitco/broadway), [GenStage](https://github.com/elixir-lang/gen_stage) |\n| **Simple background jobs** | [Oban](https://github.com/sorentwo/oban) directly |\n| **Ad-hoc parallel computation** | [Task.async_stream](https://hexdocs.pm/elixir/Task.html), [Flow](https://github.com/dashbitco/flow) |\n| **ML model training/inference** | [Nx](https://github.com/elixir-nx/nx) + distributed compute layer |\n\n**Rule of thumb:** If your data needs to survive a restart and you care about lineage, use FlowStone. If you need fast ephemeral computation across nodes with resource awareness, use Handoff.\n\n## High-Level API\n\nThe v0.5.0 API is pipeline-centric and works with zero configuration:\n\n### Running Assets\n\n```elixir\n# Run an asset (resolves dependencies automatically)\n{:ok, result} = FlowStone.run(MyPipeline, :asset_name)\n\n# Run with a specific partition\n{:ok, result} = FlowStone.run(MyPipeline, :daily_report, partition: ~D[2025-01-15])\n\n# Force re-execution (ignore cache)\n{:ok, result} = FlowStone.run(MyPipeline, :asset_name, force: true)\n\n# Run without dependencies (fails if deps not already materialized)\n{:ok, result} = FlowStone.run(MyPipeline, :asset_name, with_deps: false)\n```\n\n### Retrieving Results\n\n```elixir\n# Get cached result\n{:ok, result} = FlowStone.get(MyPipeline, :asset_name)\n{:error, :not_found} = FlowStone.get(MyPipeline, :never_run)\n\n# Check if result exists\ntrue = FlowStone.exists?(MyPipeline, :asset_name)\n\n# Get execution status\n%{state: :completed, partition: :default} = FlowStone.status(MyPipeline, :asset_name)\n```\n\n### Cache Management\n\n```elixir\n# Invalidate cached result\n{:ok, 1} = FlowStone.invalidate(MyPipeline, :asset_name)\n\n# Invalidate specific partition\n{:ok, 1} = FlowStone.invalidate(MyPipeline, :asset_name, partition: ~D[2025-01-15])\n```\n\n### Backfilling\n\n```elixir\n# Process multiple partitions\n{:ok, stats} = FlowStone.backfill(MyPipeline, :daily_report,\n  partitions: Date.range(~D[2025-01-01], ~D[2025-01-31])\n)\n\n# Parallel backfill\n{:ok, stats} = FlowStone.backfill(MyPipeline, :daily_report,\n  partitions: [:a, :b, :c, :d],\n  parallel: 4\n)\n\n# stats =\u003e %{succeeded: 31, failed: 0, skipped: 0}\n```\n\n### Introspection\n\n```elixir\n# List all assets\n[:greeting, :numbers, :doubled, :sum] = FlowStone.assets(MyPipeline)\n\n# Get asset details\n%{name: :doubled, depends_on: [:numbers]} = FlowStone.asset_info(MyPipeline, :doubled)\n\n# Get DAG visualization\nFlowStone.graph(MyPipeline)              # ASCII art\nFlowStone.graph(MyPipeline, format: :mermaid)  # Mermaid diagram\n```\n\n### Pipeline Module Shortcuts\n\nPipelines also get convenience methods:\n\n```elixir\ndefmodule MyApp.Pipeline do\n  use FlowStone.Pipeline\n  # ...\nend\n\n# These work on the pipeline module directly\n{:ok, result} = MyApp.Pipeline.run(:sum)\n{:ok, result} = MyApp.Pipeline.get(:sum)\ntrue = MyApp.Pipeline.exists?(:sum)\n[:greeting, :numbers, :doubled, :sum] = MyApp.Pipeline.assets()\n```\n\n## Configuration\n\nFlowStone works with zero configuration, but can be customized:\n\n```elixir\n# config/config.exs\n\n# Add persistence (auto-enables Postgres storage and lineage)\nconfig :flowstone, repo: MyApp.Repo\n\n# Or configure explicitly\nconfig :flowstone,\n  repo: MyApp.Repo,\n  storage: :postgres,    # :memory (default), :postgres, :s3, :parquet\n  lineage: true,         # auto-enabled when repo is set\n  async_default: false   # use sync execution by default\n```\n\nSee the [Configuration Guide](guides/configuration.md) for details.\n\n### Distributed Deployment\n\nFlowStone uses node-local ETS for runtime configuration (`FlowStone.RunConfig`). In multi-node Oban deployments:\n\n**Limitation:** Custom runtime options passed to `FlowStone.materialize/2` (like `:io`, `:registry`) will not propagate to jobs executed on different nodes.\n\n**Recommendations for multi-node deployments:**\n\n1. **Use application config** for IO managers, registries, and resource servers:\n   ```elixir\n   # config/runtime.exs - these settings are available on all nodes\n   config :flowstone,\n     io_managers: %{postgres: FlowStone.IO.Postgres},\n     default_io_manager: :postgres\n   ```\n\n2. **Use Oban's queue affinity** if you need node-specific behavior\n\n3. **Monitor fallback events** via telemetry:\n   ```elixir\n   :telemetry.attach(\"run-config-monitor\", [:flowstone, :run_config, :fallback], fn event, _, meta, _ -\u003e\n     Logger.warning(\"RunConfig fallback on #{meta.node} for run_id: #{meta.run_id}\")\n   end, nil)\n   ```\n\nFor single-node deployments, runtime options work as expected.\n\n## Core Concepts\n\n### Assets\n\nAssets are named data artifacts. Each asset declares:\n- **Dependencies** - other assets it consumes\n- **Execute function** - computation that produces the asset's value\n- **Partition** - temporal or dimensional key (dates, tuples, custom)\n\n```elixir\nasset :daily_sales do\n  depends_on [:raw_transactions, :product_catalog]\n  execute fn context, deps -\u003e\n    sales = compute_sales(deps.raw_transactions, deps.product_catalog)\n    {:ok, sales}\n  end\nend\n```\n\n### Short-Form Assets\n\nFor simple assets, use the short form:\n\n```elixir\n# Short form - value is returned directly\nasset :config, do: {:ok, %{batch_size: 100}}\n\n# Equivalent to:\nasset :config do\n  execute fn _, _ -\u003e {:ok, %{batch_size: 100}} end\nend\n```\n\n### Implicit Result Wrapping\n\nWith `wrap_results: true`, you can skip the `{:ok, ...}` wrapper:\n\n```elixir\ndefmodule MyApp.Pipeline do\n  use FlowStone.Pipeline, wrap_results: true\n\n  asset :numbers do\n    execute fn _, _ -\u003e [1, 2, 3, 4, 5] end  # Automatically wrapped as {:ok, [1, 2, 3, 4, 5]}\n  end\n\n  asset :doubled do\n    depends_on [:numbers]\n    execute fn _, %{numbers: nums} -\u003e Enum.map(nums, \u0026(\u00261 * 2)) end\n  end\nend\n```\n\n### Partitions\n\nFlowStone supports partition-aware execution for time-series and dimensional data:\n\n```elixir\n# Date partitions\n{:ok, _} = FlowStone.run(MyPipeline, :report, partition: ~D[2025-01-15])\n\n# DateTime partitions\n{:ok, _} = FlowStone.run(MyPipeline, :hourly_stats, partition: ~U[2025-01-15 14:00:00Z])\n\n# Custom partitions (atoms, tuples)\n{:ok, _} = FlowStone.run(MyPipeline, :regional_report, partition: {:us_east, ~D[2025-01-15]})\n```\n\nAccess the partition in your execute function:\n\n```elixir\nasset :daily_data do\n  execute fn ctx, _ -\u003e\n    date = ctx.partition\n    {:ok, fetch_data_for_date(date)}\n  end\nend\n```\n\n### I/O Managers\n\nPluggable storage backends for asset data:\n\n| Manager | Use Case |\n|---------|----------|\n| `FlowStone.IO.Memory` | Development, testing (default) |\n| `FlowStone.IO.Postgres` | Structured data, small-medium payloads |\n| `FlowStone.IO.S3` | Large files, data lake integration |\n| `FlowStone.IO.Parquet` | Columnar analytics data |\n\n## Advanced Features\n\n### Scatter (Dynamic Fan-Out)\n\nRuntime-discovered parallel execution:\n\n```elixir\nasset :scraped_article do\n  depends_on [:source_urls]\n\n  scatter fn %{source_urls: urls} -\u003e\n    Enum.map(urls, \u0026%{url: \u00261})\n  end\n\n  scatter_options do\n    max_concurrent 50\n    rate_limit {10, :second}\n    failure_threshold 0.02\n  end\n\n  execute fn ctx, _deps -\u003e\n    Scraper.fetch(ctx.scatter_key.url)\n  end\nend\n```\n\n### ItemReader (Streaming Scatter Inputs)\n\nRead scatter items incrementally from external sources:\n\n```elixir\nasset :processed_items do\n  scatter_from :custom do\n    init fn _config, _deps -\u003e {:ok, %{items: fetch_items(), index: 0}} end\n    read fn state, batch_size -\u003e\n      batch = Enum.slice(state.items, state.index, batch_size)\n      new_state = %{state | index: state.index + length(batch)}\n      if batch == [], do: {:ok, [], :halt}, else: {:ok, batch, new_state}\n    end\n  end\n\n  execute fn ctx, _deps -\u003e\n    {:ok, process_item(ctx.scatter_key)}\n  end\nend\n```\n\n### ItemBatcher (Batch Scatter Execution)\n\nGroup scatter items into batches for efficient execution:\n\n```elixir\nasset :batched_processor do\n  depends_on [:source_items]\n\n  scatter fn %{source_items: items} -\u003e\n    Enum.map(items, \u0026%{item_id: \u00261.id})\n  end\n\n  batch_options do\n    max_items_per_batch 20\n    batch_input fn deps -\u003e %{total: length(deps.source_items)} end\n    on_item_error :fail_batch\n  end\n\n  execute fn ctx, _deps -\u003e\n    # ctx.batch_items - list of items in this batch\n    # ctx.batch_index - zero-based batch index\n    # ctx.batch_count - total number of batches\n    sum = Enum.sum(Enum.map(ctx.batch_items, \u0026 \u00261[\"item_id\"]))\n    {:ok, %{batch_sum: sum}}\n  end\nend\n```\n\n### Signal Gate (Durable External Suspension)\n\nZero-resource waiting for external signals (webhooks, callbacks):\n\n```elixir\nasset :embedded_documents do\n  execute fn ctx, deps -\u003e\n    task_id = ECS.start_task(deps.data)\n    {:signal_gate, token: task_id, timeout: :timer.hours(1)}\n  end\n\n  on_signal fn _ctx, payload -\u003e\n    {:ok, payload.result}\n  end\n\n  on_timeout fn ctx -\u003e\n    {:error, :timeout}\n  end\nend\n```\n\n### Conditional Routing\n\nSelect a single branch at runtime:\n\n```elixir\nasset :router do\n  depends_on [:input]\n\n  route do\n    choice :branch_a, when: fn %{input: %{mode: :a}} -\u003e true end\n    default :branch_b\n  end\nend\n\nasset :branch_a do\n  routed_from :router\n  depends_on [:input]\n  execute fn _, _ -\u003e {:ok, :a} end\nend\n\nasset :branch_b do\n  routed_from :router\n  depends_on [:input]\n  execute fn _, _ -\u003e {:ok, :b} end\nend\n\nasset :merge do\n  depends_on [:branch_a, :branch_b]\n  optional_deps [:branch_a, :branch_b]\n  execute fn _, deps -\u003e {:ok, deps[:branch_a] || deps[:branch_b]} end\nend\n```\n\n### Parallel Branches\n\nRun heterogeneous branches in parallel and join results:\n\n```elixir\nasset :enrich do\n  depends_on [:input]\n\n  parallel do\n    branch :maps, final: :generate_maps\n    branch :news, final: :get_front_pages\n  end\n\n  parallel_options do\n    failure_mode :partial\n  end\n\n  join fn branches, deps -\u003e\n    %{maps: branches.maps, news: branches.news, input: deps.input}\n  end\nend\n```\n\n### Approval Gates\n\nPause execution for human review:\n\n```elixir\nasset :high_value_trade do\n  execute fn context, deps -\u003e\n    trade = compute_trade(deps)\n    if trade.value \u003e 1_000_000 do\n      {:wait_for_approval, message: \"Large trade requires sign-off\", context: trade}\n    else\n      {:ok, trade}\n    end\n  end\nend\n```\n\n### HTTP Client Resource\n\nBuilt-in HTTP client for REST API integrations with retry, rate limiting, and telemetry:\n\n```elixir\n# Register an HTTP client as a resource\nFlowStone.Resources.register(:api, FlowStone.HTTP.Client,\n  base_url: \"https://api.example.com\",\n  timeout: 30_000,\n  headers: %{\"Authorization\" =\u003e \"Bearer #{token}\"},\n  retry: %{max_attempts: 3, base_delay_ms: 1000}\n)\n\n# Use in assets\nasset :fetch_user do\n  requires [:api]\n\n  execute fn ctx, _deps -\u003e\n    client = ctx.resources[:api]\n\n    case FlowStone.HTTP.Client.get(client, \"/users/#{ctx.partition.user_id}\") do\n      {:ok, %{status: 200, body: user}} -\u003e {:ok, user}\n      {:ok, %{status: 404}} -\u003e {:error, :not_found}\n      {:error, reason} -\u003e {:error, reason}\n    end\n  end\nend\n\n# POST with idempotency key (safe retries)\nasset :create_order do\n  requires [:api]\n\n  execute fn ctx, deps -\u003e\n    idempotency_key = \"#{ctx.run_id}-#{ctx.partition}\"\n\n    FlowStone.HTTP.Client.post(ctx.resources[:api], \"/orders\", deps.order_data,\n      idempotency_key: idempotency_key\n    )\n  end\nend\n```\n\nThe HTTP client automatically:\n- Retries on 5xx errors and transport failures\n- Respects `Retry-After` headers on 429 rate limit responses\n- Uses exponential backoff with jitter\n- Emits telemetry events for monitoring\n\n### Rate Limiting\n\nDistributed rate limiting for APIs and external services:\n\n```elixir\n# Check rate limit\ncase FlowStone.RateLimiter.check(\"api:openai\", {60, :minute}) do\n  :ok -\u003e call_api()\n  {:wait, ms} -\u003e {:snooze, div(ms, 1000) + 1}\nend\n\n# Semaphore-based concurrency control\nFlowStone.RateLimiter.with_slot(\"expensive:operation\", 10, fn -\u003e\n  expensive_operation()\nend)\n```\n\n### Lineage Tracking\n\nQuery what data produced what:\n\n```elixir\n# What did :report consume?\nFlowStone.Lineage.upstream(:report, ~D[2025-01-15])\n# =\u003e [%{asset: :cleaned, partition: \"2025-01-15\"}]\n\n# What depends on :raw?\nFlowStone.Lineage.downstream(:raw, ~D[2025-01-15])\n# =\u003e [%{asset: :cleaned, partition: \"2025-01-15\"}]\n```\n\n## Testing\n\nFlowStone provides test helpers for isolated pipeline testing:\n\n```elixir\ndefmodule MyApp.PipelineTest do\n  use FlowStone.TestCase, isolation: :full_isolation\n\n  test \"runs asset with mocked dependencies\" do\n    {:ok, result} = run_asset(MyPipeline, :doubled, with_deps: %{numbers: [1, 2, 3]})\n    assert result == [2, 4, 6]\n  end\n\n  test \"asserts asset existence\" do\n    {:ok, _} = FlowStone.run(MyPipeline, :greeting)\n    assert_asset_exists(MyPipeline, :greeting)\n  end\nend\n```\n\nSee the [Testing Guide](guides/testing.md) for more patterns.\n\n## Telemetry\n\nFlowStone emits telemetry events for observability:\n\n- `[:flowstone, :materialization, :start | :stop | :exception]`\n- `[:flowstone, :scatter, :start | :complete | :failed | :instance_complete | :instance_fail]`\n- `[:flowstone, :signal_gate, :create | :signal | :timeout]`\n- `[:flowstone, :route, :start | :stop | :error]`\n- `[:flowstone, :parallel, :start | :stop | :error | :branch_start | :branch_complete | :branch_fail]`\n- `[:flowstone, :http, :request, :start | :stop | :error]`\n- `[:flowstone, :rate_limit, :check | :wait | :slot_acquired | :slot_released]`\n- `[:flowstone, :run_config, :fallback]` - Emitted when RunConfig falls back to app config (distributed mode)\n- `[:flowstone, :resources, :setup_failed]` - Emitted when a resource fails to initialize\n- `[:flowstone, :route_decisions, :cleanup]` - Emitted when route decisions are cleaned up\n\n## Low-Level API\n\nFor advanced use cases, the low-level API provides full control:\n\n```elixir\n# Manual registration with explicit options\n{:ok, _} = FlowStone.Registry.start_link(name: MyRegistry)\n{:ok, _} = FlowStone.IO.Memory.start_link(name: MyMemory)\n\nFlowStone.register(MyApp.Pipeline, registry: MyRegistry)\n\n# Configure I/O manually\nio = [config: %{agent: MyMemory}]\n\n# Materialize with explicit options\nFlowStone.materialize(:report,\n  partition: ~D[2025-01-15],\n  registry: MyRegistry,\n  io: io\n)\n\n# Materialize with all dependencies\nFlowStone.materialize_all(:report,\n  partition: ~D[2025-01-15],\n  registry: MyRegistry,\n  io: io\n)\n```\n\n## Documentation\n\n### Guides\n\n- [Getting Started](guides/getting-started.md) - Step-by-step introduction\n- [Configuration](guides/configuration.md) - Configuration options and patterns\n- [Testing](guides/testing.md) - Testing patterns and helpers\n\n### Reference\n\n- **Design overview:** `docs/design/OVERVIEW.md`\n- **Architecture decisions:** `docs/adr/README.md`\n- **Comparison with Handoff:** `docs/scratch/20251215/flowstone-vs-handoff-comparison.md`\n- **Examples:** `examples/` directory\n\n### Examples\n\nRun the examples:\n\n```bash\n# Individual examples\nmix run examples/01_hello_world.exs\nmix run examples/02_dependencies.exs\nmix run examples/03_partitions.exs\nmix run examples/04_backfill.exs\n```\n\n## Status\n\nFlowStone is in **alpha**. Core execution, persistence primitives, and safety hardening are implemented. The v0.5.2 release adds maintenance and resilience improvements including route decision cleanup, resilient resource initialization, and improved documentation for distributed deployments.\n\n## Contributing\n\nContributions are welcome. Please read the ADRs first to understand current decisions and constraints.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnshkrdotcom%2Fflowstone","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnshkrdotcom%2Fflowstone","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnshkrdotcom%2Fflowstone/lists"}