{"id":50937785,"url":"https://github.com/ddobrin/llm-council-java-original","last_synced_at":"2026-06-17T11:02:40.520Z","repository":{"id":356174965,"uuid":"1231217705","full_name":"ddobrin/llm-council-java-original","owner":"ddobrin","description":"Java implementation of an LLM Council with AI framework support","archived":false,"fork":false,"pushed_at":"2026-05-22T13:46:05.000Z","size":3037,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-22T18:55:32.670Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ddobrin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-06T18:43:59.000Z","updated_at":"2026-05-22T13:44:06.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ddobrin/llm-council-java-original","commit_stats":null,"previous_names":["ddobrin/llm-council-java-original"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ddobrin/llm-council-java-original","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddobrin%2Fllm-council-java-original","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddobrin%2Fllm-council-java-original/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddobrin%2Fllm-council-java-original/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddobrin%2Fllm-council-java-original/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ddobrin","download_url":"https://codeload.github.com/ddobrin/llm-council-java-original/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddobrin%2Fllm-council-java-original/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34445186,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-17T02:00:05.408Z","response_time":127,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-17T11:02:35.641Z","updated_at":"2026-06-17T11:02:40.514Z","avatar_url":"https://github.com/ddobrin.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLM Council - Java Implementation\n\nA Java implementation evolving the Language Model Council idea presented in the [arXiv paper](https://arxiv.org/pdf/2406.08598) and popularized by [Karpathy's LLM Council](https://github.com/karpathy/llm-council) using Spring Boot 4.x, Spring AI 2.0.0, Java 25, and Vaadin Hilla for the UI.\n\nMultiple LLMs deliberate together through a structured 5-stage process: each model answers independently, ranks peers anonymously, identifies agreements and disagreements, and a designated chairman synthesizes the final response.\n\n![LLM Council](docs/LLMCouncil.png)\n\n## Architecture\n\n```\n┌──────────────────────────────────────────────────────────────────────────────┐\n│                            Vaadin Hilla Frontend                            │\n│  ┌──────────────────┐  ┌────────────────┐  ┌──────────────────────────────┐ │\n│  │ IndividualReview │  │  PeerRanking   │  │        AnalysisPanel         │ │\n│  │      Panel       │  │     Panel      │  │  (Agreement / Disagreement)  │ │\n│  └────────┬─────────┘  └───────┬────────┘  └──────────────┬───────────────┘ │\n│           │                    │                           │                 │\n│  ┌────────┴────────────────────┴───────────────────────────┴───────────────┐ │\n│  │          FinalSynthesisPanel + ConsensusMetricsPanel + Query Input      │ │\n│  └────────────────────────────────────────────────────────────────────────┘ │\n└───────────────────────────────────┬──────────────────────────────────────────┘\n                                    │ Hilla @BrowserCallable\n              ┌─────────────────────┴──────────────────────┐\n              │              CouncilEndpoint                │\n              └─────────────────────┬──────────────────────┘\n                                    │\n              ┌─────────────────────┴──────────────────────┐\n              │              CouncilService                 │\n              │    (reactive, parallel LLM orchestration)   │\n              └─────────────────────┬──────────────────────┘\n                                    │\n              ┌─────────────────────┴──────────────────────┐\n              │            ChatClientRegistry               │\n              │       (model-id → ChatClient routing)       │\n              └────────┬──────────────┬──────────────┬─────┘\n                       │              │              │\n         ┌─────────────┴──┐  ┌───────┴──────────┐  ┌┴──────────────────────┐\n         │  Anthropic API │  │ Google GenAI API  │  │  Anthropic via        │\n         │  (Claude Haiku │  │ (Gemini 3 Flash,  │  │  Vertex AI rawPredict │\n         │   4.5, Opus,   │  │  3.1 Flash Lite,  │  │  (routes Claude       │\n         │   Sonnet)      │  │  3.1 Pro)         │  │   through GCP ADC)    │\n         └────────────────┘  └───────────────────┘  └───────────────────────┘\n```\n\n## 5-Stage Deliberation Flow\n\n1. **Stage 1: Individual Responses** — All council members answer the query in parallel via live LLM API calls. Each response includes token counts and latency metrics.\n\n2. **Stage 2: Peer Ranking** — Each model receives the other members' responses anonymized as \"Response A, B, C...\" and ranks them. Rankings are aggregated into a leaderboard with consensus metrics (Kendall's W).\n\n3. **Stage 3: Agreement Analysis** — Points of consensus across all responses are identified and surfaced, grouped by topic with mention counts.\n\n4. **Stage 4: Disagreement Analysis** — Points of divergence and conflicting positions are identified and surfaced, with per-topic severity scoring.\n\n5. **Stage 5: Final Synthesis** — The designated chairman model reads all prior stage outputs and synthesizes a single authoritative final answer.\n\n### Quick Consult mode\n\nThe REST API also exposes a 3-stage variant via `POST /api/council/quick-consult`: stages 1, 2, and 5 only — peer ranking happens, but the agreement and disagreement analysis stages are skipped. The response is the same `SavedSession` JSON, with the agreement/disagreement aggregate fields returned empty. Use this when you want a chairman synthesis informed by ranked peer responses but don't need the full deliberation breakdown.\n\n## Tech Stack\n\n- **Java 25** with preview features enabled\n- **Spring Boot 4.x**\n- **Spring AI 2.0.0** — LLM abstraction layer with multi-model support\n- **Vaadin Hilla 25.1.0** — Full-stack React + Spring integration with type-safe RPC\n- **Project Reactor** — Reactive streams (`Flux` / `Mono`) for parallel async LLM calls\n- **Caffeine** — In-memory session caching (1000 sessions, 2-hour TTL)\n- **Anthropic API** — Direct integration for Claude models\n- **Google GenAI API** — Direct integration for Gemini models\n- **Anthropic Java SDK (Vertex)** — Routes Claude models through Vertex AI `rawPredict` using GCP ADC\n- **Google Cloud Storage** — Optional session persistence to GCS\n- **GraalVM** — Native image build support\n\n## Project Structure\n\n```\nsrc/main/java/dev/council/\n├── LlmCouncilApplication.java              # Spring Boot entry point\n├── client/\n│   ├── ChatClientProvider.java             # Interface for named chat clients\n│   ├── AbstractChatClient.java             # Template method base with memory + streaming\n│   ├── ChatClientRegistry.java             # Registry: model-id → ChatClient routing\n│   ├── Opus46ChatClient.java               # Claude Opus 4.6 client bean\n│   ├── Sonnet46ChatClient.java             # Claude Sonnet 4.6 client bean\n│   ├── Haiku45ChatClient.java              # Claude Haiku 4.5 client bean\n│   ├── Gemini3FlashChatClient.java         # Gemini 3 Flash client bean\n│   ├── Gemini31FlashLiteChatClient.java    # Gemini 3.1 Flash Lite client bean\n│   ├── Gemini31ProChatClient.java          # Gemini 3.1 Pro client bean\n│   └── VertexAiAnthropicRequestInterceptor.java  # Routes Claude through Vertex AI rawPredict\n├── config/\n│   ├── CouncilConfig.java                  # Spring bean wiring for council\n│   ├── CouncilProperties.java             # @ConfigurationProperties for council.yaml\n│   ├── ObservabilityConfig.java           # Trace exporter (Cloud Trace or console)\n│   ├── WebSocketConfig.java               # WebSocket buffer tuning\n│   ├── ClientWarmupConfig.java            # ChatClient warm-up on startup (synchronous)\n│   ├── MdcTraceLoggingEnhancer.java       # Bridge MDC to Cloud Logging trace fields\n│   └── NativeRuntimeHints.java            # GraalVM reflection hints\n├── model/\n│   ├── CouncilMember.java                  # LLM model representation\n│   ├── CouncilSession.java                 # Full session state + CouncilStage enum\n│   ├── IndividualResponse.java             # Stage 1: member responses\n│   ├── IndividualRanking.java              # Stage 2: per-member peer rankings\n│   ├── IndividualAgreement.java            # Stage 3: per-member agreement points\n│   ├── IndividualDisagreement.java         # Stage 4: per-member disagreement points\n│   ├── FinalResponse.java                  # Stage 5: chairman synthesis result\n│   ├── AggregateRanking.java               # Combined ranking scores\n│   ├── AggregateAgreement.java             # Aggregated agreement points\n│   ├── AggregateDisagreement.java          # Aggregated disagreement points\n│   ├── ConsensusMetrics.java               # Kendall's W + disagreement severity\n│   ├── SavedSession.java                   # Persistence wrapper with title\n│   ├── TraceSummary.java                   # Cloud Trace view model\n│   ├── TraceSpanDetail.java                # Trace span tree node\n│   ├── TraceDuration.java                  # Trace query time range\n│   └── schema/                             # JSON schema for structured LLM output\n│       ├── RankingOutput.java\n│       ├── AgreementOutput.java\n│       └── DisagreementOutput.java\n├── service/\n│   ├── CouncilService.java                 # Orchestrates all 5 stages reactively\n│   ├── ResponseParserService.java          # LLM output parsing with format fallbacks\n│   ├── TraceRetrievalService.java          # Cloud Trace API integration\n│   └── storage/\n│       └── ConversationStorage.java        # Local file + GCS session persistence\n├── endpoint/\n│   ├── CouncilEndpoint.java                # Hilla @BrowserCallable API\n│   └── CouncilApiController.java           # REST API for programmatic access\n```\n\n```\nsrc/main/frontend/\n├── App.tsx                                 # React router setup\n├── index.ts                                # Entry point\n├── views/\n│   ├── MainLayout.tsx                      # App shell with navigation drawer\n│   ├── CouncilView.tsx                     # Main council interface, stage orchestration\n│   └── TracingView.tsx                     # Cloud Trace viewer\n├── components/\n│   ├── IndividualReviewPanel.tsx           # Stage 1: per-member response tabs\n│   ├── PeerRankingPanel.tsx                # Stage 2: rankings grid + aggregate leaderboard\n│   ├── AnalysisPanel.tsx                   # Stage 3 \u0026 4: agreement / disagreement display\n│   ├── FinalSynthesisPanel.tsx             # Stage 5: chairman synthesis + query input\n│   ├── ConsensusMetricsPanel.tsx           # Kendall's W + severity visualization\n│   ├── StageHeader.tsx                     # Shared stage header component\n│   └── TraceCard.tsx                       # Trace viewer cards\n├── utils/\n│   └── council-utils.ts                    # subscribeToFlux() helper, color utils\n└── themes/llm-council/\n    └── styles.css                          # Custom theme\n```\n\n```\nsrc/main/resources/\n├── application.properties                  # Spring config + environment variable bindings\n├── application-local.properties            # Local dev overrides (disable GCP export)\n├── application-native.properties           # GraalVM native image config\n├── council.yaml                            # Council member and chairman configuration\n├── logback-spring.xml                      # Logging config (Cloud Logging JSON / plain text)\n└── system/                                 # StringTemplate prompt files\n    ├── individual-review.st                # Stage 1 prompt\n    ├── peer-ranking.st                     # Stage 2 prompt (strict format)\n    ├── agreement-analysis.st               # Stage 3 prompt\n    ├── disagreement-analysis.st            # Stage 4 prompt\n    ├── final-response.st                   # Stage 5 prompt\n    └── title-generation.st                 # Session title generation\n```\n\n```\nsrc/test/java/dev/council/\n├── ApplicationContextTest.java             # Verifies Spring context bootstraps\n├── support/\n│   └── TestFixtures.java                   # Shared in-memory fixtures\n├── client/\n│   └── AbstractChatClientTest.java         # Memoization, concurrency, recovery\n├── endpoint/\n│   ├── CouncilApiControllerTest.java       # REST surface (pure Mockito)\n│   └── CouncilEndpointTest.java            # Hilla @BrowserCallable surface\n├── model/\n│   └── IndividualResponseTest.java\n└── service/\n    ├── CouncilServiceTest.java\n    ├── CouncilServiceFanoutTest.java       # Reactive fan-out across all 5 stages\n    ├── ResponseParserServiceTest.java      # LLM output parsing fallbacks\n    └── storage/\n        └── ConversationStorageTest.java    # Local + GCS round-trips\n```\n\n### Customizing prompts\n\nThe five stage prompts and the title-generation prompt live as plain text in `src/main/resources/system/*.st`. They are [StringTemplate](https://www.stringtemplate.org/) files rendered per-stage by `CouncilService` with the council member set, the user query, and prior-stage outputs as template variables. Edit these files directly to tune behavior — no Java changes required.\n\n## Configuration\n\n### Council Members (`council.yaml`)\n\nCouncil membership, chairman selection, and available models are driven entirely by `council.yaml` — no Java changes required to add or swap models:\n\n```yaml\ncouncil:\n  models:                                   # Active council members\n    gemini-3-flash-preview:\n      id: \"gemini-3-flash-preview\"\n      name: \"Gemini 3 Flash\"\n      provider: \"Google\"\n      modelId: \"gemini-3-flash-preview\"\n      avatarColor: \"#1da1f2\"\n    gemini-3-1-flash-lite-preview:\n      id: \"gemini-3.1-flash-lite-preview\"\n      name: \"Gemini 3.1 Flash Lite\"\n      provider: \"Google\"\n      modelId: \"gemini-3.1-flash-lite-preview\"\n      avatarColor: \"#4285f4\"\n    claude-haiku-4-5:\n      id: \"claude-haiku-4-5\"\n      name: \"Claude Haiku 4.5\"\n      provider: \"Anthropic\"\n      modelId: \"claude-haiku-4-5\"\n      avatarColor: \"#d97706\"\n\n  available:                                # Models ready to activate\n    models:\n      claude-opus-4-6:\n        id: \"claude-opus-4-6\"\n        name: \"Claude Opus 4.6\"\n        provider: \"Anthropic\"\n        modelId: \"claude-opus-4-6\"\n        avatarColor: \"#10a37f\"\n      gemini-3-1-pro-preview:\n        id: \"gemini-3.1-pro-preview\"\n        name: \"Gemini 3.1 Pro\"\n        provider: \"Google\"\n        modelId: \"gemini-3.1-pro-preview\"\n        avatarColor: \"#1da1f2\"\n      claude-sonnet-4-6:\n        id: \"claude-sonnet-4-6\"\n        name: \"Claude Sonnet 4.6\"\n        provider: \"Anthropic\"\n        modelId: \"claude-sonnet-4-6\"\n        avatarColor: \"#d97706\"\n\n  chairman:\n    model: \"gemini-3-1-flash-lite-preview\"\n\n  title:\n    model: \"gemini-3-1-flash-lite-preview\"\n\n  summarize:\n    model: \"gemini-3-1-flash-lite-preview\"\n```\n\nMove entries between `council.models` and `council.available.models` to activate or deactivate them. The `chairman.model` must match one of the model IDs defined in `council.models`.\n\n#### Special model roles\n\nThree model IDs in `council.yaml` drive non-deliberation work and can each point to a different model — typically a cheap/fast one to keep auxiliary calls inexpensive:\n\n| Role | YAML key | Used for |\n|------|----------|----------|\n| Chairman | `council.chairman.model` | Stage 5 final synthesis |\n| Title | `council.title.model` | Auto-generates a session title from the query |\n| Summarize | `council.summarize.model` | Internal summarization for long contexts |\n\nEach must reference a model ID present in `council.models` (active) or `council.available.models`. They can all point to the same model.\n\n## Tested Models\n\nSome of the models this app has been tested with:\n\n| Model | Provider | Config key | Status |\n|-------|----------|------------|--------|\n| Claude Haiku 4.5 | Anthropic | `claude-haiku-4-5` | Active |\n| Claude Opus 4.6 | Anthropic | `claude-opus-4-6` | Available |\n| Claude Sonnet 4.6 | Anthropic | `claude-sonnet-4-6` | Available |\n| Gemini 3 Flash | Google | `gemini-3-flash-preview` | Active |\n| Gemini 3.1 Flash Lite | Google | `gemini-3.1-flash-lite-preview` | Active |\n| Gemini 3.1 Pro | Google | `gemini-3.1-pro-preview` | Available |\n| Gemini 2.5 Pro | Google | `gemini-2.5-pro` | Commented out |\n| Gemini 2.5 Flash | Google | `gemini-2.5-flash` | Commented out |\n\nTo enable an available model, move its entry from `council.available.models` to `council.models` in `council.yaml`. The corresponding `ChatClient` bean must also be wired in `ChatClientRegistry`.\n\nYou can configure any model, as long as it is supported in Spring AI.\n\n## Environment Variables / API Keys\n\n### Anthropic (Direct API)\n\n```bash\nexport ANTHROPIC_API_KEY=sk-ant-...\n```\n\nOnly needed when `vertex.ai.anthropic.enabled=false`. When Vertex AI routing is enabled (the default), Claude models use Google ADC instead.\n\n### Google Cloud\n\n```bash\nexport GOOGLE_CLOUD_PROJECT=my-gcp-project-id\nexport GOOGLE_CLOUD_LOCATION=us-central1\n```\n\nGoogle authentication uses Application Default Credentials. Run `gcloud auth application-default login` before starting the application if credentials are not already configured.\n\n### Vertex AI Anthropic Routing (Default)\n\nBy default, Claude models are routed through Vertex AI `rawPredict`, using Google ADC instead of an Anthropic API key:\n\n```bash\n# Enabled by default — set to false to use ANTHROPIC_API_KEY directly\nexport VERTEX_AI_ANTHROPIC_ENABLED=true\nexport VERTEX_AI_ANTHROPIC_LOCATION=global\n```\n\n### Application Properties\n\n`application.properties` binds these environment variables to the Spring AI model configurations:\n\n```properties\nspring.ai.anthropic.opus46.api-key=${ANTHROPIC_API_KEY:}\nspring.ai.google.genai.flash3.project-id=${GOOGLE_CLOUD_PROJECT}\nspring.ai.google.genai.flash3.location=${GOOGLE_CLOUD_LOCATION}\nvertex.ai.anthropic.enabled=${VERTEX_AI_ANTHROPIC_ENABLED:true}\n```\n\nYou only need credentials for the providers whose models are enabled in `council.yaml`.\n\n### Storage\n\nSession persistence destination and paths are configurable via env vars:\n\n```bash\n# local (default) writes JSON files to data/conversations/\n# cloud writes to a GCS bucket\nexport CONVERSATIONS=local            # or 'cloud'\n```\n\n| Env var | Property | Default | Notes |\n|---------|----------|---------|-------|\n| `CONVERSATIONS` | `conversation.storage.location` | `local` | `local` or `cloud` |\n| _(none — set the property directly)_ | `conversation.dir` | `data/conversations` | Used when `local` |\n| _(none — set the property directly)_ | `bucket.name` | `llm-council-conversations` | Used when `cloud`; bucket must already exist with ADC write access |\n\n## Running the Application\n\n### Prerequisites\n\n- Java 25\n- Node.js 18+\n- Maven 3.9+\n- API keys / ADC for the providers used by active council members\n\n### Local Development (recommended for dev)\n\nRun with the `local` profile to get plain-text logs and console-printed traces — no GCP credentials needed for observability:\n\n```bash\n./mvnw spring-boot:run -Dspring-boot.run.profiles=local\n```\n\n| Signal | Destination |\n|--------|-------------|\n| **Traces** | Console (`otel.spans` logger) |\n| **Metrics** | `/actuator/metrics` only (Stackdriver export disabled) |\n| **Logs** | Plain text to stdout |\n\n### Cloud / Default Profile\n\nRun without a profile to export all three signals to Google Cloud:\n\n```bash\n./mvnw spring-boot:run\n```\n\n| Signal | Destination |\n|--------|-------------|\n| **Traces** | Google Cloud Trace (via `TraceExporter` + ADC) |\n| **Metrics** | Google Cloud Monitoring (via `micrometer-registry-stackdriver`, every 60s) |\n| **Logs** | GCP Cloud Logging JSON to stdout (auto-ingested on Cloud Run/GKE/GCE) |\n\nRequires:\n\n```bash\nexport GOOGLE_CLOUD_PROJECT=my-gcp-project-id\ngcloud auth application-default login\n```\n\nLogs include `logging.googleapis.com/trace` and `logging.googleapis.com/spanId` fields for automatic log-trace correlation in Cloud Console.\n\n### Production Build\n\n```bash\n./mvnw clean package -Pproduction\njava -jar target/llm-council-1.0.0-SNAPSHOT.jar\n```\n\nTo run with the local profile:\n\n```bash\njava -jar target/llm-council-1.0.0-SNAPSHOT.jar --spring.profiles.active=local\n```\n\n### Native Image (GraalVM) \n\n```bash\n./mvnw clean package -Pnative,production -DskipTests\n\n# local profile, no OTEL\n./target/llm-council --spring.profiles.active=native,local\n\n# GCP telemetry enabled\n./target/llm-council --spring.profiles.active=native\n```\n\n### Cloud Run Deployment\n\nDeploy from source to Cloud Run:\n\n```bash\ngcloud run deploy llm-council \\\n  --source . \\\n  --project my-gcp-project-id \\\n  --region us-central1 \\\n  --set-env-vars GOOGLE_CLOUD_PROJECT=my-gcp-project-id,GOOGLE_CLOUD_LOCATION=global \\\n  --set-build-env-vars GOOGLE_BUILD_ARGS=\"-Pproduction\" \\\n  --set-secrets ANTHROPIC_API_KEY=ANTHROPIC_API_KEY:latest \\\n  --memory 2Gi --cpu 4 \\\n  --min-instances 0 --max-instances 1 \\\n  --allow-unauthenticated\n```\n\nA `.gcloudignore` file excludes `target/`, `node_modules/`, `native/`, `.git/`, and documentation from the Cloud Build upload.\n\n#### Speeding up Cloud Build\n\nCloud Build from source has no dependency cache between builds. To avoid re-downloading all Maven dependencies on every deploy, pre-populate a local cache:\n\n```bash\n./scripts/cache-deps.sh\n```\n\nThis creates a `.m2-cache/` directory with all production dependencies (~1-2 GB). The `mvnw` wrapper automatically uses it when present, so Cloud Build finds all JARs locally.\n\nFor faster compilation, use a two-step deploy with a higher-CPU Cloud Build machine:\n\n```bash\n# Step 1: Build with more CPUs\ngcloud builds submit \\\n  --project my-gcp-project-id \\\n  --region us-central1 \\\n  --machine-type E2_HIGHCPU_8 \\\n  --pack image=us-central1-docker.pkg.dev/my-gcp-project-id/cloud-run-source-deploy/llm-council \\\n  --set-build-env-vars GOOGLE_BUILD_ARGS=\"-Pproduction\"\n\n# Step 2: Deploy the pre-built image\ngcloud run deploy llm-council \\\n  --project my-gcp-project-id \\\n  --region us-central1 \\\n  --image us-central1-docker.pkg.dev/my-gcp-project-id/cloud-run-source-deploy/llm-council \\\n  --set-env-vars GOOGLE_CLOUD_PROJECT=my-gcp-project-id,GOOGLE_CLOUD_LOCATION=global \\\n  --set-secrets ANTHROPIC_API_KEY=ANTHROPIC_API_KEY:latest \\\n  --memory 2Gi --cpu 4 \\\n  --min-instances 0 --max-instances 1 \\\n  --allow-unauthenticated\n```\n\n### Claude Code integration\n\nIf you use [Claude Code](https://claude.com/claude-code), this repo registers a `/consult-council` slash command (defined in `.claude/commands/consult-council.md`). Run it from any Claude Code session pointed at this directory and it will POST your question to a locally running council instance via `/api/council/consult`. Requires the app to be running on `localhost:8080`.\n\n## REST API\n\nIn addition to the Hilla streaming UI, the council exposes a blocking REST surface in `CouncilApiController` for programmatic access:\n\n| Method | Path | Stages run | Returns |\n|--------|------|------------|---------|\n| `POST` | `/api/council/consult` | 1, 2, 3, 4, 5 (full deliberation) | `SavedSession` JSON |\n| `POST` | `/api/council/quick-consult` | 1, 2, 5 (skips agreement \u0026 disagreement) | `SavedSession` JSON (agreement / disagreement fields empty) |\n\nBoth endpoints accept the same payload:\n\n```json\n{ \"query\": \"your question here\" }\n```\n\nExample — full deliberation:\n\n```bash\ncurl -sf -X POST http://localhost:8080/api/council/consult \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"query\": \"Should we adopt event sourcing for the order service?\"}' \\\n  --max-time 300\n```\n\nExample — quick consult (faster, no agreement/disagreement breakdown):\n\n```bash\ncurl -sf -X POST http://localhost:8080/api/council/quick-consult \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"query\": \"What are tradeoffs of CQRS for a small team?\"}' \\\n  --max-time 180\n```\n\nThese endpoints **block** the HTTP request thread for the full deliberation, bounded by `council.api.timeout` (default 5 min). For interactive use prefer the Hilla UI, which streams stage results back over WebSocket.\n\n## Testing\n\n```bash\n./mvnw test\n```\n\nThe unit-test suite uses pure Mockito and `reactor-test`'s `StepVerifier`; no Spring context, no live LLM calls. The major test classes:\n\n- `ApplicationContextTest` — Spring context bootstraps cleanly with all wired beans\n- `AbstractChatClientTest` — chat-client memoization, concurrent first-call dedup, recovery after a failed init\n- `CouncilServiceFanoutTest` — reactive fan-out across all five stages, including session-not-found and error branches\n- `CouncilServiceTest` — service-level orchestration paths\n- `CouncilApiControllerTest` — REST surface (`/consult`, `/quick-consult`) end-to-end with mocked service\n- `CouncilEndpointTest` — Hilla `@BrowserCallable` surface (session create, save, error mapping)\n- `ResponseParserServiceTest` — LLM output parsing with format fallbacks\n- `ConversationStorageTest` — local file and GCS persistence round-trips\n- `IndividualResponseTest` — model-level invariants (e.g. blank/null content treated as error)\n\nShared helpers live under `src/test/java/dev/council/support/` (`TestFixtures`).\n\n## Tuning / Operator Configuration\n\nThe application has several tunable properties beyond model selection. Most have safe defaults; override them via `application.properties`, env-specific profile properties, or `-D` flags.\n\n| Property | Default | Why change it |\n|----------|---------|---------------|\n| `council.api.timeout` | `PT5M` | Blocking ceiling for the full REST deliberation |\n| `council.endpoint.session-timeout` | `PT15S` | Hilla `createSession()` block — increase if title generation is slow |\n| `council.endpoint.save-timeout` | `PT1M` | Hilla `saveCompletedSession()` block — increase for slow GCS writes |\n| `server.tomcat.threads.max` | `30` | Headroom for concurrent blocking endpoint calls |\n| `server.tomcat.threads.min-spare` | `10` | Idle thread-pool floor |\n| `vaadin.atmosphere.maxWebSocketMessageSize` | `1048576` (1 MB) | Stage 2 peer-ranking responses can be large |\n| `server.tomcat.max-http-form-post-size` | `10MB` | Bound on form-post bodies; LLM completions can be large |\n| `logging.level.dev.council` | `INFO` | `DEBUG` adds per-Flux-element trace and prompt template rendering |\n| `logging.level.org.springframework.ai` | `INFO` | `DEBUG` plus `spring.ai.chat.observations.log-prompt=true` and `log-completion=true` logs full prompt and completion text. **Leave off in production** to avoid logging sensitive content. |\n\n**Client warmup.** `ClientWarmupConfig` triggers each `ChatClient` bean at startup so the first user request doesn't pay cold-start initialization cost. Synchronous — blocks application readiness until done.\n\n**Session cache.** Caffeine, **1000 entries / 2-hour TTL**, currently hard-coded in `CouncilService.java:243-245` (`maximumSize(1000)` / `expireAfterWrite(Duration.ofHours(2))`). Not yet exposed as a property — flagged as a future tuning knob.\n\n## Observability\n\nThe application implements three-signal observability: traces, metrics, and logs. The `local` profile switches all signals to console output; the default profile exports to Google Cloud.\n\n### Traces (OpenTelemetry)\n\nEach deliberation creates a span hierarchy:\n\n```\ncouncil.deliberation.wrapper                    (root span, per session)\n  └── council.stage1.individual-review          (parent span)\n        ├── gen_ai.client.operation             (per ChatModel call, one per member)\n        ├── gen_ai.client.operation\n        └── gen_ai.client.operation\n  └── council.stage2.peer-ranking\n        └── ...\n```\n\nSpan attributes include `council.stage`, `council.session.id`, `gen_ai.request.model`, and token counts.\n\n### Metrics (Micrometer)\n\nCustom council metrics (timers):\n\n```bash\ncurl -s localhost:8080/actuator/metrics | jq '.names[]' | grep -E 'council|gen_ai'\n\n# Per-stage timers\ncurl -s localhost:8080/actuator/metrics/council.stage1.individual-review | jq\ncurl -s localhost:8080/actuator/metrics/council.stage2.peer-ranking | jq\ncurl -s localhost:8080/actuator/metrics/council.stage3.agreement-analysis | jq\ncurl -s localhost:8080/actuator/metrics/council.stage4.disagreement-analysis | jq\ncurl -s localhost:8080/actuator/metrics/council.stage5.final-synthesis | jq\n\n# Spring AI auto-generated\ncurl -s localhost:8080/actuator/metrics/gen_ai.client.token.usage | jq\n```\n\nIn the default profile, metrics are exported to Google Cloud Monitoring (Stackdriver) every 60 seconds.\n\n### Logs\n\n- **Local profile**: Plain text to stdout via Logback\n- **Default profile**: JSON to Google Cloud Logging with `logging.googleapis.com/trace` and `logging.googleapis.com/spanId` fields for automatic log-trace correlation\n\n### Trace Viewer\n\nThe built-in Trace Viewer (`TracingView.tsx`) is mounted at `/traces` (e.g. `http://localhost:8080/traces`). It queries Google Cloud Trace and displays deliberation traces in the UI, with span trees, token counts, and links to Cloud Console. Requires `GOOGLE_CLOUD_PROJECT` to be set and ADC configured for read access to the trace API.\n\n## Session Persistence\n\nCompleted deliberation sessions can be saved and reloaded:\n\n- **Local storage** (default): JSON files in `data/conversations/`\n- **Cloud storage**: Google Cloud Storage bucket (`llm-council-conversations`)\n\nConfigure via `conversation.storage.location` (`local` or `cloud`) in `application.properties`, or set the `CONVERSATIONS` env var. See [Storage](#storage) under *Environment Variables / API Keys* for the full configuration surface.\n\n## License\nApache 2.0 License\n\n## Disclaimer\nThis is not an official Google product. \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fddobrin%2Fllm-council-java-original","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fddobrin%2Fllm-council-java-original","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fddobrin%2Fllm-council-java-original/lists"}