{"id":48926963,"url":"https://github.com/msu-denver/bili-core","last_synced_at":"2026-06-12T07:01:53.997Z","repository":{"id":277946219,"uuid":"930561837","full_name":"msu-denver/bili-core","owner":"msu-denver","description":"Open-source framework for building and testing LLM-powered applications: IRIS (single-agent orchestration), AETHER (declarative multi-agent systems), and AEGIS (adversarial security testing). Developed at MSU Denver's Community-Centered Computing (C3) Lab.","archived":false,"fork":false,"pushed_at":"2026-06-04T18:03:24.000Z","size":3541,"stargazers_count":14,"open_issues_count":24,"forks_count":2,"subscribers_count":7,"default_branch":"develop","last_synced_at":"2026-06-04T21:07:35.826Z","etag":null,"topics":["adversarial-testing","agentic-ai","ai","benchmarking","c3-lab","langchain","langgraph","llm","msu-denver","multi-agent-systems","nsf","open-source","python","rag","red-teaming","research","security-testing"],"latest_commit_sha":null,"homepage":"https://c3-lab.org/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/msu-denver.png","metadata":{"files":{"readme":"README.MD","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-02-10T20:45:12.000Z","updated_at":"2026-06-04T18:01:58.000Z","dependencies_parsed_at":"2025-05-20T21:30:55.731Z","dependency_job_id":"7256f1e9-6d1a-4830-9ecb-bc82852a465d","html_url":"https://github.com/msu-denver/bili-core","commit_stats":null,"previous_names":["msu-denver/bili-core"],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/msu-denver/bili-core","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msu-denver%2Fbili-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msu-denver%2Fbili-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msu-denver%2Fbili-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msu-denver%2Fbili-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/msu-denver","download_url":"https://codeload.github.com/msu-denver/bili-core/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/msu-denver%2Fbili-core/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34232790,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-12T02:00:06.859Z","response_time":109,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-testing","agentic-ai","ai","benchmarking","c3-lab","langchain","langgraph","llm","msu-denver","multi-agent-systems","nsf","open-source","python","rag","red-teaming","research","security-testing"],"created_at":"2026-04-17T07:04:30.481Z","updated_at":"2026-06-12T07:01:53.988Z","avatar_url":"https://github.com/msu-denver.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"bili/images/logo.png\" alt=\"BiliCore Logo\" width=\"150\"/\u003e\n\u003c/p\u003e\n\n# BiliCore: An Open-Source LLM Framework\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/msu-denver/bili-core/actions/workflows/ci-cd.yml\"\u003e\u003cimg src=\"https://github.com/msu-denver/bili-core/actions/workflows/ci-cd.yml/badge.svg?branch=develop\" alt=\"CI/CD\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/msu-denver/bili-core/releases/latest\"\u003e\u003cimg src=\"https://img.shields.io/github/v/release/msu-denver/bili-core\" alt=\"Latest Release\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://www.python.org/downloads/\"\u003e\u003cimg src=\"https://img.shields.io/badge/python-3.11+-blue.svg\" alt=\"Python 3.11+\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://opensource.org/licenses/MIT\"\u003e\u003cimg src=\"https://img.shields.io/badge/License-MIT-yellow.svg\" alt=\"License: MIT\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n**BiliCore** is an open-source, domain-agnostic framework for building and testing LLM-powered applications. It provides single-agent orchestration, multi-agent system creation, and adversarial security testing in one modular package.\n\nOriginally developed for the [**Colorado Sustainability Hub**](https://sustainabilityhub.co/) at MSU Denver's [**Community-Centered Computing (C3) Lab**](https://c3-lab.org/), bili-core now serves as the framework behind multiple research initiatives at the lab. Funded by the **[National Science Foundation (NSF)](https://www.nsf.gov/)** and the **[NAIRR Pilot](https://nairrpilot.org/)**.\n\n---\n\n## Three Components\n\nBiliCore is organized into three named components, each solving a distinct problem:\n\n### IRIS — Interactive Reasoning and Integration Services\n**Single-agent orchestration.** 60+ models across 6 providers.\n\nIRIS bridges users to LLMs, tools, and data sources. It provides a node-based workflow pipeline where each step (persona injection, tool execution, memory management, response normalization) is a composable node. Switch models mid-conversation, configure tools on the fly, and persist state across sessions.\n\n- **Providers:** AWS Bedrock, Google Vertex AI, Azure OpenAI, OpenAI, Ollama, local models\n- **Tools:** FAISS vector search, OpenSearch, weather APIs, web search, extensible tool registry\n- **Middleware:** Summarization, model call limiting, custom middleware\n- **Checkpointers:** MongoDB, PostgreSQL, in-memory — all with queryable conversation management\n- **Streaming:** Token-by-token responses via sync and async APIs\n- **Location:** `bili/iris/`\n\n### AETHER — Agent Ecosystems for Testing, Hardening, Evaluation, and Research\n**Multi-agent orchestration.** Declarative YAML configuration.\n\nAETHER lets you define multi-agent systems (MAS) in YAML and compile them into executable LangGraph workflows. Each agent can have its own LLM, tools, persona, and multi-node processing pipeline. Agents communicate through typed channels with configurable protocols.\n\n- **7 workflow types:** Sequential, hierarchical, supervisor, consensus, parallel, deliberative, custom\n- **6 communication protocols:** Direct, broadcast, request-response, pub-sub, competitive, consensus\n- **Pipeline sub-graphs:** Multi-node pipelines within individual agents\n- **Custom state fields:** Type-safe YAML state declarations with reducers and defaults\n- **Runtime injection:** `RuntimeContext` container for dependency injection into pipeline nodes\n- **Streaming:** `MASExecutor` with structured `StreamEvent` objects and `StreamFilter`\n- **Location:** `bili/aether/`\n\n### AEGIS — Adversarial Evaluation and Guarding of Intelligent Systems\n**Security testing for multi-agent systems.** Built on AETHER.\n\nAEGIS provides a systematic framework for testing how adversarial payloads propagate through multi-agent systems. It injects attacks at different phases (pre-execution, mid-execution, checkpoint), tracks propagation across agents, and evaluates compliance using a 3-tier detection system.\n\n- **7 test suites:** Prompt injection, jailbreak, memory poisoning, bias inheritance, agent impersonation, persistence, cross-model transferability\n- **3-tier detection:** Structural (CI-safe), heuristic (propagation tracking), semantic (LLM-based scoring)\n- **Baseline comparison:** Ground-truth runner for controlled before/after analysis\n- **Results viewer:** Interactive Streamlit dashboards for attack results and baseline analysis\n- **Attack GUI:** Run adversarial attacks interactively with graph visualization\n- **Location:** `bili/aegis/`\n\n---\n\n## Quick Start\n\n### Prerequisites\n\n- **Docker**: [Get Docker](https://docs.docker.com/get-docker/) — all services run in containers\n- **Git**: To clone the repository\n\n### 1. Clone and configure\n\n```bash\ngit clone https://github.com/msu-denver/bili-core.git\ncd bili-core\ncp .env.example .env\n# Edit .env with your API keys (AWS, Google, OpenAI, etc.)\n```\n\n### 2. Start the development environment\n\n```bash\ncd scripts/development\n./start-container.sh\n./attach-container.sh\n```\n\nThis starts the bili-core container along with PostgreSQL (with PostGIS), MongoDB, and LocalStack services. The container automatically activates a Python virtual environment and sets up shell aliases.\n\n### 3. Run the application\n\nInside the container:\n\n```bash\nstreamlit    # Start the Streamlit UI on port 8501\nflask        # Start the Flask API on port 5001\n```\n\n### 4. Access the application\n\n- **Streamlit UI:** http://localhost:8501\n  - `/aether` — AETHER Multi-Agent system (visualizer, chat, attack suite)\n  - `/bili` — Single-Agent RAG testing interface\n  - `/attack-results` — AEGIS attack results viewer\n  - `/results` — Baseline results viewer\n- **Flask API:** http://localhost:5001\n\n---\n\n## Architecture Overview\n\n```\nbili-core/\n├── bili/\n│   ├── iris/                  # IRIS: Single-agent orchestration\n│   │   ├── loaders/           #   Graph builder, streaming, tool/middleware/LLM loaders\n│   │   ├── nodes/             #   Pipeline nodes (persona, datetime, react agent, etc.)\n│   │   ├── graph_builder/     #   Node and edge class definitions\n│   │   ├── config/            #   LLM, tool, and middleware configurations\n│   │   ├── tools/             #   Tool implementations (FAISS, OpenSearch, weather, etc.)\n│   │   └── checkpointers/     #   State persistence (MongoDB, PostgreSQL, memory)\n│   │\n│   ├── aether/                # AETHER: Multi-agent orchestration\n│   │   ├── schema/            #   MASConfig, AgentSpec, WorkflowType, Channel definitions\n│   │   ├── compiler/          #   YAML → LangGraph compilation (graph builder, LLM resolver)\n│   │   ├── runtime/           #   MASExecutor, streaming, communication state\n│   │   ├── config/examples/   #   Example YAML configurations\n│   │   ├── integration/       #   Checkpointer factory for MAS\n│   │   ├── validation/        #   Static MAS validation engine\n│   │   └── ui/                #   Streamlit pages (chat, visualizer, attack, results)\n│   │\n│   ├── aegis/                 # AEGIS: Adversarial security testing\n│   │   ├── attacks/           #   Attack injector, propagation tracker, strategies\n│   │   ├── evaluator/         #   Semantic evaluator, scoring rubrics\n│   │   ├── security/          #   Security event detector, logger\n│   │   └── tests/             #   7 attack suites + baseline + analysis\n│   │\n│   ├── auth/                  # Shared: Authentication (Firebase, SQLite, in-memory)\n│   ├── utils/                 # Shared: Logging, LangGraph utilities, file I/O\n│   ├── prompts/               # Shared: Prompt templates\n│   ├── streamlit_ui/          # Shared: Streamlit UI components\n│   ├── flask_api/             # Shared: Flask API utilities\n│   ├── streamlit_app.py       # Unified Streamlit entry point\n│   └── flask_app.py           # Flask API entry point\n│\n├── docs/                      # Project-level documentation\n├── scripts/                   # Development and build scripts\n├── .env.example               # Environment variable template\n├── docker-compose.yml         # Full development stack\n└── requirements.txt           # Python dependencies\n```\n\n---\n\n## Code Examples\n\n### IRIS: Single-Agent Streaming\n\n```python\nfrom bili.iris.loaders.langchain_loader import build_agent_graph\nfrom bili.iris.loaders.streaming_utils import stream_agent, invoke_agent\n\nagent = build_agent_graph(checkpoint_saver=saver, node_kwargs=kwargs)\n\n# Non-streaming\nresponse = invoke_agent(agent, \"What is the weather?\", thread_id=\"user1\")\n\n# Streaming — yields tokens as they arrive\nfor token in stream_agent(agent, \"What is the weather?\", thread_id=\"user1\"):\n    print(token, end=\"\", flush=True)\n```\n\n### AETHER: Multi-Agent System\n\n```python\nfrom bili.aether import load_mas_from_yaml, compile_mas, execute_mas\n\nconfig = load_mas_from_yaml(\"bili/aether/config/examples/simple_chain.yaml\")\nresult = execute_mas(config, {\"messages\": [\"Analyze quantum computing trends\"]})\nprint(result.get_summary())\n```\n\n### AETHER: Streaming Multi-Agent\n\n```python\nfrom bili.aether.runtime import MASExecutor, StreamEventType\n\nexecutor = MASExecutor(config)\nexecutor.initialize()\n\nfor event in executor.stream(input_data):\n    if event.event_type == StreamEventType.TOKEN:\n        print(event.data[\"content\"], end=\"\", flush=True)\n```\n\n### AEGIS: Run a Security Test Suite\n\n```bash\n# Stub mode (no LLM calls — validates framework execution)\npython bili/aegis/suites/injection/run_injection_suite.py --stub\n\n# Full run (requires API credentials)\npython bili/aegis/suites/injection/run_injection_suite.py\n\n# Generate statistics report\npython bili/aegis/suites/analysis/generate_stats.py\n```\n\n---\n\n## Authentication\n\nBiliCore provides three authentication providers:\n\n| Provider | Use Case | Auto-Approve? | Configuration |\n|----------|----------|---------------|---------------|\n| **SQLite** | Local development (default) | Yes — `researcher` role | `PROFILE_DB_PATH` env var |\n| **Firebase** | Production (AWS) | No — admin approval | Firebase credentials in `.env` |\n| **In-Memory** | Testing | Yes | No configuration needed |\n\nConfigure in `bili/streamlit_app.py` via `initialize_auth_manager(auth_provider_name=...)`.\n\n---\n\n## Development\n\n### Container Aliases\n\nInside the development container:\n\n| Alias | Description |\n|-------|-------------|\n| `streamlit` | Install deps, create PG database, start Streamlit UI (port 8501) |\n| `flask` | Install deps, create PG database, start Flask API (port 5001) |\n| `deps` | Install/update Python dependencies |\n| `cleandeps` | Clean reinstall of dependencies |\n| `seeds3` | Upload data files to LocalStack S3 |\n| `createpgdb` | Create the LangGraph PostgreSQL database |\n\n### Code Quality\n\nAll code must pass formatters and linting before committing (enforced via pre-commit hooks):\n\n```bash\n./run_python_formatters.sh       # Run all formatters (Black, Autoflake, Isort)\npylint bili/ --fail-under=9      # Lint check (must score 9+/10)\n```\n\n### Running Tests\n\n```bash\n# Inside the container\npytest bili/iris/                  # IRIS unit tests\npytest bili/aether/tests/          # AETHER unit tests\npytest bili/aegis/suites/test_*.py  # AEGIS unit tests\n```\n\n### Environment Variables\n\nCopy `.env.example` to `.env` and fill in your API keys. Docker Compose reads this file automatically.\n\n- **AWS credentials:** `env/bili_root/.aws/`\n- **Google credentials:** `env/bili_root/.google/`\n- **API keys:** Set in `.env` (OpenAI, SerpAPI, weather APIs, etc.)\n\n---\n\n## Migration from v4.x to v5.0\n\nv5.0 reorganizes the codebase into the three-component architecture. Import paths have changed:\n\n| Old Path | New Path |\n|----------|----------|\n| `bili.loaders.*` | `bili.iris.loaders.*` |\n| `bili.nodes.*` | `bili.iris.nodes.*` |\n| `bili.graph_builder.*` | `bili.iris.graph_builder.*` |\n| `bili.config.*` | `bili.iris.config.*` |\n| `bili.tools.*` | `bili.iris.tools.*` |\n| `bili.checkpointers.*` | `bili.iris.checkpointers.*` |\n| `bili.aether.attacks.*` | `bili.aegis.attacks.*` |\n| `bili.aether.evaluator.*` | `bili.aegis.evaluator.*` |\n| `bili.aether.security.*` | `bili.aegis.security.*` |\n| `bili.aether.tests.injection.*` | `bili.aegis.suites.injection.*` |\n| *(other attack suites)* | `bili.aegis.suites.\u003csuite\u003e.*` |\n\n**Unchanged paths:** `bili.aether.*` (schema, compiler, runtime, config, UI), `bili.auth.*`, `bili.utils.*`, `bili.prompts.*`\n\nAll functionality is preserved — only the locations changed.\n\n---\n\n## Component Documentation\n\n- **IRIS:** See `bili/iris/` source code and inline documentation\n- **AETHER:** [`bili/aether/README.md`](bili/aether/README.md) — comprehensive MAS documentation\n- **AEGIS:** [`bili/aegis/docs/security-testing-quickstart.md`](bili/aegis/docs/security-testing-quickstart.md) — security testing guide\n\n---\n\n## Acknowledgments\n\nbili-core is developed at MSU Denver's [Community-Centered Computing (C3) Lab](https://c3-lab.org/), which houses research projects spanning sustainability, education, and community-centered computing. This work is supported by the [National Science Foundation (NSF) (Grant No. 2318730)](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2318730) and the [National Artificial Intelligence Research Resource (NAIRR) Pilot](https://nairrpilot.org/projects/awarded?_requestNumber=NAIRR240197). Their support has been instrumental in advancing AI accessibility and fostering innovation in sustainability-focused applications.\n\nFor more information, visit the [C3 Lab website](https://c3-lab.org/) or the [Sustainability Hub website](https://sustainabilityhub.co/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmsu-denver%2Fbili-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmsu-denver%2Fbili-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmsu-denver%2Fbili-core/lists"}