{"id":13749219,"url":"https://github.com/sdleffler/whisper","last_synced_at":"2025-07-14T14:31:22.426Z","repository":{"id":86101108,"uuid":"240172174","full_name":"sdleffler/whisper","owner":"sdleffler","description":"Logic programming, for Rust, from inside Rust.","archived":false,"fork":false,"pushed_at":"2020-03-06T00:58:51.000Z","size":309,"stargazers_count":39,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-05-22T21:35:11.780Z","etag":null,"topics":["backtracking-search","dsl","logic-programming","rust","rust-lang","unification"],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sdleffler.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-02-13T03:54:24.000Z","updated_at":"2024-04-20T14:27:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"54bf9dc3-767e-4f76-9cd1-c5c8d01584a5","html_url":"https://github.com/sdleffler/whisper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sdleffler%2Fwhisper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sdleffler%2Fwhisper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sdleffler%2Fwhisper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sdleffler%2Fwhisper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sdleffler","download_url":"https://codeload.github.com/sdleffler/whisper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225980897,"owners_count":17554919,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backtracking-search","dsl","logic-programming","rust","rust-lang","unification"],"created_at":"2024-08-03T07:00:57.218Z","updated_at":"2024-11-22T23:33:55.144Z","avatar_url":"https://github.com/sdleffler.png","language":"Rust","funding_links":[],"categories":["Projects"],"sub_categories":["Libraries"],"readme":"**Whisper is very much work-in-progress and still being designed. It is not in any state for contributions (probably?) except maybe by chatting with me cuz that sounds like fun. You can chat with me on [Twitter](https://twitter.com/sleffy_), on Discord (sleffy#6314), or by email. That last one's a treasure hunt, I'm sure you can find it.**\n\n# Whisper: a Logic Programming DSL for Rust\n\nWhisper is a small, embeddable logic programming language in pure Rust. The `whisper` and `whisper_ir` crates\ninclude functionality for easily constructing Whisper syntax and compiling it into bytecode\nfor execution. It supports a simplistic foreign function interface for calling into external\ncode, as well as facilities for supporting backtracking for external goals. This makes it\npossible to use Whisper for doing a sort of traced proof-search, where if Whisper finds a\nsolution, you can use external goals to track what steps it took to get there, and react\naccordingly.\n\nAdditionally through Serde, Whisper supports converting Rust structs to and from its internal term\nrepresentation. This means Whisper can be used for doing unification and general reasoning about\nRust terms, and then extracting the result back into native Rust.\n\nDue to its structure as a logic programming language, it can also work well as an embedded read-heavy\ndatabase. Writing would require recompiling a knowledge base, which is currently not terribly\nperformant since emitting a compiled heap means reading through the entire knowledge base. This could\nbe improved through incremental compilation but is not a high priority.\n\n## Motivation\n\nLogic programming is a very convenient system for problems which need inference, but mixing\nmultiple programming languages can be very inconvenient. Whisper is intended to serve as a\nsimple, embeddable, and usably performant solution for embedding logic programs in Rust.\n\nWhisper is *not* intended as:\n- A super-fast, feature-rich Pro/Hilog implementation which just happens to have good Rust interop.\n- A database capable of streaming to/from disk.\n- A super-small interpreter with a tiny dependency footprint.\n- A fully-fledged language of its own, which can run without any need for a host program.\n\n## Features\n\n- Simple, human-readable syntax, with limited support for scoping constants\n- A simple, easy to build and modify intermediate representation (IR)\n- Macros for generating Whisper syntax programmatically, with quasiquoting support\n- A parser for parsing files containing Whisper syntax\n- Simple but powerful interface for external goals written in Rust, with support for properly handling\n  backtracking\n- Simple but powerful interface for external datatypes, with support for handling unifying two external\n  values against each other\n\n## Example: Typechecker for the Simply Typed Lambda Calculus\n\nTaken from `whisper/examples/stlc.rs`.\n\n```rust\nuse ::{\n    serde::{Deserialize, Serialize},\n    whisper::{\n        builder::QueryBuilder,\n        ir::{IrNode, IrTermGraph},\n        session::DebugHandler,\n        Heap, Session, Symbol,\n    },\n};\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub enum Type {\n    Base(i32),\n    Fun(Box\u003c(Type, Type)\u003e),\n}\n\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub enum Term {\n    Const(i32, Type),\n    Var(i32),\n    Lam(i32, Type, Box\u003cTerm\u003e),\n    App(Box\u003c(Term, Term)\u003e),\n}\n\nwhisper::module! {\n    fn stlc();\n\n    context Gamma proves Term is_type Sigma if\n        context Gamma proves Term is_type Sigma in extern,\n        fail;\n\n    // Rule 1.) Variables can be typed if they are in the context.\n    context { X: Sigma | Gamma } proves (\"Var\" : X) is_type Sigma;\n    context { Y: Tau   | Gamma } proves (\"Var\" : X) is_type Sigma if\n        context Gamma proves (\"Var\" : X) is_type Sigma;\n\n    // Rule 2.) Constants are automatically typed.\n    context Gamma proves (\"Const\" : (C Tau)) is_type Tau;\n\n    // Rule 3.) If adding `X: Sigma` to the context gives `E` some type `Tau`,\n    // then the lambda expression `\\X. E` can be typed as a function\n    // `Sigma -\u003e Tau`.\n    context Gamma proves (\"Lam\" : (X Sigma E)) is_type (\"Fun\" : (Sigma Tau)) if\n        context { X: Sigma | Gamma } proves E is_type Tau;\n\n    // Rule 4.) If `E1` is a function `Sigma -\u003e Tau` and `E2` is `Sigma`, then\n    // applying `E1` to `E2` gets us a term of type `Tau`.\n    context Gamma proves (\"App\" : (E1 E2)) is_type Tau if\n        context Gamma proves E1 is_type (\"Fun\" : (Sigma Tau)),\n        context Gamma proves E2 is_type Sigma;\n\n    context Gamma proves Term is_type Sigma if\n        context Gamma proves Term is_type Sigma failed in extern,\n        fail;\n}\n\nwhisper::query! {\n    fn stlc_infer(term: \u0026IrNode);\n\n    // Note that if the line below weren't commented out, #term is\n    // parsed as `(#term)` because it is of type `IrNode` and\n    // therefore `query!` has to wrap it in a single-arity tuple.\n    //\n    // where Term is #term,\n    context { 1: Ligma } proves #term is_type Tau;\n}\n\n#[derive(Debug)]\npub struct Typechecker {\n    terms: IrTermGraph,\n    root: Symbol,\n    session: Session\u003cDebugHandler\u003e,\n}\n\nimpl Typechecker {\n    pub fn new() -\u003e Self {\n        use whisper::{ir::IrKnowledgeBase, SymbolTable};\n        let symbols = SymbolTable::new();\n        let mut terms = IrTermGraph::new(symbols.clone());\n        let mut modules = IrKnowledgeBase::new(symbols.clone());\n\n        let stlc_module = modules.new_named_module_with_root(Symbol::PUBLIC);\n        stlc(\u0026mut terms, \u0026mut modules, stlc_module);\n        modules.link(\u0026mut terms, stlc_module);\n\n        let root = modules[stlc_module].get_root().clone();\n\n        let stlc_kb = whisper::trans::knowledge_base(\u0026terms, \u0026modules);\n        println!(\n            \"Compiled knowledge base:\\n{}\",\n            stlc_kb.get(Symbol::PUBLIC_INDEX).unwrap().display()\n        );\n        let stlc_session = Session::new(symbols.clone(), stlc_kb.into());\n\n        Typechecker {\n            terms,\n            root,\n            session: stlc_session,\n        }\n    }\n\n    pub fn infer(\u0026mut self, term: \u0026Term) -\u003e Option\u003cType\u003e {\n        let mut builder = QueryBuilder::new(Heap::new(self.terms.symbol_table().clone()));\n        let term_addr = builder.bind(term);\n        let ir_query = stlc_infer(\u0026mut self.terms, \u0026self.root, \u0026term_addr);\n        let query = builder.finish(\u0026self.terms, \u0026ir_query);\n\n        self.session.load(query.into());\n\n        if self.session.resume() {\n            let addr = self.session.query_vars()[\u0026\"Tau\".into()];\n            Some(whisper_schema::serde::de::from_reader(self.session.heap().read_at(addr)).unwrap())\n        } else {\n            None\n        }\n    }\n}\n\nfn main() {\n    let mut typechecker = Typechecker::new();\n    let term = Term::App(Box::new((\n        Term::Lam(0, Type::Base(132), Box::new(Term::Var(0))),\n        Term::Var(1),\n    )));\n    let ty = typechecker.infer(\u0026term);\n\n    println!(\"Whisper input: {:?}\", term);\n    println!(\"Whisper output: {:?}\", ty);\n}\n```\n\n## Example: Configuration Validator\n\nSee `whisper_std/tests/foo_schema.rs` for the test files as well.\n\n```rust\nuse ::{\n    failure::Error,\n    im,\n    serde::{\n        de::{DeserializeOwned, Deserializer},\n        Deserialize,\n    },\n    serde_json::Value,\n    std::marker::PhantomData,\n    whisper::{prelude::*, session::DebugHandler},\n};\n\n#[derive(Debug, Clone)]\npub struct Schema(SharedKnowledgeBase);\n\nimpl Schema {\n    pub fn from_str\u003cS: AsRef\u003cstr\u003e + ?Sized\u003e(string: \u0026S) -\u003e Self {\n        let mut terms = IrTermGraph::new(SymbolTable::new());\n        let ir_kb = terms.parse_knowledge_base_str(string).expect(\"oops\");\n        Self(whisper::trans::knowledge_base(\u0026terms, \u0026ir_kb).into())\n    }\n\n    pub fn from_embedded(embedded: fn(\u0026mut IrTermGraph) -\u003e IrKnowledgeBase) -\u003e Self {\n        let mut terms = IrTermGraph::new(SymbolTable::new());\n        let ir_kb = embedded(\u0026mut terms);\n        Self(whisper::trans::knowledge_base(\u0026terms, \u0026ir_kb).into())\n    }\n}\n\nwhisper::query! {\n    fn validator_query(input: IrNode);\n\n    valid #input;\n}\n\n#[derive(Debug)]\npub struct Validator\u003cT: DeserializeOwned\u003e {\n    symbols: SymbolTable,\n    session: Session\u003cDebugHandler\u003e,\n    _phantom: PhantomData\u003cT\u003e,\n}\n\nimpl\u003cT: DeserializeOwned\u003e Validator\u003cT\u003e {\n    pub fn new(schema: \u0026Schema) -\u003e Self {\n        Self {\n            symbols: schema.0.symbol_table().clone(),\n            session: Session::new(schema.0.symbol_table().clone(), schema.0.clone()),\n            _phantom: PhantomData,\n        }\n    }\n\n    fn build_validator_query(\u0026self, value: \u0026Value) -\u003e SharedQuery {\n        let mut terms = IrTermGraph::new(self.symbols.clone());\n        let mut builder = QueryBuilder::new(Heap::new(self.symbols.clone()));\n        let bound = builder.bind(value);\n        let ir_query = validator_query(\u0026mut terms, \u0026Symbol::MOD, bound);\n        SharedQuery::from(builder.finish(\u0026terms, \u0026ir_query))\n    }\n\n    pub fn deserialize_validated\u003c'de, D: Deserializer\u003c'de\u003e\u003e(\n        \u0026mut self,\n        deserializer: D,\n    ) -\u003e Result\u003cT, Error\u003e {\n        let serializer = serde_json::value::Serializer;\n        let json_value = serde_transcode::transcode(deserializer, serializer).unwrap();\n        self.session.load(self.build_validator_query(\u0026json_value));\n\n        if self.session.resume() {\n            Ok(serde_json::from_value(json_value)?)\n        } else {\n            failure::bail!(\"Failed to validate!\");\n        }\n    }\n}\n\nwhisper::knowledge_base! {\n    fn foo_schema();\n\n    // If TLS is disabled, we dgaf about the key/cert.\n    valid Foo if\n        Foo matches { tls_enabled: false | _ } in std::map::match;\n\n    // If TLS is enabled, we want to be sure that key/cert are both `Some`.\n    valid Foo if\n        Foo matches {\n            tls_enabled: true,\n            tls: {\n                cert: _,\n                key: _,\n            }\n        } in std::map::match;\n\n    // Eventually we'll have some stuff for error reporting in the stdlib.\n    // There are a number of ways to get data out of Whisper in a way that's\n    // convenient for aggregating errors, but for now imagine this is a println.\n    valid Foo if\n        \"failed to validate!\" Foo in extern,\n        error \"TODO: add some stdlib stuff so that we can report errors!\";\n}\n\n#[derive(Debug, Clone, Deserialize)]\npub struct TlsConfig {\n    cert: Option\u003cString\u003e,\n    key: Option\u003cString\u003e,\n}\n\n#[derive(Debug, Clone, Deserialize)]\npub struct FooConfig {\n    tls: Option\u003cTlsConfig\u003e,\n    tls_enabled: bool,\n}\n\n#[test]\nfn validate_foo() -\u003e Result\u003c(), Error\u003e {\n    // First, we construct our validator from the knowledge base we made above.\n    let mut schema = Schema::from_embedded(foo_schema);\n\n    let import_point = schema.0.symbol_table().normalize(Name {\n        root: Symbol::MOD,\n        path: im::vector![Ident::from(\"std\"), Ident::from(\"map\")],\n    });\n\n    schema\n        .0\n        .to_mut()\n        .import_serialized(\u0026import_point, whisper_std::map());\n\n    let mut validator = Validator::\u003cFooConfig\u003e::new(\u0026schema);\n\n    // Now we can try to validate a few configs! Normally you'd be reading things\n    // in from a file or something but in the interest of simplicity here we'll\n    // just `include_str!` them.\n    let foo_config_a = include_str!(\"foo_schema/foo_ok.toml\");\n    let foo_config_b = include_str!(\"foo_schema/foo_bad.toml\");\n    let foo_config_c = include_str!(\"foo_schema/foo_also_ok.toml\");\n\n    // Cool... Let's try to deserialize these! We'll start with `foo_config_a`.\n    let val = validator.deserialize_validated(\u0026mut toml::Deserializer::new(foo_config_a))?;\n    println!(\"Valid: {:#?}\", val);\n\n    // Okay... so far so good, let's try `foo_config_b`.\n    match validator.deserialize_validated(\u0026mut toml::Deserializer::new(foo_config_b)) {\n        Ok(val) =\u003e println!(\"All good! Also valid: {:#?}\", val),\n        Err(err) =\u003e println!(\"Oh no! {}\", err),\n    }\n\n    // Oops, that last one failed! Alright, last but not least, let's try `foo_config_c`.\n    let val = validator.deserialize_validated(\u0026mut toml::Deserializer::new(foo_config_c))?;\n    println!(\"Valid: {:#?}\", val);\n\n    // And that one passes with flying colors.\n\n    Ok(())\n}\n```\n\n## TODO\n\nWhisper is extremely a work in progress. It's still being designed and syntax and semantics are changing\nregularly, though never too drastically. Documentation is in flux and may be old/wrong. There are a lot\nof things that need to be done before a release can even be considered:\n\n- Documentation; an eventual goal is `#[deny(missing_docs)]`.\n- Language improvements:\n    - Figure out a standard library which assists with reasoning about Rust structures\n    - Implement `let X = \u003cEXPRESSION\u003e` evaluation (currently panics)\n    - Error handling: lots of unwraps and `todo!()`s. Should be replaced with `failure`\n        derived error types\n    - Figure out a way to make imports easy from the point of view of someone integrating\n        Whisper with their program\n- I'm sure there's more but I can't think of it at the moment.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsdleffler%2Fwhisper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsdleffler%2Fwhisper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsdleffler%2Fwhisper/lists"}