{"id":19600049,"url":"https://github.com/nvim-neorg/norgopolis-module","last_synced_at":"2025-04-27T16:32:14.270Z","repository":{"id":179705012,"uuid":"664018484","full_name":"nvim-neorg/norgopolis-module","owner":"nvim-neorg","description":"A library for creating your own Norgopolis modules in Rust.","archived":false,"fork":false,"pushed_at":"2024-02-21T10:25:31.000Z","size":43,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-05T01:32:16.566Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nvim-neorg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-07-08T17:38:44.000Z","updated_at":"2024-07-21T04:28:01.000Z","dependencies_parsed_at":"2024-02-20T22:47:04.896Z","dependency_job_id":null,"html_url":"https://github.com/nvim-neorg/norgopolis-module","commit_stats":null,"previous_names":["nvim-neorg/norgopolis-module"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvim-neorg%2Fnorgopolis-module","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvim-neorg%2Fnorgopolis-module/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvim-neorg%2Fnorgopolis-module/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvim-neorg%2Fnorgopolis-module/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nvim-neorg","download_url":"https://codeload.github.com/nvim-neorg/norgopolis-module/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251171570,"owners_count":21547112,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T09:13:24.262Z","updated_at":"2025-04-27T16:32:13.963Z","avatar_url":"https://github.com/nvim-neorg.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# A Library for Creating Norgopolis Modules\n\nFor information about Norgopolis, consult https://github.com/nvim-neorg/norgopolis.\n\nThis library exposes an API for creating and maintaining a connection to the Norgopolis router.\nNorgopolis modules provide specific sets of functionality, for example multithreaded parsing, database\naccess, etc. All of the default modules created by the Neorg team are built on top of this library.\n\n\u003e [!NOTE]\n\u003e This is a library for developers. If you are a general user, feel free to check out the\n\u003e mainline [Neorg](https://github.com/nvim-neorg/neorg) repository or\n\u003e [Norgopolis itself](https://github.com/nvim-neorg/norgopolis).\n\n# Setup\n\n```sh\ncargo add norgopolis-module\n```\n\nModules are asynchronous applications that communicate with Norgopolis over stdin/stdout.\nThe format transmitted across stdin/stdout is gRPC + MessagePack. For this reason it is highly recommended\nto add both `tokio` and `tokio-stream` to your dependencies.\n\nData that is transmitted over gRPC must be serializable. Be sure to add `serde` to your dependency\nlist too!\n\n# Planning\n\nThe first phase of creating a module is deciding:\n- What sort of functionality the module will provide\n- What sort of inputs it will receive and what sort of data it will return\n\nInput/output data is not enforced at the transmit layer - that is, an application\nhas no idea what sort of data you expect as input and what sort of data you will return.\nThis is a natural technical detail of dynamic gRPC + mpack communication. For this reason it is recommended\nto describe your API in a technical document somewhere in your repository for others to see.\n\n# Usage\n\n### General Setup\n\nFirst, create a struct for your module. Name it whatever you'd like:\n\n```rs\nuse norgopolis_module::{\n    invoker_service::Service, module_communication::MessagePack, Code, Module, Status,\n};\n\n#[derive(Default)]\nstruct MyModule {\n    // add any data or state you might need to maintain here...\n}\n```\n\nSecond, implement the `norgopolis_module::invoker_service::Service` trait for your struct.\nThis forces you to implement a `call` function which will be invoked any time someone routes\na message to your module. Since async traits are not stabilized within Rust yet, tag your\ntrait implementation with `#[norgopolis_module::async_trait]`:\n\n```rs\nuse tokio_stream::wrappers::UnboundedReceiverStream;\n\n#[norgopolis_module::async_trait]\nimpl Service for MyModule {\n    type Stream = UnboundedReceiverStream\u003cResult\u003cMessagePack, Status\u003e\u003e;\n\n    async fn call(\n        \u0026self,\n        function: String,\n        args: Option\u003cMessagePack\u003e,\n    ) -\u003e Result\u003cSelf::Stream, Status\u003e {\n        todo!()\n    }\n}\n```\n\n##### `Stream`\n\nThe `Stream` type defines what sort of data will be returned back via gRPC. We recommend\nthat you set it to `UnboundedReceiverStream\u003cResult\u003cMessagePack, Status\u003e\u003e`. This means that\ngiven one request your module will be able to return an infinite amount of MessagePack responses,\nor a status code in case something went wrong.\n\n##### `call`\n\nThe `call` function gets invoked whenever a client routes a message to you. The message contains:\n- The function that they would like to invoke\n- An optional set of parameters they would like to supply to the function.\n\n### Creating the Basic Glue\n\nIn the `call` function it's recommended to match over all possible function names that your module\nsupports and returning an error code if it's unsupported:\n\n```rs\nmatch function.as_str() {\n    \"my-function\" =\u003e todo!(),\n    _ =\u003e Err(Status::new(Code::NotFound, \"Requested function not found!\")),\n}\n```\n\n\u003e [!IMPORTANT]\n\u003e It's always better to return *some* sort of status code over panicking.\n\u003e Panicking will terminate the connection to Norgopolis and the user will not receive\n\u003e any sort of error or warning.\n\n### Decoding the Parameters\n\nIf your function takes in any amount of parameters then now is the time to decode them.\nIf your parameter is complex (e.g. a dictionary) then it's recommended to create a struct\ndesignated for it. Be sure to derive `serde::Serialize`:\n\n```rs\n#[derive(serde::Serialize)]\nstruct MyParameters {\n    name: String,\n}\n```\n\nAftewards, it's a simple matter of running `decode` on your arguments:\n\n```rs\nmatch function.as_str() {\n    \"my-function\" =\u003e {\n        let args: MyParameters = args\n            .unwrap() // WARNING: Don't actually use unwrap() in your code :)\n            .decode()\n            .map_err(|err| Status::new(Code::InvalidArgument, err.to_string()))?;\n\n        // TODO: Do something with the parameters...\n    },\n}\n```\n\nWe manually provide the type of `args` so that Rust knows what type to serialize to.\nAfterwards we wrap any possible errors into a status code which can be returned back to the client.\n\n### Sending Data back to the Client\n\nNow that we have all of the input data in check we can process our data and return it back to the client.\nThe way we do this is in the form of a data stream. Thanks to data streams we can return long segments of\ndata over time instead of having to return the whole data upfront. When we return a segment of data, we\nalso return it in the form of a `Result\u003c\u003e`. This is because individual segments of data may contain errors,\nbut the whole process can complete succesfully. You should return errors from the `call` function when there\nis an irrecoverable error, but should send back an error packet when a *portion* of the internal logic fails.\n\nLet's showcase all of this via an example:\n\n```rs\nmatch function.as_str() {\n    \"my-function\" =\u003e {\n        let args: MyParameters = args\n            .unwrap() // WARNING: Don't actually use unwrap() in your code :)\n            .decode()\n            .map_err(|err| Status::new(Code::InvalidArgument, err.to_string()))?;\n\n        let (tx, rx) = tokio::sync::mpsc::unbounded_channel();\n\n        // We send back an Ok() packet to the client with an encoded message of our choice\n        // (it can be anything that's serializable with serde!)\n        tx.send(Ok(MessagePack::encode(format!(\"Hello, {}!\", args.name)))).unwrap();\n\n        Ok(UnboundedReceiverStream::new(rx))\n    },\n}\n```\n\nFirst, we create a sender and receiver via tokio's `unbounded_channel()`. This allows us to send data to the client\nand for the client to read data from the module. All return messages have to be encoded via `MessagePack::encode`.\n\n### Running the Module\n\nNow that we have all of the code set up, create an asynchronous main function. In here we will instantiate our\nmodule and kick it into full gear:\n\n```rs\n#[tokio::main]\nasync fn main() {\n    Module::new().start(MyModule::default())\n        .await\n        .unwrap()\n}\n```\n\nVoila! You now have a fundamental understanding of how modules communicate with Norgopolis and how to write your own\nnorgopolis module. Happy coding!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvim-neorg%2Fnorgopolis-module","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnvim-neorg%2Fnorgopolis-module","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvim-neorg%2Fnorgopolis-module/lists"}