{"id":16732556,"url":"https://github.com/antonmi/octopus","last_synced_at":"2026-03-06T07:32:33.303Z","repository":{"id":65376042,"uuid":"555825268","full_name":"antonmi/octopus","owner":"antonmi","description":"Declarative Interface Translation","archived":false,"fork":false,"pushed_at":"2023-11-06T09:48:56.000Z","size":284,"stargazers_count":15,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-21T22:28:41.898Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antonmi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-22T12:22:42.000Z","updated_at":"2025-01-15T18:42:44.000Z","dependencies_parsed_at":"2023-11-06T10:43:40.568Z","dependency_job_id":"ec27deb5-be0b-47d4-981f-29d14d8fd7a5","html_url":"https://github.com/antonmi/octopus","commit_stats":{"total_commits":78,"total_committers":1,"mean_commits":78.0,"dds":0.0,"last_synced_commit":"53cabad53744a1273682402cba93b7ef76b9b522"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/antonmi/octopus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmi%2Foctopus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmi%2Foctopus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmi%2Foctopus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmi%2Foctopus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antonmi","download_url":"https://codeload.github.com/antonmi/octopus/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmi%2Foctopus/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30165630,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-06T04:43:31.446Z","status":"ssl_error","status_checked_at":"2026-03-06T04:40:30.133Z","response_time":250,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-12T23:45:34.201Z","updated_at":"2026-03-06T07:32:33.277Z","avatar_url":"https://github.com/antonmi.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Octopus\n## Declarative Interface Translation\n## Specification -\u003e Elixir Code -\u003e API\n\n### The problem\nAs an application engineer, I need a simple way of interfacing\nwith programs/services that provide the required functionality.\nThe conventional approach to the problem is creating client libraries.\nSuch a library usually does three simple things:\n\n1. Translate data structures provided by the programming language (e.g. Elixir) to data required by another program (e.g. GET request to a URL with params).\n2. Call the program (e.g. make HTTP request).\n3. Translate the result (e.g. JSON response) to the language's data structures.\n\nHowever, each such translation must be explicitly coded. And this leads to a decent amount of boilerplate code.\n\n### The idea\nThese kinds of translations can be expressed in declarative way via specifications expressed as a data structure. \n\n### The solution\nThe specification can be provided using a JSON DSL that describes the interface to a service.\nThe client library code is generated from the specification.\nThe JSON is chosen as the specification language because it is easy to translate to Elixir data structures:\nJSON objects are translated to maps, JSON arrays are translated to lists, etc.\n\nConsider a simple example. Let's say we are going to use the [Agify](https://agify.io/) service. It predicts age of a person by name.\nTo use it we need to send a simple HTTP get request to it and take the age data from the response.\n\nThe JSON specification for the service would be:\n```json\n{\n  \"name\": \"agify\",\n  \"client\": {\n    \"module\": \"OctopusClientHttpFinch\",\n    \"start\": {\n      \"base_url\": \"https://api.agify.io/\"\n    }\n  },\n  \"interface\": {\n    \"age_for_name\": {\n      \"input\": {\n        \"name\": {\"type\": \"string\"}\n      },\n      \"prepare\": {\n        \"method\": \"GET\",\n        \"path\": \"/\",\n        \"params\": {\n          \"name\": \"args['name']\"\n        }\n      },\n      \"call\": {\n        \"parse_json_body\": true\n      },\n      \"transform\": {\n        \"age\": \"get_in(args, ['body', 'age'])\"\n      },\n      \"output\": {\n        \"age\": {\"type\": \"number\"}\n      }\n    }\n  }\n}\n```\nFirst, it says how what kind of client will be used - `OctopusClientHttpFinch`.\nThis is a low-level client that does basic communication with the HTTP API, and the module must exist in you app either as dependency or just a module in your code.\nSee the [OctopusClientHttpFinch](apps/octopus_client_http_finch/lib/octopus_client_http_finch.ex) and see the [Clients](#clients) section.\n\nSecond, it describes the interface of the service. In this case it has only one function - `age_for_name`.\nThere are 5 **optional** steps in the interface definition:\n1. `input` - describes the input data structure. If specified, the input data is validated against it. Octopus uses [JSON Schema](https://json-schema.org/) for data definition and validation.\n2. `prepare` - describes how the transformations needed to be done to the input data to make it ready for the call: path, method, params, headers, etc.\n3. `call` - configures the actual call to the service. Here it just says that the response body should be parsed as JSON.\n4. `transform` - describes how the result of the call should be transformed. In this case it just takes the `name` field from the response body.\n5. `output` - describes the output data structure. The output data is validated against it.\n\nThe definition can also be provided as an Elixir data structure:\n```elixir\ndefinition = %{\n  \"name\" =\u003e \"agify\",\n  \"client\" =\u003e %{\n    \"module\" =\u003e \"OctopusClientHttpFinch\",\n    \"start\" =\u003e %{\"base_url\" =\u003e \"https://api.agify.io/\"}\n  },\n  \"interface\" =\u003e %{\n    \"age_for_name\" =\u003e %{\n      \"input\" =\u003e %{\"name\" =\u003e %{\"type\" =\u003e \"string\"}},\n      \"prepare\" =\u003e %{\n        \"method\" =\u003e \"GET\",\n        \"params\" =\u003e %{\"name\" =\u003e \"args['name']\"},\n        \"path\" =\u003e \"/\"\n      },\n      \"call\" =\u003e %{\"parse_json_body\" =\u003e true},\n      \"transform\" =\u003e %{\"age\" =\u003e \"get_in(args, ['body', 'age'])\"},\n      \"output\" =\u003e %{\"age\" =\u003e %{\"type\" =\u003e \"number\"}}\n    }\n  }\n}\n```\nPlease note that strings are used as keys in the input data structure. The idea is to close to the JSON as possible, and JSON doesn't have atom type.\n\n### Transformations\nThere are two steps in the interface definition that transforms the data: `prepare` and `transform`.\nYou see \n```elixir\n\"params\" =\u003e %{\"name\" =\u003e \"args['name']\"}\n```\nand\n```elixir\n\"name\" =\u003e \"get_in(args, ['body', 'name'])\"\n```\nThe value-stings are evaluated as Elixir code. The `args` variable contains data from a previous step.\nOnly some `Kernel` functions and functions from `Access` module are available there.\nThere is also possible to add custom helpers for the transformation steps. See [Custom Helpers](#custom-helpers) section.\n\n### The magic\nHaving the declaration above (and `OctopusClientHttpFinch` also) one can create the client **service** by running:\n```elixir\nOctopus.define(definition)\n```\nThis will create the `Octpus.Services.Agify` module with a bunch of functions that are parameterized according to the specification.\nOne shouldn't use these function directly, but rather call them via `Octopus` API.\nFirst, the service should be started:\n```elixir\nOctopus.start(\"agify\")\n```\nThen, the service can be called:\n```elixir\niex(1)\u003e Octopus.call(\"agify\", \"age_for_name\", %{\"name\" =\u003e \"Anton\"})\n{:ok, %{\"age\" =\u003e 50}}\n```\nAgain, note, that strings are used as keys in the input data structure.\n\nSee [`octopus_test.exs`](apps/octopus/test/octopus_test.exs) for other functions in Octopus.\n\n### Exceptions and error handling\nWhen exception happens in any step, Octopus will return\n```elixir\n{:error, %Octopus.CallError{}}\n```\nThe `%Octopus.CallError{}` struct has the following fields:\n```text\n:step - :input | :prepare | :call | :transorm | :output | :error\n:error - original error,\n:message - string message,\n:stacktrace - stacktrace (string produced by Exception.format_stacktrace)\n```\n\nIt is possible to handle \"expected\" client errors (not exceptions). If client returns `{:error, error}` tuple than the error can be processed in the \"error\" step.\n\nThe \"step\", \"error\", \"message\", and \"stacktrace\" are available in \"args\"\n\nFor example, if the \"error\" step is specified in the \"interface\" section\n\n```json\n\"error\": {\n  \"step\": \"args['step']\",\n  \"error\": \"args['error']\",\n  \"message\": \"args['message']\",\n  \"stacktrace\": \"args['stacktrace']\",\n  \"foo\": \"unfortunately an error occured :(\"\n}\n```\n\nthen Octopus will return `{:ok, result}`, where `result` will have the fields defined in the \"error\" step.\n\nThe \"transform\" and \"output\" step will be skipped in that case.\n\nThe following diagram represent the possible flows:\n\n\u003cimg src=\"images/octopus-data-flow.png\" \nalt=\"Octopus data flow\" \nwidth=800px\u003e\n\n\n### OctopusAgent\nSince we translate the interface to JSON it becomes easy to interact with them via HTTP JSON API.\nOctopusAgent is a simple HTTP JSON API server that can be used to interact with the services.\nSee the OctopusAgent [README.md](apps/octopus_agent/README.md) for more details.\n\n### Clients\nClients are the low-level modules that do the actual communication with the service.\nOne can find the examples in the umbrella apps here:\n- [octopus_client_http_finch](apps/octopus_client_http_finch)\n- [octopus_client_cli_rambo](apps/octopus_client_cli_rambo)\n- [octopus_client_postgrex](apps/octopus_client_postgrex)\n\nYou can use them as a dependency or just copy-paste the code to your project.\nThe client must implement three functions (see the [Octopus.Client](apps/octopus/lib/octopus/client.ex) behaviour):\n\nStart:\n```elixir\n@spec start(map(), map(), atom()) :: {:ok, map()} | {:error, any()}\ndef start(args, configs, service_module) do\n  # `args` comes from Octopus.start(\"my_service\", args)\n  # `configs` comes from the \"start\" section of the specification\n  # `service_module` is the module of the defined service (like Octopus.Services.MyService) \nend\n```\nThe returned map represents the state of the client. \nIt will be passed to the `call` and `stop` functions.\n\nStop:\n```elixir\n@spec stop(map(), map(), any()) :: :ok | {:error, :not_found}\ndef stop(args, configs, state) do\n  # `args` comes from Octopus.stop(\"my_service\", args)\n  # `configs` comes from the \"stop\" section of the specification\n  # `state` is the map returned from the start function \nend\n```\n\nCall:\n```elixir\n@spec call(map(), map(), any()) :: {:ok, map()} | {:error, any()}\ndef call(args, configs, state) do\n  # `args` comes from Octopus.call(\"my_service\", \"my_function\", args)\n  # `configs` comes from the \"call\" section of the specification\n  # `state` is the map returned from the start function\nend\n```\n\n### Custom Helpers\nIt is possible to add custom helpers for the transformation steps.\nOne can add list of helper modules to the `helpers` key in the specification:\n```elixir\n```json\n{\n  \"name\": \"my_service\",\n  \"helpers\": [\"MyCustomHelpers\", \"AnotherHelpers\"],\n  \"client\": ...,\n  \"interface\": ...,\n}\n```\nThe modules must exist (be compiled) before the service is defined.\nFunctions from the modules will be available in the transformation steps.\nIf, for example, you have:\n```elixir\ndefmodule MyCustomHelpers do\n  def inc_by_one(number), do: number + 1\nend\n```\nYou can use the `inc_by_one` function in the transformation step:\n```elixir\n%{\n\"prepare\" =\u003e %{\"y\" =\u003e \"inc_by_one(args['x'])\"},\n\"transform\" =\u003e %{\"z\" =\u003e \"inc_by_one(args)\"},\n}\n```\n\n### TODO\n- use \"Octopus.Lambda\" (\"octopus.lambda\") instead of \"octopus.elixir-module-client\"\n- templates in service start/stop\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantonmi%2Foctopus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantonmi%2Foctopus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantonmi%2Foctopus/lists"}