{"id":13670887,"url":"https://github.com/uber-archive/idl","last_synced_at":"2025-04-27T13:33:08.705Z","repository":{"id":57271261,"uuid":"42240006","full_name":"uber-archive/idl","owner":"uber-archive","description":"A CLI for managing Thrift IDL files","archived":true,"fork":false,"pushed_at":"2019-10-01T19:30:57.000Z","size":226,"stargazers_count":78,"open_issues_count":14,"forks_count":19,"subscribers_count":1628,"default_branch":"dev","last_synced_at":"2025-04-11T03:23:55.933Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://www.uber.com","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uber-archive.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-09-10T11:16:18.000Z","updated_at":"2024-01-30T19:27:38.000Z","dependencies_parsed_at":"2022-09-01T00:40:11.969Z","dependency_job_id":null,"html_url":"https://github.com/uber-archive/idl","commit_stats":null,"previous_names":["uber-node/idl"],"tags_count":24,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber-archive%2Fidl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber-archive%2Fidl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber-archive%2Fidl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uber-archive%2Fidl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uber-archive","download_url":"https://codeload.github.com/uber-archive/idl/tar.gz/refs/heads/dev","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251145743,"owners_count":21543094,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T09:00:52.046Z","updated_at":"2025-04-27T13:33:03.695Z","avatar_url":"https://github.com/uber-archive.png","language":"JavaScript","readme":"**(This project is no longer under active development.)**\n\n# idl\n\n[![build status][build-png]][build]\n[![Coverage Status][cover-png]][cover]\n[![Davis Dependency status][dep-png]][dep]\n\n[![NPM][npm-png]][npm]\n\nA CLI for managing thrift IDL files.\n\n## Introduction\n\n`idl` provides a \"package manager\" for thrift interfaces.\nIt comes with two CLI commands, `idl-daemon` and `idl`.\n\n## Motivation\n\n - The source of truth for a service (the Thrift IDL file) should live\n    with the service (i.e. the code repository).\n    You want the thrift definition to be checked into the repository\n - Every service should keep a copy of the thrift definition for any\n    service that it wants to talk too. This definition should be static.\n    You do not want it to change at run-time, since that can result in\n    mismatched interfaces that cause runtime exceptions and kill a\n    process.\n - There should only be one version of the world. Your company runs at\n    a single version of each service in production; all the Thrift IDL\n    files representing the current version of every service in production\n    should ultimately be treated as a single versioned collection. This\n    versioned collection is the registry of all service definitions.\n - The one versioned collection should be live. Developers should not\n    manually publish new versions; instead we should just take `master`\n    containing the most recent Thrift IDL file for every service as the\n    source of truth.\n - When service interface definitions are fetched and copied to a\n    a project that consumes those services, they should be organized\n    such that cross-service include/import statements are possible (i.e.\n    the directory structure should support relative filepath import/include\n    statements).\n\n## The CLI\n\nThe CLI is currently broken down into two commands:\n - `idl` - CLI tool meant for end-users, but the publish command\n     can also be run a continuous integration job when interface changes\n     land in production (push-based publishing)\n - `idl-daemon` - daemonized process that can be configured to\n     poll all service repositories for interface changes (pull-based\n     publishing).\n\n### `idl`\n\nAs a developer I want to be able to talk to other services;\nTo do this I need to find their Thrift interface definitions.\n\nThe `idl` CLI solves this need.\n\nThe `idl` CLI tool has the following sub-commands:\n - `idl list` - List all available services published to the\n    registry.\n - `idl fetch \u003cservice-name\u003e` - Fetch a particular service's idl\n    and copy it to your project in a standard location.\n - `idl update` - Update all previously fetched services to the most\n    recent versions. Note: You cannot pick and choose which services to\n    update. This is intentional.\n - `idl publish` - Publish the Thrift IDL file for your service\n    to the idl registry repository. This command should set up to be\n    automatically executed when a change to the service IDL lands on\n    `master` or when that change on `master` is deployed to production.\n\nAll commands follow the unix standard of being silent if successful. If\nyou would like more information about what is happening, run the CLI\nwith the `--verbose` flag.\n\n#### `idl init`\n\nThis command will create a starter IDL file at the correct filepath in\na new service. You should use it if you've got a new service and want\nto quickly scaffold a simple thrift IDL file at the path expected\nby the `idl publish` command.\n\nExample:\n\n    $ idl init\n    $\n\n#### `idl list`\n\nThis command will list all available services published to the registry.\n\nExample:\n\n    $ idl list\n     - github.com:/foo/bar   2015-09-11T23:07:57.610Z\n     - github.com:/foo/baz   2015-09-11T23:07:58.159Z\n     - github.com:/qux/quux  2015-09-11T23:07:58.716Z\n    $\n\n#### `idl fetch \u003cservice-name\u003e`\n\nThis command will fetch a particular service's idl file and copy it\nto your project in a standard location.\n\nOnce you fetch your first service we also write the `./idl/meta.json`\nmeta file that contains the version of the registry as well as the time\nit was last changed.\n\nNote: Fetching a new service will result in an implicit update of any\nservices that have been fetched. For example, if you fetched service\n`foo` a month ago and then fetch service `bar`, `idl` will\nfirst update service foo to the most current version before fetching\n`bar`.\n\nThis command will also update the ./idl/meta.json file in your\nproject.\n\nExample:\n\n    $ idl fetch github.com/foo/bar\n    $\n\n#### `idl update`\n\nThis command will update all previously fetched services to the most\nrecent versions. Note: You cannot pick and choose which services to update. This is intentional.\n\nSince the thrift definitions define the interfaces of the services\nin production, there is only one version for all files. When you\nupdate anything, you update everything to the current version.\n\nThis command will also update the ./idl/meta.json file in your\nproject.\n\nExample.\n\n    $ idl update\n    $\n\n#### `idl publish`\n\nThis command will publish the Thrift IDL file for your service to the\nidl registry repository. This command should set up to be\nautomatically executed when a change to the service IDL lands on\n`master` or when that change on `master` is deployed to production.\n\nExample:\n\n    $ idl publish\n    $\n\n#### Command line flags\n\nThe `idl` CLI takes the following command line flags. The\nfirst flag, `--repository`, is mandatory until this tool has .rc file\nsupport.\n\n - `--repository=\u003cgit url\u003e` - The `idl` command needs to\n    know the git URL of the registry to be able to run any of the\n    commands above. e.g. `--repository=git@github.com:foo/registry`\n - `--cacheDir=\u003cpath to cache dir\u003e` - This is the path to the cache\n    directory that `idl` should use. The default value is\n    `~/.idl/`\n - `--cwd=\u003ccurrent working directory\u003e` - The path to the current\n    working directory in which to execute `idl`\n - `--verbose` - This tool follows the unix philosophy of being\n    silent on success. Use this flag if you want to see output of\n    what it is doing.\n - `--config=\u003cpath to config file\u003e` - The path to a JSON config\n    file that specifies any of the above properties.\n\n#### Configuration\n\nInternally `idl` uses [dominictarr/rc][dominictarr/rc] for\noptions configuration. This module helps with parsing configuration\nfrom arguments specified at the command line, from environment\nvariables and from `.rc` files. It will probe the following locations:\n\nGiven your application name (`appname`), rc will look in all the obvious places for configuration.\n\n  * command line arguments (parsed by [substack/minimist][substack/minimist])\n  * environment variables prefixed with `IDL_`\n    * or use \"\\_\\_\" to indicate nested properties \u003cbr/\u003e _(e.g. `IDL_REGISTRY` =\u003e `github.com/some-org/some repo`)_\n    * keys will automatically be camelCased to match the possible options\n    above.\n  * if you passed an option `--config file` then from that file\n  * a local `.idlrc` or the first found looking in `./ ../ ../../ ../../../` etc.\n  * `$HOME/.idlrc`\n  * `$HOME/.idl/config`\n  * `$HOME/.config/idl`\n  * `$HOME/.config/idl/config`\n  * `/etc/idlrc`\n  * `/etc/idl/config`\n\n### The `./idl/` directory\n\nAll services and clients will have a `./idl/` directory at the root of\nthe repo. All thrift IDL files are contained in this directory.\n\nService authors need to understand how this directory is organized and\nshould only every edit/modify the thrift IDL files for the service in\nquestion. Client authors should never have to edit/modify files in this\ndirectory and should only use the files contained therein as references\nfor the interfaces they are consuming.\n\nThe idl directory is organized so that every thrift IDL file (for the\nservice being authored or the services being consumed) is located at a\nsub-path that mirrors the git remote `origin` URL of a service.\n\nWhen you execute `git remote -v` in your service's repository, you will\nsee output similar to one of the following:\n\n    $ git remote -v\n    origin  git@github.com:uber/foo-service.git (fetch)\n    origin  git@github.com:uber/foo-service.git (push)\n\nor\n\n    $ git remote -v\n    origin  ssh://git@github.com/uber/foo-service.git (fetch)\n    origin  ssh://git@github.com/uber/foo-service.git (push)\n\nThe idl directory for your service mirrors these two addresses.\nAssuming the output above, the idl path for the service\nbeing authoring will be `./idl/github.com/uber/foo-service/`.\nThis directory will contain the thrift IDL files that will be\npublished to your idl registry repo when `idl publish`\nis executed. The IDL files in this particular sub-directory are to be\nmanually managed by service authors.\n\nAll other directories are contain service defitions for services being\nconsumed and should not be edited/modified and should only be\nconsulted as a reference when looking up a type definition for a\nservice or a function definition for a service being consumed.\n\nFor service repositories, where the service is also a client of\nother services, the unmanaged definitions for that service and\nmanaged definitions for the services being consumed will live\nside by side in sibling directories.\n\nThe reason for mixing both managed and unmanaged directories together\nis to support relative filepath includes in thrift files.\n\nFor example, if the service git@github.com:foo/bar.git references\ntype definitions from the services git@github.com:foo/baz.git and\ngit@github.com:qux/quux.git, then you might see the following\ndirectory and file structure:\n\n    $ tree\n    .\n    └── idl\n        ├── github.com\n        │   ├── foo\n        │   │   ├── bar\n        │   │   │   └── bar.thrift\n        │   │   └── baz\n        │   │       └── baz.thrift\n        │   └── qux\n        │       └── quux\n        │           └── quux.thrift\n        └── meta.json\n\n    7 directories, 4 files\n    $\n\n... and the `./idl/github.com/foo/bar/bar.thrift` would\ncontain the following includes in its header section:\n\n    include \"../baz/baz.thrift\"\n    include \"../../qux/quux/quux.thrift\"\n\nThe complexity of how this directory is organized is a necessary\nevil to support relative file includes. If, in the future, the\n`include` directive supports richer semantics, it may be possible\nto simplify this directory, but for now it is what is is.\n\n### `idl-daemon`\n\nThe `idl-daemon` will fetch all the remotes and place\ntheir thrift files in the `upstream` repository. You can use\n`idl fetch` to fetch from the upstream repository.\n\nThe `idl-daemon` is a command that should be run with\ncron.\n\nTo set up the idl repository you can run the\n`idl-daemon`. Run `idl-daemon --config-file={path}`\nand it will populate the idl remote repository.\n\nThe config file contains the following fields\n\n```json\n{\n    \"upstream\": \"git+ssh://git@github.com/my-company/idl-registry\",\n    \"repositoryDirectory\": \"/var/lib/my-company/idl/repo\",\n    \"cacheLocation\": \"/var/lib/my-company/idl/cache\",\n    \"remotes\": [{\n        \"repository\": \"git+ssh://git@github.com/my-company/user-service\",\n        \"branch\": \"master\"\n    }, {\n        \"repository\": \"git+ssh://git@github.com/my-company/product-service\",\n        \"branch\": \"master\"\n    }]\n}\n```\n\n## TODO:\n\nThis project is not done yet:\n\n - [ ] Implicit update whenever `idl fetch` is run.\n - [ ] Implement `idl validate` so that service authors can locally\n       validate the thrift IDL files for their service before publishing.\n - [ ] Implement `idl init` to automatically create a boilerplate\n       thrift IDL file using the git URL of the remote origin.\n - [ ] Implement `idl config get \u003cproperty\u003e` to get a idl\n       configuration property.\n - [ ] Implement `idl config set \u003cproperty\u003e \u003cvalue\u003e` to set a\n       idl configuration property.\n - [ ] Implement fetching from `remotes` into `upstream`.\n - [ ] Support `main` file in config to indicate service entry point.\n - [ ] Support `branch` in config.\n - [ ] Disable `publish` command using a regex saved in the .rc file.\n\n## Installation\n\n`npm install idl --global`\n\n## Tests\n\n`npm test`\n\n## Contributors\n\n - andrewdeandrade (Andrew de Andrade)\n - Raynos (Jake Verbaten)\n - kriskowal (Kris Kowal)\n - prashantv (Prashant Varanasi)\n\n## MIT Licensed\n\n  [build-png]: https://secure.travis-ci.org/uber-node/idl.png\n  [build]: https://travis-ci.org/uber-node/idl\n  [cover-png]: https://coveralls.io/repos/uber-node/idl/badge.png\n  [cover]: https://coveralls.io/r/uber-node/idl\n  [dep-png]: https://david-dm.org/uber-node/idl.png\n  [dep]: https://david-dm.org/uber-node/idl\n  [npm-png]: https://nodei.co/npm/idl.png?stars\u0026downloads\n  [npm]: https://nodei.co/npm/idl\n  [dominictarr/rc]: https://github.com/dominictarr/rc\n  [substack/minimist]: https://github.com/substack/minimist\n","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuber-archive%2Fidl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuber-archive%2Fidl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuber-archive%2Fidl/lists"}