{"id":13780319,"url":"https://github.com/typestreamio/typestream","last_synced_at":"2026-02-28T13:15:09.665Z","repository":{"id":191800549,"uuid":"685423176","full_name":"typestreamio/typestream","owner":"typestreamio","description":"Open Source streaming platform. Write and run typed data pipelines with a minimal, familiar syntax.","archived":false,"fork":false,"pushed_at":"2024-05-08T20:28:58.000Z","size":8334,"stargazers_count":64,"open_issues_count":6,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-08-03T18:14:59.491Z","etag":null,"topics":["kafka","kafka-streams","pipelines"],"latest_commit_sha":null,"homepage":"https://typestream.io","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/typestreamio.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-31T07:39:26.000Z","updated_at":"2024-07-25T22:41:06.000Z","dependencies_parsed_at":"2023-10-27T22:33:54.111Z","dependency_job_id":"36f7fbab-8f9f-471e-9e33-b74b2c27d79a","html_url":"https://github.com/typestreamio/typestream","commit_stats":null,"previous_names":["typestreamio/typestream"],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/typestreamio%2Ftypestream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/typestreamio%2Ftypestream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/typestreamio%2Ftypestream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/typestreamio%2Ftypestream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/typestreamio","download_url":"https://codeload.github.com/typestreamio/typestream/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225056802,"owners_count":17414212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kafka","kafka-streams","pipelines"],"created_at":"2024-08-03T18:01:14.472Z","updated_at":"2026-02-28T13:15:09.557Z","avatar_url":"https://github.com/typestreamio.png","language":"Kotlin","readme":"\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"/assets/avatar-transparent.png?raw=true\" width=\"86\"\u003e\n\u003c/div\u003e\n\n\u003ch1 align=\"center\"\u003eTypeStream\u003c/h1\u003e\n\n\u003cbr /\u003e\n\n\u003cdiv align=\"center\"\u003e\n    \u003ca href=\"https://github.com/typestreamio/typestream/blob/main/LICENSE\"\u003e\n        \u003cimg src=\"https://img.shields.io/github/license/typestreamio/typestream\" /\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://discord.gg/Ha9sJWXb\"\u003e\n        \u003cimg src=\"https://img.shields.io/badge/Chat-on%20Discord-blue\" alt=\"Discord invite\" /\u003e\n    \u003c/a\u003e\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"#why-typestream\"\u003eWhy TypeStream?\u003c/a\u003e\n    ·\n    \u003ca href=\"#getting-started\"\u003eGetting started\u003c/a\u003e\n    ·\n    \u003ca href=\"#how-to-contribute\"\u003eHow to contribute\u003c/a\u003e\n    ·\n    \u003ca href=\"#code-of-conduct\"\u003eCode of conduct\u003c/a\u003e\n    ·\n    \u003ca href=\"#license\"\u003eLicense\u003c/a\u003e\n\u003c/p\u003e\n\n\u003ch3 align=\"center\"\u003e\n\nTypeStream is an abstraction layer on top of Kafka that allows you to write\nand run \u003ci\u003etyped\u003c/i\u003e data pipelines with a minimal, familiar syntax.\n\n\u003c/h3 \u003e\n\n## Why TypeStream?\n\nBuilding streaming data pipelines on top of Kafka comes with some fixed costs.\nYou have to write an app, test it, then deploy and manage it in production. Even\nfor the simplest pipelines, this can be a lot of work.\n\nWith TypeStream you can write powerful, typed data pipelines the way you'd write\na simple UNIX pipeline in your terminal. For example, imagine you'd like to\nfilter a \"books\" topic. With TypeStream, it's a one liner:\n\n```sh\n$ typestream\n\u003e grep /dev/kafka/local/topics/books the \u003e /dev/kafka/local/topics/books_with_the\n```\n\nTypeStream will take care of type-checking your pipeline and then run it for\nyou. Here's how it looks like in action:\n\n![grepping with TypeStream](/assets/vhs/grep.gif?raw=true)\n\nAnother common use case that requires a lot of boilerplate is to enrich a topic\nwith data from an external source. For example, you might have a topic with an\nip address field and you may want to enrich it with country information. With\nTypeStream, you can do it (again!) in a single line:\n\n```sh\n$ typestream\n\u003e cat /dev/kafka/local/topics/page_views | enrich { view -\u003e http \"https://api.country.is/#{$view.ip_address}\" | cut .country } \u003e /dev/kafka/local/topics/page_views_with_country\n```\n\nHere's how enriching looks like in action:\n\n![enriching with TypeStream](/assets/vhs/enrich.gif?raw=true)\n\nAs you can see from the previous command, in the spirit of UNIX, we used cut to\nextract the country field from the response. Here's the kick, you can use `cut`\n(and many other Unix commands) on streams as well:\n\n```sh\n$ typestream\n\u003e cat /dev/kafka/local/topics/books | cut .title \u003e /dev/kafka/local/topics/book_titles\n```\n\nHere's how cutting looks like in action:\n\n![cutting with TypeStream](/assets/vhs/cut.gif?raw=true)\n\nAnother problem that TypeStream solves is preventing you from writing faulty\npipelines by type-checking them before executing them. Here's type checking in\naction:\n\n![type checking with TypeStream](/assets/vhs/type-checking.gif?raw=true)\n\nDeploying a pipeline to production is as simple as running:\n\n```sh\ntypestream run 'cat /dev/kafka/local/topics/books | cut .title \u003e /dev/kafka/local/topics/book_titles'\n```\n\nSee it in action here:\n\n![running a pipeline with TypeStream](/assets/vhs/run.gif?raw=true)\n\nIf you'd like to learn more about TypeStream, check out [the official\ndocumentation](https://docs.typestream.io/).\n\n## Getting started\n\nIf you use [Homebrew](https://brew.sh/):\n\n```sh\nbrew install typestreamio/tap/typestream\n$ typestream --version\n```\n\nif you see something like this:\n\n```sh\ntypestream version 2023.08.31+3 42f7762daac1872416bebab7a34d0b79a838d40a (2023-09-02 09:20:52)\n```\n\nthen you're good to go! You can now run:\n\n```sh\ntypestream local start\ntypestream local seed\n```\n\nto start a local TypeStream server and seed it with some sample data. Now you're\nready to start writing your own pipelines!\n\nCheck out our [documentation](https://docs.typestream.io/) to learn more about\nTypeStream.\n\n## How to contribute\n\nWe love every form of contribution! Good entry points to the project are:\n\n- Our [contributing guidelines](/CONTRIBUTING.md) document.\n- Issues with the tag\n  [gardening](https://github.com/typestreamio/typestream/issues?q=is%3Aissue+is%3Aopen+label%3Agardening).\n- Issues with the tag [good first\n  patch](https://github.com/typestreamio/typestream/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+patch%22).\n\nIf you're not sure where to start, open a [new\nissue](https://github.com/typestreamio/typestream/issues/new) or hop on to our\n[discord](https://discord.gg/Ha9sJWXb) server and we'll gladly help you get\nstarted.\n\n## Code of Conduct\n\nYou are expected to follow our [code of conduct](/CODE_OF_CONDUCT.md) when\ninteracting with the project via issues, pull requests, or in any other form.\nMany thanks to the awesome [contributor\ncovenant](http://contributor-covenant.org/) initiative!\n\n## License\n\n[Apache 2.0](/LICENSE)\n","funding_links":[],"categories":["Tools","\u003ca name=\"Kotlin\"\u003e\u003c/a\u003eKotlin","大数据"],"sub_categories":["Spring Cloud框架"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftypestreamio%2Ftypestream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftypestreamio%2Ftypestream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftypestreamio%2Ftypestream/lists"}