{"id":32123975,"url":"https://github.com/jlandersen/vscode-kafka","last_synced_at":"2026-05-10T08:04:59.138Z","repository":{"id":34548453,"uuid":"163522654","full_name":"jlandersen/vscode-kafka","owner":"jlandersen","description":"Apache Kafka® extension for Visual Studio Code","archived":false,"fork":false,"pushed_at":"2026-03-22T18:58:01.000Z","size":9529,"stargazers_count":104,"open_issues_count":2,"forks_count":35,"subscribers_count":8,"default_branch":"main","last_synced_at":"2026-03-23T10:18:50.793Z","etag":null,"topics":["extension","kafka","nodejs","typescript","visual-studio-code"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jlandersen.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2018-12-29T15:51:37.000Z","updated_at":"2026-03-22T18:57:14.000Z","dependencies_parsed_at":"2023-01-15T07:42:54.831Z","dependency_job_id":null,"html_url":"https://github.com/jlandersen/vscode-kafka","commit_stats":null,"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"purl":"pkg:github/jlandersen/vscode-kafka","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlandersen%2Fvscode-kafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlandersen%2Fvscode-kafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlandersen%2Fvscode-kafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlandersen%2Fvscode-kafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jlandersen","download_url":"https://codeload.github.com/jlandersen/vscode-kafka/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlandersen%2Fvscode-kafka/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31362714,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-03T15:19:21.178Z","status":"ssl_error","status_checked_at":"2026-04-03T15:19:20.670Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["extension","kafka","nodejs","typescript","visual-studio-code"],"created_at":"2025-10-20T21:31:32.075Z","updated_at":"2026-04-03T17:03:27.310Z","avatar_url":"https://github.com/jlandersen.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tools for Apache Kafka®\n\n[![CI](https://img.shields.io/github/actions/workflow/status/jlandersen/vscode-kafka/ci.yml?branch=master)](https://github.com/jlandersen/vscode-kafka/actions?query=workflow%3ACI+branch%3Amaster)\n[![Latest version](https://img.shields.io/visual-studio-marketplace/v/jeppeandersen.vscode-kafka?color=brightgreen)](https://marketplace.visualstudio.com/items?itemName=jeppeandersen.vscode-kafka)\n[![Marketplace Installs](https://img.shields.io/visual-studio-marketplace/i/jeppeandersen.vscode-kafka?logo=Installs)](https://marketplace.visualstudio.com/items?itemName=jeppeandersen.vscode-kafka)\n\n**Interact with Apache Kafka® directly in VS Code.** Manage clusters, produce and consume messages, and explore topics.\n\n---\n\n## ✨ Key Features\n\n### 🔍 **Visual Cluster Explorer**\nBrowse your Kafka infrastructure in the sidebar:\n- 📊 **Clusters** - Connect to multiple Kafka clusters simultaneously\n- 📂 **Topics** - View, create, and delete topics with real-time updates\n- 🖥️ **Brokers** - Monitor broker health and configuration\n- 👥 **Consumer Groups** - Track consumer lag and group membership\n- ⚙️ **Configurations** - Inspect and manage cluster settings\n- 🧬 **Schema Registries** - Browse subjects and schema versions linked to your clusters\n\n![Kafka Explorer](docs/assets/kafka-explorer.png)\n\n### 📤 **Message Producer**\nCreate producers using simple `.kafka` files with rich features:\n- 🎲 **Randomized Data** - Generate test data with [Faker.js](https://fakerjs.dev/) templates\n- 🔑 **Headers \u0026 Keys** - Full support for message keys and custom headers\n- ⏱️ **Scheduled Production** - Produce messages at regular intervals (`every: 5s`, `every: 1m`)\n- 🔁 **Batch Production** - Send multiple messages at once for load testing\n- 🎯 **Multiple Producers** - Define multiple producers in a single file\n- ✅ **JSON Schema Validation** - Validate `value-format: json` payloads with inline schema or `file(...)` reference\n- 🧬 **Avro \u0026 Protobuf** - Produce/consume Avro (`value-schema` inline or file) and Protobuf (`value-schema` file only)\n\n**Example:**\n```kafka\nPRODUCER user-events\ntopic: user-activity\nevery: 3s\nkey: user-{{string.uuid}}\nheaders: source=web-app, version=1.0\n{\n  \"userId\": \"{{string.uuid}}\",\n  \"event\": \"{{helpers.arrayElement(['login', 'logout', 'purchase'])}}\",\n  \"timestamp\": {{$timestamp}},\n  \"user\": {\n    \"name\": \"{{person.fullName}}\",\n    \"email\": \"{{internet.email}}\"\n  }\n}\n```\n\n**Value format examples (JSON / Avro / Protobuf):**\n```kafka\nPRODUCER\ntopic: json-events\nvalue-format: json\nvalue-schema: {\"type\":\"object\",\"required\":[\"id\"],\"properties\":{\"id\":{\"type\":\"number\"}}}\n{\"id\":1}\n###\nPRODUCER\ntopic: avro-events\nvalue-format: avro\nvalue-schema: {\"type\":\"record\",\"name\":\"UserEvent\",\"fields\":[{\"name\":\"id\",\"type\":\"int\"}]}\n{\"id\":1}\n###\nPRODUCER\ntopic: protobuf-events\nvalue-format: protobuf(demo.UserCreated)\nvalue-schema: file(./schemas/user-events.proto)\n{\"id\":1,\"email\":\"jane@example.com\",\"active\":true}\n```\n\n`./schemas/user-events.proto` used by `protobuf(demo.UserCreated)`:\n```proto\nsyntax = \"proto3\";\n\npackage demo;\n\nmessage UserCreated {\n  int32 id = 1;\n  string email = 2;\n  bool active = 3;\n}\n```\n\n![Producing Messages](docs/assets/kafka-file-producers.png)\n\n### 📥 **Flexible Consumer**\nConsume messages with the Message Viewer:\n- 📊 **Message Viewer** - Table view with search, filters, histogram, and CSV export\n- 🎯 **Targeted Consumption** - Consume from specific partitions or offsets\n- 💾 **Export Data** - Export consumed messages to CSV for analysis\n\n**Start consuming from:**\n- Right-click a topic in the explorer\n- Use Command Palette (`Ctrl+Shift+P`)\n- Define consumers in `.kafka` files\n\n```kafka\nCONSUMER analytics-team\ntopic: user-events\nfrom: earliest\npartitions: 0,1,2\n```\n\n**Consume Avro values with schema:**\n```kafka\nCONSUMER analytics-team\ntopic: user-events\nfrom: earliest\nvalue-format: avro\nvalue-schema: file(./schemas/user-event.avsc)\n```\n\n**Consume Protobuf values with schema file:**\n```kafka\nCONSUMER analytics-team\ntopic: protobuf-events\nfrom: earliest\nvalue-format: protobuf(demo.UserCreated)\nvalue-schema: file(./schemas/user-events.proto)\n```\n\n![Consumer Message Viewer](docs/assets/consumer-view.png)\n\n### 🔐 **Security**\n- 🔒 **SASL Authentication** - PLAIN, SCRAM-256, SCRAM-512 (Kafka 0.10+)\n- 🌐 **OAUTHBEARER** - OAuth 2.0 authentication with automatic token refresh\n- ☁️ **AWS MSK IAM** - Native AWS IAM authentication for Amazon MSK clusters\n- 🛡️ **SSL/TLS Support** - Secure connections with certificate validation\n- 🔑 **Secure Storage** - Credentials stored in OS keychain (macOS Keychain, Windows Credential Manager, Linux Secret Service)\n- 🧪 **Development Mode** - Optional hostname verification bypass for self-signed certificates\n\n### 🧬 **Schema Registry**\nManage Confluent-compatible Schema Registries directly from VS Code:\n- 🔗 **Reusable Connections** - Define named Schema Registry connections independent of clusters\n- 🔗 **Cluster Linking** - Link a registry to one or more Kafka clusters\n- 📂 **Explorer Browsing** - Browse subjects and schema versions in the sidebar\n- 📄 **Open \u0026 Compare** - Open schema versions in the editor and diff any two versions side-by-side\n- 🔍 **Topic Subject Discovery** - Automatically discover subjects related to a topic based on TopicNameStrategy, RecordNameStrategy, or TopicRecordNameStrategy\n- 🔒 **Auth \u0026 TLS** - Optional basic auth and custom TLS configuration per registry\n\n### 🛠️ **Advanced Administration**\n- ✅ **Create Topics** - Configure partitions, replication factor, and topic settings\n- 🗑️ **Delete Topics** - Remove unwanted topics with confirmation dialogs\n- 🧹 **Delete Records** - Empty topics by deleting all messages from all partitions\n- 📋 **Metadata Inspection** - Dump detailed metadata for clusters, brokers, and topics\n- 👥 **Consumer Group Management** - Delete consumer groups and monitor offsets\n\n---\n\n## 🚀 Getting Started\n\n### 1. Install the Extension\nSearch for \"Kafka\" in the VS Code Extensions marketplace or [install from here](https://marketplace.visualstudio.com/items?itemName=jeppeandersen.vscode-kafka).\n\n### 2. Add Your First Cluster\nClick the `+` icon in the Kafka Explorer or use `Ctrl+Shift+P` → \"Kafka: Add Cluster\"\n\n### 3. Start Exploring!\n- Browse topics and consumer groups\n- Right-click to produce or consume messages\n- Create `.kafka` files for reusable workflows\n\n\u003e 📚 **Need Help?** Open documentation inside VS Code with `Ctrl+Shift+P` → \"Kafka: Open Documentation\"\n\n![Open Documentation](docs/assets/open-doc-cmd.png)\n\n---\n\n## 📖 Documentation\n\n| Topic | Description |\n|-------|-------------|\n| [Kafka Explorer](https://github.com/jlandersen/vscode-kafka/blob/master/docs/Explorer.md) | Navigating clusters, topics, brokers, and consumer groups |\n| [Producing Messages](https://github.com/jlandersen/vscode-kafka/blob/master/docs/Producing.md) | Creating producers with Faker templates and scheduled production |\n| [Consuming Messages](https://github.com/jlandersen/vscode-kafka/blob/master/docs/Consuming.md) | Message Viewer and consumption options |\n| [.kafka File Format](https://github.com/jlandersen/vscode-kafka/blob/master/docs/KafkaFile.md) | Syntax reference for producer and consumer definitions |\n| [Settings](https://github.com/jlandersen/vscode-kafka/blob/master/docs/Settings.md) | Extension configuration options |\n\n---\n\n## 🔌 Extensibility\n\nExtend the Kafka explorer by creating custom cluster providers. Your extension can:\n- Discover clusters from external sources (cloud providers, configuration management)\n- Auto-configure connection settings\n- Provide custom authentication mechanisms\n\n**Create a Cluster Provider Extension:**\n1. Add `\"kafka-provider\"` to your extension's `package.json` keywords\n2. Implement the cluster provider API\n3. Users discover your extension via \"Discover Cluster Providers\"\n\n![Discover Cluster Providers](docs/assets/kafka-explorer-discover-providers.png)\n\n---\n\n## 🤝 Contributing\n\nWe ❤️ contributions! Whether you're:\n- 🐛 Reporting bugs\n- 💡 Suggesting features\n- 📝 Improving documentation\n- 💻 Submitting pull requests\n\nAll contributions are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\n### Development Setup\n1. Clone the repository\n2. Run `npm install`\n3. Open in VS Code and press `F5` to launch Extension Development Host\n4. Make your changes and run tests with `npm test`\n\n---\n\n## 📦 CI Builds\n\nTry the latest development version:\n1. Go to the [CI Workflow page](https://github.com/jlandersen/vscode-kafka/actions?query=workflow%3ACI+is%3Asuccess+branch%3Amaster)\n2. Click on the most recent successful run\n3. Download the `vscode-kafka` artifact\n4. Unzip and install the `.vsix` file: `code --install-extension vscode-kafka-*.vsix`\n\n---\n\n## 📄 License\n\nMIT License. See [LICENSE](LICENSE) file.\n\n---\n\n## ⚖️ Legal\n\nApache, Apache Kafka®, Kafka® and associated logos are trademarks of the Apache Software Foundation (ASF). _Tools for Apache Kafka®_ is not affiliated with, endorsed by, or otherwise associated with the Apache Software Foundation or any of its projects.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjlandersen%2Fvscode-kafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjlandersen%2Fvscode-kafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjlandersen%2Fvscode-kafka/lists"}