{"id":13394913,"url":"https://github.com/elastic/logstash","last_synced_at":"2025-12-16T20:06:51.772Z","repository":{"id":260379598,"uuid":"1090311","full_name":"elastic/logstash","owner":"elastic","description":"Logstash - transport and process your logs, events, or other data","archived":false,"fork":false,"pushed_at":"2025-12-11T10:43:42.000Z","size":132581,"stargazers_count":14744,"open_issues_count":2205,"forks_count":3519,"subscribers_count":817,"default_branch":"main","last_synced_at":"2025-12-12T05:59:31.100Z","etag":null,"topics":["etl-framework","java","jruby","logging","real-time-processing","streaming"],"latest_commit_sha":null,"homepage":"https://www.elastic.co/products/logstash","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elastic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"COPYING.csv","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE.TXT","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2010-11-18T01:31:50.000Z","updated_at":"2025-12-12T02:18:40.000Z","dependencies_parsed_at":"2024-10-31T02:07:46.384Z","dependency_job_id":"07eb0245-a835-47b5-9b1f-e9c30399c8a4","html_url":"https://github.com/elastic/logstash","commit_stats":{"total_commits":9560,"total_committers":600,"mean_commits":"15.933333333333334","dds":0.6874476987447699,"last_synced_commit":"0e1d67eda9f1ea64f9c9ebbb6ab118e251b9f956"},"previous_names":["elastic/logstash","elasticsearch/logstash","logstash/logstash"],"tags_count":419,"template":false,"template_full_name":null,"purl":"pkg:github/elastic/logstash","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Flogstash","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Flogstash/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Flogstash/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Flogstash/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elastic","download_url":"https://codeload.github.com/elastic/logstash/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Flogstash/sbom","scorecard":{"id":371268,"data":{"date":"2025-08-11","repo":{"name":"github.com/elastic/logstash","commit":"371d49537f5a3f18d19dd5692f95d7d8677b0e24"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":7.1,"checks":[{"name":"Code-Review","score":10,"reason":"all changesets reviewed","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":10,"reason":"30 commit(s) and 5 issue activity found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: SECURITY.md:1","Info: Found linked content: SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1","Info: Found text in security policy: SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Pinned-Dependencies","score":-1,"reason":"internal error: internal error: invalid Dockerfile: Syntax error - can't find = in \"locale\". Must be of the form: name=value","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Info: jobLevel 'contents' permission set to 'read': .github/workflows/catalog-info.yml:18","Info: jobLevel 'packages' permission set to 'read': .github/workflows/catalog-info.yml:19","Warn: jobLevel 'deployments' permission set to 'write': .github/workflows/docs-build.yml:17","Info: jobLevel 'contents' permission set to 'read': .github/workflows/docs-build.yml:19","Info: found token with 'none' permissions: .github/workflows/docs-cleanup.yml:12","Warn: jobLevel 'deployments' permission set to 'write': .github/workflows/docs-cleanup.yml:14","Info: jobLevel 'repository-projects' permission set to 'read': .github/workflows/mergify-labels-copier.yml:19","Info: topLevel 'contents' permission set to 'read': .github/workflows/backport-active.yml:11","Info: topLevel 'contents' permission set to 'read': .github/workflows/catalog-info.yml:12","Warn: no topLevel permission defined: .github/workflows/critical_vulnerability_scan.yml:1","Warn: no topLevel permission defined: .github/workflows/docs-build.yml:1","Warn: no topLevel permission defined: .github/workflows/docs-cleanup.yml:1","Warn: topLevel 'contents' permission set to 'write': .github/workflows/gen_release_notes.yml:17","Info: topLevel 'contents' permission set to 'read': .github/workflows/github-commands-comment.yml:10","Info: topLevel 'contents' permission set to 'read': .github/workflows/lint_docs.yml:8","Warn: no topLevel permission defined: .github/workflows/logstash_project_board.yml:1","Info: topLevel 'contents' permission set to 'read': .github/workflows/mergify-labels-copier.yml:9","Warn: no topLevel permission defined: .github/workflows/platform_ingest_docs_project_board.yml:1","Warn: no topLevel permission defined: .github/workflows/platform_logstash_project_board.yml:1","Info: topLevel 'contents' permission set to 'read': .github/workflows/pre-commit.yml:12","Warn: no topLevel permission defined: .github/workflows/project-board-assigner.yml:1","Warn: topLevel 'contents' permission set to 'write': .github/workflows/version_bumps.yml:22"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"License","score":9,"reason":"license file detected","details":["Info: project has a license file: LICENSE.txt:0","Warn: project license file does not contain an FSF or OSI license."],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Binary-Artifacts","score":9,"reason":"binaries present in source code","details":["Warn: binary detected: gradle/wrapper/gradle-wrapper.jar:1"],"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":5,"reason":"branch protection is not maximal on development and all release branches","details":["Info: 'allow deletion' disabled on branch 'main'","Info: 'force pushes' disabled on branch 'main'","Warn: 'branch protection settings apply to administrators' is disabled on branch 'main'","Warn: 'stale review dismissal' is disabled on branch 'main'","Warn: required approving review count is 1 on branch 'main'","Warn: codeowners review is not required on branch 'main'","Warn: 'last push approval' is disabled on branch 'main'","Warn: no status checks found to merge onto branch 'main'","Info: PRs are required in order to make changes on branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":9,"reason":"1 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-9hjg-9r4m-mvj7"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":7,"reason":"SAST tool is not run on all commits -- score normalized to 7","details":["Warn: 23 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-18T13:01:34.748Z","repository_id":260379598,"created_at":"2025-08-18T13:01:34.748Z","updated_at":"2025-08-18T13:01:34.748Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":27749219,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-15T02:00:09.782Z","response_time":96,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["etl-framework","java","jruby","logging","real-time-processing","streaming"],"created_at":"2024-07-30T17:01:35.971Z","updated_at":"2025-12-16T20:06:50.092Z","avatar_url":"https://github.com/elastic.png","language":"Java","readme":"# Logstash\n\nLogstash is part of the [Elastic Stack](https://www.elastic.co/products) along with Beats, Elasticsearch and Kibana. Logstash is a server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite \"stash.\" (Ours is Elasticsearch, naturally.). Logstash has over 200 plugins, and you can write your own very easily as well.\n\nFor more info, see \u003chttps://www.elastic.co/products/logstash\u003e\n\n## Documentation and Getting Started\n\nYou can find the documentation and getting started guides for Logstash\non the [elastic.co site](https://www.elastic.co/guide/en/logstash/current/getting-started-with-logstash.html)\n\nFor information about building the documentation, see the README in https://github.com/elastic/docs\n\n## Downloads\n\nYou can download officially released Logstash binaries, as well as debian/rpm packages for the\nsupported platforms, from [downloads page](https://www.elastic.co/downloads/logstash).\n\n## Need Help?\n\n- [Logstash Forum](https://discuss.elastic.co/c/logstash)\n- [Logstash Documentation](https://www.elastic.co/guide/en/logstash/current/index.html)\n- [Logstash Product Information](https://www.elastic.co/products/logstash)\n- [Elastic Support](https://www.elastic.co/subscriptions)\n\n## Logstash Plugins\n\nLogstash plugins are hosted in separate repositories under the [logstash-plugins](https://github.com/logstash-plugins) github organization. Each plugin is a self-contained Ruby gem which gets published to RubyGems.org.\n\n### Writing your own Plugin\n\nLogstash is known for its extensibility. There are hundreds of plugins for Logstash and you can write your own very easily! For more info on developing and testing these plugins, please see the [working with plugins section](https://www.elastic.co/guide/en/logstash/current/contributing-to-logstash.html)\n\n### Plugin Issues and Pull Requests\n\n**Please open new issues and pull requests for plugins under its own repository**\n\nFor example, if you have to report an issue/enhancement for the Elasticsearch output, please do so [here](https://github.com/logstash-plugins/logstash-output-elasticsearch/issues).\n\nLogstash core will continue to exist under this repository and all related issues and pull requests can be submitted here.\n\n## Developing Logstash Core\n\n### Prerequisites\n\n* Install JDK version 11 or 17. Make sure to set the `JAVA_HOME` environment variable to the path to your JDK installation directory. For example `set JAVA_HOME=\u003cJDK_PATH\u003e`\n* Install JRuby 9.2.x It is recommended to use a Ruby version manager such as [RVM](https://rvm.io/) or [rbenv](https://github.com/sstephenson/rbenv).\n* Install `rake` and `bundler` tool using `gem install rake` and `gem install bundler` respectively.\n\n### RVM install (optional)\n\nIf you prefer to use rvm (ruby version manager) to manage Ruby versions on your machine, follow these directions. In the Logstash folder:\n\n```sh\ngpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3\n\\curl -sSL https://get.rvm.io | bash -s stable --ruby=$(cat .ruby-version)\n```\n\n### Check Ruby version\n\nBefore you proceed, please check your ruby version by:\n\n```sh\n$ ruby -v\n```\n\nThe printed version should be the same as in the `.ruby-version` file.\n\n### Building Logstash\n\nThe Logstash project includes the source code for all of Logstash, including the Elastic-Licensed X-Pack features and functions; to run Logstash from source using only the OSS-licensed code, export the `OSS` environment variable with a value of `true`:\n\n``` sh\nexport OSS=true\n```\n\n* Set up the location of the source code to build\n\n``` sh\nexport LOGSTASH_SOURCE=1\nexport LOGSTASH_PATH=/YOUR/LOGSTASH/DIRECTORY\n```\n\n#### Install dependencies with `gradle` **(recommended)**[^1]\n\n* Install development dependencies\n```sh\n./gradlew installDevelopmentGems\n```\n\n* Install default plugins and other dependencies\n\n```sh\n./gradlew installDefaultGems\n```\n\n### Verify the installation\n\nTo verify your environment, run the following to start Logstash and send your first event:\n\n```sh\nbin/logstash -e 'input { stdin { } } output { stdout {} }'\n```\n\nThis should start Logstash with stdin input waiting for you to enter an event\n\n```sh\nhello world\n2016-11-11T01:22:14.405+0000 0.0.0.0 hello world\n```\n\n**Advanced: Drip Launcher**\n\n[Drip](https://github.com/ninjudd/drip) is a tool that solves the slow JVM startup problem while developing Logstash. The drip script is intended to be a drop-in replacement for the java command. We recommend using drip during development, in particular for running tests. Using drip, the first invocation of a command will not be faster but the subsequent commands will be swift.\n\nTo tell logstash to use drip, set the environment variable `` JAVACMD=`which drip` ``.\n\nExample (but see the *Testing* section below before running rspec for the first time):\n\n    JAVACMD=`which drip` bin/rspec\n\n**Caveats**\n\nDrip does not work with STDIN. You cannot use drip for running configs which use the stdin plugin.\n\n## Building Logstash Documentation\n\nTo build the Logstash Reference (open source content only) on your local\nmachine, clone the following repos:\n\n[logstash](https://github.com/elastic/logstash) - contains main docs about core features\n\n[logstash-docs](https://github.com/elastic/logstash-docs) - contains generated plugin docs\n\n[docs](https://github.com/elastic/docs) - contains doc build files\n\nMake sure you have the same branch checked out in `logstash` and `logstash-docs`.\nCheck out `master` in the `docs` repo.\n\nRun the doc build script from within the `docs` repo. For example:\n\n```\n./build_docs.pl --doc ../logstash/docs/index.asciidoc --chunk=1 -open\n```\n\n## Testing\n\nMost of the unit tests in Logstash are written using [rspec](http://rspec.info/) for the Ruby parts. For the Java parts, we use [junit](https://junit.org). For testing you can use the *test* `rake` tasks and the `bin/rspec` command, see instructions below:\n\n### Core tests\n\n1- To run the core tests you can use the Gradle task:\n\n    ./gradlew test\n\n  or use the `rspec` tool to run all tests or run a specific test:\n\n    bin/rspec\n    bin/rspec spec/foo/bar_spec.rb\n\n  Note that before running the `rspec` command for the first time you need to set up the RSpec test dependencies by running:\n\n    ./gradlew bootstrap\n\n2- To run the subset of tests covering the Java codebase only run:\n\n    ./gradlew javaTests\n\n3- To execute the complete test-suite including the integration tests run:\n\n    ./gradlew check\n\n4- To execute a single Ruby test run:\n\n    SPEC_OPTS=\"-fd -P logstash-core/spec/logstash/api/commands/default_metadata_spec.rb\" ./gradlew :logstash-core:rubyTests --tests org.logstash.RSpecTests\n\n5- To execute single spec for integration test, run:\n\n    ./gradlew integrationTests -PrubyIntegrationSpecs=specs/slowlog_spec.rb\n\nSometimes you might find a change to a piece of Logstash code causes a test to hang. These can be hard to debug.\n\nIf you set `LS_JAVA_OPTS=\"-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005\"` you can connect to a running Logstash with your IDEs debugger which can be a great way of finding the issue.\n\n### Plugins tests\n\nTo run the tests of all currently installed plugins:\n\n    rake test:plugins\n\nYou can install the default set of plugins included in the logstash package:\n\n    rake test:install-default\n\n---\nNote that if a plugin is installed using the plugin manager `bin/logstash-plugin install ...` do not forget to also install the plugins development dependencies using the following command after the plugin installation:\n\n    bin/logstash-plugin install --development\n\n## Building Artifacts\n\nBuilt artifacts will be placed in the `LS_HOME/build` directory, and will create the directory if it is not already present.\n\nYou can build a Logstash snapshot package as tarball or zip file\n\n```sh\n./gradlew assembleTarDistribution\n./gradlew assembleZipDistribution\n```\n\nOSS-only artifacts can similarly be built with their own gradle tasks:\n```sh\n./gradlew assembleOssTarDistribution\n./gradlew assembleOssZipDistribution\n\n```\n\nYou can also build .rpm and .deb, but the [fpm](https://github.com/jordansissel/fpm) tool is required.\n\n```sh\nrake artifact:rpm\nrake artifact:deb\n```\n\nand:\n\n```sh\nrake artifact:rpm_oss\nrake artifact:deb_oss\n```\n\n## Using a Custom JRuby Distribution\n\nIf you want the build to use a custom JRuby you can do so by setting a path to a custom\nJRuby distribution's source root via the `custom.jruby.path` Gradle property.\n\nE.g.\n\n```sh\n./gradlew clean test -Pcustom.jruby.path=\"/path/to/jruby\"\n```\n\n## Project Principles\n\n* Community: If a newbie has a bad time, it's a bug.\n* Software: Make it work, then make it right, then make it fast.\n* Technology: If it doesn't do a thing today, we can make it do it tomorrow.\n\n## Contributing\n\nAll contributions are welcome: ideas, patches, documentation, bug reports,\ncomplaints, and even something you drew up on a napkin.\n\nProgramming is not a required skill. Whatever you've seen about open source and\nmaintainers or community members saying \"send patches or die\" - you will not\nsee that here.\n\nIt is more important that you are able to contribute.\n\nFor more information about contributing, see the\n[CONTRIBUTING](./CONTRIBUTING.md) file.\n\n## Footnotes\n\n[^1]: \u003cdetails\u003e\u003csummary\u003eUse bundle instead of gradle to install dependencies\u003c/summary\u003e\n\n    #### Alternatively, instead of using `gradle` you can also use `bundle`:\n\n    * Install development dependencies\n\n        ```sh\n        bundle config set --local path vendor/bundle\n        bundle install\n        ```\n\n    * Bootstrap the environment:\n\n        ```sh\n        rake bootstrap\n        ```\n\n    * You can then use `bin/logstash` to start Logstash, but there are no plugins installed. To install default plugins, you can run:\n\n        ```sh\n        rake plugin:install-default\n        ```\n\n    This will install the 80+ default plugins which makes Logstash ready to connect to multiple data sources, perform transformations and send the results to Elasticsearch and other destinations.\n    \u003c/details\u003e\n","funding_links":[],"categories":["Ruby","Java","java","Data Management \u0026 Processing","Elastic Stack","其他__大数据","DevOps Tools","GitHub projects","streaming","Application Recommendation","DevNet Monitoring","Uncategorized","APM Monitoring","Logging","大数据","Observability"],"sub_categories":["Database \u0026 Cloud Management","网络服务_其他","📊 Data Analysis","SD-WAN","Uncategorized","Collector","Log Management \u0026 Log Pipelines"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felastic%2Flogstash","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felastic%2Flogstash","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felastic%2Flogstash/lists"}