{"id":13484150,"url":"https://github.com/aderyabin/sniffer","last_synced_at":"2025-03-27T15:31:07.433Z","repository":{"id":26087291,"uuid":"107301234","full_name":"aderyabin/sniffer","owner":"aderyabin","description":"Log and Analyze Outgoing HTTP Requests","archived":false,"fork":false,"pushed_at":"2023-10-12T10:50:08.000Z","size":2924,"stargazers_count":568,"open_issues_count":16,"forks_count":36,"subscribers_count":11,"default_branch":"master","last_synced_at":"2024-05-21T19:26:27.559Z","etag":null,"topics":["http","ruby","sniffer"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aderyabin.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-10-17T17:22:51.000Z","updated_at":"2024-04-24T07:36:09.000Z","dependencies_parsed_at":"2024-01-08T17:14:02.421Z","dependency_job_id":"f1cdb376-d900-46de-ae0b-86450edfc7a5","html_url":"https://github.com/aderyabin/sniffer","commit_stats":{"total_commits":102,"total_committers":13,"mean_commits":7.846153846153846,"dds":0.4019607843137255,"last_synced_commit":"cdd52e4a89a1fa6f40295e3253898c8e2c3209d5"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aderyabin%2Fsniffer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aderyabin%2Fsniffer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aderyabin%2Fsniffer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aderyabin%2Fsniffer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aderyabin","download_url":"https://codeload.github.com/aderyabin/sniffer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245871804,"owners_count":20686264,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["http","ruby","sniffer"],"created_at":"2024-07-31T17:01:19.950Z","updated_at":"2025-03-27T15:31:06.460Z","avatar_url":"https://github.com/aderyabin.png","language":"Ruby","funding_links":[],"categories":["\u003ca id=\"79499aeece9a2a9f64af6f61ee18cbea\"\u003e\u003c/a\u003e浏览嗅探\u0026\u0026流量拦截\u0026\u0026流量分析\u0026\u0026中间人","Ruby","\u003ca id=\"7bf0f5839fb2827fdc1b93ae6ac7f53d\"\u003e\u003c/a\u003e工具","HTTP Clients and tools"],"sub_categories":["\u003ca id=\"99398a5a8aaf99228829dadff48fb6a7\"\u003e\u003c/a\u003e未分类-Network","\u003ca id=\"32739127f0c38d61b14448c66a797098\"\u003e\u003c/a\u003e嗅探\u0026\u0026Sniff"],"readme":"# Sniffer\n\n[![Build](https://github.com/aderyabin/sniffer/workflows/Run%20Tests/badge.svg)](https://github.com/aderyabin/sniffer/actions) [![Gem Version](https://badge.fury.io/rb/sniffer.svg)](https://rubygems.org/gems/sniffer) [![Join the chat at https://gitter.im/aderyabin/sniffer](https://badges.gitter.im/aderyabin/sniffer.svg)](https://gitter.im/aderyabin/sniffer?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n\nSniffer aims to help:\n\n  * Log outgoing HTTP requests. Sniffer logs as JSON format for export to ELK, Logentries and etc.\n  * Debug requests. Sniffer allows to save all requests/responses in storage for future debugging\n\nSniffer supports most common HTTP accessing libraries:\n\n* [Net::HTTP](http://ruby-doc.org/stdlib-2.4.2/libdoc/net/http/rdoc/Net/HTTP.html)\n* [HTTP](https://github.com/httprb/http)\n* [HTTPClient](https://github.com/nahi/httpclient)\n* [Patron](https://github.com/toland/patron)\n* [Curb](https://github.com/taf2/curb/)\n* [Ethon](https://github.com/typhoeus/ethon)\n* [Typhoeus](https://github.com/typhoeus/typhoeus)\n* [EM-HTTP-Request](https://github.com/igrigorik/em-http-request)\n* [Excon](https://github.com/excon/excon)\n\n## Demo\n\n![demo](https://github.com/aderyabin/sniffer/blob/master/assets/demo.gif?raw=true)\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'sniffer'\n```\n\nIf you wish Sniffer to use `Module#prepend` instead of `alias_method`, you can cause individual adapters to use `prepend` instead with:\n\n```ruby\ngem 'sniffer', require: ['http_prepend', 'httpclient_prepend', 'sniffer']\n```\n\nIt's important that `'sniffer'` is the last item in the list. See the `lib` directory for a list of prependable adapters.\n\nIf you want all adapters to use `prepend`:\n\n```ruby\ngem 'sniffer', require: ['all_prepend', 'sniffer']\n```\n\nAnd then execute:\n\n    $ bundle\n\nOr install it yourself as:\n\n    $ gem install sniffer\n\n## Usage\n\nHere's some simple examples to get you started:\n\n```ruby\nrequire 'http'\nrequire 'sniffer'\n\nSniffer.enable!\n\nHTTP.get('http://example.com/?lang=ruby\u0026author=matz')\nSniffer.data[0].to_h\n# =\u003e {:request=\u003e\n#   {:host=\u003e\"example.com\",\n#    :query=\u003e\"/?lang=ruby\u0026author=matz\",\n#    :port=\u003e80,\n#    :headers=\u003e{\"Accept-Encoding\"=\u003e\"gzip;q=1.0,deflate;q=0.6,identity;q=0.3\", \"Connection\"=\u003e\"close\"},\n#    :body=\u003e\"\",\n#    :method=\u003e:get},\n#  :response=\u003e\n#   {:status=\u003e200,\n#    :headers=\u003e\n#     {\"Content-Encoding\"=\u003e\"gzip\",\n#      \"Cache-Control\"=\u003e\"max-age=604800\",\n#      \"Content-Type\"=\u003e\"text/html\",\n#      \"Date\"=\u003e\"Thu, 26 Oct 2017 13:47:00 GMT\",\n#      \"Etag\"=\u003e\"\\\"359670651+gzip\\\"\",\n#      \"Expires\"=\u003e\"Thu, 02 Nov 2017 13:47:00 GMT\",\n#      \"Last-Modified\"=\u003e\"Fri, 09 Aug 2013 23:54:35 GMT\",\n#      \"Server\"=\u003e\"ECS (lga/1372)\",\n#      \"Vary\"=\u003e\"Accept-Encoding\",\n#      \"X-Cache\"=\u003e\"HIT\",\n#      \"Content-Length\"=\u003e\"606\",\n#      \"Connection\"=\u003e\"close\"},\n#    :body=\u003e \"OK\",\n#    :timing=\u003e0.23753299983218312}}\n```\n\nYou can clear saved data\n\n```\nSniffer.clear!\n```\n\nYou can configure capacity of storage to prevent the huge memory usage and set up log rotation.\nBy default log rotation is active (when capacity is set) and log works like a queue.\nIf rotation is disabled - requests will be logged until result log size reaches the capacity.\n\n```\n# will fill the storage and stop logging\nSniffer.config.store = {capacity: 1000, rotate: false}\n\n# will rotate logs to fit 1000 results (rotate is true by default)\nSniffer.config.store = {capacity: 1000}\n```\n\nYou can reset config to default\n\n```\nSniffer.reset!\n```\n\nYou can enable and disable Sniffer\n\n```\nSniffer.enable!\nSniffer.disable!\n```\n\nBy default output log looks like that:\n\n```\nD, [2017-10-26T16:47:14.007152 #59511] DEBUG -- : {\"port\":80,\"host\":\"example.com\",\"query\":\"/?lang=ruby\u0026author=matz\",\"rq_connection\":\"close\",\"method\":\"get\",\"request_body\":\"\",\"status\":200,\"rs_accept_ranges\":\"bytes\",\"rs_cache_control\":\"max-age=604800\",\"rs_content_type\":\"text/html\",\"rs_date\":\"Thu, 26 Oct 2017 13:47:13 GMT\",\"rs_etag\":\"\\\"359670651+gzip\\\"\",\"rs_expires\":\"Thu, 02 Nov 2017 13:47:13 GMT\",\"rs_last_modified\":\"Fri, 09 Aug 2013 23:54:35 GMT\",\"rs_server\":\"ECS (lga/1385)\",\"rs_vary\":\"Accept-Encoding\",\"rs_x_cache\":\"HIT\",\"rs_content_length\":\"1270\",\"rs_connection\":\"close\",\"timing\":0.513012999901548,\"response_body\":\"OK\"}\n```\nwhere `rq_xxx` is request header and `rs_xxx` - response header\n\n## Configuration\n\nSniffer default options:\n\n```ruby\nSniffer.config do |c|\n  c.logger = Logger.new($stdout)\n  c.severity = Logger::Severity::DEBUG\n  # HTTP options to log\n  c.log = {\n    request_url: true,\n    request_headers: true,\n    request_body: true,\n    request_method: true,\n    response_status: true,\n    response_headers: true,\n    response_body: true,\n    timing: true\n  }\n  c.store =  true # save requests/responses to Sniffer.data\n  c.enabled = false  # Sniffer disabled by default\n  c.url_whitelist = nil\n  c.url_blacklist = nil\nend\n```\n\n### Whitelist\n\nYou can add specific host url to whitelist as regexp or string. Sniffer will store only requests that matched.\n\n```ruby\nSniffer.config.url_whitelist = /whitelisted.com/\n\nHTTP.get('http://example.com')\nSniffer.data[0].to_h\n# =\u003e {}\n\nHTTP.get('http://whitelisted.com/')\nSniffer.data[0].to_h\n# =\u003e {{:request=\u003e{:host=\u003e\"whitelisted.com\", ....}}\n```\n\n### Blacklist\n\nYou can add specific host url to blacklist as regexp or string. Sniffer will ignore all matched requests.\n\n```ruby\nSniffer.config.url_blacklist = /blacklisted.com/\n\nHTTP.get('http://blacklisted.com')\nSniffer.data[0].to_h\n# =\u003e {}\n\nHTTP.get('http://example.com')\nSniffer.data[0].to_h\n# =\u003e {{:request=\u003e{:host=\u003e\"example.com\", ...}}\n```\n\n### Middleware\n\nYou can add the middleware to run custom code before/after the sniffed data was logged.\n\n```ruby\nSniffer.middleware do |chain|\n  chain.add MyHook\nend\n\nclass MyHook\n  def request(data_item)\n    puts \"Before work\"\n    yield\n    puts \"After work\"\n  end\n\n  def response(data_item)\n    puts \"Before work\"\n    yield\n    puts \"After work\"\n  end\nend\n```\n\n## Development\n\nAfter checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.\n\nTo install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\n\n## Development (with Docker)\n\nGet local development environment working and tests running is very easy with docker-compose:\n```sh\ndocker-compose run app bundle\ndocker-compose run app bundle exec rspec\n```\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/aderyabin/sniffer. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.\n\n## Acknowledge\n\n* [Sergey Ponomarev](https://github.com/sponomarev)\n* [Vladimir Dementyev](https://github.com/palkan)\n* [Salahutdinov Dmitry](https://github.com/dsalahutdinov)\n* [Stanislav Chereshkevich](https://github.com/dissident)\n* [Anatoliy Kurichev](https://github.com/russo-matrosso)\n* [Dmitriy Ivliev](https://github.com/moofkit)\n* [Nate Berkopec](https://github.com/nate-at-gusto)\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).\n\n## Code of Conduct\n\nEveryone interacting in the Sniffer project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/aderyabin/sniffer/blob/master/CODE_OF_CONDUCT.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faderyabin%2Fsniffer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faderyabin%2Fsniffer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faderyabin%2Fsniffer/lists"}