{"id":16936482,"url":"https://github.com/jonhoo/trawler","last_synced_at":"2025-05-09T01:25:14.296Z","repository":{"id":49302348,"uuid":"124432339","full_name":"jonhoo/trawler","owner":"jonhoo","description":"Workload generator that emulates the traffic pattern of lobste.rs","archived":false,"fork":false,"pushed_at":"2021-06-19T03:00:48.000Z","size":238,"stargazers_count":72,"open_issues_count":1,"forks_count":9,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-07T09:56:17.783Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonhoo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-03-08T18:34:28.000Z","updated_at":"2025-04-22T02:34:22.000Z","dependencies_parsed_at":"2022-09-26T20:41:09.354Z","dependency_job_id":null,"html_url":"https://github.com/jonhoo/trawler","commit_stats":null,"previous_names":[],"tags_count":62,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Ftrawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Ftrawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Ftrawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Ftrawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonhoo","download_url":"https://codeload.github.com/jonhoo/trawler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253172812,"owners_count":21865565,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T20:57:06.528Z","updated_at":"2025-05-09T01:25:14.277Z","avatar_url":"https://github.com/jonhoo.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Crates.io](https://img.shields.io/crates/v/trawler.svg)](https://crates.io/crates/trawler)\n[![Documentation](https://docs.rs/trawler/badge.svg)](https://docs.rs/trawler/)\n[![Build Status](https://travis-ci.org/jonhoo/trawler.svg?branch=master)](https://travis-ci.org/jonhoo/trawler)\n\n# trawler-rs\n\nThis crate provides a workload generator that emulates the traffic to\n[lobste.rs](https://lobste.rs). It is a mostly open-loop benchmark similar to\n[TailBench](https://people.csail.mit.edu/sanchez/papers/2016.tailbench.iiswc.pdf), but it also\napproximates the partly open-loop designed outlined in [*Open Versus Closed: A Cautionary\nTale*](https://www.usenix.org/legacy/event/nsdi06/tech/full_papers/schroeder/schroeder.pdf) by\nhaving clients potentially issue more than one query per request.\n\nThe benchmarker main component is the \"load generator\". It generates requests according to\nactual lobste.rs traffic patterns as reported in [here](https://lobste.rs/s/cqnzl5/), records\nthe request time, and sends the request description to an implementor of\n`Service\u003cTrawlerRequest\u003e`. When the resulting future resolves, the generator logs how long the\nrequest took to process, *and* how long the request took from when it was generated until it\nwas satisfied (this is called the *sojourn time*).\n\nTrawler is written so that it can *either* be run against an instance of\nthe [lobsters Rails app](https://github.com/lobsters/lobsters) *or*\ndirectly against a backend by issuing queries. The former is done using the provided binary,\nwhereas the latter is done by linking against this crate as a library an implementing the\n`Service` trait. The latter allows benchmarking a data storage backend without also incurring\nthe overhead of the Rails frontend. Note that if you want to benchmark against the Rails\napplication, you must apply the patches in `lobsters.diff` first.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonhoo%2Ftrawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonhoo%2Ftrawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonhoo%2Ftrawler/lists"}