https://github.com/jonhoo/trawler

Workload generator that emulates the traffic pattern of lobste.rs
https://github.com/jonhoo/trawler

Last synced: over 1 year ago
JSON representation

Workload generator that emulates the traffic pattern of lobste.rs

• Host: GitHub
• URL: https://github.com/jonhoo/trawler
• Owner: jonhoo
• License: apache-2.0
• Created: 2018-03-08T18:34:28.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2021-06-19T03:00:48.000Z (about 5 years ago)
• Last Synced: 2025-05-07T09:56:17.783Z (over 1 year ago)
• Language: Rust
• Size: 232 KB
• Stars: 72
• Watchers: 3
• Forks: 9
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE-APACHE

Awesome Lists containing this project

README

          
[![Crates.io](https://img.shields.io/crates/v/trawler.svg)](https://crates.io/crates/trawler)

[![Documentation](https://docs.rs/trawler/badge.svg)](https://docs.rs/trawler/)

[![Build Status](https://travis-ci.org/jonhoo/trawler.svg?branch=master)](https://travis-ci.org/jonhoo/trawler)

# trawler-rs

This crate provides a workload generator that emulates the traffic to

[lobste.rs](https://lobste.rs). It is a mostly open-loop benchmark similar to

[TailBench](https://people.csail.mit.edu/sanchez/papers/2016.tailbench.iiswc.pdf), but it also

approximates the partly open-loop designed outlined in [*Open Versus Closed: A Cautionary

Tale*](https://www.usenix.org/legacy/event/nsdi06/tech/full_papers/schroeder/schroeder.pdf) by

having clients potentially issue more than one query per request.

The benchmarker main component is the "load generator". It generates requests according to

actual lobste.rs traffic patterns as reported in [here](https://lobste.rs/s/cqnzl5/), records

the request time, and sends the request description to an implementor of

`Service`. When the resulting future resolves, the generator logs how long the

request took to process, *and* how long the request took from when it was generated until it

was satisfied (this is called the *sojourn time*).

Trawler is written so that it can *either* be run against an instance of

the [lobsters Rails app](https://github.com/lobsters/lobsters) *or*

directly against a backend by issuing queries. The former is done using the provided binary,

whereas the latter is done by linking against this crate as a library an implementing the

`Service` trait. The latter allows benchmarking a data storage backend without also incurring

the overhead of the Rails frontend. Note that if you want to benchmark against the Rails

application, you must apply the patches in `lobsters.diff` first.