{"id":23777602,"url":"https://github.com/elmarx/miffy","last_synced_at":"2025-04-12T20:42:26.039Z","repository":{"id":270486107,"uuid":"904874817","full_name":"elmarx/miffy","owner":"elmarx","description":"A shadow-testing proxy","archived":false,"fork":false,"pushed_at":"2025-04-12T06:10:50.000Z","size":154,"stargazers_count":9,"open_issues_count":11,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-12T07:23:00.350Z","etag":null,"topics":["shadow-testing","traffic-mirroring"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elmarx.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-17T18:04:11.000Z","updated_at":"2025-04-12T06:10:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"8949a9c7-1148-49e1-b490-070b4b70bfc5","html_url":"https://github.com/elmarx/miffy","commit_stats":null,"previous_names":["elmarx/miffy"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elmarx%2Fmiffy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elmarx%2Fmiffy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elmarx%2Fmiffy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elmarx%2Fmiffy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elmarx","download_url":"https://codeload.github.com/elmarx/miffy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248631669,"owners_count":21136554,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["shadow-testing","traffic-mirroring"],"created_at":"2025-01-01T08:47:33.409Z","updated_at":"2025-04-12T20:42:26.032Z","avatar_url":"https://github.com/elmarx.png","language":"Rust","readme":"# Miffy\n\nMy diffy.\n\nA shadow-testing proxy: Send requests to a \"*reference*\" implementation, send the request to a \"*candidate*\"\nimplementation, always respond with the \"*reference*\" implementation and log/publish both responses\nif they are not\nequal.\n\n## Design goals\n\n- *reference* always wins: if the *candidate* fails, is slow, not available or whatever it should\n  not impact the\n  *reference*, always return a response as fast as possible\n- offload complex comparison: only basic comparison, in case of doubt publish both responses as\n  different to kafka\n\n## Development\n\nRequired tools:\n\n- [rust](https://www.rust-lang.org/tools/install)\n- docker-compose\n- [kcat](https://github.com/edenhill/kcat)\n\nNative libraries (required\nfor [rdakafka](https://github.com/fede1024/rust-rdkafka?tab=readme-ov-file#installation))\n\n- `libssl-dev`\n- `libsasl2-dev`\n\n## Demo\n\n* prepare the config: copy `config.sample.toml` to `config.toml`\n* start the demo-servers: `cargo run --example demo`. This will start two servers listening to\n  `localhost:3000` (the\n  reference) and `localhost:3001` (the candidate) with one endpoint: `/api/{value}`.\n* start kafka: `docker-compose up -d`\n* start **miffy**: `cargo run`.\n* send a request to a path under test: `curl http://localhost:8080/api/3`\n* send a request to any other path: `curl http://localhost:8080`\n* observe results in kafka: `kcat -b localhost:9092 -e -t miffy`\n\n# Configuration\n\nMiffy looks for a file `config.toml` in it's working-directory, but you can point to a specific file\nvia `MIFFY_CONFIG`-env-var.\n\n- get a working config by copying `config.sample.toml` to `config.toml`.\n- see `config.default.toml` for an explanation of different values and defaults.\n- all config-values values may be overriden via env-variable prefixed with `MIFFY_`.\n\nMiffy uses *rdkafka* internally and allows to set all\nits [properties](https://github.com/confluentinc/librdkafka/blob/master/CONFIGURATION.md), in\n`config.toml` section\n`[kafka]`.\nSince those values typically need to be set via environment variable (e.g. password for kafka,\n`sasl.password`), they\nmay be set/overriden via `KAFKA_SASL_PASSWORD` etc.\n\n## Deployment\n\nMiffy provides a separate management-port (default: **9000**).\n\nCurrently, only a health-endpoint `/healthz` is available.\n\n## Headers\n\nSometimes services need to know if they take part in shadow-testing and also which role they play. E.g. \"candidates\"\ncould still run in some dry-run mode where they not execute any side effects etc.\n\nMiffy therefore always sends a header name `X-Shadow-Test-Role`:\n\n- `candidate` — the service is the candidate for the current request\n- `reference` — the service is the reference for the current request\n- `upstream` — there is no experiment configured for the current request/route, so the service is just used as upstream\n\nApart from that, miffy does not touch/change/add/remove any headers.\n\n# Benchmarking\n\nTo estimate the rough overhead added by miffy, there are [just](https://just.systems/) recipes to\nrun [wrk](https://github.com/wg/wrk).\n\n* increase the open-file-limit for the shell running the demo: `ulimit -n 8182`\n* start the demo-servers: `cargo run --example demo --release`\n* start kafka: `docker-compose up -d`\n* increase open-file limit for the shell running miffy: `ulimit -n 8182`\n* start **miffy** (without access-logging): `RUST_LOG=warn cargo run --release`.\n* run the benchmark: `just bench`\n\n# License\n\nLicensed under either of\n\n* Apache License, Version 2.0\n  ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n* MIT license\n  ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\n\nat your option.\n\n## Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in the work by you, as defined in the Apache-2.0 license, shall be\ndual licensed as above, without any additional terms or conditions.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felmarx%2Fmiffy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felmarx%2Fmiffy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felmarx%2Fmiffy/lists"}