{"id":20872789,"url":"https://github.com/netr/mimicr","last_synced_at":"2025-10-24T19:33:22.939Z","repository":{"id":199158924,"uuid":"702243677","full_name":"netr/mimicr","owner":"netr","description":"Rust library for working with HTTP requests to simulate a browser.","archived":false,"fork":false,"pushed_at":"2023-10-10T23:06:06.000Z","size":66,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-11T02:38:43.134Z","etag":null,"topics":["http-client","http-requests","rust","testing"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/netr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-08T22:50:09.000Z","updated_at":"2023-10-09T00:23:57.000Z","dependencies_parsed_at":"2023-10-11T01:31:00.835Z","dependency_job_id":null,"html_url":"https://github.com/netr/mimicr","commit_stats":null,"previous_names":["netr/carte"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netr%2Fmimicr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netr%2Fmimicr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netr%2Fmimicr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netr%2Fmimicr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/netr","download_url":"https://codeload.github.com/netr/mimicr/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243247963,"owners_count":20260748,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["http-client","http-requests","rust","testing"],"created_at":"2024-11-18T06:21:04.184Z","updated_at":"2025-10-24T19:33:17.896Z","avatar_url":"https://github.com/netr.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mimicr\n\n![mimicr github actions workflow](https://github.com/netr/mimicr/actions/workflows/mimicr.yml/badge.svg)\n\nUsing this project as a learning experience for Rust.\n\nRust library for working with HTTP requests to simulate a browser. The core concept of the library is to allow you to\nconstruct multiple-step schemas to e2e test or pen test. It is based\non [reqwest](https://docs.rs/reqwest/latest/reqwest/index.html)\nand [tokio](https://docs.rs/tokio/latest/tokio/index.html) for async.\n\nBy utilizing `reqwest` and the internal `cookie stores`, you can simulate a browser session. From creating accounts,\nlogging\nin, maintaining sessions, and more.\n\nWork in progress, subject to change. Not ready for production use.\n\n## Todo / Ideas\n\n- [x] Basic functionality for a simple recursive bot with multiple steps\n- [x] Create a macro for `hdr!`(r#\"Content-type: application/json#Accept: application/json\") to make it easier to create\n  headers\n- [x] Set up a `Request` builder pattern to make it less verbose to create requests\n- [X] Move `Context` to its own module and remove `pub` from `Context` struct\n- [ ] Allow for `shared state` such as databases or other resources\n- [X] Allow for skipping steps during `on_request()`\n- [ ] Create a more robust `example using an api` that requires authentication, session management, etc.\n- [ ] More robust timeout and pause functionality [Timeout, AfterSuccess, AfterError, etc]\n- [ ] Move `Context` to a `ContextBuilder` pattern to make it less verbose to create contexts?\n- [ ] Create a `curl command parser` to transform curl commands to mimicr steps\n\n## Usage for a 2 step bot\n\n```rust\nuse mimicr::{Worker, Context, Request, StepError, Stepable};\nuse std::future::Future;\nuse std::pin::Pin;\nuse std::time::Duration;\n\n// TODO: incorporate handle_recursive into Bot\nasync fn handle_recursive(mut worker: Worker, step: String) -\u003e Result\u003c(), reqwest::Error\u003e {\n    match worker.try_step(step.as_str()).await {\n        Ok(..) =\u003e {\n            if let Some(next_step) = worker.ctx.get_next_step() {\n                let fut: Pin\u003cBox\u003cdyn Future\u003cOutput=Result\u003c(), reqwest::Error\u003e\u003e\u003e\u003e =\n                    Box::pin(handle_recursive(worker, next_step));\n                fut.await?;\n            }\n        }\n        Err(err) =\u003e panic!(\"{}\", err),\n    }\n    Ok(())\n}\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c(), reqwest::Error\u003e {\n    let mut worker: Worker = Worker::new();\n    worker.add_many_steps(vec![Arc::new(RobotsTxt {}), Arc::new(LoginPage {})]);\n\n    handle_recursive(worker, Steps::RobotsTxt.to_string()).await\n}\n\nenum Steps {\n    Google,\n    Facebook,\n}\n\n// Display\nimpl fmt::Display for Steps {\n    fn fmt(\u0026self, f: \u0026mut fmt::Formatter\u003c'_\u003e) -\u003e fmt::Result {\n        write!(\n            f,\n            \"{}\",\n            match self {\n                Steps::Google =\u003e String::from(\"Google\"),\n                Steps::Facebook =\u003e String::from(\"Facebook\"),\n            }\n        )\n    }\n}\n\n#[derive(Clone, Copy)]\nstruct Google;\n\n#[async_trait]\nimpl Stepable for Google {\n    fn name(\u0026self) -\u003e String {\n        Steps::Google.to_string()\n    }\n\n    fn on_request(\u0026mut self) -\u003e Request {\n        let headers = hdr!(\n            r#\"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36\n            Accept: */*\"#\n        );\n\n        Request::new(Method::GET, \"https://google.com\".to_string())\n            .with_headers(headers)\n            .with_timeout(Duration::new(60, 0))\n            .with_status_codes(vec![200])\n            .build();\n    }\n\n    fn on_success(\u0026self, ctx: \u0026mut Context) {\n        println!(\n            \"[{}] Success! URL: {} - Time: {}\",\n            ctx.get_current_step().unwrap(),\n            ctx.get_url(),\n            ctx.get_time_elapsed_as_string(),\n        );\n        ctx.set_next_step(Steps::Facebook.to_string());\n    }\n\n    fn on_error(\u0026self, ctx: \u0026mut Context, err: StepError) {\n        todo!()\n    }\n\n    fn on_timeout(\u0026self, ctx: \u0026mut Context) {\n        todo!()\n    }\n}\n\n\n#[derive(Clone, Copy)]\nstruct Facebook;\n\n#[async_trait]\nimpl Stepable for Facebook {\n    fn name(\u0026self) -\u003e String {\n        Steps::Facebook.to_string()\n    }\n\n    fn on_request(\u0026mut self) -\u003e Request {\n        Request::new(Method::GET, \"https://facebook.com\".to_string())\n            .with_headers(hdr!(\"Accept-Encoding: gzip, deflate, br\"))\n            .with_timeout(Duration::new(60, 0))\n            .with_status_codes(vec![200])\n            .build();\n    }\n\n    fn on_success(\u0026self, ctx: \u0026mut Context) {\n        println!(\n            \"[{}] Success! URL: {} - Time: {}\",\n            ctx.get_current_step().unwrap(),\n            ctx.get_url(),\n            ctx.get_time_elapsed_as_string(),\n        );\n        // without setting a next_step, the bot will stop\n    }\n\n    fn on_error(\u0026self, ctx: \u0026mut Context, err: StepError) {\n        todo!()\n    }\n\n    fn on_timeout(\u0026self, ctx: \u0026mut Context) {\n        todo!()\n    }\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetr%2Fmimicr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetr%2Fmimicr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetr%2Fmimicr/lists"}