{"id":50653433,"url":"https://github.com/operationalfallacy/monte-carlo-ai-coding","last_synced_at":"2026-06-07T22:03:57.535Z","repository":{"id":283567410,"uuid":"951966661","full_name":"OperationalFallacy/monte-carlo-ai-coding","owner":"OperationalFallacy","description":"A Monte Carlo simulation how long it takes AI coding assistant to actually do stuff","archived":false,"fork":false,"pushed_at":"2025-03-20T22:20:59.000Z","size":264,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-20T23:26:48.253Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OperationalFallacy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-20T14:20:56.000Z","updated_at":"2025-03-20T22:21:02.000Z","dependencies_parsed_at":"2025-03-20T23:26:52.425Z","dependency_job_id":"f7f7c75a-bb4a-4065-b39e-1e356be72de2","html_url":"https://github.com/OperationalFallacy/monte-carlo-ai-coding","commit_stats":null,"previous_names":["operationalfallacy/monte-carlo-ai-coding"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/OperationalFallacy/monte-carlo-ai-coding","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OperationalFallacy%2Fmonte-carlo-ai-coding","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OperationalFallacy%2Fmonte-carlo-ai-coding/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OperationalFallacy%2Fmonte-carlo-ai-coding/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OperationalFallacy%2Fmonte-carlo-ai-coding/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OperationalFallacy","download_url":"https://codeload.github.com/OperationalFallacy/monte-carlo-ai-coding/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OperationalFallacy%2Fmonte-carlo-ai-coding/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34039503,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-07T02:00:07.652Z","response_time":124,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-07T22:03:55.752Z","updated_at":"2026-06-07T22:03:57.528Z","avatar_url":"https://github.com/OperationalFallacy.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI Coding Time Monte Carlo Simulator\n\nThe script simulates expected time for AI coding agent to generate code. It visualizes time distribution. You can use it for realistic estimation how much time you will spend coding with AI.\n\nThis project itself took about 8 hours. ChatGPT (paid plus subscription) designed formula for simulation, and Cursor (paid 20$) subscription wrote code.\n\nAs you can see, the simulation well confirms results obtained in the experiment itself. It took 8 hours to create 400 LOC you see in sim.ts script.\n\nThe simulation shows expected time for 100 LOC is 132 min (2 hours), see plots below.\n\nThese are the values I've used for simulation.\n\n20 seconds is reasonable assumption for Cursor response with 0.05 deviation (it actually was larger values). log-normal distribution well describes how AI gets \"stuck\" if unable to generate correct response within first attempts.\n\nDefinitely change these params where `CodingTimeSimulator` class instantiated and see how it affects expected value.\n\n```\n0.9 with 0.5 Std\nMean (μ) of log-normal AI retries. How many retries AI makes before success.\n\n20 sec with 3 Std\nMean (W) AI task generation time. Base time AI takes per attempt.\n\n0.05\nMultiplicative retry delay (α). Increases slowdown per retry.\n\n1\nRetry scaling exponent (p). Controls how retry impact grows.\n\n100\nTask size (LOC). Number of lines AI is generating.\n\n0.8 with 0.05 Std\nMean AI success rate (R). Fraction of code AI gets right per attempt.\n```\n\n$$\nT_{\\text{AI}} = W \\cdot N \\cdot (1 + \\alpha N^p) \\cdot \\frac{\\text{LOC}}{R \\cdot 60}\n$$\n\n**Where:**\n\n- $W$ : Wait Time Mean — Base AI response time per attempt (normally distributed)\n- $N$ : Retry Count — Log-normally distributed number of retries before success\n- $\\alpha$ : Retry Impact — Per-retry delay multiplier\n- $p$ : Retry Power — Exponent controlling retry slowdown growth\n- $\\text{LOC}$ : Lines of Code — Task size being generated\n- $R$ : Readiness — AI effectiveness (normally distributed)\n\n## Requirements\n\n- Node.js \u003e= 16\n- yarn\n\n## Installation\n\n```bash\nyarn install\n```\n\n## Usage\n\nRun simulation with default parameters:\n\n```bash\nyarn ts-node sim.ts\n```\n\n### Parameters\n\n- `mu` (0.9): Log-normal mean of retry count\n- `sigma` (0.5): Log-normal std dev of retries\n- `readinessMean` (0.8): AI success rate mean (80%)\n- `waitTimeMean` (20): AI response time mean in seconds\n- `retryImpact` (0.05): Slowdown per retry\n- `linesOfCode` (100): Task size in LOC\n\n### Output\n\nGenerates 2 plots:\n\n- Distribution histogram (log scale)\n- Cumulative distribution (log scale)\n\nFile naming: `sim_r{readiness}_w{wait}_mu{mu}_s{sigma}_i{impact}_p{power}_{scale}.png`\n\n## Example Results\n\nBelow are example simulation results with default parameters (N=1000):\n\n### Time Distribution\n\n![Time Distribution](./media/sim_r0.8_w20_mu0.9_s0.5_i0.05_p1_linear.png)\n\nThe histogram shows right-skewed distribution of AI coding times\n\n- It completes generation within 50-200 minutes\n- Long tail extends to ~800 minutes (AI gets \"stuck\")\n- Peak around 100-150 minutes matches median time\n- Log scale reveals the exponential nature of delays in retry (if it can't complete within few retries, you better step in and do some coding)\n\n### Cumulative Distribution\n\n![Cumulative Distribution](./media/sim_cumulative_r0.8_w20_mu0.9_s0.5_i0.05_p1_linear.png)\n\nThe S-curve shows cumulative probability of generating 100 LOC:\n\n- 50% chance it completes under 116 minutes (median)\n- 90% chance it complete under 240 minutes\n- 95% chance it completes under 300 minutes\n- Steep middle section (80-200 min) indicates typical completion range\n- There is a lot tail of of longer times\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foperationalfallacy%2Fmonte-carlo-ai-coding","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foperationalfallacy%2Fmonte-carlo-ai-coding","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foperationalfallacy%2Fmonte-carlo-ai-coding/lists"}