{"id":25179583,"url":"https://github.com/tegridydev/mechamap","last_synced_at":"2026-02-28T02:44:56.326Z","repository":{"id":269904863,"uuid":"908811797","full_name":"tegridydev/mechamap","owner":"tegridydev","description":"MechaMap - Toolkit for Mechanistic Interpretability (MI) Research","archived":false,"fork":false,"pushed_at":"2025-04-25T01:00:13.000Z","size":28,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-07T04:41:49.236Z","etag":null,"topics":["explanability","llm-research","mechanistic-interpretability","open-source","python","research-tool","transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tegridydev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-27T03:10:33.000Z","updated_at":"2025-04-25T01:00:16.000Z","dependencies_parsed_at":"2024-12-27T04:22:35.451Z","dependency_job_id":"db498023-7b1e-4532-950d-2f63b5b7b775","html_url":"https://github.com/tegridydev/mechamap","commit_stats":null,"previous_names":["tegridydev/mechamap"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tegridydev/mechamap","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tegridydev%2Fmechamap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tegridydev%2Fmechamap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tegridydev%2Fmechamap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tegridydev%2Fmechamap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tegridydev","download_url":"https://codeload.github.com/tegridydev/mechamap/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tegridydev%2Fmechamap/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29923428,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-27T19:37:42.220Z","status":"online","status_checked_at":"2026-02-28T02:00:07.010Z","response_time":90,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["explanability","llm-research","mechanistic-interpretability","open-source","python","research-tool","transformers"],"created_at":"2025-02-09T15:37:11.153Z","updated_at":"2026-02-28T02:44:56.292Z","avatar_url":"https://github.com/tegridydev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MechaMap - Tool for Mechanistic Interpretability (MI) Research\n\n**MechaMap** is a scanner and analysis framework designed to help researchers working on interpreting **\"wtfisgoingon\"** within transformer-based language models. It quickly surfaces which neurons (across all layers) might be responding strongly to different semantic categories—like *location*, *food*, *numeric*, *animal*, etc.—based on a single pass of a master text/s.\n\n## What is Mechanistic Interpretability?\n\n**Mechanistic Interpretability (MI)** is the discipline of **opening the black box** of large language models (and other neural networks) to understand the **underlying circuits**, **features** and/or **mechanisms** that give rise to specific behaviors. Instead of treating a model as a monolithic function, we:\n\n1. **Trace** how input tokens propagate through attention heads, MLP layers, or neurons,  \n2. **Identify** localized “circuit motifs” or subsets of weights that implement certain tasks,  \n3. **Explain** how the distribution of learned parameters leads to emergent capabilities,  \n4. **Develop** methods to systematically break down or “edit” these circuits to confirm we understand the causal structure.\n\nMechanistic Interpretability aspires to yield **human-understandable explanations** of how advanced models represent and manipulate concepts like “zero,” “red,” “lion,” or “London.” By doing so, we gain:\n\n- **Trust \u0026 Reliability**: More confidence in model outputs if we know the circuits behind them.  \n- **Safety \u0026 Alignment**: Early detection of harmful or unintended sub-circuits.  \n- **Debugging**: Efficient fixes or interventions if a model shows undesired behaviors.\n\n\u003e **Reference \u0026 Kudos**: This project owes a great deal to the insights from [Neel Nanda’s Mechanistic Interpretability Glossary](https://www.neelnanda.io/mechanistic-interpretability/glossary). \nNeel’s research and writing efforts have significantly helped the understanding of circuits and interpretability in large language models.\n\n## Goals of MechaMap\n\n- **Rapid Discovery**: Provide a **one-and-done** pass that highlights potentially interesting neurons—particularly those that strongly respond to high-level semantic categories (like “vehicle,” “food,” “numeric,” etc.).  \n- **Foundational Baseline**: Act as a **launchpad** for deeper Mechanistic Interpretability experiments. Once MechaMap flags certain “candidate neurons,” researchers can do single-neuron hooking or more advanced circuit analysis.  \n- **Usability**: Keep the scanning code straightforward, and produce easy-to-parse CSV/JSON files that can be quickly ingested into more advanced interpretability pipelines.  \n- **Transparency**: Centralize all category tokens, scanning thresholds, and master text within a single config. This fosters reproducibility and allows for quick expansions (adding more categories or new domain tokens).\n\n## Key Features\n\n1. **Single-Pass “Master Text”**  \n   A single text that includes sample tokens from each category. MechaMap runs one forward pass per neuron (hooked at MLP outputs), computing **average activation** on tokens for each category.\n\n2. **Customizable Categories**  \n   Default categories include *date*, *location*, *animal*, *food*, *numeric*, *language*, *vehicle*, *color*, but you can easily add *sports*, *finance*, or any domain tokens you care about.\n\n3. **Partial Scanning for Large Models**  \n   If a model is huge, you can limit to the **first N neurons** per layer. That speeds up scanning while preserving the same analysis code.\n\n4. **Top Tokens**  \n   For each neuron, MechaMap also saves a short list of the **top-activating tokens** from the text. This can reveal surprising structural triggers (like punctuation or stopwords).\n\n5. **Interpretation Script**  \n   A separate `interpret_map.py` tool helps you parse the CSV, show pivot tables, detect multi-category overlaps, or compute correlations among categories.\n\n## Why Use MechaMap for Mechanistic Interpretability?\n\n- **Discovery Layer**  \n  Instead of manually investigating thousands of neurons, MechaMap quickly flags where the biggest domain-sensitive signals might be happening—particularly in later layers, or “multi-domain” neurons that consistently appear across categories.\n\n- **Hypothesis Generation**  \n  Once you find a neuron that strongly activates for *food* tokens, you can design follow-up tests (e.g., hooking that neuron in isolation, adding or removing certain tokens) to confirm if it truly encodes “foodness.”\n\n- **Comparative Studies**  \n  Scan different models (e.g., `gpt2`, `EleutherAI/pythia-70m`) with the same master text and see whether they converge on similarly specialized neurons or if they use distinct “circuits.”\n\n- **Extendable**  \n  MechaMap is **config-based**: just add tokens to a category, or define new categories, and re-run. You can also adapt the master text to reflect your specific research interests (e.g., adding legal terms, chemistry tokens, or coding keywords).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftegridydev%2Fmechamap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftegridydev%2Fmechamap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftegridydev%2Fmechamap/lists"}