{"id":25703184,"url":"https://github.com/eleutherai/tyche","last_synced_at":"2025-04-30T06:45:21.359Z","repository":{"id":252981051,"uuid":"841421794","full_name":"EleutherAI/tyche","owner":"EleutherAI","description":"Precisely estimating the volume of basins in neural net parameter space corresponding to interpretable behaviors","archived":false,"fork":false,"pushed_at":"2025-04-09T20:44:20.000Z","size":33763,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-30T06:45:09.546Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EleutherAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-12T11:27:02.000Z","updated_at":"2025-04-28T12:15:54.000Z","dependencies_parsed_at":"2025-01-07T19:50:48.552Z","dependency_job_id":"e93ba276-4ffa-42e9-80ba-ff522884c5f4","html_url":"https://github.com/EleutherAI/tyche","commit_stats":null,"previous_names":["eleutherai/sgdensity","eleutherai/basin-volume","eleutherai/tyche"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EleutherAI%2Ftyche","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EleutherAI%2Ftyche/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EleutherAI%2Ftyche/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EleutherAI%2Ftyche/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EleutherAI","download_url":"https://codeload.github.com/EleutherAI/tyche/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251658196,"owners_count":21622819,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-25T05:25:43.653Z","updated_at":"2025-04-30T06:45:21.343Z","avatar_url":"https://github.com/EleutherAI.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tyche: Estimating the sizes of basins in neural net parameter space corresponding to interpretable behaviors\n*She was often shown winged, wearing a crown, and bearing a sceptre and cornucopia; but she also appeared blindfolded and with various devices signifying uncertainty and risk. Among her monuments was a temple at Argos, where the legendary Palamedes is said to have dedicated to her the first set of dice.*\n\n## Purpose\n\nThis repo contains tool to measure the sizes of behavioral basins via Monte Carlo methods.\n\nA behavioral basin is a region in parameter space defined by an anchor point (a particular set of parameters) and a behavior-defining cost function. (We also refer to the size of the basin as the \"local volume\" of the anchor parameters.)\n\n\"Size\" here means total prior measure, where the prior is uniform (so that size = volume) or Gaussian (e.g. init distribution density).\n\nThe cost functions we implement are (1) loss over a dataset of inputs and outputs, and (2) KL divergence (from the anchor point) over a dataset of inputs.\n\nThe sizes of behavioral basins can be interpreted as a measure of the complexity of learned behaviors (by taking the negative logarithm). \n\nWhen the prior is normalized, they can also be interpreted as the prior probability of the basin-defining behavior.\n\n## Installation\n\n* Python 3.11 is recommended\n* `pip install -e .` or `pip install .`\n\nThis installs two packages: `tyche` and `palamedes`.\n\n`tyche` uses binary search to find the edge of a basin, a quick-and-dirty volume estimator to estimate volume, custom Gaussian-integral code to maintain numerical stability for very-high-dimensional Gaussian integrals, and a preconditioner matrix to ameliorate underestimation. It is very fast but can lead to dramatic underestimates of basin size. As seen in \"[Estimating the Probability of Sampling a Trained Neural Network at Random](https://arxiv.org/abs/2501.18812)\".\n\n`palamedes` (WIP) is a new project that uses SGLD and thermodynamic integration to achieve a measurement of basin volume. It is slower than `tyche` but (once it's done) will give much more precise basin size measurements.\n\nThe remaining documentation is `tyche`-specific.\n\n## Usage (HuggingFace models)\n\n```python\nfrom datasets import load_dataset\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\nimport torch\n\nfrom tyche import VolumeConfig, VolumeEstimator\n\n# Load any CausalLM model, tokenizer, and dataset\nmodel = AutoModelForCausalLM.from_pretrained(\"EleutherAI/pythia-14m\")\ntokenizer = AutoTokenizer.from_pretrained(\"EleutherAI/pythia-14m\")\ntokenizer.pad_token_id = 1  # pythia-specific\ntokenizer.eos_token_id = 0  # pythia-specific\ndataset = load_dataset(\"EleutherAI/lambada_openai\", name=\"en\", split=\"test\", trust_remote_code=True)\n\n# Configure the estimator\ncfg = VolumeConfig(model=model, \n                   tokenizer=tokenizer, \n                   dataset=dataset, \n                   text_key=\"text\",  # must match dataset field\n                   n_samples=10,  # number of MC samples\n                   cutoff=1e-2,  # KL-divergence cutoff (nats)\n                   max_seq_len=2048,  # max sequence length for tokenizer or chunk_and_tokenize\n                   val_size=10,  # number of dataset sequences to use. default (None) uses all.\n                   cache_mode=None,  # see below\n                   chunking=False,  # whether to use chunk_and_tokenize\n                   )\nestimator = VolumeEstimator.from_config(cfg)\n\n# Run the estimator\nresult = estimator.run()\n```\n\nThe result object is a `VolumeResult` with the following fields:\n\n* `estimates`: estimated log-probability of basin (natural log!)\n* `deltas`: actual KL differences (should be within ±10% of cutoff)\n* `props`, `mults`, `logabsint`: pieces of estimation calculation (for debugging)\n\nPreconditioners are not yet supported for this interface.\n\n\n### Cache mode\n\nThis setting controls how the original (unperturbed, center-of-basin) model outputs are cached:\n* `None`: (default) compute original-model outputs from scratch each iteration\n* `\"cpu\"`: keep original-model outputs on the CPU, moving to GPU in batches (slow...)\n* `\"gpu\"`: keep original-model outputs on the GPU (OOMs if val_size is too large)\n\n\n## Usage (models from the paper)\n\nAn interface for ConvNeXt and Pythia is available through `src/tyche/estimator.py`, with example usage (similar to the HuggingFace interface above) in `scripts/expt_paper.py`.\n\nThe MLP on `digits` is implemented in JAX on the branch `jax-hybrid`, which has additional dependencies:\n\n* `pip install -U \"jax[cuda12]\"`\n* `pip install -e .` or `pip install .`\n\nSee `notebooks/bigmlp_basins.ipynb` for usage.\n\n\n## Structure\n\n`notebooks/`: Jupyter notebooks\n\n`src/tyche/`: package source\n\n`.../convnext.py`: ConvNeXt on `cifar10`\n\n`.../data.py`: data preprocessing (from `sparsify`)\n\n`.../estimator.py`: classes for managing experiments and models\n\n`.../math.py`: integrals and such for high-dim geometry\n\n`.../precondition.py`: preconditioners\n\n`.../pythia.py`: Pythia on the Pile\n\n`.../utils.py`: misc helpful tools\n\n`.../volume.py`: core volume-estimation code\n\n`src/palamedes/`: Palamedes package source\n\n`scripts/`: command-line scripts (Python and shell)\n\n`.../expt_paper.py`: actual script used for results in paper\n\n`.../train_vision.py`: training script for ConvNeXt models (adapted from [https://github.com/EleutherAI/features-across-time])\n\n`old/`: large collection of old experiments and code (messy)\n\n`.../basin_precondition.ipynb`: early version of this project as a giant Jupyter notebook\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feleutherai%2Ftyche","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feleutherai%2Ftyche","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feleutherai%2Ftyche/lists"}