{"id":35003789,"url":"https://github.com/plasmacontrol/tokeye","last_synced_at":"2026-07-05T03:00:35.115Z","repository":{"id":324503068,"uuid":"1097434169","full_name":"PlasmaControl/tokeye","owner":"PlasmaControl","description":"TokEye is a open-source Python-based application for automatic classification and localization of fluctuating signals. It is designed to be used in the context of plasma physics, but can be used for any type of fluctuating signal.","archived":false,"fork":false,"pushed_at":"2026-07-01T15:20:51.000Z","size":38058,"stargazers_count":8,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-07-01T17:17:33.234Z","etag":null,"topics":["machine-learning","spectral-analysis","time-series"],"latest_commit_sha":null,"homepage":"https://control.princeton.edu","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PlasmaControl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-16T06:55:41.000Z","updated_at":"2026-06-22T06:16:47.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/PlasmaControl/tokeye","commit_stats":null,"previous_names":["plasmacontrol/tokeye"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/PlasmaControl/tokeye","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PlasmaControl%2Ftokeye","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PlasmaControl%2Ftokeye/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PlasmaControl%2Ftokeye/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PlasmaControl%2Ftokeye/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PlasmaControl","download_url":"https://codeload.github.com/PlasmaControl/tokeye/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PlasmaControl%2Ftokeye/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35141966,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-05T02:00:06.290Z","response_time":100,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","spectral-analysis","time-series"],"created_at":"2025-12-27T04:20:19.970Z","updated_at":"2026-07-05T03:00:35.070Z","avatar_url":"https://github.com/PlasmaControl.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"src/tokeye/app/assets/logo.png\" alt=\"TokEye Logo\" width=\"400\"\u003e\n\u003c/p\u003e\n\n# TokEye\n\n[![Python package](https://github.com/PlasmaControl/tokeye/actions/workflows/python-package.yml/badge.svg)](https://github.com/PlasmaControl/tokeye/actions/workflows/python-package.yml)\n\nTokEye is a open-source Python-based application for automatic classification and localization of fluctuating signals.\nIt is designed to be used in the context of plasma physics, but can be used for any type of fluctuating signal.\n\nCheck out [this poster from APS DPP 2025](assets/aps_dpp_2025.pdf) or [this preprint](https://arxiv.org/abs/2602.20317) for more information.\n\n## Example Demonstration\n\u003cvideo src=\"https://github.com/user-attachments/assets/03560db6-1941-483e-b9d7-706c164833f7\" autoplay loop muted playsinline controls width=\"100%\"\u003e\u003c/video\u003e\n\nExpected processing time:\n- V100: \u003c 0.5 seconds on any size spectrogram after warmup.\n- CPU: ~5-10 seconds.\n\n## Quickstart\n\n```bash\npip install tokeye   # or: uv tool install tokeye\ntokeye app           # opens web app on http://localhost:7860\n```\n\n- The default model downloads automatically from Hugging Face on first use (~30 MB).\n- No data on hand? Click \"Load Example Signal\" in the app, or generate one from the shell with `tokeye example`.\n- `pip install` requires Python \u003e= 3.13; `uvx`/`uv tool install` fetch a compatible Python automatically.\n\nZero-install trial: `uvx tokeye app` runs the app without installing anything into your environment.\n\n## Python API\n\nTo use TokEye inside your own program, import the `TokEye` class:\n\n```python\nimport numpy as np\nfrom tokeye import TokEye\n\neye = TokEye()  # loads the default model (auto-downloads on first use)\n\nmask = eye(signal)             # 1D time series → STFT → inference\nmask = eye(spectrogram)        # 2D spectrogram → inference directly\ncoherent, transient = mask     # (2, H, W) sigmoid scores in [0, 1]\n```\n\nInput is auto-detected by shape: a 1D array is treated as a raw time series (TokEye computes the spectrogram), a 2D array as a ready spectrogram. Standardization happens internally.\n\nIf your 2D spectrogram is stored in **linear scale** (raw STFT magnitude/power), pass `log=True` so TokEye applies `log1p` first — the model expects log-scaled input:\n\n```python\nmask = eye(linear_spectrogram, log=True)      # per call\neye = TokEye(log=True)                        # or for every call\n```\n\n`log` is off by default and ignored for 1D inputs (the STFT already log-scales). Everything is configurable through the constructor, but the defaults just work:\n\n```python\neye = TokEye(\n    model=\"big_tf_unet\",   # registry name or path to a local .pt/.pt2\n    device=\"auto\",         # \"cpu\", \"cuda\", or \"auto\"\n    n_fft=1024, hop=256,   # STFT settings (1D inputs only)\n    clip_dc=True, clip_low=1.0, clip_high=99.0,\n    log=False,             # log1p for linear-scale 2D spectrograms\n)\n```\n\n## Batch processing (CLI)\n\nFor headless / scripted use (no browser needed), run inference directly. For example:\n\n```bash\ntokeye run \"files/*.npy\" --output-dir results\n```\n\n`INPUT` arguments can be files, directories (all `*.npy` files inside are used), or quoted glob patterns. Each input is interpreted by its shape:\n- **1D array** — a raw time series. TokEye computes its STFT spectrogram using the flags below before running inference.\n- **2D array** — a precomputed spectrogram, fed to the model directly.\n\nFor each input file, `tokeye run` writes:\n- `\u003cstem\u003e_mask.npy` — float32 array, shape `(2, H, W)`, sigmoid scores per pixel (channel 0 = coherent, channel 1 = transient).\n- `\u003cstem\u003e_preview.png` — a grayscale spectrogram with the mask overlaid (green = coherent, red = transient), unless `--no-png` is passed.\n\nThe process exit code is the number of files that failed.\n\nFlags:\n| Flag | Default | Description |\n| --- | --- | --- |\n| `--model` | `big_tf_unet` | Registry name or path to a `.pt`/`.pt2` checkpoint. |\n| `--output-dir` | `tokeye_output` | Directory for masks and previews. |\n| `--n-fft` | `1024` | STFT window size (1D inputs only). |\n| `--hop` | `256` | STFT hop size (1D inputs only). |\n| `--keep-dc` | off | Keep the DC bin (dropped by default). |\n| `--clip-low` / `--clip-high` | `1.0` / `99.0` | Percentile clip bounds applied to the spectrogram. |\n| `--log` | off | Apply `log1p` to 2D spectrogram inputs stored in linear scale (1D signals are always log-scaled during the STFT). |\n| `--threshold` | `0.5` | Mask threshold used only for the preview PNG overlay. |\n| `--no-png` | off | Skip preview PNGs; write masks only. |\n| `--device` | `auto` | `cpu`, `cuda`, or `auto`. |\n\nThe released model was trained on spectrograms built with hop=128; for closest match to the training configuration use `--hop 128`.\n\nOn HPC clusters where compute nodes have no internet access, pre-fetch the weights on the login node, then run the batch job on the compute node:\n\n```bash\ntokeye download big_tf_unet   # on the login node; prints the cached path\ntokeye run ... --model big_tf_unet   # on the compute node — model is already cached\n```\n\n## Web app guide\n\n`tokeye app` (or `python -m tokeye.app`) launches a Gradio interface with three tabs:\n- **Analyze** — load a signal, compute its spectrogram, run a model, and visualize the result. Guided for first-time use: the model dropdown defaults to the bundled `big_tf_unet` model, the STFT transform has working defaults, and \"Load Example Signal\" generates a synthetic demo signal so a brand-new user needs zero files. \"Analyze\" runs the whole load-model → infer → visualize pipeline in one click. View modes: Original, Enhanced (percentile-clipped amplitude), Mask (thresholded model output), Amplitude.\n- **Annotate** — manually draw and save mask annotations over a read-only backdrop image.\n- **Utilities** — audio-format conversion and `.npy` file inspection.\n\nFlags: `tokeye app [--port 7860] [--share] [--open]` — `--share` creates a public Gradio link, `--open` opens a browser tab on launch.\n\nIf you're on a remote server (e.g. an HPC login node), forward the port over SSH instead of using `--share`:\n```bash\nssh -L 7860:localhost:7860 user@remote\n```\nThen open `http://localhost:7860` in your local browser.\n\n## Verified Datatypes\n- DIII-D Fast Magnetics (cite)\n- DIII-D CO2 Interferometer (cite)\n- DIII-D Electron Cyclotron Emission (cite)\n- DIII-D Beam Emission Spectroscopy (cite)\n\n## Evaluation\nRecall Scores:\n- TJII2021: 0.8254\n- DCLDE2011 (Delphinus capensis): 0.7708\n- DCLDE2011 (Delphinus delphis): 0.7953\n\nWith more data, comes better models. Please contribute to the project!\n\n## Installation (from source / development)\n\n[uv](https://docs.astral.sh/uv/) is the dev tool for this repo:\n```bash\ngit clone git@github.com:PlasmaControl/TokEye.git\ncd TokEye\nuv sync             # core deps\nuv sync --dev       # + pytest, ruff, etc.\nuv sync --group train  # + training deps (lightning, h5py, etc.)\n```\n\nThis creates a `.venv/`; activate it with `source .venv/bin/activate`, or prefix commands with `uv run`.\n\n## Models\n\n| Registry name | HF file | Description |\n| --- | --- | --- |\n| `big_tf_unet` | `big_tf_unet_251210.pt` | Transformer U-Net trained on multiscale (multiwindow, multihop) spectrograms. |\n\nWeights are hosted on [Hugging Face](https://huggingface.co/nc1/big_tf_unet) and download automatically the first time a registry name is used (cached in `~/.cache/huggingface`). Override the source repo with the `TOKEYE_HF_REPO` environment variable.\n\nTo use a local checkpoint instead, put `.pt`/`.pt2` files in a `model/` directory (picked up by the app's model dropdown) or pass a path directly via `--model PATH`.\n\nInput should be a tensor that has shape (B, 1, H, W) where B, H, and W can vary\nOutput will be a tensor of shape (B, 2, H, W)\n\nBest performance when spectrograms are oriented so that when they are plotted with matplotlib, the lowest frequency bin is oriented with the bottom when `origin='lower'`. Spectrograms should be standardized (mean = 0, std = 1). If baseline activity is very strong, clipping the input may help, but is generally not needed.\n\nThe first channel of the output will return preferential measurements of coherent activity (useful for most tasks)\nThe second channel of the output will return preferential measurements of transient activity\n\n## Data\nKeep signals as 1D numpy float arrays (raw time series). No need to normalize or preprocess them. The CLI also accepts 2D arrays (precomputed spectrograms) directly. The app scans a signal directory for `.npy` files (default `data/input`, configurable in the Analyze tab).\n\nBringing your own data takes two lines:\n\n```python\nimport numpy as np\n\nsignal = ...  # any 1D float array: tokamak diagnostic, hydrophone, etc.\nnp.save(\"shots/myshot.npy\", signal)\n```\n\n```bash\ntokeye run shots/myshot.npy --output-dir results\n```\n\nNo data yet? `tokeye example` writes a synthetic demo signal you can run immediately, and the web app has a matching \"Load Example Signal\" button.\n\n## Development\n\n```bash\nuv sync --dev\nuv run ruff check .\nuv run pytest\n```\n\n## Citation\nIf you use this code in your research, please cite:\n```bibtex\n@article{chen_TokEye_2026,\n  title={TokEye: Fast Signal Extraction for Fluctuating Time Series via Offline Self-Supervised Learning From Fusion Diagnostics to Bioacoustics},\n  author={Chen, Nathaniel},\n  year={2026},\n  publisher={ArXiv},\n  doi={10.48550/arXiv.2602.20317},\n  url={https://www.arxiv.org/abs/2602.20317}\n}\n```\n\n## Contact\nNathaniel Chen — nathaniel [at] princeton [dot] edu — https://nathanielchen.net\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplasmacontrol%2Ftokeye","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fplasmacontrol%2Ftokeye","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplasmacontrol%2Ftokeye/lists"}