{"id":19199822,"url":"https://github.com/rose-stl-lab/anomllm","last_synced_at":"2025-04-10T11:23:02.808Z","repository":{"id":257987128,"uuid":"841660370","full_name":"Rose-STL-Lab/AnomLLM","owner":"Rose-STL-Lab","description":null,"archived":false,"fork":false,"pushed_at":"2024-10-16T21:30:06.000Z","size":2655,"stargazers_count":16,"open_issues_count":2,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-24T10:11:10.390Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Rose-STL-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-12T21:14:52.000Z","updated_at":"2025-03-18T22:10:34.000Z","dependencies_parsed_at":"2024-10-18T02:39:05.020Z","dependency_job_id":null,"html_url":"https://github.com/Rose-STL-Lab/AnomLLM","commit_stats":null,"previous_names":["rose-stl-lab/anomllm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rose-STL-Lab%2FAnomLLM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rose-STL-Lab%2FAnomLLM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rose-STL-Lab%2FAnomLLM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rose-STL-Lab%2FAnomLLM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Rose-STL-Lab","download_url":"https://codeload.github.com/Rose-STL-Lab/AnomLLM/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248208604,"owners_count":21065203,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T12:29:06.701Z","updated_at":"2025-04-10T11:23:02.784Z","avatar_url":"https://github.com/Rose-STL-Lab.png","language":"Jupyter Notebook","funding_links":[],"categories":["异常检测"],"sub_categories":[],"readme":"\u003cp align=\"center\" \u003e\n  \u003ca href=\"https://github.com/Rose-STL-Lab/AnomLLM\"\u003e\u003cimg src=\"https://github.com/Rose-STL-Lab/AnomLLM/blob/dev/logos/AnomLLM.png?raw=true\" width=\"256\" height=\"256\" alt=\"AnomLLM\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003ch1 align=\"center\"\u003eAnomLLM\u003c/h1\u003e\n\u003ch4 align=\"center\"\u003eCan LLMs Understand Time Series Anomalies?\n\u003c/h4\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://raw.githubusercontent.com/Rose-STL-Lab/AnomLLM/refs/heads/dev/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/License-MIT-yellow.svg\" alt=\"license\"\u003e\u003c/a\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Python-3.10+-yellow\" alt=\"python\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Version-1.0.0-green\" alt=\"version\"\u003e\n\u003c/p\u003e\n\n\n## | Introduction\n\nWe challenge common assumptions about Large Language Models' capabilities in time series understanding. This repository contains the code for reproducing results and **benchmarking** your own large language models' (as long as they are compatible with OpenAI API) anomaly detection capabilities.\n\n## | Citation\n\n[[2410.05440] Can LLMs Understand Time Series Anomalies?](https://arxiv.org/abs/2410.05440)\n\n```\n@misc{zhou2024llmsunderstandtimeseries,\n      title={Can LLMs Understand Time Series Anomalies?}, \n      author={Zihao Zhou and Rose Yu},\n      year={2024},\n      eprint={2410.05440},\n      archivePrefix={arXiv},\n      primaryClass={cs.LG},\n      url={https://arxiv.org/abs/2410.05440}, \n}\n```\n\n## | Installation\n\n- Dependencies: `conda`\n- Run `export PYTHONPATH=$PYTHONPATH:$(pwd)/src` first\n- Jpyter notebook path shall be the root directory of the project.\n\n```bash\nconda env create --file environment.yml\nconda activate anomllm\npoetry install --no-root \n# Or `poetry install --no-root --with dev` if you need jupyter and etc.\n```\n\n## | Dataset Download\n\nWe recommend using [`s5cmd`](https://github.com/peak/s5cmd/tree/master) to download the dataset from the NRP S3 bucket.\n\n```bash\ns5cmd --no-sign-request --endpoint-url https://s3-west.nrp-nautilus.io cp \"s3://anomllm/data/*\" data/\n```\n\nAlternatively, you can download the dataset from the following link: [Google Drive](https://drive.google.com/file/d/19KNCiOm3UI_JXkzBAWOdqXwM0VH3xOwi/view?usp=sharing) or synthesize your own dataset using `synthesize.sh`. Make sure the dataset is stored in the `data` directory.\n\n## | API Configuration\n\nCreate a `credentials.yml` file in the root directory with the following content:\n\n```yaml\ngpt-4o:\n  api_key: \u003cYOUR_OPENAI_API_KEY\u003e\n  base_url: \"https://api.openai.com/v1\"\ngpt-4o-mini:\n  api_key: \u003cYOUR_OPENAI_API_KEY\u003e\n  base_url: \"https://api.openai.com/v1\"\ngemini-1.5-flash:\n  api_key: \u003cYOUR_GOOGLE_API_KEY\u003e\ninternvlm-76b:\n  api_key: \u003cYOUR_LOCAL_OPENAI_SERVER_API_KEY\u003e\n  base_url: \u003cYOUR_LOCAL_OPENAI_SERVER_ENDPOINT\u003e (ended with v1)\nqwen:\n  api_key: \u003cYOUR_LOCAL_OPENAI_SERVER_API_KEY\u003e\n  base_url: \u003cYOUR_LOCAL_OPENAI_SERVER_ENDPOINT\u003e (ended with v1)\n```\n\n## | Example Usage for Single Time Series\n\nCheck out the [example notebook](https://github.com/Rose-STL-Lab/AnomLLM/blob/dev/notebook/example.ipynb).\n\nTo run the example notebook, you only need the `gemini-1.5-flash` model in the `credentials.yml` file.\n\n## | Batch Run using OpenAI BatchAPI\n\n`python src/batch_api.py --data $datum --model $model --variant $variant`\n\nSee `test.sh` for comprehensive lists of models, variants, and datasets. The [Batch API](https://platform.openai.com/docs/guides/batch/overview) only works with OpenAI proprietary models and will reduce the cost by 50%, but it does not finish in real-time. Your first run will create a request file, and subsequent runs will check the status of the request and retrieve the results when they are ready.\n\n## | Online Run using OpenAI API\n\n`python src/online_api.py --data $datum --model $model --variant $variant`\n\nThe online API works with all OpenAI-compatible model hosting services.\n\n## | Evaluation\n\n`python src/result_agg.py --data $datum`\n\nThe evaluation script will aggregate the results from the API and generate the evaluation metrics, for all models and variants.\n\n## | License\n\nThis project is licensed under the MIT License.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frose-stl-lab%2Fanomllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frose-stl-lab%2Fanomllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frose-stl-lab%2Fanomllm/lists"}