{"id":20063062,"url":"https://github.com/alan-turing-institute/tcpd","last_synced_at":"2025-04-05T08:03:47.360Z","repository":{"id":39877350,"uuid":"224688676","full_name":"alan-turing-institute/TCPD","owner":"alan-turing-institute","description":"The Turing Change Point Dataset - A collection of time series for the evaluation and development of change point detection algorithms","archived":false,"fork":false,"pushed_at":"2025-01-04T10:28:47.000Z","size":4613,"stargazers_count":147,"open_issues_count":0,"forks_count":26,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-29T07:04:31.157Z","etag":null,"topics":["change-detection","change-point","change-point-detection","changepoint","dataset"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2003.06222","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alan-turing-institute.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-28T16:07:55.000Z","updated_at":"2025-03-25T14:00:24.000Z","dependencies_parsed_at":"2023-02-01T03:30:46.414Z","dependency_job_id":"68e2b031-9124-43c6-a50f-f79d31662d9c","html_url":"https://github.com/alan-turing-institute/TCPD","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alan-turing-institute%2FTCPD","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alan-turing-institute%2FTCPD/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alan-turing-institute%2FTCPD/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alan-turing-institute%2FTCPD/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alan-turing-institute","download_url":"https://codeload.github.com/alan-turing-institute/TCPD/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247305932,"owners_count":20917208,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["change-detection","change-point","change-point-detection","changepoint","dataset"],"created_at":"2024-11-13T13:40:17.921Z","updated_at":"2025-04-05T08:03:47.329Z","avatar_url":"https://github.com/alan-turing-institute.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Turing Change Point Dataset\n\n[![Build Status](https://github.com/alan-turing-institute/TCPD/actions/workflows/validate.yml/badge.svg)](https://github.com/alan-turing-institute/TCPD/actions/workflows/validate.yml)\n[![DOI](https://zenodo.org/badge/224688676.svg)](https://zenodo.org/badge/latestdoi/224688676)\n\nWelcome to the host repository of the Turing Change Point Dataset, a set of \ntime series specifically collected for the evaluation of change point \ndetection algorithms on real-world data. This dataset was introduced in [this \npaper](https://arxiv.org/abs/2003.06222). For the repository containing the \ncode used for the experiments, see \n[TCPDBench](https://github.com/alan-turing-institute/TCPDBench).\n\n**Useful links:**\n\n- [Turing Change Point Dataset](https://github.com/alan-turing-institute/TCPD) \n  on GitHub.\n- [Turing Change Point Detection \n  Benchmark](https://github.com/alan-turing-institute/TCPDBench)\n- [An Evaluation of Change Point Detection Algorithms](https://arxiv.org/abs/2003.06222) by \n  [Gertjan van den Burg](https://gertjan.dev) and [Chris \n  Williams](https://homepages.inf.ed.ac.uk/ckiw/).\n- [Annotation Tool](https://github.com/alan-turing-institute/annotatechange)\n\n## Introduction\n\nChange point detection focuses on accurately detecting moments of abrupt \nchange in the behavior of a time series. While many methods for change point \ndetection exists, past research has paid little attention to the evaluation of \nexisting algorithms on real-world data. This work introduces a benchmark study \nand a dataset ([TCPD](https://github.com/alan-turing-institute/TCPD)) that are \nexplicitly designed for the evaluation of change point detection algorithms. \nWe hope that our work becomes a proving ground for the evaluation and \ndevelopment of change point detection algorithms that work well in practice.\n\nThis repository contains the code needed to obtain the time series in the \ndataset. For the benchmark study, see \n[TCPDBench](https://github.com/alan-turing-institute/TCPDBench). Note that \nwork based on the dataset should cite [our \npaper](https://arxiv.org/abs/2003.06222):\n\n```bib\n@article{vandenburg2020evaluation,\n        title={An Evaluation of Change Point Detection Algorithms},\n        author={{Van den Burg}, G. J. J. and Williams, C. K. I.},\n        journal={arXiv preprint arXiv:2003.06222},\n        year={2020}\n}\n```\n\nThe annotations are stored in the [annotations.json](annotations.json) file, \nwhich is the same as that used in the experiments (see \n[here](https://github.com/alan-turing-institute/TCPDBench/blob/master/analysis/annotations/annotations.json)). \nAnnotations are organised in a JSON object by dataset name and annotator id, \nand use 0-based indexing. See the \n[TCPDBench](https://github.com/alan-turing-institute/TCPDBench) repository for \nmore information on extending the benchmark with your own methods or datasets.\n\n## Getting Started\n\nMany of the time series in the dataset are included in this repository. \nHowever, due to licensing restrictions, some series can not be redistributed \nand need to be downloaded locally. We've added a Python script and a Makefile \nto make this process as easy as possible. There is also a Dockerfile to \nfacilitate reproducibility.\n\n### Using Docker\n\nTo build the dataset using Docker, first build the docker image:\n\n```\n$ docker build -t tcpd https://github.com/alan-turing-institute/TCPD.git\n```\n\nthen build the dataset:\n\n```\n$ docker run -i -t -v /path/to/where/you/want/the/dataset:/TCPD/export tcpd\n```\n\n### Using the command line\n\nTo obtain the dataset, please run the following steps:\n\n1. Clone the GitHub repository and change to the new directory:\n\n   ```\n   $ git clone https://github.com/alan-turing-institute/TCPD\n   $ cd TCPD\n   ```\n\n2. Make sure you have Python (v3.2 or newer) installed, as well as \n   [virtualenv](https://virtualenv.pypa.io/en/latest/):\n   ```\n   $ pip install virtualenv\n   ```\n\n3. Next, use either of these steps:\n   - To obtain the dataset using Make, simply run:\n\n     ```\n     $ make\n     ```\n\n     This command will download all remaining datasets and verify that they \n     match the expected checksums.\n\n   - If you don't have Make, you can obtain the dataset by manually executing \n     the following commands:\n\n     ```\n     $ virtualenv ./venv\n     $ source ./venv/bin/activate\n     $ pip install -r requirements.txt\n     $ python build_tcpd.py -v collect\n     ```\n\n     If you wish to verify the downloaded datasets you can run:\n\n     ```\n     $ python ./utils/check_checksums.py -v -c ./checksums.json -d ./datasets\n     ```\n\n4. It may be convenient to export all dataset files to a single directory. \n   This can be done using Make as follows:\n\n   ```\n   $ make export\n   ```\n\nAll datasets are stored in individual directories inside the ``datasets`` \ndirectory and each has its own README file with additional metadata and \nsources. The data format used is [JSON](https://json.org/) and each file \nfollows the [JSON Schema](https://json-schema.org/) provided in \n``schema.json``.\n\n## Using the data\n\nFor your convenience, example code to load a dataset from the JSON format to a \ndata frame is provided in the ``examples`` directory in the following \nlanguages:\n\n- [Python](examples/python/)\n- [R](examples/R/)\n\nImplementations of various change point detection algorithms that use these \ndatasets are available in \n[TCPDBench](https://github.com/alan-turing-institute/TCPDBench). A script to \nplot the datasets and detection results from \n[TCPDBench](https://github.com/alan-turing-institute/TCPDBench) is also \nprovided in [utils/plot_dataset.py](tree/master/utils/plot_dataset.py).\n\nThe annotations are included in the \n[annotations.json](tree/master/annotations.json) file. They are in the format:\n\n```\n{\n  \"\u003cdataset\u003e\": {\n      \"annotator_id\": [\n          \u003cchange point index\u003e\n          \u003cchange point index\u003e\n          ...\n          ],\n      ...\n  },\n  ...\n}\n```\n\nwhere the ``annotator_id`` is a unique ID for the annotator and the change \npoint indices are \n[0-based](https://en.wikipedia.org/wiki/Zero-based_numbering). Please also see \nthe documentation in \n[TCPDBench](https://github.com/alan-turing-institute/TCPDBench) for more \ninformation about using the dataset and benchmark in your own work.\n\n## License\n\nThe code in this repository is licensed under the MIT license. See the \n[LICENSE file](LICENSE) for more details. Individual data files are often \ndistributed under different terms, see the relevant README files for more \ndetails. Work that uses this dataset should cite [our \npaper](https://arxiv.org/abs/2003.06222).\n\n## Notes\n\nIf you find any problems or have a suggestion for improvement of this \nrepository, please let us know as it will help us make this resource better \nfor everyone. You can open an issue on \n[GitHub](https://github.com/alan-turing-institute/TCPD) or send an email to \n``gertjanvandenburg at gmail dot com``.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falan-turing-institute%2Ftcpd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falan-turing-institute%2Ftcpd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falan-turing-institute%2Ftcpd/lists"}