{"id":20063038,"url":"https://github.com/alan-turing-institute/annotatechange","last_synced_at":"2025-09-10T17:33:38.501Z","repository":{"id":54423801,"uuid":"178001961","full_name":"alan-turing-institute/AnnotateChange","owner":"alan-turing-institute","description":"A simple flask application to collect annotations for the Turing Change Point Dataset, a benchmark dataset for change point detection algorithms","archived":false,"fork":false,"pushed_at":"2021-02-18T22:24:26.000Z","size":1222,"stargazers_count":20,"open_issues_count":0,"forks_count":12,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-04T08:40:39.577Z","etag":null,"topics":["bootstrap","change-detection","changepoint","changepoint-detection","d3-visualization","d3js","d3v5","flask","flask-application","flask-login","flask-sqlalchemy","open-science","reproducible-research","time-series","time-series-analysis"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2003.06222","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alan-turing-institute.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-27T13:38:30.000Z","updated_at":"2024-10-29T12:43:05.000Z","dependencies_parsed_at":"2022-08-13T15:10:48.328Z","dependency_job_id":null,"html_url":"https://github.com/alan-turing-institute/AnnotateChange","commit_stats":null,"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alan-turing-institute%2FAnnotateChange","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alan-turing-institute%2FAnnotateChange/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alan-turing-institute%2FAnnotateChange/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alan-turing-institute%2FAnnotateChange/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alan-turing-institute","download_url":"https://codeload.github.com/alan-turing-institute/AnnotateChange/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252542297,"owners_count":21764944,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bootstrap","change-detection","changepoint","changepoint-detection","d3-visualization","d3js","d3v5","flask","flask-application","flask-login","flask-sqlalchemy","open-science","reproducible-research","time-series","time-series-analysis"],"created_at":"2024-11-13T13:40:10.360Z","updated_at":"2025-05-05T17:32:03.871Z","avatar_url":"https://github.com/alan-turing-institute.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AnnotateChange\n\nWelcome to the repository of the \"AnnotateChange\" application. This \napplication was created to collect annotations of time series data in order to \nconstruct the [Turing Change Point \nDataset](https://github.com/alan-turing-institute/TCPD) (TCPD). The TCPD is a \ndataset of real-world time series used to evaluate change point detection \nalgorithms. For the change point detection benchmark that was created using \nthis dataset, see the [Turing Change Point Detection \nBenchmark](https://github.com/alan-turing-institute/TCPDBench) repository.\n\nAny work that uses this repository should cite our paper: [**Van den Burg \u0026 \nWilliams - An Evaluation of Change Point Detection Algorithms \n(2020)**](https://arxiv.org/abs/2003.06222). You can use the following BibTeX \nentry:\n\n```bib\n@article{vandenburg2020evaluation,\n        title={An Evaluation of Change Point Detection Algorithms},\n        author={{Van den Burg}, G. J. J. and Williams, C. K. I.},\n        journal={arXiv preprint arXiv:2003.06222},\n        year={2020}\n}\n```\n\nHere's a screenshot of what the application looks like during the annotation \nprocess:\n\n\u003cp align=\"center\"\u003e\n\u003cimg height=\"500px\" src=\"./annotatechange_wide.png\" alt=\"screenshot of \nAnnotateChange\" /\u003e\n\u003c/p\u003e\n\nSome of the features of AnnotateChange include:\n\n* Admin panel to add/remove datasets, add/remove annotation tasks, add/remove \n  users, and inspect incoming annotations.\n\n* Basic user management: authentication, email confirmation, forgotten \n  password, automatic log out after inactivity, etc. Users are only allowed to \n  register using an email address from an approved domain.\n\n* Task assignment of time series to user is done on the fly, ensuring no user \n  ever annotates the same dataset twice, and prioritising datasets that are \n  close to a desired number of annotations.\n\n* Interactive graph of a time series that supports pan and zoom, support for \n  multidimensional time series.\n\n* Mandatory \"demo\" to onboard the user to change point annotation.\n\n* Backup of annotations to the admin via email.\n\n* Time series datasets are verified upon upload acccording to a strict schema.\n\n## Getting Started\n\nBelow are instructions for setting up the application for local development \nand for running the application with Docker.\n\n### Basic\n\nAnnotateChange can be launched quickly for local development as follows:\n\n1. Clone the repo\n   ```\n   $ git clone https://github.com/alan-turing-institute/AnnotateChange\n   $ cd AnnotateChange\n   ```\n\n2. Set up a virtual environment and install dependencies (requires Python \n   3.7+)\n   ```\n   $ sudo apt-get install -y python3-venv # assuming Ubuntu\n   $ pip install wheel\n   $ python3 -m venv ./venv\n   $ source ./venv/bin/activate\n   $ pip install -r requirements.txt\n   ```\n\n3. Create local development environment file\n   ```\n   $ cp .env.example .env.development\n   $ sed -i 's/DB_TYPE=mysql/DB_TYPE=sqlite3/g' .env.development\n   ```\n   With ``DB_TYPE=sqlite3``, we don't have to deal with MySQL locally.\n\n4. Initialize the database (this will be a local ``app.db`` file).\n   ```\n   $ ./flask.sh db upgrade\n   ```\n\n5. Create the admin user account\n   ```\n   $ ./flask.sh admin add --auto-confirm-email\n   ```\n   The ``--auto-confirm-email`` flag automatically marks the email address of \n   the admin user as confirmed. This is mostly useful in development \n   environments when you don't have a mail address set up yet.\n\n6. Run the application\n   ```\n   $ ./flask.sh run\n   ```\n   This should tell you where its running, probably ``localhost:5000``. You \n   should be able to log in with the admin account you've just created.\n\n7. As admin, upload **ALL** demo datasets (included in [demo_data](./demo_data)) \n   through: Admin Panel -\u003e Add dataset. You should then be able to follow the \n   introduction to the app (available from the landing page).\n\n8. After completing the instruction, you then will be able to access the user \n   interface (\"Home\") to annotate your own time series.\n\n### Docker\n\nTo use AnnotateChange locally using Docker, follow the steps below. For a \nfull-fledged installation on a server, see the [deployment \ninstructions](./docs/DEPLOYMENT.md).\n\n0. Install [docker](https://docs.docker.com/get-docker/) and \n   [docker-compose](https://docs.docker.com/compose/install/).\n\n1. Clone this repository and switch to it:\n   ```\n   $ git clone https://github.com/alan-turing-institute/AnnotateChange\n   $ cd AnnotateChange\n   ```\n\n2. Build the docker image:\n   ```\n   $ docker build -t gjjvdburg/annotatechange .\n   ```\n\n3. Create the directory for persistent MySQL database storage:\n   ```\n   $ mkdir -p persist/{instance,mysql}\n   $ sudo chown :1024 persist/instance\n   $ chmod 775 persist/instance\n   $ chmod g+s persist/instance\n   ```\n\n4. Copy the environment variables file:\n   ```\n   $ cp .env.example .env\n   ```\n   Some environment variables can be adjusted if needed. For example, \n   when moving to production, you'll need to change the `FLASK_ENV` variable \n   accordingly. Please also make sure to set a proper `SECRET_KEY` and \n   `AC_MYSQL_PASSWORD` (`= MYSQL_PASSWORD`). You'll also need to configure a \n   mail account so the application can send out emails for registration etc. \n   This is what the variables prefixed with ``MAIL_`` are for. The \n   ``ADMIN_EMAIL`` is likely your own email, it is used when the app \n   encounters an error and to send backups of the annotation records. You can \n   limit the email domains users can use with the ``USER_EMAIL_DOMAINS`` \n   variable. See the [config.py](config.py) file for more info on the \n   configuration options.\n\n5. Create a local docker network for communiation between the AnnotateChange \n   app and the MySQL server:\n   ```\n   $ docker network create web\n   ```\n\n6. Launch the services with docker-compose\n   ```\n   $ docker-compose up\n   ```\n   You may need to wait 2 minutes here before the database is initialized.\n   If all goes well, you should be able to point your browser to \n   ``localhost:7831`` and see the landing page of the application. Stop the \n   service before continuing to the next step (by pressing `Ctrl+C`).\n\n7. Once you have the app running, you'll want to create an admin account so \n   you can upload datasets, manage tasks and users, and download annotation \n   results. This can be done using the following command:\n   ```\n   $ docker-compose run --entrypoint 'flask admin add --auto-confirm-email' annotatechange\n   ```\n\n8. As admin, upload **ALL** demo datasets (included in [demo_data](./demo_data)) \n   through: Admin Panel -\u003e Add dataset. You should then be able to follow the \n   introduction to the app (available from the landing page).\n\n9. After completing the instruction, you then will be able to access the user \n   interface (\"Home\") to annotate your own time series.\n\n## Notes\n\nThis codebase is provided \"as is\". If you find any problems, please raise an \nissue [on GitHub](https://github.com/alan-turing-institute/annotatechange).\n\nThe code is licensed under the [MIT License](./LICENSE).\n\nThis code was written by [Gertjan van den Burg](https://gertjan.dev) with \nhelpful comments provided by [Chris \nWilliams](https://homepages.inf.ed.ac.uk/ckiw/).\n\n## Some implementation details\n\nBelow are some thoughts that may help make sense of the codebase.\n\n* AnnotateChange is a web application build on the Flask framework. See [this \n  excellent \n  tutorial](https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world) \n  for an introduction to Flask. The [flask.sh](./flask.sh) shell script loads \n  the appropriate environment variables and runs the application.\n\n* The application handles user management and is centered around the idea of a \n  \"task\" which links a particular user to a particular time series to \n  annotate.\n\n* An admin role is available, and the admin user can manually assign and \n  delete tasks as well as add/delete users, datasets, etc. The admin user is \n  created using the [cli](./app/cli.py) (see the Getting Started documentation \n  above).\n\n* All datasets must adhere to a specific dataset schema (see \n  [utils/dataset_schema.json](app/utils/dataset_schema.json)). See the files \n  in [demo_data] for examples, as well as those in \n  [TCPD](https://github.com/alan-turing-institute/TCPD).\n\n* Annotations are stored in the database using 0-based indexing. Tasks are \n  assigned on the fly when a user requests a time series to annotate (see \n  [utils/tasks.py](app/utils/tasks.py)).\n\n* Users can only begin annotating when they have successfully passed the \n  introduction.\n\n* Configuration of the app is done through environment variables, see the \n  [.env.example](.env.example) file for an example.\n\n* Docker is used for deployment (see the deployment documentation in \n  [docs](docs)), and [Traefik](https://containo.us/traefik/) is used for SSL, \n  etc.\n\n* The time series graph is plotted using [d3.js](https://d3js.org/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falan-turing-institute%2Fannotatechange","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falan-turing-institute%2Fannotatechange","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falan-turing-institute%2Fannotatechange/lists"}