{"id":17202666,"url":"https://github.com/deepcharles/loadmydata","last_synced_at":"2026-03-12T08:20:13.688Z","repository":{"id":52504898,"uuid":"324981920","full_name":"deepcharles/loadmydata","owner":"deepcharles","description":"Utility functions for loading data sets.","archived":false,"fork":false,"pushed_at":"2023-11-15T10:23:17.000Z","size":6209,"stargazers_count":2,"open_issues_count":2,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-20T17:18:10.059Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deepcharles.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-28T10:32:44.000Z","updated_at":"2023-03-27T22:08:54.000Z","dependencies_parsed_at":"2022-09-17T15:01:41.358Z","dependency_job_id":null,"html_url":"https://github.com/deepcharles/loadmydata","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepcharles%2Floadmydata","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepcharles%2Floadmydata/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepcharles%2Floadmydata/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepcharles%2Floadmydata/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deepcharles","download_url":"https://codeload.github.com/deepcharles/loadmydata/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248782260,"owners_count":21160717,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-15T02:15:24.577Z","updated_at":"2026-03-12T08:20:13.636Z","avatar_url":"https://github.com/deepcharles.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# loadmydata\n\nUtility functions for loading **time series** data sets (Python 3.6++).\n\nThe list of available data sets currently includes:\n\n- the UEA/UCR repository,\n- the NYC taxi data set,\n- the Human locomotion data set.\n\n## Install\nThis package relies on requests, tqdm, yarl (for the download), and numpy.\n\nUse `pip` to install.\n\n```\npip install loadmydata\n```\n\nAlternatively, you can use `conda`:\n\n```\nconda config --add channels conda-forge\nconda install loadmydata\n```\n\n\n## Data format\n\nConsider a data set of *N* time series **y**\u003csup\u003e(1)\u003c/sup\u003e, **y**\u003csup\u003e(2)\u003c/sup\u003e,..., **y**\u003csup\u003e(N)\u003c/sup\u003e.\nEach **y**\u003csup\u003e(n)\u003c/sup\u003e has *T*\u003csup\u003e(n)\u003c/sup\u003e samples and *d* dimensions.\nNote that time series can have variable lengths, i.e. different *T*\u003csup\u003e(n)\u003c/sup\u003e but they share the same dimensionality *d*.\n\nSuch a data set is contained in a `numpy` array of shape (*N*, *T*, *d*) where *T*:=max\u003csub\u003en\u003c/sub\u003e *T*\u003csup\u003e(n)\u003c/sup\u003e.\nTime series with less than *T* samples are padded at the end with `numpy.nan`.\nIn addition, the extra padding is masked using [numpy's MaskedArray](https://numpy.org/doc/stable/reference/maskedarray.html).\n\n```python\nfrom loadmydata.padding import get_signal_shape\n\n# Assume that X contains a time series data set of shape (N, T, d)\nfor signal in X:\n    # signal is a masked array of shape (T, d).\n    # The true number of samples of the signal (without extra padding)\n    # can be accessed with `get_signal_shape`.\n    n_samples, n_dims = get_signal_shape(signal)\n    # To get the signal without the extra padding, do\n    signal_without_padding = signal[:n_samples]\n    # do something with signal_without_padding\n    ...\n```\n\n## UEA/UCR time series classification repository\n\nThe UEA/UCR repository focuses on time series classification.\nAs a result, each signal is associated with a label to predict.\n\nThe list of available data sets from the UEA/UCR repository is available [here](https://www.timeseriesclassification.com/dataset.php).\n\n\n### Usage example\n\n```python\nfrom loadmydata.load_uea_ucr import load_uea_ucr_data\n\ndataset_name = \"ArrowHead\"  # \"AbnormalHeartbeat\", \"ACSF1\", etc.\ndata = load_uea_ucr_data(dataset_name)\n\nprint(data.description)\nprint(data.X_train.shape)\nprint(data.X_test.shape)\nprint(data.y_train.shape)\nprint(data.y_test.shape)\n```\n\n## NYC taxi data set\n\nThis data set contains the number of New York taxi passengers aggregated in 30 minutes buckets for the period between July 2014 and January 2015. There are five anomalies occur during the NYC marathon, Thanksgiving, Christmas, New Years day, and a snow storm.\n\nThe raw data is from the NYC Taxi and Limousine Commission [1] and has been curated by [2].\n\n[1] https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page\n\n[2] Ahmad, S., Lavin, A., Purdy, S., \u0026 Agha, Z. (2017). Unsupervised real-time anomaly detection for streaming data. Neurocomputing.\n\n### Usage\n\n```python\nfrom loadmydata.load_nyc_taxi import load_nyc_taxi_dataset\n\nX, y, description = load_nyc_taxi_dataset()\n\nprint(description)\n```\n\n## Human locomotion data set\n\nThis data set consists of 1020 multivariate gait signals collected with two inertial measurement units, from 230 subjects undergoing a fixed protocol:\n\n- standing still,\n- walking 10 m,\n- turning around,\n- walking back,\n- stopping.\n\nIn total, there are 8.5 h of gait time series (sampled at 100 Hz). The measured population was composed of healthy subjects as well as patients with neurological or orthopedic disorders.\nThe start and end time stamps of more than 40,000 footsteps are available, as well as a number of contextual information about each trial. This exact data set was used in [1] to design and evaluate a step detection procedure.\n\nThe data are thoroughly described in [2].\n\n[1] Oudre, L., Barrois-Müller, R., Moreau, T., Truong, C., Vienne-Jumeau, A., Ricard, D., Vayatis, N., \u0026 Vidal, P.-P. (2018). Template-based step detection with inertial measurement units. Sensors, 18(11).\n\n[2] Truong, C., Barrois-Müller, R., Moreau, T., Provost, C., Vienne-Jumeau, A., Moreau, A., Vidal, P.-P., Vayatis, N., Buffat, S., Yelnik, A., Ricard, D., \u0026 Oudre, L. (2019). A data set for the study of human locomotion with inertial measurements units. Image Processing On Line (IPOL), 9.\n\n### Usage\n\n```python\nfrom loadmydata.load_human_locomotion import (\n    load_human_locomotion_dataset,\n    get_code_list,\n)\n\ncode_list = get_code_list()\ncode = code_list[9]  # '3-2'\ndata = load_human_locomotion_dataset(code)\n\nprint(data.signal)  # pandas array\nprint(data.left_steps)  # numpy array (n_left_steps, 2)\nprint(data.right_steps)  # numpy array (n_right_steps, 2)\nprint(data.metadata)  # dictionary\n\nprint(data.description)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepcharles%2Floadmydata","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeepcharles%2Floadmydata","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepcharles%2Floadmydata/lists"}