{"id":17632795,"url":"https://github.com/zpzim/scamp","last_synced_at":"2025-04-05T17:08:08.363Z","repository":{"id":37026587,"uuid":"127799249","full_name":"zpzim/SCAMP","owner":"zpzim","description":"The fastest way to compute matrix profiles on CPU and GPU!","archived":false,"fork":false,"pushed_at":"2024-03-11T20:20:38.000Z","size":46729,"stargazers_count":179,"open_issues_count":18,"forks_count":40,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-04-05T17:07:39.976Z","etag":null,"topics":["cuda","gpu","matrix-profile","python","time-series","time-series-analysis"],"latest_commit_sha":null,"homepage":"http://www.cs.ucr.edu/~eamonn/MatrixProfile.html","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zpzim.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-04-02T18:58:28.000Z","updated_at":"2025-04-04T15:15:53.000Z","dependencies_parsed_at":"2023-01-31T12:31:30.546Z","dependency_job_id":"f660513d-e3c6-4ee0-b77b-4ba232f5037c","html_url":"https://github.com/zpzim/SCAMP","commit_stats":{"total_commits":316,"total_committers":7,"mean_commits":"45.142857142857146","dds":0.06645569620253167,"last_synced_commit":"b55f1baf31b03ffb824c22336919cecfbf40ea92"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpzim%2FSCAMP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpzim%2FSCAMP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpzim%2FSCAMP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpzim%2FSCAMP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zpzim","download_url":"https://codeload.github.com/zpzim/SCAMP/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247369952,"owners_count":20927928,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cuda","gpu","matrix-profile","python","time-series","time-series-analysis"],"created_at":"2024-10-23T01:45:47.365Z","updated_at":"2025-04-05T17:08:08.339Z","avatar_url":"https://github.com/zpzim.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build and Test](https://github.com/zpzim/SCAMP/actions/workflows/build-and-test.yml/badge.svg)](https://github.com/zpzim/SCAMP/actions/workflows/build-and-test.yml)\n[![RTD Build Status](https://img.shields.io/readthedocs/scamp-docs)](https://scamp-docs.readthedocs.io/en/latest/)\n\n[![Docker Build and Push](https://github.com/zpzim/SCAMP/actions/workflows/docker-build.yml/badge.svg)](https://github.com/zpzim/SCAMP/actions/workflows/docker-build.yml)\n![Docker Image Version (latest semver)](https://img.shields.io/docker/v/zpzim/scamp?label=Docker%20Version)\n![Docker Image Size (latest semver)](https://img.shields.io/docker/image-size/zpzim/scamp)\n![Docker Pulls](https://img.shields.io/docker/pulls/zpzim/scamp)\n\n![PyPI](https://img.shields.io/pypi/v/pyscamp?label=pyscamp%20version)\n![PyPI - Downloads](https://img.shields.io/pypi/dm/pyscamp?label=pypi%20downloads)\n\n![Conda (channel only)](https://img.shields.io/conda/vn/conda-forge/pyscamp-gpu?label=conda-forge::pyscamp-gpu)\n![Conda](https://img.shields.io/conda/pn/conda-forge/pyscamp-gpu?label=pyscamp-gpu)\n![Conda](https://img.shields.io/conda/dn/conda-forge/pyscamp-gpu?label=Downloads%3A%20pyscamp-gpu)\n\n![Conda (channel only)](https://img.shields.io/conda/vn/conda-forge/pyscamp-cpu?label=conda-forge::pyscamp-cpu)\n![Conda](https://img.shields.io/conda/pn/conda-forge/pyscamp-cpu?label=pyscamp-cpu)\n![Conda](https://img.shields.io/conda/dn/conda-forge/pyscamp-cpu?label=Downloads%3A%20pyscamp-cpu)\n\n[![DOI](https://zenodo.org/badge/127799249.svg)](https://zenodo.org/badge/latestdoi/127799249)\n\n\n# SCAMP: SCAlable Matrix Profile\n\n## Table of Contents\n[Overview](https://github.com/zpzim/SCAMP#overview) \\\n[Documentation](https://github.com/zpzim/SCAMP#documentation) \\\n[Performance](https://github.com/zpzim/SCAMP#performance) \\\n[Python Module](https://github.com/zpzim/SCAMP#python-module) \\\n[Run Using Docker](https://github.com/zpzim/SCAMP#run-using-docker) \\\n[Distributed Operation](https://github.com/zpzim/SCAMP#distributed-operation) \\\n[Reference](https://github.com/zpzim/SCAMP#reference)\n\n## Overview\nThis is a GPU/CPU implementation of the SCAMP algorithm. SCAMP takes a time series as input and computes the matrix profile for a particular window size. You can read more at the [Matrix Profile Homepage](http://www.cs.ucr.edu/~eamonn/MatrixProfile.html)\nThis is a much improved framework over [GPU-STOMP](https://github.com/zpzim/STOMPSelfJoin) which has the following additional features:\n  * Tiling for large inputs \n  * Computation in fp32, mixed fp32/fp64, or fp64 (double is recommended for most datasets, single precision will work for some)\n  * fp32 version should get good performance on GeForce cards\n  * AB joins (you can produce the matrix profile from 2 different time series)\n  * Distributable (we use GCP but other cloud platforms can work) with verified scalability to billions of datapoints\n  * More types of matrix profiles! KNN, Matrix Summary, Sum, and 1NN without index! See the Docs!\n  * Extremely Efficient Implementation\n  * Extensible to adding optimized versions of custom join operations.\n  * CPU Support (Only enabled for double precision; does not support KNN joins yet)\n  * Handles NaN input values. The matrix profile will be computed while excluding any subsequence with a NaN value\n  * Python module: Use SCAMP in Python with pyscamp\n  * conda-forge integration: Install pyscamp seamlessly with conda.\n  * Extensive integration testing: SCAMP has thousands of input configurations tested with every PR.\n  * Automatic benchmarking: Helps ensure performance doesn't slip with future updates.\n\n## Why use SCAMP?\n\n  * It is [faster](https://scamp-docs.readthedocs.io/en/latest/performance.html) than other matrix profile libraries. For example, it is **20x** to **100x** faster than stumpy.\n  * It is very easy to install using conda and has very few dependencies.\n  * It handles real data: very large inputs, missing values, and flat regions with little issue.\n  * It can compute various other types of matrix profiles, including efficiently computing KNN matrix profiles, and matrix summaries (a.k.a. mplots). And can be extended to compute other types of profile efficiently.\n\n## Documentation\nSCAMP's documentation can be found at [readthedocs](https://scamp-docs.readthedocs.io/en/latest/).\n\n## Python module\n`pyscamp` is available through conda-forge:\n~~~\n# To install pyscamp with cpu/gpu support on Linux and Windows.\nconda install -c conda-forge pyscamp-gpu\n\n# To install pyscamp with cpu support only on Windows, Linux, or MacOS.\nconda install -c conda-forge pyscamp-cpu\n~~~\n\nNote that `pyscamp-gpu` can be installed and used even if you don't have a GPU, it will simply fall back to using your CPU. However, `pyscamp-cpu` is preferrable if you don't have a GPU because it builds with a newer compiler and does not require installing the `cudatoolkit` depencency.\n\nIf you run into problems using GPUs with `pyscamp-gpu` make sure your NVIDIA drivers are up to date. This is the most common cause of issues.\n\n### Installing from source\n\nIf you want you can build pyscamp from source which will have improved performance. A source distribution for a python3 module using pybind11 is available on pypi.org to install run:\n~~~\n# Python 3 and a c/c++ compiler is required.\n# cmake is required (if you don't have it you can pip install cmake)\npip install pyscamp\n~~~\n\nOnce installed you can use SCAMP in Python as follows:\n~~~\nimport pyscamp as mp\n\n# Allows checking if pyscamp was built with CUDA and has GPU support.\nhas_gpu_support = mp.gpu_supported()\n\n# Self join\nprofile, index = mp.selfjoin(a, sublen)\n\n# AB join using 4 threads, outputting pearson correlation.\nprofile, index = mp.abjoin(a, b, sublen, pearson=True, threads=4)\n~~~\n\nMore information and the API documentation for pyscamp is available on [readthedocs](https://scamp-docs.readthedocs.io/en/latest/)\n\n## Run Using Docker\nYou can run SCAMP via [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) using the prebuilt [image](https://hub.docker.com/r/zpzim/scamp) on dockerhub.\n\nIn order to expose the host GPUs nvidia-docker must be installed correctly. Please follow the directions provided on the nvidia-docker github page. The following example uses docker 19.03 functionality:\n~~~\ndocker pull zpzim/scamp:latest\ndocker run --gpus all \\\n   --volume /path/to/host/input/data/directory:/data \\\n   --volume /path/to/host/output/directory:/output \\\n   zpzim/scamp:latest /SCAMP/build/SCAMP \\\n   --window=\u003cwindow_size\u003e --input_a_file_name=/data/\u003cfilename\u003e \\\n   --output_a_file_name=/output/\u003cmp_filename\u003e \\\n   --output_a_index_file_name=/output/\u003cmp_index_filename\u003e\n~~~\n\n## Distributed Operation\nWe have a client/server architecture built using grpc. Tested on [GKE](https://cloud.google.com/kubernetes-engine/) but should be possible to get working on [Amazon EKS](https://aws.amazon.com/eks/) as well. \n\nFor more information on how to use the scamp client and server, please take a look at the [documentation](https://scamp-docs.readthedocs.io/en/latest/)\n\n## Reference\nIf you use SCAMP in your work, please reference the following paper:\n~~~\nZimmerman, Zachary, et al. \"Matrix Profile XIV: Scaling Time Series Motif Discovery with GPUs to Break a Quintillion Pairwise Comparisons a Day and Beyond.\" Proceedings of the ACM Symposium on Cloud Computing. 2019.\n~~~\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzpzim%2Fscamp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzpzim%2Fscamp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzpzim%2Fscamp/lists"}