{"id":29356919,"url":"https://github.com/cesnet/cesnet-tszoo","last_synced_at":"2025-07-29T15:39:48.760Z","repository":{"id":279389910,"uuid":"938647561","full_name":"CESNET/cesnet-tszoo","owner":"CESNET","description":"CESNET Ts-Zoo is a toolkit for working with large time series network traffic datasets.","archived":false,"fork":false,"pushed_at":"2025-07-25T11:08:11.000Z","size":14141,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":16,"default_branch":"main","last_synced_at":"2025-07-25T15:38:40.299Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CESNET.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-02-25T09:30:55.000Z","updated_at":"2025-07-25T11:07:10.000Z","dependencies_parsed_at":"2025-05-09T15:05:55.386Z","dependency_job_id":null,"html_url":"https://github.com/CESNET/cesnet-tszoo","commit_stats":null,"previous_names":["cesnet/cesnet-tszoo"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CESNET/cesnet-tszoo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2Fcesnet-tszoo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2Fcesnet-tszoo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2Fcesnet-tszoo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2Fcesnet-tszoo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CESNET","download_url":"https://codeload.github.com/CESNET/cesnet-tszoo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CESNET%2Fcesnet-tszoo/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267709949,"owners_count":24131932,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-29T02:00:12.549Z","response_time":2574,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-09T05:40:56.471Z","updated_at":"2025-07-29T15:39:48.753Z","avatar_url":"https://github.com/CESNET.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n    \u003cimg src=\"https://raw.githubusercontent.com/CESNET/cesnet-tszoo/main/docs/images/tszoo.svg\" width=\"450\"\u003e\n\u003c/p\u003e\n\n[![](https://img.shields.io/badge/license-BSD-blue.svg)](https://github.com/CESNET/cesnet-tszoo/blob/main/LICENSE)\n[![](https://img.shields.io/badge/docs-cesnet--tszoo-blue.svg)](https://cesnet.github.io/cesnet-tszoo/)\n[![](https://img.shields.io/badge/python-\u003e=3.10-blue.svg)](https://pypi.org/project/cesnet-tszoo/)\n[![](https://img.shields.io/pypi/v/cesnet-tszoo)](https://pypi.org/project/cesnet-tszoo/)\n\nThe goal of `cesnet-tszoo` project is to provide time series datasets with useful tools for preprocessing and reproducibility. Such as:\n\n- API for downloading, configuring and loading CESNET-TimeSeries24, CESNET-AGG23 datasets. Each with various sources and aggregations.\n- Example of configuration options:\n  - Data can be split into train/val/test sets. Split can be done by time series or by time periods.\n  - Transforming of data with built-in scalers or with custom scalers.\n  - Handling missing values built-in fillers or with custom fillers.\n- Creation and import of benchmarks, for easy reproducibility of experiments.\n- Creation and import of annotations. Can create annotations for specific time series, specific time or specific time in specific time series.\n\n## Datasets\n\n| Name                      | CESNET-TimeSeries24                                                                       | CESNET-AGG23                                                                                          |\n|---------------------------|-------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------|\n| _Published in_            | 2025                                                                                      | 2023                                                                                                  |\n| _Collection duration_     | 40 weeks                                                                                  | 10 weeks                                                                                              |\n| _Collection period_       | 9.10.2023 - 14.7.2024                                                                     | 25.2.2023 - 3.5.2023                                                                                  |\n| _Aggregation window_      | 1 day, 1 hour, 10 min                                                                     | 1 min                                                                                                 |\n| _Sources_                 | CESNET3: Institutions, Institution subnets, IP addresses                                  | CESNET2                                                                                               |\n| _Number of time series_   | Institutions: 849, Institution subnets: 1644, IP addresses: 825372                        | 1                                                                                                     |\n| _Cite_                    | [https://doi.org/10.1038/s41597-025-04603-x](https://doi.org/10.1038/s41597-025-04603-x)  | [https://doi.org/10.23919/CNSM59352.2023.10327823](https://doi.org/10.23919/CNSM59352.2023.10327823)  |\n| _Zenodo URL_              | [https://zenodo.org/records/13382427](https://zenodo.org/records/13382427)                | [https://zenodo.org/records/8053021](https://zenodo.org/records/8053021)                              |\n| _Related papers_          |                                                                                           |                                                                                                       |\n\n## Installation\n\nInstall the package from pip with:\n\n```bash\npip install cesnet-tszoo\n```\n\nor for editable install with:\n\n```bash\npip install -e git+https://github.com/CESNET/cesnet-tszoo#egg=cesnet-tszoo\n```\n\n## Examples\n\n### Initialize dataset to create train, validation, and test dataframes\n\n#### Using [`TimeBasedCesnetDataset`](https://cesnet.github.io/cesnet-tszoo/reference_time_based_cesnet_dataset/) dataset\n\n```python\nfrom cesnet_tszoo.datasets import CESNET_TimeSeries24\nfrom cesnet_tszoo.utils.enums import SourceType, AgreggationType\nfrom cesnet_tszoo.configs import TimeBasedConfig\n\ndataset = CESNET_TimeSeries24.get_dataset(data_root=\"/some_directory/\", source_type=SourceType.INSTITUTIONS, aggregation=AgreggationType.AGG_1_DAY, is_series_based=False)\nconfig = TimeBasedConfig(\n    ts_ids=50, # number of randomly selected time series from dataset\n    train_time_period=range(0, 100), \n    val_time_period=range(100, 150), \n    test_time_period=range(150, 250), \n    features_to_take=[\"n_flows\", \"n_packets\"])\ndataset.set_dataset_config_and_initialize(config)\n\ntrain_dataframe = dataset.get_train_df()\nval_dataframe = dataset.get_val_df()\ntest_dataframe = dataset.get_test_df()\n```\n\nTime-based datasets are configured with [`TimeBasedConfig`](https://cesnet.github.io/cesnet-tszoo/reference_time_based_config/).\n\n#### Using [`SeriesBasedCesnetDataset`](https://cesnet.github.io/cesnet-tszoo/reference_series_based_cesnet_dataset/) dataset\n\n```python\nfrom cesnet_tszoo.datasets import CESNET_TimeSeries24\nfrom cesnet_tszoo.utils.enums import SourceType, AgreggationType\nfrom cesnet_tszoo.configs import SeriesBasedConfig\n\ndataset = CESNET_TimeSeries24.get_dataset(data_root=\"/some_directory/\", source_type=SourceType.INSTITUTIONS, aggregation=AgreggationType.AGG_1_DAY, is_series_based=True)\nconfig = SeriesBasedConfig(\n    time_period=range(0, 250), \n    train_ts=100, # number of randomly selected time series from dataset\n    val_ts=30, # number of randomly selected time series from dataset\n    test_ts=20, # number of randomly selected time series from dataset\n    features_to_take=[\"n_flows\", \"n_packets\"])\ndataset.set_dataset_config_and_initialize(config)\n\ntrain_dataframe = dataset.get_train_df()\nval_dataframe = dataset.get_val_df()\ntest_dataframe = dataset.get_test_df()\n```\n\nSeries-based datasets are configured with [`SeriesBasedConfig`](https://cesnet.github.io/cesnet-tszoo/reference_series_based_config/).\n\n#### Using [`load_benchmark`](https://cesnet.github.io/cesnet-tszoo/benchmarks_tutorial/)\n\n```python\nfrom cesnet_tszoo.benchmarks import load_benchmark\n\nbenchmark = load_benchmark(identifier=\"2e92831cb502\", data_root=\"/some_directory/\")\ndataset = benchmark.get_initialized_dataset()\n\ntrain_dataframe = dataset.get_train_df()\nval_dataframe = dataset.get_val_df()\ntest_dataframe = dataset.get_test_df()\n```\n\nWhether loaded dataset is series-based or time-based depends on the benchmark. What can be loaded corresponds to previous datasets.\n\n## Papers","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcesnet%2Fcesnet-tszoo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcesnet%2Fcesnet-tszoo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcesnet%2Fcesnet-tszoo/lists"}