{"id":23116307,"url":"https://github.com/nabilshadman/multiprocessing-time-series-data-simulation","last_synced_at":"2026-05-05T09:31:21.697Z","repository":{"id":176914072,"uuid":"380156718","full_name":"nabilshadman/multiprocessing-time-series-data-simulation","owner":"nabilshadman","description":"A simulation using multiprocessing to generate 10,000 time dependent samples from an initial dataset of 20 samples","archived":false,"fork":false,"pushed_at":"2024-12-04T13:40:31.000Z","size":28,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-15T20:17:13.802Z","etag":null,"topics":["multiprocessing","numpy","pandas","scipy","simulation","statistics","time-series"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nabilshadman.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-25T07:15:32.000Z","updated_at":"2024-12-04T13:42:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"56616764-5abd-4aec-873e-0a20ec671793","html_url":"https://github.com/nabilshadman/multiprocessing-time-series-data-simulation","commit_stats":null,"previous_names":["nabilshadman/multiprocessing-time-series-data-simulation"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nabilshadman/multiprocessing-time-series-data-simulation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nabilshadman%2Fmultiprocessing-time-series-data-simulation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nabilshadman%2Fmultiprocessing-time-series-data-simulation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nabilshadman%2Fmultiprocessing-time-series-data-simulation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nabilshadman%2Fmultiprocessing-time-series-data-simulation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nabilshadman","download_url":"https://codeload.github.com/nabilshadman/multiprocessing-time-series-data-simulation/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nabilshadman%2Fmultiprocessing-time-series-data-simulation/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32643478,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-04T10:08:07.713Z","status":"online","status_checked_at":"2026-05-05T02:00:06.033Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["multiprocessing","numpy","pandas","scipy","simulation","statistics","time-series"],"created_at":"2024-12-17T04:15:58.246Z","updated_at":"2026-05-05T09:31:21.677Z","avatar_url":"https://github.com/nabilshadman.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Time Series Data Simulation with Multiprocessing\n\nA Python implementation leveraging multiprocessing capabilities to generate large-scale time series data (10,000 samples) from an initial dataset of 20 samples, demonstrating parallel computation techniques for efficient data simulation.\n\n## Overview\n\nThis project demonstrates time series data generation using parallel processing in Python. It calculates rates of change from time-dependent data and uses those rates to simulate new parameter values while preserving statistical properties and correlations between parameters.\n\n## Features\n\n- **Rate of Change Calculation**: Computes temporal derivatives of time series data\n- **Parallel Processing**: Utilizes Python's multiprocessing to enable efficient simulation\n- **Statistical Preservation**: Maintains statistical relationships between parameters\n- **Large-scale Generation**: Scales from 20 samples to 10,000 samples\n- **Correlation-aware**: Considers cross-parameter correlations in simulations\n\n## Technical Implementation\n\n### Core Components\n\n1. **Rate Calculation Module**\n   - Computes rate of change for time series data\n   - Handles datetime conversions and missing values\n   - Provides robust error handling\n\n2. **Simulation Engine**\n   - Implements parallel simulation using Pool's starmap\n   - Uses kernel density estimation for rate distribution\n   - Preserves parameter correlations during simulation\n\n3. **Data Processing Pipeline**\n   - Reads initial time series data\n   - Processes and validates input parameters\n   - Generates synchronized timestamps\n\n### Project Structure\n```\n.\n├── original.csv                     # Initial dataset with parameters\n├── paramX1wrtTime.csv               # Time series data for parameter X1\n├── paramX2wrtTime.csv               # Time series data for parameter X2\n└── simulate_time_series_data.ipynb  # Main implementation notebook\n```\n\n## Getting Started\n\n### Prerequisites\n- Python 3.x\n- Required packages:\n  ```python\n  numpy\u003e=1.19.0\n  pandas\u003e=1.2.0\n  scipy\u003e=1.6.0\n  ```\n\n### Installation\n1. Clone the repository:\n```bash\ngit clone https://github.com/yourusername/time-series-data-simulation.git\ncd time-series-data-simulation\n```\n\n2. Install dependencies:\n```bash \npip install -r requirements.txt\n```\n\n### Usage\n1. Place your input data files in project directory:\n   - `original.csv`: Initial parameter dataset\n   - `paramX1wrtTime.csv`: Time series for first parameter\n   - `paramX2wrtTime.csv`: Time series for second parameter\n\n2. Run the simulation:\n```python\npython simulate_time_series_data.py\n```\n\n## Implementation Details\n\n### Key Functions\n\n1. **Rate Calculation**\n```python\ndef find_rate(data):\n    \"\"\"\n    Computes rate of change for time series data\n    Args:\n        data (pandas.DataFrame): Time series data with datetime index\n    Returns:\n        pandas.DataFrame: Data with additional rate column\n    \"\"\"\n```\n\n2. **Parameter Simulation**\n```python\ndef simulate_corr(temp):\n    \"\"\"\n    Simulates correlated parameter values\n    Args:\n        temp (int): Chunk identifier for parallel processing\n    Returns:\n        pandas.DataFrame: Simulated parameter values\n    \"\"\"\n```\n\n### Workflow\n1. Calculate rates of change for time-dependent parameters\n2. Generate rate samples using kernel density estimation\n3. Initialize parallel simulation parameters\n4. Execute parallel simulation processes\n5. Combine results and add timestamps\n6. Export final simulated dataset\n\n## Results\n\nThe simulation produces:\n- 10,000 synchronized time series samples\n- Preserved statistical properties\n- Maintained parameter correlations\n- Time-stamped data points\n\n## Contributing\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/AmazingFeature`)\n3. Commit changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnabilshadman%2Fmultiprocessing-time-series-data-simulation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnabilshadman%2Fmultiprocessing-time-series-data-simulation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnabilshadman%2Fmultiprocessing-time-series-data-simulation/lists"}