{"id":30078646,"url":"https://github.com/mario-so/ohlcv-ts","last_synced_at":"2025-10-09T05:36:39.284Z","repository":{"id":294967477,"uuid":"988657244","full_name":"Mario-SO/ohlcv-ts","owner":"Mario-SO","description":null,"archived":false,"fork":false,"pushed_at":"2025-05-22T23:01:01.000Z","size":40,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-22T23:28:01.326Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://jsr.io/@mso/ohlcv","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mario-SO.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-22T22:00:56.000Z","updated_at":"2025-05-22T23:01:03.000Z","dependencies_parsed_at":"2025-05-22T23:28:10.750Z","dependency_job_id":"cc78a469-452d-47d1-a241-ec28c3e89c2f","html_url":"https://github.com/Mario-SO/ohlcv-ts","commit_stats":null,"previous_names":["mario-so/ohlcv-ts"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/Mario-SO/ohlcv-ts","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mario-SO%2Fohlcv-ts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mario-SO%2Fohlcv-ts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mario-SO%2Fohlcv-ts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mario-SO%2Fohlcv-ts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mario-SO","download_url":"https://codeload.github.com/Mario-SO/ohlcv-ts/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mario-SO%2Fohlcv-ts/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279000740,"owners_count":26082932,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-08T17:50:54.615Z","updated_at":"2025-10-09T05:36:39.279Z","avatar_url":"https://github.com/Mario-SO.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# @mso/ohlcv\n\nA high-performance TypeScript library for fetching and parsing OHLCV (Open,\nHigh, Low, Close, Volume) financial data in Deno. This library provides multiple\nparsing strategies optimized for different use cases, from simple CSV parsing to\nstreaming high-performance parsers.\n\n## 🚀 Features\n\n- **Multiple Parsing Strategies**: Choose from 4 different parsers based on your\n  performance needs\n- **Streaming Support**: Memory-efficient streaming parsers for large datasets\n- **Built-in Data Sources**: Pre-configured endpoints for BTC, ETH, SP500, and\n  Gold data\n- **Comprehensive Error Handling**: Detailed error types for robust error\n  handling\n- **Zero Dependencies**: Pure TypeScript implementation for Deno\n- **Type Safety**: Full TypeScript support with strict typing\n- **Performance Optimized**: Benchmarked parsers with different optimization\n  strategies\n\n## 📦 Installation\n\n### Add Package\n\n```bash\ndeno add jsr:@mso/ohlcv\n```\n\n### Import symbol\n\n```typescript\nimport * as ohlcv from \"@mso/ohlcv\";\n```\n\n### Or import directly with a JSR specifier\n\n```typescript\nimport * as ohlcv from \"jsr:@mso/ohlcv\";\n```\n\n### Import specific functions\n\n```typescript\nimport { DataSource, fetchCsvAsText, parseWithSimpleSplit } from \"@mso/ohlcv\";\n```\n\n### Import specific modules\n\n```typescript\nimport { fetchCsvAsStream } from \"@mso/ohlcv/provider\";\nimport { parseStreamOptimizedOhlcv } from \"@mso/ohlcv/parser\";\nimport type { Row } from \"@mso/ohlcv/types\";\n```\n\n## 🏃 Quick Start\n\n```typescript\nimport { DataSource, fetchCsvAsText, parseWithSimpleSplit } from \"@mso/ohlcv\";\n\n// Fetch and parse Bitcoin data\nconst csvData = await fetchCsvAsText(DataSource.BTC_CSV);\nconst rows = parseWithSimpleSplit(csvData);\n\nconsole.log(`Parsed ${rows.length} rows`);\nconsole.log(\"First row:\", rows[0]);\n// Output: { ts: 1640995200, o: 47686.81, h: 47865.45, l: 46617.24, c: 46498.76, v: 1234567 }\n```\n\n## 📊 Data Structure\n\nThe library works with a standardized `Row` interface:\n\n```typescript\ninterface Row {\n  ts: number; // Unix timestamp (seconds since epoch)\n  o: number; // Open price\n  h: number; // High price\n  l: number; // Low price\n  c: number; // Close price\n  v: number; // Volume\n}\n```\n\n## 🔄 Data Sources\n\nPre-configured data sources are available:\n\n```typescript\nenum DataSource {\n  BTC_CSV =\n    \"https://raw.githubusercontent.com/Mario-SO/ohlcv/main/data/btc.csv\",\n  SP500_CSV =\n    \"https://raw.githubusercontent.com/Mario-SO/ohlcv/main/data/sp500.csv\",\n  ETH_CSV =\n    \"https://raw.githubusercontent.com/Mario-SO/ohlcv/main/data/eth.csv\",\n  GOLD_CSV =\n    \"https://raw.githubusercontent.com/Mario-SO/ohlcv/main/data/gold.csv\",\n}\n```\n\n## 🔧 Parsing Strategies\n\n### 1. Simple Split Parser\n\nBest for: Small to medium datasets, simple use cases\n\n```typescript\nimport { parseWithSimpleSplit } from \"@mso/ohlcv\";\n\nconst rows = parseWithSimpleSplit(csvData, true); // skipHeader = true\n```\n\n### 2. State Machine Parser (Full String)\n\nBest for: Better error handling and validation\n\n```typescript\nimport { parseFullStringWithStateMachine } from \"@mso/ohlcv\";\n\nconst onSkipError = (error: Error, lineNumber: number, lineContent: string) =\u003e {\n  console.warn(`Skipped line ${lineNumber}: ${error.message}`);\n};\n\nconst rows = parseFullStringWithStateMachine(csvData, onSkipError);\n```\n\n### 3. Streaming State Machine Parser\n\nBest for: Large datasets, memory efficiency\n\n```typescript\nimport { fetchCsvAsStream, parseStreamWithStateMachine } from \"@mso/ohlcv\";\n\nconst stream = await fetchCsvAsStream(DataSource.BTC_CSV);\nlet processedRows = 0;\n\nconst totalRows = await parseStreamWithStateMachine(\n  stream,\n  (row: Row) =\u003e {\n    processedRows++;\n    // Process each row as it's parsed\n    console.log(`Processing row: ${row.ts}`);\n  },\n  (error, lineNumber, lineContent) =\u003e {\n    console.warn(`Error on line ${lineNumber}: ${error.message}`);\n  },\n);\n\nconsole.log(`Processed ${processedRows} of ${totalRows} total rows`);\n```\n\n### 4. Optimized OHLCV Stream Parser\n\nBest for: Maximum performance with large datasets\n\n```typescript\nimport { fetchCsvAsStream, parseStreamOptimizedOhlcv } from \"@mso/ohlcv\";\n\nconst stream = await fetchCsvAsStream(DataSource.SP500_CSV);\nconst rows: Row[] = [];\n\nconst totalRows = await parseStreamOptimizedOhlcv(\n  stream,\n  (row: Row) =\u003e {\n    rows.push(row);\n  },\n  true, // skipHeader\n  (error, lineNumber, lineContent) =\u003e {\n    console.warn(`Skipped line ${lineNumber}: ${error.message}`);\n  },\n);\n```\n\n## 🛠 Utility Functions\n\n### Date Conversion\n\n```typescript\nimport { yyyymmddToUnix } from \"@mso/ohlcv/utils\";\n\nconst timestamp = yyyymmddToUnix(\"2023-01-01\"); // Returns Unix timestamp\n```\n\n### Error Handling\n\nThe library provides comprehensive error types:\n\n```typescript\nimport {\n  FetchError,\n  HttpError,\n  InvalidFormatError,\n  InvalidTimestampError,\n  ParseError,\n} from \"@mso/ohlcv/errors\";\n\ntry {\n  const data = await fetchCsvAsText(DataSource.BTC_CSV);\n  const rows = parseWithSimpleSplit(data);\n} catch (error) {\n  if (error instanceof HttpError) {\n    console.error(`HTTP Error ${error.statusCode}: ${error.statusText}`);\n  } else if (error instanceof ParseError) {\n    console.error(`Parse Error: ${error.message}`, error.details);\n  } else if (error instanceof FetchError) {\n    console.error(`Fetch Error: ${error.message}`);\n  }\n}\n```\n\n## 📈 Performance Benchmarking\n\nRun the included benchmarks to compare parsing strategies:\n\n```bash\ndeno bench --allow-net bench.ts\n```\n\n### Benchmark Results\n\nHere are the performance results from running the benchmarks on a typical\nsystem:\n\n```\nbenchmark                               time/iter (avg)        iter/s      (min … max)           p75      p99     p995\n--------------------------------------- ----------------------------- --------------------- --------------------------\nparseWithSimpleSplit                            11.2 ms          88.9 (  9.9 ms …  11.8 ms)  11.5 ms  11.8 ms  11.8 ms\nparseFullStringWithStateMachine                  7.9 ms         127.4 (  7.4 ms …   9.5 ms)   7.9 ms   9.5 ms   9.5 ms\nparseStreamWithStateMachine                      7.7 ms         129.6 (  7.0 ms …   7.9 ms)   7.8 ms   7.9 ms   7.9 ms\nparseStreamOptimizedOhlcv                        7.4 ms         135.1 (  7.1 ms …   7.6 ms)   7.5 ms   7.6 ms   7.6 ms\n\ngroup dataset-size\nparseWithSimpleSplit - Small Dataset           193.5 µs         5,167 (174.1 µs … 259.2 µs) 200.6 µs 214.5 µs 219.5 µs\nparseWithSimpleSplit - Medium Dataset            1.9 ms         518.3 (  1.8 ms …   2.1 ms)   1.9 ms   2.0 ms   2.1 ms\nparseWithSimpleSplit - Large Dataset            11.5 ms          86.7 ( 11.0 ms …  11.9 ms)  11.7 ms  11.9 ms  11.9 ms\n\nsummary\n  parseWithSimpleSplit - Small Dataset\n     9.97x faster than parseWithSimpleSplit - Medium Dataset\n    59.59x faster than parseWithSimpleSplit - Large Dataset\n\ngroup network-fetch\nfetchCsvAsText - BTC                            17.8 ms          56.0 ( 16.2 ms …  25.9 ms)  17.9 ms  25.9 ms  25.9 ms\nfetchCsvAsStream - BTC                          18.4 ms          54.3 ( 15.8 ms …  28.5 ms)  18.9 ms  28.5 ms  28.5 ms\n\nsummary\n  fetchCsvAsText - BTC\n     1.03x faster than fetchCsvAsStream - BTC\n\ngroup end-to-end\nfetchAndParse - SimpleSplit                     19.3 ms          51.7 ( 17.3 ms …  23.9 ms)  19.3 ms  23.9 ms  23.9 ms\nfetchAndParse - StreamOptimized                 19.0 ms          52.5 ( 16.7 ms …  25.2 ms)  19.2 ms  25.2 ms  25.2 ms\n\nsummary\n  fetchAndParse - StreamOptimized\n     1.01x faster than fetchAndParse - SimpleSplit\n\ngroup stress\nmemory-stress-test                             118.6 ms           8.4 (116.9 ms … 122.0 ms) 119.5 ms 122.0 ms 122.0 ms\n```\n\n### Performance Insights\n\n- **🏆 parseStreamOptimizedOhlcv** is the fastest parser at **7.4ms** (135.1\n  iter/s)\n- **State machine parsers** outperform simple split by ~30-40%\n- **Dataset size** has linear impact on performance (59x difference between\n  small and large)\n- **Network fetch** performance is similar for text vs stream (~1ms difference)\n- **End-to-end** stream optimization provides minimal gains over simple parsing\n  for network-bound operations\n\nThe library includes comprehensive benchmarks testing:\n\n- Different parsing strategies\n- Various dataset sizes\n- Network fetch performance\n- End-to-end processing\n- Memory usage patterns\n\n## 🎯 Use Cases\n\n### Real-time Data Processing\n\n```typescript\nimport { fetchCsvAsStream, parseStreamOptimizedOhlcv } from \"@mso/ohlcv\";\n\n// Process data as it streams in\nconst stream = await fetchCsvAsStream(DataSource.BTC_CSV);\nlet latestPrice = 0;\n\nawait parseStreamOptimizedOhlcv(\n  stream,\n  (row: Row) =\u003e {\n    latestPrice = row.c; // Update latest close price\n\n    // Real-time processing logic here\n    if (row.c \u003e row.o) {\n      console.log(`Price up: ${row.c} at ${new Date(row.ts * 1000)}`);\n    }\n  },\n  true,\n);\n```\n\n### Data Analysis\n\n```typescript\nimport { DataSource, fetchCsvAsText, parseWithSimpleSplit } from \"@mso/ohlcv\";\n\nconst data = await fetchCsvAsText(DataSource.SP500_CSV);\nconst rows = parseWithSimpleSplit(data);\n\n// Calculate moving average\nconst movingAverage = (data: Row[], window: number): number[] =\u003e {\n  return data.slice(window - 1).map((_, index) =\u003e {\n    const slice = data.slice(index, index + window);\n    return slice.reduce((sum, row) =\u003e sum + row.c, 0) / window;\n  });\n};\n\nconst ma20 = movingAverage(rows, 20);\nconsole.log(`20-day moving average: ${ma20[ma20.length - 1]}`);\n```\n\n### Batch Processing\n\n```typescript\nimport { DataSource, fetchCsvAsText, parseWithSimpleSplit } from \"@mso/ohlcv\";\n\nconst dataSources = [\n  DataSource.BTC_CSV,\n  DataSource.ETH_CSV,\n  DataSource.SP500_CSV,\n];\n\nconst results = await Promise.all(\n  dataSources.map(async (source) =\u003e {\n    const data = await fetchCsvAsText(source);\n    const rows = parseWithSimpleSplit(data);\n    return {\n      source,\n      count: rows.length,\n      latestPrice: rows[rows.length - 1]?.c || 0,\n    };\n  }),\n);\n\nconsole.log(\"Batch processing results:\", results);\n```\n\n## 🔒 Permissions\n\nThis library requires the following Deno permissions:\n\n```bash\n# For fetching remote data\ndeno run --allow-net your_script.ts\n\n# For reading local files (if needed)\ndeno run --allow-read your_script.ts\n\n# For both\ndeno run --allow-net --allow-read your_script.ts\n```\n\n## 🧪 Running Examples\n\nRun the built-in demo:\n\n```bash\ndeno run --allow-net jsr:@mso/ohlcv\n```\n\nThis will demonstrate all parsing strategies with real data.\n\n## 📝 API Reference\n\n### Fetching Functions\n\n- `fetchCsvAsText(source: DataSource): Promise\u003cstring\u003e` - Fetch CSV as complete\n  string\n- `fetchCsvAsStream(source: DataSource): Promise\u003cReadableStream\u003cUint8Array\u003e\u003e` -\n  Fetch CSV as stream\n\n### Parsing Functions\n\n- `parseWithSimpleSplit(csvContent: string, skipHeader?: boolean): Row[]` -\n  Simple line-by-line parsing\n- `parseFullStringWithStateMachine(csvContent: string, onSkipError?: SkipErrorCallback): Row[]` -\n  State machine parsing with error handling\n- `parseStreamWithStateMachine(stream: ReadableStream\u003cUint8Array\u003e, onRow: RowCallback, onSkipError?: SkipErrorCallback): Promise\u003cnumber\u003e` -\n  Stream parsing with state machine\n- `parseStreamOptimizedOhlcv(stream: ReadableStream\u003cUint8Array\u003e, onRow: RowCallback, skipHeader?: boolean, onSkipError?: SkipErrorCallback): Promise\u003cnumber\u003e` -\n  Optimized stream parsing\n\n### Type Definitions\n\n- `Row` - Main data structure for OHLCV data\n- `RowCallback` - Function called for each parsed row\n- `SkipErrorCallback` - Function called when a line is skipped due to errors\n\n## 🤝 Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## 📄 License\n\nMIT License - see the [LICENSE](LICENSE) file for details.\n\n## 🔗 Links\n\n- [Deno Module Registry](https://deno.land/x/ohlcv)\n- [Source Code](https://github.com/Mario-SO/ohlcv-ts)\n- [Data Repository](https://github.com/Mario-SO/ohlcv)\n\n---\n\nBuilt with ❤️ for the Deno community\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmario-so%2Fohlcv-ts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmario-so%2Fohlcv-ts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmario-so%2Fohlcv-ts/lists"}