{"id":48477775,"url":"https://github.com/sfrechette/stream-sequelize-node","last_synced_at":"2026-04-07T07:52:18.476Z","repository":{"id":42361811,"uuid":"129408859","full_name":"sfrechette/stream-sequelize-node","owner":"sfrechette","description":"Ingest sample Market Orders Data feed from PubNub to Postgres with TimescaleDB extension installed and enabled for time series analysis.","archived":false,"fork":false,"pushed_at":"2025-05-17T01:01:38.000Z","size":62,"stargazers_count":14,"open_issues_count":0,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-17T01:33:23.921Z","etag":null,"topics":["nodejs","postgres","postgresql","pubnub","sequelize","stream","streams","timescaledb","timeseries","timeseries-analysis","timeseries-data","timeseries-database"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sfrechette.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-04-13T13:51:28.000Z","updated_at":"2025-05-17T01:00:29.000Z","dependencies_parsed_at":"2025-05-17T01:29:51.593Z","dependency_job_id":"4eaaad27-94ca-468e-adf6-c4c051d5b762","html_url":"https://github.com/sfrechette/stream-sequelize-node","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sfrechette/stream-sequelize-node","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfrechette%2Fstream-sequelize-node","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfrechette%2Fstream-sequelize-node/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfrechette%2Fstream-sequelize-node/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfrechette%2Fstream-sequelize-node/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sfrechette","download_url":"https://codeload.github.com/sfrechette/stream-sequelize-node/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sfrechette%2Fstream-sequelize-node/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31504897,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-07T03:10:19.677Z","status":"ssl_error","status_checked_at":"2026-04-07T03:10:13.982Z","response_time":105,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["nodejs","postgres","postgresql","pubnub","sequelize","stream","streams","timescaledb","timeseries","timeseries-analysis","timeseries-data","timeseries-database"],"created_at":"2026-04-07T07:52:18.366Z","updated_at":"2026-04-07T07:52:18.451Z","avatar_url":"https://github.com/sfrechette.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# stream-sequelize-node\nIngest sample Market Orders Data feed from PubNub to Postgres with TimescaleDB extension installed and enabled for time series analysis.\n### Installation    \nClone or download repository\n```\ngit clone https://github.com/sfrechette/stream-sequelize-node.git \ncd stream-sequilize-node\n```\nInstall the dependencies with the following command:\n```javascript\nnpm install\n```\n### Configuration \nCreate an .env file and add the following with your configuration values   \n```\nDB_SERVER_USER_NAME = \"\"    \nDB_SERVER_USER_PASSWORD = \"\"    \nDB_SERVER_HOST = \"\" \nDB_SERVER_PORT = \"\"     \n```\n### Usage   \n```javascript\nnode ingest.js\n```\n## Prerequesites  \nRunning this application assumes that you already have an instance of Postgres running and have installed the TimescaleDB extension. \nIf not please refer the following documentation:  \n\nGetting started, installing and setting up  \nhttps://docs.timescale.com/v0.9/getting-started\n\n### Database schema \nNext you need to execute the following SQL code snippets in your Postgres environment in order to create the database, TimescaleDB extension, schema definition, and hypertable.\n\n```sql\n\ncreate database market_orders;\n```\n\n```sql\n\\c market_orders\n```\n\n```sql\ncreate extension if not exists timescaledb cascade;\n```\n\n```sql\ncreate table orders (\n\torder_time          bigint not null, \n\ttrade_type          text not null,\n\tsymbol              text not null, \n\torder_quantity      double precision not null, \n\tbid_price           double precision not null\n);\n```\n\n```sql\ncreate index on orders(order_time desc);\ncreate index on orders(trade_type, order_time desc);\ncreate index on orders(symbol, order_time desc);\ncreate index on orders(order_time desc, order_quantity);\ncreate index on orders(order_time desc, bid_price);\n-- 86400000000 is in usecs and is equal to 1 day\nselect create_hypertable('orders', 'order_time', chunk_time_interval =\u003e 86400000000);\n```\n\nOnce completed, you can now run the application to start ingesting data in your database.\n### Sample queries\nAssuming that you have been ingesting data for a while (you can modify the time interval), run some queries to test and validate how Timescale performs nicely in fetching results while data is also being ingested in the database. \n```sql\n-- Average bid price by 5 minute intervals for Google and day trade type \nselect \ttime_bucket('5 minutes', to_timestamp(order_time)) as five_min,\n        avg(bid_price) as avg_bid_price\nfrom \torders\nwhere\tsymbol = 'Google' and trade_type = 'day'\ngroup by five_min\norder by five_min limit 20;\n```\n\n```sql\n-- Min and Max bid price by 2 minute intervals for all symbols \nselect \ttime_bucket('2 minutes', to_timestamp(order_time)) as two_min,\n        symbol, \n        min(bid_price) as min_bid_price,\n        max(bid_price) as max_bid_price \nfrom \torders\ngroup by two_min, symbol\norder by two_min desc limit 20;\n```\n\n```sql \n-- Orders with order quantity greater than 975...\nselect \torder_time,\n        trade_type, \n        order_quantity, \n        bid_price\nfrom \torders\nwhere\torder_quantity \u003e 975 and \n        symbol = 'Bespin Gas' and \n        trade_type = 'market' limit 20;\n```\n\n### Links\n**TimescaleDB**     \nhttps://www.timescale.com/\n\n**Data feed**   \nMarket Orders - artificial data stream that provides the latest market orders for a fictitious marketplace  \nhttps://www.pubnub.com/developers/realtime-data-streams/financial-securities-market-orders/","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsfrechette%2Fstream-sequelize-node","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsfrechette%2Fstream-sequelize-node","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsfrechette%2Fstream-sequelize-node/lists"}