{"id":19074393,"url":"https://github.com/maia14/data-feed","last_synced_at":"2026-05-07T02:31:16.993Z","repository":{"id":121021300,"uuid":"552109751","full_name":"MaiA14/data-feed","owner":"MaiA14","description":"An app that builds data feed using external API. User can filter data, update data, etc. Developed in TypeScript, Express \u0026 PostgreSQL.","archived":false,"fork":false,"pushed_at":"2022-10-21T13:23:04.000Z","size":81,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-22T04:41:10.228Z","etag":null,"topics":["express","nodejs","postgresql","server","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MaiA14.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-15T20:53:48.000Z","updated_at":"2022-10-22T21:06:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"9ce65fd6-1757-4888-828c-4bf4ac014d03","html_url":"https://github.com/MaiA14/data-feed","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MaiA14/data-feed","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaiA14%2Fdata-feed","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaiA14%2Fdata-feed/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaiA14%2Fdata-feed/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaiA14%2Fdata-feed/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MaiA14","download_url":"https://codeload.github.com/MaiA14/data-feed/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaiA14%2Fdata-feed/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32720109,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-07T02:14:30.463Z","status":"ssl_error","status_checked_at":"2026-05-07T02:14:29.405Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["express","nodejs","postgresql","server","typescript"],"created_at":"2024-11-09T01:50:47.294Z","updated_at":"2026-05-07T02:31:16.974Z","avatar_url":"https://github.com/MaiA14.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Data feed\n\nA company is building a system for managing data feeds inputs from external sources. \u003cbr\u003e\nData feed is a collection of data that has keys and values ordered according to specific\nstructure. \u003cbr\u003e\nEach source has a different way to login and extract the relevant data feed. \u003cbr\u003e\nUser can filter data using display options sidebar or update field value with new value.\n\n## Design\n\nThe user should be logged into the system to view the data feed.\n\n![Image](https://res.cloudinary.com/dtwqtpteb/image/upload/v1665940886/wg7sdygutef3ivwoixwo.png\n)\n\nIt means that the user can see only one feed at a time. \u003cbr\u003e\n\n\u003cb\u003e User's table: \u003c/b\u003e \u003cbr\u003e\nuser_id (Primary key) \u003cbr\u003e\nusername \u003cbr\u003e\npwd - password (in order to enable login functionality) \u003cbr\u003e\nemail \u003cbr\u003e\ncreated_on - creation time of the user \u003cbr\u003e\nlast_login (might be helpful in future feature, for example - token refresh) \u003cbr\u003e\n\n\u003cins\u003e Note: \u003c/ins\u003e \u003cbr\u003e\nin the future, there will be secured login using token. This change will required adding token to user's table. Token will be generated as hash of email \u0026 password. The pwd (user's password) that is saved in DB will be encrypted. It means that in login implemention, the server should decrypt the saved password from DB in order to compare it to the sent password by client. If passwords are equal, server returns success, otherwise - failure. On login implemention we also need to consider the case where email \u0026 password exist in DB but the token expired, then server need to generate new one and user should be able to login with the new token. (client should support updating token on his side).\n\n\u003cb\u003e Feed's table: \u003c/b\u003e \u003cbr\u003e\nfeed_id (Primary key) \u003cbr\u003e\nuser_id (foriegn key, references user table) \u003cbr\u003e\nupdated_on - timestamp that represents the last time the feed was updated \u003cbr\u003e\nfeed_data - json that represnts the data of the feed \u003cbr\u003e\n\n\n![Image](https://res.cloudinary.com/dtwqtpteb/image/upload/v1666027394/fkcssttopv7o83mgd3ll.png\n)\n\n## Workflow\n\n![Image](https://res.cloudinary.com/dtwqtpteb/image/upload/v1666006730/xmgthwfaildm4bg8xvra.jpg\n)\n\n## Prerequisites\nThe following technologies should be installed globally\n* Node (preferred version / project version - 14.15.3)\n* ts-node\n* TypeScript (latest)\n* Nodemon \n* PostgreSQL\n\n\nGet a key from this site (or use the one I supplied)\n```\nhttps://aviationstack.com/\n```\n\n## Installation\n\nBefore running this project install node modules with this command:\n\n```\nnpm install\n```\n\n\n\n## API\n\u003cins\u003e\u003cb\u003e1) Get data feed \u003c/ins\u003e\u003c/b\u003e \u003cbr\u003e\nRetrieves feed from an external source.\u003cbr\u003e\nPOST - http://localhost:9000/api/feed\n\n![Image](https://res.cloudinary.com/dtwqtpteb/image/upload/v1665951016/mdz2ihw6ikngfwyw4ylf.png\n)\n\nTry using curl:\n\n```\ncurl --location --request POST 'http://localhost:9000/api/feed' \\\n--header 'Content-Type: application/json' \\\n--header 'Cookie: connect.sid=s%3ATgv3GUfYzw7-EcNOhnaJVjy8NVcg5aWH.ZlJHeqmwkwnVIWzj0tENzz61oy2%2FiLbLnv0NRvC6CiI' \\\n--data-raw '{\n    \"source\": \"http://api.aviationstack.com/v1/flights?access_key=${API_KEY}\u0026limit=5\"\n}'\n```\n\nIn case the source is not supplied by the client, the server returns an error.\n\n![Image](https://res.cloudinary.com/dtwqtpteb/image/upload/v1665951480/cp5dzszhrxd9mwsi9dgs.png\n)\n\n\u003cins\u003e\u003cb\u003e2) Filter feed \u003c/ins\u003e\u003c/b\u003e \u003cbr\u003e\nFilters are based on a specific filter.\u003cbr\u003e\nSupported filters: \u003cbr\u003e\n- Limit results - choose the number of rows you want to see in the feed. \u003cbr\u003e\n- Show specified columns - choose which columns you want to see in the feed \u003cbr\u003e \u003cbr\u003e\n\u003cins\u003e Note:\u003c/ins\u003e \n* You can choose one of the following filters or both. \u003cbr\u003e\n* Filters like contains, greater than, lower than, will be implemented in the future.\n\nPOST - http://localhost:9000/api/feed/filter \u003cbr\u003e\n\nAn example of both filters applied:\n![Image](https://res.cloudinary.com/dtwqtpteb/image/upload/v1665952095/jlhyxpytrgz4cyedzrnh.png\n)\n\nTry using curl:\n\n```\ncurl --location --request POST 'http://localhost:9000/api/feed/filter' \\\n--header 'Content-Type: application/json' \\\n--header 'Cookie: connect.sid=s%3AcwpJbF2WrTLlpin0cGFs_9d2fsyQAW2D.a%2FMWgd3wloyb7TWyrBnCSEkhwroRWd1bQbJ4dR2hUcM' \\\n--data-raw '{\n    \"filters\": {\n        \"columns\": [\"flight\"],\n        \"limit\": 1\n    }\n}'\n```\n\nIn case no column is sent, the filter(s) is ignored since there is no way to filter an empty set of data – this scenario is equal to not sending a request at all, and in a real-world system, the client should prevent it from going out. It is added here as a safeguard\nIf the body of the request is empty, the server returns an error.\n\n\n![Image](https://res.cloudinary.com/dtwqtpteb/image/upload/v1665952367/hogscsbevd11prjbbc1e.png\n)\n\nIf the body of the request is plain, server returns error \u003cbr\u003e\n![Image](https://res.cloudinary.com/dtwqtpteb/image/upload/v1665952625/ry3xoabyj25j7jzanw4s.png\n)\n\nShow only flight date \u0026 flight status columns:\n\n![Image](https://res.cloudinary.com/dtwqtpteb/image/upload/v1665952958/y7yt4olnkf75bforw3k9.png\n)\n\n\u003cins\u003e\u003cb\u003e3) Update field \u003c/ins\u003e\u003c/b\u003e \u003cbr\u003e\nYou can update field by passing the field name you want to alter \u0026 the new value. \u003cbr\u003e\n\u003cins\u003e Note: \u003c/ins\u003e currently this feature updating row \u0026 multiple rows of specific column. \u003cbr\u003e \u003cbr\u003e\nPOST - http://localhost:9000/api/feed/updateField \u003cbr\u003e\n\nLet's say we want to update the first row (first record) with a new value for the field \"live\". \u003cbr\u003e\nOriginal field equals null as you can see: \u003cbr\u003e\n\n![Image](https://res.cloudinary.com/dtwqtpteb/image/upload/v1665963184/bftz3y8ro86dsy2uqlnx.png\n)\n\nAfter sending the request to the server, you will notice the value is altered to 5.\n\n![Image](https://res.cloudinary.com/dtwqtpteb/image/upload/v1665963225/bbxeqsfr7m7wkv1vh3cy.png\n)\n\nTry using curl:\n\n```\ncurl --location --request POST 'http://localhost:9000/api/feed/updateField' \\\n--header 'Content-Type: application/json' \\\n--header 'Cookie: connect.sid=s%3AwIcv2CWOo4MfACEOzmeyAcOHIOjz0JI-.NVuIKBNKBIAh1%2F7CVQD9d4khNV6pFOqZBSlfCdJGGK0' \\\n--data-raw '{\n    \"data\": {\n        \"rowId\": 1,\n        \"fieldToUpdate\": \"live\",\n        \"newValue\": \"5\"\n    }\n}''\n```\n\n\nAll data fields should be supplied. Otherwise, the server returns an error.\n![Image](https://res.cloudinary.com/dtwqtpteb/image/upload/v1665963344/l8eezirjrzrpuxhtxfrx.png\n)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaia14%2Fdata-feed","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaia14%2Fdata-feed","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaia14%2Fdata-feed/lists"}