{"id":21496950,"url":"https://github.com/financial-times/ft-poller","last_synced_at":"2025-10-04T08:47:13.926Z","repository":{"id":19108191,"uuid":"22336832","full_name":"Financial-Times/ft-poller","owner":"Financial-Times","description":"Scheduled, asynchronous JSON fetching for Node.js applications","archived":false,"fork":false,"pushed_at":"2024-12-03T16:17:24.000Z","size":959,"stargazers_count":12,"open_issues_count":3,"forks_count":5,"subscribers_count":93,"default_branch":"main","last_synced_at":"2024-12-03T17:29:09.208Z","etag":null,"topics":["component","customer-products","platforms-customer-products"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Financial-Times.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-07-28T09:06:00.000Z","updated_at":"2024-12-03T16:18:04.000Z","dependencies_parsed_at":"2023-12-21T02:20:09.674Z","dependency_job_id":"ca8075e6-07fd-4e2f-9a53-450549c0b93c","html_url":"https://github.com/Financial-Times/ft-poller","commit_stats":{"total_commits":146,"total_committers":27,"mean_commits":5.407407407407407,"dds":0.815068493150685,"last_synced_commit":"75a9cd236e3bad2373749f9858a763e50ac460b3"},"previous_names":[],"tags_count":53,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Financial-Times%2Fft-poller","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Financial-Times%2Fft-poller/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Financial-Times%2Fft-poller/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Financial-Times%2Fft-poller/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Financial-Times","download_url":"https://codeload.github.com/Financial-Times/ft-poller/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230382858,"owners_count":18216854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["component","customer-products","platforms-customer-products"],"created_at":"2024-11-23T16:19:59.961Z","updated_at":"2025-10-04T08:47:08.881Z","avatar_url":"https://github.com/Financial-Times.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\nScheduled, asynchronous JSON fetching for Node.js applications. [Background](#background);\n\n### installation\n\n```\nnpm install ft-poller\n```\n\n### API\nYou can create an instance of Poller like so,\n\n```\nvar Poller = require('ft-poller');\nvar p = new Poller(config);\n```\n\nWhere `config` is an object with the following properties\n\n* `url` [required]: Url to fetch data from\n* `defaultData` [recommended]: Data to return if the poller is yet to make a successful request. Typically this will be an empty object of the same type/structure as a successful response e.g. if a successful response would give you an array of users then set `defaultData: []`\n* `options` [optional]: options object to pass to `fetch`. If options is not defined or doesn't contain a `signal` property, the request timeout will be set to 4000ms by default. If `retry` is specified then the fetch will be retried the given number of times before erroring\n* `refreshInterval` [default: 60000]: Number of milliseconds to wait between request for data\n* `autostart` [default: false]: Whether to start the poller automatically when the instance is created\n* `parseData` [optional]: function to post-process the data returned by the request. Should return the post-processed data e.g\n```\nparseData: function (data) {\n    return data.rows;\n}\n```\n\n`parseData` can be any function you like and there's nothing to stop you using it to mutate any other variables in scope.\n\n\n#### methods\n\n* `start()` - Starts the poller. If passed an object `{ initialRequest: true }` it will send its first request immediately, otherwise it will wait until `config.refreshInterval` milliseconds. Returns a promise for the result of the first request (if `initialRequest` is true), or an empty resolved Promise otherwise\n\n* `stop()` - Stops polling\n\n* `getData()` - Returns the last set of data retrieved from the server (post-processed if `parseData` function exists). This will throw an `HttpError` if the most recent fetch received an error.\n\n#### Events\n\n* `error` - emits an error whenever a request returns with an error\n* `ok` - emits the response whenever a request returns successfully\n\n\n### Background\n\nThe classic request cycle for a web application follows a call from a client\nto the server, which in turn makes one or more further requests to some\nunderlying service(s).\n\n                                    +---\u003e Web service 1 --\u003e Data\n                                    |\n    Client ---\u003e Presentation tier --|---\u003e Web service 2 --\u003e Data\n                                    |\n                                    +---\u003e Web service 3 --\u003e Data\n\nOnce the data has been retrieved the response makes its way back through the\nvarious layers to the client.\n\nThis causes two problems.\n\nFirstly, your response is dependent on the slowest service to respond. If every\nrequest is hanging around waiting for 'the slow one' your performance is pegged\nto the worst performing part of your application.\n\nSecondly, by far the slowest thing in this type of architecture is the\nround-trip between the presentation tier and the service(s). The more of these\nopen connections you have hanging around, waiting to close, the greater the\nburden you place on your server.\n\n#### Async\n\nOften though, and this is especially true of News sites, the data doesn't\nchange radically from second to second so this round trip is wasted effort.\n\nIt's much more efficient for each presentation tier server to periodically\nfetch the data it needs (or listen for a message to signal when new content is\navailable), stash it in memory, then use that to service any incoming requests.\n\nThis suits a [microservice\narchitecture](http://martinfowler.com/articles/microservices.html), where many\ndiscrete modules, APIs etc. need to be assembled by a presentation tier before\nbeing rendered out to the client (as HTML, JSON etc.).\n\nThis pattern (of asynchronous fetching) allows the presentation tier to focus on\nbuilding a response from existing data (in memory) and sending it back out the\nfront door as quickly as possible.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffinancial-times%2Fft-poller","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffinancial-times%2Fft-poller","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffinancial-times%2Fft-poller/lists"}