{"id":13519126,"url":"https://github.com/yahoo/fetchr","last_synced_at":"2025-04-12T01:54:11.191Z","repository":{"id":18017267,"uuid":"21044474","full_name":"yahoo/fetchr","owner":"yahoo","description":"Universal data access layer for web applications.","archived":false,"fork":false,"pushed_at":"2025-04-01T13:26:54.000Z","size":1570,"stargazers_count":456,"open_issues_count":15,"forks_count":86,"subscribers_count":25,"default_branch":"main","last_synced_at":"2025-04-12T01:54:04.614Z","etag":null,"topics":["api","data-access","javascript","web"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yahoo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-06-20T16:22:53.000Z","updated_at":"2025-04-06T16:08:58.000Z","dependencies_parsed_at":"2023-09-25T22:57:47.138Z","dependency_job_id":"286adea6-9fbc-4e0d-82f3-4e470d195de1","html_url":"https://github.com/yahoo/fetchr","commit_stats":{"total_commits":547,"total_committers":47,"mean_commits":"11.638297872340425","dds":0.623400365630713,"last_synced_commit":"eafff6d32e8c40e5ba190da4de0ff9529b895319"},"previous_names":[],"tags_count":94,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yahoo%2Ffetchr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yahoo%2Ffetchr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yahoo%2Ffetchr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yahoo%2Ffetchr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yahoo","download_url":"https://codeload.github.com/yahoo/fetchr/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248505873,"owners_count":21115354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","data-access","javascript","web"],"created_at":"2024-08-01T05:01:54.297Z","updated_at":"2025-04-12T01:54:11.169Z","avatar_url":"https://github.com/yahoo.png","language":"JavaScript","readme":"# Fetchr\n\n[![npm version](https://badge.fury.io/js/fetchr.svg)](http://badge.fury.io/js/fetchr)\n![Build Status](https://github.com/yahoo/fetchr/actions/workflows/node.js.yml/badge.svg)\n[![Coverage Status](https://coveralls.io/repos/yahoo/fetchr/badge.png?branch=master)](https://coveralls.io/r/yahoo/fetchr?branch=master)\n\nUniversal data access layer for web applications.\n\nTypically on the server, you call your API or database directly to fetch some data. However, on the client, you cannot always call your services in the same way (i.e, cross domain policies). Instead, XHR/fetch requests need to be made to the server which get forwarded to your service.\n\nHaving to write code differently for both environments is duplicative and error prone. Fetchr provides an abstraction layer over your data service calls so that you can fetch data using the same API on the server and client side.\n\n## Install\n\n```bash\nnpm install fetchr --save\n```\n\n_Important:_ when on browser, `Fetchr` relies fully on [`Fetch`](https://fetch.spec.whatwg.org/) API. If you need to support old browsers, you will need to install a polyfill as well (eg. https://github.com/github/fetch).\n\n## Setup\n\nFollow the steps below to setup Fetchr properly. This assumes you are using the [Express](https://www.npmjs.com/package/express) framework.\n\n### 1. Configure Server\n\nOn the server side, add the Fetchr middleware into your express app at a custom API endpoint.\n\nFetchr middleware expects that you're using the [`body-parser`](https://github.com/expressjs/body-parser) middleware (or an alternative middleware that populates `req.body`) before you use Fetchr middleware.\n\n```js\nimport express from 'express';\nimport Fetcher from 'fetchr';\nimport bodyParser from 'body-parser';\nconst app = express();\n\n// you need to use body-parser middleware before fetcher middleware\napp.use(bodyParser.json());\n\napp.use('/myCustomAPIEndpoint', Fetcher.middleware());\n```\n\n### 2. Configure Client\n\nOn the client side, it is necessary for the `xhrPath` option to match the path where the middleware was mounted in the previous step\n\n`xhrPath` is an optional config property that allows you to customize the endpoint to your services, defaults to `/api`.\n\n```js\nimport Fetcher from 'fetchr';\nconst fetcher = new Fetcher({\n    xhrPath: '/myCustomAPIEndpoint',\n});\n```\n\n### 3. Register data services\n\nYou will need to register any data services that you wish to use in\nyour application. The interface for your service will be an object\nthat must define a `resource` property and at least one\n[CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete)\noperation. The `resource` property will be used when you call one of the\nCRUD operations.\n\n```js\n// app.js\nimport Fetcher from 'fetchr';\nimport myDataService from './dataService';\nFetcher.registerService(myDataService);\n```\n\n```js\n// dataService.js\nexport default {\n    // resource is required\n    resource: 'data_service',\n    // at least one of the CRUD methods is required\n    read: async function ({ req, resource, params, config }) {\n        return { data: 'foo' };\n    },\n    // other methods\n    // create: async function({ req, resource, params, body, config }) {},\n    // update: async function({ req, resource, params, body, config }) {},\n    // delete: async function({ req, resource, params, config }) {}\n};\n```\n\n### 4. Instantiating the Fetchr Class\n\nData services might need access to each individual request, for example, to get the current logged in user's session.\nFor this reason, Fetcher will have to be instantiated once per request.\n\nOn the serverside, this requires fetcher to be instantiated per request, in express middleware.\nOn the clientside, this only needs to happen on page load.\n\n```js\n// app.js - server\nimport express from 'express';\nimport Fetcher from 'fetchr';\nimport myDataService from './dataService';\nconst app = express();\n\n// register the service\nFetcher.registerService(myDataService);\n\n// register the middleware\napp.use('/myCustomAPIEndpoint', Fetcher.middleware());\n\napp.use(function (req, res, next) {\n    // instantiated fetcher with access to req object\n    const fetcher = new Fetcher({\n        xhrPath: '/myCustomAPIEndpoint', // xhrPath will be ignored on the serverside fetcher instantiation\n        req: req,\n    });\n\n    // perform read call to get data\n    fetcher\n        .read('data_service')\n        .params({ id: 42 })\n        .then(({ data, meta }) =\u003e {\n            // handle data returned from data fetcher in this callback\n        })\n        .catch((err) =\u003e {\n            // handle error\n        });\n});\n```\n\n```js\n// app.js - client\nimport Fetcher from 'fetchr';\nconst fetcher = new Fetcher({\n    xhrPath: '/myCustomAPIEndpoint', // xhrPath is REQUIRED on the clientside fetcher instantiation\n});\nfetcher\n    .read('data_api_fetcher')\n    .params({ id: 42 })\n    .then(({ data, meta }) =\u003e {\n        // handle data returned from data fetcher in this callback\n    })\n    .catch((err) =\u003e {\n        // handle errors\n    });\n\n// for create you can use the body() method to pass data\nfetcher\n    .create('data_api_create')\n    .body({ some: 'data' })\n    .then(({ data, meta }) =\u003e {\n        // handle data returned from data fetcher in this callback\n    })\n    .catch((err) =\u003e {\n        // handle errors\n    });\n```\n\n## Usage Examples\n\nSee the [simple example](https://github.com/yahoo/fetchr/tree/master/examples/simple).\n\n## Service Metadata\n\nService calls on the client transparently become fetch requests.\nIt is a good idea to set cache headers on common fetch calls.\nYou can do so by providing a third parameter in your service's callback.\nIf you want to look at what headers were set by the service you just called,\nsimply inspect the third parameter in the callback.\n\nNote: If you're using promises, the metadata will be available on the `meta`\nproperty of the resolved value.\n\n```js\n// dataService.js\nexport default {\n    resource: 'data_service',\n    read: async function ({ req, resource, params, config }) {\n        return {\n            data: 'response', // business logic\n            meta: {\n                headers: {\n                    'cache-control': 'public, max-age=3600',\n                },\n                statusCode: 200, // You can even provide a custom statusCode for the fetch response\n            },\n        };\n    },\n};\n```\n\n```js\nfetcher\n    .read('data_service')\n    .params({id: ###})\n    .then(({ data, meta }) {\n        // data will be 'response'\n        // meta will have the header and statusCode from above\n    });\n```\n\nThere is a convenience method called `fetcher.getServiceMeta` on the fetchr instance.\nThis method will return the metadata for all the calls that have happened so far\nin an array format.\nIn the server, this will include all service calls for the current request.\nIn the client, this will include all service calls for the current session.\n\n## Updating Configuration\n\nUsually you instantiate fetcher with some default options for the entire browser session,\nbut there might be cases where you want to update these options later in the same session.\n\nYou can do that with the `updateOptions` method:\n\n```js\n// Start\nconst fetcher = new Fetcher({\n    xhrPath: '/myCustomAPIEndpoint',\n    xhrTimeout: 2000,\n});\n\n// Later, you may want to update the xhrTimeout\nfetcher.updateOptions({\n    xhrTimeout: 4000,\n});\n```\n\n## Error Handling\n\nWhen an error occurs in your Fetchr CRUD method, you should throw an error object. The error object should contain a `statusCode` (default 500) and `output` property that contains a JSON serializable object which will be sent to the client.\n\n```js\nexport default {\n    resource: 'FooService',\n    read: async function create(req, resource, params, configs) {\n        const err = new Error('it failed');\n        err.statusCode = 404;\n        err.output = { message: 'Not found', more: 'meta data' };\n        err.meta = { foo: 'bar' };\n        throw err;\n    },\n};\n```\n\nAnd in your service call:\n\n```js\nfetcher\n    .read('someData')\n    .params({ id: '42' })\n    .catch((err) =\u003e {\n        // err instanceof FetchrError -\u003e true\n        // err.message -\u003e \"Not found\"\n        // err.meta -\u003e { foo: 'bar' }\n        // err.name = 'FetchrError'\n        // err.output -\u003e { message: \"Not found\", more: \"meta data\" }\n        // err.rawRequest -\u003e { headers: {}, method: 'GET', url: '/api/someData' }\n        // err.reason -\u003e BAD_HTTP_STATUS | BAD_JSON | TIMEOUT | ABORT | UNKNOWN\n        // err.statusCode -\u003e 404\n        // err.timeout -\u003e 3000\n        // err.url -\u003e '/api/someData'\n    });\n```\n\n## Abort support\n\nAn object with an `abort` method is returned when creating fetchr requests on client.\nThis is useful if you want to abort a request before it is completed.\n\n```js\nconst req = fetcher\n    .read('someData')\n    .params({ id: 42 })\n    .catch((err) =\u003e {\n        // err.reason will be ABORT\n    });\n\nreq.abort();\n```\n\n## Timeouts\n\n`xhrTimeout` is an optional config property that allows you to set timeout (in ms) for all clientside requests, defaults to `3000`.\nOn the clientside, xhrPath and xhrTimeout will be used for all requests.\nOn the serverside, xhrPath and xhrTimeout are not needed and are ignored.\n\n```js\nimport Fetcher from 'fetchr';\nconst fetcher = new Fetcher({\n    xhrPath: '/myCustomAPIEndpoint',\n    xhrTimeout: 4000,\n});\n```\n\nIf you want to set a timeout per request you can call `clientConfig` with a `timeout` property:\n\n```js\nfetcher\n    .read('someData')\n    .params({ id: 42 })\n    .clientConfig({ timeout: 5000 }) // wait 5 seconds for this request before timing out\n    .catch((err) =\u003e {\n        // err.reason will be TIMEOUT\n    });\n```\n\n## Params Processing\n\nFor some applications, there may be a situation where you need to process the service params passed in the request before they are sent to the actual service. Typically, you would process them in the service itself. However, if you need to perform processing across many services (i.e. sanitization for security), then you can use the `paramsProcessor` option.\n\n`paramsProcessor` is a function that is passed into the `Fetcher.middleware` method. It is passed three arguments, the request object, the serviceInfo object, and the service params object. The `paramsProcessor` function can then modify the service params if needed.\n\nHere is an example:\n\n```js\n/**\n    Using the app.js from above, you can modify the Fetcher.middleware\n    method to pass in the paramsProcessor function.\n */\napp.use(\n    '/myCustomAPIEndpoint',\n    Fetcher.middleware({\n        paramsProcessor: function (req, serviceInfo, params) {\n            console.log(serviceInfo.resource, serviceInfo.operation);\n            return Object.assign({ foo: 'fillDefaultValueForFoo' }, params);\n        },\n    }),\n);\n```\n\n## Response Formatting\n\nFor some applications, there may be a situation where you need to modify the response before it is passed to the client. Typically, you would apply your modifications in the service itself. However, if you need to modify the responses across many services (i.e. add debug information), then you can use the `responseFormatter` option.\n\n`responseFormatter` is a function that is passed into the `Fetcher.middleware` method. It is passed three arguments, the request object, response object and the service response object (i.e. the data returned from your service). The `responseFormatter` function can then modify the service response to add additional information.\n\nTake a look at the example below:\n\n```js\n/**\n    Using the app.js from above, you can modify the Fetcher.middleware\n    method to pass in the responseFormatter function.\n */\napp.use(\n    '/myCustomAPIEndpoint',\n    Fetcher.middleware({\n        responseFormatter: function (req, res, data) {\n            data.debug = 'some debug information';\n            return data;\n        },\n    }),\n);\n```\n\nNow when an request is performed, your response will contain the `debug` property added above.\n\n## CORS Support\n\nFetchr provides [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS) support by allowing you to pass the full origin host into `corsPath` option.\n\nFor example:\n\n```js\nimport Fetcher from 'fetchr';\nconst fetcher = new Fetcher({\n    corsPath: 'http://www.foo.com',\n    xhrPath: '/fooProxy',\n});\nfetcher.read('service').params({ foo: 1 }).clientConfig({ cors: true });\n```\n\nAdditionally, you can also customize how the GET URL is constructed by passing in the `constructGetUri` property when you execute your `read` call:\n\n```js\nimport qs from 'qs';\nfunction customConstructGetUri(uri, resource, params, config) {\n    // this refers to the Fetcher object itself that this function is invoked with.\n    if (config.cors) {\n        return uri + '/' + resource + '?' + qs.stringify(this.context);\n    }\n    // Return `falsy` value will result in `fetcher` using its internal path construction instead.\n}\n\nimport Fetcher from 'fetchr';\nconst fetcher = new Fetcher({\n    corsPath: 'http://www.foo.com',\n    xhrPath: '/fooProxy',\n});\nfetcher.read('service').params({ foo: 1 }).clientConfig({\n    cors: true,\n    constructGetUri: customConstructGetUri,\n});\n```\n\n## CSRF Protection\n\nYou can protect your Fetchr middleware paths from CSRF attacks by adding a middleware in front of it:\n\n`app.use('/myCustomAPIEndpoint', csrf(), Fetcher.middleware());`\n\nYou could use https://github.com/expressjs/csurf for this as an example.\n\nNext you need to make sure that the CSRF token is being sent with our requests so that they can be validated. To do this, pass the token in as a key in the `options.context` object on the client:\n\n```js\nconst fetcher = new Fetcher({\n    xhrPath: '/myCustomAPIEndpoint', // xhrPath is REQUIRED on the clientside fetcher instantiation\n    context: {\n        // These context values are persisted with client calls as query params\n        _csrf: 'Ax89D94j',\n    },\n});\n```\n\nThis `_csrf` will be sent in all client requests as a query parameter so that it can be validated on the server.\n\n## Service Call Config\n\nWhen calling a Fetcher service you can pass an optional config object.\n\nWhen this call is made from the client, the config object is used to set some request options and can be used to override default options:\n\n```js\n//app.js - client\nconst config = {\n    timeout: 6000, // Timeout (in ms) for each request\n    unsafeAllowRetry: false, // for POST requests, whether to allow retrying this post\n};\n\nfetcher.read('service').params({ id: 1 }).clientConfig(config);\n```\n\nFor requests from the server, the config object is simply passed into the service being called.\n\n## Retry\n\nYou can set Fetchr to automatically retry failed requests by specifying a `retry` configuration in the global or in the request configuration:\n\n```js\n// Globally\nconst fetchr = new Fetchr({\n    retry: { maxRetries: 2 },\n});\n\n// Per request\nfetchr.read('service').clientConfig({\n    retry: { maxRetries: 1 },\n});\n```\n\nWith the above configuration, Fetchr will retry twice all requests\nthat fail but only once when calling `read('service')`.\n\nYou can further customize how the retry mechanism works. These are all\nsettings and their default values:\n\n```js\nconst fetchr = new Fetchr({\n  retry: {\n    maxRetries: 2, // amount of retries after the first failed request\n    interval: 200, // maximum interval between each request in ms (see note below)\n    statusCodes: [0, 408], // response status code that triggers a retry (see note below)\n  },\n  unsafeAllowRetry: false, // allow unsafe operations to be retried (see note below)\n}\n```\n\n**interval**\n\nThe interval between each request respects the following formula, based on the exponential backoff and full jitter strategy published in [this AWS architecture blog post](https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/):\n\n```js\nMath.random() * Math.pow(2, attempt) * interval;\n```\n\n`attempt` is the number of the current retry attempt starting\nfrom 0. By default `interval` corresponds to 200ms.\n\n**statusCodes**\n\nFor historical reasons, fetchr only retries 408 responses and no\nresponses at all (for example, a network error, indicated by a status\ncode 0). However, you might find useful to also retry on other codes\nas well (502, 503, 504 can be good candidates for an automatic\nretries).\n\n**unsafeAllowRetry**\n\nBy default, Fetchr only retries `read` requests. This is done for\nsafety reasons: reading twice an entry from a database is not as bad\nas creating an entry twice. But if your application or resource\ndoesn't need this kind of protection, you can allow retries by setting\n`unsafeAllowRetry` to `true` and fetchr will retry all operations.\n\n## Context Variables\n\nBy Default, fetchr appends all context values to the request url as query params. `contextPicker` allows you to have greater control over which context variables get sent as query params depending on the request method (`GET` or `POST`). This is useful when you want to limit the number of variables in a `GET` url in order not to accidentally [cache bust](http://webassets.readthedocs.org/en/latest/expiring.html).\n\n`contextPicker` follows the same format as the `predicate` parameter in [`lodash/pickBy`](https://lodash.com/docs#pickBy) with two arguments: `(value, key)`.\n\n```js\nconst fetcher = new Fetcher({\n    context: {\n        // These context values are persisted with client calls as query params\n        _csrf: 'Ax89D94j',\n        device: 'desktop',\n    },\n    contextPicker: {\n        GET: function (value, key) {\n            // for example, if you don't enable CSRF protection for GET, you are able to ignore it with the url\n            if (key === '_csrf') {\n                return false;\n            }\n            return true;\n        },\n        // for other method e.g., POST, if you don't define the picker, it will pick the entire context object\n    },\n});\n\nconst fetcher = new Fetcher({\n    context: {\n        // These context values are persisted with client calls as query params\n        _csrf: 'Ax89D94j',\n        device: 'desktop',\n    },\n    contextPicker: {\n        GET: ['device'], // predicate can be an array of strings\n    },\n});\n```\n\n## Custom Request Headers\n\nWhen calling a Fetcher service you can add custom request headers.\n\nA request contains custom headers when you add `headers` option to 'clientConfig'.\n\n```js\nconst config = {\n    headers: {\n        'X-VERSION': '1.0.0',\n    },\n};\n\nfetcher.read('service').params({ id: 1 }).clientConfig(config);\n```\n\nAll requests contain custom headers when you add `headers` option to constructor arguments of 'Fetcher'.\n\n```js\nimport Fetcher from 'fetchr';\nconst fetcher = new Fetcher({\n    headers: {\n        'X-VERSION': '1.0.0',\n    },\n});\n```\n\n## Stats Monitoring \u0026 Analysis\n\nTo collect fetcher service's success/failure/latency stats, you can configure `statsCollector` for `Fetchr`. The `statsCollector` function will be invoked with one argumment: `stats`. The `stats` object will contain the following fields:\n\n-   **resource:** The name of the resource for the request\n-   **operation:** The name of the operation, `create|read|update|delete`\n-   **params:** The params object for the resource\n-   **statusCode:** The status code of the response\n-   **err:** The error object of failed request; null if request was successful\n-   **time:** The time spent for this request, in milliseconds\n\n### Fetcher Instance\n\n```js\nimport Fetcher from 'fetchr';\nconst fetcher = new Fetcher({\n    xhrPath: '/myCustomAPIEndpoint',\n    statsCollector: function (stats) {\n        // just console logging as a naive example.  there is a lot more you can do here,\n        // like aggregating stats or filtering out stats you don't want to monitor\n        console.log(\n            'Request for resource',\n            stats.resource,\n            'with',\n            stats.operation,\n            'returned statusCode:',\n            stats.statusCode,\n            ' within',\n            stats.time,\n            'ms',\n        );\n    },\n});\n```\n\n### Server Middleware\n\n```js\napp.use(\n    '/myCustomAPIEndpoint',\n    Fetcher.middleware({\n        statsCollector: function (stats) {\n            // just console logging as a naive example.  there is a lot more you can do here,\n            // like aggregating stats or filtering out stats you don't want to monitor\n            console.log(\n                'Request for resource',\n                stats.resource,\n                'with',\n                stats.operation,\n                'returned statusCode:',\n                stats.statusCode,\n                ' within',\n                stats.time,\n                'ms',\n            );\n        },\n    }),\n);\n```\n\n## API\n\n-   [Fetchr](https://github.com/yahoo/fetchr/blob/master/docs/fetchr.md)\n\n## License\n\nThis software is free to use under the Yahoo! Inc. BSD license.\nSee the [LICENSE file][] for license text and copyright information.\n\n[license file]: https://github.com/yahoo/fetchr/blob/master/LICENSE.md\n","funding_links":[],"categories":["Awesome React","JavaScript"],"sub_categories":["Tools"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyahoo%2Ffetchr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyahoo%2Ffetchr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyahoo%2Ffetchr/lists"}