{"id":25900860,"url":"https://github.com/pixelastic/algolia-indexing","last_synced_at":"2025-03-03T02:19:28.142Z","repository":{"id":48914072,"uuid":"137262085","full_name":"pixelastic/algolia-indexing","owner":"pixelastic","description":"Perform complex indexing operations with ease","archived":false,"fork":false,"pushed_at":"2024-10-06T14:48:41.000Z","size":3986,"stargazers_count":8,"open_issues_count":4,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-26T14:58:40.222Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pixelastic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-13T19:36:08.000Z","updated_at":"2025-01-03T14:23:21.000Z","dependencies_parsed_at":"2022-09-23T23:51:40.951Z","dependency_job_id":null,"html_url":"https://github.com/pixelastic/algolia-indexing","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pixelastic%2Falgolia-indexing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pixelastic%2Falgolia-indexing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pixelastic%2Falgolia-indexing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pixelastic%2Falgolia-indexing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pixelastic","download_url":"https://codeload.github.com/pixelastic/algolia-indexing/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241596320,"owners_count":19988057,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-03T02:19:27.529Z","updated_at":"2025-03-03T02:19:28.129Z","avatar_url":"https://github.com/pixelastic.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Algolia-indexing\n\nThis module abstract complex indexing operations under one simple command. Its\ngoal is to use as few operations as possible when updating an index, by\nperforming diff comparisons instead of a full delete/overwrite.\n\nThis package has been originally created by Algolia, but is no longer officially\nsupported.\n\n## Installation\n\nInstall through `yarn`:\n\n```shell\nyarn add algolia-indexing\n```\n\n## Usage\n\n```javascript\nimport indexing from 'algolia-indexing';\n\nconst credentials = { appId: 'XXX', apiKey: 'YYY', indexName: 'my_index' };\nconst records = [{ foo: 'bar' }];\nconst settings = { searchableAttributes: ['foo'] };\n\nawait indexing.fullAtomic(credentials, records, settings);\n```\n\nThis will update an index with new records and settings in an **atomic**\nway. It will be **fast** but will require a plan with a **large number of\nrecords** (as we'll need to duplicate the index for a short period of time).\n\nHow it works:\n\n- Set a unique objectID to each record (a hashed string of its content)\n- Copy the production index to a temporary one\n- Compare the new records and the existing records in the index\n- Patch the temporary index by removing old records and adding new ones\n- Overwrite production index with temporary one\n\n### `.verbose()`\n\nBy default, all methods are silent. By calling `indexing.verbose()`, you enable\nthe display of some progress indicators.\n\n![Example of a Full Atomic][1]\n\n## Events\n\nThe module `.pulse` key emits events at different points in time. You can listen\nto them and react accordingly. Each event is fired with an object containing\ndifferent information relative to the event that fired it.\n\n```javascript\nimport indexing from 'algolia-indexing';\n\nindexing.pulse.on('copyIndex.start', ({ source, destination }) =\u003e {\n  console.info(`Start moving ${source} to ${destination}`);\n});\nindexing.pulse.on('copyIndex.end', ({ source, destination }) =\u003e {\n  console.info(`Finished moving ${source} to ${destination}`);\n});\n```\n\nAll events have a specific key called `eventId` that is unique and shared across\nevents of the same origin. For example, a batch operation will emit\n`batch.start` when starting, `batch.end` when finished and a certain number of\n`batch.chunk` events depending on how large the batch is. All those events will\nshare the same `eventId`.\n\n| event                                              | attributes                                   |\n| -------------------------------------------------- | -------------------------------------------- |\n| `copyIndex.start`, `copyIndex.end`                 | `source`, `destination`                      |\n| `moveIndex.start`, `moveIndex.end`                 | `source`, `destination`                      |\n| `clearIndex.start`, `clearIndex.end`               | `indexName`                                  |\n| `setSettings.start`, `setSettings.end`             | `indexName`, `settings`                      |\n| `configureReplicas.start`, `configureReplicas.end` | `indexName`                                  |\n| `getAllRecords.start`, `getAllRecords.page`        | `indexName`, `currentPage`, `maxPages`       |\n| `getAllRecords.end`                                | `indexName`                                  |\n| `batch.start`, `batch.chunk`                       | `currentOperationCount`, `maxOperationCount` |\n| `batch.end`                                        |                                              |\n| `error`                                            | `message`                                    |\n\n## Config\n\n`algolia-indexing` has sensible default configuration, but allows you to turn\nknobs here and there.\n\nThe following table lists all the config keys and their default values. To\nchange a config value, you need to call `indexing.config({ keyToReplace: 'newValue' })`.\n\n| Config                | Default Value | Description                                        |\n| --------------------- | ------------: | -------------------------------------------------- |\n| `batchMaxSize`        |           100 | Number of operations to send in one batch at most. |\n| `batchMaxConcurrency` |            10 | Number of batches do we run in parallel            |\n\n## Replicas\n\nThe `settings.replicas` key can be used to define all the replicas and their\nconfiguration. They, too, will be atomically updated on a new indexing.\n\n```javascript\nconst credentials = {\n  // …\n  indexName: 'products',\n};\nconst records = [\n  // …\n];\nconst settings = {\n  // …\n  searchableAttributes: ['title'],\n  customRanking: ['desc(date)', 'desc(score)'],\n  replicas: {\n    popularity: {\n      customRanking: ['desc(score)', 'desc(date)'],\n    },\n  },\n};\nawait indexing.fullAtomic(credentials, records, settings);\n```\n\nThe above config will order results by date, then score on the main index, but\nwill also create a `products_popularity` replica ordered by score, then date.\n\n[1]: ./.github/full-atomic.gif\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpixelastic%2Falgolia-indexing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpixelastic%2Falgolia-indexing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpixelastic%2Falgolia-indexing/lists"}