{"id":14975606,"url":"https://github.com/navpreetdevpuri/mongoose-sort-encrypted-field","last_synced_at":"2026-03-04T19:01:44.914Z","repository":{"id":143635002,"uuid":"615925624","full_name":"NavpreetDevpuri/mongoose-sort-encrypted-field","owner":"NavpreetDevpuri","description":"Mongoose plugin to enable sorting on encrypted fields","archived":false,"fork":false,"pushed_at":"2023-04-03T11:25:57.000Z","size":195,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-10T18:36:41.771Z","etag":null,"topics":["mongoose-plugin","order-preserving-encryption","sorting-algorithms"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NavpreetDevpuri.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-19T04:36:54.000Z","updated_at":"2023-03-24T10:39:12.000Z","dependencies_parsed_at":"2023-05-29T16:45:31.413Z","dependency_job_id":null,"html_url":"https://github.com/NavpreetDevpuri/mongoose-sort-encrypted-field","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NavpreetDevpuri%2Fmongoose-sort-encrypted-field","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NavpreetDevpuri%2Fmongoose-sort-encrypted-field/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NavpreetDevpuri%2Fmongoose-sort-encrypted-field/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NavpreetDevpuri%2Fmongoose-sort-encrypted-field/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NavpreetDevpuri","download_url":"https://codeload.github.com/NavpreetDevpuri/mongoose-sort-encrypted-field/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241061923,"owners_count":19902794,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mongoose-plugin","order-preserving-encryption","sorting-algorithms"],"created_at":"2024-09-24T13:52:16.583Z","updated_at":"2026-03-04T19:01:44.858Z","avatar_url":"https://github.com/NavpreetDevpuri.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mongoose-sort-encrypted-field\n\nMongoose plugin to enable sorting on encrypted fields\n\n# Install\n\n```\nnpm i mongoose-sort-encrypted-field\n```\n\n## Example\n\nWe are having a user with an encrypted email, We just need to add the `sortFieldName` option to that field\n\n```javascript\nconst { encrypt, decrypt } = require(\"./encryption.js\");\nconst { getModelWithSortEncryptedFieldsPlugin } = require(\"mongoose-sort-encrypted-field\");\n\nconst userSchema = new mongoose.Schema({\n  email: {\n    type: String,\n    required: true,\n    unique: true,\n    set: encrypt,\n    get: decrypt,\n    sortFieldName: \"emailSort\",\n  },\n});\n\nconst User = getModelWithSortEncryptedFieldsPlugin(\"User\", userSchema, {\n  redisQueueClientOptions: { redis: \"redis://localhost:6379\" },\n  ignoreCases: true,\n});\n\nmodule.exports = User;\n```\n\nThen we can sort all records by email from the 'emailSort' field. For performance, we can create a MongoDB index for that field.\n\n```javascript\nconst sortedUsers = await User.find({}).sort({ emailSort: 1 }).exec();\n```\n\nNote: For values equal to `null` or `undefined`, It consider those as `''` empty strings to support proper sorting if we do multiple fields sorting.\nExample:\n\n```javascript\n// by default mongodb/javascript sort those as follow\ndocuments = [\n  { firstName: \"a\", middleName: \"\", lastName: \"b\" },\n  { firstName: \"a\", middleName: \"b\", lastName: \"b\" },\n  { firstName: \"a\", middleName: \"b\", lastName: \"b\" },\n  { firstName: \"a\", middleName: null, lastName: \"b\" },\n  { firstName: \"a\", middleName: undefined, lastName: \"b\" },\n];\n\n// If we sort as { $sort: { firstName: 1, middleName: 1, lastName: 1 } }\n// For full name then it will be wrong according to default behaviour\n[\"aa\", \"abb\", \"ab\", \"ab\"];\n\n// So, in our plugin we are doing as follow\ndocuments = [\n  { firstName: \"a\", middleName: \"\", lastName: \"b\" },\n  { firstName: \"a\", middleName: null, lastName: \"b\" },\n  { firstName: \"a\", middleName: undefined, lastName: \"b\" },\n  { firstName: \"a\", middleName: \"b\", lastName: \"b\" },\n  { firstName: \"a\", middleName: \"b\", lastName: \"b\" },\n];\n// Now it is corrent\n[\"aa\", \"ab\", \"ab\", \"abb\"];\n```\n\n### pluginOptions:\n\n1. `redisQueueClientOptions: RedisQueueClientOptions;` default:\n\n   ```\n   {\n     redis: new Redis(), // It can be an instance of [ioredis](https://www.npmjs.com/package/ioredis) or any value that we can pass to ioredis constructor\n     batchSize: 10,\n     groupVisibilityTimeoutMs: 60000,\n     pollingTimeoutMs: 10000,\n     consumerCount: 3, // Better to have consumerCount in a balance of maximum fields we have to sort vs resources usage for multiple consumers\n     redisKeyPrefix: \"mongoose-sort-encrypted-field\",\n   }\n   ```\n\n   Any options which we can pass to [redis-ordered-queue](https://www.npmjs.com/package/redis-ordered-queue) constructor and redis options can be an instance of [ioredis](https://www.npmjs.com/package/ioredis) or any value that we can pass to ioredis constructor\n\n2. `noOfCharsForSortId?: number` default: `50`\n   Number of bytes for sort ID, bigger number is mathematically better.\n\n3. `noOfCharsToIncreaseOnSaturation?: number;` default: `2` \u003cbr\u003e\n   Number of bytes to increase on saturation, for example,\n   for `04` and `05`, first, we can see there is no whole number between those\n   so, It appends an extra digit at the end and it becomes `040` and `050` and the average is `045`.\n   In the base `2^15` number system, getting a saturation like that is mathematically very unlikely.\n\n4. `ignoreCases?: boolean;` default: `false` \u003cbr\u003e\n   To ignore cases.\n\n5. `silent?: boolean;` default: `false` \u003cbr\u003e\n   Flag to turn on/off console info logs\n\n6. `selectSortFields?: boolean;` default: `false` \u003cbr\u003e\n   Flag for `select` option for sort fields. If set `false` then the sort fields will be removed from any query result.\n\n7. `revaluateAllThreshold?: number;` default: `0.5` \u003cbr\u003e\n   If the number of documents without sort ID divided by the total number of documents is less than this threshold\n   Then it will get all values, sort them, and generate sort ID for all at equal distances 0 to 2^15\n   For example, if we have 3 documents and we can 00 to 20 sort ID\n   then those documents will have 05 10 15 sort ID\n\n8. `revaluateAllCountThreshold?: number;` default: `100` \u003cbr\u003e\n   If the total number of documents is less than this value\n   then it will regenerate the sort ID the same way as revaluateAllThreshold\n\n# How does it work?\n\nWe create a sort order ID which is just a number in base `2^15`, which is a huge number system as compared to the 10 base number system. We search in DB using binary search. For `1 lakh` documents, it queries and decrypts only `18` documents (first+last+log(1lakh)) to generate a sort ID. It generates a sort order ID in `O(1)`.\n\nTo generate a sort order ID it only needs to know the previous and next sort ID, and it just averages out those to get the current sort order ID, for example in the base 10 system if need to insert between `03` and `07` then `(03+07)/02` which is `05`. for `04` and `05`, first we can see there is no whole number between those so, It append extra digit at the end and it becomes `040` and `050` and the average is `045`. In the base `2^15` number system, getting a saturation like that is mathematically very unlikely.\n\nIt uses [redis-ordered-queue](https://www.npmjs.com/package/redis-ordered-queue) to generate a sort ID. It means it only processes one document at a time as per the mathematical requirement of the sort ID generation algorithm even when we are running multiple instances of our service.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnavpreetdevpuri%2Fmongoose-sort-encrypted-field","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnavpreetdevpuri%2Fmongoose-sort-encrypted-field","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnavpreetdevpuri%2Fmongoose-sort-encrypted-field/lists"}