{"id":24701724,"url":"https://github.com/hurricanemark/pagination","last_synced_at":"2026-04-09T19:57:13.160Z","repository":{"id":43138411,"uuid":"511085888","full_name":"hurricanemark/Pagination","owner":"hurricanemark","description":"A backend server --Given a structured data source, create a middleware function generalized enough to call on with various data models and returning a diced and sliced data layout with { previous, next, results }  of whole or subset of the passed-in data object along with additional calculated statistics such as average, total, median, etc..","archived":false,"fork":false,"pushed_at":"2022-07-07T15:15:21.000Z","size":85,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-22T04:13:03.443Z","etag":null,"topics":["dotenv","express","middleware","mongodb","mongodb-compass","noidejs","pagination","rest-api"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hurricanemark.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-07-06T10:11:00.000Z","updated_at":"2022-07-07T05:59:55.000Z","dependencies_parsed_at":"2022-09-01T08:41:21.451Z","dependency_job_id":null,"html_url":"https://github.com/hurricanemark/Pagination","commit_stats":null,"previous_names":[],"tags_count":0,"template":true,"template_full_name":null,"purl":"pkg:github/hurricanemark/Pagination","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hurricanemark%2FPagination","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hurricanemark%2FPagination/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hurricanemark%2FPagination/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hurricanemark%2FPagination/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hurricanemark","download_url":"https://codeload.github.com/hurricanemark/Pagination/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hurricanemark%2FPagination/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264619378,"owners_count":23638445,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dotenv","express","middleware","mongodb","mongodb-compass","noidejs","pagination","rest-api"],"created_at":"2025-01-27T05:25:16.825Z","updated_at":"2025-12-30T19:54:08.621Z","avatar_url":"https://github.com/hurricanemark.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Pagination\n\nGiven a structured data source in the form of a json array, we can write a middleware function to dice and slice it and return a whole or subset of the same array base on the requested page number and amount of records.  \n\nWe can also include processing statistics such as average, total records, etc., in the returned JSON object.  Since the paginated function is a generalized function, processing statistics will be hit and miss.  Missing elements will yield *null* which is perfectly okay.  Ah, this is encroaching upon data minning isn't it!\n\n### Steps to Glue it Together\n\n1. Start a nodeJS project from scratch\n\n`npm init -y`\n\n`npm install express dotenv`\n\n`npm install --save-dev nodemon`\n\n2. Edit package.json file to specify how the project runs.\n\n```\n\"script\" = {\n    \"dev\": \"nodemon server.js\",\n    \"start\": \"node server.sj\"\n}\n```\n\n3. Create file *.env* and specify environment variables that can not be exposed at runtime.  e.g. PORT, NODE_ENV, SECRET, etc.\n\n* Generate random bytes string for use as SECRET\n\n```\nPS D:\\DEVEL\\NODEJS\\\u003e node -e \"console.log(require('crypto').randomBytes(32).toString('hex'))\"\nd7fce4f57c65b2d7617f9ed4600d4f8f4bce7bddc954a73330560a1d8bf32a93\n```\n\n\u003cbr\u003e   \n\n* Sample content of .env file.  If you elect to use MongoAtlas, then MONGO_URI should be defined in .env \n\n```\nPORT=1975\nNODE_ENV=development\nSECRET=d7fce4f57c65b2d7617f9ed4600d4f8f4bce7bddc954a73330560a1d8bf32a93\n```\n\n\u003cbr\u003e   \n\n4.  Create file *server.js* and put up the minimum neccessary server code.\n\n```\nif (process.env.NODE_ENV !== 'production') {\n    require('dotenv').config();\n}\nconst PORT = process.env.PORT || 3000;\nconst express = require('express');\nconst app = express();\n\napp.get('/', (req, res) =\u003e {\n    res.send(\"Oy, what's cooking?\");\n});\n\napp.listen(PORT, () =\u003e {console.log(\"Listening on port %d...\", PORT)});\n```\n\n5. Test that node project is ready for development.\n\n`npm run dev`\n\n\u003cbr\u003e   \n\n\u003cstrong\u003eLet's build the middleware!\u003c/strong\u003e\n\n1. \u003cstrong\u003eA small crossroad:\u003c/strong\u003e  Choosing between local MongoDB or cloud hosted MongoAtlas\n\n-  Option 1: By setting up a local mongodb server, you can be free of space/transaction limitation\n\n-  Option 2: By using the hosted MongoAtlas, you are constrained by the limit of one cluster by which storage and transactions are monitored.  Additional charges incurred once you surpassed the upper limits.\n    * Create an account with MongoAtlas\n    * Copy the URI token into *.env* file\n\nAlas, we choose option 1 by setting up a local db using the [MongoDB Community Server](https://fastdl.mongodb.org/windows/mongodb-windows-x86_64-5.0.9-signed.msi) for windows.\n\nEither option above should provide you with the local agent *MongoDB Compass* which displays connection and collections (ie. tables).\n\n\u003cbr\u003e  \n\n2. Create data schemas (mongoose.Schema) in file *./data.js* and export the models.  We want two collections: pagination.Users and pagination.Employees.  Your code should look close to the following:\n\n```\nconst mongoose = require('mongoose');\n\nconst userDataSchema = new mongoose.Schema({\n    name: { type: String, required: true },\n    email: { type: String, required: true }\n});\nconst userModel = mongoose.model('Users', userDataSchema);\n\n\nconst employeeDataSchema = new mongoose.Schema({\n    name: { type: String, required: true },\n    age: { type: Number, required: true },\n    role: { type: String, required: true },\n    hobbies: { type: [String], required: true }\n});\nconst employeeModel = mongoose.model('Employees', employeeDataSchema);\n\nmodule.exports = {\n    usermodel: userModel,\n    employeemodel: employeeModel\n}\n```\n\n\n3. In file *server.js*, implement db connection and populate with data for testing. \n\n```\n/* instantiate collection models */\nconst Users = require('./data.js').usermodel;        // model in db schema to be used as paramter to middleware function paginatedArrayOfObjects()\nconst Employees = require('./data.js').employeemodel;     // another model in db schema to be used as paramter to middleware function paginatedArrayOfObjects()\n\n...\n\n/* connect to db server */\nconst uri = 'mongodb://localhost/pagination';\nmongoose.connect(uri, {useNewUrlParser: true, useUnifiedTopology: true})\nconst localMongooseDB = mongoose.connection // get the connection\n\n/* populate data once */\nlocalMongooseDB.once('open', async () =\u003e {\n    if (await Users.countDocuments().exec() \u003e 0) return\n\n    // populate dabase table with Employees data\n    Promise.all([\n        Employees.create({ name: 'Mark', age: 30, role: 'Developer', hobbies: ['Coding', 'Gaming']}),\n        Employees.create({ name: 'Emily', age: 25, role: 'Designer', hobbies: ['Drawing', 'Singing']}),\n        Employees.create({ name: 'Roland', age: 35, role: 'Developer', hobbies: ['Hunting', 'Fishing']}),\n        Employees.create({ name: 'Carol', age: 40, role: 'HR', hobbies: ['Reading', 'Swimming'] }),\n\n        Users.create({ name: 'John', email: 'john@mail.com' }),\n        Users.create({ name: 'Jane', email: 'jane@mail.com' }),\n        Users.create({ name: 'Bob', email: 'bob@mail.com' }),\n        Users.create({ name: 'Mary', email: 'mary@mail.com' }),\n        Users.create({ name: 'Tom', email: 'tom@mail.com' }),\n        Users.create({ name: 'Jack', email: 'jack@mail.com' }),\n        Users.create({ name: 'Jill', email: 'jill@mail.com' }),\n        Users.create({ name: 'Bill', email: 'bill@email.com' })\n    ])\n})\n```\n\nAt the start of server.js, two collections will be populated to batabase named 'pagination'.  You can views them with *MongoDB Compass*.\n\n![pagination data](./public/MongoDB_Compass.PNG)\n\n\n\n\n\n4. Setup Test Driven Development in VSCode by creating file *request.rest*\nThis file makes use of REST Client extension and will grow with new test cases as you code along.  Here is a sample:\n\n\n```\n@baseUrl = http://localhost:1975\n\n\n\n### Test landing page ###\n# @name = LandingPage\nGET {{baseUrl}} \nContent-Type: application/x-www-form-urlencoded\n\n###====== TEST Pagination for users data ======###\n### return the users array.  We don't know how many users there are, so we just return the first 100 users ###\n# @name = UpTo100Records\nGET {{baseUrl}}/users?page=1\u0026limit=100 HTTP/1.1\n\n\n### return the first user.  set page=1 and limit=1.  Notice that the previous page = null\n# @name = FirstUser\nGET {{baseUrl}}/users?page=1\u0026limit=1\u0026order=asc HTTP/1.1\n\n### return the second user.  Set page=2 and limit=1\n# @name = SecondUser\nGET {{baseUrl}}/users?page=2\u0026limit=1 HTTP/1.1\n\n\n### return the last user.  Set page=8 and limit=1.  Notice that next page = null\n# @name = LastUser\nGET {{baseUrl}}/users?page=8\u0026limit=1\u0026order=desc HTTP/1.1\n```\n\n\u003cbr\u003e  \n\n\n### Programming the pagination middleware\n\nThe purpose of this is to make the middleware function generalized enough that collection of various models could be injected.  We created two data schemas above to demonstrate this functionality.\n\nGiven a data model, function paginatedArrayOfObjects(model) returns a diced and sliced (paginated) result in *res.paginatedResult*\n\n```\n/*\n Generalized middleware function that paginates through an array of objects.\n If the page number is not provided, it will default to 1.\n If the limit is not provided, it will default to 10.\n Return keys might be null if the array elemenets do not have the same keys.\n for example, retObj.averageAge might be null if the array elements do not have the same age property.\n*/\nfunction paginatedArrayOfObjects(model) {\n    return async (req, res, next) =\u003e {\n        const page = parseInt(req.query.page) || 1;\n        const limit = parseInt(req.query.limit) || 10;\n\n        const startIndex = (page -1) * limit;\n        const endIndex = page * limit;\n\n        const retObj = {}\n        try {\n            if (endIndex \u003c await model.countDocuments().exec()) {\n                retObj.next = {\n                    page: page + 1,\n                    limit: limit\n                }\n            } else {\n                retObj.next = null;\n            }\n        } catch (err) {\n            console.log(err);\n        }\n\n        if (startIndex \u003e 0) {\n            retObj.previous = {\n                page: page - 1,\n                limit: limit\n            }\n        } else {\n            retObj.previous = null;\n        }\n\n        try {\n            retObj.results = await model.find().limit(limit).skip(startIndex).exec();\n            \n            /* Additional statistic: total records */\n            retObj.total =  retObj.results.length;\n\n            /* Additional statistic: averageAge */\n            retObj.averageAge = retObj.results.reduce((acc, curr) =\u003e {\n                return acc + curr.age;\n            } , 0) / retObj.results.length;\n\n            res.paginatedResults = retObj;  // Attach the paginated results to the response object (res.paginatedResults) to be returned.\n            next();\n\n        } catch (err) {\n            res.status(500).json({ message: err.message });\n        }\n    }\n}\n\n```\n\n\u003cbr\u003e  \n\n\n\u003cstrong\u003eInject the function into API methods as middleware\u003c/strong\u003e\n\n```\napp.get('/users', paginatedArrayOfObjects(Users), (req, res) =\u003e {\n    res.json(res.paginatedResults);\n});\n\n\napp.get('/employees', paginatedArrayOfObjects(Employees), (req, res) =\u003e {\n    res.json(res.paginatedResults);\n});\n```\n\n\u003cbr\u003e  \n\n#### Tests\n\n\u003cstrong\u003eQuery:\u003c/strong\u003e\n\n```\n@baseUrl = http://localhost:1975\n\n### get first 4 employees, starting from the second employee ###\n# @name = UpTo100Records\nGET {{baseUrl}}/employees?page=2\u0026limit=4 HTTP/1.1\n```\n\n\u003cbr\u003e   \n\n\u003cstrong\u003eOutput\u003c/strong\u003e\n\n```\nHTTP/1.1 200 OK\nX-Powered-By: Express\nContent-Type: application/json; charset=utf-8\nContent-Length: 600\nETag: W/\"258-L/WP6BAlmR45NQwftLqa7mrFGKU\"\nDate: Wed, 06 Jul 2022 09:21:19 GMT\nConnection: close\n\n{\n  \"next\": {\n    \"page\": 3,\n    \"limit\": 4\n  },\n  \"previous\": {\n    \"page\": 1,\n    \"limit\": 4\n  },\n  \"results\": [\n    {\n      \"_id\": \"62c54bc696b2d22060ae258b\",\n      \"name\": \"Kyle\",\n      \"age\": 45,\n      \"role\": \"Staff Engineer\",\n      \"hobbies\": [\n        \"Travel\",\n        \"Youtubing\"\n      ],\n      \"__v\": 0\n    },\n    {\n      \"_id\": \"62c54bc696b2d22060ae258c\",\n      \"name\": \"Jack\",\n      \"age\": 50,\n      \"role\": \"Maintenance\",\n      \"hobbies\": [\n        \"Jogging\",\n        \"Photography\"\n      ],\n      \"__v\": 0\n    },\n    {\n      \"_id\": \"62c54bc696b2d22060ae258d\",\n      \"name\": \"Tim\",\n      \"age\": 55,\n      \"role\": \"CTO\",\n      \"hobbies\": [\n        \"Coding\",\n        \"Sleeping\",\n        \"Eating\"\n      ],\n      \"__v\": 0\n    },\n    {\n      \"_id\": \"62c54bc696b2d22060ae258f\",\n      \"name\": \"Samuel\",\n      \"age\": 65,\n      \"role\": \"Marketing\",\n      \"hobbies\": [\n        \"Volunteering\",\n        \"Callecting Rocks\"\n      ],\n      \"__v\": 0\n    }\n  ],\n  \"total\": 4,\n  \"averageAge\": 53.75\n}\n```\n\n\u003cbr\u003e  \n\n## Conclusion\n\nPagination is used in many popular websites to display similar data in sequential manner.  Youtube is a good example. Google search engine has numbering pages at the bottom.  Picture thumbnail is a pagination.  Carousel cards in web frame is pagination.  \n\nIt no doubt enables better user experience and is one of the best navigation methods for human GUI. When you take time to step through the neccessary pieces of the purzle, your skills get stacked with another layer, the pagination layer. ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhurricanemark%2Fpagination","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhurricanemark%2Fpagination","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhurricanemark%2Fpagination/lists"}