{"id":15575711,"url":"https://github.com/hsyntes/data-modeling","last_synced_at":"2026-04-10T07:53:32.982Z","repository":{"id":194201115,"uuid":"690311017","full_name":"hsyntes/data-modeling","owner":"hsyntes","description":"A Backend application that provides Advanced Data Modeling and Schema Design with MongoDB, mongoose in Node.js \u0026 Express","archived":false,"fork":false,"pushed_at":"2023-09-12T18:02:39.000Z","size":44,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-03T20:50:39.145Z","etag":null,"topics":["data","database","datamodeling","express","modeling","mongodb","mongoose","nodejs","schema"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hsyntes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-12T00:35:26.000Z","updated_at":"2024-08-06T19:22:55.000Z","dependencies_parsed_at":"2024-12-14T15:15:27.547Z","dependency_job_id":null,"html_url":"https://github.com/hsyntes/data-modeling","commit_stats":{"total_commits":14,"total_committers":1,"mean_commits":14.0,"dds":0.0,"last_synced_commit":"53f5ec0fc5fa6ea835d91e73dae2f56be5399c57"},"previous_names":["hsyntes/data-modeling"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hsyntes%2Fdata-modeling","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hsyntes%2Fdata-modeling/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hsyntes%2Fdata-modeling/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hsyntes%2Fdata-modeling/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hsyntes","download_url":"https://codeload.github.com/hsyntes/data-modeling/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246151920,"owners_count":20731700,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","database","datamodeling","express","modeling","mongodb","mongoose","nodejs","schema"],"created_at":"2024-10-02T18:40:26.459Z","updated_at":"2026-04-10T07:53:32.955Z","avatar_url":"https://github.com/hsyntes.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Data Modeling \u0026 Schema Design\n\n**Data Modeling** allows how to relate data to other data in the same or different collection such as **embedded** or **referenced** while **Schema Design** allows determmine how data will be **organized**, **validated** and **structured**. It's very similiar to Classes in **Object Oriented Programming**.\n\nBoth Data Modeling and Schema Design are crucial in the development of databases and plays a significant role in ensuring data **integrity**, **efficiency**, and **usability**.\n\n## Schema Design\n\nmongoose npm package runs top of MongoDB like a framework and it ensures to implement both **data modeling** \u0026 **schema design**.\n\n```javascript\n// User Schema Design\nconst userSchema = new mongoose.Schema({\n  username: {\n    type: String,\n    minlength: [3, \"@username cannot be shorter than 3 characters.\"],\n    required: [true, \"@username is required.\"],\n    unique: true,\n    trim: true,\n  },\n\n  // ...\n\n  password: {\n    type: String,\n    minlength: [8, \"Password cannot be shorter than 8 characters.\"],\n    maxlength: [32, \"Password cannot be longer than 32 characters.\"],\n    required: [true, \"Password is required.\"],\n    trim: true,\n    select: false,\n  },\n\n  // Password validation\n  passwordConfirm: {\n    type: String,\n    required: [true, \"Please confirm your password.\"],\n    validate: {\n      validator: function (value) {\n        return value === this.password;\n      },\n\n      message: \"Password doesn't match.\",\n    },\n  },\n\n  active: {\n    type: Boolean,\n    default: true,\n    select: false,\n  },\n});\n```\n\nFields in a **mongoose schema object** could be any data types like Int, Array, String, Boolean, or even ObjectId and can have various properties such as unique, required, max-min length, select etc.\n\n```javascript\nconst userSchema = new mongoose.Schema({\n    ...\n    // Referencing\n    followers: [\n        type: mongoose.Schema.Types.ObjectID\n    ]\n})\n\n```\n\n## Types of Data Model\n\n### One to One (1:1)\n\nOne to one relation, as the name suggests requires one entity to have an exclusive relationship with another entity and vice versa. In this case, 1 user can only have 1 username.\n\n```javascript\n{\n    \"username\": \"hsyntes\"\n}\n```\n\n### One to Many (1:Many)\n\nOne to many relation occurs when an instance of an entity has one or more related instances of another entity and keeps the documents as referenced by id or embedded.\n\n```javascript\nconst userSchema = new mongoose.Schema({\n    ...\n    // Embedded\n    \"followers\": [\n        {\n            \"_id\": ObjectId('618'),\n            \"username\": \"xyz\",\n        },\n        {\n            \"_id\": ObjectId('23'),\n            \"username\": \"abc\"\n        }\n        // ...\n    ]\n\n    // Referencing\n    \"followers\": [\n        ObjectId(\"618\"),\n        ObjectId(\"23\")\n        // ...\n    ]\n});\n\n```\n\n### Many to Many\n\nMany-to-many relation occurs when instances of at least two entities can both be related to multiple instances of another entity. In this case, 1 user can have many followers, but 1 follower can also be in many users.\n\n```javascript\n// Also known Two-Way Referencing\n{\n    \"_id\": ObjectId('618'),\n    \"username\": \"hsyntes\",\n    \"followers\": [\n        ObjectId('67'),\n    ]\n},\n{\n    \"_id\": ObjectId('67'),\n    \"username\": \"xyz\",\n    \"followers\": [\n        ObjectId('618')\n    ]\n},\n{\n    \"_id\": ObjectId('23'),\n    \"username\": \"abc\",\n    \"followers\": [\n        ObjectId('618'),\n        ObjectId('67')\n    ]\n}\n```\n\n## Virtual Referencing (Populating)\n\nOne document keeps another document(s) as a **virtual**. The referenced documents actually aren't there, but it will exist when the document is querired.\n\n**Virtual Referencing** or **Populating** would be the best solution for higher performance.\n\n### Generate Schema Key\n\n```javascript\nconst userSchema = new mongoose.Schema({\n  username: {\n    type: String,\n    // ...\n  },\n});\n\n// \"User\" will be the unique Schema Key for this collection\nconst User = mongoose.model(\"User\", userSchema);\n```\n\n### Populate Field(s)\n\nThe virtual field(s) will never exist under this document except for the document is queried with findOne.\n\n```javascript\n// User Schema\nconst userSchema = new mongoose.Schema({\n  username: {\n    type: String,\n    unique: true,\n    // ...\n  },\n},\n  // Enable virtuals\n  { versionKey: false,\n    toJSON: { virtuals: true },\n    toObject: { virtuals: true }\n  }\n);\n\n// Virtual Populating\nuserSchema.virtual(\"Posts\", { // Keep documents under \"posts\" field\n    ref: \"Posts\" // Schema key in another collection for reference\n    foreginField: \"postedBy\",\n    localField: \"_id\"\n});\n\n// Query Middleware\nuserSchema.pre('findOne', function (next) {\n    this.populate('posts');\n\n    next();\n})\n```\n\nWhen the document is queried with 'findOne', the posts will be fetched that the user document related.\n\n### Set the reference\n\n```javascript\nexports.createPost = async (req, res, next) =\u003e {\n  try {\n    const post = await Post.create({\n      title: req.body.title,\n      text: req.body.text,\n      // The current user that create this document will be settled as referenced\n      postedBy: req.user._id,\n    });\n    //...\n  }\n}\n```\n\n### Set the references as **unique**\n\n$addToSet and $pull operators prevent duplicated fields when a field added/removed from an array in MongoDB.\n\n```javascript\n// Starting the transaction\nconst session = await mongoose.startSession();\nsession.startTransaction();\nawait Post.findById(id).session(session);\n\nawait Post.findByIdAndUpdate(\n  id,\n  {\n    $addToSet: { likes: req.user._id },\n  },\n  { new: true, runValidators: true }\n);\n\nawait Post.findByIdAndUpdate(id, {\n  $pull: { likes: req.user._id },\n});\n```\n\nSession transaction starts a multi-document transaction associated with the session. Multi-document transactions are available for both shared clusters and replica sets.\n\nWant to see an example with a **real application**, please have a look at my [instamern](https://github.com/hsyntes/instamern-api) project. You can reach out advanced data modeling \u0026 schema design with aggregation pipeline, aggregation middleware, setting references with unique, etc.\n\n[![InstaMERN](https://github.com/hsyntes/instamern/blob/main/public/logo.png)](https://instamern.netlify.app)\n\n## 🔗 Contact me\n\n[![linkedin](https://img.shields.io/badge/linkedin-0A66C2?style=for-the-badge\u0026logo=linkedin\u0026logoColor=white)](https://www.linkedin.com/in/hsyntes)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhsyntes%2Fdata-modeling","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhsyntes%2Fdata-modeling","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhsyntes%2Fdata-modeling/lists"}