{"id":15190640,"url":"https://github.com/umran/apify","last_synced_at":"2026-03-03T03:39:56.256Z","repository":{"id":96191807,"uuid":"166589283","full_name":"umran/apify","owner":"umran","description":"A tool to bootstrap a headless content management and delivery system using graphql, mongodb, redis and elasticsearch","archived":false,"fork":false,"pushed_at":"2019-04-03T20:18:30.000Z","size":353,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-09-20T09:10:01.617Z","etag":null,"topics":["apify","backend","bootstrap-modern-backends","cms","cms-backend","elasticsearch","graphql","mongodb","mongoosejs","redis"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/umran.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-01-19T20:17:51.000Z","updated_at":"2019-04-03T20:18:31.000Z","dependencies_parsed_at":"2023-03-13T16:36:25.819Z","dependency_job_id":null,"html_url":"https://github.com/umran/apify","commit_stats":{"total_commits":156,"total_committers":1,"mean_commits":156.0,"dds":0.0,"last_synced_commit":"ed00301e207903245e73a264fcdb6d29916df662"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/umran%2Fapify","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/umran%2Fapify/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/umran%2Fapify/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/umran%2Fapify/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/umran","download_url":"https://codeload.github.com/umran/apify/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241110771,"owners_count":19911394,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apify","backend","bootstrap-modern-backends","cms","cms-backend","elasticsearch","graphql","mongodb","mongoosejs","redis"],"created_at":"2024-09-27T20:43:16.758Z","updated_at":"2026-03-03T03:39:51.235Z","avatar_url":"https://github.com/umran.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Apify\n\nA tool to bootstrap modern content management and delivery systems using GraphQL, MongoDB, Redis and Elasticsearch.\n\n[![Build Status](https://travis-ci.org/umran/apify.svg?branch=master)](https://travis-ci.org/umran/apify)\n[![Coverage Status](https://img.shields.io/coveralls/github/umran/apify/master.svg)](https://coveralls.io/github/umran/apify?branch=master)\n[![Known Vulnerabilities](https://snyk.io/test/github/umran/apify/badge.svg)](https://snyk.io/test/github/umran/apify)\n[![Dependency Status](https://david-dm.org/umran/apify.svg)](https://david-dm.org/umran/apify)\n\n## Introduction\n\nApify generates GraphQL CRUD APIs that are tightly coupled with MongoDB, via Mongoose, as a primary store, optionally Redis, among others as a cache store, and Elasticsearch as a full-text search engine. Apify exposes a simple configuration interface that is designed to combine configuration options for Mongoose, GraphQL schemas and Elasticsearch mappings. Apify thus effectively reduces the configuration complexity of backends that rely on MongoDB, Elasticsearch and GraphQL to a single configuration file under a universal syntax.\n\nFor each collection level document that is defined, Apify automatically creates the relevant collection in MongoDB along with its Elasticsearch mappings. It also defines GraphQL endpoints for each collection that allow CRUD operations as well as full-text search to be performed on documents right out of the box.\n\n## Installation\n\n```\nnpm install --save @umran/apify\n```\n\nTo actually run the GraphQL server you will also need the `express` and `express-graphql` packages\n\n```\nnpm install --save express express-graphql\n```\n\n## Defining Document Schemas\n\nDefining document schemas is the first and the most crucial step towards setting up the server. Documents are defined as javascript objects that have the properties: `class` and `fields`.\n\n### The `class` Property\n\nThe `class` property takes a string value indicating the class of document. There are two classes of documents that may be defined: \"collection\" and \"embedded\"\n\n#### The Collection Class\n\nCollection class documents are documents that will be stored in MongoDB under a collection. Documents classified as collection are usually standalone documents that have meaning in and of themselves.\n\nSee below for an example schema of a collection class document.\n\n```javascript\nconst Student = {\n  class: 'collection',\n  fields: {\n    firstName: {\n      type: 'string',\n      required: true,\n      es_indexed: true,\n      es_keyword: true\n    },\n    lastName: {\n      type: 'string',\n      required: true,\n      es_indexed: true,\n      es_keyword: true\n    }\n  }\n}\n```\n\n#### The Embedded Class\n\nEmbedded documents are, as the name suggests, documents that will not be stored under its own collection, but rather embedded in an existing collection. Documents classified as embedded are usually documents that do not make much sense outside the context of a parent document. Because the shape of any individual document schema is flat, Apify requires you to define a separate embedded document for each level of nesting in order to define documents with deeply nested structures.\n\nSee below for an example schema of an embedded document.\n\n```javascript\nconst Grades = {\n  class: 'embedded',\n  fields: {\n    mathematics: {\n      type: 'float',\n      required: true,\n      es_indexed: true,\n      es_boost: 2.0\n    },\n    english: {\n      type: 'float',\n      required: true,\n      es_indexed: true,\n      es_boost: 1.5\n    },\n    physics: {\n      type: 'float',\n      required: false,\n      es_indexed: true,\n      es_boost: 1.0\n    }\n  }\n}\n```\n\nIn order to reference an embedded document from within a collection level document, simply create a reference field within the collection level document that points to the embedded document, like so:\n\n```javascript\n// an embedded document\nconst Grades = {\n  class: 'embedded',\n  fields: {\n    mathematics: {\n      type: 'float',\n      required: true,\n      es_indexed: true,\n      es_boost: 2.0\n    },\n    english: {\n      type: 'float',\n      required: true,\n      es_indexed: true,\n      es_boost: 1.5\n    },\n    physics: {\n      type: 'float',\n      required: false,\n      es_indexed: true,\n      es_boost: 1.0\n    }\n  }\n}\n\n// a collection level document that references the above defined embedded document\nconst Student = {\n  class: 'collection',\n  fields: {\n    firstName: {\n      type: 'string',\n      required: true,\n      es_indexed: true,\n      es_keyword: true\n    },\n    lastName: {\n      type: 'string',\n      required: true,\n      es_indexed: true,\n      es_keyword: true\n    },\n    grades: {\n      type: 'reference',\n      ref: 'Grades',\n      required: false,\n      es_indexed: true\n    }\n  }\n}\n```\n\n### The `fields` Property\n\nThe `fields` property is an object that contains all of the fields of the document, which in turn contain information relevant to validation and search indexing of the field. Each key in the `fields` object corresponds to the name of a field, while its value is a `field` object that contains information about the field.\n\n####  The `field` Object\n\nEach field of a document has a couple of properties that Apify should know about. These properties are provided in the form of a `field` object, one for each field of the document. A field object can have a number of properties, both required and optional depending on its type. See below for details of each field type in turn.\n\n##### String Fields\n\nString fields have a required property: `type`, whose value must be set to \"string\". Refer to the table below for all the properties that may be defined on a field object of `type` \"string\".\n\n| Property | Required | Type | Description |\n| --- | --- | --- | --- |\n| `required` | true | Boolean | Tells Apify whether to always expect a value for the field |\n| `type` | true | String | Tells Apify how to parse values of the field |\n| `es_indexed` | true | Boolean | Tells Elasticsearch whether to analyze the field during indexing |\n| `es_keyword` | true | Boolean | Tells Elasticsearch whether to analyze the field as a `keyword` rather than as full-text |\n| `es_boost` | false | Number | Tells Elasticsearch how to weight the field when calculating the document's relevance score; defaults to 1.0 |\n| `es_analyzer` | false | String | Tells Elasticsearch which analyzer to use during indexing; defaults to the standard analyzer |\n| `es_search_analyzer` | false | String | Tells Elasticsearch which analyzer to use during search; defaults to the analyzer defined at `es_analyzer`|\n| `es_search_quote_analyzer` | false | String | Tells Elasticsearch which analyzer to use for quotes during search; defaults to the analyzer defined at `es_search_analyzer`|\n\n##### Integer Fields\n\nInteger fields have a required property: `type`, whose value must be set to \"integer\". Refer to the table below for all the properties that may be defined on a field object of `type` \"integer\".\n\n| Property | Required | Type | Description |\n| --- | --- | --- | --- |\n| `required` | true | Boolean | Tells Apify whether to always expect a value for the field |\n| `type` | true | String | Tells Apify how to parse values of the field |\n| `es_indexed` | true | Boolean | Tells Elasticsearch whether to analyze the field during indexing |\n| `es_boost` | false | Number | Tells Elasticsearch how to weight the field when calculating the document's relevance score; defaults to 1.0 |\n\n##### Float Fields\n\nFloat fields have a required property: `type`, whose value must be set to \"integer\". Refer to the table below for all the properties that may be defined on a field object of `type` \"float\".\n\n| Property | Required | Type | Description |\n| --- | --- | --- | --- |\n| `required` | true | Boolean | Tells Apify whether to always expect a value for the field |\n| `type` | true | String | Tells Apify how to parse values of the field |\n| `es_indexed` | true | Boolean | Tells Elasticsearch whether to analyze the field during indexing |\n| `es_boost` | false | Number | Tells Elasticsearch how to weight the field when calculating the document's relevance score; defaults to 1.0 |\n\n##### Boolean Fields\n\nInteger fields have a required property: `type`, whose value must be set to \"integer\". Refer to the table below for all the properties that may be defined on a field object of `type` \"integer\".\n\n| Property | Required | Type | Description |\n| --- | --- | --- | --- |\n| `required` | true | Boolean | Tells Apify whether to always expect a value for the field |\n| `type` | true | String | Tells Apify how to parse values of the field |\n| `es_indexed` | true | Boolean | Tells Elasticsearch whether to analyze the field during indexing |\n| `es_boost` | false | Number | Tells Elasticsearch how to weight the field when calculating the document's relevance score; defaults to 1.0 |\n\n##### Date Fields\n\nDate fields have a required property: `type`, whose value must be set to \"date\". Date fields are a special kind of field in that it takes valid ISO `Date` objects. Refer to the table below for all the properties that may be defined on a field object of `type` \"date\".\n\n| Property | Required | Type | Description |\n| --- | --- | --- | --- |\n| `required` | true | Boolean | Tells Apify whether to always expect a value for the field |\n| `type` | true | String | Tells Apify how to parse values of the field |\n| `es_indexed` | true | Boolean | Tells Elasticsearch whether to analyze the field during indexing |\n| `es_boost` | false | Number | Tells Elasticsearch how to weight the field when calculating the document's relevance score; defaults to 1.0 |\n| `default` | false | String | Specifies a default date to use during creation of the document. Takes any valid ISO Date string or the value: \"current_date\", which generates a timestamp during query time (on create or update) |\n\n##### Reference Fields\n\nReference fields have a required property: `type`, whose value must be set to \"reference\". Reference fields are a pointers to other collection class or embedded documents and are useful for nesting documents together. Refer to the table below for all the properties that may be defined on a field object of `type` \"reference\".\n\n| Property | Required | Type | Description |\n| --- | --- | --- | --- |\n| `required` | true | Boolean | Tells Apify whether to always expect a value for the field |\n| `type` | true | String | Tells Apify how to parse values of the field |\n| `es_indexed` | true | Boolean | Tells Elasticsearch whether to analyze the field containing the nested document during indexing |\n| `ref` | true | String | Tells Apify which document this field is a reference to. The value should be the document name as defined in the document schema definition and is case sensitive  |\n\n##### Array Fields\n\nArray fields have a required property: `type`, whose value must be set to \"array\". Array fields specify a list of values of some type of field. Array fields are useful for storing lists of values of the same type in a collection. Refer to the table below for all the properties that may be defined on a field object of `type` \"array\".\n\n| Property | Required | Type | Description |\n| --- | --- | --- | --- |\n| `required` | true | Boolean | Tells Apify whether to always expect a value for the field |\n| `type` | true | String | Tells Apify how to parse values of the field |\n| `item` | true | Object \u003cField\u003e | Tells Apify the field type of the values that are contained in the array. The value must be an object that represents one of the field types described above; cannot be an array type because nesting arrays within arrays is not allowed |\n\n## Composing the Document Definitions\n\nAll documents, of both the embedded and collection classes should be compiled into a single javascript object whose keys are the document names:\n\n```javascript\n// for brevity we're using the previously defined sample documents\nconst documentDefinitions = {\n  Grades,\n  Student\n}\n```\n\n## Defining the Resolver\n\nThe resolver is a single function that defines how GraphQL API calls to the `create_`, `find_`, `findOne_`, `update_`, `delete_` and `search_` endpoints are handled. Because Apify generates the relevant Mongoose models and Elasticsearch mappings during runtime, the user is expected to create a curry function, call it `createResolver`, that accepts an object (which contains the Mongoose models and Elasticsearch mappings) as a parameter and returns an `async` resolver function. The resolver function returned by `createResolver` accepts an object containing details of the method being called by the GraphQL query, the particular document involved, the arguments of the query and the GraphQL context. With this information you could write logic within the resolver function that allows you to respond with the appropriate resource. Since the GraphQL context is provided, access control logic could also be implemented from within the resolver function.\n\n### The `createResolver` function\n\n```javascript\n// for the sake of brevity, assume these methods are already defined in another place\nconst { find, findOne, search, create, update, _delete } = require('./predefinedMethods')\n\nconst createResolver = ({ mongoose_models, elastic_mappings }) =\u003e\n  async ({ method, collection, root, args, context }) =\u003e {\n\n    // example resolver logic\n    const model = mongoose_models[collection]\n    const mapping = elastic_mappings[collection]\n\n    switch(method) {\n      case 'find':\n        return await find(model, args)\n      case 'findOne':\n        return await findOne(model, args)\n      case 'search':\n        return await search(mapping, args)\n      case 'create':\n        return await create(model, args)\n      case 'update':\n        return await update(model, args)\n      case 'delete':\n        return await _delete(model, args)\n    }\n  }\n```\n\n## Building the GraphQL Schema\n\nOnce the `createResolver` function and document definitions are set up, the GraphQL schema can be built by calling the `buildGraphql` function with the document definitions and `createResolver` function as arguments.\n\n```javascript\nconst { buildGraphql } = require('@umran/apify')\n\n// for brevity assume these are already defined elsewhere\nconst documentDefinitions = require('./documentDefinitions')\nconst createResolver = require('./createResolver')\n\nconst { graphqlSchema } = buildGraphql(documentDefinitions, createResolver)\n```\n\n## Setting Up and Running the GraphQL Server\n\nSetting up and running the server is pretty straightforward. You will need to either create a new express app or use an existing one. The GraphQL server can be attached to the express instance as middleware at a path of your choice.\n\n```javascript\nconst express = require('express')\nconst graphqlHTTP = require('express-graphql')\n\n// for brevity assume the graphqlSchema is already built and available elsewhere\nconst graphqlSchema = require('./graphqlSchema')\n\n// create a new express app\nconst app = express()\n\napp.use('/api', graphqlHTTP({\n  schema: graphqlSchema,\n  graphiql: true\n}))\n\napp.listen(3000)\n```\n\n## Building the Mongoose Models and Elasticsearch Mappings Elsewhere in Your Code (Optional)\n\nSometimes it is not convenient to have code that calls the database directly live inside the GraphQL API server, for example when running a microservices architecture that decouples database operations from consumer facing services. For this reason a convenience function called `buildBackend` is available. This function can be imported and called by a separate process to generate the Mongoose models and Elasticsearch mappings needed to read and write to/from the databases.\n\nWith this capability the main resolver function which runs on the GraphQL API server can push queries to a message queue like RabbitMQ while a separate process consumes the queries and actually executes the database calls.\n\nSee below for sample code implemented in a separate NodeJS process:\n\n```javascript\nconst { buildBackend } = require('@umran/apify')\n\n// the document definitions must be available to this process\nconst documentDefinitions = require('./documentDefinitions')\n\nconst { mongoose_models, elastic_mappings } = buildBackend(documentDefinitions)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fumran%2Fapify","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fumran%2Fapify","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fumran%2Fapify/lists"}