{"id":35847283,"url":"https://github.com/cjmalloy/jasper","last_synced_at":"2026-04-26T02:02:54.954Z","repository":{"id":45683631,"uuid":"471450457","full_name":"cjmalloy/jasper","owner":"cjmalloy","description":"Knowledge Management Server","archived":false,"fork":false,"pushed_at":"2026-04-22T21:25:55.000Z","size":4566,"stargazers_count":15,"open_issues_count":13,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2026-04-22T22:26:47.786Z","etag":null,"topics":["business-intelligence","data-structures","docker","graph","protocol","rest-api","server","spring-boot"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cjmalloy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-03-18T17:01:57.000Z","updated_at":"2026-04-22T18:39:35.000Z","dependencies_parsed_at":"2026-03-08T11:02:05.975Z","dependency_job_id":null,"html_url":"https://github.com/cjmalloy/jasper","commit_stats":null,"previous_names":[],"tags_count":846,"template":false,"template_full_name":null,"purl":"pkg:github/cjmalloy/jasper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cjmalloy%2Fjasper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cjmalloy%2Fjasper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cjmalloy%2Fjasper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cjmalloy%2Fjasper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cjmalloy","download_url":"https://codeload.github.com/cjmalloy/jasper/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cjmalloy%2Fjasper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32283294,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-25T18:29:39.964Z","status":"online","status_checked_at":"2026-04-26T02:00:05.962Z","response_time":129,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["business-intelligence","data-structures","docker","graph","protocol","rest-api","server","spring-boot"],"created_at":"2026-01-08T06:12:19.616Z","updated_at":"2026-04-26T02:02:54.936Z","avatar_url":"https://github.com/cjmalloy.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Jasper\nKnowledge Management Server\n\n[![Build \u0026 Test](https://github.com/cjmalloy/jasper/actions/workflows/test.yml/badge.svg)](https://cjmalloy.github.io/jasper/reports/latest-junit/)\n[![Gatling](https://github.com/cjmalloy/jasper/actions/workflows/gatling.yml/badge.svg)](https://cjmalloy.github.io/jasper/reports/latest-gatling/)\n[![OpenAPI](https://img.shields.io/badge/OpenAPI-1.3.6-brightgreen)](https://editor.swagger.io/?url=https://raw.githubusercontent.com/cjmalloy/jasper/refs/heads/master/src/main/resources/swagger/api.yml)\n[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/jasper)](https://artifacthub.io/packages/helm/jasper/jasper)\n\n## Quickstart\nTo start the server, client, and database with a single admin user, run\nthe [quickstart](https://github.com/cjmalloy/jasper-ui/blob/master/quickstart/docker-compose.yaml)\ndocker compose file. See [Jasper App](https://github.com/cjmalloy/jasper-app) for an installable\nelectron wrapper.\n\nSee [Jasper-UI](https://github.com/cjmalloy/jasper-ui) for documentation on the reference client.\n\n## Knowledge Management\nJasper is an open source knowledge management system. It provides a generic set of tools for dealing\nwith knowledge management style problems. Knowledge management type problems include:\n* Scientific Research\n* Business Intelligence\n* Journalism\n* Web Forums\n* Wiki (Encyclopedia)\n* Task Management\n* Libraries\n* Customer Support\n* Collaborative Writing\n* Personal Knowledge Management\n* E-mail\n\nJasper can be configured to host all these products, individually or in combination.\nRun it as an app or a website and connect them together to build a networked system.\nPrevent data loss by having full or partial replication of data across the network.\nSafely ingest external data sources with one-way replication.\nEnforce conformity with a flexible data model, or simply collate unstructured reports.\n\n### Security\nJasper uses Tag Based Access Control (TBAC) to assign fine grained access controls to any object in the\nsystem. This system is simple and powerful, such that the entire security specification is contained\nin a [small, readable file](https://github.com/cjmalloy/jasper/blob/master/src/main/java/jasper/security/Auth.java).\n\n### Build your own client\nConnect to Jasper with a custom client to give users a streamlined user experience. Frontend\ndevelopers can create a bespoke interface without needing to make any server side changes. Create custom\nplugins and templates and ensure data shape with [JTD](https://jsontypedef.com/docs/jtd-in-5-minutes/)\nschemas. Fork [the reference client](https://github.com/cjmalloy/jasper-ui) or use the\n[OpenApi docs](https://editor.swagger.io/?url=https://raw.githubusercontent.com/cjmalloy/jasper/refs/heads/master/src/main/resources/swagger/api.yml) to generate API stubs.\n\n## Standards\nJasper is a standard data model and API. While JSON is used in this document, Jasper may be generalised\nto other presentations, such as XML, YAML, or TOML.\nJasper defines five entity types, an access control model, and a plugin/templating system for extending\nthe model.\n1. Ref\n2. Ext\n3. User\n4. Plugin\n5. Template\n\nThe main entity is the Ref, it represents a reference to an external resource. The main field in a Ref\nis the URL field which can be a link to a web page, or a reference to any arbitrary resources predicated\nby the URL scheme. Web content will of course use the http or https scheme. To reference a book,\none could use the [ISBN](https://en.wikipedia.org/wiki/ISBN) scheme (i.e. `isbn:978-3-16-148410-0`).\nFor comments, [Jasper-UI](https://github.com/cjmalloy/jasper-ui) uses a `comment` scheme followed by an arbitrary ID, usually a UUID\n(i.e. `comment:75b36465-4236-4d64-8c78-027d87f3c072`). For hosting internal wikis, \n[Jasper-UI](https://github.com/cjmalloy/jasper-ui) uses a `wiki` scheme followed by the\n[Wiki Page Name](https://en.wikipedia.org/wiki/Wikipedia:Page_name) (i.e. `wiki:John_Cena`).\n\nLike the [OSI model](https://en.wikipedia.org/wiki/OSI_model), Jasper's data model is defined in layers:\n1. **Identity Layer** - persistence of individual entities\n2. **Indexing Layer** - query and transport of entities\n3. **Application Layer** - custom modifications\n\n## Tagging\nJasper support hierarchical tagging of Refs. Tags are not entities, they are strings with\nregex `[_+]?[a-z0-9]+([./][a-z0-9]+)*`. Tags are part of the primary key for Tag-like entities, but no\nentities need exist to use a tag. \nRefs have a list of tags which can be used for categorization, permissions, and plugins. \nThere are three types of tags, which the type defined as a semantic ontology:\n`public`, `+protected`, `_private` tags. The character prefix defines the type while also being\npart of the tag itself. Therefore, no lookup is ever required to determine the tag type.\n * A public tag can be used freely by anyone. This includes tagging a Ref, or using it in a query.\n * A protected tag can freely be used in a query, but you cannot tag a Ref with a protected tag\nunless it is in your [read access](#access-control) list.\n * A private tag cannot be used at all unless permission is given. When fetching a Ref that includes\nprivate tags, they will be removed by the server prior to sending. See\n[access control](#access-control) for more.\n\nTags may also be fully qualified by appending the origin. (i.e. `tag@origin`). \nUse forward slashes to define hierarchical tags (i.e. `people/murray/bill` or `people/murray/anne`)\n\n## Querying\nWhen fetching a page or Refs a query may be specified. The query language uses simple set-like\noperators to match Refs according to their tag list and Origin. You may use tags, origins, or\nfully qualified tags (tag + origin). There is a special origin `@` which will match the\ndefault origin `\"\"` (the empty string). \nIf a tag is not fully qualified it will match the wildcard origin `\"@*\"`. The `*`\nwild card can be used to match anything on the default origin `\"\"` (empty string).\nValid operators in a query are:\n1. `:` and\n2. `|` or\n3. `!` not\n4. `()` groups\n\nNote: In the current implementation, groups may not be nested.\n\nExample queries:\n * `science`: All Refs that include the `science` tag\n * `science|funny`: All Refs that have either the `science` tag or the `funny` tag\n * `science:funny`: All Refs that have both the `science` tag and the `funny` tag\n * `science:!funny`: All Refs that have the `science` tag but do not have the `funny` tag\n * `(science|math):funny`: All Refs that have either the `science` or `math` tags, but\nalso the `funny` tag. This would match a ref with `['science', 'funny']`, `['math', 'funny']`,\nbut would not match `['science', 'math']`\n * `science:funny|math:funny`: Extended form of previous query. Would produce the exact same results.\n * `music:people/murray`: All Refs that have the `music` tag and `people/murray` tag. It would also\nmatch Refs with `['music', 'people/murray/anne']` or `['music', 'people/murray/bill']`\n\n## Sorting\nJasper supports dynamic sorting on fields using arrow notation (`-\u003e`). This allows sorting by\nany field within the `plugins`, `metadata`, `config`, or `external` JSONB columns without requiring\ndatabase schema changes.\n\n**Sort Syntax:**\n- Use `-\u003e` as the path separator to navigate fields\n- Append `:num` suffix for numeric sorting (otherwise values sort as strings)\n- Append `:len` suffix to sort by array length, origin nesting level, or tag levels\n- Use `[index]` notation for array element access (e.g., `external-\u003eids[0]`)\n\nThe `:num` and `:len` suffixes are automatically applied to metadata fields, so you can use\n`metadata-\u003eresponses` instead of `metadata-\u003eresponses:len`.\n\n## Modding\nJasper allows extensive modification with server reuse. Since changes are done by creating\nPlugin and Template entities, server restarts are not required. \nThis method of modding means that only client changes are required. The same Jasper server,\nwithout any code modifications, can be used. The client can define and support its own Plugins\nand Templates. This allows for much more flexible development, as writing client code (in particular\nweb clients) is much easier than writing server code. A developer with only front-end expertise \ncan extend the Jasper model to support arbitrary applications. \nIn order to extend the functionality of a Ref, a developer may choose a set of tags or URL scheme\nand a convention by which they modify the semantics of a Ref. If a custom data model is also\nrequired, a Plugin entity may be created which defines a\n[JTD](https://jsontypedef.com/docs/jtd-in-5-minutes/) schema. A Plugin is a Tag-like entity. When\na Ref is tagged with a Plugin, the Plugin may be considered active for that Ref. The Ref may then\nstore data in its config field and the server will validate it according to the schema. \nSimilarly, Ext entities may be created which extend the functionality of a tag. As Plugins define\ncustom data that can be stored in a ref, Templates may be created which allow custom data to be\nstored in Ext entities and similarly validated according to their schema.\n\nSee [Jasper-UI](https://github.com/cjmalloy/jasper-ui) for examples of Plugins and Templates, such as:\n* `plugin/thumbanail`: [This plugin](https://github.com/cjmalloy/jasper-ui/blob/master/src/app/mods/thumbnail.ts)\nallows a Ref to include a URL to a thumbnail image.\n* `user` Template: \n[This template](https://github.com/cjmalloy/jasper-ui/blob/master/src/app/mods/user.ts)\nallows a user tag to customize their experience, such as subscribing to a list of tags to show\non their home page.\n\n## Entities\nThere are two types of entities in Jasper:\n1. Refs\n2. Tags (including Exts, Plugins, Templates, and Users)\n\n![entities](./docs/entities.png)\nOrigins are used to facilitate replication and multi-tenant operation. Each origin represents a\njasper instance that that entity originated from.\n![origins](./docs/origins.png)\n\n### Ref\nRefs are the main data model in Jasper. A Ref defines a URL to a remote resource. Example:\n```json \n{\n \"url\": \"https://www.youtube.com/watch?v=9Gn4rmQTZek\",\n \"origin\": \"\",\n \"title\": \"Why does Science News Suck So Much?\",\n \"comment\": \"Sabine Hossenfelder\",\n \"tags\": [\"public\", \"youtube\", \"sabine\"],\n \"sources\": [],\n \"alternateUrls\": [],\n \"plugins\": {\n \"plugin/thumbnail\": {\"url\": \"https://...jpg\"}\n },\n \"metadata\": {\n \"responses\": 0,\n \"internalResponses\": 0,\n \"plugins\": {},\n \"modified\": \"2022-06-18T12:07:04.404272Z\"\n },\n \"published\": \"2022-06-18T12:00:07Z\",\n \"created\": \"2022-06-18T12:07:04.404272Z\",\n \"modified\": \"2022-06-18T12:07:04.404272Z\"\n}\n```\nOnly the \"url\" field is required.\n\nThe combination of URL and Origin for a Ref must be unique and may be used as a Primary Composite Key.\nImplementations may also make the modified date part of the composite primary key for version history.\n\n**URL:** The url of the resource. \n**Origin:** The Origin this Ref was replicated from, or the empty string for local. \n**Title:** Optional title for this Ref. \n**Comment:** Optional comment for this Ref, usually markdown. \n**Tags:** A list of tags used to categorise this Ref. All tags must match the regex `[_+]?[a-z0-9]+([./][a-z0-9]+)*` \n**Sources:** A list of URLs which are sources for this Ref. These may or may not have a corresponding Ref\nentity. If a source URL does correspond to a Ref, the published date of the source must predate the\npublished date of this Ref. \n**Alternate URLs:** Alternate URLs which should be considered synonymous with the URL of this Ref. This\nshould be used as part of a uniqueness check when ingesting Refs. \n**Plugins:** A JSON object with plugin tags as fields and arbitrary JSON data defined by each respective\nplugin. Must be valid according to each plugin's schema. \n**Metadata:** Optional data generated by the server for this resource. Includes response links (inverse\nsource lookup). \n**Published:** The published date of this resource. Default to create date if not known. This date must\nbe later than the published date of all sources. \n**Created:** Created date of this Ref. \n**Modified:** Last modified date of this Ref. If this is the same as the created date no modification\nhas occurred. Does not update if Metadata is modified. \n\n### Ext\nAn Ext is a Tag-like entity representing a Tag extension.\n```json \n{\n \"tag\": \"news\",\n \"origin\": \"\",\n \"name\": \"News\",\n \"config\": {\n \"pinned\":[],\n \"sidebar\": \"\"\n },\n \"modified\": \"2022-06-18T16:00:59.978700Z\"\n}\n```\nOnly the \"tag\" field is required.\n\nAn Ext allows you to customise a Tag page. For example, you could set the sidebar text or pin some links.\n\nThe combination of Tag and Origin for a Ext must be unique and may be used as a Primary Composite Key.\nImplementations may also make the modified date part of the composite primary key for version history.\n\n**Tag:** The tag of this Ext. Must match the regex `[_+]?[a-z0-9]+([./][a-z0-9]+)*`\n**Origin:** The Origin this Ext was replicated from, or the empty string for local.\n**Name:** The display name of this Ext. Used to customise the page title for the Tag page.\n**Config:** Arbitrary JSON data defined by Templates. Must be valid according to each template's schema.\n**Modified:** Last modified date of this Ext\n\n### User\nA User is a Tag-like entity representing a user.\n\n```json \n{\n \"tag\": \"+user/charlie\",\n \"origin\": \"\",\n \"name\": \"Charlie Brown\",\n \"readAccess\": [],\n \"writeAccess\": [],\n \"tagReadAccess\": [],\n \"tagWriteAccess\": [],\n \"pubKey\": \"...\",\n \"external\": {\n \"ids\": []\n },\n \"modified\": \"2022-06-18T16:00:59.978700Z\"\n}\n```\nOnly the \"tag\" field is required.\n\nA User contains the access control information for the system. Access tags work in all\nsub-origins.\n\nThe combination of Tag and Origin for a User must be unique and may be used as a Primary Composite Key.\nImplementations may also make the modified date part of the composite primary key for version history.\n\n**Tag:** The tag of this User. Must match the regex `[_+]user/[a-z0-9]+([./][a-z0-9]+)*` \n**Origin:** The Origin this User was replicated from, or the empty string for local. \n**Name:** The display name of this User. Used to customise the page title for the Tag page. \n**Read Access:** List of tags this user has complete read access to. Grants read access to all\nentities with this tag. \n**Write Access:** List of tags this user has complete write access to. Grants write access to\nall entities with this tag. \n**Tag Read Access:** List of tags this user can read. Only applies to Tag-like entities. Only needed\nfor private tags. \n**Tag Write Access:** List of tags this user can write. Only applies to Tag-like entities. \n**External IDs:** IDs used in an external auth system. Only used when external IDs are enabled. \n**Pub Key:** Base 64 encoded public RSA key. Used for verifying signatures to validate authorship. \n**Modified:** Last modified date of this User. \n\n### Plugin\nA Plugin is a Tag-like entity used to extend the functionality of Refs.\n```json \n{\n \"tag\": \"plugin/thumbnail\",\n \"origin\": \"\",\n \"name\": \"⭕️ Thumbnail\",\n \"config\": {...},\n \"defaults\": {},\n \"schema\": {\n \"optionalProperties\": {\n \"url\": {\"type\": \"string\"},\n \"width\": {\"type\": \"int32\", \"nullable\": true},\n \"height\": {\"type\": \"int32\", \"nullable\": true}\n }\n },\n \"modified\": \"2022-06-18T16:27:13.774959Z\"\n}\n```\nOnly the \"tag\" field is required.\n\nTagging a ref with a Plugin tag applies that plugin to the Ref. The Ref plugin must contain valid\ndata according to the Plugin schema. \n\nThe combination of Tag and Origin for a Plugin must be unique and may be used as a Primary Composite Key.\nImplementations may also make the modified date part of the composite primary key for version history.\n\n**Tag:** The tag of this Plugin. Must match the regex `[_+]?plugin/[a-z0-9]+([./][a-z0-9]+)*` \n**Origin:** The Origin this Plugin was replicated from, or the empty string for local. \n**Name:** The display name of this Ext. Used to customise the page title for the Tag page. \n**Config:** Arbitrary JSON. \n**Defaults:** Default plugin data if creating a new Ref with empty plugin data. May be any JSON value (object, array, or scalar). \n**Schema:** Json Type Def (JTD) schema used to validate plugin data in Ref. \n**Modified:** Last modified date of this Plugin. \n\n### Template\nA Template is a Tag-like entity used to extend the functionality of Exts.\n```json \n{\n \"tag\": \"\",\n \"origin\": \"\",\n \"name\": \"Default Template\",\n \"config\": {...},\n \"defaults\": {\n \"pinned\": []\n },\n \"schema\": {\n \"properties\": {\n \"pinned\": {\"elements\": {\"type\": \"string\"}}\n },\n \"optionalProperties\": {\n \"sidebar\": {\"type\": \"string\"}\n }\n },\n \"modified\": \"2022-06-18T16:27:13.774959Z\"\n}\n```\nOnly the \"tag\" field is required (can be the empty string).\n\nThe Tag in the case of a template is actually a Tag prefix. This Template matches all Exts\nwhere its tag followed by a forward slash is a prefix of the Ext tag. In the case of the empty\nstring the Template matches all Exts.\n\nThe combination of Tag and Origin for this Template must be unique and may be used as a Primary Composite Key.\nImplementations may also make the modified date part of the composite primary key for version history.\n\n**Tag:** The tag of this Template. Must match the regex `[_+]?[a-z0-9]+([./][a-z0-9]+)*` or the empty string. \n**Origin:** The Origin this Template was replicated from, or the empty string for local. \n**Name:** The display name of this Template. \n**Config:** Arbitrary JSON. \n**Defaults:** Default Ext config if creating a new Ext with empty config. \n**Schema:** Json Type Def (JTD) schema used to validate Ext config. \n**Modified:** Last modified date of this Template.\n\n## Layers\nThe jasper model is defined in layers. This is to facilitate lower level operations such as routing, querying,\nand archiving.\n\n### Identity Layer\nThe identity layer of the Jasper model defines how entities are stored or retrieved. A system operating\nat this layer should be extremely lenient when validating entities. Only the identity fields of the\nentity need to be considered. The identity fields are: \n1. Refs: (URL, Origin, Modified)\n2. Tags: (Tag, Origin, Modified)\n\nTogether, the (Origin, Modified) keys represent the cursor of the entity, which is used in origin based\nreplication. \n\n### Indexing Layer\nThe indexing layer of the Jasper model adds tags to Refs. A system operating at this layer should support\ntag queries, sorting, and filtering.\n\n### Validation Layer\nThe validation layer of the Jasper model includes all entity fields. Plugins and Templates are validated\naccording to their schema.\n\n#### Plugin and Template Inheritance\nPlugins and Templates behave differently in how they inherit the fields of the parent Ext.\nPlugins stack and templates merge.\nFor example, the Plugin `plugin/test` like:\n```json\n{\n \"tag\": \"plugin/test\",\n \"schema\": {\n \"properties\": {\n \"test\": { \"type\": \"string\" }\n }\n }\n}\n```\nAnd the Plugin `plugin/test/this` like:\n```json\n{\n \"tag\": \"plugin/test/this\",\n \"schema\": {\n \"properties\": {\n \"more\": { \"type\": \"string\" }\n }\n }\n}\n```\nIf we use both of these plugins in the same Ref, both plugins would have their\ndata stacked, like:\n```json\n{\n \"url\": \"test:1\",\n \"plugins\": {\n \"plugin/test\": {\n \"test\": \"data\"\n },\n \"plugin/test/this\": {\n \"more\": \"tests\"\n }\n }\n}\n```\nA template would merge all fields, overwriting at every stage, into a final result.\nFor example, the Template `a` like:\n```json\n{\n \"tag\": \"a\",\n \"schema\": {\n \"properties\": {\n \"test\": { \"type\": \"string\" }\n }\n }\n}\n```\nAnd the Template `a/b` like:\n```json\n{\n \"tag\": \"a/b\",\n \"schema\": {\n \"properties\": {\n \"more\": { \"type\": \"string\" }\n }\n }\n}\n```\nIf we use both of these plugins in the same Ext, both plugins would have their\ndata merged, like:\n```json\n{\n \"tag\": \"a/b/c\",\n \"config\": {\n \"test\": \"data\",\n \"more\": \"tests\"\n }\n}\n```\nIf a child Template defines an overlapping field in the schema, it will override the parent type.\n\n### Modding Layer\nThe modding layer of the Jasper model is entirely client side. No server changes are required in order to\nsupport new plugins or templates.\n\n## Cursor Replication\nDistributed systems must make tradeoffs according to the [CAP theorem](https://en.wikipedia.org/wiki/CAP_theorem).\nAccording to the CAP theorem you may only provide two of these three guarantees: consistency, availability,\nand partition tolerance. Jasper uses an eventually consistent model, where availability and partition\ntolerance are guaranteed. The modified date is used as a cursor to efficiently poll for modified records.\n\nTo replicate a Jasper instance simply create a Ref for that instance and tag it `+plugin/origin/pull`.\nAdd the `+plugin/cron` tag to schedule pulling, or add the `+plugin/user/run` response tag to pull a\nsingle time.\n\nThe modified date of the last entity received will be stored and used for the next poll. When polling,\nthe Jasper server requests a batch of entities from the remote instance where the modified date is\nafter the last stored modified date, sorted by modified date ascending.\n\n### Duplicate Modified Date\nJasper instances should enforce unique modified dates as the cursor for each entity type. Otherwise,\nwhen receiving\na batch of entities, it's possible that the last entity you received has a modified date that is\nexactly the same as another entity. If that is the case, requesting the next batch after that modified\ndate will skip such entities.\n\nTo prevent duplicate modified dates it's enough to add a single millisecond to the date until it\nis unique.\n\n## Deployment\nJasper is available in the following distributions:\n - [Docker image](https://github.com/cjmalloy/jasper/pkgs/container/jasper)\n - [Helm chart](https://artifacthub.io/packages/helm/jasper/jasper-ui)\n - [Jar](https://github.com/cjmalloy/jasper/releases/latest)\n\nIt supports the following configuration options:\n\n| Environment Variable | Description | Default Value (in prod) |\n|-----------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `SERVER_PORT` | Port to listen for HTTP connections. | `8081` |\n| `SPRING_PROFILES_ACTIVE` | Set the comma separated list of runtime profiles. | `default` |\n| `SPRING_DATASOURCE_URL` | PostgreSQL database connection string. | `jdbc:postgresql://localhost:5432/jasper` |\n| `SPRING_DATASOURCE_USERNAME` | PostgreSQL database username. | `jasper` |\n| `SPRING_DATASOURCE_PASSWORD` | PostgreSQL database password. | |\n| `JASPER_DEBUG` | | `false` |\n| `JASPER_LOCAL_ORIGIN` | The origin of this server. The local origin may be set to a sub origin via the `Local-Origin` header. | `\"\"` |\n| `JASPER_WORKLOAD` | List of sub-origin sandboxes for worker nodes. | |\n| `JASPER_WORKER` | ID of the worker. Must end in a number which is used to index into JASPER_WORKLOAD to set the worker origin. | |\n| `JASPER_ALLOW_USER_TAG_HEADER` | Allow pre-authentication of a user via the `User-Tag` header. | `false` |\n| `JASPER_ALLOW_USER_ROLE_HEADER` | Allows escalating user role via `User-Role` header. | `false` |\n| `JASPER_ALLOW_AUTH_HEADERS` | Allow adding additional user permissions via `Read-Access`, `Write-Access`, `Tag-Read-Access`, and `Tag-Write-Access` headers. | `false` |\n| `JASPER_MAX_ROLE` | Highest role allowed to access the server. Users with a higher role will have their role reduced to this. | `ROLE_ADMIN` |\n| `JASPER_MIN_ROLE` | Minimum role required to access the server. | `ROLE_ANONYMOUS` |\n| `JASPER_MIN_WRITE_ROLE` | Minimum role required to write to the server. | `ROLE_VIEWER` |\n| `JASPER_MIN_FETCH_ROLE` | Minimum role required to fetch external resources. | `ROLE_USER` |\n| `JASPER_MIN_CONFIG_ROLE` | Minimum role required to edit plugins and templates. | `ROLE_ADMIN` |\n| `JASPER_MIN_READ_BACKUPS_ROLE` | Minimum role required to download backups. | `ROLE_ADMIN` |\n| `JASPER_DEFAULT_ROLE` | Default role given to all users. | `ROLE_ANONYMOUS` |\n| `JASPER_DEFAULT_READ_ACCESS` | Additional read access qualified tags to apply to all users. | |\n| `JASPER_DEFAULT_WRITE_ACCESS` | Additional write access qualified tags to apply to all users. | |\n| `JASPER_DEFAULT_TAG_READ_ACCESS` | Additional tag read access qualified tags to apply to all users. | |\n| `JASPER_DEFAULT_TAG_WRITE_ACCESS` | Additional tag write access qualified tags to apply to all users. | |\n| `JASPER_INGEST_MAX_RETRY` | Maximum number of retry attempts for getting a unique modified date when ingesting a Ref. | `5` |\n| `JASPER_BACKUP_BUFFER_SIZE` | Size of buffer in bytes used to cache JSON in RAM before flushing to disk during backup. | `1000000` |\n| `JASPER_RESTORE_BATCH_SIZE` | Number of entities to restore in each transaction. | `500` |\n| `JASPER_BACKFILL_BATCH_SIZE` | Number of entities to generate Metadata for in each transaction when backfilling. | `100` |\n| `JASPER_CLEAR_CACHE_COOLDOWN_SEC` | Number of seconds to throttle clearing the config cache. | `2` |\n| `JASPER_PUSH_COOLDOWN_SEC` | Number of seconds to throttle pushing after modification. | `1` |\n| `JASPER_STORAGE` | Path to the folder to use for storage. Used by the backup system. | `/var/lib/jasper` |\n| `JASPER_NODE` | Path to node binary for running javascript deltas. | `/usr/local/bin/node` |\n| `JASPER_PYTHON` | Path to python binary for running python scripts. | `/usr/bin/python` |\n| `JASPER_SHELL` | Path to shell binary for running shell scripts. | `/usr/bin/bash` |\n| `JASPER_CACHE_API` | HTTP address of an instance where storage is enabled. | |\n| `JASPER_SSH_CONFIG_NAMESPACE` | K8s namespace to write authorized_keys config map file to. | `default` |\n| `JASPER_SSH_CONFIG_MAP_NAME` | K8s config map name to write `authorized_keys` file to. | `ssh-authorized-keys` |\n| `JASPER_SSH_SECRET_NAME` | K8s secret name to write the `host_key` file to. | `ssh-host-key` |\n| `JASPER_SECURITY_CONTENT_SECURITY_POLICY` | Set the CSP header. | `\"default-src 'self'; frame-src 'self' data:; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://storage.googleapis.com; style-src 'self' 'unsafe-inline'; img-src 'self' data:; font-src 'self' data:\"` |\n| `JASPER_OVERRIDE_SERVER_EMAIL_HOST` | Override the server email host. | |\n| `JASPER_OVERRIDE_SERVER_MAX_SOURCES` | Override the server max sources. | |\n| `JASPER_OVERRIDE_SERVER_MOD_SEALS` | Override the server mod seals. | |\n| `JASPER_OVERRIDE_SERVER_EDITOR_SEALS` | Override the server editor seals. | |\n| `JASPER_OVERRIDE_SERVER_WEB_ORIGINS` | Override the server origins with web access. | |\n| `JASPER_OVERRIDE_SERVER_MAX_REPL_ENTITY_BATCH` | Override the server maximum batch size for replicate controller. | |\n| `JASPER_OVERRIDE_SERVER_SSH_ORIGINS` | Override the server origins with SSH access. | |\n| `JASPER_OVERRIDE_SERVER_MAX_PUSH_ENTITY_BATCH` | Override the server maximum batch size for push replicate. | |\n| `JASPER_OVERRIDE_SERVER_MAX_PULL_ENTITY_BATCH` | Override the server maximum batch size for pull replicate. | |\n| `JASPER_OVERRIDE_SERVER_SCRIPT_SELECTORS` | Override the server tags and origins that can run scripts. No wildcard origins. | |\n| `JASPER_OVERRIDE_SERVER_SCRIPT_WHITELIST` | Override the server list of whitelisted script SHA-256 hashes. | |\n| `JASPER_OVERRIDE_SERVER_HOST_WHITELIST` | Override the server list of whitelisted hosts. | |\n| `JASPER_OVERRIDE_SERVER_HOST_BLACKLIST` | Override the server list of blacklisted hosts. | |\n| `JASPER_OVERRIDE_SERVER_MAX_REQUESTS` | Override the server maximum HTTP requests per origin every 500 nanoseconds. | `50` |\n| `JASPER_OVERRIDE_SERVER_MAX_CONCURRENT_REQUESTS` | Override the server global maximum concurrent HTTP requests (across all origins). | `500` |\n| `JASPER_OVERRIDE_SERVER_MAX_CONCURRENT_SCRIPTS` | Override the server maximum concurrent script executions. | `100_000` |\n| `JASPER_OVERRIDE_SERVER_MAX_CONCURRENT_REPLICATION` | Override the server maximum concurrent replication push/pull operations. | `3` |\n| `JASPER_OVERRIDE_SERVER_MAX_CONCURRENT_FETCH` | Override the server maximum concurrent fetch operations (scraping). | `10` |\n| `JASPER_OVERRIDE_SECURITY_MODE` | Override the security mode for all origins. | |\n| `JASPER_OVERRIDE_SECURITY_CLIENT_ID` | Override the security clientId for all origins. | |\n| `JASPER_OVERRIDE_SECURITY_BASE64_SECRET` | Override the security base64Secret for all origins. | |\n| `JASPER_OVERRIDE_SECURITY_SECRET` | Override the security secret for all origins. | |\n| `JASPER_OVERRIDE_SECURITY_JWKS_URI` | Override the security jwksUri for all origins. | |\n| `JASPER_OVERRIDE_SECURITY_USERNAME_CLAIM` | Override the security usernameClaim for all origins. | |\n| `JASPER_OVERRIDE_SECURITY_VERIFIED_EMAIL_CLAIM` | Override the security verifiedEmailClaim for all origins. | `unset` |\n| `JASPER_OVERRIDE_SECURITY_DEFAULT_USER` | Override the security defaultUser for all origins. | |\n| `JASPER_OVERRIDE_SECURITY_TOKEN_ENDPOINT` | Override the security tokenEndpoint for all origins. | |\n| `JASPER_OVERRIDE_SECURITY_SCIM_ENDPOINT` | Override the security scimEndpoint for all origins. | |\n| `JASPER_OVERRIDE_SECURITY_MAX_REQUESTS` | Override the security maximum HTTP requests per origin every 500 nanoseconds for all origins. | `50` |\n| `JASPER_OVERRIDE_SECURITY_MAX_CONCURRENT_SCRIPTS` | Override the security maximum concurrent script executions per origin for all origins. | `5` |\n| `JASPER_HEAP` | Set both max and initial heap size for the JVM. Only applies to the docker container. | `512m` |\n\n### Configuration Templates\nJasper uses special templates in the root origin to configure server-wide and per-origin settings.\nThese templates are automatically generated with default values if they do not exist.\n\n#### Server Config (`_config/server` Template)\nThe `_config/server` template is installed in the root origin (the local origin for this server)\nand controls server-wide settings. It is automatically created on startup if it does not exist.\n\nIf the current node is running as a worker in origin @worker, the config used will be\n`_config/server/worker` in the local origin (not the worker origin). This allows you to assign\ndifferent nodes to run different workloads.\n\n| Field | Description | Default Value |\n|----------------------------|-------------------------------------------------------------------------------------------------|--------------------------------------------|\n| `emailHost` | Email host used for sending emails. | `jasper.local` |\n| `maxSources` | Maximum number of sources allowed per Ref. | `1000` |\n| `modSeals` | List of tags that act as mod seals (protected tags that only mods can add/remove). | `[\"seal\", \"+seal\", \"_seal\", \"_moderated\"]` |\n| `editorSeals` | List of tags that act as editor seals. | `[\"plugin/qc\"]` |\n| `webOrigins` | Whitelist of origins allowed web access. Supports wildcards. | `[\"\"]` (root origin only) |\n| `maxReplEntityBatch` | Maximum batch size for the replicate controller. | `500` |\n| `sshOrigins` | Whitelist of origins allowed to open SSH tunnels. | `[\"\"]` (root origin only) |\n| `maxPushEntityBatch` | Maximum batch size for push replication. | `5000` |\n| `maxPullEntityBatch` | Maximum batch size for pull replication. | `5000` |\n| `scriptSelectors` | Whitelist of selectors (tag + origin) allowed to run scripts. No origin wildcards. | `[\"\"]` (root origin only) |\n| `scriptWhitelist` | Whitelist of script SHA-256 hashes allowed to run. If empty, all scripts are allowed. | `null` (all allowed) |\n| `hostWhitelist` | Whitelist of domains allowed to fetch from. If empty, all hosts are allowed (except blacklist). | `null` (all allowed) |\n| `hostBlacklist` | Blacklist of domains not allowed to fetch from. Takes precedence over whitelist. | `[\"*.local\"]` |\n| `maxConcurrentScripts` | Maximum concurrent script executions server-wide. | `100_000` |\n| `maxConcurrentReplication` | Maximum concurrent replication push/pull operations. | `3` |\n| `maxRequests` | Maximum HTTP requests per origin every 500 nanoseconds. | `50` |\n| `maxConcurrentRequests` | Global maximum concurrent HTTP requests across all origins. | `500` |\n| `maxConcurrentFetch` | Maximum concurrent fetch operations (scraping). | `10` |\n\n#### Security Config (`_config/security` Template)\nThe `_config/security` template is installed per-origin to configure authentication and authorization\nsettings. Each tenant can have their own security configuration. It is automatically used with default\nvalues if it does not exist. Security settings are inherited from parent origins if not set.\n\n| Field | Description | Default Value |\n|--------------------------|--------------------------------------------------------------------------------------------------|-------------------------------------------|\n| `mode` | Authentication mode (`jwt` or `jwks`). | `\"\"` (none) |\n| `clientId` | Client ID for OAuth2/JWT authentication. | `\"\"` |\n| `base64Secret` | Base64 encoded secret for JWT validation. | `\"\"` |\n| `secret` | Plain text secret for JWT validation (alternative to base64Secret). | `\"\"` |\n| `jwksUri` | URI to JWKS endpoint for token validation. | `\"\"` |\n| `tokenEndpoint` | OAuth2 token endpoint. | `\"\"` |\n| `scimEndpoint` | SCIM endpoint for user management. | `\"\"` |\n| `usernameClaim` | JWT claim to use as the username. | `sub` |\n| `externalId` | Enable external ID matching for users. | `false` |\n| `emailDomainInUsername` | Include email domain in username. | `false` |\n| `rootEmailDomain` | Root email domain for the server. | `\"\"` |\n| `verifiedEmailClaim` | JWT claim for verified email status. | `verified_email` |\n| `authoritiesClaim` | JWT claim for user authorities/roles. | `auth` |\n| `readAccessClaim` | JWT claim for read access tags. | `readAccess` |\n| `writeAccessClaim` | JWT claim for write access tags. | `writeAccess` |\n| `tagReadAccessClaim` | JWT claim for tag read access. | `tagReadAccess` |\n| `tagWriteAccessClaim` | JWT claim for tag write access. | `tagWriteAccess` |\n| `minRole` | Minimum role for basic access. | `ROLE_ANONYMOUS` |\n| `minWriteRole` | Minimum role for writing. | `ROLE_VIEWER` |\n| `minFetchRole` | Minimum role for fetching external resources. | `ROLE_USER` |\n| `minConfigRole` | Minimum role for admin configuration. | `ROLE_ADMIN` |\n| `minReadBackupsRole` | Minimum role for downloading backups. | `ROLE_ADMIN` |\n| `defaultRole` | Default role given to every user. | `ROLE_ANONYMOUS` |\n| `defaultUser` | Default user tag given to logged out users. | `\"\"` |\n| `defaultReadAccess` | Default read access tags for all users. | `null` |\n| `defaultWriteAccess` | Default write access tags for all users. | `null` |\n| `defaultTagReadAccess` | Default tag read access tags for all users. | `null` |\n| `defaultTagWriteAccess` | Default tag write access tags for all users. | `null` |\n| `maxRequests` | Maximum HTTP requests per origin every 500 nanoseconds for this origin. | `50` |\n| `maxConcurrentScripts` | Maximum concurrent script executions per origin. | `5` |\n| `scriptLimits` | Per-origin script execution limits. Map of selector patterns to max concurrent value. | `{}` (empty) |\n\n### Profiles\nSetting the active profiles is done through the `SPRING_PROFILES_ACTIVE` environment\nvariable. Multiple profiles can be activated by adding them all as a comma\nseparated list.\n\nFor production use the `prod` profile should be active. For testing, the `dev` profile will\nenable additional logging.\n\nTo enable JWT Token Authentication activate the `jwt` profile.\nEither set the `_config/security` template in the Origin receiving traffic:\n```json\n{\n \"mode\": \"jwt\",\n \"clientId\": \"\",\n \"base64Secret\": \"\",\n \"secret\": \"\",\n \"jwksUri\": \"\",\n \"usernameClaim\": \"\",\n \"tokenEndpoint\": \"\",\n \"scimEndpoint\": \"\"\n}\n```\nor set the environment variable overrides:\n- Set `JASPER_OVERRIDE_SECURITY_MODE` to either `jwt` or `jwks` \n- Set `JASPER_OVERRIDE_SECURITY_CLIENT_ID`\n- For `jwt` set `JASPER_OVERRIDE_SECURITY_BASE64_SECRET`\n- For `jwks` set `JASPER_OVERRIDE_SECURITY_JWKS_URI`\n\nIf your user management server supports SCIM, you can enable the `scim` profile to manage users.\nRequires the `_config/security` clientId, secret, and scimEndpoint set. Or\n`JASPER_OVERRIDE_SECURITY_CLIENT_ID`, `JASPER_OVERRIDE_SECURITY_BASE64_SECRET`, \nand `JASPER_OVERRIDE_SECURITY_SCIM_ENDPOINT`\nenvironment variable.\n\nThe `storage` profile is required for backups, caches, or preloading static files. Use the `JASPER_STORAGE` environment\nvariable to change the location of the storage folder.\n\nThe `preload` profile lets you preload static files. Zip files in the preload folder\n`$JASPER_STORAGE/default/preload`. If `$JASPER_LOCAL_ORIGIN` is set,\n`$JASPER_STORAGE/$JASPER_LOCAL_ORIGIN/preload` is used.\n\nThe `scripts` profile enables server side scripting through the `plugin/delta` Plugin.\n\n## Access Control\nJasper uses a combination of simple roles and Tag Based Access Control (TBAC). There are five\nhierarchical roles which cover broad access control, Admin, Mod, Editor, User, and Viewer. The\nAnonymous role is given to users who are not logged in.\nRoles are hierarchical, so they include any permissions granted to a preceding role.\n * `ROLE_ANONYMOUS`: read access to public tags and Refs.\n * `ROLE_VIEWER`: logged in user. Can be given access to private tags and Refs.\n * `ROLE_USER`: can post refs. Has read/write access to their user tag.\n * `ROLE_EDITOR`: can add/remove public tags to any post they have read access to.\n * `ROLE_MOD`: can read/write any tag or ref except plugins and templates.\n * `ROLE_ADMIN`: complete access to origin and sub-origins. Root admin can access all origins Can read/write plugins\nand templates, perform backups and restores.\n\nTags are used to provide fine-grained access to resources. For Refs, the list of tags are considered.\nFor Tags entities, their tag is considered.\n\nThe tag permissions are stored in the User entities:\n * Tag Read Access\n * Can read tag\n * Can add tag\n * Tag Write Access\n * Can edit tag Ext\n * Read Access (Refs and Tags)\n * Can read ref with tag\n * Can read tag\n * Can add tag\n * Write Access (Refs and Tags)\n * No public tags\n * Can write ref with tag\n * Can edit tag Ext\n\nThe protected and private versions of the User entity are merged when\ncalculating the tag access lists.\nIf external IDs are enabled, all matching users by external ID are also merged.\n\n### Special URL Schemas\n\n### Cache\nURLs that have the `cache:` scheme represent items stored in a file cache.\nMost URLs with a resource in a file cache have a standard `https:` scheme,\nas they are just a cache of a resource that exists elsewhere.\nWhen a file is pushed into the cache (such as a generated thumbnail), it is\ngenerated a random `cache:\u003cuuid\u003e` URL.\n\n#### Tag URLs\nURLs that point to a tag, such as `tag:/history` ignore regular tagging access rules.\nInstead, you can access this Ref if you can access the tag it points to.\n\n#### User URLs\nURLs that point to a user tag, such as `tag:/+user/chris` are always owned by the user.\nThese specials URLs can also be used to store per-plugin config data,\nsuch as `tag:/+user/chris?url=tag:/kanban`.\nVisibility of plugin setting can be set on a per-user, per-plugin basis.\nFor convenience, the user URL is used if a blank URL is passed to the tagging response controller.\nThis allows you to quickly ensure settings are initialized and fetch / edit Ref plugins and tags to read settings.\nIf a tag are passed, for example `kanban`, the default is the kanban user settings Ref: `tag:/+user/chris?url=tag:/kanban`.\nIf a blank URL and a blank tag are passed, the default is the generic user settings Ref: `tag:/+user/chris`.\nUser plugins, which follow the template `plugin/user`, may **only** be added to user URL Refs.\n\n### Special Tags\nSome public tags have special significance:\n * `public`: everyone can read\n * `internal`: don't show in UI normally, count separately in metadata\n * `locked`: No edits allowed (tagging is allowed, but not removing plugin data)\n\n### Multi-tenant\nUsers only have read-access to their own origin and sub-origins.\nFor example, if a tenant has origin `@test`, they can also read `@test.other`. As usual, writing to\norigins other than your own is never allowed.\n\n### Access Tokens\nWhen running the system with JWT authentication, roles may be added as claims. \nFor example:\n```json\n{\n \"sub\": \"username\",\n \"auth\": \"ROLE_USER\"\n}\n```\n\nNote: The claim names may be changed with the `JASPER_USERNAME_CLAIM`\nand `JASPER_AUTHORITIES_CLAIM` properties.\n\n## Backup / Restore\nJasper has a built-in backup system for mods and/or admins. Regular users should instead replicate to a separate jasper instance.\nIn order to use the backup system, the `storage` profile must be active.\n\n## Validation\nWhen ingesting entities, Jasper performs the following validation:\n * Fields must not exceed their maximum length\n * URLS are valid according to the regex `(([^:/?#]+):)?(//([^/?#]*))?([^?#]*)(\\?([^#]*))?(#(.*))`\n * Tags are valid according to their respective prefix and the general tag regex `[_+]?[a-z0-9]+([./][a-z0-9]+)*`\n * If a Ref has plugins present, any plugin data must conform to the plugin's schema\n * If an Ext matches a template prefix, any config must conform to all matching templates merged schemas\n\nPlugin and Template schemas are in JTD, which only validates the shape of the data. In addition\nto total bytes of the entity, these are the only server side data validations performed. No security related\nvalidations should be required on Ref or Ext data, so client side validation should be sufficient in most cases.\nError checking should be the first part of any script parsing user input as part of a workflow.\nWe always want to err on the side of accepting well-shaped data rather than rejecting it, as server validation\nerrors rejecting valid user input are infuriating and very common. Error correction can happen as a follow-up step\nif the client validation was somehow circumvented.\n\n## Metadata\nJasper uses metadata generation to pre-compute graph connections. This allows us to store derived data outside\nof the main data model and keeps our queries join free.\n\nJasper generates the following metadata in Refs:\n * List of responses: This is an inverse lookup of the Ref sources. Excludes any Refs with the internal tag.\n * List of internal responses: This is an inverse lookup of the Ref sources that include the internal tag.\n * List of plugin responses: A list of responses with that plugin.\n * List of user plugin responses: A list of responses with that plugin.\n * Obsolete: flag set if another origin contains the newest version of this Ref\n\nMetadata is never transferred during replication. A simplified version is sent over the client API, with\ncounts for each response type, and user plugin responses for the current user.\n\n## Server Scripting\nWhen the `scripts` profile is active, scripts may be attached to Refs with either the `plugin/delta` tag or the\n`plugin/script` tag.\nOnly admin users may install scripts and they run with very few guardrails. A regular user may invoke the script\nby tagging a Ref. The tagged ref will be serialized as UTF-8 JSON and passed to stdin. Environment variables will\ninclude the API endpoint as `JASPER_API`. Return a non-zero error code to fail the script and attach an error log.\nThe script should by writing UTF-8 JSON to stdout of the form:\n\n```json\n{\n \"ref\": [],\n \"ext\": [],\n \"user\": [],\n \"plugin\": [],\n \"template\": []\n}\n```\n\nThese entities will either be created or updated, as necessary.\n\nAdding the `+plugin/error` tag will prevent any further processing. Remove the `+plugin/error` tag to retry.\nYou can also attach any error logs for the user to see by replying to the delta with the `+plugin/log` tag. Logs should\nbe tagged `internal` to prevent clutter, and should match the visibility of the parent delta (`public` or not) with the\nsame owner so the user can clear the logs as desired.\n\n### Delta Scripts\nAny Refs with a `plugin/delta` tag will run the attached script when modified.\n\nYou can use this to mark the input Ref as completed by either:\n1. Removing the `plugin/delta` tag\n2. Adding a `+plugin/delta` Plugin response\n\nHere are examples that reply in all uppercase:\n\n#### Remove the `plugin/delta` tag:\nUse this approach when a script could be run multiple times to create multiple outputs.\n```javascript\nconst whatPlugin = {\n tag: 'plugin/delta/what',\n config: {\n timeoutMs: 30_000,\n language: 'javascript',\n // language=JavaScript\n script: `\n const ref = JSON.parse(require('fs').readFileSync(0, 'utf-8'));\n const louderRef = {\n url: 'yousaid:' + ref.url,\n sources: [ref.url],\n comment: ref.comment.toUpperCase(),\n };\n louderRef.tags = ref.tags = ref.tags.filter(t =\u003e t !== 'plugin/delta/what' \u0026\u0026 !t.startsWith('plugin/delta/what/'));\n console.log(JSON.stringify({\n ref: [ref, louderRef],\n }));\n `,\n },\n};\n```\n\n#### Add the `+plugin/delta` Plugin response:\nThis is the recommended approach as it does need to modify existing Refs and\nis less likely for a bug to cause an infinite loop.\n```javascript\nconst whatPlugin = {\n tag: 'plugin/delta/what',\n config: {\n timeoutMs: 30_000,\n language: 'javascript',\n // language=JavaScript\n script: `\n const ref = JSON.parse(require('fs').readFileSync(0, 'utf-8'));\n const louderRef = {\n url: 'yousaid:' + ref.url,\n sources: [ref.url],\n comment: ref.comment.toUpperCase(),\n tags: ['+plugin/delta/what']\n };\n console.log(JSON.stringify({\n ref: [louderRef],\n }));\n `,\n },\n};\n```\n\n### Cron scripts\nAny Refs with a `plugin/script` tag will run the attached script when the `+plugin/cron` tag is also present.\nThe `+plugin/cron` tag contains plugin data with a default interval of 15 minutes:\n```json\n{\n \"interval\": \"PT15M\"\n}\n```\n\nWhen the `+plugin/cron` tag is present the script will be run repeatedly at the interval specified. Removing the\n`+plugin/cron` tag will disable the script.\n\n#### Example\nHere is a script that outputs the current time:\n```javascript\nconst timePlugin = {\n tag: 'plugin/script/time',\n config: {\n timeoutMs: 30_000,\n language: 'javascript',\n // language=JavaScript\n script: `\n const uuid = require('uuid');\n const ref = JSON.parse(require('fs').readFileSync(0, 'utf-8'));\n const timeRef = {\n url: 'comment:' + uuid.v4(),\n sources: [ref.url],\n comment: '' + new Date(),\n tags: ['public', 'time']\n };\n console.log(JSON.stringify({\n ref: [timeRef],\n }));\n `,\n },\n};\n```\n\n## Remote Origin\nThe `+plugin/origin` tag marks a Ref as a Remote Origin and associates it with a local alias. These may be either pulled from or pushed to.\n```json\n{\n \"optionalProperties\": {\n \"local\": { \"type\": \"string\" },\n \"remote\": { \"type\": \"string\" },\n \"proxy\": { \"type\": \"string\" }\n }\n}\n```\n\n**Local:** Local alias for the remote origin. \n**Remote:** Remote origin to query, or blank for the default. \n**Proxy:** Alternate URL to replicate from. \n\n## Replicating Remote Origin\nThe `+plugin/origin/pull` tag can be used to replicate remote origins. Since this plugin\nextends `+plugin/origin`, we already have the `local` and `remote`\nfields set.\n```json\n{\n \"optionalProperties\": {\n \"query\": { \"type\": \"string\" },\n \"batchSize\": { \"type\": \"int32\" },\n \"websocket\": { \"type\": \"boolean\" },\n \"cachePrefetch\": { \"type\": \"boolean\" },\n \"cacheProxy\": { \"type\": \"boolean\" },\n \"cacheProxyPrefetch\": { \"type\": \"boolean\" },\n \"validatePlugins\": { \"type\": \"boolean\" },\n \"stripInvalidPlugins\": { \"type\": \"boolean\" },\n \"validateTemplates\": { \"type\": \"boolean\" },\n \"stripInvalidTemplates\": { \"type\": \"boolean\" },\n \"addTags\": { \"elements\": { \"type\": \"string\" } },\n \"removeTags\": { \"elements\": { \"type\": \"string\" } }\n }\n}\n```\n\n**Query:** Restrict results using a query. Can not use qualified tags as replication only works on a single origin at\na time. If you want to combine multiple origins into one, create multiple `+plugin/origin` Refs.\n**Batch Size:** The max page size to pull each request. \n**Websocket:** Listen to websocket cursor updates to pull. \n**Cache Prefetch:** Attempt to pull cached files while pulling Refs. \n**Cache Proxy:** Proxy all resources files through this origin's cache, not just cached files.\n**Cache Proxy Prefetch:** Attempt to pull all resources files through this origin's cache while pulling Refs.\n**Validate Plugins:** Flag to enable, disable plugin validation. \n**Strip Invalid Plugins:** If plugin validation is enabled, strip invalid plugins instead of skipping invalid Refs. \n**Validate Templates:** Flag to enable, disable template validation. \n**Strip Invalid Template:** If template validation is enabled, strip invalid templates instead of skipping invalid Exts.\n**Add Tags:** Tags to apply to any Refs replicated from this origin. \n**Remove Tags:** Tags to remove from any Refs replicated from this origin. \n\n## Pushing to a Remote Origin\nThe `+plugin/origin/push` tag can be used to replicate remote origins. Since this plugin\nextends `+plugin/origin`, we already have the `local` and `remote`\nfields set.\n```json\n{\n \"optionalProperties\": {\n \"query\": { \"type\": \"string\" },\n \"batchSize\": { \"type\": \"int32\" },\n \"pushOnChange\": { \"type\": \"boolean\" },\n \"cache\": { \"type\": \"boolean\" }\n }\n}\n```\n\n**Query:** Restrict push using a query. Can not use qualified tags as replication only works on a single origin at\na time. If you want to combine multiple origins into one, create multiple `+plugin/origin` Refs.\n**Batch Size:** The max page size of entities to push. \n**Push On Change:** Push entities immediately after modification.\n**Cache:** Also push cached files. \n\n## Random Number Generator\n\nThe `plugin/rng` tag can be used to generate random numbers. Random numbers are generated whenever editing, creating, or\npushing a Ref replaces an existing Ref of a different origin. When a new random number is generated it is represented in\nhex\nin the tag `+plugin/rng/6d7eb8ebb38a47d29c6a6cbc9156a1a3`, for example. When replicated, random numbers will not be\noverwritten so that spectators may verify the results. Editing of a Ref that is already the latest\nversion across all origins will preserve the existing random number or lack thereof. This ensures random numbers can't\nbe farmed, as you cannot generate a new number without cooperation from another origin.\n\nWhen delegating rng to a trusted server, users push their updates to that server and replicate the results. \nWhen playing on mutually replicating servers, each server is trusted to generate their own rng.\n\n## Release Notes\n* [v1.2](./docs/release-notes/jasper-1.2.md)\n* [v1.1](./docs/release-notes/jasper-1.1.md)\n* [v1.0](./docs/release-notes/jasper-1.0.md)\n\n## Developing\nRun a dev server with `docker compose up`. \nRun a supporting dev database and cache with `docker compose up db redis -d`.\n\n### Build\n\nRun `docker build -t jasper .` to build the project.\n\n### Running unit tests\n\nRun `docker build --target=test -t jasper-tests .` to build the tests. \nRun `docker run -it jasper-tests` to execute the unit tests.\n\n### Running end-to-end tests\n\nSee [Jasper-UI Playwright Tests](https://github.com/cjmalloy/jasper-ui/actions/workflows/playwright.yml).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcjmalloy%2Fjasper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcjmalloy%2Fjasper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcjmalloy%2Fjasper/lists"}