{"id":13530951,"url":"https://github.com/zopyx/zopyx.typesense","last_synced_at":"2025-04-14T01:10:17.481Z","repository":{"id":57478525,"uuid":"434925754","full_name":"zopyx/zopyx.typesense","owner":"zopyx","description":"Typesense integration for Plone 6","archived":false,"fork":false,"pushed_at":"2023-12-08T08:52:27.000Z","size":50598,"stargazers_count":4,"open_issues_count":1,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-14T01:09:51.081Z","etag":null,"topics":["fulltext-search","plone","typesense"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zopyx.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.rst","contributing":null,"funding":null,"license":"LICENSE.GPL","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-04T14:38:07.000Z","updated_at":"2025-03-06T09:26:59.000Z","dependencies_parsed_at":"2024-08-01T07:32:48.756Z","dependency_job_id":"09846812-3496-4ad2-91e4-798cfec5eccd","html_url":"https://github.com/zopyx/zopyx.typesense","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zopyx%2Fzopyx.typesense","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zopyx%2Fzopyx.typesense/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zopyx%2Fzopyx.typesense/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zopyx%2Fzopyx.typesense/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zopyx","download_url":"https://codeload.github.com/zopyx/zopyx.typesense/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248804825,"owners_count":21164135,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fulltext-search","plone","typesense"],"created_at":"2024-08-01T07:00:58.316Z","updated_at":"2025-04-14T01:10:17.450Z","avatar_url":"https://github.com/zopyx.png","language":"Python","funding_links":[],"categories":["Searching and Categorizing"],"sub_categories":[],"readme":"# zopyx.typesense\n\n## What is zopyx.typesense\n\n`zopyx.typesense` in an add-on for Plone 6 that provides an integration with\nthe search engine [Typesense](https://typesense.org/). The functionality is\nsimilar with `collective.solr`.\n\n![Typesense Features](https://github.com/zopyx/zopyx.typesense/raw/master/docs/typesense-features.png)\n\nThe reasons for using Typesense are\n\n- very easy to install (single binary or via Docker)\n- multi-field text indexes\n- auto-generated (but highly customizable) search UI\n- optional facted search for refining fulltext queries\n- minimal invasive into Plone (does not replace the ZCatalog)\n- very fast search\n- extensible and customizable\n- scalable (through a Typesense cluster)\n- federated search over multiple collections (aka Plone sites)\n- open-source\n- on-premise or Typesense cloud (commercial offering)\n\nClick [here](https://user-images.githubusercontent.com/594239/150671828-f6a4c993-6afa-440b-af76-66de5ff94fe5.mp4) for an intro video.\n\n### zopyx.typesense is not\n\n- a replacement of the ZCatalog/portal_catalog (document changes are pushed to\n  Typesense as an independent entity completely decoupled from Plone's\n  portal_catalog)\n- a drop-in replacement of the Plone search. `zopyx.typesense` comes with its own\n  configurable auto-generated search UI\n\n## Requirements\n\n- Plone 6 (tested)\n- Plone 5.2 (untested, supposed to work for Dexterity-only sites, under Python 3)\n\n## Architecture\n\nTypesense runs as dedicated service either on-premise or somewhere in the cloud.\n\n![Architecture](https://github.com/zopyx/zopyx.typesense/raw/master/docs/Typesense.png\n\n### Indexing\n`zopyx.typesense` pushes content changes (new documents, updated documents, deleted documents) through\nPlone lifecycle hooks into Typesense.\n\n\n### Searching\nThe search interaction between browser and Typesense happens directly *without*\nPlone being in-between.  When you open the `@@typesense-search` view, the\nsearch configuration (as defined through the search template `search.pt` and an\nassociated `app.js` file) is loaded from Plone for auto-generating the search\nUI (search input, faceted search) within the browser. The search query is\ndirectly send to Typesense and the results are directly shown within the\nbrowser as soon as they are available.  Plone is *not* involved in the search.\nNo additional filtering (e.g. regarding access control) happens through Plone.\n\n\n## Installation\n\nAdd `zopyx.typesense` to your buildout, re-run buildout and install it within Plone.\n\nFor Typesense installation, please check the installation docs of Typesense\n(either for installation through Docker or through the standalone binary).\n\nThere is no public release at this time. You need to install `zopyx.typesense`\nusing `mr.developer` as source checkout from Github.\n  \n## Configuration\n\nThe `Typesense settings` within the Plone controlpanel:\n\n![Typesense settings](https://github.com/zopyx/zopyx.typesense/raw/master/docs/typesense-settings.png)\n\n- `Name of Typesense collection` - must be a unique name for the document pool\n  of your Plone site\n- `API Key` - the administrative API key (as configured in Typesense) \n- `Search API Key` - the search API key (as configured in Typesense) \n- `URL of Typesense node X` - the URL(s) of the Typesense node or Typesense\n  cluster\n- `Collection schema` - the schema of the Typesense collecton (see Typesense\n  docs)\n\nThe `Typesense administration` within the Plone controlpanel:\n\n![Typesense administration](https://github.com/zopyx/zopyx.typesense/raw/master/docs/typesense-administration.png)\n\n## Example setup\n\nFor a typical Plone dev setup which Plone running on localhost, you want to run\nTypesense (for dev purpose) on the same machine using Docker:\n\n```\ndocker run -p 8108:8108 -v /tmp/data:/data typesense/typesense:0.22.1 --data-dir /data --api-key=some_api_key --search-only-api-key=some_search_api_key --enable-cors\n```\n\nThis command will store the Typesense data files under `/tmp/data` and open a\nconnection on port `8108`.  You need to specify two api keys: one for admin\naccess and one for search api.\n\nAfter installing `zopyx.typesense` in Plone, you would specify both API keys in\nthe Typesense control panel within Plone along with `http://localhost:8108` as\nURL for Typesense node 1.\n\n\n## Search UI\n\n![Typesense search](https://github.com/zopyx/zopyx.typesense/raw/master/docs/typesense-search.png)\n\nThe search UI is auto-generated from a minimal HTML template which defines the\nbasic layout together with the widgets to be used [see\nhere](src/zopyx/typesense/browser/search.pt).\n\nThe integration of the template with the Typesense UI generator is impleted\nthrough some lines of Javascript code in\n[src/zopyx/typesense/browser/static/app.js](src/zopyx/typesense/browser/static/app.js).\n\n\n## Views\n\n- `@@typesense-search` renders the main (auto-generated) search form. This view can be applied on\n  any folder level. This view applied on a folder implies a filter by subpath (only indexed content\n  with the given folder will be searched).\n\n- `@@typesense-settings`  renders the Plone control panel of Typesense\n\n- `@@typesense-settings`  renders the Plone control panel of Typesense\n\n- `@@typesense-admin` renders the administration control panel for Typesense\n\n- `@@typesense-indexed-content` applied on any arbitrary content object will\n  display all data indexed for the given content object\n\n## Indexing integration\n\n`zopyx.typesense` integrates into Plone using the lifecycle event for content\nbeing added, updated, deleted (plus workflow changes). By default,\n`zopyx.typesense` indexes all metadata of all content objects (basically Dublin\nCore) plus the textual content of a content objects (title, description and the\ncontent of all `RichText` fields).  Similar to Plone's `SearchableText` index,\nall textual content is being indexed within Typesense in the `text` field.\n\nIt is possible to override the indexing behavior per-content-type using an\nindexing adapter implementing `ITypesenseIndexDataProvider`. See [here for an\nexample](https://github.com/zopyx/zopyx.typesense/blob/master/tree/master/src/zopyx/typesense/adapters).\n\n### Indexing binary content and office formats\n\nThe indexer for the `File` content-type has optional support for indexing\nbinary content or office formats using Apache [TIKA](https://tika.apache.org/).\nThe server URL of the (optional) Tika server must be configured through the\n`Typesense` control panel. The most simple approach for running Tika is by\nusing Docker:\n\n```\ndocker run -d -p 9998:9998 apache/tika\n```\n\nConfigure Tika within the Typesense control panel as `http://localhost:9998`.\nEnsure that the configuration regarding host, IP address and port numbers fit\nyour security requirements (usually, you don't want expose an internal service\non public IP addresses, but localhost only). \n\n\n## Bulk reindexing\nThe Typesense admin control panel contains a button `Reindex all` which will \n\n- create a new collection whose name is defined as \u003ccollection_name\u003e-\u003ccurrent date\u003e\n- submit all Dexterity content objects for reindexing\n- alias the new collection with the official name of the configured collection\n\nThis means that an existing Typesense collection remains online until the\ncomplete reindexing process has finished. The Typesense collection with the\nfresh index content replaces then the old collection in an atomic operation.\nThere should be no impact on the availability of the search functionality\nduring reindexing.\n\n## Custom search views\n\nThe default Typesense search view `@@typesense-search` should work for most\ncommon usecases of a Plone site with the default content types. If you need\nsomething different, you have two options:\n\n- customize `search.pt` and `app.js` according to your own need (e.g. using\n  `z3c.jbot`)\n- create your search view(s) with your own search template and the related JS\n  configuration\n\n\n\n## Transactions and eventual consistency\n\nAll indexing/unindexing operations happen asyncronously to Plone and outside\nPlone's transaction system.  So, content changes might be available in\nTypesense with a short delay. \n\n## Cavecats and known issues\n\n- `zopyx.typesense` does not integrate (by-design) with Plone's security and\n  access model.  The main purpose of `zopyx.typesense` is to act as a search\n  engine for public sites.  So it is recommended at this time to index only\n  public content.\n\n## Resources\n\n- [Typesense website](https://typesense.org)\n- [Typesense API](https://typesense.org/docs/0.22.2/api)\n- [InstantSearch.js](https://www.algolia.com/doc/api-reference/widgets/js)\n\n## Author\n\nAndreas Jung | info@zopyx.com | www.zopyx.com\n\nPaid service for `zopyx.typesense` is available on request.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzopyx%2Fzopyx.typesense","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzopyx%2Fzopyx.typesense","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzopyx%2Fzopyx.typesense/lists"}