{"id":26234865,"url":"https://github.com/flowpack/flowpack.elasticsearch.contentrepositoryqueueindexer","last_synced_at":"2025-04-06T02:08:55.464Z","repository":{"id":29579752,"uuid":"33119465","full_name":"Flowpack/Flowpack.ElasticSearch.ContentRepositoryQueueIndexer","owner":"Flowpack","description":"Neos CMS ElasticSearch indexer based on the Flowpack JobQueue (to handle big indexing tags, +50'000 nodes)","archived":false,"fork":false,"pushed_at":"2025-03-27T15:17:45.000Z","size":231,"stargazers_count":8,"open_issues_count":3,"forks_count":25,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-27T15:53:39.677Z","etag":null,"topics":["beanstalkd","elasticsearch","flow-framework","neoscms"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Flowpack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-03-30T11:33:21.000Z","updated_at":"2025-03-27T15:16:24.000Z","dependencies_parsed_at":"2025-03-13T02:45:14.626Z","dependency_job_id":null,"html_url":"https://github.com/Flowpack/Flowpack.ElasticSearch.ContentRepositoryQueueIndexer","commit_stats":null,"previous_names":[],"tags_count":28,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Flowpack%2FFlowpack.ElasticSearch.ContentRepositoryQueueIndexer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Flowpack%2FFlowpack.ElasticSearch.ContentRepositoryQueueIndexer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Flowpack%2FFlowpack.ElasticSearch.ContentRepositoryQueueIndexer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Flowpack%2FFlowpack.ElasticSearch.ContentRepositoryQueueIndexer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Flowpack","download_url":"https://codeload.github.com/Flowpack/Flowpack.ElasticSearch.ContentRepositoryQueueIndexer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247423515,"owners_count":20936626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beanstalkd","elasticsearch","flow-framework","neoscms"],"created_at":"2025-03-13T02:29:34.292Z","updated_at":"2025-04-06T02:08:55.397Z","avatar_url":"https://github.com/Flowpack.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Neos CMS Elasticsearch indexer based on a job queue\n\n[![Latest Stable Version](https://poser.pugx.org/flowpack/elasticsearch-contentRepositoryQueueIndexer/v/stable)](https://packagist.org/packages/flowpack/elasticsearch-contentRepositoryQueueIndexer) [![Total Downloads](https://poser.pugx.org/flowpack/elasticsearch-contentRepositoryQueueIndexer/downloads)](https://packagist.org/packages/flowpack/elasticsearch-contentRepositoryQueueIndexer)\n\nThis package can be used to index a huge amount of nodes in Elasticsearch indexes. This\npackage use the Flowpack JobQueue packages to handle the indexing asynchronously.\n\n**Topics**\n\n* [Installation](#installation-and-configuration)\n* [Indexing](#indexing)\n* [SupervisorD configuration](#supervisord-configuration)\n* [Update Instructions](#update-instructions)\n\n\n# Installation and Configuration\n\nYou need to install the correct Queue package based on your needs.\n\nAvailable packages:\n\n  - [sqlite](https://packagist.org/packages/flownative/jobqueue-sqlite)\n  - [beanstalkd](https://packagist.org/packages/flowpack/jobqueue-beanstalkd)\n  - [doctrine](https://packagist.org/packages/flowpack/jobqueue-doctrine)\n  - [redis](https://packagist.org/packages/flowpack/jobqueue-redis)\n\nPlease check the package documentation for specific configurations.\n\nThe default configuration uses the FakeQueue, which is provided by the JobQueue.Common package. Note that with that package jobs are executed synchronous with the `flow nodeindexqueue:build` command.\n\nCheck the ```Settings.yaml``` to adapt based on the Queue package, you need to adapt the ```className```:\n\n    Flowpack:\n      JobQueue:\n        Common:\n          presets:\n            'Flowpack.ElasticSearch.ContentRepositoryQueueIndexer':\n              className: 'Flowpack\\JobQueue\\Common\\Queue\\FakeQueue'\n\nIf you use the [doctrine](https://packagist.org/packages/flownative/jobqueue-doctrine) package you have to set the ```tableName``` manually:\n\n    Flowpack:\n      JobQueue:\n        Common:\n          presets:\n            'Flowpack.ElasticSearch.ContentRepositoryQueueIndexer':\n              className: 'Flowpack\\JobQueue\\Doctrine\\Queue\\DoctrineQueue'\n          queues:\n            'Flowpack.ElasticSearch.ContentRepositoryQueueIndexer':\n              options:\n                tableName: 'flowpack_jobqueue_QueueIndexer'\n            'Flowpack.ElasticSearch.ContentRepositoryQueueIndexer.Live':\n              options:\n                tableName: 'flowpack_jobqueue_QueueIndexerLive'\n\n# Indexing\n\n## Batch Indexing\n\n### How to build indexing jobs\n\n    flow nodeindexqueue:build --workspace live\n\n#### How to process indexing jobs\n\nYou can use this CLI command to process indexing job:\n\n    flow nodeindexqueue:work --queue batch\n\n## Live Indexing\n\nYou can disable async live indexing by editing ```Settings.yaml```:\n\n    Flowpack:\n      ElasticSearch:\n        ContentRepositoryQueueIndexer:\n          enableLiveAsyncIndexing: false\n\nYou can use this CLI command to process indexing job:\n\n    flow nodeindexqueue:work --queue live\n\n# Supervisord configuration\n\nYou can use tools like ```supervisord``` to manage long running processes. Bellow you can find a basic configuration:\n\n    [supervisord]\n\n    [supervisorctl]\n\n    [program:elasticsearch_batch_indexing]\n    command=php flow nodeindexqueue:work --queue batch\n    stdout_logfile=AUTO\n    stderr_logfile=AUTO\n    numprocs=4\n    process_name=elasticsearch_batch_indexing_%(process_num)02d\n    environment=FLOW_CONTEXT=\"Production\"\n    autostart=true\n    autorestart=true\n    stopsignal=QUIT\n\n    [program:elasticsearch_live_indexing]\n    command=php flow nodeindexqueue:work --queue live\n    stdout_logfile=AUTO\n    stderr_logfile=AUTO\n    numprocs=4\n    process_name=elasticsearch_live_indexing_%(process_num)02d\n    environment=FLOW_CONTEXT=\"Production\"\n    autostart=true\n    autorestart=true\n    stopsignal=QUIT\n\n# Update Instructions\n\n## Breaking change after an upgrade to 3.0\n\n* Previously the Beanstalk queue package was installed by default, this is no longer\nthe case.\n\n## Breaking change after an upgrade to 5.0\n\n* The beanstalk queue configuration is removed. The FakeQueue is used if not configured to another queuing package.\n\nLicense\n-------\n\nLicensed under MIT, see [LICENSE](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflowpack%2Fflowpack.elasticsearch.contentrepositoryqueueindexer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflowpack%2Fflowpack.elasticsearch.contentrepositoryqueueindexer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflowpack%2Fflowpack.elasticsearch.contentrepositoryqueueindexer/lists"}