{"id":29790926,"url":"https://github.com/metabrainz/mb-solr","last_synced_at":"2026-03-09T13:02:00.615Z","repository":{"id":45444541,"uuid":"22340512","full_name":"metabrainz/mb-solr","owner":"metabrainz","description":"MusicBrainz Solr query response writer","archived":false,"fork":false,"pushed_at":"2025-05-18T20:58:08.000Z","size":409,"stargazers_count":3,"open_issues_count":1,"forks_count":13,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-05-18T21:30:43.313Z","etag":null,"topics":["java","musicbrainz","search","solr"],"latest_commit_sha":null,"homepage":"https://musicbrainz.org/doc/Development/Search_Architecture","language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/metabrainz.png","metadata":{"funding":{"custom":["https://metabrainz.org/donate"],"github":["metabrainz"]},"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2014-07-28T11:38:24.000Z","updated_at":"2025-05-18T20:56:37.000Z","dependencies_parsed_at":"2024-05-06T15:47:50.487Z","dependency_job_id":"8349d5b8-634b-4bf9-b3b2-1fe0ba2d4911","html_url":"https://github.com/metabrainz/mb-solr","commit_stats":null,"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"purl":"pkg:github/metabrainz/mb-solr","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metabrainz%2Fmb-solr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metabrainz%2Fmb-solr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metabrainz%2Fmb-solr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metabrainz%2Fmb-solr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/metabrainz","download_url":"https://codeload.github.com/metabrainz/mb-solr/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/metabrainz%2Fmb-solr/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267442905,"owners_count":24087893,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-27T02:00:11.917Z","response_time":82,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["java","musicbrainz","search","solr"],"created_at":"2025-07-28T00:13:11.193Z","updated_at":"2026-03-09T13:01:55.563Z","avatar_url":"https://github.com/metabrainz.png","language":"Java","funding_links":["https://metabrainz.org/donate","https://github.com/sponsors/metabrainz"],"categories":[],"sub_categories":[],"readme":"# MusicBrainz Solr\n\nThis package includes a\n[QueryResponseWriter](https://solr.apache.org/guide/solr/9_7/query-guide/response-writers.html)\nfor Apache Solr that will generate\n[mmd-schema](https://github.com/metabrainz/mmd-schema) compliant responses\nfor Solr cores running on an [mbsssss](https://github.com/metabrainz/mbsssss) schema.\n\n## Prerequisites\n\nIn May 2025, Solr data for indexing the whole MusicBrainz database takes ~75 GB of disk space.\nSince indexing takes hours, you might want to load the Zstandard-compressed backup archives.\nThose archives take ~60 GB of disk space and their temporarily extracted files take ~75 GB more.\nHence, it is currently recommended to provision at least 250 GB of disk space for Solr.\n\nRecommendations:\n* RAM: 4 GB\n* CPU: 8 threads, x86-64 architecture\n* Disk Space: 250 GB (or 100 GB if you don't load backups.)\n\nRequired software:\n* Git\n* Docker 2\n  (or Java SE 17, Maven 3, and Solr 9.7.0)\n\n## Installation\n\nThe generally recommended installation method is using [Docker](https://docs.docker.com).\nA Docker image is available through the eponymous [Docker Hub repository](https://hub.docker.com/r/metabrainz/mb-solr) for each release of MusicBrainz Solr.\nThese images can be used through the [MusicBrainz Docker Compose project](https://github.com/metabrainz/musicbrainz-docker) which comes with instructions.\n\nFor [development](#development) or other purposes, these images can also be run alone, see the below subsection “[Installation without MusicBrainz Server](#installation-without-musicbrainz-server)”.\n\nTo set up a cluster of Solr nodes, it can be preferred to install to the host system directly, see the below subsection “[Installation without Docker](#installation-without-docker)” and the [Apache Solr Reference Guide 9.7](https://solr.apache.org/guide/solr/9_7/).\n\n### Installation without MusicBrainz Server\n\nClone the repository with Git:\n\n```bash\ngit clone --recursive https://github.com/metabrainz/mb-solr.git\n```\n\nRun it alone on port 8983 with Docker:\n\n```bash\ndocker compose up\n```\n\n### Installation without Docker\n\n:warning: This section is outdated as it predates upgrading to SolrCloud 9.\n\n:newspaper: The MusicBrainz Solr cluster powering musicbrainz.org is using [Ansible](https://docs.ansible.com/) for deployment.\n\n#### Installing brainz-mmd2-jaxb\n\nClone the repository with Git:\n\n```bash\ngit clone https://github.com/metabrainz/mmd-schema.git\n```\n\nAnd install the package:\n\n```bash\ncd mmd-schema/brainz-mmd2-jaxb\nmvn install\n```\n\n#### Installing the query writer\n\nClone the repository with Git:\n\n```bash\ngit clone --recursive https://github.com/metabrainz/mb-solr.git\n```\n\nNavigate to the **mb-solr** folder in a terminal and build a JAR\nfile:\n\n```bash\nmvn package\n```\n\nThis will create a file called\n**solrwriter-0.0.1-SNAPSHOT-jar-with-dependencies.jar** in the **target** folder.\n\nNow you need to make this JAR file available to all Solr cores that need it.\nThe easiest option is to configure a **sharedLib** in your\n[solr.xml](https://solr.apache.org/guide/solr/9_7/configuration-guide/configuring-solr-xml.html)\nand put the JAR file into that.\n\nAll that's left to do now is enabling the Query Response Writers in your cores\n[solrconfig.xml](https://solr.apache.org/guide/solr/9_7/configuration-guide/configuring-solrconfig-xml.html).\nTo do that, add the following lines as children of the **config** element:\n\n```xml\n\u003cqueryResponseWriter name=\"mbxml\" class=\"org.musicbrainz.search.solrwriter.MBXMLWriter\"\u003e\n    \u003cstr name=\"entitytype\"\u003e$entitytype\u003c/str\u003e\n\u003c/queryResponseWriter\u003e\n\u003cqueryResponseWriter name=\"mbjson\" class=\"org.musicbrainz.search.solrwriter.MBJSONWriter\"\u003e\n    \u003cstr name=\"entitytype\"\u003e$entitytype\u003c/str\u003e\n\u003c/queryResponseWriter\u003e\n```\n\nThe solrconfig.xml of the cores defined by\n[mbsssss](https://github.com/metabrainz/mbsssss) already includes this snippet, as\nwell as the **sharedLib** configuration in the solr.xml file.\n\n**$entitytype** needs to be replaced by the entity type of the documents in the store.\nValid values are:\n\n- annotation\n- area\n- artist\n- cdstub\n- editor\n- event\n- instrument\n- label\n- place\n- recording\n- release\n- release_group\n- series\n- tag\n- work\n- url\n\nNow the core needs to be reloaded.\nAfter that, two new values for the\n[wt paramter](https://solr.apache.org/guide/solr/9_7/query-guide/common-query-parameters.html#wt-parameter)\nare available:\n\n- **mbxml**, which returns mmd-compliant XML documents\n- **mbjson**, which returns JSON document as described by the\n  [MusicBrainz API documentation page](https://musicbrainz.org/doc/MusicBrainz_API)\n\nAt the moment, the\n[field list](https://solr.apache.org/guide/solr/9_7/query-guide/common-query-parameters.html#fl-field-list-parameter)\nparameter of each query needs to include the **score** field for the code to\nwork correctly.\n\nA branch of the MusicBrainz server that can query a Solr server with this\nQueryResponseWriter is available on\n[GitHub](https://github.com/mineo/musicbrainz-server/tree/solr-search).\n\n## Development\n\nIf you followed “[Installation without MusicBrainz Server](#installation-without-musicbrainz-server)”,\njust stop the last command and run it again after having made some changes.\n\nOtherwise, if you need it to be tested with either the indexer or MusicBrainz Server:\n\n1. Make some changes in your local Git clone (of MusicBrainz Solr)\n   (The two next steps are optional but recommended to better identify images.)\n2. Commit these changes to your local Git branch.\n3. Tag this commit with a meaningful version:\n\n   ```bash\n   git tag adastrawberry-1969\n   ```\n\n4. Build a docker image:\n\n   ```bash\n   ./build.sh\n   ```\n\n5. Use this tag to [set `MB_SOLR_VERSION` in MusicBrainz Docker Compose project](https://github.com/metabrainz/musicbrainz-docker?tab=readme-ov-file#local-development-of-musicbrainz-solr).\n\nSee also [The Debugger’s Guide to MusicBrainz Solr](HACKING.md).\n\n## Helper commands\n\nSolrCloud collections backups from the MusicBrainz Solr cluster\npowering musicbrainz.org are made available twice a week to save\nmirror owners the initial cost of indexing the whole MusicBrainz data\nfrom scratch which can take hours and significant resources.\nThose backups are provided as Zstandard-compressed tar archives.\nHelper commands to handle these archives are provided in Docker images:\n\n```bash\nfetch-backup-archives\nload-backup-archives\nremove-backup-archives\n```\n\nIn May 2025, the measured time for fetching 60 GB was 12 min (depending on your bandwidth,)\nand the measured time for loading was 12 min (depending on your CPU/RAM/Disk speed.)\n\nA fourth helper command allows to delete all the Solr data if needed:\n\n```bash\ndelete-indexed-documents\n```\n\nEach of these commands is self-documented through the option `--help`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetabrainz%2Fmb-solr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmetabrainz%2Fmb-solr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmetabrainz%2Fmb-solr/lists"}