https://github.com/meilisearch/meilisearch-importer
A CLI to import massive CSV and NdJson into Meilisearch
https://github.com/meilisearch/meilisearch-importer
csv importer json jsonlines meilisearch ndjson
Last synced: 4 months ago
JSON representation
A CLI to import massive CSV and NdJson into Meilisearch
- Host: GitHub
- URL: https://github.com/meilisearch/meilisearch-importer
- Owner: meilisearch
- License: mit
- Created: 2022-10-22T00:22:24.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2025-01-12T23:01:35.000Z (9 months ago)
- Last Synced: 2025-06-09T03:43:50.300Z (4 months ago)
- Topics: csv, importer, json, jsonlines, meilisearch, ndjson
- Language: Rust
- Homepage: https://crates.io/crates/meilisearch-importer
- Size: 119 KB
- Stars: 31
- Watchers: 2
- Forks: 4
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Meilisearch Importer
The most efficient CLI tool to import massive CSVs, NSJSON or JSON (array of objects) into Meilisearch.
This tool has been tested with multiple datasets from hundreds of thousand documents to some with more than forty millions documents. The progress bar is very handy in this case.
## Features
- Uploads millions of documents to Meilisearch.
- Automatically retries on error.
- Shows the upload progress along with the estimated time of arrival (ETA).
- [Works on the Cloud](https://www.meilisearch.com/pricing) and on self-hosted instances.
## Installation
You can download the latest version of this tool [on the release page](https://github.com/meilisearch/meilisearch-importer/releases).
## Example Usage
### Send Documents to the Cloud
It's straightforward to [create a project on the Cloud](https://www.meilisearch.com/pricing) and send your documents into it.
If you cannot send your dataset directly from the website by drag-and-dropping it, this tool is perfect for you. You can send them by running the following command:
```bash
meilisearch-importer \
--url 'https://ms-************.sfo.meilisearch.io'
--index crunchbase \
--primary-key uuid \
--api-key 'D2jkS***************' \
--files ./dataset/organizations.csv
```
### Send Documents to a Local Instance
This tool is also useful when you want to test Meilisearch locally. The only mandatory parameters to define are the URL, the index name and your dataset.
However, you can also increase the batch size to make meilisearch index faster.
```bash
meilisearch-importer \
--url 'http://localhost:7700'
--index movies \
--files movies.json \
--batch-size 100MB
```