https://github.com/haohanyang/mongodb-toolkit
MongoDB utility libraries for data import/export and schema analysis
https://github.com/haohanyang/mongodb-toolkit
database mongodb node
Last synced: 6 months ago
JSON representation
MongoDB utility libraries for data import/export and schema analysis
- Host: GitHub
- URL: https://github.com/haohanyang/mongodb-toolkit
- Owner: haohanyang
- License: other
- Created: 2025-10-18T09:30:51.000Z (9 months ago)
- Default Branch: master
- Last Pushed: 2025-10-19T16:27:32.000Z (9 months ago)
- Last Synced: 2025-10-19T19:51:10.014Z (9 months ago)
- Topics: database, mongodb, node
- Language: JavaScript
- Homepage: http://npmjs.com/package/mongodb-toolkit
- Size: 152 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG
- License: LICENSE
Awesome Lists containing this project
README
# MongoDB Toolkit
MongoDB Toolkit is a utility library that wraps utilities from [MongoDB Compass](https://github.com/mongodb-js/compass) for exporting data, importing data, analyzing schemas, and more.
## Features
- **Export Data**: Export MongoDB data to CSV or JSON files/streams.
- **Import Data**: Import data from CSV or JSON files/streams into MongoDB.
- **Schema Analysis**: Analyze MongoDB collections to generate schema definitions.
## Installation
To install MongoDB Toolkit, use npm:
```bash
npm install mongodb-toolkit
```
To use the library with typescript, either set `"skipLibCheck": false` in your `tsconfig.json` or install type definitions:
```
npm i @types/json-schema @types/reservoir -D
```
## Usage
Examples use [MongoDB Sample Datasets](https://www.mongodb.com/docs/atlas/sample-data/)
### Export Data to CSV
```ts
import { exportCSV } from 'mongodb-toolkit';
const mongoClient = new MongoClient(process.env['MONGO_URI']!);
const cursor = mongoClient
.db('sample_airbnb')
.collection('listingsAndReviews')
.aggregate([{ $limit: 10 }, { $project: { listing_url: 1 } }]);
exportCSV(cursor, fs.createWriteStream('output.csv'), {
delimiter: ';',
progressCallback: (idx, phase) => {
console.log(phase, idx);
},
})
.then((result) => {
console.log('Export result', result);
})
.catch((err) => {
console.error(err);
})
.finally(() => {
mongoClient.close();
process.exit(0);
});
```
### Import CSV File
CSV content (import.csv):
```csv
id,name
1,John
2,Jane
3,Doe
```
```ts
import { importCSV } from 'mongodb-toolkit';
const mongoClient = new MongoClient(process.env['MONGO_URI']!);
const coll = mongoClient.db('mydb').collection('mycollection');
importCSV(coll, fs.createReadStream('./import.csv'), {
fields: { id: 'int', name: 'string' },
delimiter: ',',
})
.then((result) => {
console.log('Import Result:', result);
})
.catch((err) => {
console.error(err);
})
.finally(() => {
mongoClient.close();
process.exit(0);
});
```
### Analyze Schema
```ts
import { analyzeSchema } from 'mongodb-toolkit';
const mongoClient = new MongoClient(process.env['MONGO_URI']!);
// Sample 100 docs
const cursor = mongoClient
.db('sample_mflix')
.collection('comments')
.aggregate([{ $limit: 100 }]);
const controller = new AbortController();
// Abort the analysis after 1 second
setTimeout(() => {
controller.abort();
}, 1000);
analyzeSchema(cursor, { abortSignal: controller.signal })
.then((schema) => schema.getMongoDBJsonSchema())
.then((jsonSchema) => {
if (!schema) {
// Aborted
throw new Error('Schema analysis aborted');
}
console.log(JSON.stringify(jsonSchema, null, 2));
})
.catch((err) => {
console.error(err);
})
.finally(() => {
mongoClient.close();
process.exit(0);
});
```
Output
```json
{
"bsonType": "object",
"required": ["_id", "date", "email", "movie_id", "name", "text"],
"properties": {
"_id": {
"bsonType": "objectId"
},
"date": {
"bsonType": "date"
},
"email": {
"bsonType": "string"
},
"movie_id": {
"bsonType": "objectId"
},
"name": {
"bsonType": "string"
},
"text": {
"bsonType": "string"
}
}
}
```