Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/music-of-the-ainur/mongodb.almaren
MongoDB connector for Almaren Framework
https://github.com/music-of-the-ainur/mongodb.almaren
Last synced: 2 months ago
JSON representation
MongoDB connector for Almaren Framework
- Host: GitHub
- URL: https://github.com/music-of-the-ainur/mongodb.almaren
- Owner: music-of-the-ainur
- License: apache-2.0
- Created: 2020-05-08T14:01:58.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2023-11-03T04:48:44.000Z (about 1 year ago)
- Last Synced: 2023-12-15T06:29:13.220Z (about 1 year ago)
- Language: Scala
- Size: 837 KB
- Stars: 0
- Watchers: 4
- Forks: 10
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# MongoDB Connector
[](https://github.com/music-of-the-ainur/mongodb.almaren/actions/workflows/mongodb-almaren-githubactions.yml)
To add Mongodb Almaren dependency to your sbt build:
```
libraryDependencies += "com.github.music-of-the-ainur" %% "mongodb-almaren" % "0.0.9-3.4"
libraryDependencies += "org.mongodb.spark" % "mongo-spark-connector" % "10.2.0"
```
To run in spark-shell:
```
spark-shell --master local[*] --packages "com.github.music-of-the-ainur:almaren-framework_2.12:0.9.10-3.4,com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.9-3.4,org.mongodb.spark:mongo-spark-connector:10.2.0"
```
MongoDB Connector is available in [Maven Central](https://mvnrepository.com/artifact/com.github.music-of-the-ainur)
repository.
| version | Connector Artifact |
|----------------------------|----------------------------------------------------------------|
| Spark 3.4.x and scala 2.13 | `com.github.music-of-the-ainur:mongodb-almaren_2.13:0.0.9-3.4` |
| Spark 3.4.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.9-3.3` |
| Spark 3.3.x and scala 2.13 | `com.github.music-of-the-ainur:mongodb-almaren_2.13:0.0.9-3.3` |
| Spark 3.3.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.9-3.3` |
| Spark 3.2.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.9-3.2` |
| Spark 3.1.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.9-3.1` |
| Spark 2.4.x and scala 2.12 | `com.github.music-of-the-ainur:mongodb-almaren_2.12:0.0.9-2.4` |
| Spark 2.4.x and scala 2.11 | `com.github.music-of-the-ainur:mongodb-almaren_2.11:0.0.9-2.4` |
## Source and Target
Connector was implemented using: [https://github.com/mongodb/mongo-spark](https://github.com/mongodb/mongo-spark).
## Example
### Source
Parameters:
| Parameters | Description |
|------------|-------------------------|
| hosts | localhost:27017 |
| database | foo |
| collection | bar |
| user | username |
| password | password |
|stringPrefix| this is used to specify MongoDb Connection type(srv) https://docs.mongodb.com/manual/reference/connection-string/|
| options | extra connector options |
#### For Standalone Connection Type Mongo
```scala
import com.github.music.of.the.ainur.almaren.Almaren
import com.github.music.of.the.ainur.almaren.builder.Core.Implicit
import com.github.music.of.the.ainur.almaren.mongodb.MongoDb.MongoImplicit
almaren.builder.sourceMongoDb("localhost","foo","bar")
```
#### For srv Connection Type Mongo
```scala
import com.github.music.of.the.ainur.almaren.Almaren
import com.github.music.of.the.ainur.almaren.builder.Core.Implicit
import com.github.music.of.the.ainur.almaren.mongodb.MongoDb.MongoImplicit
almaren.builder.sourceMongoDb("localhost","foo","bar",None,None,Some("srv"))
```
Parameters for Uri:
| Parameters | Description |
|------------|-------------------------------------------------------------------------------------------------------------------|
| uri | mongodb://localhost:27017/foo |
| collection | bar |
| options | extra connector options |
#### For Connection Uri Type Mongo
```scala
import com.github.music.of.the.ainur.almaren.Almaren
import com.github.music.of.the.ainur.almaren.builder.Core.Implicit
import com.github.music.of.the.ainur.almaren.mongodb.MongoDb.MongoImplicit
almaren.builder.sourceMongoDbUri("mongodb://localhost:27017/foo","bar")
```
### Target
Parameters:
| Parameters | Description |
|------------|-------------------------|
| hosts | localhost:27017 |
| database | foo |
| collection | bar |
| user | username |
| password | password |
|stringPrefix| this is used to specify MongoDb Connection type(srv) https://docs.mongodb.com/manual/reference/connection-string/|
| options | extra connector options |
| saveMode | SaveMode.Overwrite |
#### For Standalone Connection Type Mongo
```scala
import org.apache.spark.sql.SaveMode
import com.github.music.of.the.ainur.almaren.Almaren
import com.github.music.of.the.ainur.almaren.builder.Core.Implicit
import com.github.music.of.the.ainur.almaren.mongodb.MongoDb.MongoImplicit
almaren.builder.targetMongoDb("localhost","foo","bar", saveMode = SaveMode.Overwrite)
```
#### For srv Connection Type Mongo
```scala
import org.apache.spark.sql.SaveMode
import com.github.music.of.the.ainur.almaren.Almaren
import com.github.music.of.the.ainur.almaren.builder.Core.Implicit
import com.github.music.of.the.ainur.almaren.mongodb.MongoDb.MongoImplicit
almaren.builder.targetMongoDb("localhost","foo","bar",None,None,Some("srv"), saveMode = SaveMode.Overwrite)
```
Parameters for Uri:
| Parameters | Description |
|------------|-------------------------------------------------------------------------------------------------------------------|
| uri | mongodb://localhost:27017/foo |
| collection | bar |
| options | extra connector options |
| saveMode | SaveMode.Overwrite |
#### For Connection Uri Type Mongo
```scala
import com.github.music.of.the.ainur.almaren.Almaren
import com.github.music.of.the.ainur.almaren.builder.Core.Implicit
import com.github.music.of.the.ainur.almaren.mongodb.MongoDb.MongoImplicit
almaren.builder.targetMongoDbUri("mongodb://localhost:27017/foo","bar", saveMode = SaveMode.Overwrite)
```