https://github.com/conduitio-labs/conduit-connector-weaviate
Conduit connector for Weaviate
https://github.com/conduitio-labs/conduit-connector-weaviate
conduit go golang weaviate
Last synced: 3 months ago
JSON representation
Conduit connector for Weaviate
- Host: GitHub
- URL: https://github.com/conduitio-labs/conduit-connector-weaviate
- Owner: conduitio-labs
- Created: 2023-08-04T15:10:47.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2025-02-12T18:10:53.000Z (4 months ago)
- Last Synced: 2025-02-12T19:25:44.957Z (4 months ago)
- Topics: conduit, go, golang, weaviate
- Language: Go
- Homepage:
- Size: 330 KB
- Stars: 0
- Watchers: 10
- Forks: 1
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# Conduit Connector for Weaviate
A [Conduit](https://conduit.io) destination connector for Weaviate.
## How to build?
Run `make build` to build the connector. For instructions on how to use built connector
with Conduit, check the [Conduit documentation](https://conduit.io/docs/connectors/installing).
## Testing
To run the unit test, execute `make test`.
The integration tests require a running Weaviate instance. The provided `make` targets run
a Weaviate instance using Docker. To run the integration tests, you need to:
1. set the `OPENAI_APIKEY` environment variable, which stores an OpenAI API key.
2. execute `make test-integration`.
The Docker compose file at `test/docker-compose.yml` can be used to run an instance of Weaviate locally.
## Destination
The Weaviate destination connectors handles all the changes supported by Conduit,
which are: inserts, updates, and deletes.
### Configuration
| name | description | required | default value |
|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------|---------------|
| `endpoint` | Host of the Weaviate instance. | true | "" |
| `scheme` | Scheme of the Weaviate instance.
Values: `https`, `http`. | false | "https" |
| `class` | The class name as defined in the schema.
A record will be saved under this class unless it has the `weaviate.class` metadata field. | true | "" |
| `moduleHeader.name` | Name of the header configuring a module (e.g. `X-OpenAI-Api-Key`). | false | "" |
| `moduleHeader.value` | API key for the module defined above. | false | "" |
| `generateUUID` | Generate a UUID for records (an MD5 sum of a record's key). | false | "false" |
| `auth.mechanism` | Specifies in which way the connector will authenticate to Weaviate.
Values: `none`, `apiKey`, `wcsCreds`. | false | "none" |
| `auth.apiKey` | A Weaviate API key. | Required if `auth.mechanism = apiKey`. | "" |
| `auth.wcsCreds.username` | Weaviate Cloud Services (WCS) username. | Required if `auth.mechanism = wcsCreds`. | "" |
| `auth.wcsCreds.password` | Weaviate Cloud Services (WCS) password. | Required if `auth.mechanism = wcsCreds`. | "" |
(For more information about authentication in Weaviate, refer to the Weaviate https://weaviate.io/developers/wcs/guides/authentication.)
