Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/indexnetwork/index
Discovery Protocol
https://github.com/indexnetwork/index
agent-frameworks ceramic-network composability composedb creator-tools discovery ethereum event-driven interoperability ipfs lit-protocol web3
Last synced: about 1 month ago
JSON representation
Discovery Protocol
- Host: GitHub
- URL: https://github.com/indexnetwork/index
- Owner: indexnetwork
- License: mit
- Created: 2023-06-29T19:17:48.000Z (over 1 year ago)
- Default Branch: dev
- Last Pushed: 2024-08-21T17:42:09.000Z (3 months ago)
- Last Synced: 2024-08-22T11:03:42.844Z (3 months ago)
- Topics: agent-frameworks, ceramic-network, composability, composedb, creator-tools, discovery, ethereum, event-driven, interoperability, ipfs, lit-protocol, web3
- Language: TypeScript
- Homepage: https://index.network
- Size: 90.1 MB
- Stars: 43
- Watchers: 2
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Discovery protocol 🚀
## About Index Network
Index Network is a decentralized discovery protocol that enables personalized and autonomous information retrieval through a composable vector database and an agent-based query engine.
To achieve this, Index provides a decentralized semantic index that eliminates data fragmentation, a composable discovery protocol that allows data to be queried from multiple sources, in a user-centric manner. On top of this, it provides a real-time environment for agents that facilitates integration with algorithms and services and ensures that information acquires a fluid, social, and autonomous structure.
> [!NOTE]
> Full documentation and additional resources are available on the [Index Network documentation site →](https://docs.index.network)
## Table of contents
1. [Example](#examples)
2. [Components](#components)
3. [Getting started](#getting-started)
4. [Contributing](#contributing)
5. [Credits](#built-with-open-protocols)
6. [Resources](#resources)
7. [License](#license)
## Example
To illustrate, here is an example of an agent message that’s possible using Index Network:
## Components
### Web App
The [Web App](web-app) component is where you can find the code for the web application. The web app provides a user-friendly interface that allows creators to explore indexes, manage their data, and configure them.
### Indexer
The [Indexer](indexer) component hosts the code for the indexer. It acts as the backbone of the natural language operations, providing the necessary methods to consume data from Ceramic, interact with large language models. It listenes Apache Kafka to consume Ceramic Network events consumes to ChromaDB by running `yarn consumer`
### API
The [API](api) repository also hosts the code for all the API operations To start, you can simply run `yarn api`
Full protocol documentation can be found on the [Index Network documentation site →](https://docs.index.network)
### Ceramic Network Node
ComposeDB on Ceramic is a composable graph database built for Web3. Here you can find the steps to deploy along with an IPFS Node:
[https://composedb.js.org/docs/0.6.x/guides/composedb-server](https://composedb.js.org/docs/0.6.x/guides/composedb-server)
### IPFS Node
ComposeDB on Ceramic is a composable graph database built for Web3. Here you can find the steps to deploy along with an IPFS Node:
[https://composedb.js.org/docs/0.6.x/guides/composedb-server](https://composedb.js.org/docs/0.6.x/guides/composedb-server)
### Others
We use [PostgreSQL](https://www.postgresql.org/) indexing feature of [Ceramic Network](https://ceramic.network/) and use [KafkaConnect CDC](https://docs.confluent.io/cloud/current/connectors/cc-postgresql-source.html) to produce db changes to [Apache Kafka](https://kafka.apache.org/) and finally write data to [ChromaDB](https://www.trychroma.com/) via the consumer service.
## Getting started
This code snippet demonstrates how to interact with the protocol using the Index client. It begins by initializing the client for and authenticating with a DID session. Then, it creates an index titled "Future of publishing" and a web page titled "Post medium publishing" with a specified URL. The code adds the web page to the index and performs a natural language query using the keyword "summarize," retrieving and displaying the natural language query response.
```js
const indexClient = new Index({
network: "testnet",
});
indexClient.authenticate(session);
// Create an index
const indexId = await indexClient.createIndex({
title: "Future of publishing",
signerPublicKey: "0x047955f0df748d..1708fc8c965",
signerFunction: "QmaD9FZJYst2Tntf9UwSd3QUeD68XpaPhKBSyy5wWLHq2m",
});
// Create a web page
const webPageId = await indexClient.createWebPage({
title: "Post medium publishing",
url: "http://www.paulgraham.com/publishing.html",
});
// Add the web page to the index
await indexClient.addIndexItem(indexId, webPageId);
// Perform a natural language query
const queryResponse = await indexClient.query({
query: "summarize",
indexes: [indexId],
});
console.log("Query response:", queryResponse);
```
```json
{
"response": "This article discusses the intricacies and challenges of publishing in the modern digital era, emphasizing the importance of content quality and audience engagement. The author shares insights from personal experiences and outlines strategies for successful online publishing.",
"sources": [
{
"itemId": "kjzl6kcym7w8y7fjc89gmnkne7qpdz5ws5ryfji3i8dndjh2wxuii7z1anybovo",
"indexId": "indexIdValue"
}
]
}
```
## Contributing
First of all, thanks for thinking of contributing to this project. Before sending a Pull Request, please make sure that you're assigned the task on a GitHub issue.
- If a relevant issue already exists, discuss on the issue and get it assigned to yourself on GitHub.
- If no relevant issue exists, open a new issue and get it assigned to yourself on GitHub.
And join our community to get support, provide feedback, or just to say hello on...
## Built with Open Protocols
Index Network leverages the power of several open protocols to enhance its capabilities:
- [**Ceramic**](https://ceramic.network/): Ceramic is a decentralized data network that brings unlimited data composability to Web3 applications. It enables Index Network to leverage the decentralized data network and utilize its features in building discovery engines.
- [**Lit Protocol**](https://litprotocol.com/): Lit Protocol is a decentralized access control infrastructure designed to bring more utility to the web. Index Network integrates Lit Protocol to provide enhanced access control features, allowing creators to have fine-grained control over their data.
- [**IPFS**](https://ipfs.io/): IPFS (InterPlanetary File System) is a peer-to-peer hypermedia protocol designed to preserve and grow humanity's knowledge by making the web upgradeable, resilient, and more open. Index Network utilizes IPFS to store and distribute data, ensuring its availability and resilience.
## Resources
- **[Index Network](https://index.network)** to explore the app.
- **[Discord](https://discord.gg/wvdxP6XvYu)** for support and discussions with the community and the team.
- **[GitHub](https://github.com/indexas/indexas)** for source code, project board, issues, and pull requests.
- **[Twitter](https://twitter.com/indexas)** for the latest updates on the product and published blogs.
## License
Index Network is under the MIT license. See the [LICENSE](LICENSE) for more information.