{"id":13531694,"url":"https://github.com/Aquila-Network/aquila","last_synced_at":"2025-04-01T20:30:28.522Z","repository":{"id":47098874,"uuid":"182265441","full_name":"Aquila-Network/aquila","owner":"Aquila-Network","description":"An easy to use Neural Search Engine. Index latent vectors along with JSON metadata and do efficient k-NN search.","archived":false,"fork":false,"pushed_at":"2024-05-06T18:26:02.000Z","size":1578,"stargazers_count":376,"open_issues_count":13,"forks_count":25,"subscribers_count":21,"default_branch":"main","last_synced_at":"2024-08-27T20:15:33.613Z","etag":null,"topics":["approximate-nearest-neighbor-search","aquila","embedding","faiss","feature-vectors","image-search","information-retrieval","information-retrieval-engine","knn-search","nearest-neighbor-search","neural-information-retrieval","neural-search","retrieval","search-engine","similarity-search","similarity-searches","vector-database","video-search"],"latest_commit_sha":null,"homepage":"https://aquila.network","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Aquila-Network.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-19T13:01:51.000Z","updated_at":"2024-07-25T15:12:22.000Z","dependencies_parsed_at":"2024-05-06T18:53:44.417Z","dependency_job_id":null,"html_url":"https://github.com/Aquila-Network/aquila","commit_stats":null,"previous_names":["aquila-network/aquiladb"],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aquila-Network%2Faquila","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aquila-Network%2Faquila/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aquila-Network%2Faquila/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aquila-Network%2Faquila/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Aquila-Network","download_url":"https://codeload.github.com/Aquila-Network/aquila/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246709921,"owners_count":20821296,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["approximate-nearest-neighbor-search","aquila","embedding","faiss","feature-vectors","image-search","information-retrieval","information-retrieval-engine","knn-search","nearest-neighbor-search","neural-information-retrieval","neural-search","retrieval","search-engine","similarity-search","similarity-searches","vector-database","video-search"],"created_at":"2024-08-01T07:01:04.926Z","updated_at":"2025-04-01T20:30:27.360Z","avatar_url":"https://github.com/Aquila-Network.png","language":"HTML","funding_links":[],"categories":["Awesome Vector Search Engine","HTML","vector-database"],"sub_categories":["Standalone Service"],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://aquila.network\"\u003e\n    \u003cimg\n      src=\"https://user-images.githubusercontent.com/19545678/133918727-5a37c6be-676f-427b-8c86-dd50f58d1287.png\"\n      alt=\"Aquila Network Logo\"\n      height=\"64\"\n    /\u003e\n  \u003c/a\u003e\n  \u003cbr /\u003e\n  \u003cp\u003e\n    \u003ch3\u003e\n      \u003cb\u003e\n        Aquila DB\n      \u003c/b\u003e\n    \u003c/h3\u003e\n  \u003c/p\u003e\n  \u003cp\u003e\n    \u003cb\u003e\n      Easy to use Neural Search Engine\n    \u003c/b\u003e\n  \u003c/p\u003e\n  \u003cbr/\u003e\n\u003c/div\u003e\n\n**[Aquila DB](https://github.com/Aquila-Network/AquilaDB)** is a Neural search engine. In other words, it is a database to index **Latent Vectors** generated by ML models along with **JSON Metadata** to perform **k-NN** retrieval. It is dead simple to set up, language-agnostic, and drop in addition to your Machine Learning Applications. Aquila DB, as of current features is a ready solution for Machine Learning engineers and Data scientists to build **[Neural Information Retrieval](https://www.microsoft.com/en-us/research/uploads/prod/2017/06/INR-061-Mitra-neuralir-intro.pdf)** applications out of the box with minimal dependencies.\n\n\u003e This project is still in alpha version \u0026 we're already using it in production to power semantic search at https://aquila.network. \n\n\nWanna support this project? Yes, we love getting a **star** ⭐ and **shout-out** 🗣️ 🤗\n\nJoin [Community chat and get support: ![discord chatroom for discussions](https://www.freeiconspng.com/minicovers/flat-discord-material-like-icon--2.png)](https://discord.gg/5YP7zHS)\n\n\n# Who is this for\n\n* If you are working on a data science project and need to store a hell of a lot of data and retrieve similar data based on some feature vector, this will be a useful tool to you, with extra benefits a real world web application needs.\n* Are you dealing with a lot of images and related metadata? Want to find similar ones? You are at the right place.\n* If you are looking for a document database, this is not the right place for you.\n\n# Technology\nAquila DB powers search features of Aquila Network. Here is where Aquila DB fits in the entire ecosystem:\n\u003cdiv align=\"center\"\u003e\n  \u003cimg\n    src=\"https://user-images.githubusercontent.com/19545678/133918438-997af8ec-67ef-4f7c-95ab-16d2f28e79e3.png\"\n    alt=\"Aquila DB Architecture\"\n    height=\"400\"\n  /\u003e\n \u003cbr/\u003e\n\u003c/div\u003e\n\n\n\nIf you are serious and wanna dive down the rabbit hole, read our **[whitepapers](https://github.com/Aquila-Network/whitepaper)** and **[technical specifications](https://github.com/Aquila-Network/specs)** (being actively worked on). \n\n\n\n**As a side note**, everything in **[Aquila Network](https://github.com/Aquila-Network)** is defined by the specifications and a large chunk of our efforts goes into it. We also maintain quality implementations of those specifications with non-technical users in mind. This is to make sure that - Aquila Network is fully open, decentralized by design, and Fair. You can follow those specifications to implement your alternative software and still interact with the network without any restrictions.\n\n# Install\n### Debian\n\nRun `curl -s -L https://raw.githubusercontent.com/Aquila-Network/AquilaDB/master/install.sh | /bin/bash -s -- -d 1 `.\n\n### Docker\n\n**You need docker installed in your system**\n\nBuild image (lite): `docker build https://raw.githubusercontent.com/Aquila-Network/AquilaDB/master/Dockerfile -t aquiladb:local`\n\nBuild image (big data): `docker build https://raw.githubusercontent.com/Aquila-Network/AquilaDB/master/DockerfileBig -t aquiladb:localbg`\n\nRun image (to deploy Aquila DB lite): `docker run -p 5001:5001 -d aquiladb:local`\n\nRun image (to deploy Aquila DB big): `docker run -p 5001:5001 -d aquiladb:localbg`\n\n# Client SDKs\nWe currently have multiple client libraries in progress to abstract the communication between deployed Aquila DB and your applications.\n\n[Python](https://github.com/Aquila-Network/AquilaPy)\n\n[Node JS](https://github.com/Aquila-Network/AquilaJS)\n\n## Where to get private key (wallet key) for client authentication\nWhen you use a client library to authenticate with AquilaDB, you might need access the same private key (wallet key) used by AquilaDB. This key is located inside `/ossl/` directory within AquilaDB docker container (in your computer if you have installed AquilaDB directly without docker). To access the keys inside your AquilaDB container, follow below steps:\n\n* identify `CONTAINER ID` for the already running `aquiladb` docker instance:\n`docker ps`\n* take a copy of private keys from docker container to your host machine:\n`docker cp CONTAINER_ID:/ossl/ ./`\n* now you will see a new directory named `ossl` at your current location. Use the keys inside it.\n#### tips for advanced users\nIf your pipeline requires the private keys to be generated in advance, you can do it in your host machine and then mount it to the container's `/ossl/` directory. \n\nRun:\n```\nmkdir -p \u003chost\u003e/ossl/\nopenssl genrsa -passout pass:1234 -des3 -out \u003chost\u003e/ossl/private.pem 2048\nopenssl rsa -passin pass:1234 -in \u003chost\u003e/ossl/private.pem -outform PEM -pubout -out \u003chost\u003e/ossl/public.pem\nopenssl rsa -passin pass:1234 -in \u003chost\u003e/ossl/private.pem -out \u003chost\u003e/ossl/private_unencrypted.pem -outform PEM\n```\n\n# Progress\nThis project is still and will be under active development with intermediate production releases. It can either be used as a standalone database or as a participating node in Aquila Network. Please note, [Aquila Port](https://github.com/Aquila-Network/specs/blob/main/README.md#aquila-port) (peer-peer network layer for Aquila DB nodes) is also a work in progress. Currently, you need to deploy your custom models to feed vector embeddings to Aquila DB, until [Aquila Hub](https://github.com/Aquila-Network/specs/blob/main/README.md#aquila-hub) developments get started.\n\n# Contribute\nWe have [prepared a document](https://docs.google.com/document/d/1bT2_9FQIxQpx_rdYbkTukn_DJRi_haVK_ixTf8uTaDE/edit?usp=sharing) to get anyone interested to contribute, immediately started with Aquila DB.\nHere is our high-level [release roadmap](https://user-images.githubusercontent.com/19545678/62313851-5af82880-b4af-11e9-84f6-21e24bf46e8a.png).\n\n# Learn\n\nWe have started meeting developers and do small talks on Aquila DB. Here are the slides that we use on those occasions: http://bit.ly/AquilaDB-slides \n\n**Video:**\n\n[\u003cimg alt=\"introduction to Neural Information retrieval with AquilaDB\" src=\"http://img.youtube.com/vi/-VYpjpLXU5Q/0.jpg\" width=\"300\" /\u003e](http://www.youtube.com/watch?v=-VYpjpLXU5Q)\n\nAs of current AquilaDB release features, you can build **[Neural Information Retrieval](https://www.microsoft.com/en-us/research/uploads/prod/2017/06/INR-061-Mitra-neuralir-intro.pdf)** applications out of the box without any external dependencies. Here are some useful links to learn more about it and start building:\n\n* Microsoft published a paper and youtube video on this to onboard anyone interested: \n  * paper: https://www.microsoft.com/en-us/research/uploads/prod/2017/06/INR-061-Mitra-neuralir-intro.pdf\n  * video: https://www.youtube.com/watch?v=g1Pgo5yTIKg\n* Embeddings for Everything: Search in the Neural Network Era: https://www.youtube.com/watch?v=JGHVJXP9NHw\n* Autoencoders are one such deep learning algorithms that will help you to build semantic vectors - foundation for Neural Information retrieval. Here are some links to Autoencoders based IR:\n  * go to chapter 15 in this link: https://www.cs.toronto.edu/~hinton/coursera_lectures.html\n  * https://www.coursera.org/lecture/ml-foundations/examples-of-document-retrieval-in-action-CW25H\n  * https://www.coursera.org/lecture/intro-to-deep-learning/autoencoders-101-QqBOa\n* Note that, the idea of information retrieval applies not only to text data but for any data. All you need to do is, encode any source datatype to a dense vector with deep neural networks.\n\n\u003cbr/\u003e\u003cbr/\u003e\n\u003ch1 align=\"center\"\u003eOur Sponsors\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\u003cb\u003e\u003c/b\u003e\u003c/p\u003e\n\n\u003cbr/\u003e\n\n\u003e email us to sponsor this project [adbadmin@protonmail.ch](mailto:adbadmin@protonmail.ch).\n\n\u003cbr/\u003e\u003cbr/\u003e\n\n# Citing Aquila DB\nIf you use Aquila DB in an academic paper, we would 😍 to be cited. Here are the two ways of citing Aquila DB:\n```\n\\footnote{https://github.com/Aquila-Network/AquilaDB}\n```\n```\n@misc{AquilaNetwork2019AquilaDB,\n  title={AquilaDB: Neural Search Engine},\n  author={Jubin Jose, Nibin Peter},\n  howpublished={\\url{https://github.com/Aquila-Network/AquilaDB}},\n  year={2019}\n}\n```\n\n# License\n\nApache License 2.0 [license file](https://github.com/Aquila-Network/AquilaDB/blob/master/LICENSE)\n\ncreated by ❤️ with a-mma (a_മ്മ)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAquila-Network%2Faquila","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAquila-Network%2Faquila","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAquila-Network%2Faquila/lists"}