An open API service indexing awesome lists of open source software.

https://github.com/obscurely/rekosearch-public

AI-powered search engine that understands the content of your files
https://github.com/obscurely/rekosearch-public

ai aws flask kubernetes python react rust search terraform

Last synced: 3 months ago
JSON representation

AI-powered search engine that understands the content of your files

Awesome Lists containing this project

README

          





Logo


RekoSearch is an AI-powered file search engine for images, videos, documents and audio that understands the content of your files. It enables simultaneous semantic search across them.



Explore the docs ยป




View Demo
ยท
Report Bug
ยท
Request Feature



Table of Contents



  1. ๐Ÿชฝ About The Project


  2. ๐Ÿ—๏ธ Architecture Diagrams

  3. ๐Ÿ’ Contributing

  4. โ“ FAQ

  5. ๐Ÿ”ฐ Security

  6. ๐Ÿชช License

  7. ๐Ÿ“ง Contact

## ๐Ÿชฝ About The Project

RekoSearch is an AI-powered file search engine for images, videos, documents and audio that understands the content of your files. It enables simultaneous semantic search across:

- **Images & Videos**: Find objects, scenes, activities, landmarks, facial attributes and text.
- **Documents**: Search through the actual content (text).
- **Audio**: Search through what's being said.

You can plainly search for what you want, or use advanced search syntax like the example below:

```
label:dog AND face:smiling NOT text:warning
```

(This will find images and videos with a dog and a person smiling, but no text spelling "warning".)

Alternatively, you can also download just the results files.

If you've got any questions please give the [FAQ](#faq) a check.

(back to top)

### ๐Ÿ’ก Features

- [x] AI-Powered Content Search
- [x] Multi-Format Support
- [x] Advanced Search Syntax
- [x] Object and Scene Detection
- [x] Facial Analysis
- [x] Text Recognition
- [x] Audio Transcription
- [x] Batch Processing
- [x] Search Result Downloads
- [x] Secure Data Handling
- [x] Job Management
- [x] Transparent Pricing

(back to top)

### ๐ŸŒ Demo

Check out the [demo](https://rekosearch.com/demo) to see how it works in action.

(back to top)

### ๐Ÿ” Built with

- [AWS](https://aws.amazon.com/)
- [Kubernetes](https://kubernetes.io/)
- [Rust](https://www.rust-lang.org/)
- [Python](https://www.python.org/)
- [Flask](https://flask.palletsprojects.com/)
- [React](https://reactjs.org/)
- [MUI](https://mui.com/)
- [Terraform](https://www.terraform.io/)

(back to top)

## ๐Ÿ—๏ธ Architecture Diagrams

**IMPORTANT NOTE**:

- What is below referred to as the **FRONTEND** includes **ONLY** the **DASHBOARD**.
- The **HOMEPAGE** is a statically exported (**prerendered**) **Next.js website hosted on S3** behind CloudFront. (using the same technologies as the Dashboard: React, MUI, etc.)
- What this means is that the **homepage is very inexpensive to run** and infinitely scalable, at a cost that is easy to bear, while the **dashboard**, which costs more to run, **will be paid for by the users**.

### High-Level Overview

![High Overview Diagram](assets/diagrams/high_overview.png)

A comprehensive overview of RekoSearch's architecture, showing how authentication, job processing, data upload, and result retrieval work together. [View detailed documentation](docs/OVERVIEW.md)

### New Job Flow

![New Job Diagram](assets/diagrams/new_job.png)

Step-by-step process of creating a new file processing job, from user request through authentication, credit checking, S3 upload generation, and job queuing. [View detailed documentation](docs/Application/NEW_JOB.md)

### Server Processing

![Server Processing Diagram](assets/diagrams/server_processor.png)

Detailed workflow of how jobs are processed on the server side, including file processing, AI analysis, error handling, and status updates. [View detailed documentation](docs/Application/SERVER.md)

### User API

![User API Diagram](assets/diagrams/user_api.png)

Architecture of user-related API endpoints including profile management, user updates, terms acceptance, and account deletion processes. [View detailed documentation](docs/Application/USER_API.md)

### Jobs API

![Jobs API Diagram](assets/diagrams/jobs_api.png)

Structure of job-related API endpoints for listing jobs, retrieving job details, accessing file URLs, and deleting jobs. [View detailed documentation](docs/Application/JOBS_API.md)

### K3s Cluster

![K3s Cluster Diagram](assets/diagrams/k3s.png)

Infrastructure diagram showing the K3s Kubernetes cluster deployment on AWS with load balancers, auto-scaling, and CloudFront integration. [View detailed documentation](docs/Application/K3S.md)

NOTE: Currently using Digital Ocean K8s cluster to slowly integrate the K3s cluster in. Making sure the launch is smooth and without issues. Might actually stick with DO K8s cluster for some time as at the current scale it's cheaper and easier to manage. K3s cluster will get cheaper, but at a higher scale.

(back to top)

## ๐Ÿ’ Contributing

While the actual source code is private if you have any suggestions feel free to open an [Issue](https://github.com/Obscurely/RekoSearch-Public/issues) about them.

Please read [CONTRIBUTING.md](CONTRIBUTING.md) and
[CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md).

(back to top)

## โ“FAQ

See [FAQ.md](./docs/FAQ.md) for frequently asked questions.

(back to top)

## ๐Ÿ”ฐ Security

If you happen to find any issues please carefully read
[SECURITY.md](SECURITY.md) and strictly follow everything stated.

(back to top)

## ๐Ÿชช License

Is licensed under [Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International](https://creativecommons.org/licenses/by-nc-nd/4.0/)

(back to top)

## ๐Ÿ“ง Contact

Either post an issue in the
[Issues Tab](https://github.com/Obscurely/RekoSearch-Public/issues) or contact
me at this email address if you have more to say:
[contact@rekosearch.com](mailto:contact@rekosearch.com)

(back to top)

[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/K3K3H29LV)