https://github.com/varppi/goramq
A fast Golang program to load files into memory and search through then via an API
https://github.com/varppi/goramq
backend-api data-search database-search file-search golang rest-api
Last synced: 10 months ago
JSON representation
A fast Golang program to load files into memory and search through then via an API
- Host: GitHub
- URL: https://github.com/varppi/goramq
- Owner: varppi
- License: gpl-3.0
- Created: 2023-07-24T05:22:18.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2025-08-21T14:26:07.000Z (10 months ago)
- Last Synced: 2025-08-21T17:01:35.237Z (10 months ago)
- Topics: backend-api, data-search, database-search, file-search, golang, rest-api
- Language: Go
- Homepage:
- Size: 188 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# GORAMQ (Golang Ram Query)
Like the name suggests, Goramq loads user supplied files into memory and then lets you search strings inside the files via a restful API.
## How it works
Goramq reads all files provided in chunks (defined by blocksize) and adds the read data to a variable. By default after reading the files Goramq will ask Golang garbage collector to do a cycle, this frees up memory.
When a query comes through, Goramq activates a "lock" that prevents other queries from being taken while the current one is being executed, to the person/bot doing the query that was put on hold, it will look like the website is just loading slowly.
## Installation
Note: Make sure you have Golang installed and the go bin directory added to your PATH variable.
```bash
go install github.com/Varppi/goramq@latest
```
## API Docs
```
API endpoint: `/search/`
Parameters:
q: The query
Example values:
- badperson@gmail.com --> Searches for a line that contains single string ("badperson@gmail.com")
- peter::@randomcompany.com --> Searches for a line that contains both strings, "peter" and "@randomcompany.com"
caseins: Make the query case insensitive
amount: How many results you want to receive (faster and doesn't require as much bandwith)
Example value:
- 10 --> Returns first 10 results
Example requests:
- http://127.0.0.1:9112/search?q=randomguy::Estonia::2003&caseins=true
- http://127.0.0.1:9112/search?q=username@protonmail.com&pass=verysecretAPIpassword
```
## Basic usage examples
```
#Load a file and start the web server in local only mode
goramq --filenames "database.sql"
#Loads 2 files and let the password protected API listen on all interfaces on port 80
goramq --listener :80 --filenames "documents/database.sql::documents/randomtextfile.txt" --password APIsecret
#Loads a file and does garbage collection only after all files have been loaded
goramq --filenames "database.sql" --pcgo
```
## Help page
```
________________ ________ _______ ______ __________
__ ____/__ __ \___ __ \___ |___ |/ /__ __ \
_ / __ _ / / /__ /_/ /__ /| |__ /|_/ / _ / / /
/ /_/ / / /_/ / _ _, _/ _ ___ |_ / / / / /_/ /
\____/ \____/ /_/ |_| /_/ |_|/_/ /_/ \___\_\
Usage of goramq:
-blocksize int
How big chunks the files will be loaded in (default 1024)
-filenames string
Files to serve (separated with ::)
-listener string
Web server listener (IP:PORT) (default "127.0.0.1:9112")
-nogc
Disables garbage collection (uses twice as much ram)
-password string
Password protects the API
-pcgo
Only does garbage collection after all files are loaded.
-quiet
When used, Goramq will not display the file loading progress (perfomance boost)
-result-limit int
How many results you can receive (over limit = error message) (default 50000)
```