Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/pilotpirxie/coresearch
🔎 .NET Core cross-platform, in-memory, full text search library for building search engines. Made to learn C#.
https://github.com/pilotpirxie/coresearch
algorithm algorithms beginner-project big-data dotnet dotnet-core dotnetcore full-text-search inmemory-db learning text-search
Last synced: 3 months ago
JSON representation
🔎 .NET Core cross-platform, in-memory, full text search library for building search engines. Made to learn C#.
- Host: GitHub
- URL: https://github.com/pilotpirxie/coresearch
- Owner: pilotpirxie
- License: mit
- Created: 2019-11-08T21:39:08.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2019-12-02T23:42:17.000Z (about 5 years ago)
- Last Synced: 2024-11-09T03:52:54.450Z (3 months ago)
- Topics: algorithm, algorithms, beginner-project, big-data, dotnet, dotnet-core, dotnetcore, full-text-search, inmemory-db, learning, text-search
- Language: C#
- Homepage:
- Size: 133 KB
- Stars: 42
- Watchers: 1
- Forks: 27
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# coresearch
.NET Core cross-platform, in-memory, full text search library for building search engines
## About
**Coresearch** uses an inverted index with a boosted trie data structure for indexing atomic search criterion from content to resources. Trie algorithm makes Coresearch more elastic and allows both exact word querying and operations like fuzzy search, wildcards and character matching. Entire trie structure is stored in memory for better performance. The entire project was written as a learning project so be aware of putting in on the production 😉
## Features
* Efficient **inverted index** algorithm (mapping from content to resources)
* Uses **Prefix Tree (Trie)** data structure for quick search and horizontal scaling
* **Exact search** with super fast O(k) worst case time, where k is the key length
* **Wildcard search** for recursive depth-first search tree traversal
* **Character match** search with deep level k+1
* Simple **command line** tool and **library**
* **Cross-platform**, runs on Linux, MacOS and Windows
* Easily handle **large amount of different resources**
[Trie on Wikipedia](https://en.wikipedia.org/wiki/Trie)
## CLI commands
### Load data from files in specific path and extension (recursive)
```
source
alias: load
```
example:
```
> source ./ *.txt
```
### Get resource names for exact key (word)
```
get
alias: search
```
example:
```
> search cars
output:
```
### Search for every resource name which key starts with prefix
Query modes:
* Question mark (?) is for select of all resource names belong to children of specific node (if any).
* Wildcard sign (\*) is for select of all resource names recursively under specific prefix.
* Exact matching (without any sign) produces equal output as get/search command.
```
query
```
example 1:
```
> query c ?
output:
```
example 2:
```
> query c *
output:
```
example 3:
```
> query cars
output:
```
### Add resource under key (word)
```
add
alias: insert
```
example:
```
> add english-dict.txt house
```
### Remove specific key (word)
```
delete
```
example:
```
> delete sport
```
### Echo
```
echo
```
example:
```
> echo hello
output:
hello
```
### Turn on/off debug mode
```
debug
```
example:
```
> debug true
```
### Remove every node in structure and collect memory
```
flush
```
example:
```
> flush
```
### Show information
```
info
```
example:
```
> flush
output:
Nodes in trie: 4651175
Words inserted: 15513389
Resource files: 2227
Memory usage: 1044944640 bytes
```
### Clear console
```
clear
```
example:
```
> clear
```
## CLI arguments
Turn debug mode
```
--debug
-d
```
Set max size of memory (in bytes)
```
--memory-limit
-m
```
Pre-process every word before insert
```
--normalize
-n
```
Pattern for removing unwanted characters, used for each word before insert
```
--pattern
-p
```
Load data from specific path at start
```
--source
-s
```
Set extension for loading data at start
```
--extension
-e
```
License
-----
coresearch is licensed under the MIT.