https://github.com/bootjp/word_detector

https://github.com/bootjp/word_detector

Last synced: 21 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/bootjp/word_detector
• Owner: bootjp
• License: mit
• Created: 2019-08-22T06:39:40.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2019-08-22T07:54:34.000Z (over 5 years ago)
• Last Synced: 2024-05-01T12:52:28.853Z (9 months ago)
• Language: Go
• Size: 3.91 KB
• Stars: 1
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# word_detector

## that is this?

Fast search for words in huge log files.

## use case

When migrating a database, I wanted to determine whether all query logs included incompatible queries. 

## method

Here we expand the word into on-memory and search in a single thread.

If the log file is large, it should be multi threaded.

If a word file changes frequently, you should create a inverted index.

## how to use 

WORD_FILE=/path/to/word.txt TARGET_FILE=/path/to/log.txt go run cli/main.go