https://github.com/pchaparro/search-engine
Full stack search-engine created from youtube videos obtained using "web-scraping"
https://github.com/pchaparro/search-engine
data opensearch python python3 react scraper scraping scraping-websites search search-engine semantic-search sentence-transformers typescript website
Last synced: 2 months ago
JSON representation
Full stack search-engine created from youtube videos obtained using "web-scraping"
- Host: GitHub
- URL: https://github.com/pchaparro/search-engine
- Owner: PChaparro
- License: agpl-3.0
- Created: 2022-11-10T20:56:56.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2022-11-14T22:45:33.000Z (over 3 years ago)
- Last Synced: 2025-06-09T06:46:36.349Z (about 1 year ago)
- Topics: data, opensearch, python, python3, react, scraper, scraping, scraping-websites, search, search-engine, semantic-search, sentence-transformers, typescript, website
- Language: Jupyter Notebook
- Homepage:
- Size: 361 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Search engine
**Note:** You cand find more information on `data/`, `back-end` and `front-end` folders.
## Resume
### On load
```mermaid
sequenceDiagram
autonumber
participant frontend
participant python_api
participant open_search
participant sentence_transformers
frontend ->> python_api: HTTP Get request to get random videos
python_api ->> open_search: Query with random score
open_search ->> python_api: Random videos
python_api ->> frontend: HTTP Json response
```
### On user submit
```mermaid
sequenceDiagram
autonumber
participant frontend
participant python_api
participant open_search
participant sentence_transformers
frontend ->> python_api: HTTP Post request to get results
python_api ->> sentence_transformers: Transform user query into vector
sentence_transformers ->> python_api: Vector
python_api ->> open_search: Query using a knn filter
open_search ->> python_api: Query results
python_api ->> frontend: HTTP Json response
```