{"id":18187756,"url":"https://github.com/ravikisha/relaxsearch","last_synced_at":"2025-10-28T18:31:01.025Z","repository":{"id":260809453,"uuid":"882384465","full_name":"Ravikisha/RelaxSearch","owner":"Ravikisha","description":"RelaxSearch is a lightweight, demonstration-based search engine project built with Go and Elasticsearch.","archived":false,"fork":false,"pushed_at":"2024-11-04T10:12:26.000Z","size":4945,"stargazers_count":23,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-12-21T03:21:40.729Z","etag":null,"topics":["elasticsearch","golang","ranking-algorithm","search-engine"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ravikisha.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-02T17:31:06.000Z","updated_at":"2024-12-19T17:41:48.000Z","dependencies_parsed_at":"2024-11-02T18:38:52.964Z","dependency_job_id":null,"html_url":"https://github.com/Ravikisha/RelaxSearch","commit_stats":null,"previous_names":["ravikisha/relaxsearch"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ravikisha%2FRelaxSearch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ravikisha%2FRelaxSearch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ravikisha%2FRelaxSearch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ravikisha%2FRelaxSearch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ravikisha","download_url":"https://codeload.github.com/Ravikisha/RelaxSearch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238689942,"owners_count":19514125,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","golang","ranking-algorithm","search-engine"],"created_at":"2024-11-03T02:03:10.789Z","updated_at":"2025-10-28T18:31:00.046Z","avatar_url":"https://github.com/Ravikisha.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Banner](./docs/banner.jpg)\n\n# RelaxSearch\n\n\u003cp float=\"left\"\u003e\n\n\u003cimg src=\"https://img.shields.io/badge/Go-00ADD8?logo=Go\u0026logoColor=white\u0026style=for-the-badge\" alt=\"GoLang\" /\u003e\n\u003cimg src=\"https://img.shields.io/badge/Elasticsearch-005571?logo=Elasticsearch\u0026logoColor=white\u0026style=for-the-badge\" alt=\"Elasticsearch\" /\u003e\n\u003cimg src=\"https://img.shields.io/badge/Docker-2496ED?logo=Docker\u0026logoColor=white\u0026style=for-the-badge\" alt=\"Docker\" /\u003e\n\u003cimg src=\"https://img.shields.io/badge/REST_API-0096D8?logo=Postman\u0026logoColor=white\u0026style=for-the-badge\" alt=\"REST API\" /\u003e\n\u003cimg src=\"https://img.shields.io/badge/Full_Text_Search-00ADD8?logo=Go\u0026logoColor=white\u0026style=for-the-badge\" alt=\"Full Text Search\" /\u003e\n\u003cimg src=\"https://img.shields.io/badge/Web_Crawler-005571?logo=WebCrawler\u0026logoColor=white\u0026style=for-the-badge\" alt=\"Web Crawler\" /\u003e\n\n\n\u003c/p\u003e\n\nRelaxSearch is a lightweight, demonstration-based search engine project built with Go and Elasticsearch. It comprises two main components:\n\n\n1. **RelaxEngine**: A web scraper using cron jobs to periodically crawl and index content.\n2. **RelaxWeb**: An API server that enables full-text search capabilities on the indexed data.\n\nThis project showcases the essentials of a search engine by scraping content from specified URLs and storing them in Elasticsearch, where they are accessible for keyword-based searches.\n\n\u003e Inspire by the Full Document Search in go: [Blog Here](https://medium.com/@ravikishan63392/building-a-high-performance-full-text-search-engine-in-go-f2e4ec63e643)\n\n\u003e Also Inspired by the [Web Crawler in Cpp](https://github.com/kshru9/Web-Crawler)\n\n## Components\n\n### 1. RelaxEngine\n- **Function**: A web crawler that scrapes and indexes content from the web. It runs periodically via cron jobs.\n- **Technology**: Go and Elasticsearch.\n- **Details**: RelaxEngine reads a list of seed URLs, crawls the web starting from these URLs, and indexes the retrieved content in Elasticsearch.\n\n### 2. RelaxWeb\n- **Function**: Provides a REST API to search the indexed content.\n- **Technology**: Go and Elasticsearch.\n- **API Endpoint**: `/search` - Accepts a keyword query and returns relevant content with optional pagination, date filtering, and content highlighting.\n\n## Prerequisites\n\n- **Golang**: v1.16 or higher.\n- **Docker**: For containerized deployment.\n- **Elasticsearch**: v8.6.0 (configured in Docker Compose).\n\n## Setup and Installation\n\n1. Clone the repository:\n   ```bash\n   git clone https://github.com/Ravikisha/RelaxSearch.git\n   cd RelaxSearch\n   ```\n\n2. Configure `.env` files:\n   - **RelaxEngine** and **RelaxWeb** use configuration files for connecting to Elasticsearch and defining scraping parameters.\n   - Set up your Elasticsearch credentials in the `.env` files under each service as required.\n\n3. Build and Run with Docker Compose:\n   ```bash\n   docker-compose up --build\n   ```\n\n   - This will start the Elasticsearch service, RelaxEngine (scraping in background), and RelaxWeb (serving the search API on port 7000).\n\n![docker setup](./docs/dockersetup.png)\n\n![Docker Cron Jobs](./docs/cronjob.png)\n\n## Usage\n\n### Search API\n\n- **Endpoint**: `GET /search`\n- **Parameters**:\n  - `keyword` (required): The term to search for.\n  - `from` (optional): Pagination start index.\n  - `size` (optional): Number of results to return (default: 10).\n  - `dateRangeStart` and `dateRangeEnd` (optional): Filter results based on timestamp.\n\nExample using `curl`:\n```bash\ncurl -X GET \"http://localhost:7000/search?keyword=example\" -H \"Authorization: Basic \u003cbase64_credentials\u003e\"\n```\n\n![search api](./docs/searchIndex.png)\n\n## Project Structure\n\n```\nRelaxSearch/\n├── relaxengine/           # Crawler and indexing component\n│   ├── cmd/               # Main application for crawling\n│   ├── config/            # Configuration files\n│   └── crawler/           # Crawler logic and utilities\n└── relaxweb/              # Search API server\n    ├── config/            # Configuration files\n    └── search/            # Search functionality and API\n```\n\n![Diagram](./docs/diagram.png)\n\n## Known Issues and Improvements\n\n- **Error handling**: Current error handling can be improved for more robust operation.\n- **Scalability**: Limited to demonstration use; suitable for small to medium datasets but you scale with the elasticsearch nodes.\n\n## License\n\nThis project is open-source and available under the [MIT License](LICENSE).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fravikisha%2Frelaxsearch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fravikisha%2Frelaxsearch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fravikisha%2Frelaxsearch/lists"}