{"id":18073874,"url":"https://github.com/roshan-r/cherava","last_synced_at":"2025-08-26T19:15:40.596Z","repository":{"id":109165970,"uuid":"609269708","full_name":"Roshan-R/Cherava","owner":"Roshan-R","description":"Cherava, a web based scraper environment that focuses on simplicity.","archived":false,"fork":false,"pushed_at":"2025-03-14T06:45:20.000Z","size":215,"stargazers_count":13,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-12T05:29:00.458Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://cherava.vercel.app","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Roshan-R.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-03-03T18:28:41.000Z","updated_at":"2025-03-14T06:45:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"9e15205d-2598-4d3c-80cb-7f6db49bcc31","html_url":"https://github.com/Roshan-R/Cherava","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Roshan-R/Cherava","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Roshan-R%2FCherava","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Roshan-R%2FCherava/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Roshan-R%2FCherava/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Roshan-R%2FCherava/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Roshan-R","download_url":"https://codeload.github.com/Roshan-R/Cherava/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Roshan-R%2FCherava/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259942758,"owners_count":22935325,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-31T10:09:49.049Z","updated_at":"2025-06-15T08:04:02.728Z","avatar_url":"https://github.com/Roshan-R.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cherava\n\nAn open source, zero-code web scraping automation tool. \u003cbr /\u003e\nIts purpose is to create alert systems for various tasks such as monitoring university notice boards, e-commerce site price alerts, product alerts, and more.\n\n## Features that work\n\n- GUI-based scraper task creation with zero code required for any website.\n- Users can add workflows, which are saved into a database based on the user's session.\n- Automation of scraping tasks with cron job-like scheduling.\n- Notification system that alerts the user when the contents of a specific HTML selector changes over time using email.\n\n## Future proposed features\n\n- Use a preview of the website within the UI itself to pick the CSS selector.\n- More notification provider options.\n- UI-based or script engine features to further process the data received from scraping.\n\n## How it works\n\n- Workflows for scraping automation tasks are created using a browser session as an ID. Future plans are to use proper authentication.\n- The Cheerio Node.js package is used to run the web scraping tasks, and a preview is generated for the URL and CSS selector specified in the add workflow UI. The CSS selector needs to be taken from the website using Inspect Element.\n- When a workflow is created, it is added into the PostgreSQL database hosted on Railway.app, along with the notification recipient emails and the interval for checking updates on the website.\n- The Node-cron package is used for scheduling the scraping workflow.\n- The Nodemailer package then sends an email to the recipients when the workflow detects a change in the CSS selector's contents on the website versus the contents stored in the database during the first run when the workflow was added.\n\n## Timeline\n- Initially, the web scraper and the basic UI for adding a workflow were implemented.\n- Next, the workflow UI was connected to the PostgreSQL database.\n- Next, the Nodemailer notifier module was added.\n- Next, the cron job module was added.\n- Finally, all of these components were integrated together.\n\n## Demo Video\n\n[![](https://img.youtube.com/vi/Eqarz4dFGnU/0.jpg)](https://youtu.be/Eqarz4dFGnU)\n\n## Built by\n\n\u003ctable\u003e\n  \u003ctr\u003e\n\n\u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/Roshan-R\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/43182697?v=4\" width=\"180px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eRoshan R Chandar\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\n\n  \u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/AJAYK-01\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/55079486?v=4\" width=\"180px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eAjay Krishna K V\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\n\n  \u003ctd align=\"center\"\u003e\u003ca href=\"https://github.com/GameGodS3\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/54617167?v=4\" width=\"180px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eSudev Suresh Sreedevi\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\n\n\u003c/tr\u003e\n\u003c/table\u003e\n\n## To run locally\n\nClone the Repo\n\n#### Frontend\n\n```\ncd frontend\nnpm i\nnpm run dev\n```\n\n#### Backend\n\n```\ncd backend\nnpm i\nnpm run dev\n```\n\n#### Env file format for frontend\n\n\u003cimg width=\"402\" alt=\"image\" src=\"https://user-images.githubusercontent.com/55079486/222959773-772e428c-2e0f-4acd-bc5f-07ffe23b689c.png\"\u003e\n\n#### Env file format for backend\n\n\n\u003cimg width=\"720\" alt=\"Screenshot 2023-03-05 at 5 44 38 PM\" src=\"https://user-images.githubusercontent.com/55079486/222959847-dfdefb8f-d426-451c-8a0e-071689fb3161.png\"\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Froshan-r%2Fcherava","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Froshan-r%2Fcherava","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Froshan-r%2Fcherava/lists"}