{"id":13642628,"url":"https://github.com/webwhiz-ai/webwhiz","last_synced_at":"2025-04-20T20:32:22.727Z","repository":{"id":164292868,"uuid":"633738492","full_name":"webwhiz-ai/webwhiz","owner":"webwhiz-ai","description":"WebWhiz allows you to create an AI chatbot that knows everything about your product and can instantly respond to your customer's queries.","archived":false,"fork":false,"pushed_at":"2024-09-08T07:42:05.000Z","size":2238,"stargazers_count":969,"open_issues_count":48,"forks_count":165,"subscribers_count":16,"default_branch":"main","last_synced_at":"2025-04-13T02:42:18.514Z","etag":null,"topics":["ai","chat","chatbot","chatgpt","openai","self-hosted"],"latest_commit_sha":null,"homepage":"https://www.webwhiz.ai/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/webwhiz-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"license.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-28T07:05:49.000Z","updated_at":"2025-04-02T06:00:51.000Z","dependencies_parsed_at":"2023-10-02T20:14:48.063Z","dependency_job_id":"75d157ed-d6d9-4ee8-87c4-1a0ce6f406b5","html_url":"https://github.com/webwhiz-ai/webwhiz","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/webwhiz-ai%2Fwebwhiz","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/webwhiz-ai%2Fwebwhiz/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/webwhiz-ai%2Fwebwhiz/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/webwhiz-ai%2Fwebwhiz/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/webwhiz-ai","download_url":"https://codeload.github.com/webwhiz-ai/webwhiz/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249958828,"owners_count":21351721,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","chat","chatbot","chatgpt","openai","self-hosted"],"created_at":"2024-08-02T01:01:34.150Z","updated_at":"2025-04-20T20:32:22.174Z","avatar_url":"https://github.com/webwhiz-ai.png","language":"TypeScript","funding_links":[],"categories":["TypeScript","Chatbots","SDK, Libraries, Frameworks"],"sub_categories":["JavaScript/Typescript"],"readme":"# WebWhiz\n\nTrain ChatGPT on Your Website Data and Build an AI Chatbot that can instantly answer your customers queries.\n\n![webwhiz](https://user-images.githubusercontent.com/6586706/236858939-4f3e4ac0-f3f7-4f76-8fee-add747b09ce1.png)\n    \n\n## 🔥 Core features\n\n- Easy Integration\n- Data-Specific Responses\n- Regular Data Updates\n- No code builder\n- Customise chatbot\n- Fine tuning\n- Offline message\n\n\n\n\n## 🤔 How it works ❓\nCreate and train a chatbot for your website in just a few simple steps.\n\n- Just enter your website URL to get started. We'll automatically fetch and prepare training data.\n- We’ll automatically train ChatGPT on your website based on the selected parameters and create the chatbot for you.\n- To embed the chatbot to your website, simply add the tiny script tag to your website.\n\n\n\u003chr\u003e\n\n## 🙋‍♂️ Frequently Asked Question ❓\n\n### ***What is WebWhiz?***\nWebWhiz allows you to train ChatGPT on your website data and build a chatbot that you can add to your website. No coding required.\n\n### ***How frequently do you crawl my website?***\nCurrently we crawl your website once every month. Please contact us if you need your website to be scanned more frequently\n\n### ***What data do you collect from my website?***\nWebWhiz collects data from your website pages to train your chatbot. This includes text data from the pages as well as any metadata such as page titles or descriptions. We do not collect any personally identifiable information (PII) or sensitive data from your website. We scan only public data available to search engines\n\n### ***What happens if I exceed my plan's limits?***\nIf you exceed your plan's limits for projects or pages, we will notify you. However, if you exceed the token limit for your plan, your chatbots will stop generating AI responses and will instead respond with a predefined message.\n\n### ***What are tokens?***\nTokens are a unit of measurement used to calculate the amount of text data that is processed by your chatbot. Each token corresponds to a variable number of characters, depending on the complexity of the language used in the message. Each message your chatbot sends uses a certain number of tokens based on the length and complexity of the input and the AI response. You can view the current token usage of your account on the dashboard.\n\n### ***Can I train custom data?***\nYes, you can train custom data by simply pasting content to WebWhiz\n\n### ***Can I bring my own open ai Key***\nNot at the moment, but, it will be possible in a couple of days.\n\n### ***What is the maximum size of context?***\nWebWhiz have any limitations on the size of context. However, please note that the number of pages you can crawl may be limited based on the plan you choose. Please refer to our plans page to learn more about the specific limitations of each plan.\n\n## 📑 License\nWebwhiz is open-source under the **GNU Affero General Public License Version 3 (AGPLv3)**\n\n## WebWhiz SDK\nWebWhiz SDK is available on NPM, CDNs, and GitHub.\n\n-   [NPM](https://www.npmjs.com/) - NPM is a package manager for the JavaScript\n    programming language. You can install `webwhiz` using the following\n    command\n\n    ```sh\n    npm install webwhiz\n    ```\n- [CDN](https://www.unpkg.com/webwhiz@1.0.0/dist/sdk.js) Use directly from CDN\n\n  ```sh\n    https://www.unpkg.com/webwhiz@1.0.0/dist/sdk.js\n  ```\n\n\n\n## ☁️ Self Hosting\n\n1. Docker - Easy\n2. Manual Setup - Involved (but provides more flexibility)\n\n### 📦 Docker\n\nPrerequisites\n\n- Docker \u0026 docker-compose\n\nRunning Webwhiz with docker\n\n1. Clone the repo\n2. Edit the `.env.docker` file present in the root of the repo and add your `OPENAI_KEY` \u0026 `OPENAI_KEY_2`\n3. Use docker-compose to start the stack\n\n```bash\n# Bring up webwhiz\n# Once the building is done and webwhiz starts the UI will be available at\n# http://localhost:3030, backend is available at http://localhost:3000\n# To exit Press Ctrl-C\ndocker-compose up\n\n# Alternatively Run webwhiz as a daemon\ndocker-compose up -d\n\n# Stop Webwhiz\ndocker-compose down\n\n# Force rebuild all containers (required only if some change is not picked up)\nsudo docker-compose up --build --force-recreate\n```\n\n### 🛃 Manual\n\nWebWhiz is designed to be used as a production grade Chatbot that can be scaled up or down to handle any volume of data.\n\nWebWhiz consists of mainly 3 components\n\n1. The API server - This is the main webwhiz backend web server using NestJS\n2. JS Celery Worker - Handles crawling, embeddings generation\n3. Python Celery Worker - Container Cosine similarity calculator \u0026 HTML / PDF content extractor\n\nFor Database and Caching Webwhiz uses\n\n- MongoDB\n- Redis\n\nThe backend server uses third part services (including OpenAI) for powering the chatbot, as well as for error monitoring etc. Only OpenAI key is mandatory and you can ignore the others if you prefer to.\n\n\u003e NOTE: WebWhiz keeps embeddings in Redis to improve the performance of chatbot responses. For most organisations the chatbots created would be conatins data for a few hundren or thousands of pages, and Redis should work well while providing better performance. If you would like to use a dedicated vector database for searching relavant chunks please reach out to us.\n\n#### Prerequisites\n\n- MongoDB v6\n- Redis v7\n- Node v18 + Yarn\n- Python v3.6+\n\n#### Setting Enviornment Variables\n\n1. Create a copy of the `.env.sample` file and rename as `.env`\n\nThe following variables as mandatory\n\n- HOST - IP to which the web server should bind to typically `0.0.0.0`\n- PORT - Port on which web server should listen to (Default 3000)\n- SECRET_KEY - Secret used for encryption (JWT, etc).\n- MONGO_URI - MongoDB uri to use\n- MONGO_DBNAME - Name of Database inside MongoDB\n- OPENAI_KEY - OpenAI API key\n- OPENAI_KEY_2 - Alternate OpenAI API key, used when the primary one raises error. You can use the same API key for both if you don't want to provide two separate API keys\n\n2. Inside the workers folder create a copy of the `.env.sample` and rename as `.env`.\n\nSet the value for the following variables - `MONGO_URI`, `MONGO_DBNAME`, `REDIS_HOST`, `REDIS_PORT`\n\n#### Installing dependencies and running app\n\nFrom the root folder run the following commands\n\n```bash\n# Install node dependencies\nyarn install\n\n# Install python worker dependencies\ncd workers\npython3 -m venv venv\nsource venv/bin/activate\npip install -r requirements.txt\n\n# Run application with pm2\ncd ..\nyarn run build\nnpm install -g pm2 # Use sudo if required\npm2 start ecosystem.config.js\n```\n\nThis will start the backend http server, the js worker and the python worker\n\n#### Frontend \n\nCreate `.env` file in the frontend folder and add the following variables \n\n```bash\nREACT_APP_BASE_URL='https://api.website.com'\nGOOGLE_AUTH_ID='Only if you need google login'\n```\n\nFrom the frontend folder run the following commands to start the server\n```bash\n# Install dependencies\nnpm install\n\n# Run front end app\nnpm run start\n```\n\nRun `npm run build` to package the frontend app\n\nIf you face any issues, reach out to hi@webwhiz.ai\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwebwhiz-ai%2Fwebwhiz","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwebwhiz-ai%2Fwebwhiz","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwebwhiz-ai%2Fwebwhiz/lists"}