https://github.com/oevortex/webscout
Webscout is the all-in-one search and AI toolkit you need. Discover insights with Yep.com, DuckDuckGo, and Phind; access cutting-edge AI models; transcribe YouTube videos; generate temporary emails and phone numbers; perform text-to-speech conversions; and much more!
https://github.com/oevortex/webscout
ai api free freeai g4f gguf llamacpp localgpt ml ollama openai openinterpreter python tempmail tempnumber text-generation websearch youtube youtube-api
Last synced: 7 days ago
JSON representation
Webscout is the all-in-one search and AI toolkit you need. Discover insights with Yep.com, DuckDuckGo, and Phind; access cutting-edge AI models; transcribe YouTube videos; generate temporary emails and phone numbers; perform text-to-speech conversions; and much more!
- Host: GitHub
- URL: https://github.com/oevortex/webscout
- Owner: OEvortex
- License: other
- Created: 2024-02-27T05:02:20.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2025-03-28T17:06:03.000Z (10 months ago)
- Last Synced: 2025-03-29T14:17:42.934Z (10 months ago)
- Topics: ai, api, free, freeai, g4f, gguf, llamacpp, localgpt, ml, ollama, openai, openinterpreter, python, tempmail, tempnumber, text-generation, websearch, youtube, youtube-api
- Language: Python
- Homepage: https://pypi.org/project/webscout/
- Size: 6.71 MB
- Stars: 207
- Watchers: 4
- Forks: 39
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE.md
Awesome Lists containing this project
README
Webscout
Your All-in-One Python Toolkit for Web Search, AI Interaction, Digital Utilities, and More
Access diverse search engines, cutting-edge AI models, temporary communication tools, media utilities, developer helpers, and powerful CLI interfaces – all through one unified library.
## 📋 Table of Contents
- [🌟 Key Features](#-features)
- [⚙️ Installation](#️-installation)
- [🖥️ Command Line Interface](#️-command-line-interface)
- [📚 Documentation Hub](docs/README.md)
- [🔄 OpenAI-Compatible API Server](docs/openai-api-server.md)
- [🕸️ Scout: HTML Parser & Web Crawler](docs/scout.md)
- [🎭 Awesome Prompts Manager](docs/awesome-prompts.md)
- [🔗 GitAPI: GitHub Data Extraction](docs/gitapi.md)
- [🤖 AI Models and Voices](#-ai-models-and-voices)
- [💬 AI Chat Providers](#-ai-chat-providers)
- [👨💻 Advanced AI Interfaces](#-advanced-ai-interfaces)
- [🤝 Contributing](#-contributing)
- [🙏 Acknowledgments](#-acknowledgments)
> [!IMPORTANT]
> **Webscout supports three types of compatibility:**
>
> - **Native Compatibility:** Webscout's own native API for maximum flexibility
> - **OpenAI Compatibility:** Use providers with OpenAI-compatible interfaces
> - **Local LLM Compatibility:** Run local models with OpenAI-compatible servers
>
> Choose the approach that best fits your needs! For OpenAI compatibility, check the [OpenAI Providers README](webscout/Provider/OPENAI/README.md) or see the [OpenAI-Compatible API Server](#-openai-compatible-api-server) section below.
> [!NOTE]
> Webscout supports over 90 AI providers including: LLAMA, C4ai, Copilot, HuggingFaceChat, PerplexityLabs, DeepSeek, WiseCat, GROQ, OPENAI, GEMINI, DeepInfra, Meta, YEPCHAT, TypeGPT, ChatGPTClone, ExaAI, Claude, Anthropic, Cloudflare, AI21, Cerebras, and many more. All providers follow similar usage patterns with consistent interfaces.
## 🚀 Features
Search & AI
- **Comprehensive Search:** Access multiple search engines including DuckDuckGo, Yep, Bing, Brave, Yahoo, Yandex, Mojeek, and Wikipedia for diverse search results ([Search Documentation](docs/search.md))
- **AI Powerhouse:** Access and interact with various AI models through three compatibility options:
- **Native API:** Use Webscout's native interfaces for providers like OpenAI, Cohere, Gemini, and many more
- **[OpenAI-Compatible Providers](webscout/Provider/OPENAI/README.md):** Seamlessly integrate with various AI providers using standardized OpenAI-compatible interfaces
- **Local LLMs:** Run local models with OpenAI-compatible servers (see [Inferno documentation](docs/inferno.md))
- **[AI Search](webscout/Provider/AISEARCH/README.md):** AI-powered search engines with advanced capabilities
- **[OpenAI-Compatible API Server](docs/openai-api-server.md):** Run a local API server that serves any Webscout provider through OpenAI-compatible endpoints
- **[Python Client API](docs/client.md):** Use Webscout providers directly in Python with OpenAI-compatible format
Media & Content Tools
- **[YouTube Toolkit](webscout/Extra/YTToolkit/README.md):** Advanced YouTube video and transcript management with multi-language support
- **[Text-to-Speech (TTS)](webscout/Provider/TTS/README.md):** Convert text into natural-sounding speech using multiple AI-powered providers
- **[Text-to-Image](webscout/Provider/TTI/README.md):** Generate high-quality images using a wide range of AI art providers
- **[Weather Tools](docs/weather.md):** Retrieve detailed weather information for any location
Developer Tools
- **[GitAPI](docs/gitapi.md):** Powerful GitHub data extraction toolkit without authentication requirements for public data
- **[SwiftCLI](docs/swiftcli.md):** A powerful and elegant CLI framework for beautiful command-line interfaces
- **[LitPrinter](docs/litprinter.md):** Styled console output with rich formatting and colors
- **[LitAgent](docs/litagent.md):** Modern user agent generator that keeps your requests undetectable
- **[Scout](docs/scout.md):** Advanced web parsing and crawling library with intelligent HTML/XML parsing
- **[GGUF Conversion](webscout/Extra/gguf.md):** Convert and quantize Hugging Face models to GGUF format
- **[Utility Decorators](docs/decorators.md):** Easily measure function execution time (`timeIt`) and add retry logic (`retry`) to any function
- **[Stream Sanitization Utilities](docs/sanitize.md):** Advanced tools for cleaning, decoding, and processing data streams
- **[Command Line Interface](docs/cli.md):** Comprehensive CLI for all search engines and utilities
Privacy & Utilities
- **[Tempmail](webscout/Extra/tempmail/README.md) & Temp Number:** Generate temporary email addresses and phone numbers
- **[Awesome Prompts Manager](docs/awesome-prompts.md):** Curated collection of system prompts for specialized AI personas with comprehensive management capabilities
## ⚙️ Installation
Webscout supports multiple installation methods to fit your workflow:
### 📦 Standard Installation
```bash
# Install from PyPI
pip install -U webscout
# Install with API server dependencies
pip install -U "webscout[api]"
# Install with development dependencies
pip install -U "webscout[dev]"
```
### ⚡ UV Package Manager (Recommended)
[UV](https://github.com/astral-sh/uv) is a fast Python package manager. Webscout has full UV support:
```bash
# Install UV first (if not already installed)
pip install uv
# Install Webscout with UV
uv add webscout
# Install with API dependencies
uv add "webscout[api]"
# Run Webscout directly with UV (no installation needed)
uv run webscout --help
# Run with API dependencies
uv run webscout --extra api webscout-server
# Install as a UV tool for global access
uv tool install webscout
# Use UV tool commands
webscout --help
webscout-server
```
### 🔧 Development Installation
```bash
# Clone the repository
git clone https://github.com/OEvortex/Webscout.git
cd Webscout
# Install in development mode with UV
uv sync --extra dev --extra api
# Or with pip
pip install -e ".[dev,api]"
# Or with uv pip
uv pip install -e ".[dev,api]"
```
### 🐳 Docker Installation
```bash
# Pull and run the Docker image
docker pull OEvortex/webscout:latest
docker run -it OEvortex/webscout:latest
```
### 📱 Quick Start Commands
After installation, you can immediately start using Webscout:
```bash
# Check version
webscout version
# Search the web
webscout text -k "python programming"
# Start API server
webscout-server
# Get help
webscout --help
```
## 🖥️ Command Line Interface
Webscout provides a comprehensive command-line interface with support for multiple search engines and utilities. You can use it in multiple ways:
### 🚀 Direct Commands (Recommended)
After installing with `uv tool install webscout` or `pip install webscout`:
```bash
# Get help and list all commands
webscout --help
# Show version
webscout version
# Start API server
webscout-server
# Web search commands
webscout text -k "python programming" # DuckDuckGo text search
webscout images -k "mountain landscape" # DuckDuckGo image search
webscout news -k "AI breakthrough" -t w # News from last week
webscout weather -l "New York" # Weather information
webscout translate -k "Hello" -to es # Translation
# Alternative search engines
webscout yahoo_text -k "machine learning" -r us # Yahoo search
webscout bing_text -k "climate change" # Bing search
webscout yep_text -k "latest news" # Yep search
# Search with advanced options
webscout images -k "cat" --size large --type-image photo --license-image any
webscout maps -k "coffee shop" --city "New York" --radius 5
```
### 🔧 UV Run Commands (No Installation Required)
```bash
# Run directly with UV (downloads and runs automatically)
uv run webscout --help
uv run webscout text -k "latest news"
uv run --extra api webscout-server
```
### 📦 Python Module Commands
```bash
# Traditional Python module execution
python -m webscout --help
python -m webscout text -k "search query"
python -m webscout-server
```
### 🌐 Supported Search Providers
Webscout CLI supports multiple search backends:
- **DuckDuckGo** (default): `text`, `images`, `videos`, `news`, `answers`, `maps`, `translate`, `suggestions`, `weather`
- **Yahoo**: `yahoo_text`, `yahoo_images`, `yahoo_videos`, `yahoo_news`, `yahoo_answers`, `yahoo_maps`, `yahoo_translate`, `yahoo_suggestions`, `yahoo_weather`
- **Bing**: `bing_text`, `bing_images`, `bing_news`, `bing_suggestions`
- **Yep**: `yep_text`, `yep_images`, `yep_suggestions`
For detailed command reference and all available options, see [CLI Documentation](docs/cli.md).
## 🤖 AI Models and Voices
Webscout provides easy access to a wide range of AI models and voice options.
LLM Models
Access and manage Large Language Models with Webscout's model utilities.
```python
from webscout import model
from rich import print
# List all available LLM models
all_models = model.llm.list()
print(f"Total available models: {len(all_models)}")
# Get a summary of models by provider
summary = model.llm.summary()
print("Models by provider:")
for provider, count in summary.items():
print(f" {provider}: {count} models")
# Get models for a specific provider
provider_name = "PerplexityLabs"
available_models = model.llm.get(provider_name)
print(f"\n{provider_name} models:")
if isinstance(available_models, list):
for i, model_name in enumerate(available_models, 1):
print(f" {i}. {model_name}")
else:
print(f" {available_models}")
```
TTS Voices
Access and manage Text-to-Speech voices across multiple providers.
```python
from webscout import model
from rich import print
# List all available TTS voices
all_voices = model.tts.list()
print(f"Total available voices: {len(all_voices)}")
# Get a summary of voices by provider
summary = model.tts.summary()
print("\nVoices by provider:")
for provider, count in summary.items():
print(f" {provider}: {count} voices")
# Get voices for a specific provider
provider_name = "ElevenlabsTTS"
available_voices = model.tts.get(provider_name)
print(f"\n{provider_name} voices:")
if isinstance(available_voices, dict):
for voice_name, voice_id in list(available_voices.items())[:5]: # Show first 5 voices
print(f" - {voice_name}: {voice_id}")
if len(available_voices) > 5:
print(f" ... and {len(available_voices) - 5} more")
```
## 💬 AI Chat Providers
Webscout offers a comprehensive collection of AI chat providers, giving you access to various language models through a consistent interface.
### Popular AI Providers
| Provider | Description | Key Features |
| ---------------- | ------------------------ | ---------------------------------- |
| `OPENAI` | OpenAI's models | GPT-3.5, GPT-4, tool calling |
| `GEMINI` | Google's Gemini models | Web search capabilities |
| `Meta` | Meta's AI assistant | Image generation, web search |
| `GROQ` | Fast inference platform | High-speed inference, tool calling |
| `LLAMA` | Meta's Llama models | Open weights models |
| `DeepInfra` | Various open models | Multiple model options |
| `Cohere` | Cohere's language models | Command models |
| `PerplexityLabs` | Perplexity AI | Web search integration |
| `YEPCHAT` | Yep.com's AI | Streaming responses |
| `ChatGPTClone` | ChatGPT-like interface | Multiple model options |
| `TypeGPT` | TypeChat models | Multiple model options |
Example: Using Meta AI
```python
from webscout import Meta
# For basic usage (no authentication required)
meta_ai = Meta()
# Simple text prompt
response = meta_ai.chat("What is the capital of France?")
print(response)
# For authenticated usage with web search and image generation
meta_ai = Meta(fb_email="your_email@example.com", fb_password="your_password")
# Text prompt with web search
response = meta_ai.ask("What are the latest developments in quantum computing?")
print(response["message"])
print("Sources:", response["sources"])
# Image generation
response = meta_ai.ask("Create an image of a futuristic city")
for media in response.get("media", []):
print(media["url"])
```
Example: GROQ with Tool Calling
```python
from webscout import GROQ, DuckDuckGoSearch
import json
# Initialize GROQ client
client = GROQ(api_key="your_api_key")
# Define helper functions
def calculate(expression):
"""Evaluate a mathematical expression"""
try:
result = eval(expression)
return json.dumps({"result": result})
except Exception as e:
return json.dumps({"error": str(e)})
def search(query):
"""Perform a web search"""
try:
ddg = DuckDuckGoSearch()
results = ddg.text(query, max_results=3)
return json.dumps({"results": results})
except Exception as e:
return json.dumps({"error": str(e)})
# Register functions with GROQ
client.add_function("calculate", calculate)
client.add_function("search", search)
# Define tool specifications
tools = [
{
"type": "function",
"function": {
"name": "calculate",
"description": "Evaluate a mathematical expression",
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "The mathematical expression to evaluate"
}
},
"required": ["expression"]
}
}
},
{
"type": "function",
"function": {
"name": "search",
"description": "Perform a web search",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The search query"
}
},
"required": ["query"]
}
}
}
]
# Use the tools
response = client.chat("What is 25 * 4 + 10?", tools=tools)
print(response)
response = client.chat("Find information about quantum computing", tools=tools)
print(response)
```
GGUF Model Conversion
Webscout provides tools to convert and quantize Hugging Face models into the GGUF format for offline use.
> **Note (2026.01.01)**: GGUF conversion now uses lazy imports for `huggingface_hub`. The library can be imported without requiring `huggingface_hub`, and it's only loaded when GGUF features are actually used. Install it with `pip install huggingface_hub` if you need GGUF conversion.
```python
from webscout.Extra.gguf import ModelConverter
# Create a converter instance
converter = ModelConverter(
model_id="mistralai/Mistral-7B-Instruct-v0.2", # Hugging Face model ID
quantization_methods="q4_k_m" # Quantization method
)
# Run the conversion
converter.convert()
```
#### Available Quantization Methods
| Method | Description |
| -------- | ------------------------------------------------------------- |
| `fp16` | 16-bit floating point - maximum accuracy, largest size |
| `q2_k` | 2-bit quantization (smallest size, lowest accuracy) |
| `q3_k_l` | 3-bit quantization (large) - balanced for size/accuracy |
| `q3_k_m` | 3-bit quantization (medium) - good balance for most use cases |
| `q3_k_s` | 3-bit quantization (small) - optimized for speed |
| `q4_0` | 4-bit quantization (version 0) - standard 4-bit compression |
| `q4_1` | 4-bit quantization (version 1) - improved accuracy over q4_0 |
| `q4_k_m` | 4-bit quantization (medium) - balanced for most models |
| `q4_k_s` | 4-bit quantization (small) - optimized for speed |
| `q5_0` | 5-bit quantization (version 0) - high accuracy, larger size |
| `q5_1` | 5-bit quantization (version 1) - improved accuracy over q5_0 |
| `q5_k_m` | 5-bit quantization (medium) - best balance for quality/size |
| `q5_k_s` | 5-bit quantization (small) - optimized for speed |
| `q6_k` | 6-bit quantization - highest accuracy, largest size |
| `q8_0` | 8-bit quantization - maximum accuracy, largest size |
#### Command Line Usage
```bash
python -m webscout.Extra.gguf convert -m "mistralai/Mistral-7B-Instruct-v0.2" -q "q4_k_m"
```
## 🤝 Contributing
Contributions are welcome! If you'd like to contribute to Webscout, please follow these steps:
1. Fork the repository
2. Create a new branch for your feature or bug fix
3. Make your changes and commit them with descriptive messages
4. Push your branch to your forked repository
5. Submit a pull request to the main repository
## 🙏 Acknowledgments
- All the amazing developers who have contributed to the project
- The open-source community for their support and inspiration
Made with ❤️ by the Webscout team