{"id":30106772,"url":"https://github.com/glav/deepresearch","last_synced_at":"2025-08-10T01:09:51.471Z","repository":{"id":302000835,"uuid":"1010899759","full_name":"glav/deepresearch","owner":"glav","description":"Experimentation with Deep research API's and systems","archived":false,"fork":false,"pushed_at":"2025-07-23T05:25:25.000Z","size":134,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-23T06:22:15.704Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/glav.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-30T02:26:15.000Z","updated_at":"2025-07-23T05:24:05.000Z","dependencies_parsed_at":"2025-06-30T03:37:33.820Z","dependency_job_id":"d292b616-d4c0-4fea-935b-f65df4334b00","html_url":"https://github.com/glav/deepresearch","commit_stats":null,"previous_names":["glav/deepresearch"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/glav/deepresearch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glav%2Fdeepresearch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glav%2Fdeepresearch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glav%2Fdeepresearch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glav%2Fdeepresearch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/glav","download_url":"https://codeload.github.com/glav/deepresearch/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/glav%2Fdeepresearch/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269660456,"owners_count":24455292,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-09T02:00:10.424Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-10T01:09:50.264Z","updated_at":"2025-08-10T01:09:51.385Z","avatar_url":"https://github.com/glav.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep Research Tests\n\nA comprehensive multi-provider research platform for experimenting with advanced AI research models. This project provides a unified framework for conducting structured research using multiple AI providers including OpenAI, Azure OpenAI, and Azure AI Foundry, with built-in evaluation capabilities.\n\n## Overview\n\nThis project demonstrates how to use advanced AI research capabilities across multiple providers to perform automated research tasks. It includes multiple research clients, an evaluation system, and modular components that can process complex queries and generate structured outputs with reasoning traces.\n\n## Features\n\n- **Multi-Provider Support**: Integrates with OpenAI, Azure OpenAI, and Azure AI Foundry research models\n- **Advanced Model Support**: Utilizes o3-deep-research, o4-mini-deep-research, and Azure model variants\n- **AI Output Evaluation**: Built-in Azure AI evaluation system for assessing research quality\n- **Custom Prompts**: Configurable research prompts and system messages for different research scenarios\n- **Structured Output**: Generates organized markdown files with research results and reasoning traces\n- **Background Processing**: Supports long-running research tasks with real-time status monitoring\n- **Modular Architecture**: Reusable components for output processing and client management\n- **Flexible Tools**: Extensible tool configuration for different research scenarios\n\n## Project Structure\n\n```\ndeepresearch/\n├── src/\n│   ├── app.py                    # Main application entry point\n│   ├── basic_client.py           # Basic OpenAI completion client\n│   ├── openai_deep_client.py     # OpenAI deep research client\n│   ├── azure_deep_client.py      # Azure OpenAI deep research client\n│   ├── aifoundry_deep_client.py  # Azure AI Foundry research client\n│   ├── azure_evaluator.py        # Azure AI evaluation component\n│   ├── output_processor.py       # Reusable output processing module\n│   ├── terminal_spinner.py       # Terminal progress indicators\n│   ├── models.py                 # Model configurations and mappings\n│   ├── input_parameters.py       # Research input parameter definitions\n│   ├── load_env.py               # Environment variable loader\n│   ├── prompts.py                # Research prompts and system messages\n│   └── README_EVALUATOR.md       # Azure evaluator documentation\n├── experiments/                  # Research experiment results\n│   └── experiment-1/             # Sample experiment outputs\n├── output/                       # Generated research outputs\n│   ├── output_items*.md          # Complete research results by provider\n│   └── reasoning*.md             # Reasoning traces and analysis\n├── data/                         # Sample datasets and test data\n│   └── sample_dataset.json       # Sample evaluation dataset\n├── .copilot/                     # Project modification documentation\n├── requirements.txt              # Production dependencies\n├── requirements-dev.txt          # Development dependencies\n└── README.md                     # This file\n```\n\n## Setup\n\n### Prerequisites\n\n- Python 3.8+\n- API access to one or more of the following:\n  - OpenAI API key with access to deep research models\n  - Azure OpenAI endpoint and API key\n  - Azure AI Foundry project endpoint and credentials\n\n### Installation\n\n1. Clone the repository:\n   ```bash\n   git clone \u003crepository-url\u003e\n   cd deepresearch\n   ```\n\n2. Install dependencies:\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n3. Set up environment variables:\n   Create a `.env` file in the root directory with the relevant provider credentials:\n\n   **For OpenAI:**\n   ```env\n   OPENAI_API_KEY=your_openai_api_key_here\n   ```\n\n   **For Azure OpenAI:**\n   ```env\n   AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/\n   AZURE_OPENAI_API_KEY=your_azure_api_key_here\n   AZURE_OPENAI_API_VERSION=2024-12-01-preview\n   ```\n\n   **For Azure AI Foundry:**\n   ```env\n   PROJECT_ENDPOINT=your_project_endpoint\n   BING_RESOURCE_NAME=your_bing_connection_name\n   MODEL_DEPLOYMENT_NAME=your_model_deployment\n   DEEP_RESEARCH_MODEL_DEPLOYMENT_NAME=your_deep_research_model\n   ```\n\n## Usage\n\n### Basic Usage\n\nRun the main application:\n```bash\npython src/app.py\n```\n\nThis will execute the configured research client. By default, it runs Azure OpenAI research, but you can modify `src/app.py` to use different providers:\n\n```python\n# Choose your research provider:\ndo_openai_research(prompts.example_system_message, prompts.example_user_query)     # OpenAI\ndo_azure_research(prompts.example_system_message, prompts.example_user_query)      # Azure OpenAI\ndo_aifoundry_research(prompts.example_system_message, prompts.example_user_query)  # Azure AI Foundry\n```\n\n### Provider-Specific Usage\n\n**OpenAI Deep Research:**\n```bash\n# Modify app.py to uncomment the OpenAI client\npython src/openai_deep_client.py\n```\n\n**Azure OpenAI Research:**\n```bash\n# Modify app.py to uncomment the Azure client\npython src/azure_deep_client.py\n```\n\n**Azure AI Foundry Research:**\n```bash\n# Modify app.py to uncomment the AI Foundry client\npython src/aifoundry_deep_client.py\n```\n\n### AI Output Evaluation\n\nEvaluate the quality of generated research using the Azure AI evaluation component:\n\n```bash\n# Basic evaluation\npython src/azure_evaluator.py --input data/sample_dataset.json\n\n# Specify evaluation metrics\npython src/azure_evaluator.py --input data/sample_dataset.json --metrics relevance,coherence,bleu\n\n# Use AI-assisted metrics (requires Azure OpenAI)\npython src/azure_evaluator.py --input data/sample_dataset.json --metrics relevance,coherence \\\n  --model-endpoint \"https://your-endpoint.openai.azure.com/\" \\\n  --model-api-key \"your-api-key\" \\\n  --model-deployment \"your-deployment-name\"\n```\n\nSee `src/README_EVALUATOR.md` for detailed evaluation documentation.\n\n### Customizing Research\n\n1. **Modify prompts**: Edit `src/prompts.py` to change the research focus:\n   - `system_message`: Define the research assistant's role and behavior\n   - `user_query`: Specify the research question or topic\n   - `tools_input`: Configure available tools for the research\n\n2. **Example prompt configurations**:\n   ```python\n   # Health economics research\n   system_message = \"You are a professional researcher preparing a structured, data-driven report...\"\n   user_query = \"Research the economic impact of semaglutide on global healthcare systems.\"\n\n   # Creative research\n   system_message = \"You are a dad joke researcher...\"\n   user_query = \"Research dad jokes about nerds.\"\n   ```\n\n### Understanding Output\n\nThe research generates provider-specific output files in the `output/` directory:\n\n**OpenAI Research Output:**\n- **`output_items_openai_[model_name].md`**: Complete research results including all content types\n- **`reasoning_openai_[model_name].md`**: Detailed reasoning traces showing the model's thought process\n\n**Azure OpenAI Research Output:**\n- **`output_items_azure_[model_name].md`**: Complete Azure research results\n- **`reasoning_azure_[model_name].md`**: Azure reasoning traces\n\n**Azure AI Foundry Research Output:**\n- **`output_items_aifoundry_o3_deep_research.md`**: AI Foundry research results with token usage metrics\n\n**Evaluation Output:**\n- **`evaluation_results_[timestamp].json`**: Detailed evaluation metrics in JSON format\n- **`evaluation_report_[timestamp].md`**: Human-readable evaluation report\n\nEach output file includes comprehensive metadata, token usage statistics, and execution timing information.\n\n## Configuration\n\n### Model Selection\n\nThe project supports multiple research models across different providers:\n\n**OpenAI Models:**\n- `o3-deep-research`: Advanced research model with comprehensive capabilities\n- `o4-mini-deep-research-2025-06-26`: Lightweight research model for faster execution\n\n**Azure OpenAI Models:**\n- `o3-mini`: Fast, efficient model for quick research tasks\n- `o3-pro`: Professional-grade model with enhanced capabilities\n- `o4-mini`: Latest generation lightweight model\n\n**Azure AI Foundry Models:**\n- Configurable deep research models with Bing integration\n- Custom model deployments based on your AI Foundry setup\n\nConfigure the model in the respective client files:\n```python\n# In openai_deep_client.py\nmodel = OPENAI_03  # or OPENAI_04_MINI\n\n# In azure_deep_client.py\nmodel = AZURE_03_MINI  # or AZURE_03_PRO, AZURE_04_MINI\n\n# In aifoundry_deep_client.py\n# Model configured via environment variables\n```\n\n### Research Parameters\n\nKey parameters you can adjust across all providers:\n- **Background processing**: Set `background=True` for long-running tasks\n- **Tools**: Configure available research tools (varies by provider)\n- **Reasoning**: Customize reasoning prompts for specific research styles\n- **Model selection**: Choose appropriate model for your use case and performance needs\n\n## Development\n\n### Running in Development Mode\n\nFor development, install additional dependencies:\n```bash\npip install -r requirements-dev.txt\n```\n\n### Project Components\n\n- **`app.py`**: Main entry point that orchestrates the research process across providers\n- **`openai_deep_client.py`**: OpenAI deep research client with response processing\n- **`azure_deep_client.py`**: Azure OpenAI research client with Azure-specific configurations\n- **`aifoundry_deep_client.py`**: Azure AI Foundry client with agent-based research\n- **`azure_evaluator.py`**: Standalone evaluation component for AI output assessment\n- **`output_processor.py`**: Reusable module for processing and formatting research outputs\n- **`terminal_spinner.py`**: Progress indicators for long-running research tasks\n- **`models.py`**: Model configurations and provider mappings\n- **`basic_client.py`**: Simple completion client for basic tasks\n- **`load_env.py`**: Environment configuration management\n- **`prompts.py`**: Research prompt definitions and configurations\n- **`input_parameters.py`**: Parameter definitions for research inputs\n\n## Dependencies\n\nKey dependencies include:\n- **Core Libraries:**\n  - `openai`: OpenAI API client for OpenAI and Azure OpenAI services\n  - `azure-ai-projects`: Azure AI Foundry project integration\n  - `azure-ai-agents`: Azure AI agent framework for foundry clients\n  - `azure-ai-evaluation`: Azure AI evaluation metrics and tools\n  - `azure-identity`: Azure authentication and credential management\n- **Utilities:**\n  - `python-dotenv`: Environment variable management\n  - `tqdm`: Progress tracking for research operations\n  - `GitPython`: Git integration capabilities\n- **Development:**\n  - Additional packages in `requirements-dev.txt` for development workflows\n\n## Examples\n\n### Multi-Provider Research Comparison\n```python\n# Health economics research across providers\nsystem_message = \"\"\"\nYou are a professional researcher preparing a structured, data-driven report...\n\"\"\"\nuser_query = \"Research the economic impact of semaglutide on global healthcare systems.\"\n\n# Run the same research across different providers for comparison\ndo_openai_research(system_message, user_query)\ndo_azure_research(system_message, user_query)\ndo_aifoundry_research(system_message, user_query)\n```\n\n### Specialized Research Scenarios\n```python\n# Environmental research with EPA focus\ndeep_research_epa_system_prompt = \"\"\"\nYou are an AI agent that assists in deep research experiments based on\nEnvironmental Protection Authority information and documentation...\n\"\"\"\nepa_query = \"Are there any Proposal Elements with Greenhouse Gas Emissions\nthat involve electricity generation for Hope Downs?\"\n\n# Creative research\ndadjoke_system_message = \"You are a dad joke researcher...\"\ndadjoke_user_query = \"Research dad jokes about nerds.\"\n```\n\n## Output Format\n\nResearch results are structured with comprehensive metadata and multiple content types:\n\n### Content Types by Provider\n- **OpenAI/Azure OpenAI**: Text responses, web searches, reasoning traces, code interpreter calls\n- **Azure AI Foundry**: Agent responses, URL citations, deep research tool outputs\n- **All Providers**: Token usage metrics, execution timing, status updates\n\n### Output Structure\n- **Item Types**: Different content categories (reasoning, web_search_call, text, code_interpreter_call)\n- **Summaries**: Key findings and insights with structured data presentation\n- **Status Updates**: Real-time progress tracking throughout the research process\n- **Source Citations**: References, URLs, and metadata for all retrieved information\n- **Performance Metrics**: Token usage, execution time, and provider-specific statistics\n\n### Evaluation Metrics\nWhen using the Azure evaluator component:\n- **Quality Metrics**: Relevance, Coherence, Fluency scores\n- **Similarity Metrics**: BLEU, ROUGE, F1, METEOR scores\n- **Custom Metrics**: Configurable evaluation criteria based on research needs\n\n## Experiments\n\nThe `experiments/` directory contains sample research runs demonstrating:\n- **experiment-1/**: Comparative outputs across OpenAI models (o3, o4-mini) showing different research approaches\n- Provider-specific output formats and reasoning patterns\n- Performance comparisons between model variants\n- Real-world research scenarios with actual data and citations\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Test thoroughly across multiple providers\n5. Update documentation as needed\n6. Submit a pull request\n\nWhen contributing:\n- Test changes with multiple AI providers when applicable\n- Update the `.copilot/` documentation for significant modifications\n- Follow the existing code patterns for new client implementations\n- Ensure new features work with the modular output processing system\n\n## License\n\nThis project is a sandbox for experimentation and learning purposes with advanced AI research capabilities.\n\n---\n\n*Last updated: July 11, 2025*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fglav%2Fdeepresearch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fglav%2Fdeepresearch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fglav%2Fdeepresearch/lists"}