{"id":30805316,"url":"https://github.com/ajitashwath/researcher","last_synced_at":"2025-09-06T00:55:33.946Z","repository":{"id":302884744,"uuid":"1013613631","full_name":"ajitashwath/researcher","owner":"ajitashwath","description":"A web based AI application that creates comprehensive reports on any topic using AI agents.","archived":false,"fork":false,"pushed_at":"2025-08-14T11:11:21.000Z","size":352,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-14T11:31:42.018Z","etag":null,"topics":["crewai","docker","gcp","streamlit"],"latest_commit_sha":null,"homepage":"https://create-report-kar.streamlit.app/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ajitashwath.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-04T07:24:19.000Z","updated_at":"2025-08-14T11:11:24.000Z","dependencies_parsed_at":"2025-07-30T19:17:50.654Z","dependency_job_id":"44ec7e4a-60e4-4419-b527-ecf0bd92da6b","html_url":"https://github.com/ajitashwath/researcher","commit_stats":null,"previous_names":["ajitashwath/create-report","ajitashwath/researcher"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ajitashwath/researcher","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajitashwath%2Fresearcher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajitashwath%2Fresearcher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajitashwath%2Fresearcher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajitashwath%2Fresearcher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ajitashwath","download_url":"https://codeload.github.com/ajitashwath/researcher/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ajitashwath%2Fresearcher/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273842847,"owners_count":25177921,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-05T02:00:09.113Z","response_time":402,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crewai","docker","gcp","streamlit"],"created_at":"2025-09-06T00:55:30.852Z","updated_at":"2025-09-06T00:55:33.935Z","avatar_url":"https://github.com/ajitashwath.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Researcher Agent\nA web based AI application that creates comprehensive reports on any topic using AI agents.\n## Features\n\n- 🤖 **Multi-Agent AI System**: Research, analysis, writing, and review agents working together\n- 📊 **Comprehensive Reports**: Generate detailed reports on any topic\n- 🌐 **Multiple Interfaces**: Web UI (Streamlit), REST API (FastAPI), and CLI\n- ☁️ **Cloud-Ready**: Docker containerization with GCP deployment support\n- 🔧 **Configurable**: Customizable agents, tasks, and report formats\n- 🔍 **Advanced Research**: Integration with search tools and OpenAI for comprehensive research\n\n## Installation\n\nEnsure you have Python \u003e=3.10 \u003c3.14 installed on your system. This project uses [UV](https://docs.astral.sh/uv/) for dependency management and package handling, offering a seamless setup and execution experience.\n\nFirst, if you haven't already, install uv:\n\n```bash\npip install uv\n```\n\nNext, navigate to your project directory and install the dependencies:\n\n```bash\n# Install using pip\npip install -r requirements.txt\n\n# Or using crewai CLI (optional)\ncrewai install\n```\n\n## Environment Configuration\n\nCreate a `.env` file in the root directory with the following variables:\n\n```env\n# Required API Keys\nOPENAI_API_KEY=your_openai_api_key_here\nGROQ_API_KEY=your_groq_api_key_here\nSERPER_API_KEY=your_serper_api_key_here\n\n# Optional Configuration\nPYTHONDONTWRITEBYTECODE=1\nPYTHONUNBUFFERED=1\n```\n\n**Required API Keys:**\n- `OPENAI_API_KEY`: For AI content generation and analysis\n- `GROQ_API_KEY`: For enhanced AI processing capabilities\n- `SERPER_API_KEY`: For web search functionality\n\n## Usage Options\n\n### 1. Streamlit Web Interface\n\nRun the interactive web application:\n\n```bash\nstreamlit run app.py\n```\n\nAccess the application at `http://localhost:8501`\n\n### 2. FastAPI REST API\n\nStart the API server:\n\n```bash\n# Development\nuvicorn api:app --reload --host 0.0.0.0 --port 8000\n\n# Production\ngunicorn -w 1 --threads 2 -k uvicorn.workers.UvicornWorker --bind 0.0.0.0:8080 api:app\n```\n\n**API Endpoints:**\n- `GET /`: Health check\n- `POST /research`: Generate reports\n\n**Example API Usage:**\n```bash\ncurl -X POST \"http://localhost:8000/research\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"topic\": \"How to improve infrastructure in Bangalore?\",\n    \"user_personalization\": \"Focus on transportation and urban planning\"\n  }'\n```\n\n### 3. Command Line Interface\n\nRun directly from the command line:\n\n```bash\ncrewai run\n```\n\nThis will generate a `report.md` file with research on LLMs (default example).\n\n## Docker Deployment\n\n### Building the Docker Image\n\nThe project includes a production-ready Dockerfile optimized for deployment:\n\n```bash\n# Build the image\ndocker build -t createreport-crew .\n\n# Run locally\ndocker run -p 8080:8080 \\\n  -e OPENAI_API_KEY=your_key_here \\\n  -e GROQ_API_KEY=your_key_here \\\n  -e SERPER_API_KEY=your_key_here \\\n  createreport-crew\n```\n\n### Docker Configuration\n\nThe Dockerfile is configured with:\n- **Base Image**: Python 3.11 slim (Debian Bullseye)\n- **Port**: 8080 (Cloud Run compatible)\n- **Server**: Gunicorn with Uvicorn workers\n- **Optimization**: Multi-threaded, production-ready configuration\n\n## Google Cloud Platform (GCP) Deployment\n\n### Prerequisites\n\n1. Install Google Cloud SDK:\n```bash\n# macOS\nbrew install google-cloud-sdk\n\n# Ubuntu/Debian\nsudo apt-get install google-cloud-sdk\n\n# Or download from: https://cloud.google.com/sdk/docs/install\n```\n\n2. Authenticate with GCP:\n```bash\ngcloud auth login\n```\n\n### Step-by-Step GCP Deployment\n\n#### 1. Project Setup\n\n```bash\n# List available projects\ngcloud projects list\n\n# Set your project ID\ngcloud config set project YOUR_PROJECT_ID\n\n# Enable required services\ngcloud services enable cloudbuild.googleapis.com artifactregistry.googleapis.com run.googleapis.com\n```\n\n#### 2. Configure Variables\n\n```powershell\n# PowerShell (Windows)\n$REPO_NAME = \"createreport-crew\"\n$REGION = \"us-central1\"  # or your preferred region\n$SERVICE_NAME = \"createreport-api\"\n```\n\n```bash\n# Bash (Linux/macOS)\nREPO_NAME=\"createreport-crew\"\nREGION=\"us-central1\"  # or your preferred region\nSERVICE_NAME=\"createreport-api\"\n```\n\n#### 3. Create Artifact Registry Repository\n\n```powershell\n# PowerShell\ngcloud artifacts repositories create $REPO_NAME `\n    --repository-format=docker `\n    --location=$REGION `\n    --description=\"CreateReport Crew Docker Repository\"\n```\n\n```bash\n# Bash\ngcloud artifacts repositories create $REPO_NAME \\\n    --repository-format=docker \\\n    --location=$REGION \\\n    --description=\"CreateReport Crew Docker Repository\"\n```\n\n#### 4. Build and Push Docker Image\n\n```powershell\n# PowerShell\n$PROJECT_ID = $(gcloud config get-value project)\n$IMAGE_TAG = \"$($REGION)-docker.pkg.dev/$($PROJECT_ID)/$($REPO_NAME)/createreport-api:latest\"\n\n# Build and push image\ngcloud builds submit --tag $IMAGE_TAG\n```\n\n```bash\n# Bash\nPROJECT_ID=$(gcloud config get-value project)\nIMAGE_TAG=\"$REGION-docker.pkg.dev/$PROJECT_ID/$REPO_NAME/createreport-api:latest\"\n\n# Build and push image\ngcloud builds submit --tag $IMAGE_TAG\n```\n\n#### 5. Deploy to Cloud Run\n\n```powershell\n# PowerShell\ngcloud run deploy $SERVICE_NAME `\n    --image=$IMAGE_TAG `\n    --platform=managed `\n    --region=$REGION `\n    --allow-unauthenticated `\n    --port=8080 `\n    --memory=2Gi `\n    --cpu=1 `\n    --timeout=900 `\n    --set-env-vars=\"PYTHONDONTWRITEBYTECODE=1,PYTHONUNBUFFERED=1\"\n```\n\n```bash\n# Bash\ngcloud run deploy $SERVICE_NAME \\\n    --image=$IMAGE_TAG \\\n    --platform=managed \\\n    --region=$REGION \\\n    --allow-unauthenticated \\\n    --port=8080 \\\n    --memory=2Gi \\\n    --cpu=1 \\\n    --timeout=900 \\\n    --set-env-vars=\"PYTHONDONTWRITEBYTECODE=1,PYTHONUNBUFFERED=1\"\n```\n\n#### 6. Set Environment Variables (Secrets)\n\nFor production deployment, set your API keys as environment variables:\n\n```bash\n# Set environment variables with secrets\ngcloud run services update $SERVICE_NAME \\\n    --region=$REGION \\\n    --set-env-vars=\"OPENAI_API_KEY=your_openai_key_here,GROQ_API_KEY=your_groq_key_here,SERPER_API_KEY=your_serper_key_here\"\n```\n\n**Better approach using Secret Manager:**\n\n```bash\n# Create secrets\necho \"your_openai_key_here\" | gcloud secrets create openai-api-key --data-file=-\necho \"your_groq_key_here\" | gcloud secrets create groq-api-key --data-file=-\necho \"your_serper_key_here\" | gcloud secrets create serper-api-key --data-file=-\n\n# Deploy with secrets\ngcloud run deploy $SERVICE_NAME \\\n    --image=$IMAGE_TAG \\\n    --region=$REGION \\\n    --set-secrets=\"OPENAI_API_KEY=openai-api-key:latest,GROQ_API_KEY=groq-api-key:latest,SERPER_API_KEY=serper-api-key:latest\"\n```\n\n### Post-Deployment\n\nAfter successful deployment, you'll receive a service URL. You can:\n\n1. **Test the API**:\n```bash\ncurl -X GET \"https://your-service-url.run.app/\"\n```\n\n2. **View logs**:\n```bash\ngcloud run services logs tail $SERVICE_NAME --region=$REGION\n```\n\n3. **Monitor the service**:\n```bash\ngcloud run services describe $SERVICE_NAME --region=$REGION\n```\n\n## Customization\n\n### Agents Configuration\n\nModify `src/create_report/config/agents.yaml` to define your agents:\n\n```yaml\nresearcher:\n  role: \"Senior Research Analyst\" \n  goal: \"Conduct comprehensive research...\"\n  backstory: \"You are a senior research analyst...\"\n```\n\n### Tasks Configuration\n\nModify `src/create_report/config/tasks.yaml` to define your tasks:\n\n```yaml\nresearch_task:\n  description: \"Conduct comprehensive research on...\"\n  expected_output: \"A comprehensive research summary...\"\n  agent: researcher\n```\n\n### Custom Logic\n\n- **Modify `src/create_report/crew.py`**: Add custom logic, tools, and specific arguments\n- **Modify `src/create_report/main.py`**: Add custom inputs and orchestration logic\n- **Modify `api.py`**: Customize API endpoints and request handling\n- **Modify `app.py`**: Customize the Streamlit interface\n\n## Project Structure\n\n```\ncreate_report/\n├── .dockerignore           # Docker ignore patterns\n├── .gitignore             # Git ignore patterns  \n├── Dockerfile             # Production Docker configuration\n├── README.md              # This file\n├── api.py                 # FastAPI REST API\n├── app.py                 # Streamlit web interface\n├── requirements.txt       # Python dependencies\n├── pyproject.toml        # Project configuration\n├── knowledge/            # Knowledge base files\n│   └── user_preference.txt\n├── src/create_report/    # Main package\n│   ├── __init__.py\n│   ├── main.py           # Main orchestration logic\n│   ├── crew.py           # Crew management\n│   ├── config/           # Configuration files\n│   │   ├── agents.yaml   # Agent definitions\n│   │   └── tasks.yaml    # Task definitions\n│   └── tools/            # Custom tools\n│       ├── __init__.py\n│       └── custom_tool.py\n```\n\n## Understanding Your Crew\n\nThe create-report Crew is composed of multiple AI agents, each with unique roles, goals, and tools:\n\n- **🔍 Researcher**: Conducts comprehensive research and gathers information\n- **📊 Analyst**: Analyzes data and identifies trends and insights  \n- **✍️ Writer**: Creates well-structured, engaging reports\n- **🔍 Reviewer**: Reviews and improves report quality and accuracy\n- **📋 Strategist**: Develops strategic recommendations and implementation plans\n\nThese agents collaborate on a series of tasks, leveraging their collective skills to achieve complex objectives.\n\n## Troubleshooting\n\n### Common Issues\n\n1. **API Key Errors**: Ensure all required API keys are set in your environment\n2. **Docker Build Failures**: Check that all dependencies are properly listed in `requirements.txt`\n3. **GCP Deployment Issues**: Verify that all required GCP services are enabled\n4. **Memory Issues**: Increase Cloud Run memory allocation if processing large reports\n\n### Docker Debugging\n\n```bash\n# Build and run locally for debugging\ndocker build -t createreport-crew-debug .\ndocker run -it --entrypoint /bin/bash createreport-crew-debug\n\n# Check logs\ndocker logs \u003ccontainer_id\u003e\n```\n\n### GCP Debugging\n\n```bash\n# View detailed logs\ngcloud run services logs tail $SERVICE_NAME --region=$REGION --format=\"value(textPayload)\"\n\n# Check service status\ngcloud run services list --region=$REGION\n\n# Describe service configuration\ngcloud run services describe $SERVICE_NAME --region=$REGION\n```\n\n## Performance Optimization\n\nFor better performance in production:\n\n1. **Increase Resources**: Adjust CPU and memory in Cloud Run deployment\n2. **Enable Caching**: Implement caching for frequently requested reports\n3. **Database Integration**: Add database support for storing reports and user data\n4. **Load Balancing**: Use multiple Cloud Run instances for high traffic\n\n## Support\n\nFor support, questions, or feedback regarding the CreateReport Crew or crewAI:\n\n- Visit our [documentation](https://docs.crewai.com)\n- Reach out to us through our [GitHub repository](https://github.com/joaomdmoura/crewai)\n- [Join our Discord](https://discord.com/invite/X4JWnZnxPb)\n- [Chat with our docs](https://chatg.pt/DWjSBZn)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajitashwath%2Fresearcher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fajitashwath%2Fresearcher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fajitashwath%2Fresearcher/lists"}