{"id":31442251,"url":"https://github.com/strcoder4007/voice-sentiment-analysis","last_synced_at":"2026-07-14T23:32:00.967Z","repository":{"id":311257545,"uuid":"1041850742","full_name":"strcoder4007/voice-sentiment-analysis","owner":"strcoder4007","description":"Customer call analysis app with:  Speech-to-Text + speaker diarization via ElevenLabs Structured conversation analysis via OpenAI React frontend for multi-file upload and rich results display. Takes into account not only the text but tone, rythym etc.","archived":false,"fork":false,"pushed_at":"2025-10-17T04:57:21.000Z","size":10997,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-26T03:29:00.991Z","etag":null,"topics":["call-analysis","speaker-diarization","speech-to-text"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/strcoder4007.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-21T05:29:30.000Z","updated_at":"2025-10-17T04:57:25.000Z","dependencies_parsed_at":"2025-10-16T16:07:26.490Z","dependency_job_id":null,"html_url":"https://github.com/strcoder4007/voice-sentiment-analysis","commit_stats":null,"previous_names":["strcoder4007/voice-sentiment-analysis"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/strcoder4007/voice-sentiment-analysis","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/strcoder4007%2Fvoice-sentiment-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/strcoder4007%2Fvoice-sentiment-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/strcoder4007%2Fvoice-sentiment-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/strcoder4007%2Fvoice-sentiment-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/strcoder4007","download_url":"https://codeload.github.com/strcoder4007/voice-sentiment-analysis/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/strcoder4007%2Fvoice-sentiment-analysis/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35483857,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-14T02:00:06.603Z","response_time":114,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["call-analysis","speaker-diarization","speech-to-text"],"created_at":"2025-09-30T18:48:21.099Z","updated_at":"2026-07-14T23:32:00.955Z","avatar_url":"https://github.com/strcoder4007.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Voice Sentiment Analysis\n\nCustomer call analysis app with:\n- Speech-to-Text + speaker diarization via ElevenLabs\n- Structured conversation analysis via OpenAI\n- React frontend for multi-file upload and rich results display\n- FastAPI backend orchestrating transcription, diarization, and analysis\n\nThis repository contains both the backend API (FastAPI) and the frontend web app (React).\n\n- GitHub: https://github.com/strcoder4007/voice-sentiment-analysis\n\n---\n\n## Features\n\n- Upload one or more audio files (wav, mp3, m4a, flac)\n- Automatic transcription with timestamps and speaker diarization\n- Grouped human-readable transcript by speaker turns with HH:MM:SS.mmm ranges\n- Structured JSON analysis including:\n  - emotion_overall + confidence\n  - satisfaction + confidence\n  - summary, customer_intent, issues\n  - action_items with owner and due date\n  - agent_speaker_label + identification confidence\n  - agent_improvement_opportunities (category, evidence, impact, recommendation)\n  - post_call_recommendations\n  - follow_up_message_draft\n  - sentiment_analysis narrative\n- Health check endpoint and robust error handling\n- CORS enabled for local development\n- Simple UI with results cards and JSON viewer\n\n---\n\n## Architecture and Flow\n\n- Frontend (React)\n  - Multi-file audio selection and upload\n  - Displays result cards and raw JSON\n  - Default backend URL: http://localhost:8000/analyze/\n\n- Backend (FastAPI)\n  - POST /analyze/: accepts multipart form-data (field key: files, repeated)\n  - Transcribes using ElevenLabs STT with diarization and word-level timestamps\n  - Groups words into speaker turns; renders readable transcript with timecodes\n  - Sends transcript summary to OpenAI for structured conversation analysis\n  - Returns combined metadata, transcript, and analysis JSON per file\n\n- External Services\n  - ElevenLabs Speech-to-Text API\n  - OpenAI Responses API\n\n---\n\n## Repository Structure\n\n```\nvoice-sentiment-analysis/\n├─ backend/\n│  ├─ main.py                # FastAPI app and orchestration logic\n│  └─ requirements.txt       # Python dependencies\n├─ frontend/\n│  ├─ package.json           # React app and scripts\n│  ├─ public/                # CRA public assets\n│  └─ src/\n│     ├─ App.js              # Upload UI and results grid\n│     ├─ App.css             # Styles\n│     └─ components/\n│        ├─ ResultCard.jsx   # Result card UI\n│        └─ JsonViewer.jsx   # JSON viewer component\n├─ TODO.md                   # Project roadmap\n├─ voxtral.py                # (not used by app runtime)\n└─ README.md                 # This file\n```\n\n---\n\n## Prerequisites\n\n- macOS, Linux, or Windows\n- Python 3.10+ recommended\n- Node.js 18+ and npm\n- API keys:\n  - OpenAI API key (with access to the selected model)\n  - ElevenLabs API key\n\nCosts: Using OpenAI and ElevenLabs APIs incurs usage charges. Ensure your accounts are configured appropriately.\n\n---\n\n## Quick Start\n\n1) Configure environment variables (create a .env file in the project root):\n```\nOPENAI_API_KEY=sk-...\nELEVENLABS_API_KEY=eleven-...\n```\n\n2) Start the backend (FastAPI):\n```\npython -m venv .venv\nsource .venv/bin/activate    # Windows: .venv\\Scripts\\activate\npip install -r backend/requirements.txt\n\n# Option A: run via uvicorn (recommended during dev)\nuvicorn backend.main:app --reload --host 0.0.0.0 --port 8000\n\n# Option B: run the module directly\npython backend/main.py\n```\n\n3) Start the frontend (React):\n```\ncd frontend\nnpm install\nnpm start\n```\n\n- Frontend dev server: http://localhost:3000\n- Backend API: http://localhost:8000\n- The default frontend expects the backend at http://localhost:8000/analyze/\n\n---\n\n## Configuration\n\n- Environment variables (in project root .env; loaded by python-dotenv):\n  - `OPENAI_API_KEY` — required\n  - `ELEVENLABS_API_KEY` — required\n\n- Backend ports\n  - Default is 8000; change via uvicorn flag if desired\n  - CORS is enabled for all origins by default (FastAPI middleware)\n\n- OpenAI model\n  - Backend code uses `model=\"gpt-5\"` in `backend/main.py`\n  - Ensure your account has access to this model; otherwise update the model string to an available one (e.g., `gpt-4o` or `gpt-4o-mini`) in `backend/main.py`\n\n- Frontend API URL\n  - Currently hardcoded in `frontend/src/App.js`:\n    ```\n    fetch(\"http://localhost:8000/analyze/\", { ... })\n    ```\n  - If deploying or changing ports, update this URL accordingly\n\n---\n\n## API\n\nBase URL: `http://localhost:8000`\n\n- GET `/`\n  - Returns: `{ \"message\": \"Voice Sentiment Analysis API is running\" }`\n\n- GET `/health`\n  - Returns:\n    ```\n    {\n      \"status\": \"healthy\",\n      \"openai_configured\": true|false,\n      \"elevenlabs_configured\": true|false\n    }\n    ```\n\n- POST `/analyze/`\n  - Content-Type: multipart/form-data\n  - Field name for files: `files` (repeat for multiple)\n  - Returns:\n    ```\n    {\n      \"results\": [\n        {\n          \"filename\": \"call1.mp3\",\n          \"date\": \"YYYY-MM-DD\",\n          \"time\": \"HH:MM:SS\",\n          \"audio_length\": \"HH:MM:SS.mmm\",\n          \"file_size\": 12345,\n          \"transcription\": \"...\",\n          \"analysis\": {\n            \"emotion_overall\": \"very_negative | negative | neutral | positive | very_positive\",\n            \"emotion_confidence\": 0.0,\n            \"satisfaction\": \"very_unsatisfied | unsatisfied | neutral | satisfied | very_satisfied\",\n            \"satisfaction_confidence\": 0.0,\n            \"summary\": \"2-4 sentences...\",\n            \"customer_intent\": \"one sentence...\",\n            \"issues\": [\"...\"],\n            \"action_items\": [\n              { \"owner\": \"agent|customer|other\", \"item\": \"...\", \"due\": \"YYYY-MM-DD|null\" }\n            ],\n            \"agent_speaker_label\": \"Speaker 1 | Speaker 2 | Speaker 3 | unknown\",\n            \"agent_identification_confidence\": 0.0,\n            \"agent_improvement_opportunities\": [\n              {\n                \"category\": \"empathy|discovery|clarity|solution_quality|ownership|pace|listening|policy_adherence|product_knowledge\",\n                \"observation\": \"...\",\n                \"evidence\": \"\\\"short quote\\\"\",\n                \"recommended_change\": \"...\",\n                \"impact\": \"low|medium|high\"\n              }\n            ],\n            \"post_call_recommendations\": [\"...\"],\n            \"follow_up_message_draft\": \"short paragraph...\",\n            \"sentiment_analysis\": \"2-4 sentences of critical-thinking analysis...\"\n          }\n        }\n      ],\n      \"total_processed\": 1\n    }\n    ```\n\nExample curl (single file):\n```\ncurl -X POST http://localhost:8000/analyze/ \\\n  -F \"files=@/path/to/audio.mp3;type=audio/mpeg\"\n```\n\nExample curl (multiple files):\n```\ncurl -X POST http://localhost:8000/analyze/ \\\n  -F \"files=@/path/to/call1.wav;type=audio/wav\" \\\n  -F \"files=@/path/to/call2.m4a;type=audio/m4a\"\n```\n\n---\n\n## Frontend\n\n- Tech: React (CRA), Result cards and JSON viewer\n- File input accepts multiple audio files\n- Action button posts to `/analyze/`\n- Error states shown inline\n\nTo change the API URL:\n- Edit `frontend/src/App.js` and update the fetch URL to your backend endpoint\n\nRun:\n```\ncd frontend\nnpm install\nnpm start\n```\n\nBuild for production:\n```\nnpm run build\n```\n\n---\n\n## Backend\n\n- Tech: FastAPI, httpx, python-dotenv\n- Entrypoints:\n  - `backend/main.py` (direct run)\n  - `uvicorn backend.main:app --reload --port 8000`\n- Key pipeline (per file):\n  1) Validate and read bytes\n  2) ElevenLabs STT with diarization and `timestamps_granularity=\"word\"`\n  3) Group words into speaker turns, render readable transcript with time ranges\n  4) Send enriched prompt to OpenAI Responses API for structured analysis JSON\n  5) Safe-parse JSON; add metadata and return\n\nPython dependencies: see `backend/requirements.txt`\n\n---\n\n## ElevenLabs and OpenAI Notes\n\n- ElevenLabs Speech-to-Text API:\n  - Endpoint: `POST https://api.elevenlabs.io/v1/speech-to-text`\n  - Requires `xi-api-key` header\n  - This app requests diarization and word-level timestamps\n  - Supported formats include common audio types (mp3, wav, m4a, flac)\n\n- OpenAI Responses API:\n  - Model string configurable in `backend/main.py` (`gpt-5` by default)\n  - If your account lacks access to the default model, change it to one you can use (e.g., `gpt-4o`)\n\n---\n\n## Troubleshooting\n\n- 500: OpenAI API key not configured\n  - Ensure `.env` contains `OPENAI_API_KEY` and the backend process can read it\n- 500: ElevenLabs API key not configured\n  - Ensure `.env` contains `ELEVENLABS_API_KEY`\n- 502 from ElevenLabs STT\n  - Check file format, account plan/limits, and API key validity\n- Empty or invalid OpenAI response\n  - Ensure model access; if needed, switch to a supported model in `backend/main.py`\n- CORS or network errors in the browser\n  - Confirm backend running at http://localhost:8000\n  - Verify fetch URL in `frontend/src/App.js`\n- Large files/slow responses\n  - Backend uses httpx timeout of 120s; adjust if needed\n\n---\n\n## Security and Privacy\n\n- API keys are loaded from environment variables; do not commit them to source control\n- Uploaded audio is processed in-memory for analysis then returned in results\n- Be mindful of sensitive content in audio/transcripts and downstream storage\n\n---\n\n## Roadmap\n\nSee [TODO.md](./TODO.md). Planned items include:\n- Improved multiple upload UX\n- Robust error states and retries\n- Accuracy validation and diarization quality checks\n- Deployment docs and environment management\n- CI, unit/integration tests with sample audio\n\n---\n\n## Development Tips\n\n- Run backend with `--reload` for hot reload during API edits:\n  ```\n  uvicorn backend.main:app --reload --port 8000\n  ```\n- Adjust the analysis schema/prompt in `backend/main.py` under `schema_template`, `system_msg`, and `user_prompt`\n- To change diarization behavior or language hints, modify `transcribe_with_elevenlabs` parameters in `backend/main.py`\n\n---\n\n## License\n\nNo license specified.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstrcoder4007%2Fvoice-sentiment-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstrcoder4007%2Fvoice-sentiment-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstrcoder4007%2Fvoice-sentiment-analysis/lists"}