{"id":23908180,"url":"https://github.com/themihirmathur/speechgrade","last_synced_at":"2026-04-06T09:02:36.714Z","repository":{"id":270385416,"uuid":"910203686","full_name":"themihirmathur/SpeechGrade","owner":"themihirmathur","description":"'SpeechGrade' is an innovative educational platform designed to streamline speech assessment and enhance teaching by integrating the MERN stack, machine learning models, large language models (LLMs), and Google's Generative AI.","archived":false,"fork":false,"pushed_at":"2024-12-30T18:49:53.000Z","size":4578,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-23T15:15:35.526Z","etag":null,"topics":["expressjs","flask","gemini","gemini-embeddings","google-generative-ai","langchain","llm","llma-index","machine-learning","material-ui","mern-stack","mongodb","nodejs","openai-whisper","reactjs"],"latest_commit_sha":null,"homepage":"https://github.com/themihirmathur/SpeechGrade","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/themihirmathur.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-30T17:44:46.000Z","updated_at":"2024-12-30T18:49:56.000Z","dependencies_parsed_at":"2024-12-30T19:34:19.558Z","dependency_job_id":null,"html_url":"https://github.com/themihirmathur/SpeechGrade","commit_stats":null,"previous_names":["themihirmathur/speechgrade"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/themihirmathur/SpeechGrade","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themihirmathur%2FSpeechGrade","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themihirmathur%2FSpeechGrade/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themihirmathur%2FSpeechGrade/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themihirmathur%2FSpeechGrade/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/themihirmathur","download_url":"https://codeload.github.com/themihirmathur/SpeechGrade/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/themihirmathur%2FSpeechGrade/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266120400,"owners_count":23879314,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["expressjs","flask","gemini","gemini-embeddings","google-generative-ai","langchain","llm","llma-index","machine-learning","material-ui","mern-stack","mongodb","nodejs","openai-whisper","reactjs"],"created_at":"2025-01-05T04:25:45.266Z","updated_at":"2025-12-30T18:56:31.123Z","avatar_url":"https://github.com/themihirmathur.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SpeechGrade 🎤💬 Educational Platform for Speech Assessment and Question Generation  \n\n\u003cp align=\"left\"\u003e\n  \u003cimg src=\"https://www.animatedimages.org/data/media/562/animated-line-image-0184.gif\" width=\"1920\" \n\u003c/p\u003e\n\n![Screenshot 2024-12-31 at 12 13 25 AM](https://github.com/user-attachments/assets/2dd01e0a-102f-4d19-b7c0-823189bfbf17)\n\n`SpeechGrade` is an innovative educational platform designed to streamline speech assessment and enhance teaching methodologies through cutting-edge technologies. By integrating the MERN stack, machine learning models, large language models (LLMs), and Google's Generative AI, SpeechGrade SpeechGrade empowers educators and students with tools for automated transcription, similarity analysis, and personalized learning.\n\nThis platform enhances teaching methodologies by streamlining speech evaluations and offering dynamic test creation capabilities, making it an indispensable tool for educational institutions and training programs.\n\n\u003cp align=\"left\"\u003e\n  \u003cimg src=\"https://www.animatedimages.org/data/media/562/animated-line-image-0184.gif\" width=\"1920\" \n\u003c/p\u003e\n\n## Features\n\n### **1. Speech Assessment**\n#### **Audio Transcription**\n- Users can upload audio files through a React-based client application.\n- The `Flask` server leverages `OpenAI’s Whisper` machine learning model for high-accuracy speech-to-text transcription.\n\n#### **Similarity Analysis**\n- Transcribed text is compared to a predefined target text (e.g., speeches or learning material) using `Google's Generative AI`.\n- The platform calculates a similarity score and provides detailed insights into alignment with the target text.\n\n#### **Results and Feedback**\n- Performance analysis highlights areas of strength and improvement.\n- Results are stored for further review, enabling continuous learning and progress tracking.\n\n---\n\n### **2. Question Generation and Metadata Management**\n#### **Metadata Management**\n- A `Node.js` server efficiently handles metadata related to teachers, students, and classes.\n- Enables organized storage and retrieval of educational resources.\n\n#### **LLM-Powered Question Generation**\n- Teachers can generate test questions tailored to specific topics and objectives, including:\n  - Multiple-choice questions\n  - Short-answer questions\n  - Essay prompts\n- Questions are designed using advanced large language models, ensuring alignment with curriculum goals.\n\n---\n\n### 3. **Personalized Learning**\nSpeechGrade incorporates data-driven insights to offer personalized feedback, enabling students to focus on specific areas for improvement. This personalization enhances the learning experience and boosts student outcomes.  \n\n\u003cp align=\"left\"\u003e\n  \u003cimg src=\"https://www.animatedimages.org/data/media/562/animated-line-image-0184.gif\" width=\"1920\" \n\u003c/p\u003e\n\n## **Technologies Used**  \n\n### **Frontend**  \n- **React.js**: Interactive and user-friendly interface for uploading audio and viewing results.  \n- **Material-UI**: Modern and responsive component library for a seamless user experience.  \n\n---\n\n### **Backend**  \n- **Flask**: Server-side processing for speech transcription and similarity analysis.  \n- **Node.js**: Server for metadata management and integration with LLMs.  \n- **Express.js**: Framework for efficient API management.  \n\n---\n\n### **Database**  \n- **MongoDB**: NoSQL database for storing user profiles, performance logs, and question sets.  \n\n---\n\n### **Machine Learning**  \n- **Whisper Model**: OpenAI’s state-of-the-art speech-to-text model for accurate transcription.  \n- **Google's Generative AI**: Advanced model for text similarity assessment and educational content analysis.  \n\n\u003cp align=\"left\"\u003e\n  \u003cimg src=\"https://www.animatedimages.org/data/media/562/animated-line-image-0184.gif\" width=\"1920\" \n\u003c/p\u003e\n\n## **Installation**  \n\n1. **Clone the Repository**  \n   ```bash\n   git clone https://github.com/themihirmathur/SpeechGrade.git\n   cd SpeechGrade\n   ```\n\n2. **Install Dependencies**  \n   Install required packages for both the Flask and Node.js servers.  \n   ```bash\n   # Flask server\n   cd server-flask\n   pip install -r requirements.txt\n\n   # Node.js server\n   cd ../server-node\n   npm install\n   ```\n\n3. **Run the Servers**  \n   Start both servers.  \n   ```bash\n   # Flask server\n   python app.py\n\n   # Node.js server\n   npm start\n   ```\n\n4. **Run the Client Application**  \n   Navigate to the client folder and start the `React` application.  \n   ```bash\n   cd ../client\n   npm install\n   npm start\n   ```\n\n5. **Access the Application**  \n   Open your browser and navigate to `http://localhost:3000`.  \n\n\u003cp align=\"left\"\u003e\n  \u003cimg src=\"https://www.animatedimages.org/data/media/562/animated-line-image-0184.gif\" width=\"1920\" \n\u003c/p\u003e\n\n## **Usage**  \n\n### 1. **Speech Assessment**\n- Navigate to the speech assessment page on the platform.  \n- Upload an audio file of your speech or presentation.  \n- View the transcription and similarity score along with detailed feedback.  \n\n---\n\n### 2. **Question Generation**\n- Teachers can log in to access the question generation tool.  \n- Input a topic or learning material, and the platform will generate custom test questions.  \n- Save the questions for use in assessments or classroom activities.  \n\n\u003cp align=\"left\"\u003e\n  \u003cimg src=\"https://www.animatedimages.org/data/media/562/animated-line-image-0184.gif\" width=\"1920\" \n\u003c/p\u003e\n\n## **Evaluation Metrics**  \nThe platform employs the following metrics to assess speech similarity and system performance:  \n- **Word Error Rate (WER):** Measures transcription accuracy.  \n- **Semantic Similarity Score:** Quantifies alignment between the transcribed and target text.  \n- **Performance Analytics:** Tracks user progress over time, providing actionable insights for improvement.  \n\n---\n\n## **Benefits**  \n- **Educator Efficiency:** Automates transcription, assessment, and test creation.  \n- **Improved Learning Outcomes:** Personalized feedback fosters targeted skill development.  \n- **Scalable and Flexible:** Supports diverse use cases, from language learning to public speaking training.  \n\n\u003cp align=\"left\"\u003e\n  \u003cimg src=\"https://www.animatedimages.org/data/media/562/animated-line-image-0184.gif\" width=\"1920\" \n\u003c/p\u003e\n\n## **Future Work**  \n- **Real-Time Feedback:** Implementing real-time transcription and scoring during live presentations.  \n- **Multi-language Support:** Expanding the platform to support multiple languages for global accessibility.  \n- **Time-Series Analysis:** Incorporating temporal patterns in speech to provide deeper insights into fluency and pacing.  \n- **Enhanced Analytics:** Advanced dashboards for educators to analyze class-wide performance trends.  \n\n![Screenshot 2024-12-31 at 12 10 18 AM](https://github.com/user-attachments/assets/cc7542cf-889a-4fc8-9c96-77d3fb9d92b5)\n\n\u003cp align=\"left\"\u003e\n  \u003cimg src=\"https://www.animatedimages.org/data/media/562/animated-line-image-0184.gif\" width=\"1920\" \n\u003c/p\u003e\n\n## Contribution\n\nContributions to SpeechGrade are welcome. Please follow these steps:\n1. Fork the repository.\n2. Create a new branch for your feature or bug fix.\n3. Commit your changes and open a pull request.\n\nFor major changes, open an issue to discuss the proposed modifications beforehand.\n\n---\n\n## **Contact**\nIf you have any questions or feedback, feel free to reach out:  \n**Name**: Mihir Mathur  \n**Email**: [themihirmathur@gmail.com](mailto:themihirmathur@gmail.com)  \n**Phone**: +91-7525099328  \n\n\u003cp align=\"left\"\u003e\n  \u003cimg src=\"https://www.animatedimages.org/data/media/562/animated-line-image-0184.gif\" width=\"1920\" \n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthemihirmathur%2Fspeechgrade","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthemihirmathur%2Fspeechgrade","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthemihirmathur%2Fspeechgrade/lists"}