{"id":24072479,"url":"https://github.com/zoraizmohammad/academicinsights","last_synced_at":"2026-06-17T07:31:40.811Z","repository":{"id":269384405,"uuid":"903990736","full_name":"zoraizmohammad/academicInsights","owner":"zoraizmohammad","description":"AcademicInsights: A Template Web Extension for using NLP and Behavioral Data for EdTech Research","archived":false,"fork":false,"pushed_at":"2024-12-31T18:18:56.000Z","size":35,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-26T22:14:03.352Z","etag":null,"topics":["edtech","nlp","research-tool","template-project","web-extension","web-scraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zoraizmohammad.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-16T03:24:33.000Z","updated_at":"2024-12-31T18:18:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"ad146257-54cb-4558-b6f5-1f6c942fa2d3","html_url":"https://github.com/zoraizmohammad/academicInsights","commit_stats":null,"previous_names":["zoraizmohammad/academicinsights"],"tags_count":0,"template":true,"template_full_name":null,"purl":"pkg:github/zoraizmohammad/academicInsights","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zoraizmohammad%2FacademicInsights","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zoraizmohammad%2FacademicInsights/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zoraizmohammad%2FacademicInsights/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zoraizmohammad%2FacademicInsights/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zoraizmohammad","download_url":"https://codeload.github.com/zoraizmohammad/academicInsights/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zoraizmohammad%2FacademicInsights/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34439294,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-17T02:00:05.408Z","response_time":127,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["edtech","nlp","research-tool","template-project","web-extension","web-scraping"],"created_at":"2025-01-09T17:17:47.759Z","updated_at":"2026-06-17T07:31:40.796Z","avatar_url":"https://github.com/zoraizmohammad.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# **AcademicInsights**\n## A Template Web Extension using NLP and Behavioral Data for EdTech Research\n\n## **Overview**\n\n**AcademicInsights** is a Chrome web extension paired with a backend that enables researchers to analyze students' online academic behavior. This tool uses **web scraping**, **machine learning (ML)**, and **natural language processing (NLP)** to provide insights into learning styles, resource usage, and topic relevance. \n\nThis project serves as a customizable template for **edtech researchers**, facilitating studies on how students utilize online resources for learning and how their behavior correlates with academic success.\n\n---\n\n## **Features**\n- **Web Scraping**: Extracts and analyzes visited website content.\n- **Learning Style Identification**: Classifies behavior into Visual, Auditory, Reading/Writing, or Kinesthetic styles.\n- **Topic Categorization**: Matches content to admin-defined academic topics.\n- **Google Sign-In**: Authenticates users securely.\n- **Backend Integration**: Provides a robust backend API for data processing and storage.\n- **Customizable**: Designed for easy adaptation to specific research needs.\n\n---\n\n## **Project Structure**\n\n### **Backend**\n```plaintext\nbackend/\n├── app/\n│   ├── scraping/               # Web scraping and analysis\n│   │   ├── scraper.py          # Scrapes website data\n│   │   ├── nlp_analysis.py     # NLP for topic categorization\n│   │   ├── ml_models.py        # ML models for learning styles\n│   ├── __init__.py             # Initializes Flask app\n│   ├── routes.py               # API routes\n│   ├── models.py               # Data models for backend (optional)\n├── app.py                      # Entry point for the backend\n├── config.py                   # Configuration for Firebase and other settings\n├── firebase-admin-key.json     # Firebase Admin SDK credentials\n├── Dockerfile                  # Docker container configuration\n├── requirements.txt            # Backend dependencies\n```\n\n### **Frontend**\n```plaintext\nfrontend/\n├── icons/                      # Extension icons\n│   ├── icon16.jpg\n│   ├── icon48.jpg\n│   ├── icon128.jpg\n├── background.js               # Background script for the extension\n├── firebase-config.js          # Firebase configuration for frontend\n├── manifest.json               # Chrome extension configuration\n├── popup.html                  # HTML for the extension's popup UI\n├── popup.js                    # Frontend logic for the popup\n├── styles.css                  # CSS for the popup\n```\n\n---\n\n## **Getting Started**\n\n### **1. Prerequisites**\n- Python 3.9+\n- Node.js (optional for additional tools)\n- Chrome Browser\n- Docker (for backend deployment)\n- Firebase Project for Authentication\n\n---\n\n### **2. Backend Setup**\n\n#### **Local Setup**\n1. Navigate to the `backend/` directory:\n   ```bash\n   cd backend\n   ```\n2. Create a virtual environment and activate it:\n   ```bash\n   python3 -m venv venv\n   source venv/bin/activate\n   ```\n3. Install dependencies:\n   ```bash\n   pip install -r requirements.txt\n   ```\n4. Run the backend:\n   ```bash\n   python app.py\n   ```\n   The backend will be available at `http://localhost:5000`.\n\n#### **Docker Setup**\n1. Build the Docker image:\n   ```bash\n   docker build -t academic-insights-backend .\n   ```\n2. Run the Docker container:\n   ```bash\n   docker run -p 5000:5000 academic-insights-backend\n   ```\n\n#### **Files of Interest**\n- **API Endpoints**: `app/routes.py`\n- **Web Scraping Logic**: `app/scraping/scraper.py`\n- **NLP Analysis**: `app/scraping/nlp_analysis.py`\n- **Learning Style ML Models**: `app/scraping/ml_models.py`\n\n---\n\n### **3. Frontend Setup**\n\n#### **Local Setup**\n1. Open Chrome and navigate to `chrome://extensions/`.\n2. Enable **Developer Mode**.\n3. Click **Load unpacked** and select the `frontend/` folder.\n\n#### **Files of Interest**\n- **Extension Configuration**: `frontend/manifest.json`\n- **Popup Logic**: `frontend/popup.js`\n- **Firebase Configuration**: `frontend/firebase-config.js`\n\n---\n\n### **4. Firebase Setup**\n1. Create a Firebase project at [Firebase Console](https://console.firebase.google.com/).\n2. Enable **Google Authentication** under `Authentication \u003e Sign-in method`.\n3. Download the **Admin SDK JSON key** and place it in `backend/firebase-admin-key.json`.\n4. Add Firebase configuration to `frontend/firebase-config.js`:\n   ```javascript\n   export const firebaseConfig = {\n     apiKey: \"YOUR_API_KEY\",\n     authDomain: \"YOUR_AUTH_DOMAIN\",\n     projectId: \"YOUR_PROJECT_ID\",\n     storageBucket: \"YOUR_STORAGE_BUCKET\",\n     messagingSenderId: \"YOUR_MESSAGING_SENDER_ID\",\n     appId: \"YOUR_APP_ID\",\n   };\n   ```\n\n---\n\n### **5. Deployment**\n\n#### **Backend Deployment**\nFor more detailed deployment instructions with more details about how to setup using various Backend hosting services, please for [here](https://github.com/zoraizmohammad/academicInsights/blob/0fdf09d4e04d48ee14b7eb02042c2c7459edb417/backendnstructions.md)\n- Deploy the backend using **Heroku**, **AWS**, or **Google Cloud Run**.\n- Update `frontend/popup.js` with the deployed backend URL:\n   ```javascript\n   const BASE_URL = \"https://your-backend-url.com\";\n   ```\n\n#### **Frontend Deployment**\n1. Package the extension:\n   - Zip the `frontend/` folder.\n2. Submit to the [Chrome Web Store Developer Dashboard](https://chrome.google.com/webstore/devconsole/).\n\n---\n\n## **Using the Tool**\n\n1. **Google Sign-In**:\n   - Authenticate using a Google account.\n2. **Track Web Activity**:\n   - Visit websites and let the extension scrape content.\n3. **Analyze Insights**:\n   - View learning styles and topic categorization in the extension popup.\n\n---\n\n## **Customization Guide**\n\n1. **Add New Topics**:\n   - Edit `app/scraping/nlp_analysis.py` to include new topics and keywords.\n\n2. **Modify Learning Style Models**:\n   - Update or retrain models in `app/scraping/ml_models.py` for different user behavior datasets.\n\n3. **Extension UI Customization**:\n   - Edit `frontend/popup.html` and `frontend/styles.css`.\n\n---\n\n## **For Researcher Use**\n\n- **Data Access**:\n  - Use the `/dashboard/\u003cuser_id\u003e` endpoint in `backend/app/routes.py` to retrieve user data.\n- **Documentation**:\n  - Refer to `backendinstructions.md` for backend setup and `setupLocal.md` or `setupHosted.md` for deployment.\n\n---\n\n## **Future Enhancements**\n- Add more advanced NLP models (e.g., GPT-based summarization).\n- Enable real-time monitoring with WebSockets.\n- Add more robust privacy features for anonymizing user data.\n\n---\n\n## **Support**\n\nFor issues, please contact me or open an issue on the project repository! Thanks!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzoraizmohammad%2Facademicinsights","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzoraizmohammad%2Facademicinsights","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzoraizmohammad%2Facademicinsights/lists"}