https://github.com/rushilpatel21/sentique
Sentique is a full‑stack feedback analytics platform that ingests user reviews from App Store, Google Play, Trustpilot, Reddit and Twitter/X, processes them with a fine‑tuned BERT model into 16 categories and sentiment labels, and provides comprehensive analytics and insights.
https://github.com/rushilpatel21/sentique
bert-model celery chatbot data-visualization deep-learning django feedback-analysis finetuned-model gemini-api machine-learning nlp postgresql rag rag-chatbot react review-analysis scraping sentiment-analysis sentique vector-search
Last synced: about 2 months ago
JSON representation
Sentique is a full‑stack feedback analytics platform that ingests user reviews from App Store, Google Play, Trustpilot, Reddit and Twitter/X, processes them with a fine‑tuned BERT model into 16 categories and sentiment labels, and provides comprehensive analytics and insights.
- Host: GitHub
- URL: https://github.com/rushilpatel21/sentique
- Owner: rushilpatel21
- License: apache-2.0
- Created: 2025-02-22T03:36:26.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-11-02T09:50:49.000Z (5 months ago)
- Last Synced: 2025-11-02T11:31:11.935Z (5 months ago)
- Topics: bert-model, celery, chatbot, data-visualization, deep-learning, django, feedback-analysis, finetuned-model, gemini-api, machine-learning, nlp, postgresql, rag, rag-chatbot, react, review-analysis, scraping, sentiment-analysis, sentique, vector-search
- Language: TypeScript
- Homepage: https://sentique.vercel.app
- Size: 83.7 MB
- Stars: 2
- Watchers: 1
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Sentique
**Sentique** is a full‑stack feedback analytics platform that ingests user reviews from App Store, Google Play, Trustpilot, Reddit and Twitter/X, processes them with a fine‑tuned BERT model into 16 categories and sentiment labels, and provides comprehensive analytics and insights.
## Overview
Sentique transforms scattered customer feedback into actionable intelligence through sophisticated data processing and AI-powered analysis. The platform combines traditional machine learning with generative AI to deliver both structured analytics and conversational insights, helping businesses understand customer sentiment at scale.
## Screenshots
### Onboarding Form
*The onboarding form collects essential application details via the `/register` endpoint to set up Sentique:*
- Apple App Store Product ID (numeric ID, e.g. `123456789`)
- Apple App Store Product Name (e.g. `MyAwesomeApp`)
- Google Play Store Package Name (e.g. `com.example.app`)
- Company Website URL (e.g. `example.com`)
*Once you submit this information, Sentique will automatically fetch and process reviews from your configured app stores and web channels for sentiment analysis and insights.*
### Dashboard Overview
*The main dashboard provides a comprehensive view of sentiment trends across time periods, sources, and categories. Key metrics are highlighted with interactive filtering capabilities.*
### Sentiment Analysis
*Detailed sentiment breakdown showing positive, negative, and neutral distribution across different sources and time periods, enabling targeted analysis of customer feedback patterns.*
### Actionable Insights
*Summarization of review distribution across the 16 categories identified by our fine-tuned BERT model, helping identify the most discussed aspects of products or services.*
*Actionable Insights of review distribution across the 16 categories identified by our fine-tuned BERT model, helping identify the most discussed aspects of products or services.*
### RAG-Based Chatbot Interface
*Natural language interface allowing users to query the review database conversationally, with responses generated by Gemini API using relevant review context.*
### Insights Report
*Summary report highlighting key strengths and improvement areas based on analysis of customer reviews across selected time periods and categories.*
## Key Features
- **Multi-Source Data Collection**: Automated scraping of reviews from App Store, Google Play, Trustpilot, Reddit, and Twitter/X
- **ML-Powered Classification**: Fine-tuned BERT model categorizes reviews into 16 distinct classes
- **Sentiment Analysis**: Identifies positive, negative, and neutral opinions across all feedback sources
- **Interactive Dashboards**: Real-time visualization of sentiment trends, source breakdowns, and feature-specific feedback
- **AI-Powered Insights**: Actionable summaries of what users like and areas for improvement via Gemini API
- **RAG-Driven Chatbot**: Natural language Q&A over vectorized review data using SentenceTransformer + Gemini
## Tech Stack
- **Frontend**: React, Vite, TypeScript, Tailwind CSS
- **Backend**: Django, Django REST Framework, FastAPI
- **Database**: PostgreSQL, PGVector, Redis (Celery broker)
- **Authentication**: Django all-auth
- **LLM**: Gemini 1.5 Pro
- **Deep Learning Models**: Fine-tuned ROBERTA based classifier model, j-hartmann/sentiment-roberta-large-english-3-classes
- **Network Tunneling**: ngrok
- **Process Management**: Celery
- **Package Management**: pnpm (for frontend), pip (for backend)
## System Architecture
### Backend Architecture
#### Data Collection Framework
- **Multi-Source Scrapers**: Specialized modules for each data source (App Store, Play Store, Reddit, Trustpilot, Twitter)
- **Asynchronous Processing**: Django + Celery architecture for efficient task queuing and execution
- **Error Handling**: Robust retry mechanisms for failed scraping attempts
- **Data Storage**: PostgreSQL database optimized for review data and analysis results
#### Machine Learning Pipeline
- **Sentiment Analysis**: Classifies reviews as positive, negative, or neutral
- **Category Classification**: Fine-tuned BERT model sorts reviews into 16 specific categories
- **Vector Embeddings**: Transforms review text into vector representations for similarity search
- **Model Persistence**: Efficient storage and retrieval of trained models
#### AI-Powered Insights
- **Gemini API Integration**: Generates comprehensive insights about specific feedback categories
- **Structured Analysis**: Creates organized summaries of positive and negative aspects
- **Time-Series Analysis**: Tracks sentiment changes and feature performance over time
- **Fallback Handling**: Alternative text extraction when primary parsing methods fail
#### RAG-Based Chatbot
- **Vector Search**: SentenceTransformer converts queries into searchable vectors
- **Contextual Retrieval**: Finds and ranks relevant reviews based on semantic similarity
- **Context Augmentation**: Enhances AI responses with retrieved review data
- **Response Generation**: Gemini API produces human-like answers grounded in actual review data
### Frontend Architecture
#### Component Structure
- **React + TypeScript**: Modern frontend with strong typing
- **Dashboard Views**: Main analysis dashboards with sentiment trends
- **Detailed Feedback Explorer**: Deep-dive interface for individual reviews
- **Product Feedback Analysis**: Category-specific review insights
- **Interactive Chat Interface**: Conversational UI for the RAG system
#### Visualization Components
- **Trend Charts**: Time-based visualization of sentiment patterns
- **Source Distribution**: Breakdown of sentiment by data source
- **Feature Analysis**: Comparative visualization of product features
- **Topic Modeling**: Visual representation of common themes in reviews
## Data Flow
1. **Collection**: Scrapers gather reviews from multiple platforms
2. **Processing**: ML models analyze sentiment and categorize reviews
3. **Enrichment**: Reviews are vectorized and stored with metadata
4. **Analysis**: Gemini API generates insights from processed data
5. **Presentation**: Frontend displays visualizations and interactive reports
6. **Interaction**: Users explore data and query the RAG chatbot
## Project Setup Guide
### Notes
- Requires **pnpm**, **Node.js**, **Python 3.9+**
- On Windows, use `venv\Scripts\activate` instead of `source venv/bin/activate`
- Environment variables must be configured in `.env` file
### Frontend
1. Navigate to the frontend directory:
```sh
cd frontend
```
2. Install dependencies:
```sh
pnpm install
```
3. Start the development server:
```sh
pnpm dev
```
### Database Setup
#### Install PostgreSQL
If you haven't installed PostgreSQL, install it using:
**Ubuntu:**
```sh
sudo apt update
sudo apt install postgresql postgresql-contrib
```
**Mac (Homebrew):**
```sh
brew install postgresql
```
**Windows:**
Download from [official PostgreSQL website](https://www.postgresql.org/download/).
#### Enable and Start PostgreSQL
```sh
sudo systemctl enable postgresql
sudo systemctl start postgresql
```
#### Create a Database and User
```sh
sudo -u postgres psql
```
Then, inside the PostgreSQL shell:
```sql
CREATE DATABASE sentique;
CREATE USER sentique_user WITH PASSWORD 'my_password';
ALTER ROLE sentique_user SET client_encoding TO 'utf8';
ALTER ROLE sentique_user SET default_transaction_isolation TO 'read committed';
ALTER ROLE sentique_user SET timezone TO 'UTC';
GRANT ALL PRIVILEGES ON DATABASE sentique TO sentique_user;
\q
```
#### Install `pgvector` Extension
Connect to your database:
```sh
psql -U sentique_user -d sentique
```
Then, enable the `pgvector` extension:
```sql
CREATE EXTENSION vector;
```
### Redis Setup
#### Install Redis
**Ubuntu:**
```sh
sudo apt update
sudo apt install redis-server
```
**Mac (Homebrew):**
```sh
brew install redis
```
**Windows:**
Use [Memurai](https://www.memurai.com/) (a Redis alternative for Windows) or install Redis via WSL.
#### Start and Enable Redis
```sh
sudo systemctl enable redis-server
sudo systemctl start redis-server
```
#### Verify Redis is Running
```sh
redis-cli ping
```
You should see `PONG` as output.
### Backend
1. Create and activate a virtual environment:
```sh
python -m venv venv
source venv/bin/activate # For macOS/Linux
# For Windows:
# venv\Scripts\activate
```
2. Install required dependencies:
```sh
pip install -r requirements.txt
```
3. Apply database migrations:
```sh
python manage.py makemigrations
python manage.py migrate
```
4. Run the celery worker:
```sh
celery -A backend worker --loglevel=info
```
5. Run the development server:
```sh
python manage.py runserver
```