{"id":28490232,"url":"https://github.com/adiyaan010205/price-pulse","last_synced_at":"2026-04-09T02:31:06.296Z","repository":{"id":297447404,"uuid":"996775365","full_name":"adiyaan010205/Price-Pulse","owner":"adiyaan010205","description":"PricePulse is a full-stack web application that automates product price tracking across popular e-commerce platforms like Amazon and eBay. It enables users to receive real-time alerts for price drops, monitor trends via analytics dashboards, and manage tracked items efficiently—all through a modern and responsive interface.","archived":false,"fork":false,"pushed_at":"2025-06-05T13:44:55.000Z","size":41186,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-05T14:33:41.863Z","etag":null,"topics":["antibot-detection","apscheduler","fastapi","jwt-authentication","postgresql","price-tracker","react","scrapy","sendgrid","smtp","sqlalchemy","tailwindcss","webscraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adiyaan010205.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-05T12:53:02.000Z","updated_at":"2025-06-05T13:44:57.000Z","dependencies_parsed_at":"2025-06-05T14:44:33.756Z","dependency_job_id":null,"html_url":"https://github.com/adiyaan010205/Price-Pulse","commit_stats":null,"previous_names":["adiyaan010205/price-pulse"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/adiyaan010205/Price-Pulse","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adiyaan010205%2FPrice-Pulse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adiyaan010205%2FPrice-Pulse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adiyaan010205%2FPrice-Pulse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adiyaan010205%2FPrice-Pulse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adiyaan010205","download_url":"https://codeload.github.com/adiyaan010205/Price-Pulse/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adiyaan010205%2FPrice-Pulse/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262671725,"owners_count":23346460,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["antibot-detection","apscheduler","fastapi","jwt-authentication","postgresql","price-tracker","react","scrapy","sendgrid","smtp","sqlalchemy","tailwindcss","webscraping"],"created_at":"2025-06-08T07:07:55.244Z","updated_at":"2025-12-30T22:44:05.908Z","avatar_url":"https://github.com/adiyaan010205.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# PricePulse - Smart Price Tracking Application 🏷️\n\nA comprehensive full-stack web application that monitors product prices across multiple e-commerce platforms, sends automated alerts on price drops, and provides detailed price analytics through an intuitive dashboard.\n\n![Screenshot 2025-06-03 080321](https://github.com/user-attachments/assets/b35c9614-c1f7-4d82-9625-db8af84c41a2)\n\n![Screenshot 2025-06-03 080352](https://github.com/user-attachments/assets/2f5d0acd-aee2-4786-b065-20f909d17481)\n\n\n## ✨ Features\n\n- 🕷️ **Real-time Web Scraping** - Automated price extraction from Amazon, eBay, and other platforms\n- 📧 **Smart Alerts** - Email notifications when prices drop below target thresholds\n- 📊 **Price Analytics** - Interactive charts showing price history and trends\n- 🔄 **Background Monitoring** - Scheduled price checks using APScheduler\n- 👤 **User Management** - Secure user authentication and product management\n- 📱 **Responsive Design** - Modern React frontend with Tailwind CSS\n- 🛡️ **Anti-Bot Protection** - Advanced scraping techniques to avoid detection\n\n## 🛠️ Tech Stack\n\n### Backend\n- **Framework**: FastAPI (Python)\n- **Web Scraping**: Scrapy with custom middlewares\n- **Database**: PostgreSQL with SQLAlchemy ORM\n- **Email Service**: SendGrid / SMTP\n- **Scheduling**: APScheduler for background tasks\n- **Authentication**: JWT tokens\n\n### Frontend\n- **Framework**: React.js 18+\n- **Styling**: Tailwind CSS 3.x\n- **HTTP Client**: Axios\n- **Notifications**: React Toastify\n- **State Management**: React Hooks\n\n### Infrastructure\n- **Server**: Uvicorn ASGI server\n- **Database**: PostgreSQL 15+\n- **Environment**: Python 3.10+, Node.js 18+\n\n## 📁 Project Structure\n\n```\nprice-tracker/\n├── backend/\n│   ├── app/\n│   │   ├── main.py                 # FastAPI application entry point\n│   │   ├── config.py               # Configuration settings\n│   │   ├── routes/\n│   │   │   └── tracker.py          # API endpoints for products/users\n│   │   ├── models/\n│   │   │   └── product.py          # SQLAlchemy database models\n│   │   ├── schemas/\n│   │   │   └── product.py          # Pydantic request/response schemas\n│   │   ├── services/\n│   │   │   ├── scraper_runner.py   # Scrapy integration service\n│   │   │   ├── email_alert.py      # Email notification service\n│   │   │   └── scheduler.py        # Background task scheduler\n│   │   └── db/\n│   │       └── database.py         # Database connection setup\n│   ├── scrapy_spiders/\n│   │   ├── __init__.py\n│   │   ├── product_spider.py       # Main scraping spider\n│   │   └── settings.py             # Scrapy configuration\n│   ├── requirements.txt            # Python dependencies\n│   └── .env                        # Environment variables\n├── frontend/\n│   ├── src/\n│   │   ├── components/\n│   │   │   ├── UploadForm.jsx      # Product URL input form\n│   │   │   └── ProductCard.jsx     # Product display component\n│   │   ├── App.jsx                 # Main React application\n│   │   ├── main.jsx                # React DOM entry point\n│   │   └── index.css               # Tailwind CSS imports\n│   ├── public/\n│   │   └── index.html\n│   ├── package.json                # Node.js dependencies\n│   ├── tailwind.config.js          # Tailwind configuration\n│   └── postcss.config.js           # PostCSS configuration\n├── README.md\n├── .gitignore\n└── docker-compose.yml              # Docker deployment configuration\n```\n\n## 📦 Dependencies\n\n### Backend Dependencies (`requirements.txt`)\n```\nfastapi==0.104.1\nuvicorn[standard]==0.24.0\nsqlalchemy==2.0.21\npsycopg2-binary==2.9.7\npydantic==2.4.2\nscrapy==2.12.0\nrequests==2.31.0\npython-dotenv==1.0.0\napscheduler==3.10.4\nsendgrid==6.10.0\npython-multipart==0.0.6\nalembic==1.12.0\nbcrypt==4.0.1\npython-jose[cryptography]==3.3.0\n```\n\n### Frontend Dependencies (`package.json`)\n```\n{\n  \"dependencies\": {\n    \"react\": \"^18.2.0\",\n    \"react-dom\": \"^18.2.0\",\n    \"axios\": \"^1.5.0\",\n    \"react-toastify\": \"^9.1.3\",\n    \"@headlessui/react\": \"^1.7.17\",\n    \"@heroicons/react\": \"^2.0.18\"\n  },\n  \"devDependencies\": {\n    \"@types/react\": \"^18.2.15\",\n    \"@types/react-dom\": \"^18.2.7\",\n    \"@vitejs/plugin-react\": \"^4.0.3\",\n    \"tailwindcss\": \"^3.3.0\",\n    \"postcss\": \"^8.4.24\",\n    \"autoprefixer\": \"^10.4.14\",\n    \"vite\": \"^4.4.5\"\n  }\n}\n```\n\n## 🚀 Installation \u0026 Setup\n\n### Prerequisites\n- Python 3.10 or higher\n- Node.js 18+ and npm\n- PostgreSQL 15+\n- Git\n\n### 1. Clone Repository\n```\ngit clone https://github.com/adiyaan010205/Price-Pulse.git\ncd Price-Pulse\n```\n\n### 2. Backend Setup\n```\ncd backend\n\n# Create virtual environment\npython -m venv venv\n\n# Activate virtual environment\n# Windows:\nvenv\\Scripts\\activate\n# macOS/Linux:\nsource venv/bin/activate\n\n# Install dependencies\npip install -r requirements.txt\n\n# Setup environment variables\ncp .env.example .env\n# Edit .env with your credentials\n```\n\n### 3. Database Configuration\n```\n# Create PostgreSQL database\ncreatedb price_tracker\n\n# Run migrations (if using Alembic)\nalembic upgrade head\n```\n\n### 4. Frontend Setup\n```\ncd ../frontend\n\n# Install dependencies\nnpm install\n\n# Install Tailwind CSS\nnpm install -D tailwindcss postcss autoprefixer\nnpx tailwindcss init -p\n```\n\n### 5. Environment Variables (`.env`)\n```\n# Database Configuration\nDB_HOST=localhost\nDB_NAME=price_tracker\nDB_USER=postgres\nDB_PASSWORD=your_password\nDB_PORT=5432\n\n# Email Configuration (SMTP/SendGrid)\nSMTP_SERVER=smtp.gmail.com\nSMTP_PORT=587\nSMTP_USER=your_email@gmail.com\nSMTP_PASSWORD=your_app_password\n\n# SendGrid (Alternative)\nSENDGRID_API_KEY=your_sendgrid_api_key\n\n# Application Settings\nSECRET_KEY=your_secret_key_here\nALGORITHM=HS256\nACCESS_TOKEN_EXPIRE_MINUTES=30\n```\n\n## 🏃‍♂️ Running the Application\n\n### Development Mode\n\n**Backend:**\n```\ncd backend\nuvicorn app.main:app --reload --host 0.0.0.0 --port 8000\n```\n\n**Frontend:**\n```\ncd frontend\nnpm run dev\n```\n\n### Production Mode\n```\n# Backend\nuvicorn app.main:app --host 0.0.0.0 --port 8000\n\n# Frontend\nnpm run build\nnpm run preview\n```\n\n## 🔌 API Endpoints\n\n### Products\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| `POST` | `/api/v1/products/` | Create new product |\n| `GET` | `/api/v1/products/` | List all products |\n| `GET` | `/api/v1/products/{id}` | Get product details |\n| `PUT` | `/api/v1/products/{id}` | Update product |\n| `DELETE` | `/api/v1/products/{id}` | Delete product |\n| `GET` | `/api/v1/products/{id}/price-history` | Get price history |\n| `POST` | `/api/v1/products/{id}/check-price` | Manual price check |\n\n### Users\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| `POST` | `/api/v1/users/` | Create user |\n| `GET` | `/api/v1/users/` | List users |\n\n### Example API Usage\n```\n# Add new product\ncurl -X POST \"http://localhost:8000/api/v1/products/\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"name\": \"Example Product\",\n    \"url\": \"https://www.amazon.in/dp/B09V4MXBSN\",\n    \"target_price\": 299.99,\n    \"platform\": \"amazon\"\n  }'\n\n# Get all products\ncurl -X GET \"http://localhost:8000/api/v1/products/\"\n\n# Check price manually\ncurl -X POST \"http://localhost:8000/api/v1/products/1/check-price\"\n```\n\n## 🕷️ Web Scraping Implementation\n\n### Spider Configuration\nThe application uses Scrapy with custom settings for reliable scraping:\n\n```\n# scrapy_spiders/settings.py\nUSER_AGENT = \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36\"\nDOWNLOAD_DELAY = 3\nRANDOMIZE_DOWNLOAD_DELAY = True\nROBOTSTXT_OBEY = False\nCOOKIES_ENABLED = True\nHTTPERROR_ALLOW_ALL = True\n```\n\n### Supported Platforms\n- **Amazon** (amazon.in, amazon.com)\n- **eBay** (ebay.com, ebay.in)\n- **Generic** e-commerce sites\n\n### Anti-Detection Features\n- Rotating User-Agents\n- Random download delays\n- Cookie handling\n- Proxy support (configurable)\n\n## 📧 Email Alert System\n\n### Configuration\n```\n# Email via SMTP\nSMTP_SERVER = \"smtp.gmail.com\"\nSMTP_PORT = 587\n\n# Email via SendGrid\nSENDGRID_API_KEY = \"your_api_key\"\n```\n\n### Alert Triggers\n- Price drops below target price\n- Price increases above threshold\n- Product availability changes\n- Daily/weekly price summaries\n\n## 🕒 Background Scheduling\n\nThe application uses APScheduler for automated price checking:\n\n```\n# Default schedule: Every 6 hours\nscheduler.add_job(\n    func=check_all_products,\n    trigger=\"interval\",\n    hours=6,\n    id='price_check_job'\n)\n```\n\n\n## 🔧 Troubleshooting\n\n### Common Issues\n\n**1. Scrapy Signal Handler Error**\n```\n# Solution: Use multiprocessing in scraper_runner.py\nprocess = multiprocessing.Process(target=_run_spider, args=(url, queue))\n```\n\n**2. PostgreSQL Connection Error**\n```\n# Check database credentials in .env\n# Ensure PostgreSQL service is running\nsudo service postgresql start\n```\n\n**3. Tailwind CSS Not Working**\n```\n# Reinstall Tailwind dependencies\nnpm uninstall tailwindcss\nnpm install -D tailwindcss@3.4.4 postcss autoprefixer\n```\n\n**4. Amazon Blocking Requests**\n```\n# Update user agent and add delays in scrapy settings\nDOWNLOAD_DELAY = 5\nRANDOMIZE_DOWNLOAD_DELAY = True\n```\n\n## 🤝 Contributing\n\n1. Fork the repository\n2. Create feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit changes (`git commit -m 'Add amazing feature'`)\n4. Push to branch (`git push origin feature/amazing-feature`)\n5. Open Pull Request\n\n\n**Built with ❤️ by Mohammed Adiyaan R**  \n*Powered by FastAPI ⚡ React ⚛️ and Scrapy 🕷️*\n\nFor support or questions, please open an issue on GitHub or contact [adiyaan126@gmail.com](mailto:adiyaan126@gmail.com)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadiyaan010205%2Fprice-pulse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadiyaan010205%2Fprice-pulse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadiyaan010205%2Fprice-pulse/lists"}