{"id":26753150,"url":"https://github.com/igopalakrishna/high-perf-chatbot-torchscript","last_synced_at":"2026-04-29T18:34:24.310Z","repository":{"id":283844371,"uuid":"953072623","full_name":"igopalakrishna/high-perf-chatbot-torchscript","owner":"igopalakrishna","description":"High-performance conversational AI chatbot built with PyTorch, TorchScript, and Luong attention. Optimized for fast inference, scripted for deployment, and trained on movie dialogs with hyperparameter tuning and profiling.","archived":false,"fork":false,"pushed_at":"2025-03-22T15:26:46.000Z","size":558,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-22T15:27:51.487Z","etag":null,"topics":["ai","attention","chatbot","deep-learning","deployment","hyperparameter-tuning","nlp","pytorch","seq2seq","torchscript"],"latest_commit_sha":null,"homepage":"https://wandb.ai/ga2664-new-york-university/chatbot/sweeps/iwgnqx8h?nw=nwuserga2664","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/igopalakrishna.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-22T14:05:41.000Z","updated_at":"2025-03-22T15:26:50.000Z","dependencies_parsed_at":"2025-03-22T15:38:48.307Z","dependency_job_id":null,"html_url":"https://github.com/igopalakrishna/high-perf-chatbot-torchscript","commit_stats":null,"previous_names":["igopalakrishna/high-perf-chatbot-torchscript"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/igopalakrishna/high-perf-chatbot-torchscript","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/igopalakrishna%2Fhigh-perf-chatbot-torchscript","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/igopalakrishna%2Fhigh-perf-chatbot-torchscript/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/igopalakrishna%2Fhigh-perf-chatbot-torchscript/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/igopalakrishna%2Fhigh-perf-chatbot-torchscript/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/igopalakrishna","download_url":"https://codeload.github.com/igopalakrishna/high-perf-chatbot-torchscript/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/igopalakrishna%2Fhigh-perf-chatbot-torchscript/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32439277,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-29T18:12:22.909Z","status":"ssl_error","status_checked_at":"2026-04-29T18:11:33.322Z","response_time":110,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","attention","chatbot","deep-learning","deployment","hyperparameter-tuning","nlp","pytorch","seq2seq","torchscript"],"created_at":"2025-03-28T13:17:59.244Z","updated_at":"2026-04-29T18:34:24.283Z","avatar_url":"https://github.com/igopalakrishna.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TorchScript-Optimized Conversational AI Chatbot\n\n\u003e A high-performance, GPU-accelerated conversational AI chatbot trained on the Cornell Movie Dialog Corpus using a Sequence-to-Sequence architecture with Luong attention. Optimized via Weights \u0026 Biases hyperparameter sweeps and exported with TorchScript for deployment in non-Python environments.\n\n---\n\n## Project Summary\n\nThis project implements and optimizes a conversational chatbot trained on movie dialogues using PyTorch. It uses a Seq2Seq architecture with GRU layers and Luong-style attention, supports real-time greedy decoding, and is exportable via TorchScript for deployment.\n\nThe chatbot was optimized using Weights \u0026 Biases (W\u0026B) hyperparameter sweeps and benchmarked using PyTorch Profiler to improve memory and compute efficiency. TorchScript conversion enables portable inference outside Python (e.g., mobile or C++ environments).\n\n---\n\n## Features\n\n- Sequence-to-Sequence GRU architecture with Luong attention\n- Trained on the Cornell Movie Dialogs Corpus\n- Hyperparameter sweeps via Weights \u0026 Biases (W\u0026B)\n- GPU training with PyTorch \u0026 profiling\n- TorchScript conversion (traced + scripted) for deployment\n- Performance profiling via `torch.profiler`\n- Exportable to CPU-compatible `.pt` model for inference in C++ (tested)\n\n---\n\n## Technical Stack\n\n- **Python 3.11**, **PyTorch**\n- **TorchScript** for model export\n- **Weights \u0026 Biases** for hyperparameter tuning\n- **torch.profiler** for performance analysis\n- Jupyter Notebook for experimentation\n- Google Colab (GPU backend) for training\n\n---\n\n## Model Training and Tuning\n\n- Model: Seq2Seq with 2-layer GRU (encoder \u0026 decoder)\n- Attention: Luong (\"dot\") attention mechanism\n- Dataset: Cornell Movie Dialogs\n- Embedding size: 500  \n- Training iterations: 4000  \n- Batch size: 64  \n- Loss achieved: **2.88**\n\n### Hyperparameter Sweep (W\u0026B)\n\nTested 50 combinations with:\n- Learning Rate: [0.0001, 0.00025, 0.0005, 0.001]\n- Gradient Clipping: [0, 25, 50, 100]\n- Decoder LR ratio: [1, 3, 5, 10]\n- Optimizer: Adam / SGD\n- Teacher Forcing: [0, 0.5, 1.0]\n\n**Best configuration (jumping-sweep-17)**:\n- Loss: 2.88  \n- Clip: 100  \n- LR: 0.0005  \n- Optimizer: Adam  \n- Teacher Forcing: 1.0  \n- Decoder LR Ratio: 3.0\n\n---\n\n## TorchScript Conversion\n\nConverted models for non-Python environments:\n- Traced Encoder → `traced_encoder.pt`\n- Traced Decoder → `traced_decoder.pt`\n- Scripted GreedySearchDecoder → `scripted_searcher.pt`\n\n```python\ntorch.jit.save(scripted_searcher, \"scripted_chatbot_cpu.pth\")\n```\n\n✔️ Fully compatible with TorchScript static graph  \n✔️ Dynamic control flow handled via `torch.jit.script()`  \n✔️ Exported for CPU (map_location=\"cpu\") to support C++ deployment\n\n---\n\n## Performance Profiling\n\nUsed `torch.profiler` and Chrome Trace Viewer (`chrome://tracing`) to analyze:\n- CUDA time\n- Memory usage\n- Execution bottlenecks\n\n### Latency Comparison\n\n| Model Type       | Inference Time | Speedup |\n|------------------|----------------|---------|\n| Native PyTorch   | 0.0651 sec     | 1x      |\n| TorchScript      | 0.0519 sec     | **1.25x** |\n\n---\n\n## Sample Responses\n\n| Input             | Response                    |\n|------------------|-----------------------------|\n| hello            | hello . ? ? ? ?             |\n| what's up?       | i want to talk . . !        |\n| who are you?     | i am your father . . !      |\n| where are you from? | i am not home . .        |\n\nNote: Some responses reflect dataset bias and should not be used in production without moderation.\n\n---\n\n## Run Instructions\n\n### Training (Colab)\n\n```bash\npython chatbot_train.py\n```\n\n### Evaluation\n\n```bash\npython evaluate.py\n```\n\n### TorchScript Export\n\n```bash\npython export_torchscript.py\n```\n\n### Inference (Scripted)\n\n```bash\npython chatbot_infer.py\n```\n\n---\n\n## What I Learned\n\n- End-to-end ML pipeline: preprocessing → training → tuning → deployment  \n- TorchScript conversion for portability  \n- GPU profiling using `torch.profiler`  \n- W\u0026B for effective hyperparameter optimization  \n- Latency benchmarking \u0026 model efficiency tuning\n\n---\n\n## Directory Structure\n\n```\nchatbot.ipynb               # Main training + inference notebook\nnonPython_chatbot.cpp       # C++ inference attempt (TorchScript)\nchatbot_model.pt            # PyTorch model checkpoint\nscripted_searcher.pt        # Final TorchScript model (for deployment)\ntraced_encoder.pt\ntraced_decoder.pt\nlibtorch-v2.1.0.zip         # LibTorch for Apple Silicon\nREADME.md                   # You're here!\n```\n\n---\n\n## References\n\n- [Weights \u0026 Biases](https://wandb.ai/)\n- [TorchScript Docs](https://pytorch.org/docs/stable/jit.html)\n- [Cornell Movie Dialogs Corpus](https://www.cs.cornell.edu/~cristian/Cornell_Movie-Dialogs_Corpus.html)\n- [LibTorch for Apple Silicon](https://github.com/mlverse/libtorch-mac-m1)\n\n---\n\n## Author\n\n**Gopala Krishna Abba**  \n[LinkedIn](https://linkedin.com/igopalakrishna) • [W\u0026B Project](https://wandb.ai/ga2664-new-york-university/chatbot/sweeps/iwgnqx8h?nw=nwuserga2664)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Figopalakrishna%2Fhigh-perf-chatbot-torchscript","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Figopalakrishna%2Fhigh-perf-chatbot-torchscript","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Figopalakrishna%2Fhigh-perf-chatbot-torchscript/lists"}