{"id":27000237,"url":"https://github.com/amul-thantharate/kafkaconnectapp","last_synced_at":"2026-04-28T12:02:09.362Z","repository":{"id":285709271,"uuid":"959070011","full_name":"Amul-Thantharate/KafkaConnectApp","owner":"Amul-Thantharate","description":"A modern, distributed chat system that combines the speed of Go, the reliability of MySQL, and the scalability of Kafka. Perfect for building real-time communication platforms that can handle thousands of concurrent users with ease.","archived":false,"fork":false,"pushed_at":"2025-04-02T08:14:03.000Z","size":19,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-20T00:47:35.063Z","etag":null,"topics":["docker","docker-compose","golang","kafka","mysql"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Amul-Thantharate.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-02T08:11:15.000Z","updated_at":"2025-04-02T08:14:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"bd28398e-3766-4409-985f-01d6b0388193","html_url":"https://github.com/Amul-Thantharate/KafkaConnectApp","commit_stats":null,"previous_names":["amul-thantharate/kafkaconnectapp"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Amul-Thantharate/KafkaConnectApp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Amul-Thantharate%2FKafkaConnectApp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Amul-Thantharate%2FKafkaConnectApp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Amul-Thantharate%2FKafkaConnectApp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Amul-Thantharate%2FKafkaConnectApp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Amul-Thantharate","download_url":"https://codeload.github.com/Amul-Thantharate/KafkaConnectApp/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Amul-Thantharate%2FKafkaConnectApp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32379629,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-28T11:25:28.583Z","status":"ssl_error","status_checked_at":"2026-04-28T11:25:05.435Z","response_time":56,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","docker-compose","golang","kafka","mysql"],"created_at":"2025-04-04T03:19:40.323Z","updated_at":"2026-04-28T12:02:09.356Z","avatar_url":"https://github.com/Amul-Thantharate.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 💬 KafkaConnect Chat Application 📚\n\n[![Go Version](https://img.shields.io/badge/Go-1.16+-00ADD8?style=flat\u0026logo=go)](https://golang.org)\n[![MySQL](https://img.shields.io/badge/MySQL-8.0+-00758F?style=flat\u0026logo=mysql)](https://www.mysql.com/)\n[![Apache Kafka](https://img.shields.io/badge/Apache%20Kafka-3.0+-231F20?style=flat\u0026logo=apache-kafka)](https://kafka.apache.org/)\n[![Docker](https://img.shields.io/badge/Docker-20.10+-2496ED?style=flat\u0026logo=docker)](https://www.docker.com/)\n[![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)\n\n🚀 A modern, distributed chat system that combines the speed of Go, the reliability of MySQL, and the scalability of Kafka. Perfect for building real-time communication platforms that can handle thousands of concurrent users with ease. Whether you're creating a team chat, community platform, or learning distributed systems - this project has you covered!\n\nA scalable, real-time chat application built with modern technologies:\n- **Go (Golang)** for high-performance server-side operations\n- **MySQL** for reliable user and group data persistence\n- **Apache Kafka** for robust message queuing and real-time communication\n- **Docker** for containerized deployment and easy scaling\n\n## ✨ Key Features\n\n### 👤 User Management\n- Secure user authentication with bcrypt password encryption\n- User registration and login system\n- Real-time online/offline status tracking\n- User session management\n\n### 💬 Messaging System\n- Real-time message delivery\n- Private messaging between users\n- Broadcast messages to all online users\n- Persistent message history through Kafka\n\n### 👥 Group Chat\n- Create and manage chat groups\n- Join/leave group functionality\n- Group-specific messaging\n- Member count tracking\n- Real-time group updates\n\n### 🔔 System Features\n- Real-time notifications for user activities\n- Join/leave event broadcasting\n- Online user list updates\n- Group membership tracking\n- Error handling and user feedback\n\n## 📋 Prerequisites\n\n### Required Software\n- Go 1.16 or higher\n- Docker 20.10+ and Docker Compose\n- NetCat (nc) for client connections\n\n### System Requirements\n- Minimum 4GB RAM\n- 10GB free disk space\n- Linux/Unix-based system (recommended)\n\n## 🛠️ Setup\n\n1. **Clone the repository**\n   ```bash\n   git clone https://github.com/Amul-Thantharate/KafkaConnectApp.git\n   cd KafkaConnectApp\n   ```\n\n2. **Start the infrastructure**\n   ```bash\n   docker-compose up -d\n   ```\n   This will start:\n   - MySQL on port 3307\n   - Zookeeper on port 2181\n   - Kafka on port 9092\n\n3. **Run the chat server**\n   ```bash\n   go run main.go\n   ```\n   The server will start on port 8080.\n\n## 💬 Usage\n\n1. **Connect to the chat server**\n   ```bash\n   nc localhost 8080\n   ```\n\n2. **Available Commands**\n   ```\n   /help                          - Show all available commands\n   /register \u003cusername\u003e \u003cpassword\u003e - Create a new account\n   /login \u003cusername\u003e \u003cpassword\u003e    - Login to your account\n   /logout                        - Logout from current session\n   /list_user                     - Show online users\n   /pm \u003cusername\u003e \u003cmessage\u003e       - Send private message\n   /broadcast \u003cmessage\u003e           - Send message to all users\n   /group_create \u003cgroup_name\u003e     - Create a new group\n   /group_join \u003cgroup_name\u003e       - Join an existing group\n   /group_leave \u003cgroup_name\u003e      - Leave a group\n   /group_msg \u003cgroup_name\u003e \u003cmsg\u003e  - Send message to group\n   ```\n\n## 🔍 Debugging and Monitoring\n\n### Database Inspection\n\n1. **Connect to MySQL**\n   ```bash\n   mysql -h localhost -P 3307 -u root -proot chat_db\n   ```\n\n2. **Useful MySQL queries**\n   ```sql\n   -- List all users\n   SELECT * FROM users;\n\n   -- Show online users\n   SELECT * FROM users WHERE online = true;\n\n   -- List all groups\n   SELECT * FROM groups;\n\n   -- Show group members\n   SELECT g.name, u.username \n   FROM group_members gm \n   JOIN groups g ON gm.group_id = g.id \n   JOIN users u ON gm.user_id = u.id;\n   ```\n\n### Kafka Message Inspection\n\n1. **View broadcast messages**\n   ```bash\n   # Install kafkacat if not available\n   sudo apt-get install kafkacat\n\n   # Listen to broadcast messages\n   kafkacat -b localhost:9092 -t broadcast -C\n\n   # Listen to system messages\n   kafkacat -b localhost:9092 -t system -C\n   ```\n\n2. **Using Kafka Console Consumer**\n   ```bash\n   # If you have Kafka tools installed\n   kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic broadcast --from-beginning\n   ```\n\n## 🏗️ Architecture\n\n- **TCP Server**: Handles client connections and command processing\n- **MySQL**: Stores user accounts, groups, and membership information\n- **Kafka**: Manages message delivery and persistence\n- **GORM**: ORM for database operations\n- **bcrypt**: Secure password hashing\n\n## 📝 Database Schema\n\n```sql\n-- Users table\nCREATE TABLE users (\n    id BIGINT PRIMARY KEY AUTO_INCREMENT,\n    username VARCHAR(255) UNIQUE,\n    password VARCHAR(255),\n    online BOOLEAN\n);\n\n-- Groups table\nCREATE TABLE groups (\n    id BIGINT PRIMARY KEY AUTO_INCREMENT,\n    name VARCHAR(255) UNIQUE\n);\n\n-- Group Members table\nCREATE TABLE group_members (\n    id BIGINT PRIMARY KEY AUTO_INCREMENT,\n    user_id BIGINT,\n    group_id BIGINT,\n    FOREIGN KEY (user_id) REFERENCES users(id),\n    FOREIGN KEY (group_id) REFERENCES groups(id)\n);\n```\n\n## 🛑 Stopping the Application\n\n1. **Stop the Go server**\n   ```bash\n   # Press Ctrl+C in the terminal running the server\n   # Or find and kill the process\n   pkill -f \"go run main.go\"\n   ```\n\n2. **Stop the infrastructure**\n   ```bash\n   docker-compose down\n   ```\n\n## 🔒 Security Notes\n\n- Passwords are hashed using bcrypt before storage\n- All database credentials should be changed in production\n- Consider adding TLS for secure communication in production\n\n## 🐛 Troubleshooting\n\n1. **MySQL Connection Issues**\n   - Ensure MySQL is running: `docker-compose ps`\n   - Check port availability: `netstat -an | grep 3307`\n   - Verify credentials in database/database.go\n\n2. **Kafka Connection Issues**\n   - Ensure Kafka is running: `docker-compose ps`\n   - Check Kafka logs: `docker-compose logs kafka`\n   - Verify connection string in kafka/kafka.go\n\n3. **Common Issues**\n   - Port conflicts: Change ports in docker-compose.yml\n   - Database migration: Check GORM logs for schema updates\n   - Message delivery: Monitor Kafka consumer lag\n\n## 🤝 Contributing\n\nFeel free to submit issues, fork the repository, and create pull requests for any improvements.\n\n## 📝 License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n\n## 🌐 Contact\n\nFor questions or support, please contact Amul Thantharate at amulthantharate@gmail.com\nThanks for using this project! ❤️\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famul-thantharate%2Fkafkaconnectapp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Famul-thantharate%2Fkafkaconnectapp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famul-thantharate%2Fkafkaconnectapp/lists"}