{"id":50348130,"url":"https://github.com/musadiqpasha/file_transfer","last_synced_at":"2026-05-29T20:01:37.651Z","repository":{"id":354927133,"uuid":"1226043149","full_name":"MusadiqPasha/file_transfer","owner":"MusadiqPasha","description":null,"archived":false,"fork":false,"pushed_at":"2026-04-30T23:41:02.000Z","size":180,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-01T01:17:25.080Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MusadiqPasha.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-30T23:10:18.000Z","updated_at":"2026-04-30T23:41:06.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/MusadiqPasha/file_transfer","commit_stats":null,"previous_names":["musadiqpasha/file_transfer"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/MusadiqPasha/file_transfer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MusadiqPasha%2Ffile_transfer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MusadiqPasha%2Ffile_transfer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MusadiqPasha%2Ffile_transfer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MusadiqPasha%2Ffile_transfer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MusadiqPasha","download_url":"https://codeload.github.com/MusadiqPasha/file_transfer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MusadiqPasha%2Ffile_transfer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33668186,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-29T02:00:06.066Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-29T20:01:35.785Z","updated_at":"2026-05-29T20:01:37.628Z","avatar_url":"https://github.com/MusadiqPasha.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# File Transfer Protocol — BigEndian Challenge 1\n\nA complete client-server file transfer system built from scratch using raw TCP sockets. Implements chunking, sequence numbers, out-of-order delivery simulation, SHA-256 integrity verification, packet drop simulation (random and manual), and automatic retransmission via ACK/NACK — with live web dashboards for both server and client.\n\n---\n\n## Features\n\n| Feature | Details |\n|---|---|\n| TCP socket communication | Raw `socket` module, no libraries |\n| 1024-byte chunking | Fixed-size chunks with sequence numbers |\n| Out-of-order simulation | Server shuffles chunks before sending |\n| SHA-256 checksum | Computed server-side, verified client-side |\n| Packet drop simulation | **Random** (configurable %) or **Manual** (pick exact chunks) |\n| ACK / NACK retransmission | Client detects missing chunks, sends NACK with missing seq numbers, server resends — loops until all chunks arrive |\n| Transfer speed + timing | Measured end-to-end including all retransmit rounds |\n| Live web dashboards | Real-time monitor for both server and client |\n\n---\n\n## Project Structure\n\n```\nfile_transfer/\n├── server.py         # TCP socket server (port 9999) + Flask API (port 5001)\n├── client_web.py     # Flask web app (port 5002) + TCP socket client\n├── utils.py          # Shared: split_into_chunks, reassemble_chunks, compute_checksum\n├── server_ui.html    # Live server dashboard + drop control panel\n├── client_ui.html    # File upload dashboard + transfer progress\n└── README.md\n```\n\n---\n\n## How to Run\n\n### 1. Install dependencies\n\n```bash\npip install flask\n```\n\n### 2. Terminal 1 — Start the server\n\n```bash\npython server.py\n```\n\nOpen **http://localhost:5001** → server dashboard loads.\n\n### 3. Terminal 2 — Start the client\n\n```bash\npython client_web.py\n```\n\nOpen **http://localhost:5002** → client dashboard loads.\n\n### 4. (Optional) Configure packet drops before transferring\n\nOn the **server dashboard** (http://localhost:5001), use the **Packet Drop Control** panel on the left:\n\n- **None** — no drops, clean transfer\n- **Random** — drag the slider to set drop probability (0–90%), every chunk has that % chance of being dropped on round 1\n- **Manual** — type specific sequence numbers (e.g. `0`, `3`, `7`), click Add → those exact chunks will be dropped\n\nClick **Apply Settings** before the client sends a file.\n\n### 5. Transfer a file\n\n- Go to **http://localhost:5002**\n- Drop or select any file\n- Click **Send File →**\n- Watch both dashboards update live\n\n---\n\n## Screenshots\n\n### Client Dashboard\n![Client UI](client.png)\n\n### Server Dashboard\n![Server UI](server.png)\n\n---\n\n## Full Protocol — With Retransmission\n\n```\nCLIENT                                  SERVER\n------                                  ------\n1. Send: [4B file_size][file_data]\n                                        2. Receive file\n                                        3. split_into_chunks() → 1024B chunks\n                                        4. Build chunk_map {seq: data}\n                                        5. Shuffle chunks (out-of-order sim)\n                                        6. Send: [4B total_chunks]\n\n──── ROUND 1 (and retransmit rounds) ────────────────────────────\n\n                                        7. For each chunk:\n                                           should_drop(seq, round)?\n                                           → YES: skip (log [DROP])\n                                           → NO:  send [4B seq][4B len][data]\n                                        8. Send: b\"END_\"\n\n9. Collect chunks → chunk_dict {seq: data}\n10. missing = set(range(total)) - set(chunk_dict.keys())\n\n    If missing is empty:\n11a. Send: [4B: 0]  ← ACK\n                                        12a. Receive 0 → break loop\n\n    If missing chunks exist:\n11b. Send: [4B: count][count × 4B seq_nos]  ← NACK\n                                        12b. Receive NACK\n                                        13.  Rebuild to_send from chunk_map\n                                             (retransmits never dropped)\n                                        → back to step 7\n\n──── ALL CHUNKS RECEIVED ────────────────────────────────────────\n\n                                        14. compute_checksum(original_file)\n                                        15. Send: [64B SHA-256 hex string]\n\n16. recv_all(sock, 64) → expected checksum\n17. reassemble_chunks(chunk_dict)\n    → sorted(keys) → b\"\".join(chunks)\n18. compute_checksum(reassembled) → actual\n19. actual == expected?\n    → YES: save to received/, print \"Transfer Successful\"\n    → NO:  print \"CHECKSUM MISMATCH\"\n```\n\n---\n\n## Design Decisions\n\n### Why TCP?\nTCP guarantees delivery and byte ordering at the network layer. We still implement out-of-order handling at the **application layer** (by shuffling chunks ourselves) to demonstrate sequence-number-based reassembly — the same technique used in real streaming protocols.\n\n### Why `recv_all()`?\nTCP is a stream protocol — it does not preserve message boundaries. Sending 1024 bytes may arrive as 512 + 512 across two `recv()` calls. `recv_all()` loops until exactly the requested number of bytes has been collected.\n\n### Why `struct.pack(\"\u003eI\", n)`?\nIntegers must be serialized to bytes to be sent over a socket. `\"\u003eI\"` means big-endian unsigned 32-bit integer — \"network byte order\", the universal convention for all major network protocols (RFC 1700). The receiver uses `struct.unpack` to reconstruct the integer.\n\n### Why a separate `chunk_map` dictionary?\n```python\nchunk_map = {seq: data for seq, data in chunks}\n```\nIn retransmit rounds the server receives a NACK with a list of missing sequence numbers and must look them up instantly. A dict gives O(1) lookup vs O(n) scan of a list.\n\n### Why two separate state dicts with two separate locks?\n`drop_config` is written by the Flask thread (UI Apply button) and read by the socket thread. `state` is written by the socket thread and read by Flask for status polls. Separate locks mean one thread writing drop config never blocks another thread reading transfer state — better concurrency and cleaner separation of concerns.\n\n### Why retransmits are never dropped\n```python\ndef should_drop(seq_no, round_num):\n    if round_num \u003e 1:\n        return False\n```\nGuarantees the retransmission loop always terminates. In real networks retransmits can also fail, but for a demo system we guarantee convergence.\n\n---\n\n## Example Output\n\n**Server terminal (with 25% random drop):**\n```\n[10:42:01]  Socket server listening on 127.0.0.1:9999\n[10:42:05]  Client connected: ('127.0.0.1', 52341)\n[10:42:05]  File received (3072 bytes)\n[10:42:05]  Drop mode: RANDOM (25% per chunk)\n[10:42:05]  File split into 3 chunks\n[10:42:05]  Round 1: sending all 3 chunks (shuffled)\n[10:42:05]    Sent chunk #2 (1024 B)\n[10:42:05]    [DROP] Chunk #0 dropped\n[10:42:05]    Sent chunk #1 (1024 B)\n[10:42:05]  Round 1 done — waiting for ACK/NACK\n[10:42:05]  NACK — 1 missing: [0]\n[10:42:05]  Round 2: retransmitting 1 missing chunk(s)\n[10:42:05]    Sent chunk #0 (1024 B)\n[10:42:05]  ACK received — all chunks delivered in 2 round(s)\n[10:42:05]  Checksum sent: 3b4c1d8eaf29c7...\n```\n\n**Client terminal:**\n```\n[10:42:05]  Connected to server 127.0.0.1:9999\n[10:42:05]  Sent 3072 bytes to server\n[10:42:05]  Server will send 3 chunks (with drop simulation)\n[10:42:05]    Received chunk #2 (1024 B)\n[10:42:05]    Received chunk #1 (1024 B)\n[10:42:05]  Round 1: missing 1 chunk(s): [0], sending NACK\n[10:42:05]    Received chunk #0 (1024 B)\n[10:42:05]  ACK sent — all 3 chunks received after 2 round(s)\n[10:42:05]  Checksum received: 3b4c1d8eaf29c7...\n[10:42:05]  Reassembling chunks in sequence order...\n[10:42:05]  VERIFIED — checksums match. Saved as received_sample.txt\n[10:42:05]  Transfer Successful\n```\n\n---\n\n## What Each File Does\n\n| File | Responsibility |\n|---|---|\n| `utils.py` | `split_into_chunks()` — splits bytes into 1024B chunks with seq numbers. `reassemble_chunks()` — sorts by seq number and joins. `compute_checksum()` — SHA-256 hex digest. |\n| `server.py` | Receives file over TCP, chunks it, simulates drops (random/manual), sends chunks shuffled, handles NACK/retransmit loop, sends checksum. Flask API exposes status + drop config to the dashboard. |\n| `client_web.py` | Flask web app receives file upload from browser, runs socket client in a thread, implements NACK/retransmit loop, reassembles, verifies checksum. Flask API exposes transfer state to dashboard. |\n| `server_ui.html` | Drop control panel (mode tabs, rate slider, manual chunk picker). Live stats: file size, chunks sent, dropped, retransmit rounds. Chunk grid (white = sent, red = dropped). Scrollable log. |\n| `client_ui.html` | File drop zone + Send button. Progress bar. Checksum comparison panel (both boxes turn green on match). Chunk arrival visualizer (shows shuffled order). Transfer speed + time. Log. |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmusadiqpasha%2Ffile_transfer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmusadiqpasha%2Ffile_transfer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmusadiqpasha%2Ffile_transfer/lists"}