{"id":17156331,"url":"https://github.com/seamile/fastcopy","last_synced_at":"2025-04-13T13:30:56.858Z","repository":{"id":48587853,"uuid":"365557615","full_name":"seamile/FastCopy","owner":"seamile","description":"🚅 FastCopy is a remote file copy program similar to scp. It uses concurrent transmission, so the speed is faster than scp.","archived":false,"fork":false,"pushed_at":"2023-07-07T04:35:15.000Z","size":139,"stargazers_count":28,"open_issues_count":1,"forks_count":7,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-27T04:41:31.464Z","etag":null,"topics":["rsync","scp","sftp-client","ssh"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/seamile.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-08T16:10:11.000Z","updated_at":"2024-12-20T07:13:11.000Z","dependencies_parsed_at":"2022-09-05T22:21:02.646Z","dependency_job_id":null,"html_url":"https://github.com/seamile/FastCopy","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seamile%2FFastCopy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seamile%2FFastCopy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seamile%2FFastCopy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seamile%2FFastCopy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/seamile","download_url":"https://codeload.github.com/seamile/FastCopy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248720982,"owners_count":21151021,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["rsync","scp","sftp-client","ssh"],"created_at":"2024-10-14T22:06:10.346Z","updated_at":"2025-04-13T13:30:56.568Z","avatar_url":"https://github.com/seamile.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FastCopy\n\n[![PyPI Version](https://img.shields.io/pypi/v/fastcopy?color=blue\u0026label=Version\u0026logo=python\u0026logoColor=white)](https://pypi.org/project/fastcopy/)\n[![Installs](https://static.pepy.tech/personalized-badge/fastcopy?period=total\u0026units=international_system\u0026left_color=grey\u0026right_color=blue\u0026left_text=Installs)](https://pepy.tech/project/fastcopy)\n\n基于 SSH 协议的多线程文件传输工具。\n\n目标是用来替换 `scp` 和 `rsync`。\n\n\n## 特点\n\n- 文件切块处理，并行传输，速度更快\n- 支持使用 *文件名通配符* 及 *正则表达式* 来匹配需要传输的文件\n- 自动跳过本地与远程相同内容的文件\n- 自动保持 *发送端* 与 *接收端* 文件权限完全相同\n- 支持 SSH Config\n- 支持 SSH Agent\n\n\n## TODO\n\n- [ ] 断点续传支持\n- [ ] 改进配置管理方式\n- [ ] 握手时确认会话参数，取消全局变量方式\n- [ ] 版本前后兼容\n- [ ] 编写测试用例\n- [ ] 保持软链接\n\n\n## 安装\n\n使用前须在 *服务器* 和 *本地* 同时安装本程序。\n\n```shell\npip install fastcopy\n```\n\n## 使用\n\n1. 服务器\n\n    *运行前首先确保服务端的 7523 端口未被占用*\n\n    ```shell\n    fcpd -d\n    ```\n\n2. 本地\n\n    - 下载\n\n        ```shell\n        fcp user@host:/foo/bar ./\n        ```\n\n        [![asciicast](https://asciinema.org/a/430555.svg)](https://asciinema.org/a/430555)\n\n    - 上传\n\n        ```shell\n        fcp ./fake/file user@host:/foo/bar\n        ```\n\n        [![asciicast](https://asciinema.org/a/430553.svg)](https://asciinema.org/a/430553)\n\n\n## 报文设计\n\n所有数据包均采用**大端字节序**\n\n### 1. 报文统一格式\n\n|  flag   | chksum  | length  | payload |\n| :-----: | :-----: | :-----: | :-----: |\n| 1 Bytes | 4 Bytes | 2 Bytes |   ...   |\n\n### 2. 报文类型\n\n1. 推送申请: `0x1`\n2. 拉取申请: `0x2`\n3. 建立会话: `0x3`\n4. 后续连接: `0x4`\n5. 传输模式: `0x5`\n6. 目录信息: `0x6`\n7. 文件信息: `0x7`\n8. 文件数量: `0x8`\n9. 文件就绪: `0x9`\n10. 数据传输: `0xa`\n11. 传输完成: `0xb`\n12. 异常退出: `0xc`\n\n\n### 3. 报文详情\n\n1. 数据请求\n\n    连接建立后，客户端首先需要向服务器申请 *拉取* 或 *推送*，并将 *目的路径* 传给服务器\n\n    - 拉取、推送的标识由 `flag` 字段决定\n    - 方向: Client -\u003e Server\n    - Payload 格式为:\n\n        | connection info |\n        | :-------------: |\n        |   json string   |\n\n2. 建立会话\n\n    服务器收到第一步的申请后，会产生一个 SessionID，并回传给客户端，客户端需要在自己本地保存\n\n    - 方向: Server -\u003e Client\n    - Payload 格式为:\n\n        | session_id |\n        | :--------: |\n        |  16 Bytes  |\n\n3. 后续连接\n\n    客户端后续与服务器建立的并发连接，第一个报文须告诉服务器 SessionID\n\n    - 方向: Client -\u003e Server\n    - Payload 格式为:\n\n        | session_id |\n        | :--------: |\n        |  16 Bytes  |\n\n4. 文件总量\n\n    连接就绪后，发送端需告知接收端文件总量\n\n    - Payload 长度 4 字节，所以最大允许传输文件数量为 4,294,967,296\n    - 方向: Sender -\u003e Receiver\n    - Payload 格式:\n\n        | n_files |\n        | :-----: |\n        | 4 Bytes |\n\n5. 文件信息\n\n    文件发送发需将每一个文件的信息告知接收端。\n    包括文件的编号、权限、大小、创建时间、修改时间、访问时间、校验和、路径。\n    其中路径为相对路径。\n\n    - 方向: Sender -\u003e Receiver\n    - Payload 格式:\n\n        | file_id |  perm   |  size   |  mtime  |  chksum  | path  |\n        | :-----: | :-----: | :-----: | :-----: | :------: | :---: |\n        | 4 Bytes | 2 Bytes | 8 Bytes | 8 Bytes | 16 Bytes |  ...  |\n\n6. 接收端文件准备就绪\n\n    接收端收到文件信息后，需将文件信息记录起来，并在本地创建同样大小的空文件\n\n    - 方向: Receiver -\u003e Sender\n    - Payload 格式:\n\n        | file_id |\n        | :-----: |\n        | 4 Bytes |\n\n7. 文件数据块传输报文\n\n    Chunk Sequence 占用 4 字节，所以支持的单个文件最大为: 4 GB * ChunkSize\n\n    - 方向: Sender -\u003e Receiver\n    - Payload 格式:\n\n        | file_id |   seq   | data  |\n        | :-----: | :-----: | :---: |\n        | 4 Bytes | 4 Bytes |  ...  |\n\n\n### 4. 握手过程\n\n| 序号 |                  客户端                   |               服务器                |\n| ---- | :---------------------------------------: | :---------------------------------: |\n| 1    |                客户端启动                 |             服务端启动              |\n| 2    |                                           |           等待客户端连接            |\n| 3    |               发起连接请求                |                                     |\n| 4    |                                           |           接收客户端连接            |\n| 5    |                                           |   等待客户端请求 (请求超时则断开)   |\n| 6    |        发送 `PUSH` 或 `PULL` 请求         |                                     |\n| 7    |                                           |           产生 SessionID            |\n| 8    |                                           |       将 SessionID 传回客户端       |\n| 9    |           接收 SessionID 并保存           |                                     |\n| 10   |           循环创建多个并行连接            |                                     |\n| 11   | 新连接携带 SessionID 逐一发送`ATTACH`请求 |                                     |\n| 12   |                                           |         确认 SessionID 无误         |\n| 13   |                                           | 将新连接添加至对应 Session 的连接池 |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseamile%2Ffastcopy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fseamile%2Ffastcopy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseamile%2Ffastcopy/lists"}