{"id":22901286,"url":"https://github.com/mooerslab/multiplefiletransfer","last_synced_at":"2026-01-26T03:02:42.771Z","repository":{"id":165234156,"uuid":"585623946","full_name":"MooersLab/multipleFileTransfer","owner":"MooersLab","description":"Transfer multiple files by scp without repeated manual password entry","archived":false,"fork":false,"pushed_at":"2025-08-13T21:46:17.000Z","size":41,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-13T23:31:27.118Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MooersLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-01-05T16:39:58.000Z","updated_at":"2025-08-13T21:46:20.000Z","dependencies_parsed_at":"2024-04-10T12:38:31.381Z","dependency_job_id":"dcd40b55-2a86-42a3-bad2-8f18244b4640","html_url":"https://github.com/MooersLab/multipleFileTransfer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MooersLab/multipleFileTransfer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MooersLab%2FmultipleFileTransfer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MooersLab%2FmultipleFileTransfer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MooersLab%2FmultipleFileTransfer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MooersLab%2FmultipleFileTransfer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MooersLab","download_url":"https://codeload.github.com/MooersLab/multipleFileTransfer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MooersLab%2FmultipleFileTransfer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28765570,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-26T02:25:41.078Z","status":"ssl_error","status_checked_at":"2026-01-26T02:24:28.809Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-14T01:32:42.764Z","updated_at":"2026-01-26T03:02:42.764Z","avatar_url":"https://github.com/MooersLab.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"![version](https://img.shields.io/static/v1?label=multipleFileTranser\u0026message=0.6\u0026color=brightcolor)\n[![license: mit](https://img.shields.io/badge/license-mit-blue.svg)](https://opensource.org/licenses/mit)\n\n# Transfer multiple files by scp without repeated manual password entry\n\n*Problem:* Transferring multiple tar files from a remote computer can be laborious because you are asked to enter your password after each file transfer. \n\nThis solution requires making an authentication key using the program *sshpass*. You have to have ssh installed to use this solution.\n\n1. Generate an authentication key. You should use a short passphrase when prompted for one.\n```bash\nssh-keygen -t rsa -b 4096\n```\n2. Copy the authentication key to the remote server(s).\n```bash\nssh-copy-id bmooers@schooner2.oscer.ou.edu:/home/bmooers\n```\n3. You will be prompted for the passphrase the next time you use `ssh` to log in or `scp` to transfer files. The prompt with scp will occur just once instead of the number of times corresponding to the list's files.\n\n\nThe above steps are done only once. \nThe steps below must be done at the start of a session of multiple file transfers. \nYou will have to enter your passphrase only once. \nDue to ethernet instability, you may have to limit the amount of tar files transferred to a quarter TB.\n\n4. Enter: `eval \"$(ssh-agent -s)\"`\n5. Enter: `ssh-add ~/.ssh/id_rsa`\n\n\nStore the following two functions in your `.zshrc` or `.bashrc` or store in a .bashFunctions file and source from your `.zshrc` or `.bashrc` file.\nThis first function is for putting file(s) onto the OURdisk.\n\n```bash\nfunction dtn2mput {\nprint \"Enter the next two lines once for a day of transfers, then you can run this script multiple times with different batches of files.\"\nprintf 'eval \"$(ssh-agent -s)\"'\nprintf 'ssh-add ~/.ssh/id_rsa'\nif [ $# -lt 1 ]; then\n echo 1\u003e\u00262 \"$0: not enough arguments. Need one filename or a list of two or more separated by whitespaces.\"\n echo \"Usage1: smbmget fileName1 fileName2\"\n echo \"Note the dropped leading forward slash\"\n return 2\nfi\ndestination=\"/ourdisk/hpc/bmooers/bmooers/dont_archive\"\n  for file in \"$@\"; do\n    echo \"Put '$file' to dtn2.\"\n    scp -i ~/.ssh/id_rsa -pr \"$file\" bmooers@dtn2.oscer.ou.edu:\"$destination\"/.\n  done\necho \"All done.\"  \necho \"Function stored in ~/.bashFunctions3.\"\n# Call the function with the list of files \"$@\" to be transferred to OURdisk computer.  \n}\n```\n\n6. Enter `dtn2mput file1.tar file2.tar file3.tar file4.tar` and so on. Note the use of whitespaces to separate the files.\n\nThis second function is for getting file9(s) from OURdisk.\n\n```bash\nfunction dtn2mget {\nprintf \"Enter the next two lines once for a day of transfers, then you can run this script mutliple times with different batches of files.\"\nprintf 'eval \"$(ssh-agent -s)\"'\nprintf 'ssh-add ~/.ssh/id_rsa'\nif [ $# -lt 1 ]; then\n echo 1\u003e\u00262 \"$0: not enough arguments. Need one filename or a list of two or more separated by whitespaces.\"\n echo \"Usage1: dtn2mget fileName1 fileName2\"\n return 2\nfi\nsource=\"/data/bmooers\"\n  for file in \"$@\"; do\n    echo \"Fetch '$file' from dtn2.\"\n    scp -i ~/.ssh/id_rsa -pr mooers@smbcopy.slac.stanford.edu:\"$source\"/\"$file\" .\n  done\necho \"All done.\"  \necho \"Function stored in ~/.bashFunctions3.\"\n# Call the function with the list of files \"$@\" to be transferred from OURdisk computer.  \n}\n```\n\n## Long Duration Transfers of Large Files\n\nStart a tmux session and launch the dtn2mput function with a list of tar files.\nFortunately, the tmux session will continue despite your premature disconnections from the server.\nYou can launch up to 10 or more tmux sessions in parallel from an account on a HPC computer.\n\n\n## Update History\n\n|Version      | Changes                                         | Date            |\n|:-----------:|:-----------------------------------------------:|:---------------:|\n| Version 0.3 |  Fixed typos in README.md                       | 2024 April 10   |\n| Version 0.4 |  Added Solution 1.                              | 2024 August 13  |\n| Version 0.5 |  Revised dramatically.                          | 2025 January 12 |\n| Version 0.6 |  Note about using tmux.                         | 2025 Augues 13 |\n\n\n## Sources of funding\n\n- NIH: R01 CA242845\n- NIH: R01 AI088011\n- NIH: P30 CA225520 (PI: R. Mannel)\n- NIH P20GM103640 and P30GM145423 (PI: A. West)\n\n\n  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmooerslab%2Fmultiplefiletransfer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmooerslab%2Fmultiplefiletransfer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmooerslab%2Fmultiplefiletransfer/lists"}