https://github.com/abhi18av/ena-genomes-link-aggregator
https://github.com/abhi18av/ena-genomes-link-aggregator
aria2c browser-automation clojure ena python
Last synced: 3 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/abhi18av/ena-genomes-link-aggregator
- Owner: abhi18av
- License: epl-1.0
- Created: 2019-04-28T06:52:53.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2020-08-15T04:43:24.000Z (almost 5 years ago)
- Last Synced: 2025-01-05T10:08:24.790Z (5 months ago)
- Topics: aria2c, browser-automation, clojure, ena, python
- Language: Python
- Size: 172 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# ena-genomes-link-aggregator
## STEP-1 : Aggregate the links using the excel file from Emilyn
- We did this via browser automation in `etaoin` and `clojure`
## STEP-2 : Download the links using an FTP enabled downloader
#### TRIAL-1 : Python script
This didn't work since the speed of the Remote-database-server is not good at all!
#### TRIAL-2 : A Python package
https://github.com/julienc91/multidl
Didn't work in an assuring way.
#### TRIAL-3 : aria2
Have relied on `aria2` to download things in parallel.
```
nohup aria2c -i urls.txt -j4 &
```
## STEP-3 : Upload all files in Emilyn's OneDrive
## License
Copyright © 2019 Abhinav Sharma ( @abhi18av )
Distributed under the Eclipse Public License either version 1.0 or (at
your option) any later version.