{"id":21117462,"url":"https://github.com/nileshsah/harwest-tool","last_synced_at":"2025-04-09T12:07:13.796Z","repository":{"id":36996720,"uuid":"243549763","full_name":"nileshsah/harwest-tool","owner":"nileshsah","description":"A one-shot tool to harvest submissions from different OJs onto one single VCS managed repository http://bit.ly/harwest","archived":false,"fork":false,"pushed_at":"2023-10-10T19:47:30.000Z","size":64,"stargazers_count":132,"open_issues_count":17,"forks_count":15,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-02T10:38:43.446Z","etag":null,"topics":["atcoder","codeforces","command-line-tool","competitive-programming","git","ojs","pypi"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nileshsah.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-27T15:27:37.000Z","updated_at":"2025-03-11T17:11:42.000Z","dependencies_parsed_at":"2024-12-24T10:22:46.954Z","dependency_job_id":"6ceb95a8-1673-440a-8b89-4ee1c21f8b57","html_url":"https://github.com/nileshsah/harwest-tool","commit_stats":{"total_commits":60,"total_committers":5,"mean_commits":12.0,"dds":0.25,"last_synced_commit":"cd4a601918a06f891daae77e1a44580d70a0049a"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nileshsah%2Fharwest-tool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nileshsah%2Fharwest-tool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nileshsah%2Fharwest-tool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nileshsah%2Fharwest-tool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nileshsah","download_url":"https://codeload.github.com/nileshsah/harwest-tool/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248036067,"owners_count":21037092,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atcoder","codeforces","command-line-tool","competitive-programming","git","ojs","pypi"],"created_at":"2024-11-20T02:46:08.703Z","updated_at":"2025-04-09T12:07:13.769Z","avatar_url":"https://github.com/nileshsah.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 📦 Harwest ⛏\n\n[![PyPI](https://img.shields.io/pypi/v/harwest.svg)](https://pypi.python.org/pypi/harwest)\n[![Downloads](https://pepy.tech/badge/harwest)](https://pepy.tech/project/harwest)\n[![PyPI](https://img.shields.io/pypi/l/harwest.svg)](https://github.com/nileshsah/harwest-tool/blob/master/LICENSE)\n\n**Harwest** takes away the hassle of managing your submission files on different online-judges by\nautomating the entire process of collecting and organizing your code submissions in one single Git repository.\n\nHere's a sample repository created using Harwest: [harwest-sample](https://github.com/nileshsah/harwest-sample)\n\n## Highlights\n* Fully automated collection of all yours submissions with minimal effort setup\n* Simple and easy to use interface to get you started in minutes\n* Extensive traceability for your submissions with reference to the problem, tags, submission date and more\n* Single commit for each submission stamped with the original submission date for building rich and accurate contributions graph\n* Automated git pushes to the remote repository with every update\n* Requires little to no knowledge of operating Git (though would strongly recommend learning it)\n\n## Platforms\n\nHarwest currently has extensive support for the following platforms:\n* [Codeforces](https://codeforces.com/)\n* [AtCoder](https://atcoder.jp/) _contributed by [@s-i-d-d-i-s](https://github.com/s-i-d-d-i-s)_\n\nwhile integration with various other OJs are still in the kitchen. Contributions are always welcomed.\n\n\n## Installation\n\nYou will require `Python 3.5+` along with `pip3` in order to be able to install and use Harwest.\nRefer to the documentation for installing `pip` on [windows](https://phoenixnap.com/kb/install-pip-windows), \n[ubuntu/linux](https://phoenixnap.com/kb/how-to-install-python-3-ubuntu) or\n[macOS](https://docs.python-guide.org/starting/install3/osx/)\n\nThe package is available at \u003chttps://pypi.python.org/pypi/harwest\u003e [![PyPI](https://img.shields.io/pypi/v/harwest.svg)](https://pypi.python.org/pypi/harwest)\n\nRun the following command in the terminal to install the package:\n```bash\n$ pip3 install harwest\n```\n\n\n## Getting Started\n\nAfter installing the package, run the following command in the terminal:\n```bash\n$ harwest\n```\n\nIn case you're using Harwest for the first time, you'd be greeted with a set of configuration steps\nthat you'll have to complete to set up the tool.\n\n- **Step [1]** requires you to select a directory name where all your code submissions will be stored. \n  The directory will be created under the same path from where you executed the command. \n  \n  In case\n  you'd like to set up the directory at some other location then press \\\u003cCtrl\\\u003e+\\\u003cC\\\u003e to exit from\n  the setup and execute the command again from your desired location.\n- **Step [2]** is straight-forward and asks you to enter your full-name and email address which will be \n  used for setting up the git repository. \n  \n  NOTE: For the contributions to show up in the contributions streak graph, the provided email \n  address must be the same as the email address associated with your GitHub/BitBucket account \n  \n- **Step [3]** though optional, takes away the effort of even pushing the changes to the Git repository\n  from you. To take advantage of this feature, create an **empty** git repository in [GitHub](https://github.com/new) \n  or BitBucket _(without any README, .gitignore or license)_ and copy and paste the git remote url\n  as input for this step. \n  \n  If you however don't want automated pushes for your repository then leave \n  the input as empty and press \\\u003center\\\u003e. You can always push the repository to remote manually.\n\n\n```bash\nnellex@HQ:~$ harwest\n\n      __  __                              __\n     / / / /___ _______      _____  _____/ /_\n    / /_/ / __ `/ ___/ | /| / / _ \\/ ___/ __/\n   / __  / /_/ / /   | |/ |/ /  __(__  ) /_\n  /_/ /_/\\__,_/_/    |__/|__/\\___/____/\\__/\n\n  ==========================================\n\nHey there! 👋 Looks like you're using Harwest for the first time. Let's get you started 🚀\n\n[1] We'll need to create a directory to store all your files\n    The directory will be created as /home/nellex/\u003cyour-input\u003e\n\u003e So, what would you like your directory to be called? accepted\n👍 Alright, so you're directory will be created at /home/nellex/accepted\n\n[2] Then let's build your author tag which will appear in your Git commits as:\n    Author: Steve Jobs \u003csteve.jobs@apple.com\u003e\n\u003e So what would your beautiful (Author) Full Name be? Nilesh Sah\n\u003e And of course, your magical (Author) Email Address? nilesh.sah13@gmail.com\n\n[3] Guess what? We can automate the Git pushes for you too! 🎉\n   In case you'd like that, then please specify the remote Git Url for an \"empty\" repository\n   It would be somewhat like https://github.com/nileshsah/harwest-tool.git\n   But it's optional, in case you'd like to skip then leave it empty and just hit \u003center\u003e\n\u003e (Optional) So, what would be the remote url for the repository again? https://github.com/nileshsah/accepted.git\n\n 🥳 You rock! We're all good to go now\n```\n\nOnce the initial set up is complete, you can then execute the command\n\n```bash\n$ harwest \u003cplatform\u003e\n$ harwest codeforces # example\n```\nto harvest your submissions from the Codeforces platform. If it's the first time you're running the \ncommand, you'll be prompted for providing your Codeforces handle name\n```bash\n\u003e So what's your prestigious Codeforces Handle Name? nellex\n```\n\nHarwest will then start scraping all your submissions, starting from page 1 till the very end.\n\n```bash\nnellex@HQ:~$ harwest codeforces\n\n      __  __                              __\n     / / / /___ _______      _____  _____/ /_\n    / /_/ / __ `/ ___/ | /| / / _ \\/ ___/ __/\n   / __  / /_/ / /   | |/ |/ /  __(__  ) /_\n  /_/ /_/\\__,_/_/    |__/|__/\\___/____/\\__/\n\n  ==========================================\n\n⛏ ️Harvesting Codeforces (nellex) Submissions to /home/nellex/accepted\n⌛  Currently scanning page #1: (24/24) Phoenix and Beauty https://codeforces.com/contest/1348/problem/B\nUsername for 'https://github.com': nileshsah\nPassword for 'https://nileshsah@github.com':\n👌 The updates were automatically pushed to the remote repository\n✅ The repository was successfully updated!\n```\n\nIn case scanning stops at any page due to some server side error, you can restart scraping from the\nfailed page by running the command:\n\n```bash\n$ harwest \u003cplatform\u003e --start-page N\n$ harwest codeforces --start-page 3 # example\n```\n\nor instead force Harwest to re-scan the entire submission list for the platform by running:\n ```bash\n$ harwest \u003cplatform\u003e --full-scan\n$ harwest atcoder --full-scan # example\n ```\n\n## Reconfigure\n\nHarwest settings can be reconfigured by running the following command which will then restart the\nentire configuration steps.\n\n```bash\n$ harwest --init\n```\nHarwest provides the ability to re-use an existing directory previously created by this tool for \nfurther updates.\n\nTo change the handle name for a specific platform, you can run:\n```bash\n$ harwest \u003cplatform\u003e --setup\n$ harwest codeforces --setup # example\n```\n\n\n## License\n\nMIT License\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnileshsah%2Fharwest-tool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnileshsah%2Fharwest-tool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnileshsah%2Fharwest-tool/lists"}