{"id":15763498,"url":"https://github.com/casperdcl/gitcrawl","last_synced_at":"2026-01-08T20:09:55.139Z","repository":{"id":92567301,"uuid":"379981654","full_name":"casperdcl/gitcrawl","owner":"casperdcl","description":"Parses a python repository and creates a nice env.yml for conda. Aims to ease the headache that comes with transferring code and conda environments.","archived":false,"fork":false,"pushed_at":"2021-06-24T16:07:36.000Z","size":437,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2024-10-11T12:01:23.202Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"nsultova/gitcrawl","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/casperdcl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-24T16:05:45.000Z","updated_at":"2021-06-24T16:05:47.000Z","dependencies_parsed_at":null,"dependency_job_id":"599aff27-6243-45f6-b4c6-a544fcc6ca60","html_url":"https://github.com/casperdcl/gitcrawl","commit_stats":{"total_commits":18,"total_committers":3,"mean_commits":6.0,"dds":0.2222222222222222,"last_synced_commit":"e23c3644d20945152029aafbd66284c50c99cb75"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casperdcl%2Fgitcrawl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casperdcl%2Fgitcrawl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casperdcl%2Fgitcrawl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casperdcl%2Fgitcrawl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/casperdcl","download_url":"https://codeload.github.com/casperdcl/gitcrawl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246450478,"owners_count":20779421,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-04T12:00:23.413Z","updated_at":"2026-01-08T20:09:55.099Z","avatar_url":"https://github.com/casperdcl.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Gitcrawl \n* exports required modules from python code \n* bakes you a nice environments.yml for your conda\n* aims to ease the headache that comes with transferring conda environments\n* allows you to create a neat environments.yml for any python project you feed it\n\n[Docs](https://nsultova.github.io/gitcrawl/)\n\n## Motivation\nCurrently there is no way known to me to transfer conda environments in a non-cumbersome way across operating systems. The current commands as stated in the official docs are:\n\n```\nconda env export \u003e environment.yml \n\nconda env export --no-builds \u003e environment.yml \n\nconda env export --from-history.yml\n\n```\n\nExample:\n\nWithin your environment, call `conda install numpy=1.17.5`\n\n`conda env export` produces a .yml file including version numbers, local dependencies + hashes: \n```\n- numpy=1.19.5=py38h6ced74f_1\n- olefile=0.46=pyh9f0ad1d_1\n...\n```\n\n`conda env export --no-builds` produces a .yml file including version numbers and local dependencies: \n```\n- numpy=1.19.5\n- olefile=0.46\n...\n```\n\n`conda env export --from-history` produces a .yml file with no version information included, Also it includes only packages which have ben manually installed into the environment. \n\n```\n- numpy\n- pandas\n...\n```\n\nIf you want to get a better grasp of the commands [here](https://github.com/nsultova/conda_reproduce_testcase) is a small test-case.\n\n\nAbove behavior makes transferring conda environments across systems often quite challenging. More often than not, you'll end up writing the entire environment.yml by hand. \n\n## Gitcrawl\n\nI wrote gitcrawl out of the desire to make life a bit easier for my colleagues and me, as we often exchange/test/experiment with various code. \n\nGitcrawl offers another handy feature: You can clone any python repository and it will create a environment.yml for you, thus saving you a lot of time and helping you keep your system neat and tidy when you just simply want to test some code. \n(Especially nice, if not even a requirements.txt is provided)\n\n### Note:\n* This tool is a workaround  and PoC, keep this in mind\n* Adjusting the resulting environment.yml is sometimes unavoidable\n\n\n## Install: \n\n\n**Way 0:**\n\n`pip install gitcrawl`\n\n**Way 1**\nThe manual way (if you want to tinker around):\n\nClone the repository:\n\n`git clone https://github.com/nsultova/gitcrawl.git`\n\nUse the provided gitcrawl-env.yml to create a conda environment:\n\n`conda env create -f gitcrawl-env.yml`\n\nActivate the new environment: \n`conda activate gitcrawl-env`\n\nVerify that the new environment was installed correctly:\n`conda env list`\n\n(If you prefer to use pip you should be able to extract what you need from the environments.yml)\n\n## Usage:\n\n**Way 0**\n\n`gitcrawl -s ../repo/to/be/parsed`\n\n**Way 1**\n\n`cd \u003cpath/to/gitcrawl\u003e`\n\n`python3 -m gitcrawl.gitcrawl -s ../repo/to/be/parsed`\n\n\nThe code is designed in a heavily interactive way. If you're lazy you can set the `--leave-me-alone` flag. This will make gitcrawl run trough all channels and ask you only if there are several installation candidates to choose from.\n\n*In either ways `--help` and the docs are your friends. \u003c3* [Have a look](https://nsultova.github.io/gitcrawl/)\n\n\n## Workflow\n\n\u003cimg src=\"https://raw.githubusercontent.com/nsultova/gitcrawl/master/imgs/workflow.png\" width=\"50%\"\u003e\n\n## Notes\n\nThis code relies on:\n* [findimports](https://github.com/mgedmin/findimports)\n* [requirements-parser](https://github.com/davidfischer/requirements-parser)\n* [pip_search](https://pypi.org/project/pip-search/#files) ..as for now pip-search is \n\nMake sure you have them installed!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcasperdcl%2Fgitcrawl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcasperdcl%2Fgitcrawl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcasperdcl%2Fgitcrawl/lists"}