{"id":19911297,"url":"https://github.com/adtech-labs/conda-mirror","last_synced_at":"2025-07-16T21:11:58.170Z","repository":{"id":78005288,"uuid":"71492747","full_name":"adtech-labs/conda-mirror","owner":"adtech-labs","description":"Mirror upstream conda channels","archived":false,"fork":false,"pushed_at":"2019-06-26T21:00:08.000Z","size":349,"stargazers_count":72,"open_issues_count":20,"forks_count":62,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-04-07T10:51:37.180Z","etag":null,"topics":["conda","conda-mirror","mirroring","python","repodata","team-platform"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adtech-labs.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-10-20T18:35:17.000Z","updated_at":"2024-10-17T08:42:33.000Z","dependencies_parsed_at":"2023-03-18T22:48:17.557Z","dependency_job_id":null,"html_url":"https://github.com/adtech-labs/conda-mirror","commit_stats":null,"previous_names":["adtech-labs/conda-mirror","vericast/conda-mirror"],"tags_count":35,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adtech-labs%2Fconda-mirror","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adtech-labs%2Fconda-mirror/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adtech-labs%2Fconda-mirror/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adtech-labs%2Fconda-mirror/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adtech-labs","download_url":"https://codeload.github.com/adtech-labs/conda-mirror/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252137539,"owners_count":21700235,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["conda","conda-mirror","mirroring","python","repodata","team-platform"],"created_at":"2024-11-12T21:24:41.708Z","updated_at":"2025-05-03T03:31:10.333Z","avatar_url":"https://github.com/adtech-labs.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# conda-mirror\n\n[![Project Status: Inactive – The project has reached a stable, usable state but is no longer being actively developed; support/maintenance will be provided as time allows.](https://www.repostatus.org/badges/latest/inactive.svg)](https://www.repostatus.org/#inactive)\n[![Build Status](https://travis-ci.org/Valassis-Digital-Media/conda-mirror.svg?branch=master)](https://travis-ci.org/Valassis-Digital-Media/conda-mirror)\n[![PyPI version](https://badge.fury.io/py/conda-mirror.svg)](https://badge.fury.io/py/conda-mirror)\n[![codecov](https://codecov.io/gh/Valassis-Digital-Media/conda-mirror/branch/master/graph/badge.svg)](https://codecov.io/gh/Valassis-Digital-Media/conda-mirror)\n\nMirrors an upstream conda channel to a local directory.\n\n## Install\n\n`conda-mirror` is available on PyPI and conda-forge.\n\nInstall with:\n\n`pip install conda-mirror`\n\nor:\n\n`conda install conda-mirror -c conda-forge`\n\n## Compatibility\n\n`conda-mirror` is intentionally a py3 only package\n\n## CLI\n\nCLI interface for `conda-mirror.py`\n\n```\nusage: conda-mirror [-h] [--upstream-channel UPSTREAM_CHANNEL]\n                    [--target-directory TARGET_DIRECTORY]\n                    [--temp-directory TEMP_DIRECTORY] [--platform PLATFORM]\n                    [-v] [--config CONFIG] [--pdb] [--num-threads NUM_THREADS]\n                    [--version] [--dry-run] [--no-validate-target]\n                    [--minimum-free-space MINIMUM_FREE_SPACE]\n\nCLI interface for conda-mirror.py\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --upstream-channel UPSTREAM_CHANNEL\n                        The target channel to mirror. Can be a channel on\n                        anaconda.org like \"conda-forge\" or a full qualified\n                        channel like \"https://repo.continuum.io/pkgs/free/\"\n  --target-directory TARGET_DIRECTORY\n                        The place where packages should be mirrored to\n  --temp-directory TEMP_DIRECTORY\n                        Temporary download location for the packages. Defaults\n                        to a randomly selected temporary directory. Note that\n                        you might need to specify a different location if your\n                        default temp directory has less available space than\n                        your mirroring target\n  --platform PLATFORM   The OS platform(s) to mirror. one of: {'linux-64',\n                        'linux-32','osx-64', 'win-32', 'win-64'}\n  -v, --verbose         logging defaults to error/exception only. Takes up to\n                        three '-v' flags. '-v': warning. '-vv': info. '-vvv':\n                        debug.\n  --config CONFIG       Path to the yaml config file\n  --pdb                 Enable PDB debugging on exception\n  --num-threads NUM_THREADS\n                        Num of threads for validation. 1: Serial mode. 0: All\n                        available.\n  --version             Print version and quit\n  --dry-run             Show what will be downloaded and what will be removed.\n                        Will not validate existing packages\n  --no-validate-target  Skip validation of files already present in target-\n                        directory\n  --minimum-free-space MINIMUM_FREE_SPACE\n                        Threshold for free diskspace. Given in megabytes.\n```\n\n## Example Usage\n\nWARNING: Invoking this command will pull ~10GB and take at least an hour\n\n`conda-mirror --upstream-channel conda-forge --target-directory local_mirror --platform linux-64`\n\n## More Details\n\n### blacklist/whitelist configuration\n\nexample-conf.yaml:\n\n```yaml\nblacklist:\n    - license: \"*agpl*\"\n    - license: None\n    - license: \"\"\n\nwhitelist:\n    - name: system\n```\n\n`blacklist` removes package(s) that match the condition(s) listed from the\nupstream repodata.\n\n`whitelist` re-includes any package(s) from blacklist that match the\nwhitelist conditions.\n\nblacklist and whitelist both take lists of dictionaries. The keys in the\ndictionary need to be values in the `repodata.json` metadata. The values are\n(unix) globs to match on. Go here for the full repodata of the upstream\n\"defaults\" channel:\nhttp://conda.anaconda.org/anaconda/linux-64/repodata.json\n\nHere are the contents of one of the entries in repodata['packages']\n\n```python\n{'botocore-1.4.10-py34_0.tar.bz2': {'arch': 'x86_64',\n  'binstar': {'channel': 'main',\n   'owner_id': '55fc8527d3234d09d4951c71',\n   'package_id': '56b88ea1be1cc95a362b218e'},\n  'build': 'py34_0',\n  'build_number': 0,\n  'date': '2016-04-11',\n  'depends': ['docutils \u003e=0.10',\n   'jmespath \u003e=0.7.1,\u003c1.0.0',\n   'python 3.4*',\n   'python-dateutil \u003e=2.1,\u003c3.0.0'],\n  'license': 'Apache',\n  'md5': 'b35a5c1240ba672e0d9d1296141e383c',\n  'name': 'botocore',\n  'platform': 'linux',\n  'requires': [],\n  'size': 1831799,\n  'version': '1.4.10'}}\n```\n\nSee implementation details in the `conda_mirror:match` function for more\ninformation.\n\n#### Common usage patterns\n##### Mirror **only** one specific package\nIf you wanted to match exactly the botocore package listed above with your\nconfig, then you could use the following configuration to first blacklist\n**all** packages and then include just the botocore packages:\n\n```yaml\nblacklist:\n    - name: \"*\"\nwhitelist:\n    - name: botocore\n      version: 1.4.10\n      build: py34_0\n```\n##### Mirror everything but agpl licenses\n```yaml\nblacklist:\n    - license: \"*agpl*\"\n```\n\n##### Mirror only python 3 packages\n```yaml\nblacklist:\n    - name: \"*\"\nwhitelist:\n    - build: \"*py3*\"\n```\n\n## Testing\n\n### Install test requirements\n\nNote: Will install packages from pip\n\n```\n$ pip install -r test-requirements.txt\nRequirement already satisfied: pytest in /home/edill/miniconda/lib/python3.5/site-packages (from -r test-requirements.txt (line 1))\nRequirement already satisfied: coverage in /home/edill/miniconda/lib/python3.5/site-packages (from -r test-requirements.txt (line 2))\nRequirement already satisfied: pytest-ordering in /home/edill/miniconda/lib/python3.5/site-packages (from -r test-requirements.txt (line 3))\nRequirement already satisfied: py\u003e=1.4.29 in /home/edill/miniconda/lib/python3.5/site-packages (from pytest-\u003e-r test-requirements.txt (line 1))\n```\n\n### Run the tests, invoking with the `coverage` tool.\n\n```\n$ coverage run run_tests.py\nsys.argv=['run_tests.py']\n========================================= test session starts ==========================================\nplatform linux -- Python 3.5.3, pytest-3.0.6, py-1.4.31, pluggy-0.4.0 -- /home/edill/miniconda/bin/python\ncachedir: .cache\nrootdir: /home/edill/dev/maxpoint/github/conda-mirror, inifile:\nplugins: xonsh-0.5.2, ordering-0.4\ncollected 4 items\n\ntest/test_conda_mirror.py::test_match PASSED\ntest/test_conda_mirror.py::test_cli[https://repo.continuum.io/pkgs/free-linux-64] PASSED\ntest/test_conda_mirror.py::test_cli[conda-forge-linux-64] PASSED\ntest/test_conda_mirror.py::test_handling_bad_package PASSED\n\n======================================= 4 passed in 4.41 seconds =======================================\n```\n\n### Show the coverage statistics\n\n```\n$ coverage report -m\nName                           Stmts   Miss  Cover   Missing\n------------------------------------------------------------\nconda_mirror/__init__.py           3      0   100%\nconda_mirror/conda_mirror.py     236     20    92%   203-205, 209-210, 214, 240, 249-254, 262-264, 303, 366, 497, 542-543, 629\n------------------------------------------------------------\nTOTAL                            239     20    92%\n```\n\n## Other\n\nAfter a new contributor makes a pull-request that is approved, we will reach out\nand invite you to be a maintainer of the project.\n\n\n## Releasing\n\nTo release you need three things\n\n1. Commit rights to conda-mirror\n2. A github token\n3. The version number that you want to use for the new tag\n\nAfter you have all three of these things, run the release.sh script (on a unix machine) and\npass it the tag that you want to use and your github token:\n\n```bash\nGITHUB_TOKEN=\u003cgithub_token\u003e ./release.sh \u003ctag\u003e\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadtech-labs%2Fconda-mirror","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadtech-labs%2Fconda-mirror","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadtech-labs%2Fconda-mirror/lists"}