Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/cthoyt/zenodo-client
A tool for automated uploading and version management of scientific data to Zenodo
https://github.com/cthoyt/zenodo-client
Last synced: 18 days ago
JSON representation
A tool for automated uploading and version management of scientific data to Zenodo
- Host: GitHub
- URL: https://github.com/cthoyt/zenodo-client
- Owner: cthoyt
- License: mit
- Created: 2021-03-01T18:15:14.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-10-09T15:12:57.000Z (about 1 month ago)
- Last Synced: 2024-10-14T02:23:37.793Z (about 1 month ago)
- Language: Python
- Homepage:
- Size: 98.6 KB
- Stars: 25
- Watchers: 3
- Forks: 5
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- Contributing: .github/CONTRIBUTING.md
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
Zenodo Client
A wrapper for the Zenodo API.
## 💪 Getting Started
The first example shows how you can set some configuration then never worry about whether it's been
uploaded already or not - all baked in with [`pystow`](https://github.com/cthoyt/pystow). On the
first time this script is run, the new deposition is made, published, and the identifier is stored
with the given key in your `~/.config/zenodo.ini`. Next time it's run, the deposition will be looked
up, and the data will be uploaded. Versioning is given automatically by date, and if multiple
versions are uploaded on one day, then a dash and the revision are appended.
```python
from zenodo_client import Creator, Metadata, ensure_zenodo
# Define the metadata that will be used on initial upload
data = Metadata(
title='Test Upload 3',
upload_type='dataset',
description='test description',
creators=[
Creator(
name='Hoyt, Charles Tapley',
affiliation='Harvard Medical School',
orcid='0000-0003-4423-4370',
),
],
)
res = ensure_zenodo(
key='test3', # this is a unique key you pick that will be used to store
# the numeric deposition ID on your local system's cache
data=data,
paths=[
'/Users/cthoyt/Desktop/test1.png',
],
sandbox=True, # remove this when you're ready to upload to real Zenodo
)
from pprint import pprint
pprint(res.json())
```
A real-world example can be found here: https://github.com/cthoyt/nsockg.
The following example shows how to use the Zenodo uploader if you already know what your deposition
identifier is.
```python
from zenodo_client import update_zenodo
# The ID from your deposition
SANDBOX_DEP_ID = '724868'
# Paths to local files. Good to use in combination with resources that are always
# dumped to the same place by a given script
paths = [
# os.path.join(DATABASE_DIRECTORY, 'alts_sample.tsv')
'/Users/cthoyt/Desktop/alts_sample.tsv',
]
# Don't forget to set the ZENODO_API_TOKEN environment variable or
# any valid way to get zenodo/api_token from PyStow.
update_zenodo(SANDBOX_DEP_ID, paths)
```
The following example shows how to look up the latest version of a record.
```python
from zenodo_client import Zenodo
zenodo = Zenodo()
OOH_NA_NA_RECORD = '4020486'
new_record = zenodo.get_latest_record(OOH_NA_NA_RECORD)
```
Even further, the latest version of `names.tsv.gz` can be automatically downloaded to the
`~/.data/zenodo///` via `pystow` with:
```python
from zenodo_client import Zenodo
zenodo = Zenodo()
OOH_NA_NA_RECORD = '4020486'
new_record = zenodo.download_latest(OOH_NA_NA_RECORD, 'names.tsv.gz')
```
A real-world example can be found [here](https://github.com/pyobo/pyobo/blob/master/src/pyobo/resource_utils.py)
where the latest build of the [Ooh Na Na](https://cthoyt.com/2020/04/18/ooh-na-na.html) nomenclature
database is automatically downloaded from Zenodo, even though the PyOBO package only hardcodes the
first deposition ID.
### Command Line Interface
The zenodo_client command line tool is automatically installed. It can be used from the shell with
the `--help` flag to show all subcommands:
```shell
$ zenodo_client --help
```
It can be run with `zenodo_client ... `
## 🚀 Installation
The most recent release can be installed from
[PyPI](https://pypi.org/project/zenodo_client/) with:
```shell
$ pip install zenodo_client
```
The most recent code and data can be installed directly from GitHub with:
```bash
$ pip install git+https://github.com/cthoyt/zenodo-client.git
```
## 👐 Contributing
Contributions, whether filing an issue, making a pull request, or forking, are appreciated. See
[CONTRIBUTING.md](https://github.com/cthoyt/zenodo-client/blob/master/.github/CONTRIBUTING.md) for more information on getting involved.
## 👋 Attribution
### ⚖️ License
The code in this package is licensed under the MIT License.
### 🍪 Cookiecutter
This package was created with [@audreyfeldroy](https://github.com/audreyfeldroy)'s
[cookiecutter](https://github.com/cookiecutter/cookiecutter) package using [@cthoyt](https://github.com/cthoyt)'s
[cookiecutter-snekpack](https://github.com/cthoyt/cookiecutter-snekpack) template.
## 🛠️ For Developers
See developer instructions
The final section of the README is for if you want to get involved by making a code contribution.
### Development Installation
To install in development mode, use the following:
```bash
$ git clone git+https://github.com/cthoyt/zenodo-client.git
$ cd zenodo-client
$ pip install -e .
```
### 🥼 Testing
After cloning the repository and installing `tox` with `pip install tox`, the unit tests in the `tests/` folder can be
run reproducibly with:
```shell
$ tox
```
Additionally, these tests are automatically re-run with each commit in a [GitHub Action](https://github.com/cthoyt/zenodo-client/actions?query=workflow%3ATests).
### 📖 Building the Documentation
The documentation can be built locally using the following:
```shell
$ git clone git+https://github.com/cthoyt/zenodo-client.git
$ cd zenodo-client
$ tox -e docs
$ open docs/build/html/index.html
```
The documentation automatically installs the package as well as the `docs`
extra specified in the [`setup.cfg`](setup.cfg). `sphinx` plugins
like `texext` can be added there. Additionally, they need to be added to the
`extensions` list in [`docs/source/conf.py`](docs/source/conf.py).
### 📦 Making a Release
After installing the package in development mode and installing
`tox` with `pip install tox`, the commands for making a new release are contained within the `finish` environment
in `tox.ini`. Run the following from the shell:
```shell
$ tox -e finish
```
This script does the following:
1. Uses [Bump2Version](https://github.com/c4urself/bump2version) to switch the version number in the `setup.cfg`,
`src/zenodo_client/version.py`, and [`docs/source/conf.py`](docs/source/conf.py) to not have the `-dev` suffix
2. Packages the code in both a tar archive and a wheel using [`build`](https://github.com/pypa/build)
3. Uploads to PyPI using [`twine`](https://github.com/pypa/twine). Be sure to have a `.pypirc` file configured to avoid the need for manual input at this
step
4. Push to GitHub. You'll need to make a release going with the commit where the version was bumped.
5. Bump the version to the next patch. If you made big changes and want to bump the version by minor, you can
use `tox -e bumpversion -- minor` after.