https://github.com/girder/large_image_wheels
manylinux wheel files for girder/large_image dependencies
https://github.com/girder/large_image_wheels
binary gdal glymur libtiff libvips mapnik openslide openslide-python pylibtiff pyproj python-javabridge pyvips wheels
Last synced: 2 days ago
JSON representation
manylinux wheel files for girder/large_image dependencies
- Host: GitHub
- URL: https://github.com/girder/large_image_wheels
- Owner: girder
- License: apache-2.0
- Created: 2018-10-31T14:15:22.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2026-01-27T15:28:13.000Z (3 days ago)
- Last Synced: 2026-01-28T00:55:01.349Z (3 days ago)
- Topics: binary, gdal, glymur, libtiff, libvips, mapnik, openslide, openslide-python, pylibtiff, pyproj, python-javabridge, pyvips, wheels
- Language: Dockerfile
- Homepage: https://girder.github.io/large_image_wheels
- Size: 10.4 GB
- Stars: 22
- Watchers: 1
- Forks: 4
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# large_image_wheels
manylinux wheel files for girder/large_image dependencies.
## Use
You can install from the wheels in this repository using a command like:
```
pip install pylibtiff openslide_python pyvips gdal mapnik glymur javabridge -f https://girder.github.io/large_image_wheels
```
## Building
The wheels are all generated with the accompanying Dockerfile.
Building the wheels:
```
docker build --force-rm -t girder/large_image_wheels .
```
To extract the wheel files from the docker image:
```
mkdir -p wheels
docker run -v wheels:/opt/mount --rm --entrypoint bash girder/large_image_wheels -c 'cp /io/wheelhouse/*many* /opt/mount/.'
```
This will use the last recorded versions (stored in the `versions.txt` file). To update the versions that are used, you can do (before the docker build):
```
python3 -u check_versions.py > versions.txt
```
## Results
This makes wheels for the main libraries:
- GDAL
- Glymur
- mapnik
- openslide_python
- pylibmc
- pylibtiff
- python_javabridge
- pyvips
Currently, wheels are built for Python 3.9, 3.10, 3.11, 3.12, 3.13, 3.14 and for architectures x86_64 and aarch64. Some wheels are also being built for 3.13t and 3.14t. Some libraries have older versions available for older versions of Python that were built before support for those versions was ended. There are also some libraries from before their projects published binary wheels (psutil, pyproj, ujson).
This also builds some non-version specific libraries to ensure they have recent dependencies:
- bioformats
In general version numbers have some additional component to the base library (e.g., an extra .1 added to the end) to ensure that they do not have conflicts with exact version numbers on pypi.
## Extras
Various related executables are bundled with the Python packages. These are added as package data in a `bin` directory within the main package. There is a python wrapper script exposing these in the Python binary path. For instance, `gdalinfo` is available from the `GDAL` package. It is located in the Python site-packages `osgeo/bin` directory. To access it directly (rather than through the wrapper script), the appropriate directory can be gleaned from Python as part of a bash command, e.g., `` `python -c 'import os,sys,osgeo;sys.stdout.write(os.path.dirname(osgeo.__file__))'`/bin/gdalinfo --version``.
## Issues
In order to find the built libraries, this modifies how pylibtiff, openslide_python, and pyvips load those libraries. The other libraries are patched in place. There is probably a better way to do this.
There are some differences in features exposed on different architectures; this is largely based on the source libraries and not on how they are built.
## Example Use
This makes is more convenient to use large_image. For instance, you can create a Jupyter Notebook with large_image.
```
docker run --rm -p 8888:8888 jupyter/minimal-notebook bash -c 'pip install git+git://github.com/girder/large_image.git#egg=large_image[openslide,mapnik] -f https://girder.github.io/large_image_wheels matplotlib && start.sh jupyter notebook --NotebookApp.token="" --ip=0.0.0.0'
```
In the Jupyter interface, create a new notebook. In the first cell run:
```
import large_image
import requests
import matplotlib.pyplot as plt
wsi_url = 'https://data.kitware.com/api/v1/file/5899dd6d8d777f07219fcb23/download'
wsi_path = 'TCGA-02-0010-01Z-00-DX4.07de2e55-a8fe-40ee-9e98-bcb78050b9f7.svs'
open(wsi_path, 'wb').write(requests.get(wsi_url, allow_redirects=True).content)
ts = large_image.getTileSource(wsi_path)
tile_info = ts.getSingleTile(
tile_size=dict(width=1000, height=1000),
scale=dict(magnification=20),
tile_position=1000
)
```
And in the second cell run:
```
plt.imshow(tile_info['tile'])
```
## Rationale
It is an inconvenience to have to add `--find-links https://girder.github.io/large_image_wheels` to pip install commands to use these wheels. There are alternatives: (a) convince upstream repositories to publish wheels, or (b) publish these under unique names (e.g., large_image_dependency_gdal). None of these are perfect solutions.
Using `--find-links` requires modifying pip install commands. Further, if a newer non-wheel version of a package is published on pypi, it will be used in favor of the wheel unless the explicit version is used. Since this repository won't maintain old versions, this means that the wheels must be rebuilt as soon as new versions are released.
It isn't practical to produce wheels for some upstream packages. For instance, GDAL can be compiled with optional components that require licensing. A publicly published wheel can't contain such components, and, therefore, it makes sense for the package on pypi to expect that the library has been installed separately from the python module.
If these wheels were published under alternate names, they could be published to pypi. However, this would require a wheel for every supported OS or have conditionals in downstream packages' setup.py install_requires. Further, it would preclude using custom-built libraries (such as GDAL with licensed additions).
Using `--find-links` seems like the best choice of these options. Downstream packages can list the expected modules in `install_requires`. Installation doesn't become any harder for platforms without wheels, and custom-built libraries are supported.
Newer versions usually have an extra `.1` added to the original python package's version number to ensure that pip and other installers will resolve to our published wheel uniquely.
## Special use cases
### GeoDjango
GeoDjango expects libgdal and libgeos_c to be in standard locations. These libraries can be installed by installing the GDAL wheel. Starting with GDAL 3.1.2, the `osgeo` module from these wheels exposes two values that can be used [to tell Django where these libraries are located](https://docs.djangoproject.com/en/5.2/ref/contrib/gis/install/geolibs/). Specifically, Django can be configured using these values:
```
import osgeo
django.conf.settings.configure()
django.conf.settings.GDAL_LIBRARY_PATH = osgeo.GDAL_LIBRARY_PATH
django.conf.settings.GEOS_LIBRARY_PATH = osgeo.GEOS_LIBRARY_PATH
```
Inside a Django application's `settings.py` file, this is somewhat simpler::
```
import osgeo
GDAL_LIBRARY_PATH = osgeo.GDAL_LIBRARY_PATH
GEOS_LIBRARY_PATH = osgeo.GEOS_LIBRARY_PATH
```
## Future Work
- More optional libraries
Several packages (GDAL, ImageMagick, vips, and probably others) could have additional libraries added to the built wheels. As time permits, those libraries that are appropriately licensed may be gradually added. See the Dockerfile for comments on what else could be built. If there is a specific need for a library that hasn't yet been included, please create an issue for it.
- More executables
A variety of executables are bundled with the wheels in appropriate `bin` directories. There are additional tools that could be added. If there is a specific need for an executable that hasn't been included, please create an issue for it.
- Fully automate releases
The wheels should be published from a successful CI run rather than from a user commit.