Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jdidion/xphyle
Python library that facilitates opening, reading, and writing files (and file-like entities like URLs and streams) agnostic of compression format. (production)
https://github.com/jdidion/xphyle
Last synced: 23 days ago
JSON representation
Python library that facilitates opening, reading, and writing files (and file-like entities like URLs and streams) agnostic of compression format. (production)
- Host: GitHub
- URL: https://github.com/jdidion/xphyle
- Owner: jdidion
- License: other
- Created: 2016-10-18T15:07:27.000Z (about 8 years ago)
- Default Branch: develop
- Last Pushed: 2024-07-30T01:55:23.000Z (5 months ago)
- Last Synced: 2024-10-02T20:18:00.184Z (2 months ago)
- Language: Python
- Homepage:
- Size: 2.38 MB
- Stars: 21
- Watchers: 4
- Forks: 3
- Open Issues: 15
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGES.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Citation: CITATION
Awesome Lists containing this project
- starred-awesome - xphyle - Python library that facilitates opening, reading, and writing files (and file-like entities like URLs and streams) agnostic of compression format. (production) (Python)
README
# xphyle: extraordinarily simple file handling
[](https://pypi.python.org/pypi/xphyle)
[](https://travis-ci.org/jdidion/xphyle)
[](https://coveralls.io/github/jdidion/xphyle?branch=master)
[](https://www.codacy.com/app/jdidion/xphyle?utm_source=github.com&utm_medium=referral&utm_content=jdidion/xphyle&utm_campaign=Badge_Grade)
[](http://xphyle.readthedocs.io/en/latest/?badge=latest)
[](https://zenodo.org/badge/latestdoi/71260678)
[](http://joss.theoj.org/papers/10.21105/joss.00255)
xphyle is a small python library that makes it easy to open compressed
files. Most importantly, xphyle will use the appropriate program (e.g. 'gzip') to compress/decompress a file if it is available on your system; this is almost always faster than using the corresponding python library. xphyle also provides methods that simplify common file I/O operations.
Recent version of xphyle (4.0.0+) require python 3.6. Older versions of xphyle support python 3.4+.
Please note that xphyle may work on Windows, but it is not tested.
# Installation
```
pip install xphyle
```
# Building from source
Clone this repository and run
```
make
```
# Example usages:
```python
from xphyle import *
from xphyle.paths import STDIN, STDOUT
# Open a compressed file...
myfile = xopen('infile.gz')
# ...or a compressed stream
# e.g. gzip -c afile | python my_program.py
stdin = xopen(STDIN)
# Easily write to the stdin of a subprocess
with open_('|cat', 'wt') as process:
process.write('foo')
# We have to tell xopen what kind of compression
# to use when writing to stdout
stdout = xopen(STDOUT, compression='gz')
# The `open_` method ensures that the file is usable with the `with` keyword.
# Print all lines in a compressed file...
with open_('infile.gz') as myfile:
for line in myfile:
print(line)
# ... or a compressed URL
with open_('http://foo.com/myfile.gz') as myfile:
for line in myfile:
print(line)
# Transparently handle paths and file objects
def dostuff(path_or_file):
with open_(path_or_file) as myfile:
for line in myfile:
print(line)
# Read all lines in a compressed file into a list
from xphyle.utils import read_lines
lines = list(read_lines('infile.gz'))
# Sum the rows in a compressed file where each line is an integer value
total = sum(read_lines('infile.gz', convert=int))
```
See the [Documentation](https://xphyle.readthedocs.io/en/latest/) for full usage information.
# Supported compression formats
* `gzip` (uses `igzip` or `pigz` if available)
* `bgzip`
* `bzip2` (uses `pbzip2` if available)
* `lzma`
* `zstd`
# Issues
Please report bugs and request enhancements using the [issue tracker](https://github.com/jdidion/xphyle).
# Roadmap
Future releases are mapped out using [GitHub Projects](https://github.com/jdidion/xphyle/projects).
# Citing xphyle
[Didion, JP (2017) xphyle: Extraordinarily simple file handling. Journal of Open Source Software; doi:10.21105/joss.00255](https://joss.theoj.org/papers/10.21105/joss.00255#)
# Acknowledgements
* [Dependencies scanned by PyUp.io](http://pyup.io/)
* Thanks to [@ctb](https://github.com/ctb) for reviewing the xphyle paper