https://github.com/edinburgh-genome-foundry/flametree
:fire: Python file and zip operations made easy
https://github.com/edinburgh-genome-foundry/flametree
archive file filesystem python zip
Last synced: 7 months ago
JSON representation
:fire: Python file and zip operations made easy
- Host: GitHub
- URL: https://github.com/edinburgh-genome-foundry/flametree
- Owner: Edinburgh-Genome-Foundry
- License: mit
- Created: 2017-02-03T19:50:06.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2022-05-04T13:33:09.000Z (over 3 years ago)
- Last Synced: 2025-03-28T23:07:04.630Z (7 months ago)
- Topics: archive, file, filesystem, python, zip
- Language: Python
- Homepage:
- Size: 120 KB
- Stars: 149
- Watchers: 10
- Forks: 12
- Open Issues: 5
-
Metadata Files:
- Readme: README.rst
Awesome Lists containing this project
README
.. raw:: html
![]()
Python file operations made easy
.. image:: https://github.com/Edinburgh-Genome-Foundry/Flametree/actions/workflows/build.yml/badge.svg
:target: https://github.com/Edinburgh-Genome-Foundry/Flametree/actions/workflows/build.yml
:alt: GitHub CI build status.. image:: https://coveralls.io/repos/github/Edinburgh-Genome-Foundry/Flametree/badge.svg
:target: https://coveralls.io/github/Edinburgh-Genome-Foundry/FlametreeFlametree is a Python library which provides a simple syntax for handling files and folders
(no ``os.path.join``, ``os.listdir`` etc.), and works the same way for different file systems.Write a Flametree program to read/write files in disk folders, and your code will also be
able to read/write in zip archives and virtual (in-memory) archives - which is particularly
useful on web servers.As an illustration, here is how to use Flametree to read a file ``texts/poems/the_raven.txt``, replace all
occurences of the word "raven" by "seagull" in the text, and write the result to a new
file ``the_seagull.txt`` in the same folder:.. code:: python
from flametree import file_tree
with file_tree("texts") as root:
poem_text = root.poems.the_raven_txt.read()
new_text = poem_text.replace("raven", "seagull")
root.poems._file("the_seagull.txt").write(new_text)Even in this very simple use case, the syntax is clearer than the ``os`` way,
which would write as follows:.. code:: python
import os
with open(os.path.join("poems", "the_raven.txt"), "r") as f:
poem_text = f.read()
new_text = poem_text.replace("raven", "seagull")
with open(os.path.join("poems", "the_seagull.txt"), "w") as f:
content = f.write(new_text)Moreover, the same Flametree code also works for files inside a zip archive:
.. code:: python
with file_tree("my_archive.zip") as root:
poem_text = root.poems.the_raven_txt.read()
new_text = poem_text.replace("raven", "seagull")
root.poems._file("the_seagull.txt").write(new_text)Now in hard mode: suppose that your server receives binary zip data of an
archive containing ``poems/the_raven.txt``, and must return back a new zip
containing a file ``poems/the_seagull.txt``. Here again, the syntax of the core
operations is the same:.. code:: python
destination_zip = file_tree("@memory") # Create a new virtual zip
with file_tree(the_raven_zip_data) as root:
poem_text = root.poems.the_raven_txt.read()
new_text = poem_text.replace("raven", "seagull")
destination_zip._dir("poems")._file("the_seagull.txt").write(new_text)
destination_zip_data = destination_zip._close()
# Now send the data to the clientSee section *Usage* below for more examples and features.
Installation
------------Flametree should work on Windows/Max/Linux, with Python 2 and 3, and has no external dependency.
It can be installed by unzipping the source code in one directory and using this command: ::
python setup.py install
You can also install it directly from the Python Package Index with this command: ::
pip install flametree
Contribute
----------Flametree is an open-source software originally written by Zulko_ and released on Github_
under the MIT licence (Copyright 2017 Edinburgh Genome Foundry, University of Edinburgh).
Everyone is welcome to contribute!
In particular if you have ideas of new kinds of file systems to add to Flametree.Usage
-----Opening a file tree
~~~~~~~~~~~~~~~~~~~Here is how you open different kinds of file systems:
.. code:: python
from flametree import file_tree
# Open a directory from the disk's file system:
root = file_tree("my_folder/")# Open a zip archive on the disk:
root = file_tree("my_archive.zip")# Connect to a file-like object (file handle, StringIO...) of a zip:
root = file_tree(file_like_object)# Create a virtual 'in-memory' zip file:
root = file_tree("@memory")# Open some data string representing a zip to read
root = file_tree(some_big_zip_data_string)In the two first examples, if ``my_folder`` or ``my_archive.zip`` do not exist, they
will be automatically created. If they do exist, it is possible to completely overwrite
them with the option ``replace=True``.Exploring a file tree:
~~~~~~~~~~~~~~~~~~~~~~Once you have created the ``root`` element with one of the methods above, you can display the whole
file tree with ``root._tree_view()`` :.. code::
>>> print (root._tree_view())
texts/
poems/
dover_beach.txt
the_raven.txt
the_tyger.txt
todo_list.txt
figures/
figure1.png
figure2.png
Readme.mdThe attributes of a directory like ``root`` are its files and subdirectories.
For instance to print the content of ``dover_beach.txt`` you would write:.. code:: python
print( root.texts.poems.dover_beach_txt.read() )
or even simpler:
.. code:: python
root.texts.poems.dover_beach_txt.print_content()
Notice that the ``.`` before ``txt`` was replaced by ``_`` so as to form a valid
attribute name.This syntactic sugar is particularly useful to explore a file tree in
IPython Notebooks or other editors offering auto-completion:.. image:: https://raw.githubusercontent.com/Edinburgh-Genome-Foundry/Flametree/master/images/autocomplete.png
:alt: [illustration]
:align: centerAlternatively, you can access files and directories using dictionary calls:
.. code:: python
root["texts"]["poems"]["dover_beach.txt"]
To iterate through the subdirectories of a directory, use the ``_dirs`` attribute:
.. code:: python
for subdirectory in root._dirs:
print (subdirectory._name) # Will print 'texts' and 'figures'To iterate through the files of a directory, use the ``_files`` attribute:
.. code:: python
for f in root.figures._files:
print (f._name) # Will print 'figure1.png' and 'figure2.png'Finally, use ``_all_files`` to iterate through all files nested in a directory.
The snippet below prints the content of all ``.txt`` files in the file tree:.. code:: python
for f in root._all_files:
if f._name.endswith(".txt"):
f.print_content()Creating files and folders
~~~~~~~~~~~~~~~~~~~~~~~~~~To create a new subdirectory use ``_dir``:
.. code:: python
root._dir("data") # create a 'data' folder at the root
root.data._dir("reports") # create a 'reports' folder under `root/data`To create a new file use ``_file``:
.. code:: python
root._file("joke.txt") # create a 'joke.txt' file at the root.
root.texts._file("hello.txt") # create 'hello.txt' in `root/texts`.To write content in a file, use ``.write``:
.. code:: python
root.joke_txt.write("A plateau is the highest form of flattery.")
Writing to a file will use mode ``a`` (append) by default. To overwrite
the file set the write mode to ``"w"``. Let's erase and rewrite that ``joke.txt``:.. code:: python
root.joke_txt.write("'DNA' stands for National Dyslexic Association.", "w")
File and directory creation commands can be chained.
Let us create some new folders ``data/`` and ``data/test_1/``, and
write to file ``data/test_1/values.csv``, all in a single line:.. code:: python
root._dir("data")._dir("test_1")._file("values.csv").write("1,15,25")
Beware that ``._dir`` and ``._file`` **overwrite their target by default**, which means that if you write:
.. code:: python
root._dir("data")._file("values_1.csv").write("1,4,7")
root._dir("data")._file("values_2.csv").write("2,9,7")The directory ``data`` will only contain ``values_2.csv``, because the second
line's ``_dir("data")`` erases the ``data`` directory and starts a new one. To avoid this,
either use ``root.data`` in the second line:.. code:: python
root._dir("data")._file("values_1.csv").write("1,4,7")
root.data._file("values_2.csv").write("2,9,7")Or use ``replace=False`` in ``_dir``:
.. code:: python
root._dir("data")._file("values_1.csv").write("1,4,7")
root._dir("data", replace=False)._file("values_2.csv").write("2,9,7")Other operations
~~~~~~~~~~~~~~~~You can move, copy, and delete a file with ``.move(folder)``, ``.copy(folder)``,
``.delete()``, and a directory with ``._move(folder)``, ``._copy(folder)``,
``._delete()``... code:: python
root.data.values1_csv.delete() # delete file 'values1.csv'
root.data._delete() # delete directory 'data'
# Move folder `plots` from `root/figures` to `other_root/figures`
root.figures.plots._move(other_root.figures)
# Move file `fig.png` from `root/figures` to `other_root/figures`
root.figures.fig_png.move(other_root.figures)Special rules for ZIP archives
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~It is not currently possible to modify/delete a file that is already zipped
into an archive (because zips are not really made for that, it would
be doable but would certainly be a hack).When creating files and folders in a zip with Flametree, the changes in the actual zip
will only be performed by closing the ``root`` with ``root._close()``
(after which the ``root`` can't be used any more). If it is an in-memory zip, ``root._close()``
returns the value of the zip content as a string (Python 2) or bytes (Python 3).Here are a few examples:
.. code:: python
root = file_tree("archive.zip")
root._file("hello.txt").write("Hi there !")
root._close()# Equivalent to the previous, using `with`:
with file_tree("archive.zip") as root:
root._file("hello.txt").write("Hi there !")# Getting binary data of an in-memory zip file:
root = file_tree("@memory")
root._file("hello.txt").write("Hi there !")
binary_data = root._close()Using file writers from other libraries
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Some libraries have file-generating methods which expect a file name or a file
object to write too.
You can also feed Flametree files to these functions. for instance here is
how to use Weasyprint to create a PDF ``pdfs/report.pdf``.. code:: python
import weasyprint
from flametree import file_tree
root = file_tree(".") # or 'archive.zip' to write in an archive.
html = weasyprint.HTML(string="Hello world!", base_url='.')
html.write_pdf(root._dir("pdfs")._file("test.pdf"))And here is how you would save a Matplotlib figure in a zip archive:
.. code:: python
import matplotlib.pyplot as plt
from flametree import file_tree
fig, ax = plt.subplots(1)
ax.plot([1, 2, 3], [3, 1, 2])
with file_tree("archive.zip") as root:
fig.savefig(root._dir("plots")._file("figure.png"), format="png")That's all folks !
.. _Zulko: https://github.com/Zulko/
.. _Github: https://github.com/Edinburgh-Genome-Foundry/flametree