https://github.com/zipfile/python-getdents
Python binding to Linux syscall getdents64
https://github.com/zipfile/python-getdents
getdents64 python-library
Last synced: 7 months ago
JSON representation
Python binding to Linux syscall getdents64
- Host: GitHub
- URL: https://github.com/zipfile/python-getdents
- Owner: ZipFile
- License: bsd-2-clause
- Created: 2016-12-08T15:19:35.000Z (about 9 years ago)
- Default Branch: master
- Last Pushed: 2025-04-01T10:02:01.000Z (11 months ago)
- Last Synced: 2025-04-27T07:38:19.361Z (10 months ago)
- Topics: getdents64, python-library
- Language: Python
- Homepage: https://pypi.org/project/getdents/
- Size: 50.8 KB
- Stars: 8
- Watchers: 1
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.rst
- License: LICENSE
Awesome Lists containing this project
README
===============
Python getdents
===============
Iterate large directories efficiently with python.
About
=====
``python-getdents`` is a simple wrapper around Linux system call ``getdents64`` (see ``man getdents`` for details). `More details `_ on approach.
TODO
====
* Verify that implementation works on platforms other than ``x86_64``.
Install
=======
.. code-block:: sh
pip install getdents
For development
---------------
.. code-block:: sh
python3 -m venv env
. env/bin/activate
pip install -e .[test]
Building Wheels
~~~~~~~~~~~~~~~
.. code-block:: sh
pip install cibuildwheel
cibuildwheel --platform linux --output-dir wheelhouse
Run tests
=========
.. code-block:: sh
ulimit -v 33554432 && py.test tests/
Or
.. code-block:: sh
ulimit -v 33554432 && ./setup.py test
Usage
=====
.. code-block:: python
from getdents import getdents
for inode, type, name in getdents('/tmp', 32768):
print(name)
Advanced
--------
.. code-block:: python
import os
from getdents import *
fd = os.open('/tmp', O_GETDENTS)
for inode, type, name in getdents_raw(fd, 2**20):
print({
DT_BLK: 'blockdev',
DT_CHR: 'chardev ',
DT_DIR: 'dir ',
DT_FIFO: 'pipe ',
DT_LNK: 'symlink ',
DT_REG: 'file ',
DT_SOCK: 'socket ',
DT_UNKNOWN: 'unknown ',
}[type], {
True: 'd',
False: ' ',
}[inode == 0],
name,
)
os.close(fd)
CLI
---
Usage
~~~~~
::
python-getdents [-h] [-b N] [-o NAME] PATH
Options
~~~~~~~
+--------------------------+-------------------------------------------------+
| Option | Description |
+==========================+=================================================+
| ``-b N`` | Buffer size (in bytes) to allocate when |
| | iterating over directory. Default is 32768, the |
| | same value used by glibc, you probably want to |
+--------------------------+ increase this value. Try starting with 16777216 |
| ``--buffer-size N`` | (16 MiB). Best performance is achieved when |
| | buffer size rounds to size of the file system |
| | block. |
+--------------------------+-------------------------------------------------+
| ``-o NAME`` | Output format: |
| | |
| | * ``plain`` (default) Print only names. |
| | * ``csv`` Print as comma-separated values in |
+--------------------------+ order: inode, type, name. |
| ``--output-format NAME`` | * ``csv-headers`` Same as ``csv``, but print |
| | headers on the first line also. |
| | * ``json`` output as JSON array. |
| | * ``json-stream`` output each directory entry |
| | as single json object separated by newline. |
+--------------------------+-------------------------------------------------+
Exit codes
~~~~~~~~~~
* 3 - Requested buffer is too large
* 4 - ``PATH`` not found.
* 5 - ``PATH`` is not a directory.
* 6 - Not enough permissions to read contents of the ``PATH``.
Examples
~~~~~~~~
.. code-block:: sh
python-getdents /path/to/large/dir
python -m getdents /path/to/large/dir
python-getdents /path/to/large/dir -o csv -b 16777216 > dir.csv