https://github.com/mutating/dirstree
Another library for iterating through the contents of a directory
https://github.com/mutating/dirstree
Last synced: 20 days ago
JSON representation
Another library for iterating through the contents of a directory
- Host: GitHub
- URL: https://github.com/mutating/dirstree
- Owner: mutating
- License: mit
- Created: 2025-09-27T19:10:15.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2026-05-29T08:52:36.000Z (22 days ago)
- Last Synced: 2026-05-29T09:25:47.568Z (22 days ago)
- Language: Python
- Size: 168 KB
- Stars: 3
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
ⓘ
[](https://pepy.tech/project/dirstree)
[](https://pepy.tech/project/dirstree)
[](https://coveralls.io/github/mutating/dirstree?branch=main)
[](https://github.com/boyter/scc/)
[](https://hitsofcode.com/github/mutating/dirstree/view?branch=main)
[](https://github.com/mutating/dirstree/actions/workflows/tests_and_coverage.yml)
[](https://pypi.python.org/pypi/dirstree)
[](https://badge.fury.io/py/dirstree)
[](http://mypy-lang.org/)
[](https://github.com/astral-sh/ruff)
[](https://deepwiki.com/mutating/dirstree)
There are many libraries for traversing directories. You can also do this using the standard library. What makes this library different:
- 💎 Beautiful, laconic syntax.
- ⚗️ Filtering by file extensions, text patterns in [`.gitignore` format](https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository#_ignoring), and using custom callables.
- 🐍 Natively works with both [`Path` objects](https://docs.python.org/3/library/pathlib.html#basic-use) from the standard library and strings.
- ❌ Support for [cancellation tokens](https://github.com/pomponchik/cantok).
- 👯♂️ Combining multiple crawling methods in one object.
## Table of contents
- [**Installation**](#installation)
- [**Basic usage**](#basic-usage)
- [**Applying a function to each path**](#applying-a-function-to-each-path)
- [**Filtering**](#filtering)
- [**Working with Cancellation Tokens**](#working-with-cancellation-tokens)
- [**Combination**](#combination)
- [**Transactionality**](#transactionality)
## Installation
You can install [`dirstree`](https://pypi.org/project/dirstree/) with `pip`:
```bash
pip install dirstree
```
You can also use [`instld`](https://github.com/pomponchik/instld) to quickly try out this package and others without installing them.
## Basic usage
The library is easy to use:
- Create a crawler object, passing the path to the base directory and, if necessary, additional arguments.
- Iterate through it.
The simplest example would look like this:
```python
from dirstree import Crawler
crawler = Crawler('.')
for file in crawler:
print(file)
```
> ↑ This recursively prints all files in the current directory, including files in nested directories. At each iteration, we get a new [`Path` object](https://docs.python.org/3/library/pathlib.html#basic-use).
## Applying a function to each path
If you just want to run a function for each file the crawler finds, you don't have to write the loop yourself — every crawler has an `apply()` method:
```python
Crawler('src', exclude=['tests/**']).apply(print)
```
> ↑ This will print the entire contents of the directory, except for the excluded locations.
> ⓘ All of the crawler's settings are respected, exactly as they would be during normal iteration.
## Filtering
By default, crawlers iterate over files only. If you need every filesystem entity found under the base directory, pass `only_files=False`:
```python
crawler = Crawler('.', only_files=False)
```
Iterating through the files in the directory, you may not want to view all files, but only files of a certain type. To do this, ignore all other files. How to do it? There are three ways:
- Bypass only files with the specified [extensions](https://en.wikipedia.org/wiki/Filename_extension), such as `.txt`, `.doc`, or `.py`.
- Bypass files whose paths follow a specific text pattern.
- Use an arbitrary function to determine whether you need each specific path or not.
To select a specific method, you need to pass a specific parameter when creating the crawler object. Of course, all the methods can be combined with each other.
To set the file extensions you are interested in, use the `extensions` parameter:
```python
crawler = Crawler('.', extensions=['.txt']) # Iterate only on .txt files.
```
> ⓘ The `extensions` parameter is available only in the default file-only mode, so it cannot be combined with `only_files=False`.
Also, if you only need Python files, you can use a special class to bypass them only, without specifying extensions:
```python
from dirstree import PythonCrawler
crawler = PythonCrawler('.') # Iterate only on .py files.
```
> ⓘ `PythonCrawler` is always file-only.
To specify which files and directories you do NOT want to iterate over, use the `exclude` parameter:
```python
crawler = Crawler('.', exclude=['.git', 'venv']) # Exclude ".git" and "venv" directories.
```
> ↑ Please note that we use the [`.gitignore` format](https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository#_ignoring) here.
If you need a universal way to filter out unnecessary paths, pass your function as the `filter` parameter:
```python
crawler = Crawler('.', filter=lambda path: len(str(path)) == 7) # Iterate only on paths that are 7 characters long.
```
## Working with Cancellation Tokens
You can set an arbitrary condition under which file traversal will stop using [cancellation tokens](https://cantok.readthedocs.io/en/latest/the_pattern/) from the [`cantok`](https://github.com/pomponchik/cantok) library.
> There are two ways to do this ↓
1. If you use the crawler as a one-time object for a single iteration, set the token when creating it:
```python
for path in Crawler('.', token=TimeoutToken(0.0001)): # Limit the iteration time to 0.0001 seconds.
print(path)
```
2. If you plan to use the crawler object several times, use the `go()` method for iteration and pass a new token to it every time:
```python
crawler = Crawler('.')
for path in crawler.go(token=TimeoutToken(0.0001)): # Limit the iteration time to 0.0001 seconds.
print(path)
```
> ↑ Follow these rules to avoid accidentally "baking" an expired token inside a crawler object.
By default, cancellation stops iteration silently — the caller cannot tell it apart from natural exhaustion. Pass `raise_on_cancel=...` to make the crawler raise an exception on cancellation instead:
```python
for path in Crawler('.', token=TimeoutToken(0.0001), raise_on_cancel=True):
print(path)
```
> ↑ `raise_on_cancel=True` re-raises the native `cantok` exception; `raise_on_cancel=MyError("...")` raises that exact instance; `raise_on_cancel=MyError` instantiates the class with the cantok message and raises that. Default is `False` (silent).
## Combination
You can combine multiple crawler objects into one using the usual addition operator, like this:
```python
for path in Crawler('../dirstree') + Crawler('../cantok'):
print(path)
```
> ↑ The paths that you will iterate over will be automatically deduplicated.
> ↑ You can also impose arbitrary restrictions on each of the summed objects, all of them will be taken into account.
You can also pass multiple paths to a single crawler object:
```python
for path in Crawler('../dirstree', '../cantok'):
print(path)
```
> ↑ In this case, there is no deduplication of paths.
## Transactionality
If you plan to modify the directory while iterating over it — for example, deleting or moving files inside an `apply()` callback — pass `freeze=True` to take a snapshot of every matching path up front, then iterate that snapshot instead of the live filesystem:
```python
Crawler('path/to/directory', freeze=True).apply(lambda p: p.unlink())
```
> ↑ The snapshot is built on the first step of iteration, with every filter and cancellation token already applied. After that, any creation, renaming or deletion happening in the directory does not affect what is yielded — each call to `go()` or `iter()` produces its own fresh snapshot.
> ↑ Without `freeze=True` the order of yielded paths depends on the live state of the filesystem, so mid-iteration mutation may silently skip or duplicate entries.