https://github.com/pytask-dev/pytask-julia
A pytask plugin for Julia.
https://github.com/pytask-dev/pytask-julia
pytask
Last synced: 9 months ago
JSON representation
A pytask plugin for Julia.
- Host: GitHub
- URL: https://github.com/pytask-dev/pytask-julia
- Owner: pytask-dev
- License: mit
- Created: 2022-01-14T07:49:32.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2025-09-15T18:05:23.000Z (10 months ago)
- Last Synced: 2025-09-15T20:13:09.625Z (10 months ago)
- Topics: pytask
- Language: Python
- Homepage:
- Size: 205 KB
- Stars: 3
- Watchers: 3
- Forks: 0
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGES.md
- License: LICENSE
Awesome Lists containing this project
README
# pytask-julia
[](https://pypi.org/project/pytask-julia)
[](https://pypi.org/project/pytask-julia)
[](https://anaconda.org/conda-forge/pytask-julia)
[](https://anaconda.org/conda-forge/pytask-julia)
[](https://pypi.org/project/pytask-julia)
[](https://github.com/pytask-dev/pytask-julia/actions?query=branch%3Amain)
[](https://codecov.io/gh/pytask-dev/pytask-julia)
[](https://results.pre-commit.ci/latest/github/pytask-dev/pytask-julia/main)
[](https://github.com/ambv/black)
______________________________________________________________________
Run Julia scripts with pytask.
## Installation
pytask-julia is available on [PyPI](https://pypi.org/project/pytask-julia) and
[Anaconda.org](https://anaconda.org/conda-forge/pytask-julia). Install it with
```console
$ pip install pytask-julia
# or
$ conda install -c conda-forge pytask-julia
```
You also need to have Julia installed and `julia` on your command line. Test it by
typing the following on the command line
```console
julia -h
```
If an error is shown instead of a help page, you can install Julia on Unix systems with
```console
conda install -c conda-forge julia
```
or choose one of the installers on this [page](https://julialang.org/downloads/).
## Usage
To create a task that runs a Julia script, define a task function with the
`@pytask.mark.julia` decorator. The `script` keyword provides a path relative to the
task module to the Julia script.
```python
import pytask
from pathlib import Path
from pytask import task
@task(kwargs={"path": Path("out.csv")})
@pytask.mark.julia(script="script.jl")
def task_run_jl_script():
pass
```
If you are wondering why the function body is empty, know that pytask-julia replaces the
body with a predefined internal function. See the section on implementation details for
more information.
### Dependencies and Products
Dependencies and products can be added as usual. Read this
[tutorial](https://pytask-dev.readthedocs.io/en/stable/tutorials/defining_dependencies_products.html).
For example, with the `@pytask.task` decorator as shown before. (The choice of the kwarg
name, here `path`, is arbitrary.)
### Accessing dependencies and products in the script
To access the paths of dependencies and products in the script, pytask-julia stores the
information by default in a `.json` file. The path to this file is passed as a
positional argument to the script. Inside the script, you can read the information.
```julia
import JSON
path_to_json = ARGS[1] # Contains the path to the .json file.
config = JSON.parse(read(path_to_json, String)) # A dictionary.
config["path"] # Is the path to the output file "../out.csv".
```
The `.json` file is stored in the same folder as the task in a `.pytask` directory.
To parse the JSON file, you need to install
[JSON.jl](https://github.com/JuliaIO/JSON.jl).
You can also pass any other information to your script by using the `@task` decorator.
```python
@task(kwargs={"path": Path("out.csv"), "number": 1})
@pytask.mark.julia(script="script.jl")
def task_run_jl_script():
pass
```
and inside the script use
```julia
config["number"] # Is 1.
```
### Debugging
In case a task throws an error, you might want to execute the script independently from
pytask. After a failed execution, you see the command that executed the Julia script in
the report of the task. It looks roughly like this
```console
julia -- script.jl /.pytask/pytask-julia/.json
```
### Managing Julia environments
Julia has support for environments to execute your tasks via `Pkg.jl` which is explained
[here](https://pkgdocs.julialang.org/v1/environments/).
pytask-julia allows you define a default environment via your
[pytask configuration file](https://pytask-dev.readthedocs.io/en/stable/tutorials/configuration.html).
Use the `julia_project` key to define a path to your configuration file to point to your
environment.
Probably your environment files `Manifest.toml` and `Project.toml` reside at the root of
your project folder as well as your pytask configuration file. Then, the content will
look like this.
```toml
[tool.pytask.ini_options]
julia_project = "."
```
You can also define environments for each task which will overwrite any other default
with the `project` keyword argument. Pass a path to the task module.
```python
@task(kwargs={"path": Path("out.csv")})
@pytask.mark.julia(script="script.jl", project=".")
def task_run_jl_script():
pass
```
### Command Line Options
Command line options can be passed via the `options` keyword argument.
```python
@task(kwargs={"path": Path("out.csv")})
@pytask.mark.julia(script="script.jl", options=["--threads", "2"])
def task_run_jl_script():
pass
```
This example will execute the script using threads.
### Repeating tasks with different scripts or inputs
You can also repeat the execution of tasks, meaning executing multiple Julia scripts or
passing different command line arguments to the same Julia script.
The following task executes two Julia scripts, `script_1.jl` and `script_2.jl`, which
produce different outputs.
```python
for i in range(2):
@task(kwargs={"path": Path(f"out_{i}.csv")})
@pytask.mark.julia(script=f"script_{i}.jl")
def task_execute_julia_script():
pass
```
If you want to pass different inputs to the same Julia script, pass these arguments with
the `kwargs` keyword of the `@task` decorator.
```python
for i in range(2):
@task(kwargs={"path": Path(f"out_{i}.csv"), "i": i})
@pytask.mark.julia(script="script.jl")
def task_execute_julia_script():
pass
```
and inside the task access the argument `i` with
```julia
import JSON
path_to_json = ARGS[1] # Contains the path to the .json file.
config = JSON.parse(read(path_to_json, String)) # A dictionary.
config["path"] # Is the path to the output file "../output_{i}.csv".
config["i"] # Is the number.
```
### Serializers
You can also serialize your data with any other tool you like. By default, pytask-julia
also supports YAML (if PyYaml is installed).
Use the `serializer` keyword arguments of the `@pytask.mark.julia` decorator with
```python
@pytask.mark.julia(script="script.jl", serializer="yaml")
def task_example(): ...
```
And in your Julia script use
```julia
import YAML
config = YAML.load_file(ARGS[1])
```
Note that the `YAML` package needs to be installed.
If you need a custom serializer, you can also provide any callable for `serializer`
which transforms data into a string. Use `suffix` to set the correct file ending.
Here is a replication of the JSON example.
```python
import json
@pytask.mark.julia(script="script.jl", serializer=json.dumps, suffix=".json")
def task_example(): ...
```
### Configuration
You can influence the default behavior of pytask-julia with some configuration values.
Place them into a `pyproject.toml` file.
**`julia_serializer`**
Use this option to change the default serializer.
```toml
[tool.pytask.ini_options]
julia_serializer = "json"
```
**`julia_suffix`**
Use this option to set the default suffix of the file which contains serialized paths to
dependencies, products and more.
```toml
[tool.pytask.ini_options]
julia_suffix = ".json"
```
**`julia_options`**
Use this option to set default options for each task which are separated by whitespace.
```toml
[tool.pytask.ini_options]
julia_options = ["--threads", 2]
```
**`julia_project`**
Use this option to set a default environment for each task. If your environment with
`Manifest.toml` and `Project.toml` is defined in the same directory as the configuration
file `pyproject.toml`, just use a dot.
```toml
[tool.pytask.ini_options]
julia_project = "."
```
If the environment files were in a folder next to the configuration file called
`environment` use
```toml
[tool.pytask.ini_options]
julia_project = "environment"
```
## Changes
Consult the [release notes](CHANGES.md) to find out about what is new.
## Development
After setting up the environment, enter
```console
$ julia
$ ]
$ pkg> activate .
$ pkg> instantiate
```
to set up the Julia environment.