https://github.com/nicolay-r/bulk-translate

No-string Python framework for translation texts with optionally pre-annotated object spans from CSV tabular data; powered by AREkit
https://github.com/nicolay-r/bulk-translate

arekit googletrans iterator pipeline span span-based spreadsheet spreadsheets translate

Last synced: about 1 month ago
JSON representation

No-string Python framework for translation texts with optionally pre-annotated object spans from CSV tabular data; powered by AREkit

• Host: GitHub
• URL: https://github.com/nicolay-r/bulk-translate
• Owner: nicolay-r
• License: mit
• Created: 2024-11-10T13:44:50.000Z (2 months ago)
• Default Branch: master
• Last Pushed: 2024-12-10T10:41:43.000Z (about 1 month ago)
• Last Synced: 2024-12-10T11:35:39.278Z (about 1 month ago)
• Topics: arekit, googletrans, iterator, pipeline, span, span-based, spreadsheet, spreadsheets, translate
• Language: Python
• Homepage:
• Size: 325 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# bulk-translate 0.24.1

![](https://img.shields.io/badge/Python-3.9-brightgreen.svg)

![](https://img.shields.io/badge/AREkit-0.25.0-orange.svg)

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/nicolay-r/bulk-translate/blob/master/bulk_translate_demo.ipynb)

[![PyPI downloads](https://img.shields.io/pypi/dm/bulk-translate.svg)](https://pypistats.org/packages/bulk-translate)

A tiny Python no-string package for performing translation of a massive `CSV`/`JSONL` files that 

natively provides support for annotating **fixed-spans** that are optionally invariant for translator.

  



    



The out-of-the box features of the `bulk-translate` are:

* ✅ Support of the `spans` for annotation / optional translation.

* ✅ Native Implementation of two translation modes:

  - `fast-mode`: exploits extra chars that could be used for grouping all the text parts into single batch with further deconstruction.

  - `accurate`: pefroms individual translation of each text part.

* ✅ No strings: you're free to adopt any LM / LLM backend.

  - Support `googletrans` by default.

## Installation

```bash

pip install git+https://github.com/nicolay-r/bulk-translate

```

## Usage

> **NOTE:** If you wish to translate parse entities, you can use `parse-entities` flag

For the following [`test.tsv` example data](/test/data/test.tsv) with annotated entities enclosed in square brackets:

```bash

python -m bulk_translate.translate \

    --src "test/data/test.tsv" \

    --prompt "{text}" \

    --adapter "dynamic:models/googletrans_310a.py:GoogleTranslateModel" \

    --output "test-translated.jsonl" \

    --parse-entities \

    %% \

    --src "auto" \

    --dest "ru"

```

## Powered by

* AREkit [[github]](https://github.com/nicolay-r/AREkit)