Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/reubano/csv2ofx
A Python library and command line tool for converting csv to ofx and qif files
https://github.com/reubano/csv2ofx
cli csv data featured finance library ofx qif
Last synced: about 2 months ago
JSON representation
A Python library and command line tool for converting csv to ofx and qif files
- Host: GitHub
- URL: https://github.com/reubano/csv2ofx
- Owner: reubano
- License: mit
- Created: 2012-10-01T13:53:08.000Z (almost 12 years ago)
- Default Branch: master
- Last Pushed: 2024-02-22T19:45:20.000Z (5 months ago)
- Last Synced: 2024-05-02T00:07:36.382Z (2 months ago)
- Topics: cli, csv, data, featured, finance, library, ofx, qif
- Language: Python
- Homepage:
- Size: 4.38 MB
- Stars: 189
- Watchers: 15
- Forks: 109
- Open Issues: 24
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Lists
- awesome - csv2ofx - A Python library and command line tool for converting csv to ofx and qif files (Python)
- my-awesome-stars - reubano/csv2ofx - A Python library and command line tool for converting csv to ofx and qif files (Python)
README
# csv2ofx
[](https://app.travis-ci.com/github/reubano/csv2ofx)
[](https://pypi.python.org/pypi/csv2ofx)
[](https://pypi.python.org/pypi/csv2ofx)
## INTRODUCTION
[csv2ofx](http://github.com/reubano/csv2ofx) is a [Python library](#library-examples) and [command line interface program](#cli-examples) that converts CSV files to OFX and QIF files for importing into GnuCash or similar financial accounting programs. csv2ofx has built in support for importing csv files from mint, yoodlee, and xero.
## Requirements
csv2ofx has been tested and is known to work on Python 3.7, 3.8, and 3.9; and PyPy3.7.
## INSTALLATION
(You are using a [virtualenv](http://www.virtualenv.org/en/latest/index.html), right?)
sudo pip install csv2ofx
## Usage
csv2ofx is intended to be used either directly from Python or from the command line.
### Library Examples
*normal OFX usage*
```python
import itertools as it
from meza.io import read_csv, IterStringIO
from csv2ofx import utils
from csv2ofx.ofx import OFX
from csv2ofx.mappings.default import mapping
ofx = OFX(mapping)
records = read_csv('path/to/file.csv', has_header=True)
groups = ofx.gen_groups(records)
trxns = ofx.gen_trxns(groups)
cleaned_trxns = ofx.clean_trxns(trxns)
data = utils.gen_data(cleaned_trxns)
content = it.chain([ofx.header(), ofx.gen_body(data), ofx.footer()])
for line in IterStringIO(content):
print(line)
```
*normal QIF usage*
```python
import itertools as it
from tabutils.io import read_csv, IterStringIO
from csv2ofx import utils
from csv2ofx.qif import QIF
from csv2ofx.mappings.default import mapping
qif = QIF(mapping)
records = read_csv('path/to/file.csv', has_header=True)
groups = qif.gen_groups(records)
trxns = qif.gen_trxns(groups)
cleaned_trxns = qif.clean_trxns(trxns)
data = utils.gen_data(cleaned_trxns)
content = it.chain([qif.gen_body(data), qif.footer()])
for line in IterStringIO(content):
print(line)
```
### CLI Examples
*show help*
csv2ofx -h
```bash
usage: csv2ofx [options]
description: csv2ofx converts a csv file to ofx and qif
positional arguments:
source the source csv file (default: stdin)
dest the output file (default: stdout)
optional arguments:
-h, --help show this help message and exit
-a TYPE, --account TYPE
default account type 'CHECKING' for OFX and 'Bank' for QIF.
-e DATE, --end DATE end date (default: today)
-B BALANCE, --ending-balance BALANCE
ending balance (default: None)
-l LANGUAGE, --language LANGUAGE
the language (default: ENG)
-s DATE, --start DATE
the start date
-y, --dayfirst interpret the first value in ambiguous dates (e.g. 01/05/09) as the day
-m MAPPING_NAME, --mapping MAPPING_NAME
the account mapping (default: default)
-x FILE_PATH, --custom FILE_PATH
path to a custom mapping file
-c FIELD_NAME, --collapse FIELD_NAME
field used to combine transactions within a split for double entry statements
-C ROWS, --chunksize ROWS
number of rows to process at a time (default: 2 ** 14)
-r ROWS, --first-row ROWS
the first row to process (zero based)
-R ROWS, --last-row ROWS
the last row to process (zero based, negative values count from the end)
-O COLS, --first-col COLS
the first column to process (zero based)
-L, --list-mappings list the available mappings
-V, --version show version and exit
-q, --qif enables 'QIF' output instead of 'OFX'
-M, --ms-money enables MS Money compatible 'OFX' output
-o, --overwrite overwrite destination file if it exists
-D DATE, --server-date DATE
OFX server date (default: source file mtime)
-E ENCODING, --encoding ENCODING
File encoding (default: utf-8)
-d, --debug display the options and arguments passed to the parser
-v, --verbose verbose output
```
*normal usage*
csv2ofx file.csv file.ofx
*print output to stdout*
csv2ofx ~/Downloads/transactions.csv
*read input from stdin*
cat file.csv | csv2ofx
*qif output*
csv2ofx -q file.csv
*specify date range from one year ago to yesterday with qif output*
csv2ofx -s '-1 year' -e yesterday -q file.csv
*use yoodlee settings*
csv2ofx -m yoodlee file.csv
#### Special cases
Some banks, like *UBS Switzerland*, may provide CSV exports that are not
readily tractable by csv2ofx because of extra header or trailing lines,
redundant or unwanted columns. These input files can be preprocessed with the
shipped `utilz/csvtrim` shell script. F.i., with mapping `ubs-ch-fr`:
csvtrim untrimmed.csv | csv2ofx -m ubs-ch-fr
## CUSTOMIZATION
### Code modification
If you would like to import csv files with field names different from the default, you can modify the mapping file or create your own. New mappings must be placed in the `csv2ofx/mappings` folder (otherwise you must use the ). The mapping object consists of a dictionary whose keys are OFX/QIF attributes and whose values are functions which should return the corresponding value from a record (csv row). The mapping function will take in a record, e.g.,
```python
{'Account': 'savings 2', 'Date': '1/3/15', 'Amount': 5000}
```
The most basic mapping function just returns a specific field or value, e.g.,
```python
from operator import itemgetter
mapping = {
'bank': 'BetterBank',
'account': itemgetter('Account'),
'date': itemgetter('Date'),
'amount': itemgetter('Amount')}
```
But more complex parsing is also possible, e.g.,
```python
mapping = {
'account': lambda r: r['Details'].split(':')[0],
'date': lambda r: '%s/%s/%s' % (r['Month'], r['Day'], r['Year']),
'amount': lambda r: r['Amount'] * 2,
'first_row': 1,
'last_row': 10,
'filter': lambda r: r['Amount'] > 10,
}
```
### Required field attributes
attribute | description | default field | example
----------|-------------|---------------------|--------
`account`|transaction account|Account|BetterBank Checking
`date`|transaction date|Date|itemgetter('Transaction Date')
`amount`|transaction amount|Amount|itemgetter('Transaction Amount')
### Optional field attributes
attribute | description | default field | default value | example
----------|-------------|---------------|---------------|--------
`desc`|transaction description|Reference|n/a|shell station
`payee`|transaction payee|Description|n/a|Shell
`notes`|transaction notes|Notes|n/a|for gas
`check_num`|the check or transaction number|Row|n/a|2
`id`|transaction id|`check_num`|Num|n/a|531
`bank`|the bank name|n/a|`account`|Bank
`account`|transaction account type|n/a|checking|savings
`account_id`|transaction account id|n/a|hash of `account`|bb_checking
`type`|transaction type (either debit or credit)|n/a|CREDIT if amount > 0 else DEBIT|debit
`balance`|account balance|n/a|n/a|$23.00
`class`|transaction class|n/a|n/a|travel
### Optional value attributes
attribute | description | default value | example
----------|-------------|---------------|--------
`has_header`|does the csv file have a header row|True
`custom_header`|header row to use (e.g. if not provided in csv)|None|["Account","Date","Amount"]
`is_split`|does the csv file contain split (double entry) transactions|False
`currency`|the currency ISO code|USD|GBP
`delimiter`|the csv field delimiter|,|;
`date_fmt`|custom QIF date output format|%m/%d/%y|%m/%d/%Y
`dayfirst`|interpret the first value in ambiguous dates (e.g. 01/05/09) as the day (ignored if `parse_fmt` is present)|False|True
`parse_fmt`|transaction date parsing format||%m/%d/%Y
`first_row`|the first row to process (zero based)|0|2
`last_row`|the last row to process (zero based, negative values count from the end)|inf|-2
`first_col`|the first column to process (zero based)|0|2
`filter`|keep transactions for which function returns true||lambda tr: tr['amount'] > 10
## Scripts
csv2ofx comes with a built in task manager `manage.py`.
### Setup
pip install -r dev-requirements.txt
### Examples
*Run python linter and nose tests*
```bash
manage lint
manage test
```
## Contributing
Please mimic the coding style/conventions used in this repo. If you add new classes or functions, please add the appropriate doc blocks with examples. Also, make sure the python linter and nose tests pass.
Ready to contribute? Here's how:
1. Fork and clone.
```bash
git clone [email protected]:/csv2ofx.git
cd csv2ofx
```
2. Setup a new [virtualenv](http://www.virtualenv.org/en/latest/index.html)
```bash
mkvirtualenv -i pkutils csv2ofx
activate csv2ofx
python setup.py develop
pip install -r dev-requirements.txt
```
3. Create a branch for local development
```bash
git checkout -b name-of-your-bugfix-or-feature
```
4. Make your changes, run linter and tests (see above), and submit a pull request through the GitHub website.
### Adding Mappings
How to contribute a mapping:
1. Add the mapping in `csv2ofx/mappings/`
2. Add a simple example CSV file in `data/test/`.
3. Add the OFX or QIF file that results from the mapping and example CSV file in `data/converted/`.
4. Add a `csv2ofx` call for your mapping to the tests in `tests/test.py`, in `PRE_TESTS`. If you added an OFX (not QIF) converted file, pay attention to the `-e` (end date) and `-D` (server date) arguments in the test- otherwise tests may pass on your workstation and fail on the build server.
5. Ensure your test succeeds (see above).
## License
csv2ofx is distributed under the [MIT License](http://opensource.org/licenses/MIT), the same as [meza](https://github.com/reubano/meza).