https://github.com/mla/iconv-chunks

Process huge files with iconv
https://github.com/mla/iconv-chunks

Last synced: 2 days ago
JSON representation

Process huge files with iconv

• Host: GitHub
• URL: https://github.com/mla/iconv-chunks
• Owner: mla
• License: mit
• Created: 2015-05-02T20:53:20.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2015-05-02T21:18:15.000Z (over 9 years ago)
• Last Synced: 2023-04-12T23:47:09.959Z (over 1 year ago)
• Language: Perl
• Size: 129 KB
• Stars: 11
• Watchers: 2
• Forks: 3
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGES
  • License: LICENSE

Awesome Lists containing this project

README

        
NAME

====

iconv-chunks - Process huge files with iconv

SYNOPSIS

========

iconv-chunks  [iconv-options]

DESCRIPTION

===========

The standard iconv program reads the entire input file into memory,

which doesn't work for large files (such as database exports).

This script is just a wrapper that processes the input file in

manageable chunks and writes it to standard output.

The first argument is the input filename (use - to specify standard

input). Anything else is passed through to iconv.

The real iconv needs to be somewhere in your PATH.

EXAMPLES

========

    # Convert latin1 to utf-8:

    ./iconv-chunks database.txt -f latin1 -t utf-8 > out.txt

    # Input filename of - means standard input:

    ./iconv-chunks - -f iso8859-1 -t utf8 < database.txt > out.txt

    # More complex example, using compressed input/output to minimize disk use:

    zcat database.txt.gz | ./iconv-chunks - -f iso8859-1 -t utf8 | \

    gzip - > database-utf.dump.gz

AUTHOR

======

Maurice Aubrey 

LICENSE

=======

MIT