https://github.com/Genivia/ugrep
🔍 ugrep 7.3 file pattern searcher -- a more powerful, ultra fast, user-friendly, compatible grep replacement. Includes a TUI, Google-like Boolean search with AND/OR/NOT, fuzzy search, hexdumps, searches (nested) archives (zip, 7z, tar, pax, cpio), compressed files (gz, Z, bz2, lzma, xz, lz4, zstd, brotli), pdfs, docs, and more
https://github.com/Genivia/ugrep
code-search file-indexing file-search fuzzy-search grep hexdump interactive recursively-search regex ripgrep search silver-searcher tar tui unicode zip
Last synced: about 1 year ago
JSON representation
🔍 ugrep 7.3 file pattern searcher -- a more powerful, ultra fast, user-friendly, compatible grep replacement. Includes a TUI, Google-like Boolean search with AND/OR/NOT, fuzzy search, hexdumps, searches (nested) archives (zip, 7z, tar, pax, cpio), compressed files (gz, Z, bz2, lzma, xz, lz4, zstd, brotli), pdfs, docs, and more
- Host: GitHub
- URL: https://github.com/Genivia/ugrep
- Owner: Genivia
- License: bsd-3-clause
- Created: 2019-04-29T01:42:38.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2025-03-10T14:40:02.000Z (about 1 year ago)
- Last Synced: 2025-03-11T13:17:31.836Z (about 1 year ago)
- Topics: code-search, file-indexing, file-search, fuzzy-search, grep, hexdump, interactive, recursively-search, regex, ripgrep, search, silver-searcher, tar, tui, unicode, zip
- Language: C++
- Homepage: https://ugrep.com
- Size: 139 MB
- Stars: 2,762
- Watchers: 32
- Forks: 116
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE.txt
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
- awesome-for-oneliner - ugrep - a faster, more user-friendly and more featureful replacement for grep (The name of command: `ug`) (Grep Replacement / Open USP Tsukubai)
- awesome-cli-apps-in-a-csv - ugrep - Ultra fast grep with interactive TUI, fuzzy search, boolean queries, hexdumps and more. (<a name="text-search"></a>Text search (alternatives to grep))
- awesome-cli-apps - ugrep - Ultra fast grep with interactive TUI, fuzzy search, boolean queries, hexdumps and more. (<a name="text-search"></a>Text search (alternatives to grep))
- my-awesome-github-stars - Genivia/ugrep - 🔍 ugrep 7.8 file pattern searcher -- a user-friendly, faster, more capable grep replacement. Includes a TUI, Google-like Boolean search with AND/OR/NOT, fuzzy search, hexdumps, searches (nested) archi (C++)
- awesome-starred - Genivia/ugrep - 🔍 ugrep 7.8 file pattern searcher -- a user-friendly, faster, more capable grep replacement. Includes a TUI, Google-like Boolean search with AND/OR/NOT, fuzzy search, hexdumps, searches (nested) archi (C++)
- command-line-tools - ugrep - Ultra fast grep with interactive query UI. (Other)
README
The ugrep file pattern searcher
[ README | User Guide | Indexing | Benchmarks | Q&A ]
option -Q opens a query TUI to search files as you type!
Why use ugrep?
--------------
- ugrep is fast, user-friendly, and equipped with a ton of new features that users wanted
- includes an interactive TUI with built-in help, Google-like search with AND/OR/NOT patterns, fuzzy search, search (nested) zip/7z/tar/pax/cpio archives, tarballs and compressed files gz/Z/bz/bz2/lzma/xz/lz4/zstd/brotli, search and hexdump binary files, search documents such as PDF, doc, docx, and output in JSON, XML, CSV or your own customized format
- Unicode extended regex pattern syntax with multi-line pattern matching without requiring special command-line options
- includes a file indexer to speed up searching slow and cold file systems
- a true drop-in replacement for GNU grep (assuming you [copy or symlink `ug` to `grep`, and to `egrep` and to `fgrep`](#grep)), unlike other popular grep claiming to be "grep alternatives" or "replacements" when those actually implement incompatible command-line options and use an incompatible regex matcher, i.e. Perl regex only versus POSIX BRE (grep) and ERE (egrep) when ugrep supports all regex modes
- benchmarks show that [ugrep is (one of) the fastest grep](https://github.com/Genivia/ugrep-benchmarks) using the high-performance DFA-based regex matcher [RE/flex](https://github.com/Genivia/RE-flex)
Development roadmap
-------------------
*if something should be improved or added to ugrep, then let me know!*
- #1 priority is quality assurance to continue to make sure ugrep has no bugs and is reliable
- make ugrep run even faster, see for example [#432](https://github.com/Genivia/ugrep/issues/432), [#421](https://github.com/Genivia/ugrep/issues/421)
- share [reproducible performance results](https://github.com/Genivia/ugrep-benchmarks)
Overview
--------
### Commands
- `ug` is for interactive use, which loads an optional .ugrep configuration file with your preferences located in the working directory or home directory, `ug+` also searches pdfs, documents, e-books, image metadata
- `ugrep` for batch use like GNU grep without a .ugrep configuration file, `ugrep+` also searches pdfs, documents, e-books, image metadata
### What does ugrep add that GNU grep does not support?
- Matches Unicode patterns by default and automatically searches UTF-8, UTF-16 and UTF-32 encoded files
- Matches multiple lines with `\n` or `\R` in regex patterns, *no special options are required to do so!*
- Built-in help: `ug --help`, where `ug --help WHAT` displays options related to `WHAT` you are looking for
💡 `ug --help regex`, `ug --help globs`, `ug --help fuzzy`, `ug --help format`.
- User-friendly with customizable [configuration files](#config) used by the `ug` command intended for interactive use that loads a .ugrep configuration file with your preferences
ug PATTERN ... ugrep --config PATTERN ...
💡 `ug --save-config ...options-you-want-to-save...` saves a .ugrep config file in the working directory so that the next time you run `ug` there it uses these options. Do this in your home directory to save a .ugrep config file with options you generally want to use.
- Interactive [query TUI](#query), press F1 or CTRL-Z for help and TAB/SHIFT-TAB to navigate to dirs and files
ug -Q ug -Q -e PATTERN
💡 `-Q` replaces `PATTERN` on the command line to let you enter patterns interactively in the TUI. In the TUI use ALT+letter keys to toggle short "letter options" on/off, for example ALT-n (option `-n`) to show/hide line numbers.
- Search the contents of [archives](#archives) (zip, tar, pax, jar, cpio, 7z) and [compressed files](#archives) (gz, Z, bz, bz2, lzma, xz, lz4, zstd, brotli)
ug -z PATTERN ... ug -z --zmax=2 PATTERN ...
💡 specify `-z --zmax=2` to search compressed files and archives nested within archives. The `--zmax` argument may range from 1 (default) to 99 for up to 99 decompression and de-archiving steps to search nested archives
- Search with Google-like [Boolean query patterns](#bool) using `-%` patterns with `AND` (or just space), `OR` (or a bar `|`), `NOT` (or a dash `-`), using quotes to match exactly, and grouping with `( )` (shown on the left side below); or with options `-e` (as an "or"), `--and`, `--andnot`, and `--not` regex patterns (shown on the right side below):
ug -% 'A B C' ... ug -e 'A' --and 'B' --and 'C' ...
ug -% 'A|B C' ... ug -e 'A' -e 'B' --and 'C' ...
ug -% 'A -B -C' ... ug -e 'A' --andnot 'B' --andnot 'C' ...
ug -% 'A -(B|C)'... ug -e 'A' --andnot 'B' --andnot 'C' ...
ug -% '"abc" "def"' ... ug -e '\Qabc\E' --and '\Qdef\E' ...
where `A`, `B` and `C` are arbitrary regex patterns (use option `-F` to search strings)
💡 specify option `-%%` (`--bool --files`) to apply the Boolean query to files as a whole: a file matches if all Boolean conditions are satisfied by matching patterns file-wide. Otherwise, Boolean conditions apply to single lines by default, since grep utilities are generally line-based pattern matchers. Option `--stats` displays the query in human-readable form after the search completes.
- Search pdf, doc, docx, e-book, and more with `ug+` [using filters](#filter) associated with filename extensions:
ug+ PATTERN ...
or specify `--filter` with a file type to use a filter utility:
ug --filter='pdf:pdftotext % -' PATTERN ...
ug --filter='doc:antiword %' PATTERN ...
ug --filter='odt,docx,epub,rtf:pandoc --wrap=preserve -t plain % -o -' PATTERN ...
ug --filter='odt,doc,docx,rtf,xls,xlsx,ppt,pptx:soffice --headless --cat %' PATTERN ...
ug --filter='pem:openssl x509 -text,cer,crt,der:openssl x509 -text -inform der' PATTERN ...
ug --filter='latin1:iconv -f LATIN1 -t UTF-8' PATTERN ...
💡 the `ug+` command is the same as the `ug` command, but also uses filters to search PDFs, documents, and image metadata
- Display horizontal context with option `-o` (`--only-matching`) and context options `-ABC`, e.g. to find matches in very long lines, such as Javascript and JSON sources:
ug -o -C20 -nk PATTERN longlines.js
💡 `-o -C20` fits all matches with context in 20 characters before and 20 charactess after a match (i.e. 40 Unicode characters total), `-nk` outputs line and column numbers.
- Find approximate pattern matches with [fuzzy search](#fuzzy), within the specified Levenshtein distance
ug -Z PATTERN ... ug -Z3 PATTTERN ...
💡 `-Zn` matches up to `n` extra, missing or replaced characters, `-Z+n` matches up to `n` extra characters, `-Z-n` matches with up to `n` missing characters and `-Z~n` matches up to `n` replaced characters. `-Z` defaults to `-Z1`.
- Fzf-like search with regex (or fixed strings with `-F`), fuzzy matching with up to 4 extra characters with `-Z+4` and words only with `-w`, using `-%%` for file-wide Boolean searches
ug -Q -%% -l -w -Z+4 --sort=best
💡 `-l` lists the matching files in the TUI, press `TAB` then `ALT-y` to view a file, `SHIFT-TAB` and `Alt-l` to go back to view the list of matching files ordered by best match
- Search [binary files](#binary) and display hexdumps with binary pattern matches (Unicode text or `-U` for byte patterns)
ug --hexdump -U BYTEPATTERN ... ug --hexdump TEXTPATTERN ...
ug -X -U BYTEPATTERN ... ug -X TEXTPATTERN ...
ug -W -U BYTEPATTERN ... ug -W TEXTPATTERN ...
💡 `--hexdump=4chC1` displays `4` columns of hex without a character column `c`, no hex spacing `h`, and with one extra hex line `C1` before and after a match.
- Include files to search by [file types or file "magic bytes"](#magic) or exclude them with `^`
ug -t TYPE PATTERN ... ug -t ^TYPE PATTERN ...
ug -M 'MAGIC' PATTERN ... ug -M '^MAGIC' PATTERN ...
- Include files and directories to search that match [gitignore-style globs](#globs) or exclude them with `^`
ug -g 'FILEGLOB' PATTERN ... ug -g '^FILEGLOB' PATTERN ...
ug -g 'DIRGLOB/' PATTERN ... ug -g '^DIRGLOB/' PATTERN ...
ug -g 'PATH/FILEGLOB' PATTERN ... ug -g '^PATH/FILEGLOB' PATTERN ...
ug -g 'PATH/DIRGLOB/' PATTERN ... ug -g '^PATH/DIRGLOB/' PATTERN ...
- Include files to search by [filename extensions](#magic) (suffix) or exclude them with `^`, a shorthand for `-g"*.EXT"`
ug -O EXT PATTERN ... ug -O ^EXT PATTERN ...
- Include [hidden files (dotfiles) and directories](#hidden) to search (omitted by default)
ug -. PATTERN ... ug -g'.*,.*/' PATTERN ...
💡 specify `hidden` in your .ugrep to always search hidden files with `ug`.
- Exclude files specified by [.gitignore](#ignore) etc.
ug --ignore-files PATTERN ... ug --ignore-files=.ignore PATTERN ...
💡 specify `ignore-files` in your .ugrep to always ignore them with `ug`. Add additional `ignore-files=...` as desired.
- Search patterns excluding [negative patterns](#not) ("match this but not that")
ug -e PATTERN -N NOTPATTERN ... ug -e '[0-9]+' -N 123 ...
- Use [predefined regex patterns](#source) to search source code, javascript, XML, JSON, HTML, PHP, markdown, etc.
ug PATTERN -f c++/zap_comments -f c++/zap_strings ...
ug PATTERN -f php/zap_html ...
ug -f js/functions ... | ug PATTERN ...
- Sort matching files by [name, best match, size, and time](#sort)
ug --sort PATTERN ... ug --sort=size PATTERN ...
ug --sort=changed PATTERN ... ug --sort=created PATTERN ...
ug -Z --sort=best PATTERN ... ug --no-sort PATTERN ...
- Output results in [CSV, JSON, XML](#json), and [user-specified formats](#format)
ug --csv PATTERN ... ug --json PATTERN ...
ug --xml PATTERN ... ug --format='file=%f line=%n match=%O%~' PATTERN ...
💡 `ug --help format` displays help on format `%` fields for customized output.
- Search with PCRE's Perl-compatible regex patterns and display or replace [subpattern matches](#replace)
ug -P PATTERN ... ug -P --format='%1 and %2%~' 'PATTERN(SUB1)(SUB2)' ...
- Replace patterns in the output with [-P and --replace](#replace) replacement text, optionally containing `%` [formatting fields](#format), using `-y` to pass the rest of the file through:
ug --replace='TEXT' PATTERN ... ug -y --replace='TEXT' PATTERN ...
ug --replace='(%m:%o)' PATTERN ... ug -y --replace='(%m:%o)' PATTERN ...
ug -P --replace='%1' PATTERN ... ug -y -P --replace='%1' PATTERN ...
💡 `ug --help format` displays help on format `%` fields to optionally use with `--replace`.
- Search files with a specific [encoding](#encoding) format such as ISO-8859-1 thru 16, CP 437, CP 850, MACROMAN, KOI8, etc.
ug --encoding=LATIN1 PATTERN ...
Table of contents
-----------------
- [How to install](#install)
- [Performance comparisons](#speed)
- [Using ugrep within Vim](#vim)
- [Using ugrep within Emacs](#emacs)
- [Using ugrep to replace GNU/BSD grep](#grep)
- [Equivalence to GNU/BSD grep](#equivalence)
- [Short and quick command aliases](#aliases)
- [Notable improvements over grep](#improvements)
- [Tutorial](#tutorial)
- [Examples](#examples)
- [Advanced examples](#advanced)
- [Displaying helpful info](#help)
- [Configuration files](#config)
- [Interactive search with -Q](#query)
- [Recursively list matching files with -l, -R, -r, --depth, -g, -O, and -t](#recursion)
- [Boolean query patterns with -%, -%%, --and, --not](#bool)
- [Search this but not that with -v, -e, -N, -f, -L, -w, -x](#not)
- [Search non-Unicode files with --encoding](#encoding)
- [Matching multiple lines of text](#multiline)
- [Displaying match context with -A, -B, -C, and -y](#context)
- [Searching source code using -f, -O, and -t](#source)
- [Searching compressed files and archives with -z](#archives)
- [Find files by file signature and shebang "magic bytes" with -M, -O and -t](#magic)
- [Fuzzy search with -Z](#fuzzy)
- [Search hidden files with -.](#hidden)
- [Using filter utilities to search documents with --filter](#filter)
- [Searching and displaying binary files with -U, -W, and -X](#binary)
- [Ignore binary files with -I](#nobinary)
- [Ignoring .gitignore-specified files with --ignore-files](#ignore)
- [Using gitignore-style globs to select directories and files to search](#globs)
- [Including or excluding mounted file systems from searches](#fs)
- [Counting the number of matches with -c and -co](#count)
- [Displaying file, line, column, and byte offset info with -H, -n, -k, -b, and -T](#fields)
- [Displaying colors with --color and paging the output with --pager](#color)
- [Output matches in JSON, XML, CSV, C++](#json)
- [Customize output with --format](#format)
- [Replacing matches with -P --replace and --format using backreferences](#replace)
- [Limiting the number of matches with -1,-2...-9, -K, -m, and --max-files](#max)
- [Matching empty patterns with -Y](#empty)
- [Case-insensitive matching with -i and -j](#case)
- [Sort files by name, best match, size, and time](#sort)
- [Tips for advanced users](#tips)
- [More examples](#more)
- [Man page](#man)
- [Regex patterns](#patterns)
- [POSIX regular expression syntax](#posix-syntax)
- [POSIX and Unicode character classes](#posix-classes)
- [POSIX and Unicode character categories](#posix-categories)
- [Perl regular expression syntax](#perl-syntax)
- [Troubleshooting](#bugs)
How to install
--------------
### MacOS
Install the latest ugrep with [Homebrew](https://brew.sh):
$ brew install ugrep
or install with [MacPorts](https://www.macports.org):
$ sudo port install ugrep
This installs the `ugrep` and `ug` commands, where `ug` is the same as `ugrep`
but also loads the configuration file .ugrep when present in the working
directory or home directory.
### Windows
Install with [Winget](https://learn.microsoft.com/en-us/windows/package-manager/)
`winget install Genivia.ugrep`
Or install with [Chocolatey](https://community.chocolatey.org/packages/ugrep)
`choco install ugrep`
Or install with [Scoop](https://scoop.sh) `scoop install ugrep`
Or download the full-featured `ugrep.exe` executable as release artifact from
. The zipped release contains the
main `ugrep.exe` binary as well as `ug.exe`. The `ug` command, intended for
interactive use, loads and reads in settings from the _`.ugrep`_ configuration
file (when present in the working directory or home directory).
Add `ugrep.exe` and `ug.exe` to your execution path: go to *Settings* and
search for "Path" in *Find a Setting*. Select *environment variables* ->
*Path* -> *New* and add the directory where you placed the `ugrep.exe` and
`ug.exe` executables.
>[!TIP]
> _Practical hints on using `ugrep.exe` and `ug.exe` on the Windows command line:_
>
>- when quoting patterns and arguments on the command line, do not use single
>`'` quotes but use `"` instead; most Windows command utilities consider
>the single `'` quotes part of the command-line argument!
>- file and directory globs are best specified with option `-g/GLOB` instead of
>the usual `GLOB` command line arguments to select files and directories to
>search, especially for recursive searches;
>- when specifying an empty pattern `""` to match all input, this may be ignored
>by some Windows command interpreters such as Powershell, in that case you
>must specify option `--match` instead;
>- to match newlines in patterns, you may want to use `\R` instead of `\n` to
>match any Unicode newlines, such as `\r\n` pairs and single `\r` and `\n`.
### Alpine Linux
$ apk add ugrep ugrep-doc
Check for version info.
### Arch Linux
$ pacman -S ugrep
Check for version info.
### CentOS
First enable the [EPEL repository](https://docs.fedoraproject.org/en-US/epel/),
then you can install ugrep.
$ dnf install ugrep
Check for version info.
### Debian
$ apt-get install ugrep
Check for version info. To build and try
`ugrep` locally, see "All platforms" build steps further below.
### Fedora
$ dnf install ugrep
Check for version info.
### FreeBSD
$ pkg install ugrep
Check for version info.
### Haiku
$ pkgman install cmd:ugrep
Check for
version info. To build and try `ugrep` locally, see "All platforms" build
steps further below.
### NetBSD
You can use the standard NetBSD package installer (pkgsrc):
### OpenBSD
$ pkg_add ugrep
Check for version info.
### OpenSUSE
$ zypper install ugrep
Check for version info.
### RHEL
First enable the [EPEL repository](https://docs.fedoraproject.org/en-US/epel/),
then you can install ugrep.
$ dnf install ugrep
Check for version info.
### Other platforms: step 1 download
Clone `ugrep` with
$ git clone https://github.com/Genivia/ugrep
Or visit to download a specific release.
### Other platforms: step 2 consider optional dependencies
You can always add these later, when you need these features:
- Option `-P` (Perl regular expressions) requires either the PCRE2 library
(recommended) or the Boost.Regex library (optional fallback). If PCRE2 is
not installed, install PCRE2 with e.g. `sudo apt-get install -y libpcre2-dev`
or [download PCRE2](https://www.pcre.org) and follow the installation
instructions. Alternatively,
[download Boost.Regex](https://www.boost.org/users/download) and run
`./bootstrap.sh` and `sudo ./b2 --with-regex install`. See
[Boost: getting started](https://www.boost.org/doc/libs/1_72_0/more/getting_started/unix-variants.html).
- Option `-z` (compressed files and archives search) requires the
[zlib](https://www.zlib.net) library installed. It is installed on most
systems. If not, install it, e.g. with `sudo apt-get install -y libz-dev`.
To search `.bz` and `.bz2` files, install the
[bzip2](https://www.sourceware.org/bzip2) library (recommended), e.g. with
`sudo apt-get install -y libbz2-dev`. To search `.lzma` and `.xz` files,
install the [lzma](https://tukaani.org/xz) library (recommended), e.g. with
`sudo apt-get install -y liblzma-dev`. To search `.lz4` files, install the
[lz4](https://github.com/lz4/lz4) library (optional, not required), e.g.
with `sudo apt-get install -y liblz4-dev`. To search `.zst` files, install
the [zstd](http://facebook.github.io/zstd) library (optional, not required),
e.g. with `sudo apt-get install -y libzstd-dev`. To search `.br` files,
install the [brotli](https://github.com/google/brotli) library (optional, not
required), e.g. with `sudo apt-get install -y libbrotli-dev`. To search
`.bz3` files, install the [bzip3](https://github.com/kspalaiologos/bzip3)
library (optional, not required), e.g. with `sudo apt-get install -y bzip3`.
>[!TIP]
>Even if your system has command line utilities, such as `bzip2`, that
>does not necessarily mean that the development libraries such as `libbz2` are
>installed. The *development libraries* should be installed.
>
>Some Linux systems may not be configured to load dynamic libraries from
>`/usr/local/lib`, causing a library load error when running `ugrep`. To
>correct this, add `export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib"`
>to your `~/.bashrc` file. Or run `sudo ldconfig /usr/local/lib`.
### Other platforms: step 3 build
Execute the `./build.sh` script to build `ugrep`:
$ cd ugrep
$ ./build.sh
This builds the `ugrep` executable in the `ugrep/src` directory with
`./configure` and `make -j`, verified with `make test`. When all tests pass,
the `ugrep` executable is copied to `ugrep/bin/ugrep` and the symlink
`ugrep/bin/ug -> ugrep/bin/ugrep` is added for the `ug` command.
Note that `ug` is the same as `ugrep` but also loads the configuration file
.ugrep when present in the working directory or home directory. This means
that you can define your default options for `ug` in .ugrep.
Alternative paths to installed or local libraries may be specified with
`./build.sh`. To get help on the available build options:
$ ./build.sh --help
You can build static executables by specifying:
$ ./build.sh --enable-static
This may fail if libraries don't link statically, such as brotli. In that case
try `./build.sh --enable-static --without-brotli`.
You can build `ugrep` with customized defaults enabled, such as a pager:
$ ./build.sh --enable-pager
Options to select defaults for builds include:
- `--help` display build options
- `--enable-static` build static executables, if possible
- `--enable-hidden` always search hidden files and directories
- `--enable-pager` always use a pager to display output on terminals
- `--enable-pretty` colorize output to terminals and add filename headings
- `--disable-auto-color` disable automatic colors, requires ugrep option `--color=auto` to show colors
- `--disable-mmap` disable memory mapped files
- `--disable-sse2` disable SSE2 and AVX optimizations
- `--disable-avx2` disable AVX2 and AVX512BW optimizations, but compile with SSE2 when supported
- `--disable-neon` disable ARM NEON/AArch64 optimizations
- `--with-grep-path` the default `-f` path if `GREP_PATH` is not defined
- `--with-grep-colors` the default colors if `GREP_COLORS` is not defined
After the build completes, copy `ugrep/bin/ugrep` and `ugrep/bin/ug` to a
convenient location, for example in your `~/bin` directory. Or, if you may want
to install the `ugrep` and `ug` commands and man pages:
$ sudo make install
This also installs the pattern files with predefined patterns for option `-f`
at `/usr/local/share/ugrep/patterns/`. Option `-f` first checks the working
directory for the presence of pattern files, if not found checks environment
variable `GREP_PATH` to load the pattern files, and if not found reads the
installed predefined pattern files.
### Troubleshooting
#### Git and timestamps
Unfortunately, git clones do not preserve timestamps which means that you may
run into "WARNING: 'aclocal-1.15' is missing on your system." or that
autoheader was not found when running `make`.
To work around this problem, run:
$ autoreconf -fi
$ ./build.sh
#### Compiler warnings
GCC 8 and greater may produce warnings of the sort *"note: parameter passing
for argument ... changed in GCC 7.1"*. These warnings should be ignored.
### Dockerfile for developers
A Dockerfile is included to build `ugrep` in a Ubuntu container.
Developers may want to use sanitizers to verify the **ugrep** code when making
significant changes, for example to detect data races with the
[ThreadSanitizer](https://clang.llvm.org/docs/ThreadSanitizer.html):
$ ./build.sh CXXFLAGS='-fsanitize=thread -O1 -g'
We checked `ugrep` with the clang AddressSanitizer, MemorySanitizer,
ThreadSanitizer, and UndefinedBehaviorSanitizer. These options incur
significant runtime overhead and should not be used for the final build.
🔝 [Back to table of contents](#toc)
Performance comparisons
-----------------------
Please note that the **ugrep** and **ug** commands search binary files by
default and do not ignore .gitignore specified files, which will not make
recursive search performance comparisons meaningful unless options `-I` and
`--ignore-files` are used. To make these options the default for **ug**,
simply add `ignore-binary` and `ignore-files` to your .ugrep configuration
file.
For an up-to-date performance comparison of the latest ugrep, please see the
[ugrep performance benchmarks](https://github.com/Genivia/ugrep-benchmarks).
Ugrep is faster than GNU grep, Silver Searcher, ack, sift. Ugrep's speed beats
ripgrep in most benchmarks.
Using ugrep within Vim
----------------------
First, let's define the `:grep` command in Vim to search files recursively. To
do so, add the following lines to your `.vimrc` located in the root directory:
if executable('ugrep')
set grepprg=ugrep\ -RInk\ -j\ -u\ --tabs=1\ --ignore-files
set grepformat=%f:%l:%c:%m,%f+%l+%c+%m,%-G%f\\\|%l\\\|%c\\\|%m
endif
This specifies `-j` [case insensitive searches](#case) with the Vim `:grep`
command. For case sensitive searches, remove `\ -j` from `grepprg`. Multiple
matches on the same line are listed in the quickfix window separately. If this
is not desired, remove `\ -u` from `grepprg`. With this change, only the first
match on a line is shown. Option `--ignore-files` skips files specified in
`.gitignore` files, when present. To limit the depth of recursive searches to
the current directory only, append `\ -1` to `grepprg`.
You can now invoke the Vim `:grep` command in Vim to search files on a
specified `PATH` for `PATTERN` matches:
:grep PATTERN [PATH]
If you omit `PATH`, then the working directory is searched. Use `%` as `PATH`
to search only the currently opened file in Vim:
:grep PATTERN %
The `:grep` command shows the results in a
[quickfix](http://vimdoc.sourceforge.net/htmldoc/quickfix.html#:grep) window
that allows you to quickly jump to the matches found.
To open a quickfix window with the latest list of matches:
:copen
Double-click on a line in this window (or select a line and press ENTER) to
jump to the file and location in the file of the match. Enter commands `:cn`
and `:cp` to jump to the next or previous match, respectively. To update the
search results in the quickfix window, just grep them. For example, to
recursively search C++ source code marked `FIXME` in the working directory:
:grep -tc++ FIXME
To close the quickfix window:
:cclose
You can use **ugrep** options with the `:grep` command, for example to
select single- and multi-line comments in the current file:
:grep -f c++/comments %
Only the first line of a multi-line comment is shown in quickfix, to save
space. To show all lines of a multi-line match, remove `%-G` from
`grepformat`.
A popular Vim tool is [ctrlp.vim](http://kien.github.io/ctrlp.vim), which is
installed with:
$ cd ~/.vim
$ git clone https://github.com/kien/ctrlp.vim.git bundle/ctrlp.vim
CtrlP uses **ugrep** by adding the following lines to your `.vimrc`:
if executable('ugrep')
set runtimepath^=~/.vim/bundle/ctrlp.vim
let g:ctrlp_match_window='bottom,order:ttb'
let g:ctrlp_user_command='ugrep "" %s -Rl -I --ignore-files -3'
endif
where `-I` skips binary files, option `--ignore-files` skips files specified in
`.gitignore` files, when present, and option `-3` restricts searching
directories to three levels (the working directory and up to two levels below).
Start Vim then enter the command:
:helptags ~/.vim/bundle/ctrlp.vim/doc
To view the CtrlP documentation in Vim, enter the command:
:help ctrlp.txt
🔝 [Back to table of contents](#toc)
Using ugrep within Emacs
------------------------
Thanks to [Manuel Uberti](https://github.com/emacs-mirror/emacs/commits?author=manuel-uberti),
you can now use **ugrep** in Emacs. To use **ugrep** instead of GNU grep
within Emacs, add the following line to your `.emacs.d/init.el` file:
(setq-default xref-search-program ‘ugrep)
This means that Emacs commands such as `project-find-regexp` that rely on
[Xref](https://www.gnu.org/software/emacs/manual/html_node/emacs/Xref.html) can
now leverage the power of **ugrep**.
Furthermore, it is possible to use `grep` in the [Emacs grep
commands](https://www.gnu.org/software/emacs/manual/html_node/emacs/Grep-Searching.html).
For instance, you can run `lgrep` with `ugrep` by customizing `grep-template`
to something like the following:
(setq-default grep-template "ugrep --color=always -0Iinr -e ")
If you do not have Emacs version 29 (or greater) you can download and build
Emacs from the [Emacs master branch](https://github.com/emacs-mirror/emacs),
or enable Xref integration with **ugrep** manually:
(with-eval-after-load 'xref
(push '(ugrep . "xargs -0 ugrep --null -ns -e ")
xref-search-program-alist)
(setq-default xref-search-program 'ugrep))
🔝 [Back to table of contents](#toc)
Using ugrep to replace GNU/BSD grep
-----------------------------------
**ugrep** supports all standard GNU/BSD grep command-line options and improves
many of them too. See [notable improvements over grep](#improvements).
In fact, executing `ugrep` with options `-U`, `-Y`, `-.` and `--sort` makes it
behave like `egrep`, permitting empty patterns to match and search hidden files
instead of ignoring them. See [grep equivalence](#equivalence).
- You can create [convenient grep aliases](#aliases) with or without options
`-Y`, `-.` and `--sort` or include other options as desired. If you really
must stick exactly to GNU/BSD grep ASCII/LATIN1 patterns, use options `-U`
and `--grep` to disable Unicode pattern matching and to reassign options `-z`
and `-Z` to `--null-data` and `--null`, respectively.
- You can also create `grep`, `egrep` and `fgrep` executables by symlinking or
copying `ugrep` to those names. When the `ugrep` (or `ugrep.exe`) executable
is copied as `grep` (`grep.exe`), `egrep` (`egrep.exe`), `fgrep`
(`fgrep.exe`), then options `-Y` and `-.` are automatically enabled together
with either `-G` for `grep`, `-E` for `egrep` and `-F` for `fgrep`. In
addition, when copied as `zgrep`, `zegrep` and `zfgrep`, option
`--decompress` is enabled. For example, when `ugrep` is copied as `zegrep`,
options `--decompress`, `-E`, `-Y`, `-.` and `--sort` are enabled.
- Likewise, symlinks and hard links can be used to create `grep`, `egrep` and
`fgrep` replacements in the usual installation directories. For example:
sudo ln -s `which ugrep` /opt/local/bin/grep
sudo ln -s `which ugrep` /opt/local/bin/egrep
sudo ln -s `which ugrep` /opt/local/bin/fgrep
sudo ln -s `which ugrep` /opt/local/bin/zgrep
sudo ln -s `which ugrep` /opt/local/bin/zegrep
sudo ln -s `which ugrep` /opt/local/bin/zfgrep
The `/opt/local/bin` here is an example and may or may not be in your `$path`
and may or may not be found, so please adjust as necessary. **Caution:**
*bash does not obey the linked name when executing the program, reverting to
the name `ugrep` instead, which negates all internal compatibility settings.
To avoid this, copy the executables instead of linking!*
When linking or copying `ugrep` to `grep`, `egrep`, `fgrep`, `zgrep`, `zegrep`,
`zfgrep`, options `-z` and `-Z` are reassigned for compatibility to GNU/BSD
grep options `--null-data` and `--null`, respectively.
### Equivalence to GNU/BSD grep
When the `ugrep` executable file is symlinked or copied to `grep`, `egrep`,
`fgrep`, `zgrep`, `zegrep` and `zfgrep` executables, then those executables
will behave as GNU grep equivalents. This behavior is implicit and automatic,
essentially using the following translations:
grep = ugrep -G -Y -. --sort
egrep = ugrep -E -Y -. --sort
fgrep = ugrep -F -Y -. --sort
zgrep = ugrep -z -G -Y -. --sort
zegrep = ugrep -z -E -Y -. --sort
zfgrep = ugrep -z -F -Y -. --sort
please note that:
- `-Y` enables empty matches, so for example the pattern `a*` matches every
line instead of a sequence of `a`'s. By default in ugrep, the pattern `a*`
matches a sequence of `a`'s. Moreover, in ugrep the pattern `a*b*c*` matches
what it is supposed to match by default. See [improvements](#improvements).
- `-.` searches hidden files (dotfiles). By default, hidden files are ignored,
like most Unix utilities.
- `--sort` specifies output sorted by pathname, showing sorted matching files
first followed by sorted recursive matches in subdirectories. Otherwise,
matching files are reported in no particular order to improve performance;
- options `-z` and `-Z` are reassigned to `--null-data` and `--null` and no
longer enable `--decompress` and `--fuzzy` searching modes.
There is one minor difference with GNU/BSD grep:
- GNU/BSD grep defaults to `-Dread` and `-dread` which are not recommended, see
[improvements](#improvements) for an explanation.
🔝 [Back to table of contents](#toc)
### Short and quick command aliases
Commonly-used aliases to add to `.bashrc` to increase productivity:
alias uq = 'ug -Q' # interactive TUI search (uses .ugrep config)
alias uz = 'ug -z' # compressed files and archives search (uses .ugrep config)
alias ux = 'ug -U --hexdump' # binary pattern search (uses .ugrep config)
alias ugit = 'ug -R --ignore-files' # works like git-grep & define your preferences in .ugrep config
alias grep = 'ug -G' # search with basic regular expressions (BRE) like grep
alias egrep = 'ug -E' # search with extended regular expressions (ERE) like egrep
alias fgrep = 'ug -F' # find string(s) like fgrep
alias zgrep = 'ug -zG' # search compressed files and archives with BRE
alias zegrep = 'ug -zE' # search compressed files and archives with ERE
alias zfgrep = 'ug -zF' # find string(s) in compressed files and/or archives
alias xdump = 'ugrep -X ""' # hexdump files without searching (don't use .ugrep config)
alias zmore = 'ugrep+ -z -I -+ --pager ""' # view compressed, archived and regular files (don't use .ugrep config)
🔝 [Back to table of contents](#toc)
### Notable improvements over grep
- **ugrep** starts an interactive query TUI with option `-Q`.
- **ugrep** matches patterns across multiple lines when patterns match `\n`.
- **ugrep** matches full Unicode by default (disabled with option `-U`).
- **ugrep** supports Boolean patterns with AND, OR and NOT (option `--bool`).
- **ugrep** supports gitignore with option `--ignore-files`.
- **ugrep** supports fuzzy (approximate) matching with option `-Z`.
- **ugrep** supports user-defined global and local configuration files.
- **ugrep** searches compressed files and archives with option `-z`.
- **ugrep** searches cpio, jar, pax, tar, zip and 7z archives with option `-z`.
- **ugrep** searches cpio, jar, pax, tar, zip and 7z archives recursively
stored within archives with `-z` and `--zmax=NUM` for up to `NUM` levels deep.
- **ugrep** searches pdf, doc, docx, xls, xlsx, epub, and more with `--filter`
using third-party format conversion utilities as plugins.
- **ugrep** searches a directory when the FILE argument is a directory, like
most Unix/Linux utilities; use option `-r` to search directories recursively.
- **ugrep** does not match hidden files by default like most Unix/Linux
utilities (hidden dotfile file matching is enabled with `-.`).
- **ugrep** regular expression patterns are more expressive than GNU grep and
BSD grep POSIX ERE and support Unicode pattern matching. Extended regular
expression (ERE) syntax is the default (i.e. option `-E` as egrep, whereas
`-G` enables BRE).
- **ugrep** spawns threads to search files concurrently to improve search
speed (disabled with option `-J1`).
- **ugrep** produces hexdumps with `-W` (output binary matches in hex with text
matches output as usual) and `-X` (output all matches in hex).
- **ugrep** can output matches in JSON, XML, CSV and user-defined formats (with
option `--format`).
- **ugrep** option `-f` uses `GREP_PATH` environment variable or the predefined
patterns installed in `/usr/local/share/ugrep/patterns`. If `-f` is
specified and also one or more `-e` patterns are specified, then options
`-F`, `-x`, and `-w` do not apply to `-f` patterns. This is to avoid
confusion when `-f` is used with predefined patterns that may no longer work
properly with these options.
- **ugrep** options `-O`, `-M`, and `-t` specify file extensions, file
signature magic byte patterns, and predefined file types, respectively. This
allows searching for certain types of files in directory trees, for example
with recursive search options `-R` and `-r`. Options `-O`, `-M`, and `-t`
also applies to archived files in cpio, jar, pax, tar, zip and 7z files.
- **ugrep** option `-k`, `--column-number` to display the column number, taking
tab spacing into account by expanding tabs, as specified by option `--tabs`.
- **ugrep** option `-P` (Perl regular expressions) supports backreferences
(with `--format`) and lookbehinds, which uses the PCRE2 or Boost.Regex
library for fast Perl regex matching with a PCRE-like syntax.
- **ugrep** option `-b` with option `-o` or with option `-u`, ugrep displays
the exact byte offset of the pattern match instead of the byte offset of the
start of the matched line reported by GNU/BSD grep.
- **ugrep** option `-u`, `--ungroup` to not group multiple matches per line.
This option displays a matched input line again for each additional pattern
match on the line. This option is particularly useful with option `-c` to
report the total number of pattern matches per file instead of the number of
lines matched per file.
- **ugrep** option `-Y` enables matching empty patterns. Grepping with
empty-matching patterns is weird and gives different results with GNU grep
versus BSD grep. Empty matches are not output by **ugrep** by default, which
avoids making mistakes that may produce "random" results. For example, with
GNU/BSD grep, pattern `a*` matches every line in the input, and actually
matches `xyz` three times (the empty transitions before and between the `x`,
`y`, and `z`). Allowing empty matches requires **ugrep** option `-Y`.
Patterns that start with `^` or end with `$`, such as `^\h*$`, match empty.
These patterns automatically enable option `-Y`.
- **ugrep** option `-D, --devices=ACTION` is `skip` by default, instead of
`read`. This prevents unexpectedly hanging on named pipes in directories
that are recursively searched, as may happen with GNU/BSD grep that `read`
devices by default.
- **ugrep** option `-d, --directories=ACTION` is `skip` by default, instead of
`read`. By default, directories specified on the command line are searched,
but not recursively deeper into subdirectories.
- **ugrep** offers *negative patterns* `-N PATTERN`, which are patterns of the
form `(?^X)` that skip all `X` input, thus removing `X` from the search.
For example, negative patterns can be used to skip strings and comments when
searching for identifiers in source code and find matches that aren't in
strings and comments. Predefined `zap` patterns use negative patterns, for
example, use `-f cpp/zap_comments` to ignore pattern matches in C++ comments.
- **ugrep** ignores the `GREP_OPTIONS` environment variable, because the
behavior of **ugrep** must be portable and predictable on every system. Also
GNU grep abandoned `GREP_OPTIONS` for this reason. Please use the `ug`
command that loads the .ugrep configuration file located in the working
directory or in the home directory when present, or use shell aliases to
create new commands with specific search options.
🔝 [Back to table of contents](#toc)
Tutorial
--------
### Examples
To perform a search using a configuration file `.ugrep` placed in the working
directory or home directory (note that `ug` is the same as `ugrep --config`):
ug PATTERN FILE...
To save a `.ugrep` configuration file to the working directory, then edit this
file in your home directory to customize your preferences for `ug` defaults:
ug --save-config
To search the working directory and recursively deeper for `main` (note that
`-r` recurse symlinks is enabled by default if no file arguments are
specified):
ug main
Same, but only search C++ source code files recursively, ignoring all other
files:
ug -tc++ main
Same, using the interactive query TUI, starting with the initial search pattern
`main` (note that `-Q` with an initial pattern requires option `-e` because
patterns are normally specified interactively and all command line arguments
are considered files/directories):
ug -Q -tc++ -e main
To search for `#define` (and `# define` etc) using a regex pattern in C++ files
(note that patterns should be quoted to prevent shell globbing of `*` and `?`):
ug -tc++ '#[\t ]*define'
To search for `main` as a word (`-w`) recursively without following symlinks
(`-r`) in directory `myproject`, showing the matching line (`-n`) and column
(`-k`) numbers next to the lines matched:
ug -r -nkw main myproject
Same, but only search `myproject` without recursing deeper (note that directory
arguments are searched at one level by default):
ug -nkw main myproject
Same, but search `myproject` and one subdirectory level deeper (two levels)
with `-2`:
ug -2 -nkw main myproject
Same, but only search C++ files in `myproject` and its subdirectories with
`-tc++`:
ug -tc++ -2 -nkw main myproject
Same, but also search inside archives (e.g. zip and tar files) and compressed
files with `-z`:
ug -z -tc++ -2 -nkw main myproject
Search recursively the working directory for `main` while ignoring gitignored
files (e.g. assuming `.gitignore` is in the working directory or below):
ug --ignore-files -tc++ -nkw main
To list all files in the working directory and deeper that are not ignored by
`.gitignore` file(s):
ug --ignore-files -l ''
To display the list of file name extensions and "magic bytes" (shebangs)
that are searched corresponding to `-t` arguments:
ug -tlist
To list all shell files recursively, based on extensions and shebangs with `-l`
(note that `''` matches any non-empty file):
ug -l -tShell ''
🔝 [Back to table of contents](#toc)
### Advanced examples
To search for `main` in source code while ignoring strings and comment blocks
you can use *negative patterns* with option `-N` to skip unwanted matches in
C/C++ quoted strings and comment blocks:
ug -r -nkw -e 'main' -N '"(\\.|\\\r?\n|[^\\\n"])*"|//.*|/\*(.*\n)*?.*\*+\/' myproject
This is a lot of work to type in correctly! If you are like me, I don't want
to spend time fiddling with regex patterns when I am working on something more
important. There is an easier way by using **ugrep**'s predefined patterns
(`-f`) that are installed with the `ugrep` tool:
ug -r -nkw 'main' -f c/zap_strings -f c/zap_comments myproject
This query also searches through other files than C/C++ source code, like
READMEs, Makefiles, and so on. We're also skipping symlinks with `-r`. So
let's refine this query by selecting C/C++ files only using option `-tc,c++`
and include symlinks to files and directories with `-R`:
ug -R -tc,c++ -nkw 'main' -f c/zap_strings -f c/zap_comments myproject
What if you only want to look for the identifier `main` but not as a function
`main(`? In this case, use a negative pattern for this to skip unwanted
`main\h*(` pattern matches:
ug -R -tc,c++ -nkw -e 'main' -N 'main\h*\(' -f c/zap_strings -f c/zap_comments myproject
This uses the `-e` and `-N` options to explicitly specify a pattern and a
negative pattern, respectively, which is essentially forming the pattern
`main|(?^main\h*\()`, where `\h` matches space and tab. In general, negative
patterns are useful to filter out pattern matches that we are not interested
in.
As another example, let's say we may want to search for the word `FIXME` in
C/C++ comment blocks. To do so we can first select the comment blocks with
**ugrep**'s predefined `c/comments` pattern AND THEN select lines with `FIXME`
using a pipe:
ug -R -tc,c++ -nk -f c/comments myproject | ug -w 'FIXME'
Filtering results with pipes is generally easier than using AND-OR logic that
some search tools use. This approach follows the Unix spirit to keep utilities
simple and use them in combination for more complex tasks.
Let's produce a sorted list of all identifiers found in Java source code while
skipping strings and comments:
ug -R -tjava -f java/names myproject | sort -u
This matches Java Unicode identifiers using the regex
`\p{JavaIdentifierStart}\p{JavaIdentifierPart}*` defined in
`patterns/java/names`.
With traditional grep and grep-like tools it takes great effort to recursively
search for the C/C++ source file that defines function `qsort`, requiring
something like this:
ug -R --include='*.c' --include='*.cpp' '^([ \t]*[[:word:]:*&]+)+[ \t]+qsort[ \t]*\([^;\n]+$' myproject
Fortunately, with **ugrep** we can simply select all function definitions in
files with extension `.c` or `.cpp` by using option `-Oc,cpp` and by using a
predefined pattern `functions` that is installed with the tool to produce
all function definitions. Then we select the one we want:
ug -R -Oc,cpp -nk -f c/functions | ug 'qsort'
Note that we could have used `-tc,c++` to select C/C++ files, but this also
includes header files when we want to only search `.c` and `.cpp` files.
We can also skip files and directories from being searched that are defined in
`.gitignore`. To do so we use `--ignore-files` to exclude any files and
directories from recursive searches that match the globs in `.gitignore`, when
one or more `.gitignore` files are found:
ug -R -tc++ --ignore-files -f c++/defines
This searches C++ files (`-tc++`) in the working directory for `#define`
lines (`-f c++/defines`), while skipping files and directories declared in
`.gitignore`. If you find this too long to type then define an alias to search
GitHub directories:
alias ugit='ugrep -R --ignore-files'
ugit -tc++ -f c++/defines
To highlight matches when pushed through a chain of pipes we should use
`--color=always`:
ugit --color=always -tc++ -f c++/defines | ugrep -w 'FOO.*'
This returns a color-highlighted list of all `#define FOO...` macros in C/C++
source code files, skipping files defined in `.gitignore`.
Note that the complement of `--exclude` is not `--include`, because exclusions
always take precedence over inclusions, so we cannot reliably list the files
that are ignored with `--include-from='.gitignore'`. Only files explicitly
specified with `--include` and directories explicitly specified with
`--include-dir` are visited. The `--include-from` from lists globs that are
considered both files and directories to add to `--include` and
`--include-dir`, respectively. This means that when directory names and
directory paths are not explicitly listed in this file then it will not be
visited using `--include-from`.
Because ugrep checks if the input is valid UTF-encoded Unicode (unless `-U` is
used), it is possible to use it as a filter to ignore non-UTF output produced
by a program:
program | ugrep -I ''
If the program produces valid output then the output is passed through,
otherwise the output is filtered out option `-I`. If the output is initially
valid for a very large portion but is followed by invalid output, then ugrep
may initially show the output up to but excluding the invalid output after
which further output is blocked.
To filter lines that are valid ASCII or UTF-encoded, while removing lines that
are not:
program | ugrep '[\p{Unicode}--[\n]]+'
Note that `\p{Unicode}` matches `\n` but we don't want to matche the whole
file! Just lines with `[\p{Unicode}--[\n]]+`.
🔝 [Back to table of contents](#toc)
### Displaying helpful info
The ugrep man page:
man ugrep
To show a help page:
ug --help
To show options that mention `WHAT`:
ug --help WHAT
To show a list of `-t TYPES` option values:
ug -tlist
In the interactive query TUI, press F1 or CTRL-Z for help and options:
ug -Q
🔝 [Back to table of contents](#toc)
### Configuration files
--config[=FILE], ---[FILE]
Use configuration FILE. The default FILE is `.ugrep'. The working
directory is checked first for FILE, then the home directory. The
options specified in the configuration FILE are parsed first,
followed by the remaining options specified on the command line.
The ug command automatically loads a `.ugrep' configuration file,
unless --config=FILE or --no-config is specified.
--no-config
Do not load the default .ugrep configuration file.
--save-config[=FILE] [OPTIONS]
Save configuration FILE to include OPTIONS. Update FILE when
first loaded with --config=FILE. The default FILE is `.ugrep',
which is automatically loaded by the ug command. When FILE is a
`-', writes the configuration to standard output. Only part of the
OPTIONS are saved that do not cause searches to fail when combined
with other options. Additional options may be specified by editing
the saved configuration file. A configuration file may be modified
manually to specify one or more config[=FILE] to indirectly load
the specified FILEs, but recursive config loading is not allowed.
#### The ug command versus the ugrep command
The `ug` command is intended for context-dependent interactive searching and is
equivalent to the `ugrep --config` command to load the configuration file
`.ugrep` when present in the working directory or, when not found, in the home
directory:
ug PATTERN ...
ugrep --config PATTERN ...
The `ug` command also sorts files by name per directory searched. A
configuration file contains `NAME=VALUE` pairs per line, where `NAME` is the
name of a long option (without `--`) and `=VALUE` is an argument, which is
optional and may be omitted depending on the option. Empty lines and lines
starting with a `#` are ignored:
# Color scheme
colors=cx=hb:ms=hiy:mc=hic:fn=hi+y+K:ln=hg:cn=hg:bn=hg:se=
# Disable searching hidden files and directories
no-hidden
# ignore files specified in .ignore and .gitignore in recursive searches
ignore-files=.ignore
ignore-files=.gitignore
Command line options are parsed in the following order: first the (default or
named) configuration file is loaded, then the remaining options and
arguments on the command line are parsed.
Option `--stats` displays the configuration file used after searching.
#### Named configuration files
Named configuration files are intended to streamline custom search tasks, by
reducing the number of command line options to just one `---FILE` to use the
collection of options specified in `FILE`. The `--config=FILE` option and its
abbreviated form `---FILE` load the specified configuration file located in the
working directory or, when not found, located in the home directory:
ug ---FILE PATTERN ...
ugrep ---FILE PATTERN ...
An error is produced when `FILE` is not found or cannot be read.
Named configuration files can be used to define a collection of options that
are specific to the requirements of a task in the development workflow of a
project. For example to report unresolved issues by checking the source code
and documentation for comments with FIXME and TODO items. Such named
configuration file can be localized to a project by placing it in the project
directory, or it can be made global by placing it in the home directory. For
visual feedback, a color scheme specific to this task can be specified with
option `colors` in the configuration `FILE` to help identify the output
produced by a named configuration as opposed to the default configuration.
#### Saving a configuration file
The `--save-config` option saves a `.ugrep` configuration file to the working
directory using the current configuration loaded with `--config`. This saves
the current configuration combined with additional options when specified also.
Only those options that cannot conflict with other options and options that
cannot negatively impact search results will be saved.
The `--save-config=FILE` option saves the configuration to the specified `FILE`.
The configuration is written to standard output when `FILE` is a `-`.
Alternatively, a configuration file may be manually created or modified. A
configuration file may include one or more `config[=FILE]` to indirectly load
the specfified `FILE`, but recursive config loading is prohibited. The
simplest way to manuall create a configuration file is to specify `config` at
the top of the file, followed by the long options to override the defaults.
🔝 [Back to table of contents](#toc)
### Interactive search with -Q
-Q[=DELAY], --query[=DELAY]
Query mode: start a TUI to perform interactive searches. This mode
requires an ANSI capable terminal. An optional DELAY argument may
be specified to reduce or increase the response time to execute
searches after the last key press, in increments of 100ms, where
the default is 3 (300ms delay). No whitespace may be given between
-Q and its argument DELAY. Initial patterns may be specified with
-e PATTERN, i.e. a PATTERN argument requires option -e. Press F1
or CTRL-Z to view the help screen. Press F2 or CTRL-Y to invoke a
command to view or edit the file shown at the top of the screen.
The command can be specified with option --view, or defaults to
environment variable PAGER when defined, or EDITOR. Press Tab and
Shift-Tab to navigate directories and to select a file to search.
Press Enter to select lines to output. Press ALT-l for option -l
to list files, ALT-n for -n, etc. Non-option commands include
ALT-] to increase context. See also options --no-confirm, --delay,
--split and --view.
--no-confirm
Do not confirm actions in -Q query TUI. The default is confirm.
--delay=DELAY
Set the default -Q key response delay. Default is 3 for 300ms.
--split
Split the -Q query TUI screen on startup.
--view[=COMMAND]
Use COMMAND to view/edit a file in -Q query TUI by pressing CTRL-Y.
This option starts a user interface to enter search patterns interactively:
- Press F1 or CTRL-Z to view a help screen and to enable or disable options.
- Press Alt with a key corresponding to a ugrep option letter or digit to
enable or disable the ugrep option. For example, pressing Alt-c enables
option `-c` to count matches. Pressing Alt-c again disables `-c`. Options
can be toggled with the Alt key while searching or when viewing the help
screen. If Alt/Meta keys are not working (e.g. X11 xterm), then press
CTRL-O followed by the key corresponding to the option. Alt keys may work
in xterm by adding `xterm*metaSendsEscape: true` to ~/.Xdefaults`.
- Press Alt-g to enter or edit option `-g` file and directory matching globs, a
comma-separated list of gitignore-style glob patterns. Presssing ESC returns
control to the query pattern prompt (the globs are saved). When a glob is
preceded by a `!` or a `^`, skips files whose name matches the glob When a
glob contains a `/`, full pathnames are matched. Otherwise basenames are
matched. When a glob ends with a `/`, directories are matched.
- The query TUI prompt switches between `Q>` (normal), `F>` (fixed strings),
`G>` (basic regex), `P>` (Perl matching), and `Z>` (fuzzy matching).
When the `--glob=` prompt is shown, a comma-separated list of gitignore-style
glob patterns may be entered. Presssing ESC returns control to the pattern
prompt.
- Press CTRL-T to split the TUI screen to preview a file in the bottom pane.
- Press CTRL-Y to view a file with a pager specified with `--view`.
- Press Enter to switch to selection mode to select lines to output when ugrep
exits. Normally, ugrep in query mode does not output any results unless
results are selected. While in selection mode, select or deselect lines with
Enter or Del, or press A to select all results.
- The file listed or shown at the top of the screen, or beneath the cursor in
selection mode, is edited by pressing F2 or CTRL-Y. A file viewer or editor
may be specified with `--view=COMMAND`. Otherwise, the `PAGER` or `EDITOR`
environment variables are used to invoke the command with CTRL-Y. Filenames
must be enabled and visible in the output to use this feature.
- Press TAB to chdir one level down into the directory of the file listed
or viewed at the top of the screen. If no directory exists, the file itself
is selected to search. Press Shift-TAB to go back up one level.
- Press CTRL-] to toggle colors on and off. Normally ugrep in query mode uses
colors and other markup to highlight results. When colors are turned off,
selected results are also not colored in the output produced by ugrep when
ugrep exits. When colors are turned on (the default), selected results are
colored depending on the `--color` option.
- The query engine is optimized to limit system load by performing on-demand
searches to produce results only for the visible parts shown in the
interface. That is, results are shown on demand, when scrolling down and
when exiting when all results are selected. When the search pattern is
modified, the previous search query is cancelled when incomplete. This
effectively limits the load on the system to maintain a high degree of
responsiveness of the query engine to user input. Because the search results
are produced on demand, occasionally you may notice a flashing "Searching..."
message when searching files.
- To display results faster, specify a low `DELAY` value such as 1. However,
lower values may increase system load as a result of repeatedly initiating
and cancelling searches by each key pressed.
- To avoid long pathnames to obscure the view, `--heading` is enabled by
default. Press Alt-+ to switch headings off.
Query TUI key mapping:
key(s) | function
---------------- | -------------------------------------------------
`Alt-key` | toggle ugrep command-line option corresponding to `key`
`Alt-/`xxxx`/` | insert Unicode hex code point U+xxxx
`Esc` `Ctrl-C` | go back or exit
`Ctrl-Q` | quick exit and output the results selected in selection mode
`Tab` | chdir to the directory of the file shown at the top of the screen or select file
`Shift-Tab` | chdir one level up or deselect file
`Enter` | enter selection mode and toggle selected lines to output on exit
`Up` `Ctrl-P` | move up
`Down` `Ctrl-N` | move down
`Left` `Ctrl-B` | move left
`Right` `Ctrl-F` | move right
`PgUp` `Ctrl-G` | move display up by a page
`PgDn` `Ctrl-D` | move display down by a page
`Alt-Up` | move display up by 1/2 page (MacOS `Shift-Up`)
`Alt-Down` | move display down by 1/2 page (MacOS `Shift-Down`)
`Alt-Left` | move display left by 1/2 page (MacOS `Shift-Left`)
`Alt-Right` | move display right by 1/2 page (MacOS `Shift-Right`)
`Home` `Ctrl-A` | move cursor to the beginning of the line
`End` `Ctrl-E` | move cursor to the end of the line
`Ctrl-K` | delete after cursor
`Ctrl-L` | refresh screen
`Ctrl-O`+`key` | toggle ugrep command-line option corresponding to `key`, same as `Alt-key`
`Ctrl-R` `F4` | jump to bookmark
`Ctrl-S` | jump to the next dir/file/context
`Ctrl-T` `F5` | toggle split screen (`--split` starts a split-screen TUI)
`Ctrl-U` | delete before cursor
`Ctrl-V` | verbatim character
`Ctrl-W` | jump back one dir/file/context
`Ctrl-X` `F3` | set bookmark
`Ctrl-Y` `F2` | view or edit the file shown at the top of the screen
`Ctrl-Z` `F1` | view help and options
`Ctrl-^` | chdir back to the starting working directory
`Ctrl-]` | toggle color/mono
`Ctrl-\` | terminate process
To interactively search the files in the working directory and below:
ug -Q
Same, but restricted to C++ files only and ignoring `.gitignore` files:
ug -Q -tc++ --ignore-files
To interactively search all makefiles in the working directory and below:
ug -Q -g 'Makefile*' -g 'makefile*'
Same, but for up to 2 directory levels (working and one subdirectory level):
ug -Q -2 -g 'Makefile*' -g 'makefile*'
To interactively view the contents of `main.cpp` and search it, where `-y`
shows any nonmatching lines as context:
ug -Q -y main.cpp
To interactively search `main.cpp`, starting with the search pattern `TODO` and
a match context of 5 lines (context can be interactively enabled and disabled,
this also overrides the default context size of 2 lines):
ug -Q -C5 -e TODO main.cpp
To view and search the contents of an archive (e.g. zip, tarball):
ug -Q -z archive.tar.gz
To interactively select files from `project.zip` to decompress with `unzip`,
using ugrep query selection mode (press Enter to select lines):
unzip project.zip `zipinfo -1 project.zip | ugrep -Q`
🔝 [Back to table of contents](#toc)
### Recursively list matching files with -l, -R, -r, -S, --depth, -g, -O, and -t
-L, --files-without-match
Only the names of files not containing selected lines are written
to standard output. Pathnames are listed once per file searched.
If the standard input is searched, the string ``(standard input)''
is written.
-l, --files-with-matches
Only the names of files containing selected lines are written to
standard output. ugrep will only search a file until a match has
been found, making searches potentially less expensive. Pathnames
are listed once per file searched. If the standard input is
searched, the string ``(standard input)'' is written.
-R, --dereference-recursive
Recursively read all files under each directory. Follow all
symbolic links to files and directories, unlike -r.
-r, --recursive
Recursively read all files under each directory, following symbolic
links only if they are on the command line. Note that when no FILE
arguments are specified and input is read from a terminal,
recursive searches are performed as if -r is specified.
-S, --dereference-files
When -r is specified, symbolic links to files are followed, but not
to directories. The default is not to follow symbolic links.
--depth=[MIN,][MAX], -1, -2, -3, ... -9, -10, -11, -12, ...
Restrict recursive searches from MIN to MAX directory levels deep,
where -1 (--depth=1) searches the specified path without recursing
into subdirectories. Note that -3 -5, -3-5, and -35 search 3 to 5
levels deep. Enables -r if -R or -r is not specified.
-g GLOBS, --glob=GLOBS
Search only files whose name matches the specified comma-separated
list of GLOBS, same as --include='glob' for each `glob' in GLOBS.
When a `glob' is preceded by a `!' or a `^', skip files whose name
matches `glob', same as --exclude='glob'. When `glob' contains a
`/', full pathnames are matched. Otherwise basenames are matched.
When `glob' ends with a `/', directories are matched, same as
--include-dir='glob' and --exclude-dir='glob'. A leading `/'
matches the working directory. This option may be repeated and may
be combined with options -M, -O and -t to expand searches. See
`ugrep --help globs' and `man ugrep' section GLOBBING for details.
-O EXTENSIONS, --file-extension=EXTENSIONS
Search only files whose filename extensions match the specified
comma-separated list of EXTENSIONS, same as --include='*.ext' for
each `ext' in EXTENSIONS. When `ext' is preceded by a `!' or a
`^', skip files whose filename extensions matches `ext', same as
--exclude='*.ext'. This option may be repeated and may be combined
with options -g, -M and -t to expand the recursive search.
-t TYPES, --file-type=TYPES
Search only files associated with TYPES, a comma-separated list of
file types. Each file type corresponds to a set of filename
extensions passed to option -O and filenames passed to option -g.
For capitalized file types, the search is expanded to include files
with matching file signature magic bytes, as if passed to option
-M. When a type is preceded by a `!' or a `^', excludes files of
the specified type. This option may be repeated.
--stats
Output statistics on the number of files and directories searched,
and the inclusion and exclusion constraints applied.
If no FILE arguments are specified and input is read from a terminal, recursive
searches are performed as if `-r` is specified. To force reading from standard
input, specify `-` as the FILE argument.
To recursively list all non-empty files in the working directory:
ug -r -l ''
To list all non-empty files in the working directory but not deeper (since a
FILE argument is given, in this case `.` for the working directory):
ug -l '' .
To list all non-empty files in directory `mydir` but not deeper (since a FILE
argument is given):
ug -l '' mydir
To list all non-empty files in directory `mydir` and deeper while following
symlinks:
ug -R -l '' mydir
To recursively list all non-empty files on the path specified, while visiting
subdirectories only, i.e. directories `mydir/` and subdirectories at one
level deeper `mydir/*/` are visited (note that `-2 -l` can be abbreviated to
`-l2`):
ug -2 -l '' mydir
To recursively list all non-empty files in directory `mydir`, not following any
symbolic links (except when on the command line such as `mydir`):
ug -rl '' mydir
To recursively list all Makefiles matching the text `CPP`:
ug -l -tmake 'CPP'
To recursively list all `Makefile.*` matching `bin_PROGRAMS`:
ug -l -g'Makefile.*' 'bin_PROGRAMS'
To recursively list all non-empty files with extension .sh, with `-Osh`:
ug -l -Osh ''
To recursively list all shell scripts based on extensions and shebangs with
`-tShell`:
ug -l -tShell ''
To recursively list all shell scripts based on extensions only with `-tshell`:
ug -l -tshell ''
🔝 [Back to table of contents](#toc)
### Boolean query patterns with -%, -%%, --and, --not
--bool, -%, -%%
Specifies Boolean query patterns. A Boolean query pattern is
composed of `AND', `OR', `NOT' operators and grouping with `(' `)'.
Spacing between subpatterns is the same as `AND', `|' is the same
as `OR' and a `-' is the same as `NOT'. The `OR' operator binds
more tightly than `AND'. For example, --bool 'A|B C|D' matches
lines with (`A' or `B') and (`C' or `D'), --bool 'A -B' matches
lines with `A' and not `B'. Operators `AND', `OR', `NOT' require
proper spacing. For example, --bool 'A OR B AND C OR D' matches
lines with (`A' or `B') and (`C' or `D'), --bool 'A AND NOT B'
matches lines with `A' without `B'. Quoted subpatterns are matched
literally as strings. For example, --bool 'A "AND"|"OR"' matches
lines with `A' and also either `AND' or `OR'. Parentheses are used
for grouping. For example, --bool '(A B)|C' matches lines with `A'
and `B', or lines with `C'. Note that all subpatterns in a Boolean
query pattern are regular expressions, unless -F is specified.
Options -E, -F, -G, -P and -Z can be combined with --bool to match
subpatterns as strings or regular expressions (-E is the default.)
This option does not apply to -f FILE patterns. The double short
option -%% enables options --bool --files. Option --stats displays
the Boolean search patterns applied. See also options --and,
--andnot, --not, --files and --lines.
--files
Boolean file matching mode, the opposite of --lines. When combined
with option --bool, matches a file if all Boolean conditions are
satisfied. For example, --bool --files 'A B|C -D' matches a file
if some lines match `A', and some lines match either `B' or `C',
and no line matches `D'. See also options --and, --andnot, --not,
--bool and --lines. The double short option -%% enables options
--bool --files.
--lines
Boolean line matching mode for option --bool, the default mode.
--and [[-e] PATTERN] ... -e PATTERN
Specify additional patterns to match. Patterns must be specified
with -e. Each -e PATTERN following this option is considered an
alternative pattern to match, i.e. each -e is interpreted as an OR
pattern. For example, -e A -e B --and -e C -e D matches lines with
(`A' or `B') and (`C' or `D'). Note that multiple -e PATTERN are
alternations that bind more tightly together than --and. Option
--stats displays the search patterns applied. See also options
--not, --andnot, and --bool.
--andnot [[-e] PATTERN] ...
Combines --and --not. See also options --and, --not, and --bool.
--not [-e] PATTERN
Specifies that PATTERN should not match. Note that -e A --not -e B
matches lines with `A' or lines without a `B'. To match lines with
`A' that have no `B', specify -e A --andnot -e B. Option --stats
displays the search patterns applied. See also options --and,
--andnot, and --bool.
--stats
Output statistics on the number of files and directories searched,
and the inclusion and exclusion constraints applied.
Note that the `--and`, `--not`, and `--andnot` options require `-e PATTERN`.
The `-%` option makes all patterns Boolean-based, supporting the following
logical operations listed from the highest level of precedence to the lowest:
operator | alternative | result
-------- | ----------- | -------
`"x"` | | match `x` literally and exactly as specified (using the standard regex escapes `\Q` and `\E`)
`( )` | | Boolean expression grouping
`-x` | `NOT x` | inverted match, i.e. matches if `x` does not match
`x\|y` | `x OR y` | matches lines with `x` or `y`
`x y` | `x AND y` | matches lines with both `x` and `y`
- `x` and `y` are subpatterns that do not start with the special symbols `|`,
`-`, and `(` (use quotes or a `\` escape to match these);
- `-` and `NOT` are the same and take precedence over `OR`, which means that
`-x|y` == `(-x)|y` for example.
- `|` and `OR` are the same and take precedence over `AND`, which means that
`x y|z` == `x (y|z)` for example;
The `--stats` option displays the Boolean queries in human-readable form
converted to CNF (Conjunctive Normal Form), after the search is completed.
To show the CNF without a search, read from standard input terminated by an
EOF, like `echo | ugrep -% '...' --stats`.
Subpatterns are color-highlighted in the output, except those negated with
`NOT` (a `NOT` subpattern may still show up in a matching line when using an
OR-NOT pattern like `x|-y`). Note that subpatterns may overlap. In that
case only the first matching subpattern is color-highlighted.
Multiple lines may be matched when subpatterns match newlines. There is one
exception however: subpatterns ending with `(?=X)` lookaheads may not match
when `X` spans multiple lines.
Empty patterns match any line (grep standard). Therefore, `-% 'x|""|y'`
matches everything and `x` and `y` are not color-highlighted. Option `-y`
should be used to show every line as context, for example `-y 'x|y'`.
Fzf-like interactive querying (Boolean search with fixed strings with fuzzy
matching to allow e.g. up to 4 extra characters matched with `-Z+4` in words
with `-w`), press TAB and ALT-y to view a file with matches. Press SHIFT-TAB
and ALT-l to go back to the list of matching files:
ug -Q -%% -l -w -F -Z+4 --sort=best
To recursively find all files containing both `hot` and `dog` anywhere in the
file with option `--files`:
ug -%% 'hot dog'
ug --files -e hot --and dog
To find lines containing both `hot` and `dog` in `myfile.txt`:
ug -% 'hot dog' myfile.txt
ug -e hot --and dog myfile.txt
To find lines containing `place` and then also `hotdog` or `taco` (or both) in
`myfile.txt`:
ug -% 'hotdog|taco place' myfile.txt
ug -e hotdog -e taco --and place myfile.txt
Same, but exclude lines matching `diner`:
ug -% 'hotdog|taco place -diner' myfile.txt
ug -e hotdog -e taco --and place --andnot diner myfile.txt
To find lines with `diner` or lines that match both `fast` and `food` but not `bad` in `myfile.txt`:
ug -% 'diner|(fast food -bad)' myfile.txt
To find lines with `fast food` (exactly) or lines with `diner` but not `bad` or `old` in `myfile.txt`:
ug -% '"fast food"|diner -bad -old' myfile.txt
Same, but using a different Boolean expression that has the same meaning:
ug -% '"fast food"|diner -(bad|old)' myfile.txt
To find lines with `diner` implying `good` in `myfile.txt` (that is, show lines
with `good` without `diner` and show lines with `diner` but only those with
`good`, which is logically implied!):
ug -% 'good|-diner' myfile.txt
ug -e good --not diner myfile.txt
To find lines with `foo` and `-bar` and `"baz"` in `myfile.txt` (not that `-`
and `"` should be matched using `\` escapes and with `--and -e -bar`):
ug -% 'foo \-bar \"baz\"' myfile.txt
ug -e foo --and -e -bar --and '"baz"' myfile.txt
To search `myfile.cpp` for lines with `TODO` or `FIXME` but not both on the
same line, like XOR:
ug -% 'TODO|FIXME -(TODO FIXME)' myfile.cpp
ug -e TODO -e FIXME --and --not TODO --not FIXME myfile.cpp
🔝 [Back to table of contents](#toc)
### Search this but not that with -v, -e, -N, -f, -L, -w, -x
-e PATTERN, --regexp=PATTERN
Specify a PATTERN to search the input. An input line is selected
if it matches any of the specified patterns. This option is useful
when multiple -e options are used to specify multiple patterns, or
when a pattern begins with a dash (`-'), or to specify a pattern
after option -f or after the FILE arguments.
-f FILE, --file=FILE
Read newline-separated patterns from FILE. White space in patterns
is significant. Empty lines in FILE are ignored. If FILE does not
exist, the GREP_PATH environment variable is used as path to FILE.
If that fails, looks for FILE in /usr/local/share/ugrep/pattern.
When FILE is a `-', standard input is read. This option may be
repeated.
-L, --files-without-match
Only the names of files not containing selected lines are written
to standard output. Pathnames are listed once per file searched.
If the standard input is searched, the string ``(standard input)''
is written.
-N PATTERN, --neg-regexp=PATTERN
Specify a negative PATTERN to reject specific -e PATTERN matches
with a counter pattern. Note that longer patterns take precedence
over shorter patterns, i.e. a negative pattern must be of the same
length or longer to reject matching patterns. Option -N cannot be
specified with -P. This option may be repeated.
-v, --invert-match
Selected lines are those not matching any of the specified
patterns.
-w, --word-regexp
The PATTERN is searched for as a word, such that the matching text
is preceded by a non-word character and is followed by a non-word
character. Word-like characters are Unicode letters, digits and
connector punctuations such as underscore.
-x, --line-regexp
Select only those matches that exactly match the whole line, as if
the patterns are surrounded by ^ and $.
See also [Boolean query patterns with -%, -%%, --and, --not](#bool) for
more powerful Boolean query options than the traditional GNU/BSD grep options.
To display lines in file `myfile.sh` but not lines matching `^[ \t]*#`:
ug -v '^[ \t]*#' myfile.sh
To search `myfile.cpp` for lines with `FIXME` and `urgent`, but not `Scotty`:
ugrep FIXME myfile.cpp | ugrep urgent | ugrep -v Scotty
Same, but using `-%` for Boolean queries:
ug -% 'FIXME urgent -Scotty' myfile.cpp
To search for decimals using pattern `\d+` that do not start with `0` using
negative pattern `0\d+` and excluding `555`:
ug -e '\d+' -N '0\d+' -N 555 myfile.cpp
To search for words starting with `disp` without matching `display` in file
`myfile.py` by using a "negative pattern" `-N '/'` where `-N`
specifies an additional negative pattern to skip matches:
ug -e '\' myfile.py
To search for lines with the word `display` in file `myfile.py` skipping this
word in strings and comments, where `-f` specifies patterns in files which are
predefined patterns in this case:
ug -n -w 'display' -f python/zap_strings -f python/zap_comments myfile.py
To display lines that are not blank lines:
ug -x -e '.*' -N '\h*' myfile.py
Same, but using `-v` and `-x` with `\h*`, i.e. pattern `^\h*$`:
ug -v -x '\h*' myfile.py
To recursively list all Python files that do not contain the word `display`,
allowing the word to occur in strings and comments:
ug -RL -tPython -w 'display' -f python/zap_strings -f python/zap_comments
🔝 [Back to table of contents](#toc)
### Search non-Unicode files with --encoding
--encoding=ENCODING
The encoding format of the input. The default ENCODING is binary
and UTF-8 which are the same. Note that option -U specifies binary
PATTERN matching (text matching is the default.)
Binary, ASCII and UTF-8 files do not require this option to search them. Also
UTF-16 and UTF-32 files do not require this option to search them, assuming
that UTF-16 and UTF-32 files start with a UTF BOM
([byte order mark](https://en.wikipedia.org/wiki/Byte_order_mark)) as usual.
Other file encodings require option `--encoding=ENCODING`:
encoding | parameter
---------------------- | --------------
ASCII | *n/a*
UTF-8 | *n/a*
UTF-16 with BOM | *n/a*
UTF-32 with BOM | *n/a*
UTF-16 BE w/o BOM | `UTF-16` or `UTF-16BE`
UTF-16 LE w/o BOM | `UTF-16LE`
UTF-32 w/o BOM | `UTF-32` or `UTF-32BE`
UTF-32 w/o BOM | `UTF-32LE`
Latin-1 | `LATIN1` or `ISO-8859-1`
ISO-8859-1 | `ISO-8859-1`
ISO-8859-2 | `ISO-8859-2`
ISO-8859-3 | `ISO-8859-3`
ISO-8859-4 | `ISO-8859-4`
ISO-8859-5 | `ISO-8859-5`
ISO-8859-6 | `ISO-8859-6`
ISO-8859-7 | `ISO-8859-7`
ISO-8859-8 | `ISO-8859-8`
ISO-8859-9 | `ISO-8859-9`
ISO-8859-10 | `ISO-8859-10`
ISO-8859-11 | `ISO-8859-11`
ISO-8859-13 | `ISO-8859-13`
ISO-8859-14 | `ISO-8859-14`
ISO-8859-15 | `ISO-8859-15`
ISO-8859-16 | `ISO-8859-16`
MAC (CR=newline) | `MAC`
MacRoman (CR=newline) | `MACROMAN`
EBCDIC | `EBCDIC`
DOS code page 437 | `CP437`
DOS code page 850 | `CP850`
DOS code page 858 | `CP858`
Windows code page 1250 | `CP1250`
Windows code page 1251 | `CP1251`
Windows code page 1252 | `CP1252`
Windows code page 1253 | `CP1253`
Windows code page 1254 | `CP1254`
Windows code page 1255 | `CP1255`
Windows code page 1256 | `CP1256`
Windows code page 1257 | `CP1257`
Windows code page 1258 | `CP1258`
KOI8-R | `KOI8-R`
KOI8-U | `KOI8-U`
KOI8-RU | `KOI8-RU`
Note that regex patterns are always specified in UTF-8 (includes ASCII). To
search binary files with binary patterns, see
[searching and displaying binary files with -U, -W, and -X](#binary).
To recursively list all files that are ASCII (i.e. 7-bit):
ug -L '[^[:ascii:]]'
To recursively list all files that are non-ASCII, i.e. UTF-8, UTF-16, and
UTF-32 files with non-ASCII Unicode characters (U+0080 and up):
ug -l '[^[:ascii:]]'
To check if a file contains non-ASCII Unicode (U+0080 and up):
ug -q '[^[:ascii:]]' myfile && echo "contains Unicode"
To remove invalid Unicode characters from a file (note that `-o` may not work
because binary data is detected and rejected and newlines are added, but
`--format="%o%` does not check for binary and copies the match "as is"):
ug "[\p{Unicode}\n]" --format="%o" badfile.txt
To recursively list files with invalid UTF content (i.e. invalid UTF-8 byte
sequences or files that contain any UTF-8/16/32 code points that are outside
the valid Unicode range) by matching any code point with `.` and by using a
negative pattern `-N '\p{Unicode}'` to ignore each valid Unicode character:
ug -l -e '.' -N '\p{Unicode}'
To display lines containing laughing face emojis:
ug '[😀-😏]' emojis.txt
The same results are obtained using `\x{hhhh}` to select a Unicode character
range:
ug '[\x{1F600}-\x{1F60F}]' emojis.txt
To display lines containing the names Gödel (or Goedel), Escher, or Bach:
ug 'G(ö|oe)del|Escher|Bach' GEB.txt wiki.txt
To search for `lorem` in lower or upper case in a UTF-16 file that is marked
with a UTF-16 BOM:
ug -iw 'lorem' utf16lorem.txt
To search utf16lorem.txt when this file has no UTF-16 BOM, using `--encoding`:
ug --encoding=UTF-16 -iw 'lorem' utf16lorem.txt
To search file `spanish-iso.txt` encoded in ISO-8859-1:
ug --encoding=ISO-8859-1 -w 'año' spanish-iso.txt
🔝 [Back to table of contents](#toc)
### Matching multiple lines of text
-o, --only-matching
Output only the matching part of lines. If -A, -B or -C is
specified, fits the match and its context on a line within the
specified number of columns.
Multiple lines may be matched by patterns that match newline characters. Use
option `-o` to output the match only, not the full lines(s) that match.
To match a `\n` line break, include `\n` in the pattern to match the LF
character. If you want to match `\r\n` and `\n` line breaks, use `\r?\n` or
simply use `\R` to match any Unicode line break `\r\n`, `\r`, `\v`, `\f`, `\n`,
U+0085, U+2028 and U+2029.
To match C/C++ `/*...*/` multi-line comments:
ug '/\*(.*\n)*?.*\*+\/' myfile.cpp
To match C/C++ comments using the predefined `c/comments` patterns with
`-f c/comments`, restricted to the matching part only with option `-o`:
ug -of c/comments myfile.cpp
Same as `sed -n '/begin/,/end/p'`: to match all lines between a line containing
`begin` and the first line after that containing `end`, using lazy repetition:
ug -o '.*begin(.|\n)*?end.*' myfile.txt
🔝 [Back to table of contents](#toc)
### Displaying match context with -A, -B, -C, -y, and --width
-A NUM, --after-context=NUM
Output NUM lines of trailing context after matching lines. Places
a --group-separator between contiguous groups of matches. If -o is
specified, output the match with context to fit NUM columns after
the match or shortens the match. See also options -B, -C and -y.
-B NUM, --before-context=NUM
Output NUM lines of leading context before matching lines. Places
a --group-separator between contiguous groups of matches. If -o is
specified, output the match with context to fit NUM columns before
the match or shortens the match. See also options -A, -C and -y.
-C NUM, --context=NUM
Output NUM lines of leading and trailing context surrounding each
matching line. Places a --group-separator between contiguous
groups of matches. If -o is specified, output the match with
context to fit NUM columns before and after the match or shortens
the match. See also options -A, -B and -y.
-y, --any-line
Any line is output (passthru). Non-matching lines are output as
context with a `-' separator. See also options -A, -B, and -C.
--width[=NUM]
Truncate the output to NUM visible characters per line. The width
of the terminal window is used if NUM is not specified. Note that
double wide characters in the output may result in wider lines.
-o, --only-matching
Output only the matching part of lines. If -A, -B or -C is
specified, fits the match and its context on a line within the
specified number of columns.
To display two lines of context before and after a matching line:
ug -C2 'FIXME' myfile.cpp
To show three lines of context after a matched line:
ug -A3 'FIXME.*' myfile.cpp:
To display one line of context before each matching line with a C function
definition (C names are non-Unicode):
ug -B1 -f c/functions myfile.c
To display one line of context before each matching line with a C++ function
definition (C++ names may be Unicode):
ug -B1 -f c++/functions myfile.cpp
To display any non-matching lines as context for matching lines with `-y`:
ug -y -f c++/functions myfile.cpp
To display a hexdump of a matching line with one line of hexdump context:
ug -C1 -UX '\xaa\xbb\xcc' a.out
Context within a line is displayed with option `-o` with a context option:
ug -o -C20 'pattern' myfile.cpp
Same, but with pretty output with headings, line numbers and column numbers
(`-k`) and showing context:
ug --pretty -oC20 'pattern' myfile.cpp
🔝 [Back to table of contents](#toc)
### Searching source code using -f, -g, -O, and -t
-f FILE, --file=FILE
Read newline-separated patterns from FILE. White space in patterns
is significant. Empty lines in FILE are ignored. If FILE does not
exist, the GREP_PATH environment variable is used as path to FILE.
If that fails, looks for FILE in /usr/local/share/ugrep/pattern.
When FILE is a `-', standard input is read. This option may be
repeated.
--ignore-files[=FILE]
Ignore files and directories matching the globs in each FILE that
is encountered in recursive searches. The default FILE is
`.gitignore'. Matching files and directories located in the
directory of the FILE and in subdirectories below are ignored.
Globbing syntax is the same as the --exclude-from=FILE gitignore
syntax, but files and directories are excluded instead of only
files. Directories are specifically excluded when the glob ends in
a `/'. Files and directories explicitly specified as command line
arguments are never ignored. This option may be repeated to
specify additional files.
-g GLOBS, --glob=GLOBS
Search only files whose name matches the specified comma-separated
list of GLOBS, same as --include='glob' for each `glob' in GLOBS.
When a `glob' is preceded by a `!' or a `^', skip files whose name
matches `glob', same as --exclude='glob'. When `glob' contains a
`/', full pathnames are matched. Otherwise basenames are matched.
When `glob' ends with a `/', directories are matched, same as
--include-dir='glob' and --exclude-dir='glob'. A leading `/'
matches the working directory. This option may be repeated and may
be combined with options -M, -O and -t to expand searches. See
`ugrep --help globs' and `man ugrep' section GLOBBING for details.
-O EXTENSIONS, --file-extension=EXTENSIONS
Search only files whose filename extensions match the specified
comma-separated list of EXTENSIONS, same as --include='*.ext' for
each `ext' in EXTENSIONS. When `ext' is preceded by a `!' or a
`^', skip files whose filename extensions matches `ext', same as
--exclude='*.ext'. This option may be repeated and may be combined
with options -g, -M and -t to expand the recursive search.
-t TYPES, --file-type=TYPES
Search only files associated with TYPES, a comma-separated list of
file types. Each file type corresponds to a set of filename
extensions passed to option -O and filenames passed to option -g.
For capitalized file types, the search is expanded to include files
with matching file signature magic bytes, as if passed to option
-M. When a type is preceded by a `!' or a `^', excludes files of
the specified type. This option may be repeated.
--stats
Output statistics on the number of files and directories searched,
and the inclusion and exclusion constraints applied.
The file types are listed with `ugrep -tlist`. The list is based on
established filename extensions and "magic bytes". If you have a file type
that is not listed, use options `-O` and/or `-M`. You may want to define an
alias, e.g. `alias ugft='ugrep -Oft'` as a shorthand to search files with
filename suffix `.ft`.
To recursively display function definitions in C/C++ files (`.h`, `.hpp`, `.c`,
`.cpp` etc.) with line numbers with `-tc++`, `-o`, `-n`, and `-f c++/functions`:
ug -on -tc++ -f c++/functions
To recursively display function definitions in `.c` and `.cpp` files with line
numbers with `-Oc,cpp`, `-o`, `-n`, and `-f c++/functions`:
ug -on -Oc,cpp -f c++/functions
To recursively list all shell files with `-tShell` to match filename extensions
and files with shell shebangs, except files with suffix `.sh`:
ug -l -tShell -O^sh ''
To recursively list all non-shell files with `-t^Shell`:
ug -l -t^Shell ''
To recursively list all shell files with shell shebangs that have no shell
filename extensions:
ug -l -tShell -t^shell ''
To search for lines with `FIXME` in C/C++ comments, excluding `FIXME` in
multi-line strings:
ug -n 'FIXME' -f c++/zap_strings myfile.cpp
To read patterns `TODO` and `FIXME` from standard input to match lines in the
input, while excluding matches in C++ strings:
ug -on -f - -f c++/zap_strings myfile.cpp <
### Searching compressed files and archives with -z
-z, --decompress
Search compressed files and archives. Archives (.cpio, .pax, .tar)
and compressed archives (e.g. .zip, .7z, .taz, .tgz, .tpz, .tbz,
.tbz2, .tb2, .tz2, .tlz, .txz, .tzst) are searched and matching
pathnames of files in archives are output in braces. When used
with option --zmax=NUM, searches the contents of compressed files
and archives stored within archives up to NUM levels. If -g, -O,
-M, or -t is specified, searches files stored in archives whose
filenames match globs, match filename extensions, match file
signature magic bytes, or match file types, respectively.
Supported compression formats: gzip (.gz), compress (.Z), zip, 7z,
bzip2 (requires suffix .bz, .bz2, .bzip2, .tbz, .tbz2, .tb2, .tz2),
lzma and xz (requires suffix .lzma, .tlz, .xz, .txz),
lz4 (requires suffix .lz4),
zstd (requires suffix .zst, .zstd, .tzst),
brotli (requires suffix .br),
bzip3 (requires suffix .bz3).
--zmax=NUM
When used with option -z (--decompress), searches the contents of
compressed files and archives stored within archives by up to NUM
expansion stages. The default --zmax=1 only permits searching
uncompressed files stored in cpio, pax, tar, zip and 7z archives;
compressed files and archives are detected as binary files and are
effectively ignored. Specify --zmax=2 to search compressed files
and archives stored in cpio, pax, tar, zip and 7z archives. NUM
may range from 1 to 99 for up to 99 decompression and de-archiving
steps. Increasing NUM values gradually degrades performance.
Files compressed with gzip (`.gz`), compress (`.Z`), bzip2 (`.bz`, `.bz2`,
`.bzip2`), lzma (`.lzma`), xz (`.xz`), lz4 (`.lz4`), zstd (`.zst`, `.zstd`),
brotli (`.br`) and bzip3 (`.bz3`) are searched with option `-z` when the
corresponding libraries are installed and compiled with ugrep. This option
does not require files to be compressed. Uncompressed files are searched also,
although slower.
Other compression formats can be searched with **ugrep** [filters](#filter).
Archives (cpio, jar, pax, tar, zip and 7z) are searched with option `-z`.
Regular files in an archive that match are output with the archive pathnames
enclosed in `{` and `}` braces. Supported tar formats are v7, ustar, gnu,
oldgnu, and pax. Supported cpio formats are odc, newc, and crc. Not supported
is the obsolete non-portable old binary cpio format. Archive formats cpio,
tar, and pax are automatically recognized with option `-z` based on their
content, independent of their filename suffix.
By default, uncompressed archives stored within zip archives are also searched:
all cpio, pax, and tar files stored in zip and 7z archives are automatically
recognized and searched. However, by default, compressed files stored within
archives are not recognized, e.g. zip files stored within tar files are not
searched but rather all compressed files and archives are searched as if they
are binary files without decompressing them.
Specify `--zmax=NUM` to search archives that contain compressed files and
archives for up to `NUM` levels deep. The value of `NUM` may range from 1 to
99 for up to 99 decompression and de-archiving steps to expand up to 99 nested
archives. Larger `--zmax=NUM` values degrade performance. It is unlikely you
will ever need 99 as `--zmax=2` suffices for most practical use cases, such as
searching zip files stored in tar files.
When option `-z` is used with options `-g`, `-O`, `-M`, or `-t`, archives and
compressed and uncompressed files that match the filename selection criteria
(glob, extension, magic bytes, or file type) are searched only. For example,
`ugrep -r -z -tc++` searches C++ files such as `main.cpp` and zip and tar
archives that contain C++ files such as `main.cpp`. Also included in the
search are compressed C++ files such as `main.cpp.gz` and `main.cpp.xz` when
present. Also any cpio, pax, tar, zip and 7z archives when present are
searched for C++ files that they contain, such as `main.cpp`. Use option
`--stats` to see a list of the glob patterns applied to filter file pathnames
in the recursive search and when searching archive contents.
When option `-z` is used with options `-g`, `-O`, `-M`, or `-t` to search cpio,
jar, pax, tar, zip and 7z archives, archived files that match the filename
selection criteria are searched only.
The gzip, compress, and zip formats are automatically detected, which is useful
when reading gzip-compressed data from standard input, e.g. input redirected
from a pipe. Other compression formats require a filename suffix: `.bz`,
`.bz2`, or `.bzip2` for bzip2, `.lzma` for lzma, `.xz` for xz, `.lz4` for lz4,
`.zst` or `.zstd` for zstd, `.br` for brotli and `.bz3` for bzip3. Also the
compressed tar archive shorthands `.taz`, `.tgz` and `.tpz` for gzip, `.tbz`,
`.tbz2`, `.tb2`, and `.tz2` for bzip2, `.tlz` for lzma, `.txz` for xz, and
`.tzst` for zstd are recognized. To search these formats with ugrep from
standard input, use option `--label='stdin.bz2'` for bzip2,
`--label='stdin.lzma'` for lzma, `--label='stdin.xz'` for xz,
`--label='stdin.lz4` for lz4 and `--label='stdin.zst` for zstd and so on. The
name `stdin` is arbitrary and may be omitted:
format | filename suffix | tar/pax archive short suffix | suffix required? | ugrep from stdin | library |
--------- | ----------------------- | ------------------------------- | ---------------- | ---------------- | ------------ |
gzip | `.gz` | `.taz`, `.tgz`, `.tpz` | no | automatic | libz |
compress | `.Z` | `.taZ`, `.tZ` | no | automatic | *built-in* |
zip | `.zip`, `.zipx`, `.ZIP` | | no | automatic | libz |
7zip | `.7z` | | yes | `--label=.7z` | *built-in* |
bzip2 | `.bz`, `.bz2`, `.bzip2` | `.tb2`, `.tbz`, `.tbz2`, `.tz2` | yes | `--label=.bz2` | libbz2 |
lzma | `.lzma` | `.tlz` | yes | `--label=.lzma` | liblzma |
xz | `.xz` | `.txz` | yes | `--label=.xz` | liblzma |
lz4 | `.lz4` | | yes | `--label=.lz4` | liblz4 |
zstd | `.zst`, `.zstd` | `.tzst` | yes | `--label=.zst` | libzstd |
brotli | `.br` | | yes | `--label=.br`