https://github.com/jftuga/freq
Display the frequency of each line in a file or from STDIN
https://github.com/jftuga/freq
Last synced: 11 months ago
JSON representation
Display the frequency of each line in a file or from STDIN
- Host: GitHub
- URL: https://github.com/jftuga/freq
- Owner: jftuga
- License: mit
- Created: 2018-11-15T17:54:09.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2023-11-24T21:57:56.000Z (over 2 years ago)
- Last Synced: 2024-06-20T14:20:37.234Z (almost 2 years ago)
- Language: Go
- Homepage:
- Size: 40 KB
- Stars: 3
- Watchers: 3
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# freq
Display the line frequency of each line in a file or from STDIN
The [Releases Page](https://github.com/jftuga/freq/releases) contains binaries for Windows, MacOS, Linux and FreeBSD.
## Usage
```
Usage for freq:
-N int
only output the last N results, useful with -a
-a output results in ascending order
-b bare: don't display numeric frequencies; only display a sorted, unique list
-d if line only contains IP address, resolve to hostname
-l convert to lowercase first
-n int
only output the first N results
-p output using percentages
-se int
substring end position
-ss int
substring start position
-v display version and then exit
```
## Installation
* macOS: `brew update; brew install jftuga/tap/freq`
* Binaries for Linux, macOS and Windows are provided in the [releases](https://github.com/jftuga/freq/releases) section.
## Examples
```
R:\freq>type con > test.txt
d
b
a
b
c
c
d
c
d
d
^Z
R:\freq>type test.txt | freq.exe
4 d
3 c
2 b
1 a
me@linux:~/freq$ freq -n 2 test.txt
4 d
3 c
me@linux:~/freq$ ls -l | grep -v ^total ; echo ; ls -l | grep -v ^total | freq -ss 2 -se 6
-rw-r--r-- 1 jftuga jftuga 1068 Feb 2 20:33 LICENSE
-rw-r--r-- 1 jftuga jftuga 214 Feb 2 20:33 Makefile
-rw-r--r-- 1 jftuga jftuga 1292 Feb 2 20:41 README.md
drwxr-xr-x 9 jftuga jftuga 4096 Feb 2 20:37 dist
-rwxr-xr-x 1 jftuga jftuga 2398016 Feb 2 20:43 freq
-rw-rw-r-- 1 jftuga jftuga 7654 Feb 2 20:33 freq.go
-rwxr-xr-x 1 jftuga jftuga 3597 Feb 2 20:33 increment_version.sh
3 rw-r-
3 rwxr-
1 rw-rw
```
## Speed
For input greater than a few hundred megs in size, `freq` is faster than:
sort | uniq -c | sort -nr
# or
awk '{a[$0]++}END{for(i in a){print a[i] " " i}}' | sort -nr
but slower than something like:
export LC_ALL=C
sort -S 8G --parallel=4 -T /mnt/fast_ssd/tmp | uniq -c | sort -n -r -S 8G --parallel=4 -T /mnt/fast_ssd/tmp
See also: https://www.reddit.com/r/commandline/comments/a7hq5n/psa_improving_gnu_sort_speed/
## Compile with Docker
* see also: https://hub.docker.com/_/golang
* docker pull golang
* git clone https://github.com/jftuga/freq.git
* cd freq
* docker run --rm -v "$PWD":/usr/src/freq -w /usr/src/freq golang:latest make