Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/kashav/fsql

Search for files using a fun query language
https://github.com/kashav/fsql

find golang

Last synced: 2 days ago
JSON representation

Search for files using a fun query language

Awesome Lists containing this project

README 

        
# fsql [![Go](https://github.com/kashav/fsql/actions/workflows/go.yml/badge.svg)](https://github.com/kashav/fsql/actions/workflows/go.yml)



>Search through your filesystem with SQL-esque queries.



## Contents



- [Demo](#demo)

- [Installation](#installation)

- [Usage](#usage)

- [Query Syntax](#query-syntax)

- [Examples](#usage-examples)

- [Contribute](#contribute)

- [License](#license)



## Demo



[![fsql.gif](./media/fsql.gif)](https://asciinema.org/a/120534)



## Installation



#### Binaries



[View latest release](https://github.com/kashav/fsql/releases/latest).



#### Via Go



```sh

$ go get -u -v github.com/kashav/fsql/...

$ which fsql

$GOPATH/bin/fsql

```



#### Via Homebrew



```sh

$ brew install fsql

$ which fsql

/usr/local/bin/fsql

```



#### Build manually



```sh

$ git clone https://github.com/kashav/fsql.git $GOPATH/src/github.com/kashav/fsql

$ cd $_ # $GOPATH/src/github.com/kashav/fsql

$ make

$ ./fsql

```



## Usage



fsql expects a single query via stdin. You may also choose to use fsql in interactive mode.



View the usage dialogue with the `-help` flag.



```sh

$ fsql -help

usage: fsql [options] [query]

  -v  print version and exit (shorthand)

  -version

      print version and exit

```



## Query syntax



In general, each query requires a `SELECT` clause (to specify which attributes will be shown), a `FROM` clause (to specify which directories to search), and a `WHERE` clause (to specify conditions to test against).



```console

>>> SELECT attribute, ... FROM source, ... WHERE condition;

```



You may choose to omit the `SELECT` and `WHERE` clause.



If you're providing your query via stdin, quotes are **not** required, however you'll have to escape _reserved_ characters (e.g. `*`, `<`, `>`, etc).



### Attribute



Currently supported attributes include `name`, `size`, `time`, `hash`, `mode`.



Use `all` or `*` to choose all; if no attribute is provided, this is chosen by default.



**Examples**:



Each group features a set of equivalent clauses.



```console

>>> SELECT name, size, time ...

>>> name, size, time ...

```



```console

>>> SELECT all FROM ...

>>> all FROM ...

>>> FROM ...

```



### Source



Each source should be a relative or absolute path to a directory on your machine.



Source paths may include environment variables (e.g. `$GOPATH`) or tildes (`~`). Use a hyphen (`-`) to exclude a directory. Source paths also support usage of [glob patterns](https://en.wikipedia.org/wiki/Glob_(programming)).



In the case that a directory begins with a hyphen (e.g. `-foo`), use the following to include it as a source:



```console

>>> ... FROM ./-foo ...

```



**Examples**:



```console

>>> ... FROM . ...

```



```console

>>> ... FROM ~/Desktop, ./*/**.go ...

```



```console

>>> ... FROM $GOPATH, -.git/ ...

```



### Condition



#### Condition syntax



A single condition is made up of 3 parts: an attribute, an operator, and a value.



- **Attribute**:



  A valid attribute is any of the following: `name`, `size`, `mode`, `time`.



- **Operator**:



  Each attribute has a set of associated operators.



  - `name`:



    | Operator | Description |

    | :---: | --- |

    | `=` | String equality |

    | `<>` / `!=` | Synonymous to using `"NOT ... = ..."` |

    | `IN` | Basic list inclusion |

    | `LIKE` |  Simple pattern matching. Use `%` to match zero, one, or multiple characters. Check that a string begins with a value: `%`, ends with a value: `%`, or contains a value: `%%`. |

    | `RLIKE` | Pattern matching with regular expressions. |



  - `size` / `time`:



    - All basic algebraic operators: `>`, `>=`, `<`, `<=`, `=`, and `<>` / `!=`.



  - `hash`:



    - `=` or `<>` / `!=`



  - `mode`:



    - `IS`



- **Value**:



  If the value contains spaces, wrap the value in quotes (either single or double) or backticks.



  The default unit for `size` is bytes.



  The default format for `time` is `MMM DD YYYY HH MM` (e.g. `"Jan 02 2006 15 04"`).



  Use `mode` to test if a file is regular (`IS REG`) or if it's a directory (`IS DIR`).



  Use `hash` to compute and/or compare the hash value of a file. The default algorithm is `SHA1`



#### Conjunction / Disjunction



Use `AND` / `OR` to join conditions. Note that precedence is assigned based on order of appearance.



This means `WHERE a AND b OR c` is **not** the same as `WHERE c OR b AND a`. Use parentheses to get around this behaviour, i.e. `WHERE a AND b OR c` **is** the same as `WHERE c OR (b AND a)`.



**Examples**:



```console

>>> ... WHERE name = main.go OR size = 5 ...

```



```console

>>> ... WHERE name = main.go AND size > 20 ...

```



#### Negation



Use `NOT` to negate a condition. This keyword **must** precede the condition (e.g. `... WHERE NOT a ...`).



Note that negating parenthesized conditions is currently not supported. However, this can easily be resolved by applying [De Morgan's laws](https://en.wikipedia.org/wiki/De_Morgan%27s_laws) to your query. For example, `... WHERE NOT (a AND b) ...` is _logically equivalent_ to `... WHERE NOT a OR NOT b ...` (the latter is actually more optimal, due to [lazy evaluation](https://en.wikipedia.org/wiki/Lazy_evaluation)).



**Examples**:



```console

>>> ... WHERE NOT name = main.go ...

```



### Attribute Modifiers



Attribute modifiers are used to specify how input and output values should be processed. These functions are applied directly to attributes in the `SELECT` and `WHERE` clauses.



The table below lists currently-supported modifiers. Note that the first parameter to `FORMAT` is always the attribute name.



| Attribute | Modifier  | Supported in `SELECT` | Supported in `WHERE` |

| :---: | --- | :---: | :---: |

| `hash` | `SHA1(, n)` | ✔️ | ✔️ |

| `name` | `UPPER` (synonymous to `FORMAT(, UPPER)`) | ✔️ | ✔️ |

| | `LOWER` (synonymous to `FORMAT(, LOWER)`) | ✔️ | ✔️ |

| | `FULLPATH` | ✔️ |  |

| | `SHORTPATH`  | ✔️ |  |

| `size` | `FORMAT(, unit)` | ✔️ | ✔️ |

| `time` | `FORMAT(, layout)` | ✔️ | ✔️ |



- **`n`**:



  Specify the length of the hash value. Use a negative integer or `ALL` to display all digits.



- **`unit`**:



  Specify the size unit. One of: `B` (byte), `KB` (kilobyte), `MB` (megabyte), or `GB` (gigabyte).



- **`layout`**:



  Specify the time layout. One of: [`ISO`](https://en.wikipedia.org/wiki/ISO_8601), [`UNIX`](https://en.wikipedia.org/wiki/Unix_time), or [custom](https://golang.org/pkg/time/#Time.Format). Custom layouts must be provided in reference to the following date: `Mon Jan 2 15:04:05 -0700 MST 2006`.



**Examples**:



```console

>>> SELECT SHA1(hash, 20) ...

```



```console

>>> ... WHERE UPPER(name) ...

```



```console

>>> SELECT FORMAT(size, MB) ...

```



```console

>>> ... WHERE FORMAT(time, "Mon Jan 2 2006 15:04:05") ...

```



### Subqueries



Subqueries allow for more complex condition statements. These queries are recursively evaluated while parsing. SELECTing multiple attributes in a subquery is not currently supported; if more than one attribute (or `all`) is provided, only the first attribute is used.



Support for referencing superqueries is not yet implemented, see [#4](https://github.com/kashav/fsql/issues/4) if you'd like to help with this.



**Examples**:



```console

>>> ... WHERE name IN (SELECT name FROM ../foo) ...

```



## Usage Examples



List all attributes of each directory in your home directory (note the escaped `*`):



```console

$ fsql SELECT \* FROM ~ WHERE mode IS DIR

```



List the names of all files in the Desktop and Downloads directory that contain `csc` in the name:



```console

$ fsql "SELECT name FROM ~/Desktop, ~/Downloads WHERE name LIKE %csc%"

```



List all files in the current directory that are also present in some other directory:



```console

$ fsql

>>> SELECT all FROM . WHERE name IN (

...   SELECT name FROM ~/Desktop/files.bak/

... );

```



Passing queries via stdin without quotes is a bit of a pain, hopefully the next examples highlight that, my suggestion is to use interactive mode or wrap the query in quotes if you're doing anything with subqueries or attribute modifiers.



List all files named `main.go` in `$GOPATH` which are larger than 10.5 kilobytes or smaller than 100 bytes:



```console

$ fsql SELECT all FROM $GOPATH WHERE name = main.go AND \(FORMAT\(size, KB\) \>= 10.5 OR size \< 100\)

$ fsql "SELECT all FROM $GOPATH WHERE name = main.go AND (FORMAT(size, KB) >= 10.5 OR size < 100)"

$ fsql

>>> SELECT

...   all

... FROM

...   $GOPATH

... WHERE

...   name = main.go

...   AND (

...     FORMAT(size, KB) >= 10.5

...     OR size < 100

...   )

... ;

```



List the name, size, and modification time of JavaScript files in the current directory that were modified after April 1st 2017:



```console

$ fsql SELECT UPPER\(name\), FORMAT\(size, KB\), FORMAT\(time, ISO\) FROM . WHERE name LIKE %.js AND time \> \'Apr 01 2017 00 00\'

$ fsql "SELECT UPPER(name), FORMAT(size, KB), FORMAT(time, ISO) FROM . WHERE name LIKE %.js AND time > 'Apr 01 2017 00 00'"

$ fsql

>>> SELECT

...   UPPER(name),

...   FORMAT(size, KB),

...   FORMAT(time, ISO)

... FROM

...   .

... WHERE

...   name LIKE %.js

...   AND time > 'Apr 01 2017 00 00'

... ;

```



## Contribute



This project is completely open source, feel free to [open an issue](https://github.com/kashav/fsql/issues) or [submit a pull request](https://github.com/kashav/fsql/pulls).



Before submitting code, please ensure that tests are passing and the linter is happy. The following commands may be of use, refer to the [Makefile](./Makefile) to see what they do.



```sh

$ make install \

       get-tools

$ make fmt \

       vet \

       lint

$ make test \

       coverage

$ make bootstrap-dist \

       dist

```



## License



fsql source code is available under the [MIT license](./LICENSE).