{"id":16150657,"url":"https://github.com/funilrys/tivilsta","last_synced_at":"2025-06-29T09:38:27.216Z","repository":{"id":50681140,"uuid":"500170323","full_name":"funilrys/tivilsta","owner":"funilrys","description":"A different whitelisting mechanism for blocklist maintainers.","archived":false,"fork":false,"pushed_at":"2024-04-27T12:26:51.000Z","size":105,"stargazers_count":5,"open_issues_count":1,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-12T10:02:52.830Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/funilrys.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["funilrys"],"ko_fi":"funilrys","custom":["https://www.paypal.me/funilrys"]}},"created_at":"2022-06-05T17:49:31.000Z","updated_at":"2024-10-23T07:43:14.000Z","dependencies_parsed_at":"2024-04-09T21:45:23.781Z","dependency_job_id":"9dd259d8-3605-4aae-a9b0-3ed78125bd6d","html_url":"https://github.com/funilrys/tivilsta","commit_stats":{"total_commits":36,"total_committers":2,"mean_commits":18.0,"dds":0.02777777777777779,"last_synced_commit":"33449c31b95a986de8afde32151c272401f64fc5"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/funilrys%2Ftivilsta","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/funilrys%2Ftivilsta/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/funilrys%2Ftivilsta/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/funilrys%2Ftivilsta/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/funilrys","download_url":"https://codeload.github.com/funilrys/tivilsta/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234840941,"owners_count":18895129,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-10T00:50:34.710Z","updated_at":"2025-06-29T09:38:27.204Z","avatar_url":"https://github.com/funilrys.png","language":"Rust","funding_links":["https://github.com/sponsors/funilrys","https://ko-fi.com/funilrys","https://www.paypal.me/funilrys"],"categories":[],"sub_categories":[],"readme":"# Tivilsta - A different whitelisting mechanism for blocklist maintainers \u003c!-- omit in toc --\u003e\n\nTivilsta aims to provide a different and in our view better whitelisting mechanism\nfor blocklist maintainers.\n\n# Table of Content\n\n- [Table of Content](#table-of-content)\n- [Installation](#installation)\n- [The Format](#the-format)\n  - [Introduction](#introduction)\n  - [The flags](#the-flags)\n    - [No Flag: The purest rule](#no-flag-the-purest-rule)\n    - [`ALL `: The \"ends-with\" rule](#all--the-ends-with-rule)\n    - [`REG `: The regular expression rule](#reg--the-regular-expression-rule)\n    - [`RZD `: The broad and powerful rule](#rzd--the-broad-and-powerful-rule)\n- [Usage \\\u0026 Examples](#usage--examples)\n  - [Library](#library)\n  - [CLI](#cli)\n    - [Overview](#overview)\n    - [Help Output](#help-output)\n    - [Simple whitelisting example](#simple-whitelisting-example)\n- [License](#license)\n\n# Installation\n\nYou can install the tivilsta CLI or library through [crates.io](https://crates.io/crates/tivilsta).\n\n```shell\n$ cargo install tivilsta\n\n$ tivilsta -V  ## Assuming that it is in your ${PATH}\n```\n\n# The Format\n\n## Introduction\n\nIn a world where blocklists and whitelist lists are getting bigger and bigger,\nthe whitelisting mechanism we all use is still the same: list all whitelisted\ndomains and use some kind of shell magic to processed the whitelisting.\n\nWhat if we want more ? That's what Tivilsta tries to provide: A better way of\nwriting whitelist list.\n\nWith Tivilsta you still have 1 domain per line but you also get some nice features\nlike for example regular expression _(regex)_. In fact, the Tivilsta project\nhas the same set of of \"pure\" rule that you know but also some flags like `ALL `,\n`REG ` or `RDZ ` to fulfill many possible use cases that list maintainer may\nneed during the whitelisting process.\n\n## The flags\n\nTivilsta provides a set of flags to make whitelist maintenance easier.\n\n### No Flag: The purest rule\n\nThis is the purest of all rules. It is what we all know an cherish. The single\nline without any flag.\n\n```\nexample.org\n```\n\nIn this example, any subject of your source file that literally matches `example.org`\nwill be whitelisted.\n\n### `ALL `: The \"ends-with\" rule\n\nSometime when working with highly volatile dataset, you may want to whitelist\nevery subjects that ends with for example `gov.uk`.\n\nWith Tivilsta you can do that through the `ALL ` flag.\n\n```\nALL .gov.uk\n```\n\nIn this example, any subject of your source file that ends with `.gov.uk` -\n`gov.uk` included - will be whitelisted.\n\n### `REG `: The regular expression rule\n\nYou are a fan of regex ? We are too! When working with highly volatile dataset,\nwe want to simply use a regular expression (short regex) to do the task.\n\nWith Tivilsta you can do that through the `REG ` flag.\n\n```\nREG ^(?!.*\\.?(watchdog\\.ohio|dap\\.digitalgov|stats\\.ssa|adgallery\\.whitehousedrugpolicy)).*\\.gov$\n```\n\nIn this example, any subject of your source file that ends with `.gov` will be\nwhitelisted except the following:\n\n- `watchdog.ohio.gov`\n- `dap.digitalgov.gov`\n- `stats.ssa.gov`\n- `adgallery.whiteshousedrugpolicy.gov`\n\n### `RZD `: The broad and powerful rule\n\nHave you ever wondered if it is possible to somehow whitelist all combination of\na company name with all possible Top Level Domain ?\n\nWith Tivilsta you can do that through the `RDZ ` flag. This flag is extremely\nbroad and powerful as it will fetch the\n[IANA Root Zone Database](https://www.iana.org/domains/root/db) and the\n[Public Suffix List](https://publicsuffix.org/)\nto build a set of rules with all possible gTLDs or extensions - if you prefer.\n\n```\nRZD example\n```\n\nIn this example, any subject matching `example.[gTLD]` will be whitelisted.\n\n\n# Usage \u0026 Examples\n\n## Library\n\n```rust\nuse tivilsta::Ruler;\n\nfn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\n    let my_subjects: Vec\u003cString\u003e = vec![\n        String::from(\"example.com\"),\n        String::from(\"example.org\"),\n        String::from(\"api.example.org\"),\n        String::from(\"test.example.com\"),\n    ];\n\n    let whitelisting_rules: Vec\u003cString\u003e = vec![\n        String::from(\"api.example.org\"),\n        String::from(\"ALL .com\"),\n    ];\n\n    let mut ruler = Ruler::new(false);\n    ruler.parse_vec(\u0026whitelisting_rules);\n\n    for subject in my_subjects {\n        if ruler.is_whitelisted(\u0026subject) {\n            println!(\"{} is WHITELISTED\", subject)\n        } else {\n            println!(\"{} is still BLOCKLISTED\", subject)\n        }\n    }\n\n    Ok(())\n}\n```\n\n**Output:**\n\n```\nexample.com is WHITELISTED\nexample.org is still BLOCKLISTED\napi.example.org is WHITELISTED\ntest.example.com is WHITELISTED\n```\n\n## CLI\n\n### Overview\n\nExample for argument with multiple values or files:\n\n- `tivilsta -s test.list -w whitelist1.list -w  whitelist2.list`\n- `tivilsta -s test.list --reg reg1.list --reg reg2.list`\n\n\n| Argument                | Required           | Multiple Values Allowed | Description                                                                                                                                                         |\n| ----------------------- | ------------------ | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `--source` \\| `-s`      | :white_check_mark: | :x:                     | The source file. In other word the block list to process.                                                                                                           |\n| `--whitelist` \\| `-w`   | :white_check_mark: | :white_check_mark:      | The whitelist schema file-s. Each line will be processed **AS IT IS.**                                                                                              |\n| `--all`                 | :x:                | :white_check_mark:      | The whitelist schema file-s. Each line will be prefixed with the `ALL ` flag.                                                                                       |\n| `--allow-complements`   | :x:                | :x:                     | Whether we should consider complements when whitelisting. A complement is `www.example.org` when `example.org` is given and vice-versa.                             |\n| `--help` \\| `-h`        | :x:                | :x:                     | Prints the help message and exit.                                                                                                                                   |\n| `--output` \\| `-o`      | :x:                | :x:                     | The output file. By default the tool will output to `stdout`. You can use this argument to explicitly set the output file.                                          |\n| `--reg`                 | :x:                | :white_check_mark:      | The whitelist schema file-s. Each line will be prefixed with the `REG ` flag.                                                                                       |\n| `--rzd`                 | :x:                | :white_check_mark:      | The whitelist schema file-s. Each line will be prefixed with the `RDZ ` flag.                                                                                       |\n| `-m` \\| `--multithread` | :x:                | :x:                     | Whether we should process the given information with multithreading. This is useful for large files. Note: This is not recommended for small files.                 |\n| `--max-threads` \\| `-t` | :x:                | :x:                     | The maximum number of threads to use. Note: This is not recommended for small files. If not given, the number of threads will be set to the number of CPU cores - 2 |\n| `--version` \\| `-V`     | :x:                | :x:                     | Prints the version and exit.                                                                                                                                        |\n\n### Help Output\n\n```\nA different whitelisting mechanism for blocklist maintainers.\n\nUSAGE:\n    tivilsta [OPTIONS] --source \u003cSOURCE\u003e --whitelist \u003cWHITELIST\u003e...\n\nOPTIONS:\n        --all \u003cALL\u003e...                 One or multiple space separated whitelisting schema in form\n                                       of a file path or URL to read. Each rule/line will be\n                                       automatically prefixed with the `ALL ` flag while parsing.\n                                       Note: When using a URL, the file will be downloaded and\n                                       stored in a temporary file that will be deleted when the\n                                       program exits\n        --allow-complements            Whether we consider complements while parsing rules. Note:\n                                       Complements are `www.example.org` if `example.org` is given -\n                                       and vice-versa\n    -h, --help                         Print help information\n    -m, --multithread                  Whether we should process the given information with\n                                       multithreading. This is useful for large files. Note: This is\n                                       not recommended for small files\n    -o, --output \u003cOUTPUT\u003e              The output file\n        --reg \u003cREG\u003e...                 One or multiple space separated whitelisting schema in form\n                                       of a file path or URL to read. Each rule/line will be\n                                       automatically prefixed with the `REG ` flag while parsing.\n                                       Note: When using a URL, the file will be downloaded and\n                                       stored in a temporary file that will be deleted when the\n                                       program exits\n        --rzd \u003cRZD\u003e...                 One or multiple space separated whitelisting schema in form\n                                       of a file path or URL to read. Each rule/line will be\n                                       automatically prefixed with the `RZD ` flag while parsing.\n                                       Note: When using a URL, the file will be downloaded and\n                                       stored in a temporary file that will be deleted when the\n                                       program exits\n    -s, --source \u003cSOURCE\u003e              The file to cleanup\n    -t, --max-threads \u003cMAX_THREADS\u003e    The maximum number of threads to use. Note: This is not\n                                       recommended for small files. If not given, the number of\n                                       threads will be set to the number of CPU cores - 2\n    -V, --version                      Print version information\n    -w, --whitelist \u003cWHITELIST\u003e...     One or multiple space separated whitelisting schema in form\n                                       of a file path or URL. Each rule/line will be parsed\n                                       as-it-is. Note: When using a URL, the file will be downloaded\n                                       and stored in a temporary file that will be deleted when the\n                                       program exits\n\n```\n\n\n### Simple whitelisting example\n\n```shell\n$ cat test.list\nexample.org\nexample.com\napi.example.org\ntest.example.com\n\n$ cat whitelist.list\napi.example.org\nALL .com\n\n$ tivilsta -s test.list -w whitelist.list\nexample.org\n```\n\n# License\n\n```\nCopyright (c) 2022, 2023, 2024, 2025 Nissar Chababy\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffunilrys%2Ftivilsta","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffunilrys%2Ftivilsta","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffunilrys%2Ftivilsta/lists"}