{"id":28384055,"url":"https://github.com/datumbox/vernamveil","last_synced_at":"2025-06-25T22:31:12.304Z","repository":{"id":290089759,"uuid":"972354886","full_name":"datumbox/VernamVeil","owner":"datumbox","description":"VernamVeil: A Function-Based Stream Cypher","archived":false,"fork":false,"pushed_at":"2025-06-15T11:09:41.000Z","size":6058,"stargazers_count":3,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-15T11:34:14.364Z","etag":null,"topics":["blake3","cryptography","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datumbox.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-25T00:15:16.000Z","updated_at":"2025-06-15T11:09:21.000Z","dependencies_parsed_at":"2025-05-10T21:22:41.890Z","dependency_job_id":"6537684d-d915-4df6-a0a1-423eaff89785","html_url":"https://github.com/datumbox/VernamVeil","commit_stats":null,"previous_names":["datumbox/vernamveil"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/datumbox/VernamVeil","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datumbox%2FVernamVeil","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datumbox%2FVernamVeil/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datumbox%2FVernamVeil/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datumbox%2FVernamVeil/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datumbox","download_url":"https://codeload.github.com/datumbox/VernamVeil/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datumbox%2FVernamVeil/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261962196,"owners_count":23236884,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blake3","cryptography","python"],"created_at":"2025-05-30T07:38:46.473Z","updated_at":"2025-06-25T22:31:12.288Z","avatar_url":"https://github.com/datumbox.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cpicture\u003e\n  \u003csource srcset=\"https://raw.githubusercontent.com/datumbox/VernamVeil/main/docs/logo.png\" media=\"(prefers-color-scheme: light)\"\u003e\n  \u003csource srcset=\"https://raw.githubusercontent.com/datumbox/VernamVeil/main/docs/logo-dark.png\" media=\"(prefers-color-scheme: dark)\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/datumbox/VernamVeil/main/docs/logo.png\" alt=\"VernamVeil Logo\"\u003e\n\u003c/picture\u003e\n\n# VernamVeil: A Function-Based Stream Cypher\n\n[![CI](https://github.com/datumbox/VernamVeil/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/datumbox/VernamVeil/actions) [![Docs](https://img.shields.io/badge/docs-online-brightgreen.svg)](https://datumbox.github.io/VernamVeil/) [![License](https://img.shields.io/github/license/datumbox/VernamVeil?color=brightgreen)](https://github.com/datumbox/VernamVeil/blob/main/LICENSE)\n\n---\n\n\u003e ⚠️ **DISCLAIMER:** This is an educational encryption prototype and **not** meant for real-world use. It has **not** been audited or reviewed by cryptography experts, and **should not** be used to store, transmit, or protect sensitive data.\n\n## 🚀 Quick Start\n\nMinimal Installation (without vectorisation or C extension support):\n```bash\npip install .\n```\n\nMinimal Example:\n```python\nimport hashlib\nfrom vernamveil import FX, VernamVeil\n\n\n# Step 1: Define a custom key stream function; remember to store this securely\ndef keystream_fn(i: int, seed: bytes) -\u003e bytes:\n    # Simple cryptographically safe fx; see below for more examples\n    hasher = hashlib.blake2b(seed)\n    hasher.update(i.to_bytes(8, \"big\"))\n    return hasher.digest()\n\nfx = FX(keystream_fn, block_size=64, vectorise=False)\n\n\n# Step 2: Generate a random initial seed for encryption\ninitial_seed = VernamVeil.get_initial_seed()  # remember to store this securely\n\n# Step 3: Encrypt and decrypt a single message\ncypher = VernamVeil(fx)\nencrypted, _ = cypher.encode(b\"Hello!\", initial_seed)\ndecrypted, _ = cypher.decode(encrypted, initial_seed)\n```\n\n---\n\n## 🔎 Overview\n\n**VernamVeil** is an experimental cypher inspired by the **One-Time Pad (OTP)** developed in Python. The name honours **Gilbert Vernam**, who is credited with the theoretical foundation of the OTP.\n\nInstead of using a static key, VernamVeil allows the key to be represented by a function `fx(i: int | np.ndarray, seed: bytes | bytearray) -\u003e bytes | np.ndarray`:\n- `i`: the index of the block of bytes in the message; a scalar integer or a 1D uint64 NumPy array with a continuous enumeration for vectorised operations.\n- `seed`: a byte string that provides context and state; should be kept secret.\n- **Output**: bytes or a 2D uint8 NumPy array representing the key stream values.\n\n_Note: `numpy` is an optional but highly recommended dependency, used to accelerate vectorised operations when available._\n\nThis approach enables novel forms of key generation, especially for those who enjoy playing with maths and code. While this is not a secure implementation by cryptographic standards, it offers a fun and flexible way to experiment with function-based encryption. If you're curious about how encryption works, or just want to mess with maths and code in a cool way, this project is a fun starting point. For more information, read the accompanying [blog post](https://blog.datumbox.com/vernamveil-a-fresh-take-on-function-based-encryption/).\n\n---\n\n## 💡 Why VernamVeil?\n\n- **Function-based keystreams for enhanced flexibility**: Unlike traditional cyphers that rely on fixed-size keys, VernamVeil allows the keystream to be generated by a user-defined function (`fx`). This enables the creation of arbitrarily long, non-repeating keystreams, provided that the function and seed combination allows it.\n- **Modular design for cryptographic experimentation**: VernamVeil's architecture makes it easy to swap in different cryptographic primitives and keystream generation strategies. This modularity supports experimentation with various secure pseudorandom functions, hash-based constructions, or even true one-time pad keystreams, while encouraging the use of well-established cryptographic techniques.\n- **Inspired by the One-Time Pad (OTP)**: VernamVeil supports an OTP mode when provided with a truly random, externally sourced keystream. The One-Time Pad offers strong cryptographic guarantees if, and only if, it is implemented correctly: the cypher is bug-free, the keystream is truly random, at least as long as the message, used only once, and kept completely secret.\n\n---\n\n## ✨ Cryptographic Characteristics\n\n1. **Function-Based, Symmetric Cypher with OTP Support**: VernamVeil uses a user-defined function `fx` and an `initial_seed` (both secret) to dynamically generate the key stream. This approach is symmetric: identical secrets and encryption configuration are required for both encryption and decryption, making the process fully reversible. VernamVeil also offers the `OTPFX` callable wrapper that allows it to operate in one-time pad (OTP) mode when provided with a truly random, externally sourced keystream.\n2. **Synthetic IV Seed Initialisation, Stateful Seed Evolution \u0026 Avalanche Effects**: Instead of a traditional nonce, the first internal seed is derived using a Synthetic IV computed as a keyed hash of the user-provided initial seed, the full plaintext, and the current timestamp (inspired by [RFC 5297](https://datatracker.ietf.org/doc/html/rfc5297)). For each chunk, the seed is further evolved by key-hashing the previous seed with the chunk's plaintext, maintaining state between operations. This hash-based seed refreshing ensures each keystream is unique, prevents keystream reuse, provides resilience against seed reuse and deterministic output, and produces an avalanche effect: small changes in input result in large, unpredictable changes in output. Including the current timestamp in the SIV ensures each encryption produces a unique keystream, making output non-deterministic and providing resilience against accidental seed reuse, even for identical messages and seeds. The scheme does not allow backward derivation of seeds, if a current seed is leaked, past messages remain secure (backward secrecy is preserved).\n3. **Message Obfuscation, Zero Metadata \u0026 Authenticated Encryption**: The cypher injects decoy chunks, pads real chunks with dummy bytes, and shuffles output to obscure chunk boundaries, complicating cryptanalysis methods such as traffic analysis or block boundary detection. Chunk delimiters are randomly generated, encrypted, and not exposed; block delimiters are randomly generated and refreshed for every block. The cyphertext contains no embedded metadata, minimising the risk of attackers identifying recurring patterns or structural information. All encryption details are deterministically recovered from the `fx`, except for the configuration parameters (e.g., chunk and delimiter sizes) which must be provided and matched exactly during decryption, or the MAC check will fail. During encryption, Message Authentication is enforced using a standard encrypt-then-MAC (EtM) construction. During decryption verification-before-decryption is used to detect tampering and prevent padding oracle-style issues.\n4. **Modular \u0026 Configurable Keystream Design**: The `fx` function can be swapped to explore different styles of pseudorandom generation, including custom PRNGs, cryptographic hashes or OTP. The implementation also allows full adjustment of configuration, offering flexibility to tailor encryption to specific needs. \n5. **Vectorisation \u0026 Optional C-backed Fast Hashing**: All operations are vectorised using `numpy` when `vectorise=True`, with a slower pure Python fallback available. For even faster vectorised `fx` functions, an optional C module (`nphash`) can be compiled (with `cffi` and system dependencies), enabling high-performance BLAKE2b, BLAKE3 and SHA-256 hashing for NumPy-based key stream generation. BLAKE3, in particular, is only available via this extension and benefits from hardware acceleration, including SIMD and assembly optimisations where supported. This is supported both in user-defined `fx` methods and automatically by helpers like `generate_default_fx`. See [`nphash/README.md`](nphash/README.md) for details.\n\n---\n\n## ⚠️ Caveats \u0026 Best Practices\n\n- **Not Secure for Real Use**: This is an educational tool and experimental toy, not production-ready cryptography.\n- **Use Strong `fx` Functions**: The entire system's unpredictability hinges on the entropy and behaviour of your `fx`. Avoid anything guessable or biased and the use of periodic mathematical functions which can lead to predictable or repeating outputs.\n- **Use Secure Seeds \u0026 Avoid Reuse**: Generate initial seeds using the provided `VernamVeil.get_initial_seed()` method which is cryptographically safe. Treat each `initial_seed` as a one-time-use context and use a fresh initial seed for every encode/decode session. During the same session, the API returns the next seed you should use for the following call.\n- **True One-Time Pad Support**: If you use the `OTPFX` callable wrapper with a truly random keystream (generated from a physical entropy source), VernamVeil can be configured to operate as an One-Time Pad cypher. The keystream must be at least as long as the message, used only once, and never reused for any other message. Generating such keystreams is challenging in practice; pseudo-random generators (including cryptographically secure ones) do not provide the same guarantees as a true one-time pad.\n- **Message Ordering \u0026 Replay**: VernamVeil is designed to be nonce-free by evolving the seed with each message or chunk, ensuring keystream uniqueness as long as each session uses a distinct `initial_seed`. The Synthetic IV mechanism, by incorporating the current timestamp, ensures each cyphertext is unique even for identical messages and seeds, and specifically provides resilience against accidental seed reuse for the first message. However, the cypher itself does not guarantee full replay protection or enforce message ordering; these must be handled by the application. For strict anti-replay or ordering requirements, implement explicit mechanisms (such as sequence numbers or nonces) at a higher layer.\n\n---\n\n## 📝 Examples\n\n### ✉️ Encrypting and Decrypting Multiple Messages\n\n```python\nfrom vernamveil import VernamVeil, generate_default_fx\n\n\n# Step 1: Generate a custom fx using the helper\nfx = generate_default_fx()  # remember to store fx.source_code securely\n\n# Step 2: Generate a random initial seed for encryption\ninitial_seed = VernamVeil.get_initial_seed()  # remember to store this securely\n\n# Step 3: Initialise VernamVeil with the custom fx and parameters\ncypher = VernamVeil(fx, chunk_size=64, decoy_ratio=0.2)\n\n# Step 4: Encrypt multiple messages in one session\nmessages = [\n    \"This is a secret message!\",\n    \"another one\",\n    \"and another one\"\n]\nencrypted = []\nseed = initial_seed\nfor msg in messages:\n    # Each message evolves the seed for the next one\n    enc, seed = cypher.encode(msg.encode(), seed)\n    encrypted.append(enc)\n\n# Step 5: Decrypt multiple messages in one session\nseed = initial_seed\nfor original, enc in zip(messages, encrypted):\n    # Each message evolves the seed for the next one\n    dec, seed = cypher.decode(enc, seed)\n    assert dec.decode() == original\n```\n\n### 📂 Encrypting and Decrypting Files\n\n```python\nfrom vernamveil import VernamVeil, generate_default_fx\n\n\n# Step 1: Generate a custom fx using the helper\nfx = generate_default_fx()  # remember to store fx.source_code securely\n\n# Step 2: Generate a random initial seed for encryption\ninitial_seed = VernamVeil.get_initial_seed()  # remember to store this securely\n\n# Step 3: Initialise VernamVeil with the custom fx and parameters\ncypher = VernamVeil(fx, chunk_size=64, decoy_ratio=0.2)\n\n# Step 4: Encrypt a file\ncypher.process_file(\"encode\", \"plain.txt\", \"encrypted.dat\", initial_seed)\n\n# Step 5: Decrypt the file\ncypher.process_file(\"decode\", \"encrypted.dat\", \"decrypted.txt\", initial_seed)\n```\n\n\u003e **Note:**\n\u003e The `process_file` method uses background threads and queues to perform asynchronous I/O for both reading and writing, enabling efficient processing of large files without blocking the main thread.\n\n---\n\n## 🧪 How to Design a Custom `fx`\n\n\u003e ⚠️ **Warning: Designing cryptographic functions is difficult and risky**\n\u003e\n\u003e Creating your own cryptographic methods is a major undertaking, and even small mistakes can introduce severe vulnerabilities. The greatest weakness of this cypher is that it allows users to supply their own `fx` functions: a non-expert can easily \"shoot themselves in the foot\" by designing a function that is predictable, biased, or otherwise insecure, potentially making the encryption trivial to break. This project is strictly educational and not intended for real-world security. The following section provides some basic principles for designing `fx` functions, but it is not a comprehensive guide to cryptographic engineering.\n\nWhen creating your own key stream function (`fx`), it is essential to follow best practices to ensure the unpredictability and security of your cypher. Poorly designed functions can introduce vulnerabilities, bias, or even make the encryption reversible by attackers. Use the following guidelines:\n\n- **Uniform \u0026 Non-Constant Output**: Your `fx` should produce diverse, unpredictable outputs for different input indices. Avoid constant, biased, low-entropy, or periodic mathematical functions. The distribution of outputs should be as uniform as possible. Use a standard cryptographic pseudorandom function (PRF) before returning the output.\n- **Seed Sensitivity**: The output of `fx` must depend on the secret seed. Changing the seed should result in completely different outputs.\n- **Type Correctness**: The function must return a `bytes` or a NumPy `uint8` array in vectorised mode.\n- **Determinism**: `fx` must be deterministic for the same inputs. Do not use external state or randomness inside your function.\n- **Avoid Data-Dependent Branching or Timing**: Do not introduce data-dependent branching or timing in your `fx`, as this can lead to side-channel attacks.\n- **Performance**: Complex or slow `fx` functions will slow down encryption and decryption. Test performance if speed is important for your use case.\n\n**Recommended approach:**\nApply a unique transformation to the input index using a function that incorporates constant but randomly sampled parameters to make each `fx` instance unpredictable. Then, combine the result with the secret seed using a cryptographically secure keyed hash method or HMAC. This ensures your keystream is both unpredictable and securely bound to your secret.\n\n**Always test your custom `fx`** with the provided `check_fx_sanity` utility before using it for encryption. Note that this method only performs very basic checks and cannot guarantee cryptographic security; it may catch common mistakes, but passing all checks does not mean your function is secure.\n\nBelow we provide some example `fx` methods to illustrate these principles in practice:\n\n### 🧠 A more robust, Keyed Hash-based scalar `fx` (not cryptographically standard)\n\n```python\nimport hashlib\nfrom vernamveil import FX\n\n\ndef keystream_fn(i: int, seed: bytes) -\u003e bytes:\n    # Implements a customisable fx function based on a 10-degree polynomial transformation of the index,\n    # followed by a cryptographically secure keyed hash (BLAKE2b) output.\n    # Note: The security of `fx` relies entirely on the secrecy of the seed and the strength of the keyed hash.\n    # The polynomial transformation adds uniqueness to each fx instance but does not contribute additional entropy.\n    weights = [24242, 68652, 77629, 55585, 32284, 78741, 70249, 39611, 54080, 73198, 12426]\n\n    # Transform index i using a polynomial function to introduce uniqueness on fx\n    current_pow = 1\n    result = 0\n    for weight in weights:\n        result = (result + weight * current_pow) \u0026 0xFFFFFFFFFFFFFFFF\n        current_pow = (current_pow * i) \u0026 0xFFFFFFFFFFFFFFFF\n\n    # Hash using BLAKE2b\n    return hashlib.blake2b(i.to_bytes(8, \"big\"), key=seed).digest()\n\n\nfx = FX(keystream_fn, block_size=64, vectorise=False)\n```\n\n### 🏎️ A fast version of the above `fx` that uses NumPy vectorisation and the `nphash` C module\n\n```python\nimport numpy as np\nfrom vernamveil import FX, hash_numpy\n\n\ndef keystream_fn(i: np.ndarray, seed: bytes) -\u003e np.ndarray:\n    # Implements a customisable fx function based on a 10-degree polynomial transformation of the index,\n    # followed by a cryptographically secure keyed hash (BLAKE2b) output.\n    # Note: The security of `fx` relies entirely on the secrecy of the seed and the strength of the keyed hash.\n    # The polynomial transformation adds uniqueness to each fx instance but does not contribute additional entropy.\n    weights = np.array([24242, 68652, 77629, 55585, 32284, 78741, 70249, 39611, 54080, 73198, 12426], dtype=np.uint64)\n\n    # Transform index i using a polynomial function to introduce uniqueness on fx\n    # Compute all powers: shape (len(i), degree + 1)\n    powers = np.power.outer(i, np.arange(11, dtype=np.uint64))\n\n    # Weighted sum (polynomial evaluation)\n    result = np.dot(powers, weights)\n\n    # Hash using BLAKE2b\n    return hash_numpy(result, seed, \"blake2b\")  # uses C module if available, else NumPy fallback\n\n\nfx = FX(keystream_fn, block_size=64, vectorise=True)\n```\n\n### 🛡️ A cryptographically strong Keyed Hash BLAKE2b `fx` (vectorised \u0026 C-accelerated)\n\n```python\nimport numpy as np\nfrom vernamveil import FX, hash_numpy\n\n\ndef keystream_fn(i: np.ndarray, seed: bytes) -\u003e np.ndarray:\n    # The secure default `fx` of the VernamVeil cypher.\n    # Implements a standard keyed hash-based pseudorandom function (PRF) using BLAKE2b.\n    # The output is deterministically derived from the input index `i` and the secret `seed`.\n    # Security relies entirely on the secrecy of the seed and the cryptographic strength of the keyed hash.\n\n    # Hash using BLAKE2b\n    return hash_numpy(i, seed, \"blake2b\")  # uses C module if available, else NumPy fallback\n\n\nfx = FX(keystream_fn, block_size=64, vectorise=True)\n```\n\n### 🏁️ A cryptographically strong and Fast Hash BLAKE3 `fx` (only available with C-acceleration)\n\n```python\nimport numpy as np\nfrom vernamveil import FX, hash_numpy\n\n\ndef keystream_fn(i: np.ndarray, seed: bytes) -\u003e np.ndarray:\n    # Implements a standard keyed hash-based pseudorandom function (PRF) using BLAKE3.\n    # BLAKE3 is only available when the C extension is installed, and benefits from hardware acceleration\n    # (SIMD and assembly) for maximum performance.\n    # The output is deterministically derived from the input index `i` and the secret `seed`.\n    # Security relies entirely on the secrecy of the seed and the cryptographic strength of the keyed hash.\n\n    # Hash using BLAKE3\n    return hash_numpy(i, seed, \"blake3\", hash_size=512)  # requires the C module\n\n\nfx = FX(keystream_fn, block_size=512, vectorise=True)\n```\n\n---\n\n## 🎲 One-Time Pad (OTP) Mode\n\nVernamVeil can operate as a one-time pad (OTP) cypher when provided with a truly random keystream. By supplying a keystream of random bytes (generated from a physical entropy source) through the `fx` interface, VernamVeil achieves the properties of an OTP: the keystream must be at least as long as the message, used only once, and never reused for any other message. Note that pseudo-random generators, including cryptographically secure ones, do not provide the same guarantees as a true one-time pad.\n\nTo facilitate OTP mode, VernamVeil provides the `OTPFX` utility. `OTPFX` is a callable wrapper that allows you to use an externally generated, truly random keystream as a drop-in replacement for a function-based `fx`. You provide a list of random byte blocks (each of a fixed size), and `OTPFX` ensures these are used sequentially as the keystream during encryption and decryption.\n\nExample:\n\n```python\nfrom vernamveil import OTPFX, VernamVeil\n\n\ndef get_true_random_bytes(n: int) -\u003e bytes:\n    # Replace with a function that returns n bytes from a true random source.\n    # For real OTP, use a true random source (e.g., hardware RNG, quantum RNG, etc.)\n    # Using `secrets` or `os.urandom` is not truly random and does not provide the same guarantees.\n    raise NotImplementedError()\n\n# Prepare a keystream of random blocks\nblock_size = 64\nkeystream = [get_true_random_bytes(block_size) for _ in range(100)]\n\n# Create a cypher with the OTPFX instance\nfx = OTPFX(keystream, block_size=block_size, vectorise=False)\ncypher = VernamVeil(fx)\n\n# Encrypt a message as per usual\ninitial_seed = VernamVeil.get_initial_seed()  # remember to store this securely\nencrypted_message = cypher.encrypt(b\"some message\", initial_seed)\n\n# Optionally clip the keystream to the used portion\nfx.keystream = fx.keystream[:fx.position]  # remember to store this securely\n\n# Reset the pointer for decryption\nfx.position = 0\n\n# Decrypt the message\ndecrypted_message = cypher.decrypt(encrypted_message, initial_seed)\n\n```\n\n**Note:** The keystream must be truly random, at least as long as the message, and never reused. Reusing a keystream completely breaks the security of OTP encryption.\n\n\u003e **Warning:**\n\u003e Do **not** use or test your `OTPFX` instance (e.g., by calling it or running sanity checks) before actual encryption or decryption. Any use will consume part of the keystream, which cannot be recovered, and will cause decryption to fail. Always use a fresh, unused `OTPFX` instance for each encryption/decryption operation or reset its `fx.position` to `0`.\n\n---\n\n## 🧰 Provided `fx` Utilities\n\nVernamVeil includes helper tools to make working with key stream functions easier:\n\n- `OTPFX`: A callable wrapper for using externally generated, one-time-pad keystreams as a drop-in replacement for function-based `fx`.\n- `check_fx_sanity`: Runs basic sanity checks on your custom `fx` to ensure it produces diverse and seed-sensitive outputs.\n- `generate_keyed_hash_fx` (same as `generate_default_fx`): Generates a deterministic `fx` function that applies a specified hash algorithm (e.g., BLAKE2b, BLAKE3 or SHA-256) directly to the index and seed. The seed is the only secret key but the keyed hash is a cryptographically strong and proven `fx`. Supports both scalar and vectorised (NumPy) modes. This is the recommended secure default `fx` for the VernamVeil cypher. The BLAKE3 option is only available with the C extension.\n- `generate_polynomial_fx`: Generates a random `fx` function that first transforms the index using a polynomial with random weights, then applies keyed hashing (BLAKE2b) for cryptographic output. Supports both scalar and vectorised (NumPy) modes.\n- `load_fx_from_file`: Loads a custom `fx` function from a Python file. This is useful for testing and validating your own implementations. It uses `importlib` internally to import the `fx`. **Never use this with files from untrusted sources, as it can run arbitrary code on your system.**\n\nThese utilities help you prototype and validate your own key stream functions before using them in encryption.\n\nExample:\n\n```python\nfrom vernamveil import generate_default_fx, check_fx_sanity\n\n# Generate a vectorised fx function\nfx = generate_default_fx(vectorise=True)\n\n# Show the generated function's source code\nprint(\"Generated fx source code:\\n\", fx.source_code)\n\n# Check if the generated fx passes basic sanity checks\nseed = b\"mysecretseed\"\nnum_samples = 100\npassed = check_fx_sanity(fx, seed, num_samples)\nprint(\"Sanity check passed:\", passed)\n```\n\n---\n\n## 🕵️ Plausible Deniability Utilities\n\nPlausible deniability in cryptography enables users to plausibly claim that an encrypted message contains different content from its true meaning. This is especially valuable in situations where an adversary may compel a user to reveal keys or decrypt sensitive data. By constructing alternative cryptographic parameters (such as a fake `fx` function and seed) the user can make the cyphertext decrypt to an innocuous decoy message, while the genuine message remains secure and undisclosed.\n\nHere is an example of Forging a Decoy Decryption:\n\n```python\nfrom vernamveil import VernamVeil, generate_default_fx, forge_plausible_fx\n\n# Original cypher and encryption\nfx = generate_default_fx()\nreal_cypher = VernamVeil(fx, padding_range=(5, 25), chunk_size=32, decoy_ratio=0.3)\nsecret_message = b\"Top secret!\"\nseed = real_cypher.get_initial_seed()\ncyphertext, _ = real_cypher.encode(secret_message, seed)\n\n# Decoy message to plausibly reveal\ndecoy = b\"This is a harmless message. Nothing to see here. Look away!\"\n\n# Forge plausible fx and seed\nplausible_fx, fake_seed = forge_plausible_fx(real_cypher, cyphertext, decoy)\n\n# Use the forged fx and seed to decrypt the cyphertext to the decoy\n# Note: The SIV and MAC must be turned off for this to work.\ndecoy_cypher = VernamVeil(plausible_fx, padding_range=(5, 25), chunk_size=32, decoy_ratio=0.3,\n                          siv_seed_initialisation=False, auth_encrypt=False)\nrevealed, _ = decoy_cypher.decode(cyphertext, fake_seed)\nprint(revealed)  # b'This is a harmless message. Nothing to see here. Look away!'\n```\n\nThis approach allows you to demonstrate that a given cyphertext could plausibly contain a harmless message, providing a credible alternative explanation under duress, while the original secret remains protected.\n\n---\n\n## 🖥️ Command-Line Interface (CLI)\n\nVernamVeil provides a convenient CLI for file encryption and decryption. The CLI supports both encoding (encryption) and decoding (decryption) operations, allowing you to specify custom key stream functions (`fx`) and seeds, or have them generated automatically.\n\n### ⚙️ Features\n\n- **Encrypt and decrypt files or streams** using a user-defined or auto-generated `fx` function and seed represented in hex.\n- **Auto-generate `fx.py` and `seed.hex`** during encoding if not provided; these files are saved in the current working directory.\n- **Custom `fx` and seed support**: Supply your own `fx.py` and `seed.hex` for both encoding and decoding.\n- **Configurable parameters**: Adjust chunk size, delimiter size, padding, decoy ratio, and more. Set `--verbosity info` to see progress information (off by default).\n- **Sanity checks**: Optionally verify that your `fx` function is suitable for cryptographic use. These checks are automatically skipped for `OTPFX` to avoid consuming the keystream.\n\n### 💻 Usage\n\n```bash\n# Encrypt a file with auto-generated fx and seed\nvernamveil encode --infile plain.txt --outfile encrypted.dat\n\n# Encrypt a file with a custom fx function and seed\nvernamveil encode --infile plain.txt --outfile encrypted.dat --fx-file fx.py --seed-file seed.hex\n\n# Decrypt a file (requires the same fx and seed used for encryption)\nvernamveil decode --infile encrypted.dat --outfile decrypted.txt --fx-file fx.py --seed-file seed.hex\n\n# Encrypt and Decrypt from stdin to stdout (using - or omitting the argument)\nvernamveil encode --infile - --outfile - --fx-file fx.py --seed-file seed.hex \u003c plain.txt \u003e encrypted.dat\nvernamveil decode --infile - --outfile - --fx-file fx.py --seed-file seed.hex \u003c encrypted.dat \u003e decrypted.txt\n\n# Enable sanity check for fx and seed during encryption\nvernamveil encode --infile plain.txt --outfile encrypted.dat --fx-file fx.py --seed-file seed.hex --check-sanity\n```\n\n\u003e ⚠️ **Warning: CLI Parameter Consistency**\n\u003e\n\u003e When decoding, you **must** use the exact same parameters (such as `--chunk-size`, `--delimiter-size`, `--padding-range`, `--decoy-ratio`, `--siv-seed-initialisation`, `--auth-encrypt` and `--hash-name`) as you did during encoding.\n\u003e\n\u003e For example, the following will **fail** with a `Authentication failed: MAC tag mismatch.` error because the `--chunk-size` parameter differs between encoding and decoding:\n\u003e\n\u003e ```bash\n\u003e vernamveil encode --infile plain.txt --outfile encrypted.dat --chunk-size 2048\n\u003e vernamveil decode --infile encrypted.dat --outfile decrypted.txt --chunk-size 1024 --fx-file fx.py --seed-file seed.hex\n\u003e ```\n\u003e\n\u003e **Always use identical parameters for both encoding and decoding.** Any mismatch will result in decryption failure. The only exception is the `--buffer-size` parameter, which can be different for encoding and decoding.\n\n### 🗄️ File Handling\n\n- For both `--infile` and `--outfile`, passing `-` or omitting the argument means `stdin`/`stdout` will be used. This allows for piping and streaming data directly.\n- When encoding **without** `--fx-file` or `--seed-file`, the CLI generates `fx.py` and `seed.hex` in the current working directory. The absolute paths to these files are displayed after generation. **Store these files securely**; they are required for decryption.\n- When decoding, you **must** provide both `--fx-file` and `--seed-file` pointing to the originals used for encryption.\n- For safety, the CLI will **not overwrite** existing output files, `fx.py`, or `seed.hex`. If these files already exist, you must delete or rename them manually before running the command. Overwrite protection does **not** apply when outputting to `stdout`.\n\n\u003e ⚠️ **Warning: Binary Output to Terminals**\n\u003e\n\u003e If you use `-` or omit `--outfile`, the output will be written to `stdout` in binary mode. Writing binary data directly to a terminal may corrupt your session. Only redirect binary output to files or pipes, not to an interactive terminal.\n\nSee `vernamveil encode --help` and `vernamveil decode --help` for all available options.\n\n---\n\n## 🛠️ Technical Details\n\n- **Compact Implementation**: The core cypher implementation (`_vernamveil.py`) is about 200 lines of code, excluding comments, documentation and empty lines.\n- **External Dependencies**: Built using only Python's standard library, with NumPy being optional for vectorisation.\n- **Optional C/C++ Module for Fast Hashing and Byte Search**: An optional C/C++ module (`nphash`) is provided, built using [cffi](https://cffi.readthedocs.io/), which enables fast BLAKE2b, BLAKE3, and SHA-256 keyed hashing for NumPy arrays, and also provides efficient byte search utilities for internal use. BLAKE3 support is exclusive to this extension and benefits from hardware acceleration for optimal performance. Specifically, BLAKE3 uses the [official implementation](https://github.com/BLAKE3-team/BLAKE3) and can utilise SIMD instruction sets such as SSE2, SSE4.1, AVX2, AVX512F, AVX512VL (on x86_64), and NEON (on ARM), as well as hand-written assembly, where supported by your hardware and compiler. These acceleration features are detected and enabled automatically during the build process. The extension includes both C and C++ code (the C++ component is from the BLAKE3 project), so both a C and a C++ compiler (for example, gcc, g++, or MSVC) are required to build it. See the [`nphash` README](nphash/README.md) for further details.\n- **Tested with**: Python 3.10 and NumPy 2.2.5.\n\n### 🔧 Installation\n\nTo install the library with all optional dependencies (development tools, NumPy for vectorisation, and cffi for the C module):\n```\npip install .[dev,numpy,cffi]\n```\n\n- The `[dev]` extra installs development and testing dependencies.\n- The `[numpy]` extra enables fast vectorised operations.\n- The `[cffi]` extra is required for building the `nphash` C extension for accelerated BLAKE2b, BLAKE3 and SHA-256 in NumPy-based `fx` functions. **You need to compile the C extension afterwards. See below.**\n\n### ⚡ Fast Vectorised `fx` Functions\n\nIf you want to use fast vectorised key stream functions, install with both `numpy` and `cffi` enabled. The included `nphash` C module provides high-performance BLAKE2b, BLAKE3, and SHA-256 keyed hash implementations for NumPy arrays, which are automatically used by `generate_default_fx(vectorise=True)` when available. BLAKE3 is only available via the C extension and is hardware-accelerated for maximum speed. If the extension is not present, a slower pure NumPy fallback is used (excluding BLAKE3).\n\n**To use the C extension you must build it from source.** For more details, see [`nphash/README.md`](nphash/README.md).\n\n---\n\n## 🚦 Benchmarks: VernamVeil vs AES-256-CBC\n\nVernamVeil is competitive with OpenSSL's AES-256-CBC in terms of speed. As both cyphers leverage hardware acceleration, the exact benchmarks vary significantly from machine to machine. On an Apple MacBook Pro with an M1 Max chip, VernamVeil is about 15% faster. Note that our benchmarks test VernamVeil with all its default security features enabled (Authenticated Encryption, Synthetic IV, and Obfuscation), whereas AES-256-CBC performs raw, unauthenticated encryption.\n\n### 📊 Summary\n\nThe following benchmarks compare VernamVeil (NumPy vectorisation, C extension enabled, and `blake3` hashing) to OpenSSL's AES-256-CBC on the same machine. The tests were run on a 1GB random file, measuring the time taken for encoding and decoding operations.\n\n| Algorithm    | Encode Time | Decode Time | Total Time |\n|--------------|-------------|-------------|------------|\n| VernamVeil   | 1.119 s     | 1.220 s     | 2.339 s    |\n| AES-256-CBC  | 1.760 s     | 0.950 s     | 2.710 s    |\n\n### ‍💻 Benchmark Setup\n\n**1. Create a 1GB random file:**\n```bash\ndd if=/dev/urandom of=/tmp/original.bin bs=1M count=1024 status=progress\n```\n\n**2. Generate a random 256-bit key and IV for AES-256-CBC:**\n```bash\nopenssl rand -hex 32 \u003e key.hex\nopenssl rand -hex 16 \u003e iv.hex\n```\n\n### 🐇 VernamVeil (Vectorised + C extension + Keyed Hash `fx` using BLAKE3)\n\n**Encoding:**\n```bash\nvernamveil encode --infile /tmp/original.bin --outfile /tmp/output.enc --fx-file fx.py --seed-file seed.hex --buffer-size 134217728 --chunk-size 1048576 --delimiter-size 64 --padding-range 100 200 --decoy-ratio 0.01 --hash-name blake3 --verbosity info\n```\n_Time: 1.119s_\n\n**Decoding:**\n```bash\nvernamveil decode --infile /tmp/output.enc --outfile /tmp/output.dec --fx-file fx.py --seed-file seed.hex --buffer-size 136349200 --chunk-size 1048576 --delimiter-size 64 --padding-range 100 200 --decoy-ratio 0.01 --hash-name blake3 --verbosity info\n```\n_Time: 1.220s_\n\n### 🐢 AES-256-CBC (OpenSSL)\n\n**Encoding:**\n```bash\ntime openssl enc -aes-256-cbc -in /tmp/original.bin -out /tmp/output.enc -K $(cat key.hex) -iv $(cat iv.hex) -pbkdf2\n```\n_Time: 1.760s_\n\n**Decoding:**\n```bash\ntime openssl enc -d -aes-256-cbc -in /tmp/output.enc -out /tmp/output.dec -K $(cat key.hex) -iv $(cat iv.hex) -pbkdf2\n```\n_Time: 0.950s_\n\n---\n\n## 📚 Documentation\n\nFull API and usage docs are available at: [https://datumbox.github.io/VernamVeil/](https://datumbox.github.io/VernamVeil/)\n\n---\n\n## 🤝 Contributing\n\nContributions, bug reports, and feature requests are welcome! Please open an issue or pull request on [GitHub](https://github.com/datumbox/VernamVeil).\n\n---\n\n## 📄 Copyright \u0026 License\n\nCopyright (C) 2025 [Vasilis Vryniotis](http://blog.datumbox.com/author/bbriniotis/).\n\nThe code is licensed under the [Apache License, Version 2.0](https://github.com/datumbox/VernamVeil/blob/main/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatumbox%2Fvernamveil","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatumbox%2Fvernamveil","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatumbox%2Fvernamveil/lists"}