https://github.com/southwinds-io/volta
An embedded, zero-dependency Go library for multi-tenant data encryption and zero-trust secret management.
https://github.com/southwinds-io/volta
cryptography encryption-decryption gdpr go golang golang-library hipaa pci-dss pii secret-management
Last synced: 3 months ago
JSON representation
An embedded, zero-dependency Go library for multi-tenant data encryption and zero-trust secret management.
- Host: GitHub
- URL: https://github.com/southwinds-io/volta
- Owner: southwinds-io
- License: other
- Created: 2025-06-25T14:50:09.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2025-06-28T10:03:23.000Z (3 months ago)
- Last Synced: 2025-06-28T11:22:44.730Z (3 months ago)
- Topics: cryptography, encryption-decryption, gdpr, go, golang, golang-library, hipaa, pci-dss, pii, secret-management
- Language: Go
- Homepage:
- Size: 2.18 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
- License: LICENSE
- Audit: audit/file_logger.go
Awesome Lists containing this project
README
# Volta: An Embedded Go Library for Multi-Tenant Encryption and Secrets Management
Volta is a zero-dependency Go library designed for multi-tenant encryption and secret management, intended to be
embedded directly within an application.
It is used by SouthWinds Tech to protect Personally Identifiable Information (PII) and other sensitive data within their
own applications. It establishes a cryptographic boundary within the application's process, providing data protection
controls without reliance on external services.
### Why Volta? An Architectural and Comparative Context
A common question for architects and developers is, *"Why use Volta instead of a dedicated secrets manager like HashiCorp Vault, AWS
Secrets Manager, or just a simpler approach like AES-GCM with keys in environment variables?"*
The answer lies in understanding the specific problem Volta is designed to solve. Volta does not aim to compete with
centralised secret vaults; it is designed to **complement them** by addressing a narrow, yet critical, challenge: *
*scalable, application-level, multi-tenant data encryption with minimal operational overhead.**
Volta is best understood as an implementation of the **[Embedded Library Model](docs/arc_patterns.md)** for data protection, distinct from the *
*External Service Model** of traditional vaults.
Here is a pragmatic comparison of the architectural patterns:
| Aspect | Centralised Vaults (e.g. HashiCorp Vault, AWS SM) | Embedded Library Pattern (e.g. Volta) |
|:---------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Primary Problem Domain** | **Secret Brokering & Identity Management.** Centralising and brokering access to secrets (database credentials, API keys) for infrastructure and applications. | **Application-Level Data Encryption.** Per-tenant cryptographic isolation of sensitive data (like PII) stored in a shared database or storage layer. |
| **Operational Model** | Requires deploying, managing, and securing a **separate, highly-available service cluster.** | **Embedded directly into the application.** No external service to manage, reducing operational scope to application configuration. |
| **Typical Workload** | **Low-frequency operations.** Applications fetch configuration or short-lived credentials at startup or infrequently. | **High-frequency operations.** Encrypting and decrypting data fields on nearly every read/write path that is considered to manage [PII](https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/personal-information-what-is-it/what-is-personal-information-a-guide/) within the application's business logic. |
| **Network Overhead** | Each cryptographic operation (if using Encryption-as-a-Service) incurs network latency. | **Zero network latency** for cryptographic operations, as they occur in-process. |
| **Multi-Tenancy** | Handled via policies, paths, and namespaces at the infrastructure level. | A **core, native architectural primitive.** The library is fundamentally designed to manage distinct cryptographic contexts for a large amount of tenants. |
#### The Hybrid Pattern: Complementing, Not Competing
The most powerful approach is a **[hybrid model](docs/arc_patterns.md)** that leverages the strengths of both patterns. While Volta excels at
high-frequency, in-process encryption, it still faces the classic "first secret" problem: *where does the application get
the master passphrase to unlock a tenant's vault?*
This is where centralised vaults shine. The ideal architecture becomes:
1. **Bootstrapping (The "First Secret"):** An application, using a platform-managed identity (e.g., AWS IAM Role,
Kubernetes Service Account), authenticates to a centralised secret manager (AWS Secrets Manager, HashiCorp Vault).
2. **Secure Handoff:** The secret manager securely provides the master passphrase to the application instance at
startup. This is a low-frequency, highly secure event.
3. **Application-Level Encryption:** Once bootstrapped, the embedded Volta library uses the passphrase to manage its key
hierarchy and perform high-frequency, low-latency encryption and decryption of tenant data for the application's
entire lifecycle.
In this model, the centralised vault handles **infrastructure-level identity and bootstrapping**, while Volta provides
the specialised, high-performance **application-level cryptographic engine**.
The following diagram shows how the hybrid pattern could work:
#### What About Simpler Approaches (e.g., AES-GCM)?
For a single-tenant application with one or two keys, a direct implementation of a cryptographic cipher might seem
sufficient. However, this approach does not scale to a multi-tenant SaaS environment and lacks critical features that
Volta provides out-of-the-box:
* **Key Lifecycle Management:** No built-in support for secure key generation, rotation, or destruction per tenant.
* **Auditing:** No integrated, pluggable audit trail for all cryptographic and administrative operations.
* **Secure Storage:** Lacks a structured, encrypted format for persisting a large number of tenant keys.
* **In-Memory Protection:** Does not provide safeguards for keys and plaintext while resident in application memory.
Volta offers a practical, robust implementation for a specific and complex problem, providing the architectural
components necessary for secure, compliant, and maintainable multi-tenant applications. Its goal is to serve as a
reference pattern for this problem, reducing redundant-engineering effort across the industry.
### Design Principles
Volta's architecture is guided by the following principles:
* **Multi-tenancy:** Provides cryptographic isolation for distinct tenants (e.g., applications, services, or
organizational units). Each tenant operates with its own dedicated keys, secrets, and audit trails.
* **Pluggable Backends:** Storage and audit logging are defined by interfaces (`persist.Store`, `audit.Logger`),
enabling integration with various persistence and observability systems.
* **In-Memory Protection:** Utilizes memory-protection libraries (e.g., `memguard`) to secure keys and plaintext secrets
while resident in application memory.
* **Embedded, API-less Design:** Operates as a library without an external REST API. This minimizes the attack surface
and places the control of cryptographic operations within the application's own security context.
* **Non-Exportable Keys:** Encryption keys are managed internally and are not designed to be exported.
* **Administrative CLI:** A command-line interface is provided for out-of-band vault management and administrative
operations.
### Key Components
1. **Vault Manager (`VaultManagerService`):**
The top-level service that manages the lifecycle of all tenant vaults. It is the primary entry point for an
application, responsible for retrieving, initializing, and closing tenant-specific vault instances.
2. **Vault (`VaultService`):**
Represents an individual tenant's vault. It is the interface for all cryptographic operations and secret management
for a single tenant, including encryption, decryption, secret storage, and key management.
### [Feature Set](docs/features.md)
* **Per-Tenant Cryptography:** Securely encrypt and decrypt data with keys isolated to each tenant, forming the foundation of the multi-tenant security model.
* **Secret Management:** Provides full CRUD (Create, Retrieve, Update, Delete) operations for managing secrets like API keys, credentials, or other sensitive configuration.
* **Key Lifecycle Management:** Automates secure, per-tenant key management, including generation, rotation, and destruction, complete with auditable reasoning.
* **Backup and Restore:** Supports the creation of encrypted, passphrase-protected backups of individual tenant vaults and enables secure restoration for disaster recovery.
* **In-Memory Protection:** Limits the exposure of plaintext secrets in application memory using techniques like memory-protection libraries and scoped access functions.
* **Pluggable Backends:** Designed for extensibility with pluggable interfaces for both the persistence layer (storage) and the audit logging system, allowing integration with existing infrastructure.
* **Administrative CLI:** Includes a command-line interface for performing out-of-band administrative tasks, such as bulk operations, backups, and maintenance across tenants.
### Basic Usage
#### Storing and Retrieving Secrets
The example [here](examples/secrets/main.go) demonstrates initialising the vault manager, accessing a tenant's vault,
storing and retrieving secrets.
#### Encryption of sensitive data
The example [here](examples/encryption/main.go) demonstrates initialising the vault manager, accessing a tenant's vault,
and using it to directly **encrypt and decrypt sensitive data**, such as PII.
#### Key Encryption Key (KEK) Rotation
The example [here](examples/kek_rotation/main.go) demonstrates the process of Key Encryption Key (KEK) rotation, while maintaining access to existing encrypted data without requiring data re-encryption.
#### Data Encryption Key (DEK) Rotation
The example [here](examples/dek_rotation/main.go) demonstrates the complete lifecycle of Data Encryption Key (DEK) rotation and its impact on encrypted data accessibility.
#### Volta's Backup & Restore
The example [here](examples/backup/main.go) demonstrates the complete backup/restore cycle of a vault with encrypted PII and secrets.
#### KEK Rotation Performance Testing
The example [here](examples/performance/readme.md) provides a simple performance test tool to evaluate the speed and reliability of Key Encryption Key (KEK) rotation operations across
multiple tenants using Volta.
### Installation
```sh
go get southwinds.dev/volta
```
### Security Model
The security of a tenant's vault relies on the strength and confidentiality of its **passphrase**. This passphrase is
used to derive the Key Encryption Key (KEK) that protects all of the tenant's Data Encryption Keys (DEKs).
Volta uses libraries like `memguard` to provide in-memory protection for sensitive data (e.g., keys, plaintext secrets),
making it more difficult for other processes on the host to inspect the application's memory. This is part of a
defense-in-depth strategy and is not a substitute for securing the host environment.
### Extensibility
Volta is designed to be adaptable to different infrastructures. Custom backends can be created by implementing the
following interfaces:
* **`persist.Store`:** For connecting to different storage systems (e.g., databases, cloud object storage, key-value
stores).
* **`audit.Logger`:** For routing audit events to centralized logging or observability platforms (e.g., ELK Stack,
Splunk, CloudWatch).
### Detailed Documentation
For a deeper understanding of the architecture, data structures, security patterns, and compliance guides, please see
the `/docs` directory:
- [Rationale, Architecture, and Advantages](docs/arc.md)
- [Feature Set](docs/features.md)
- [Architectural Patterns in Secret Management and Data Protection](docs/arc_patterns.md)
- [Architectural Patterns for GDPR Compliance using an Embedded Cryptographic Library](docs/gdpr.md)
- [Supporting HIPAA Technical Safeguards with an Embedded Cryptographic Library](docs/hipaa.md)
- [Architectural Patterns for PCI DSS Compliance using an Embedded Cryptographic Library](docs/pci_dss.md)
- [Facilitating Zero Trust Architecture with an Embedded Cryptographic Library](docs/zta.md)
- [Architectural Patterns for Secure Secret Handling in Concurrent Applications](docs/secret_handling.md)
- [Envelop Encryption and Key Rotation Scenarios](docs/envelope_enc.md)
### Core Concepts
Understanding the following concepts is fundamental to working with Volta:
1. **Tenancy:**
Volta's multitenant architecture enables a single instance to serve multiple, isolated tenants. Each tenant (e.g., a
distinct application, microservice, or department) operates within its own secure context, with dedicated encryption
keys, secrets, and audit trails. This isolation is crucial for managing data access and compliance across different
entities.
2. **Vault (`VaultService`):**
The `VaultService` interface represents an individual tenant's vault. It is the primary interaction point for all
cryptographic operations and secret management activities pertinent to that tenant. This includes encrypting and
decrypting data, managing the lifecycle of secrets (storing, retrieving, updating, deleting), and handling
tenant-specific encryption keys. Each `VaultService` instance is scoped to a single tenant.
3. **Vault Manager (`VaultManagerService`):**
The `VaultManagerService` interface acts as the top-level orchestrator for the entire Volta deployment. It manages
the lifecycle of tenant vaults (e.g., retrieving a vault for a specific tenant, closing tenant sessions). It also
supports administrative operations that span across multiple tenants, such as bulk key rotations, passphrase changes,
and querying aggregated audit logs from various tenants. This service is the main entry point for applications
interacting with Volta.
4. **Secrets:**
In Volta, a "secret" refers to any piece of sensitive information that requires secure storage and controlled access.
Examples include API keys, database credentials, personal identifiable information (PII), or configuration
parameters. Volta ensures secrets are encrypted at rest using strong cryptographic measures, with access governed by
the tenant's vault.
5. **Encryption Keys:**
Cryptographic keys are the cornerstone of Volta's security. Volta manages the lifecycle of these keys—including
generation, rotation, and secure destruction—on a per-tenant basis. Data and secrets are encrypted using these keys,
ensuring that even if the underlying storage is compromised, the sensitive information remains protected.
6. **Storage Backend (`Store` Interface):**
Volta abstracts its persistence layer through the `Store` interface. This design allows developers to integrate Volta
with various storage mechanisms, such as local file systems, distributed key-value stores, or cloud-based storage
services. While Volta may provide default implementations, the pluggable nature of the `Store` interface ensures that
organizations can use storage solutions that align with their existing infrastructure and policies. The store is
responsible for persisting encrypted key metadata, encrypted secrets data, and other essential vault state.
7. **Audit Backend (`audit.Logger` Interface):**
Comprehensive and immutable audit trails are critical for security monitoring, compliance, and forensic analysis.
Volta utilizes the `audit.Logger` interface to log significant events. These events include key management
operations (creation, rotation, destruction), secret access patterns (creation, retrieval, updates, deletion),
administrative actions, and failed operations. Like the storage backend, the audit backend is pluggable, enabling
integration with various logging systems, Security Information and Event Management (SIEM) tools, or custom audit
repositories.
8. **In-Memory Protection:**
Protecting sensitive data, especially cryptographic keys, while it is actively being used in application memory is a
significant challenge. Volta incorporates techniques to mitigate risks associated with memory exposure (e.g., from
memory dumps or certain side-channel attacks). This is achieved through mechanisms like leveraging secure memory
enclaves and careful handling of sensitive byte slices. The `VaultService.SecureMemoryProtection()` method provides
insight into the status or configuration of these protections.
### High-Level Operational Flow
A typical interaction with Volta involves the following conceptual steps:
1. **Initialization:**
An application initializes the `VaultManagerService`. This step typically involves configuring the desired storage
and audit backends.
2. **Tenant Vault Access:**
The application requests a `VaultService` instance for a specific `tenantID` from the `VaultManagerService`. If a
vault for this tenant doesn't exist and auto-creation is configured (or a specific provisioning step is taken), it
might be initialized.
3. **Secure Operations:**
Using the obtained `VaultService` instance, the application can:
* Encrypt and decrypt data.
* Store, retrieve, update, and delete secrets securely.
* Manage encryption keys (e.g., initiate key rotation).
* Utilize features like `UseSecret` for safely working with secret data in memory for a limited scope.
4. **Auditing:**
All significant operations are automatically audited through the configured audit backend, providing a trail of
activities.
5. **Tenant Closure:**
When a tenant's operations are complete, or during application shutdown, the `VaultManagerService` can be used to
close individual tenant vaults or all active vaults to release resources and ensure data is securely persisted.
## Operational Guidance and Extensibility
This section provides guidance on good practices for operating Volta, discusses important security considerations, and
outlines how Volta's pluggable architecture allows for extensibility.
### Good Practices for Utilizing Volta
Effective and secure operation of Volta relies on adhering to sound security principles and operational disciplines:
1. **Secure Tenant Passphrase Management:**
Volta uses a single passphrase to generate a derived key. The security of the entire tenant vault hinges on the
secrecy and strength of this initial passphrase. Applications integrating Volta must implement robust mechanisms for
managing these passphrases, ensuring they are not hardcoded, exposed in logs, or insecurely stored. Consider using
environment variables, dedicated secret management systems (for bootstrapping Volta's passphrase), or user-provided
input where appropriate.
2. **Regular Key Rotation:**
Volta supports key rotation (`VaultService.RotateKey` and `VaultManagerService.RotateAllTenantKeys`). Regularly
rotating Data Encryption Keys (DEKs) is a crucial security hygiene practice. It limits the amount of data encrypted
with any single key, reducing the impact if a key is ever compromised. Establish a policy for key rotation frequency
based on your organization's risk assessment and compliance requirements. Provide clear, auditable `reason` strings
for each rotation.
3. **Backup Management and Testing:**
* **Secure Backup Passphrases:** Backup archives created by Volta are encrypted. The passphrase used for backup
encryption must be strong and managed with extreme care, separately from the backup files themselves. Losing this
passphrase means losing access to the backup.
* **Regular Backups:** Implement a regular schedule for backing up tenant vaults. The frequency should align with
your Recovery Point Objectives (RPOs).
* **Test Restore Procedures:** Regularly test the restoration process (`VaultService.RestoreBackup`) in a
non-production environment to ensure backup integrity and verify that your recovery procedures are effective.
Untested backups provide a false sense of security.
* **Secure Backup Storage:** Store backup files in a secure, access-controlled location, separate from the primary
operational environment.
4. **Principle of Least Privilege:**
When designing applications that interact with `VaultManagerService`, ensure that access to administrative
functions (like `RotateAllTenantKeys`, `RotateAllTenantPassphrases`, or broad audit queries) is restricted to
privileged components or users. Individual application instances should typically only interact with their designated
`VaultService`.
5. **Audit Log Monitoring:**
Volta provides comprehensive auditing capabilities. Regularly monitor and review audit logs (`QueryAuditLogs`,
`GetAuditSummary`, etc.) for suspicious activities, unauthorized access attempts, frequent failed operations, or
unusual patterns. Integrate audit logs with your central Security Information and Event Management (SIEM) system if
possible.
6. **Graceful Shutdown and Resource Management:**
Ensure that `VaultService.Close()` (for individual tenant vaults) and `VaultManagerService.CloseTenant()` or
`VaultManagerService.CloseAll()` are called appropriately during application shutdown or when a tenant's session
ends. This ensures that resources are released, pending writes are flushed, and sensitive data is cleared from
memory.
7. **Scope of Plaintext Secret Exposure:**
While methods like `Decrypt` provide direct access to plaintext, exercise caution. Prefer using scoped access
patterns like those implied by `UseSecret` (not explicitly defined but a common secure pattern) where plaintext
exists in memory for the shortest necessary duration and within a controlled function scope. When handling plaintext
byte slices, ensure they are explicitly cleared from memory as soon as they are no longer needed, if not managed by
Volta's secure memory enclaves.
8. **Backend Selection:**
Choose storage (`StoreConfig`) and audit backends that align with your operational environment, security
requirements, and scalability needs. Consider factors like data durability, availability, access control mechanisms,
and existing infrastructure.
9. **Understanding Memory Protection:**
Be aware of the configured `memoryProtectionLevel` (as suggested by the internal `Vault` struct). Understand what
protections are offered (e.g., non-swappable memory, guard pages) and their limitations. While Volta aims to protect
critical data in memory, this is part of a defense-in-depth strategy.
### Security Considerations
Volta is designed with security as a core tenet, but its overall security posture also depends on the environment and
practices surrounding its deployment.
1. **Root of Trust - Tenant Passphrase/Key:**
The ultimate security of a tenant's data within Volta relies on the protection of its primary secret (e.g.,
passphrase or master key) from which its Key Encryption Key (KEK) is derived. If this root secret is compromised, the
encrypted data it protects can be decrypted. Volta itself does not manage this initial secret but uses it to
bootstrap its own internal key hierarchy.
2. **In-Memory Protection (`memguard`):**
Volta's use of libraries like `memguard` to protect sensitive data (keys, temporarily held secrets) in memory
significantly raises the bar for attackers. However, no in-memory protection is infallible. Sophisticated attackers
with sufficient privileges on the host OS (e.g., root access) or those exploiting severe kernel vulnerabilities might
still be able to bypass or undermine these protections. Physical memory attacks (e.g., cold boot attacks) also remain
a theoretical concern for extremely high-security scenarios.
3. **Storage Backend Security:**
Volta encrypts data before writing it to the `persist.Store`. However, the underlying storage system (filesystem,
database, cloud storage) must also be secured. This includes proper access controls, encryption at rest for the
storage medium itself (as an additional layer), and protection against unauthorized administrative access to the
backend.
4. **Audit Log Integrity and Security:**
The audit backend should be configured to be as tamper-evident as possible. For instance, append-only logs, logs
shipped to a separate, secured system, or cryptographic signing of log entries can enhance integrity. Unauthorized
modification or deletion of audit logs can mask malicious activity.
5. **Application-Layer Vulnerabilities:**
Volta secures secrets *within its boundary*. If the application embedding Volta has vulnerabilities (e.g., SQL
injection, Remote Code Execution, insecure API endpoints), these could be exploited to misuse Volta's `VaultService`
API, potentially leading to secret exposure. Application security is a critical complementary layer.
6. **Backup Security:**
Backup files (`BackupContainer`) contain encrypted copies of highly sensitive data. These files must be protected
with strong access controls, both in transit and at rest in their storage location. The backup encryption passphrase
is a critical secret.
7. **Physical Security:**
Physical access to the systems running Volta can bypass many software-based security controls. Ensure appropriate
physical security measures are in place for the host infrastructure.
8. **Operational Security:**
Secure operational practices, such as restricting access to production environments, robust identity and access
management, and regular security patching of the underlying OS and Go runtime, are essential.
### Extensibility and Pluggability
A key design principle of Volta is its adaptability through pluggable components. This allows integration into diverse
environments and extension with custom functionality.
1. **Pluggable Storage Backends (`persist.Store`):**
Volta allows applications to define their own persistence mechanisms for storing encrypted vault data. This is
achieved by implementing a `persist.Store` interface (the exact definition of which would be provided by Volta). A
custom storage backend might interact with various systems:
* Relational databases (PostgreSQL, MySQL, SQL Server)
* NoSQL databases (MongoDB, Cassandra, DynamoDB)
* Distributed key-value stores (etcd, Consul, ZooKeeper)
* Cloud storage services (AWS S3, Google Cloud Storage, Azure Blob Storage) beyond any built-in support.
* Proprietary in-house storage solutions.
The implementation would need to handle safe storage and retrieval of byte arrays (representing encrypted data and
metadata) keyed by tenant ID and potentially other identifiers.
The `StoreConfig` structure facilitates configuring these custom (or pre-built) backends.
2. **Pluggable Audit Backends (`audit.Logger`):**
Similarly, Volta enables custom audit logging implementations by adhering to an `audit.Logger` interface. This allows
audit events to be routed to various destinations:
* Local log files in specific formats (e.g., JSON, CEF).
* Centralized logging systems (Splunk, ELK Stack, Graylog).
* Cloud-native monitoring services (AWS CloudWatch Logs, Google Cloud Logging, Azure Monitor).
* Databases designed for audit trails.
A custom logger would receive `audit.Event` data (or similar structured information) and be responsible for its
durable and secure recording.
Volta allows to tailor its deployment to specific infrastructure and security requirements, ensuring that it can evolve
with changing needs without compromising its core mission of providing simple, secure secret management.
### License
Volta is licensed under the Apache 2.0 License. See [LICENSE](LICENSE) for the full license text.