An open API service indexing awesome lists of open source software.

https://github.com/wowemulation-dev/cascette-rs

A collection of Blizzard NGDP tools written for Rust.
https://github.com/wowemulation-dev/cascette-rs

blizzard casc ngdp reverse-engineering tact

Last synced: 6 months ago
JSON representation

A collection of Blizzard NGDP tools written for Rust.

Awesome Lists containing this project

README

          

# cascette-rs

Rust implementation of Blizzard's NGDP (Next Generation Distribution Pipeline)
for World of Warcraft emulation.

[![Discord](https://img.shields.io/discord/1394228766414471219?logo=discord&style=flat-square)](https://discord.gg/Q44pPMvGEd)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE-APACHE)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE-MIT)
[![CI Status](https://github.com/wowemulation-dev/cascette-rs/workflows/CI/badge.svg)](https://github.com/wowemulation-dev/cascette-rs/actions)
[![Crates.io Version](https://img.shields.io/crates/v/ngdp-client)](https://crates.io/crates/ngdp-client)
[![docs.rs](https://img.shields.io/docsrs/ngdp-client)](https://docs.rs/ngdp-client)

## ๐ŸŽฏ Project Status

**Current Version**: 0.4.3

### Core Components

| Component | Version | Status | Description |
| --------------- | ------- | ----------- | -------------------------------------------------- |
| `ngdp-bpsv` | 0.4.3 | โœ… Stable | BPSV parser/writer for NGDP formats |
| `ribbit-client` | 0.4.3 | โœ… Stable | Ribbit protocol client with signature verification |
| `tact-client` | 0.4.3 | โœ… Stable | TACT HTTP client with retry logic and batching |
| `tact-parser` | 0.4.3 | โœ… Stable | TACT file format parser (encoding, install, etc.) |
| `ngdp-cdn` | 0.4.3 | โœ… Stable | CDN client with fallback hosts and connection pooling |
| `ngdp-cache` | 0.4.3 | โœ… Stable | Comprehensive caching layer with LRU eviction |
| `blte` | 0.4.3 | โœ… Stable | BLTE decompression with memory pooling |
| `ngdp-crypto` | 0.4.3 | โœ… Stable | Modern encryption with Salsa20 and key service |
| `ngdp-client` | 0.4.3 | โœ… Stable | CLI tool for NGDP operations |
| `casc-storage` | 0.4.3 | ๐Ÿšง Beta | CASC storage implementation (in development) |
| `ngdp-patch` | 0.4.3 | ๐Ÿšง Beta | Patch file support (in development) |

### Implementation Progress

- โœ… **Version Discovery**: HTTP-first approach with HTTPS endpoints, Ribbit fallback
for compatibility
- โœ… **TACT Protocol**: HTTP/HTTPS clients for version and CDN queries with retry
logic
- โœ… **BPSV Format**: Parser and builder with zero-copy optimizations
- โœ… **TACT Parsers**: Support for encoding, install, download, size, build config,
TVFS
- โœ… **BLTE Decompression**: Compression modes (ARC4/Frame deprecated in v0.4.0)
- โœ… **Encryption**: Salsa20 cipher with key management (ARC4 deprecated)
- โœ… **CDN Operations**: Parallel downloads, streaming, retry logic, rate limiting
- โœ… **Caching**: HTTP-first caching with fallbacks and TTL support
- โœ… **Install Command**: Client installation with .build.info generation for restoration
- โœ… **Build Config**: Uncompressed handling and download order (encoding before manifests)
- ๐Ÿšง **CASC Storage**: Local storage implementation (in development)
- ๐Ÿ”„ **TVFS**: Parser implemented, needs real-world data testing

## ๐Ÿš€ Quick Start

### Library Usage

Add to your `Cargo.toml`:

```toml
[dependencies]
ribbit-client = "0.4.3"
ngdp-bpsv = "0.4.3"
tact-parser = "0.4.3"
blte = "0.4.3"
ngdp-crypto = "0.4.3"
```

Basic example (HTTP-first approach):

```rust
use ngdp_cache::hybrid_version_client::HybridVersionClient;
use ribbit_client::Region;

#[tokio::main]
async fn main() -> Result<(), Box> {
// Create a hybrid client (HTTP primary, Ribbit fallback)
let client = HybridVersionClient::new(Region::US).await?;

// Request WoW versions with HTTPS endpoints
let versions = client.get_product_versions("wow").await?;

// Print version information
for entry in &versions.entries {
println!(
"{}: {} (build {})",
entry.region, entry.versions_name, entry.build_id
);
}

Ok(())
}
```

## ๐Ÿงช Testing with Real WoW Data

Many tests and examples can work with real WoW installation data for comprehensive
testing. This is optional - all tests will skip gracefully if no data is available.

### Setup Environment Variables

Set environment variables pointing to your WoW installation Data directories:

```bash
# Classic Era (1.15.x)
export WOW_CLASSIC_ERA_DATA="$HOME/Downloads/wow/1.15.2.55140.windows-win64/Data"

# Classic with expansions
export WOW_CLASSIC_DATA="$HOME/Downloads/wow/classic/Data"

# Retail (current)
export WOW_RETAIL_DATA="$HOME/Downloads/wow/retail/Data"
```

### Valid Data Directory Structure

The Data directory should contain CASC structure:

```
Data/
โ”œโ”€โ”€ data/ # CASC archive files (required)
โ”œโ”€โ”€ indices/ # CASC index files
โ”œโ”€โ”€ config/ # CASC configuration files
โ””โ”€โ”€ ...
```

### Running Tests with Real Data

```bash
# Run all tests (will skip those requiring WoW data if not available)
cargo test

# Run specific integration tests
cargo test -p casc-storage --test real_data_integration

# Run examples that use real data
cargo run -p casc-storage --example list_casc_files
```

Tests and examples will automatically:

- Use environment variables when available
- Fall back to common installation paths
- Skip gracefully with helpful setup instructions if no data found
- Work in CI environments without WoW data

## ๐Ÿ“ฆ Installation

### CLI Tool

#### Install with Cargo

```bash
cargo install ngdp-client
```

#### Install with Script (Unix/Linux/macOS)

```bash
curl -fsSL https://raw.githubusercontent.com/wowemulation-dev/cascette-rs/main/install.sh | bash
```

#### Install with Script (Windows PowerShell)

```powershell
irm https://raw.githubusercontent.com/wowemulation-dev/cascette-rs/main/install.ps1 | iex
```

### Library Usage

```bash
cargo add ribbit-client ngdp-bpsv tact-client tact-parser ngdp-cdn ngdp-cache blte ngdp-crypto
```

### From source

```bash
git clone https://github.com/wowemulation-dev/cascette-rs
cd cascette-rs
cargo build --release
# CLI binary will be at target/release/ngdp
```

## ๐Ÿ–ฅ๏ธ CLI Usage

The `ngdp-client` provides a comprehensive command-line interface for NGDP operations:

### Install Game Clients

```bash
# Install WoW Classic Era (minimal installation)
ngdp install game wow_classic_era --install-type minimal --path ./wow-client

# Full installation with verification
ngdp install game wow --install-type full --verify --path ./wow-retail

# Dry-run to see what would be installed
ngdp install game wow_classic --install-type minimal --dry-run --path ./test
```

### Version and CDN Information

```bash
# Query product versions (uses HTTPS endpoints)
ngdp query versions wow

# Get CDN configuration
ngdp query cdns wow

# Inspect build configurations
ngdp inspect build-config
```

### BPSV and File Analysis

```bash
# Parse and display BPSV files
ngdp inspect bpsv ./file.bpsv

# Analyze BLTE files
ngdp inspect blte ./encrypted.blte
```

All commands support multiple output formats (`--format json|text|bpsv`) and the install command automatically creates `.build.info` files for client restoration.

## ๐Ÿ“š Documentation

- [BPSV Format Specification](docs/bpsv-format.md)
- [BPSV Examples](ngdp-bpsv/examples)
- [Ribbit Protocol](docs/ribbit-protocol.md)
- [Ribbit Examples](ribbit-client/examples)
- [TACT Protocol](docs/tact-protocol.md)

## ๐Ÿ“š Online References

- [TACT Reference](https://wowdev.wiki/TACT)
- [Ribbit Reference](https://wowdev.wiki/Ribbit)
- [CASC Reference](https://wowdev.wiki/CASC)

## ๐Ÿ”ง Features

### Complete

- **BPSV Parser/Writer** (`ngdp-bpsv`)
- โœ… BPSV format support with zero-copy parsing
- โœ… Type-safe field definitions (STRING, HEX, DEC)
- โœ… Schema validation and sequence number handling
- โœ… Builder pattern for document creation
- โœ… Round-trip compatibility

- **Ribbit Protocol Client** (`ribbit-client`)
- โœ… Blizzard regions (US, EU, CN, KR, TW, SG)
- โœ… V1 (MIME) and V2 (raw) protocol support
- โœ… Typed API for all endpoints
- โœ… PKCS#7/CMS signature verification
- โœ… Certificate and OCSP support
- โœ… Retry with exponential backoff
- โœ… DNS caching for performance

- **TACT HTTP Client** (`tact-client`)
- โœ… Version and CDN configuration queries
- โœ… Support for V1 (port 1119) and V2 (HTTPS) protocols
- โœ… Typed response parsing
- โœ… Retry handling
- โœ… Blizzard regions supported

- **CDN Content Delivery** (`ngdp-cdn`)
- โœ… Parallel downloads with progress tracking
- โœ… Streaming operations for large files
- โœ… Retry with rate limit handling
- โœ… Content verification
- โœ… Configurable connection pooling
- โœ… Fallback to backup CDN servers
- โœ… Support for community mirrors (arctium.tools, reliquaryhq.com)

- **Caching Layer** (`ngdp-cache`)
- โœ… Transparent caching for all NGDP operations
- โœ… TTL-based expiration policies
- โœ… Streaming I/O for memory efficiency
- โœ… CDN-compatible directory structure
- โœ… Batch operations for performance

- **TACT File Parsers** (`tact-parser`)
- โœ… Encoding files (CKey โ†” EKey mapping)
- โœ… Install manifests with tag-based filtering
- โœ… Download manifests with priority sorting
- โœ… Size files for installation calculations
- โœ… Build configurations (key-value format)
- โœ… TVFS (TACT Virtual File System)
- โœ… 40-bit integer and varint support

- **BLTE Decompression** (`blte`)
- โœ… Compression modes (None, ZLib, LZ4, Frame, Encrypted)
- โœ… Multi-chunk file support
- โœ… Checksum verification
- โœ… Integration with ngdp-crypto for encrypted blocks
- โœ… Memory processing

- **Encryption Support** (`ngdp-crypto`)
- โœ… Salsa20 stream cipher (WoW encryption)
- โœ… ARC4/RC4 cipher (legacy content)
- โœ… Key management and loading
- โœ… Multiple key file formats (CSV, TXT, TSV)
- โœ… TACTKeys repository integration

- **CLI Tool** (`ngdp-client`)
- โœ… Product queries and version information
- โœ… Certificate operations
- โœ… BPSV inspection and build config analysis
- โœ… Encryption key management commands
- โœ… Inspect commands with BLTE support
- โœ… Multiple output formats (text, JSON, BPSV)
- โœ… Terminal formatting

## ๐Ÿค Contributing

Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

Special thanks to the WoW emulation community and the documentation efforts at
[wowdev.wiki](https://wowdev.wiki).

## ๐Ÿ“„ License

This project is dual-licensed under either:

- MIT license ([LICENSE-MIT](LICENSE-MIT) or )
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or )

at your option.

### Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in the work by you, as defined in the Apache-2.0 license, shall be
dual licensed as above, without any additional terms or conditions.

---

**Note**: This project is not affiliated with or endorsed by Blizzard Entertainment.
It is an independent implementation based on reverse engineering efforts by the
community for educational and preservation purposes.