Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/mitsuhiko/frostbite2-stuff

Python libs for accessing Frostbite stuff
https://github.com/mitsuhiko/frostbite2-stuff

Last synced: 3 months ago
JSON representation

Python libs for accessing Frostbite stuff

Awesome Lists containing this project

README 

        



    \ | /

   - < > -    Frostbite2 Stuff

     | |



  This repository contains libraries that access various

  formats that Battlefield 3 uses.  Currently it reads most

  of the .sb/.toc/.m/.cas and .cat files as well as the

  usersettings of the game.



  // Superbundle Files



    - Overall Format



      I don't know how the engine calls these files internally

      but they are by far the most common file format that

      Frostbite 2 has.  They exist in two formats: unencrypted

      and signed and XOR encrypted.  To keep them apart from

      each other the engine seems to use the 4 byte header

      00D1CE00 at the beginning of files.



      If a file starts with 00D1CE00 a hash will be located at

      0x08 in the file and start with the ascii letter "x",

      then 256 ascii letters which are the hash and then again

      the letter "x".  What this hash is used for I do not know.



      The lookup table for the XOR encryption then is located

      at 0x0128 and is 257 characters long.  It is followed by

      XOR encrypted data.  For each character read the following

      decryption has to be applied:



        for (int i = 0; i < data_size; i++)

          data[i] = data[i] ^ magic[i % 257] ^ 0x7b;



      If the file does not start with that header the payload

      starts at byte zero directly.



    - File Format Description



      The file can be described as a form of binary JSON.  It

      generally is a nested structure of dictionarys or lists

      and some scalar types as well as strings and blobs.



      Each object is prefixed with a typecode and some flags:



        uint8 byte = read_single_byte();

        int flags = byte >> 5;

        int typecode = byte & 0x1f;



      I don't know what the flags are doing but for reading the

      files it seems that you can ignore them.  The following

      typecodes exist:



        0           None / nil

        1           List

        2           Dict

        5           Unknown (8 bytes)

        6           Bool

        7           Binary String

        8           int32

        9           int64

        15          UUID

        16          SHA1

        19          Variable Length Quantity (varint)



      Strings are prefixed with the number of bytes as varint.

      Lists and dicts are prefixed with the number of elements

      they contain of but they are also delimited by `None` at

      the end.



      Dicts are interesting in that they move the typecode of

      the value of the dict before the key which means that

      you need to read "typecode key value" where value is no

      longer prefixed with a typecode.



  // CAS / CAT files



    - CAS File Format



      CAS files are the data source for everything the engine

      reads.  All the assets are stored in CAS files.  The CAS

      files by themselves can be dumped to the filesystem

      without the help of the CAT file but finding things in

      that file would be O(n) as such the CAT file exists which

      is a catalog of all the CAS files.



      struct entry {

          char header[4];

          char sha1[20];

          int32 data_length;

          char padding[4];

      };



      The header for all entries in the CAS file are always

      FACE0FF0.



    - CAT File Format



      CAS Catalogs are very similar.  They start with the

      header "NyanNyanNyanNyan" and after that they list all

      the contents of all CAS files with the additional

      information of position, size and which CAS file they

      are from:



      struct entry {

          char sha1[20];

          int32 offset;

          int32 size;

          int32 cas_num;

      }



      The cas num is the name of the cas file.  "2" would

      indicate "cas_02.cas" etc.



  // How Does The Engine Find Stuff?



    - File Identifiers



      The sha1 hashes in the CAS/CAT are the SHA1 hash of the

      contents of the file.  This is also how the engine looks

      them up which indicates that it does not do a md5 hash

      of the filename.  The .toc/.sb files in combination have

      UUIDs for bundles which then have a bunch of SHA1 hashes

      which point to files in the CAS files.



      Additionally there is a bom.fb2 file which seems to be

      unused by the game.  Once decrypted with the algorithm

      from above it exposes a zip file made up of .m files

      which point to various parts in the archives.  Yet the

      patch does not have such a bom.fb2 file.