{"id":18411389,"url":"https://github.com/alfg/quick-dive-into-mp4","last_synced_at":"2025-04-07T11:31:41.046Z","repository":{"id":142573758,"uuid":"247595156","full_name":"alfg/quick-dive-into-mp4","owner":"alfg","description":"🎥 A technical intro the MP4 container byte structure.","archived":false,"fork":false,"pushed_at":"2021-05-01T04:07:01.000Z","size":2967,"stargazers_count":49,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-22T17:01:47.017Z","etag":null,"topics":["guide","isobmff","mp4","parser","quicktime","video"],"latest_commit_sha":null,"homepage":"https://dev.to/alfg/a-quick-dive-into-mp4-57fo","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alfg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-16T02:41:54.000Z","updated_at":"2025-02-21T20:28:10.000Z","dependencies_parsed_at":null,"dependency_job_id":"5f7133e3-fcc4-4489-b2bd-7e6e920971dc","html_url":"https://github.com/alfg/quick-dive-into-mp4","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alfg%2Fquick-dive-into-mp4","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alfg%2Fquick-dive-into-mp4/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alfg%2Fquick-dive-into-mp4/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alfg%2Fquick-dive-into-mp4/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alfg","download_url":"https://codeload.github.com/alfg/quick-dive-into-mp4/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247644105,"owners_count":20972223,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["guide","isobmff","mp4","parser","quicktime","video"],"created_at":"2024-11-06T03:36:45.991Z","updated_at":"2025-04-07T11:31:40.210Z","avatar_url":"https://github.com/alfg.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\u003cimg src=\"header.png\" /\u003e\u003c/p\u003e\n\n# A Quick Dive Into MP4\n\nWhat is an MP4? We all know it as a file format for playing video with sound. It's used for streaming video by Netflix, YouTube, Instagram, and also for capturing video on your iPhone, but how does it work? How is it used? What is the byte structure? What is a container?\n\nThis guide is an introduction and a quick dive into the MP4 file format, also known as the ISO Base Media File Format ([ISO-BMFF MPEG-4 Part 14](https://en.wikipedia.org/wiki/MPEG-4_Part_14)). Fancy name, I know.\n\nI won't go into the playback details in this guide, but more of the MP4 byte format commonly known as the MP4 Box Structure.\n\n## Introduction\nThe MPEG-4 Part 14 (MP4) is one of the most common [container formats](https://en.wikipedia.org/wiki/Comparison_of_video_container_formats) for video and has an extension of `.mp4`. You may already know of other container formats, such as `wav`, `mov`, `mp3` or more recently `webm`. A container just \"contains\" the video or audio track, or both. It can also support embedded subtitle tracks too.\n\nMP4 is an extension of the ISO Base Media File Format ([ISOBMFF, MPEG-4 Part 12)](https://en.wikipedia.org/wiki/ISO/IEC_base_media_file_format), which is a format designed to contain timed media information.\n\nThe ISO-BMFF format is directly based on [QuickTime](https://en.wikipedia.org/wiki/QuickTime_File_Format), therefore the MP4 is essentially identical to the QuickTime file format.\n\n```\n     MPEG-4 Part 14 - MP4 File Format\n ┌──────────────────────────────────────┐\n ├────────────────────┐                 │\n │  ISO Base Media    │                 │\n │    File Format     │  MP4 Extension  │\n │  (MPEG-4 Part 12)  │                 │\n └────────────────────┴─────────────────┘\n```\n\nIn order to fully understand the MP4 structure, you'll need to obtain a copy of the [ISO](https://en.wikipedia.org/wiki/International_Organization_for_Standardization) documents:\n* 14496-12 – MPEG-4 Part 12\n* 14496-14 - MPEG-4 Part 14\n\nA Google search should result in a few resources to get a copy of the PDF.\n\n## What's in a container?\nSince MP4 is a [container](https://en.wikipedia.org/wiki/Comparison_of_video_container_formats) format, it doesn't actually handle the decoding of the video and audio streams, it just contains them as tracks along with their metadata.\n\nA container can store some of the following information:\n* General metadata such as file type and compatibility\n* Video, audio and subtitle tracks and codec details\n* Metadata: duration, timescale, bitrate, width/height, etc\n* Progressive and fragmented metadata details\n* A series of video frames or audio samples known as \"Sample Data\"\n\nThis is all the information a player needs to decode and play the content.\n\nAt a high-level, this is what an MP4 structure typically looks like:\n\n```\nvideo.mp4\n├───general file metadata\n├───movie data\n├───tracks\n│   ├───video\n│   │   ├───video metadata\n│   │   └───video sample data\n│   └───audio\n│       ├───audio metadata\n│       └───audio sample data\n└───more metadata\n```\n\n## Movie Boxes\nThe MP4 byte structure is composed of a series of boxes, also known as \"atoms\", according to the QuickTime specification. Each box describes and contains data to build the MP4 container format.\n\nBoxes typically have a four letter name, also known as a [FourCC](https://en.wikipedia.org/wiki/FourCC). This is the shortened version of the full box name, enough to fit into 4 bytes. This is important for when you are reading and writing boxes into or from the byte format.\n\nBefore we jump into the byte structure details, here is a more technical view of the MP4 box tree, compared to the high-level view above:\n\n```\nvideo.mp4\n├───ftyp -------------------\u003e FileType Box\n├───mdat -------------------\u003e Movie Data Box\n├───moov -------------------\u003e Movie Boxes\n│   ├───trak ---------------\u003e Track Box\n│   │   ├─── tkhd ----------\u003e Track Header\n│   │   └─── mdia ----------\u003e Media Box\n│   │        └─── ...\n│   └───trak\n│   │   ├─── tkhd ----------\u003e Track Header\n│   │   └─── mdia ----------\u003e Media Box\n│   │        └─── ...\n└───udta -------------------\u003e Userdata Box\n```\n\nThis is just a simplified view. However, there are many more boxes defined in the MP4 specification.\n\n## What's in the box?\nAn MP4 \"Box\" contains just enough information to read and parse the box name, size and data.\n\nEach of these boxes have a different purpose, containing a bit of information and details on a specific piece of data. Some boxes describe the file type, and others can describe codec detail, picture resolution, frame rate, duration, sample sizes and more. There's also boxes containing the encoded video and audio data too.\n\nA box typically contains the following base information:\n* Size of the box (in bytes)\n* Box Name (FourCC)\n* Box Data\n\n```\n┌─────────────────────┐\n|      Box Header     |\n| Size (4) | Type (4) | Box Header = 8 Bytes\n| --------------------|\n|     Box Data (N)    | Box Data = N Bytes\n└─────────────────────┘\n           └─────────── Box Size = 8 + N bytes\n```\n\nThis is just enough information we need to know how to parse a box, along with the MP4 specification document to understand the box fields.\n\n## Parse a Box\nSo let's parse our first box!\n\nAs mentioned above, the first 8 bytes of each box is known as the \"Box Header\", where the first 4 bytes are the size of the box, and the next 4 bytes are the box name. These are the two values you need to know to iterate and parse each box, byte by byte.\n\nHere's a box header struct for example:\n```go\ntype Box struct {\n  Size    int32\n  Name    string\n}\n```\n\nReading the box data from each atom requires the box size, name and byte structure of each box you are parsing. You can refer to the MPEG-4 Part 14 specification for the byte structure of each known box, or just refer to some existing MP4 parsing open-source code.\n\nAccording to the specification, the `ftyp` box has the following structure:\n```c\naligned(8) class FileTypeBox\n  extends Box(‘ftyp’) {\n  unsigned int(32) major_brand;\n  unsigned int(32) minor_version;\n  unsigned int(32) compatible_brands[]; // to end of the box\n}\n```\n\n* `major_brand` – is a brand identifier\n* `minor_version` – is an informative integer for the minor version of the major brand\n* `compatible_brands` – is a list, to the end of the box, of brands\n\nFor example, reading the FileTypeBox (`ftyp`) would look something like the following (in Golang):\n\n```go\ntype FtypBox struct {\n  *Box\n  MajorBrand       string\n  MinorVersion     uint32\n  CompatibleBrands []string\n}\n\nfunc (b *FtypBox) parse() {\n  data := b.ReadBoxData() // Read box header.\n  b.MajorBrand = string(data[0:4])\n  b.MinorVersion = binary.BigEndian.Uint32(data[4:8])\n  if len(data) \u003e 8 {\n    for i := 8; i \u003c len(data); i += 4 {\n      b.CompatibleBrands = append(b.CompatibleBrands, string(data[i:i+4]))\n    }\n  }\n}\n```\n\nGoing over the above:\n* Reading the first 4 bytes of the box header as an unsigned 32 bit integer (big endian) gives us the box size: `32 bytes`.\n* The next 4 bytes gives us: `0x66747970` in hexidecimal, or `ftyp` as a string.\n* Next 4 bytes gives us the Major Brand: `0x69736F6D` in hexidecimal, or `isom` as a string.\n* Next 4 bytes gives us the Minor Version: `512`\n* The next 16 bytes, read as `uint32be` (into a string) at a time gives us an array of compatible brands: `isom`, `iso2`, `avc1`,\n and `mp41`.\n* We have read a total of `32 bytes` as defined in the box header. \n\nSee a minimal example:\nhttps://gist.github.com/alfg/7375aee32fda490de4bf62fbced49d2e#file-mp4_example-go\n\n```\n$ go run mp4.go tears-of-steel.mp4\nftyp.name:  ftyp\nftyp.major_brand:  isom\nftyp.minor_version:  512\nftyp.compatible_brands:  [isom iso2 avc1 mp41]\n```\n\nIf you were to open the mp4 into a hex editor, it would look something like this for the `ftyp` box:\n\n```\n0x00 00 00 00 20 66 74 79 70 | 69 73 6F 6D 00 00 02 00 ... ftypisom....\n0x10 69 73 6F 6D 69 73 6F 32 | 61 76 63 31 6D 70 34 31 isomiso2avc1mp41\n```\n\n## Next Box!\n\nNow that we've read the `ftyp` box, we can move on to the next box header, which happens to be the `moov` box:\n\n```go\ntype MoovBox struct {\n  *Box\n  Mvhd  *MvhdBox\n}\n\nfunc (b *MoovBox) parse() {\n  boxes := readBoxes(b.Reader, b.Start+BoxHeaderSize, b.Size-BoxHeaderSize)\n\n  for _, box := range boxes {\n    switch box.Name {\n    case \"mvhd\":\n    b.Mvhd = \u0026MvhdBox{Box: box}\n    b.Mvhd.parse()\n  }\n}\n```\n\nThe `moov` box contains a nested `mvhd` box, so we also need to define `mvhd` too:\n\n```go\ntype MvhdBox struct {\n  *Box\n  Flags            uint32\n  Version          uint8\n  CreationTime     uint32\n  ModificationTime uint32\n  Timescale        uint32\n  Duration         uint32\n  Rate             Fixed32\n  Volume           Fixed16\n}\n\nfunc (b *MvhdBox) parse() {\n  data := b.ReadBoxData()\n  b.Version = data[0]\n  b.Timescale = binary.BigEndian.Uint32(data[12:16])\n  b.Duration = binary.BigEndian.Uint32(data[16:20])\n  b.Rate = fixed32(data[20:24])\n  b.Volume = fixed16(data[24:26])\n}\n```\n\nSee the example with `moov` and `mvhd` box included:\nhttps://gist.github.com/alfg/7375aee32fda490de4bf62fbced49d2e#file-mp4_example_2-go\n\n```\n$ go run mp4.go tears-of-steel.mp4\nftyp.name:  ftyp\nftyp.major_brand:  isom\nftyp.minor_version:  512\nftyp.compatible_brands:  [isom iso2 avc1 mp41]\nmoov.name:  moov 3170\nmoov.mvhd.name:  mvhd\nmoov.mvhd.version:  0\nmoov.mvhd.volume:  1\n```\n\n## EOF\n\nNow that we've parsed 3 boxes, hopefully you have an idea on how to implement more. The process is iterative when using a reader:\n* Read the box header, containing the box size and name.\n* Refer to the specification to read and/or skip fields.\n* Skip any remaining bytes left in the box size.\n* Read the next box (or skip).\n\n##### Some things to keep in mind:\n* Some boxes have multiple versions, and therefore can differ in the struct and overall size of the box.\n* You can skip properties, but the reader must know how many bytes to skip.\n* There are various MP4 specifications beyond `MPEG-4 Part 14` as more boxes are being added throughout the years.\n* Fragmented MP4 files (fMP4) are segmented as a series of `moof` and `mdat` boxes. This is more common and optimial for streaming delivery. I'll cover this in a future post.\n\n## Thanks for reading!\nFor a more complete example of reading MP4 boxes in Go, check out:\nhttps://github.com/alfg/mp4\n\nI also have a more advanced MP4 reader and writer in Rust:\nhttps://github.com/alfg/mp4rs\n\nI highly suggest some of the following tools for inspecting MP4 files:\n* [MP4Box](https://github.com/gpac/gpac/wiki/MP4Box)\n* [mp4box.js](https://gpac.github.io/mp4box.js)\n* [MediaInfo](https://github.com/MediaArea/MediaInfo)\n* [FFProbe](https://ffmpeg.org/ffprobe.html)\n\nFind me on GitHub at: https://github.com/alfg\n\nHappy Hacking! 🎥\n\n# References and Resources\n* https://developer.apple.com/library/archive/documentation/QuickTime/QTFF\n* https://en.wikipedia.org/wiki/MPEG-4_Part_14\n* https://en.wikipedia.org/wiki/ISO/IEC_base_media_file_format\n* https://en.wikipedia.org/wiki/Comparison_of_video_container_formats\n* https://en.wikipedia.org/wiki/FourCC\n* https://gist.github.com/alfg/7375aee32fda490de4bf62fbced49d2e\n* https://github.com/alfg\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falfg%2Fquick-dive-into-mp4","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falfg%2Fquick-dive-into-mp4","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falfg%2Fquick-dive-into-mp4/lists"}