Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/pietroborrello/CustomProcessingUnit

The first analysis framework for CPU microcode
https://github.com/pietroborrello/CustomProcessingUnit

Last synced: 2 months ago
JSON representation

The first analysis framework for CPU microcode

Awesome Lists containing this project

README 

        
# Custom Processing Unit



[![DOI](https://zenodo.org/badge/522085221.svg)](https://zenodo.org/badge/latestdoi/522085221)






Custom Processing Unit is the first dynamic analysis framework able to hook, patch and trace CPU microcode at the software level.



It works by leveraging [undocumented instructions](https://github.com/chip-red-pill/udbgInstr) in Intel CPUs that allow access to the CRBUS.

Using our [microcode decompiler](https://github.com/pietroborrello/ghidra-atom-microcode) we reverse engineered how the CPU uses the CRBUS and by replicating the interactions we have full control of the CPU.



Find the static analysis framework as subtree in [this](./ghidra-processor-module) folder, or at https://github.com/pietroborrello/ghidra-atom-microcode.



Check out our slides describing this work [here](./slides.pdf).



**Note**: Custom Processing Unit requires a Red-Unlocked CPU: currently, only [Goldmont CPUs](https://en.wikipedia.org/wiki/Goldmont) (GLM) have a [public Red Unlock](https://github.com/ptresearch/IntelTXE-PoC). We tested Gigabyte GB-BPCE-3350C with CPU stepping 0x9 and 0xa (cpuid 0x000506C9 and 0x000506CA).



Custom Processing Unit is made up of a UEFI application and a few libraries. The UEFI application interacts with the GLM CPU, while the libraries provide different helpers to compile microcode into the UEFI application and analyze its output.



## Prerequisites



1. Follow the steps to red unlock your Goldmont CPU from https://github.com/ptresearch/IntelTXE-PoC.

2. Create a bootable USB key with an EFI shell

3. Install [gnu-efi](https://wiki.osdev.org/GNU-EFI) on your main host



## Setup



```

GNU_EFI_DIR= make

```



This will build the source microcode files and the UEFI application into `cpu.efi`.

Copy `cpu.efi` into the `\EFI\` folder of the USB key, plug it in the GLM and boot into the EFI shell.



Run `map -r` in the efi shell to identify the USB key device and `:` to mount it.



## Run Custom Processing Unit



Run `./cpu.efi` to print the help:



```

Usage:

  patch:         p

  patch & exec:  x

  perf:          f

  zero out m&p:  z

  hook:          h  [m&p idx] [uop addr] [patch addr]

  template:      m

  dump imms:     di

  dump rom:      dr

  dump msrs:     dm

  dump SMM:      ds [address] [size]

  cpuid:         c  [rax] [rcx]

  rdmsr:         rm [msr]

  wrmsr:         wm [msr]

  read:          r  [cmd] [addr]

  write:         w  [cmd] [addr] [value]

  invoke:        i  [addr]

  update ucode:  u  [size]

  ldat read:     lr [port] [array] [bank] [idx] [addr] [optional size]

  ldat write:    lw [port] [array] [bank] [idx] [addr] [value]

```



### Simple instructions



`cpu` provides helpers to run simple instructions from the command line:

* cpuid

* rdmsr

* wrmsr



### Complex actions

`cpu` provides interfaces to complex CPU routines that are interesting to execute to study cpu behavior:

* `u`: update the CPU ucode with the provided (signed) patch

* `f`: collect performance counters while running microcode



### Raw udbgrd and udbwr



`cpu` provides raw interfaces to the undocumented instructions `udbrd` and `udbgwr`.

The most interesting commands they provide are:

* 0x0:  access CRBUS

* 0x10: access UROM

* 0x40: access stgbuf

* 0xd8: invoke ucode routine from address



### LDAT access

`cpu` exposes LDAT access routines to read and write. Specify the parameters  `[port] [array] [bank] [idx] [addr]` to read or write there.

Interesting ports are:

* 0x6a0: microcode sequencer, which has access to the internal the ucode ROM and RAM

* 0x120: load/store buffers

* 0x3c0: instruction cache

* 0x630: ITLB



Please notice that accessing some of these internal components may cause the CPU to freeze.



### Patch microcode



`cpu` provides functionalities to install patches in the microcode. 

1. Write your microcode patch in `bios/ucode_patches/ucode_patch.u` (look at the other patches for examples)

2. Build the UEFI application

3. Execute `cpu.efi p` to install the patch at the address provided in `.org`.



Notice that in the microcode, only the addresses between 0x7c00 and 0x7e00 are writable and meaningful to patch.



Running `cpu.efi x`, it will also execute the microcode patched and print the `rax, rbx, rcx, rdx` registers as result.



### Match & Patch



To automatically execute microcode at certain CPU events or microcode points, `cpu` leverages the Match and Patch. 

It defines a microcode address to hook and the microcode address to jump to when the hook is triggered.



* `z`: resets all the match & patch.

* `h`: installs an hook, given an index (0-0x20), an address to hook (0-0x7c00) and a target address to execute (0x7c00-0x7e00).



### Tracing microcode



By installing multiple hooks and continuously executing an instruction, `cpu` is able to trace the microoperations performed by such an instruction, and dump them. To trace:



1. Write the instruction to be traced after the `// [TRACED INSTRUCTION HERE]` in `get_trace_clock_at()`.

2. Build the UEFI application.

3. Trace with: `cpu.efi m`.

It will create a `trace.txt` file that contains all the addresses that have been hit.

4. Execute `uasm-lib/uasm.py -t trace.txt > parsed_trace.txt`.

It will generate a full trace of the microcode executed during the instruction.



Notice that `uasm.py` will leverage the `ms_arrayX.txt` files in its folder to generate a disassembly of the microinstructions executed. These are for GLM with stepping 0x9 (cpuid 0x000506C9). Please generate the proper arrays in case you have a different stepping.

You can use the LDAT dump functionalities for this purpose.



### Secret memory dumpers



The CPU has different inaccessible buffers from the architecture, for which we provide routines to dump:

* `smm`: SMROM (or any other address while disabling SMM protection)

* `rom`: internal ROM

* `imms`: CPU hardcoded immediates

* `msrs`: internal MSRs configurations



### Writing microcode patches



We provide an assembler that generates header files to be compiled into the `cpu.efi` UEFI application.

Look into the provided patches in `bios/ucode_patches` for the syntax.

It supports simple operations and labels.

Assemble a microcode patch with `uasm.py -i ucode_patch.u -o ucode_patch.h`.

`cpu.efi` will be compiled and automatically include the microcode patch that you want to apply.



#### Example



file: `code_patch.u`

```

.org 0x7c00



rax:= ZEROEXT_DSZ32(0x00001337)

rbx:= ZEROEXT_DSZ32(0x00001337)

rcx:= ZEROEXT_DSZ32(0x00001337)

rdx:= ZEROEXT_DSZ32(0x00001337)

```



recompile, then run in the GLM:

```

cpu.efi z # zero out match & patch

cpu.efi p # apply the patch

cpu.efi h 0 0x0428 0x7c00 # rdrand entry point

```

now every time `rdrand` is executed, it will return `0x1337` in the registers.



### Cite Us



Our work has been published in a [paper](https://pietroborrello.com/publication/woot23/woot23.pdf) at [WOOT](https://wootconference.org/) 2023: 

```

@inproceedings{Borrello2023CustomProcessingUnit,

    title = {{CustomProcessingUnit}: Reverse Engineering and Customization of Intel Microcode},

    author = {Borrello, Pietro and Easdon, Catherine and Schwarzl, Martin and Czerny, Roland and Schwarz, Michael},

    booktitle = {IEEE Workshop on Offensive Technologies (WOOT 23)},

    year = {2023},

}

```



### Experiments



The experiments described in the paper can be run with:

```

cpu.efi e [exp_idx]

```



with `[exp_idx]`:



0. Fast ucode breakpoints

1. Constant-Time ucode division

2. x86 PAC

3. Attack x86 PAC with PACMAN

4. Conditional Hardware Breakpoints