Visual diagram of ModR/M + REX decoding

1. Full Instruction Layout (x86-64)

A typical instruction looks like this:

[ REX ] [ OPCODE ] [ MODR/M ] [ SIB ] [ DISP ] [ IMM ]

Not all fields are always present.

2. Focus: REX + ModR/M Interaction

Step-by-step decoding pipeline

            +----------------+
            |   REX Prefix   |
            | 0100WRXB       |
            +----------------+
                     |
                     v
     +--------------------------------+
     | Extends ModR/M + SIB fields    |
     | Adds 4th bit to registers      |
     +--------------------------------+
                     |
                     v
        +--------------------------+
        |     ModR/M Byte         |
        |  MOD | REG | R/M        |
        |  2b  | 3b  | 3b         |
        +--------------------------+
                     |
                     v
     +--------------------------------+
     | Final register =              |
     | (REX bit << 3) + 3-bit field  |
     +--------------------------------+

3. ModR/M Byte Structure (Core)

Bit layout:
  7 6 | 5 4 3 | 2 1 0
  MOD |  REG  |  R/M

Meaning:

4. Where REX Fits In

REX does NOT replace ModR/M.

It extends its fields:

              REX byte
        +-------------------+
        | W R X B           |
        +-------------------+
          | | | |
          | | | +--> extends R/M or BASE
          | | +----> extends SIB INDEX
          | +------> extends REG field
          +--------> 64-bit operand size

5. Visual Mapping: REG Field Expansion

Without REX (3-bit only)

REG field:
000 → RAX
001 → RCX
010 → RDX
...
111 → RDI

Diagram:

REG (3-bit)
   ↓
[ 000 ] → RAX
[ 001 ] → RCX
[ 010 ] → RDX
...
[ 111 ] → RDI

With REX (adds 1 bit)

REX.R + REG becomes:
REX.R | REG
   1   | 010  → 1010 (R10)
   1   | 011  → 1011 (R11)

Diagram:

        REX.R bit
             ↓
        +----------+
        |    1     |
        +----------+
             |
             v
REG bits → [010]
             |
             v
      4-bit register:
           1010 → R10

6. Full Decoding Example (Visual Walkthrough)

Instruction:

mov r10, rax

Machine code:

4C 89 D0

STEP 1 — REX Byte

4C = 01001100

Split:

0100 W R X B
     1 1 0 0

Diagram:

REX
+-------------------+
| W = 1 (64-bit)    |
| R = 1 (extend REG)|
| X = 0             |
| B = 0             |
+-------------------+

STEP 2 — ModR/M Byte

D0 = 11010000

Split:

11 010 000
MOD REG R/M

Diagram:

ModR/M
+--------------------+
| MOD = 11           |  → register mode
| REG = 010          |
| R/M = 000          |
+--------------------+

STEP 3 — Apply REX Extensions

REG = REX.R : REG
     = 1 : 010
     = 1010 → R10
R/M = REX.B : R/M
     = 0 : 000
     = 0000 → RAX

FINAL RESULT

mov r10, rax

7. Complete Visual Pipeline

        MACHINE CODE
        -------------
        4C 89 D0
             |
             v
     +----------------+
     | REX = 4C      |
     +----------------+
             |
             v
     W=1  R=1  X=0  B=0
             |
             v
     +------------------------+
     | ModR/M = D0           |
     | REG=010  R/M=000      |
     +------------------------+
             |
             v
     +------------------------+
     | Apply REX extensions   |
     | REG → 1010 = R10       |
     | R/M → 0000 = RAX       |
     +------------------------+
             |
             v
     FINAL:
     mov r10, rax

8. SIB Extension (Visual Add-on)

When memory indexing is used:

[ base + index * scale ]

SIB byte:

SS | INDEX | BASE

REX.X extends INDEX:

REX.X + INDEX → 4-bit register

Diagram:

          SIB byte
   +--------------------+
   | SS | INDEX | BASE |
   +--------------------+
            |
            v
     REX.X extends INDEX
            |
            v
     4-bit register space

9. Key Insight Diagram (Mental Model)

Original x86:
   REG = 3 bits
   → 8 registers
x86-64:
   REX adds 1 bit
   → 16 registers
Visual:
   3-bit:  [xxx]
   REX:     x[xxx]
            ↑
         new MSB

10. Final Summary

• ModR/M encodes register fields in 3 bits

• REX adds a missing 4th bit

• Final register = 0–15 range

• Enables R8–R15 without breaking old x86