x86-64 Machine Code Encoding: Encoding Examples

- MOV, ADD, JMP, CALL.

Introduction

This section provides concrete examples of machine code encoding for several fundamental x86-64 instructions: MOV, ADD, JMP, and CALL. These instructions are central to virtually every program and understanding their encoding is critical for designing a functional assembler. Examples cover different operand types and addressing modes, illustrating the interplay of prefixes, opcode bytes, ModR/M, SIB, displacement, and immediate fields.

1. MOV Instruction Encoding Examples

The MOV instruction copies data from a source operand to a destination operand. Its encoding varies depending on operand types (register, memory, immediate) and operand size.

Example 1: Register-to-Register MOV — MOV RAX, RBX

• Instruction: Move contents of RBX into RAX

• Encoding:

  • No prefix needed for 64-bit registers except REX

  • REX prefix required to access 64-bit registers and extended registers (RAX and RBX are low registers but 64-bit size requires REX.W)

  • Opcode: 0x89 for MOV r/m64, r64

  • ModR/M byte: Mod=11 (register direct), Reg=RBX (011), R/M=RAX (000)

• Machine code breakdown:

  • REX prefix: 0x48 (01001000) — W=1 (64-bit), R=0, X=0, B=0

  • Opcode: 0x89

  • ModR/M: 0xD8 (11011000 binary: Mod=11, Reg=011, R/M=000)

• Final encoding: 48 89 D8

Example 2: Immediate to Register — MOV RAX, 0x12345678

• Moves immediate 32-bit value into RAX (zero-extended to 64 bits)

• Encoding:

  • REX prefix: 0x48 (for 64-bit operand)

  • Opcode: 0xB8 + reg where reg=000 for RAX → 0xB8

  • Immediate: 4 bytes (0x78 0x56 0x34 0x12) — little endian

• Final encoding: 48 B8 78 56 34 12 00 00 00 00

Note: Since immediate is 32-bit, assembler zero-extends to 64-bit in this case.

Example 3: MOV from Memory to Register — MOV RAX, [RBP-0x10]

• Moves 8 bytes from memory at address [RBP - 0x10] to RAX

• Encoding:

  • REX prefix: 0x48 (64-bit)

  • Opcode: 0x8B for MOV r64, r/m64

  • ModR/M: Mod=01 (8-bit displacement), Reg=RAX(000), R/M=RBP(101)

  • Displacement: 0xF0 (two’s complement of -0x10)

• Final encoding: 48 8B 45 F0

2. ADD Instruction Encoding Examples

The ADD instruction performs addition between two operands and stores the result in the destination operand.

Example 1: Register to Register ADD — ADD RAX, RCX

• Adds RCX to RAX, result in RAX

• Encoding:

  • REX prefix: 0x48 (64-bit operand)

  • Opcode: 0x01 for ADD r/m64, r64

  • ModR/M: Mod=11 (register), Reg=RCX(001), R/M=RAX(000)

• Final encoding: 48 01 C8

Example 2: Immediate to Register ADD — ADD RDX, 5

• Adds immediate 8-bit value 5 to RDX

• Encoding:

  • REX prefix: 0x48 (64-bit operand)

  • Opcode: 0x83 for ADD r/m64, imm8

  • ModR/M: Mod=11 (register), Reg=000 (ADD opcode extension), R/M=RDX(010)

  • Immediate: 0x05

• Final encoding: 48 83 C2 05

3. JMP Instruction Encoding Examples

The JMP instruction performs an unconditional jump to a target address.

Example 1: Near Relative JMP — JMP +0x10

• Jump forward 16 bytes relative to next instruction

• Encoding:

  • Opcode: 0xE9

  • Displacement: 4-byte signed little endian (0x10 00 00 00)

• Final encoding: E9 10 00 00 00

Example 2: Short Relative JMP — JMP +0x7

• Jump forward 7 bytes relative to next instruction (within -128 to +127 range)

• Encoding:

  • Opcode: 0xEB

  • Displacement: 1 byte (0x07)

• Final encoding: EB 07

4. CALL Instruction Encoding Examples

The CALL instruction transfers control to a procedure/function, pushing the return address onto the stack.

Example 1: Near Relative CALL — CALL +0x20

• Call subroutine 32 bytes ahead relative to next instruction

• Encoding:

  • Opcode: 0xE8

  • Displacement: 4-byte signed little endian (0x20 00 00 00)

• Final encoding: E8 20 00 00 00

Example 2: Indirect CALL via Register — CALL RAX

• Calls address stored in RAX register

• Encoding:

  • Opcode: 0xFF

  • ModR/M: Mod=11 (register), Reg=010 (CALL opcode extension), R/M=RAX(000)

• Final encoding: FF D0

5. Summary Table of Key Encoding Patterns

6. Important Notes

• The REX prefix is mandatory for 64-bit operand size unless default operand size is 64 bits in a particular mode.

• Opcode extensions often use bits in the ModR/M byte’s Reg field to specify instruction variants.

• Displacement and immediate values are encoded in little-endian order.

• Relative displacements for JMP and CALL are calculated relative to the address of the next instruction.

• The exact encoding depends on the addressing mode and operand sizes; an assembler must dynamically generate these fields based on the operands.

Conclusion

Understanding detailed encoding of instructions like MOV, ADD, JMP, and CALL is fundamental to assembler design. This section has demonstrated how each part of an instruction contributes to its final machine code form, providing a foundation for implementing the encoding logic in an assembler targeting the x86-64 architecture.