Flash CTF – Opermutated

## Initial sanity-checks

$ file opermutated
opermutated: ELF 64-bit LSB executable, x86-64, statically linked, stripped

$ ./opermutated
Enter the flag (without MetaCTF{ and }):

Pretty bare. strings shows the success/fail strings but not the prompt, a hint that the real work is in an embedded payload.

A symbol scan confirms this:

$ nm -C opermutated | grep encoded
00000000006040b0 T encoded_shellcode
0000000000608000 T encoded_shellcode_len
0000000000608008 T encoded_shellcode_entry

So the ELF is a loader plus an XOR-encoded blob.


## Yanking the shell-code out

# extract_blob.py
import lief, struct, pathlib
b   = lief.parse('opermutated')
base = b.get_symbol('encoded_shellcode').value
size = struct.unpack('<Q', b.get_content_from_virtual_address(
        b.get_symbol('encoded_shellcode_len').value, 8))[0]
raw  = bytes(b.get_content_from_virtual_address(base, size))
key  = 0x5A                    # see next note
pathlib.Path('stage2.bin').write_bytes(bytes(c ^ key for c in raw))

Finding the XOR key – dump the first 16 raw bytes, XOR with common values (0x20..0x7F) until you recognize 55 48 89 E5 (push rbp; mov rsp,rbp). 0x5A works.

stage2.bin is ~4 kB of x86-64 shell-code.


## First look at the decrypted blob

Noise everywhere

addq $0, %r11     ;  ←───┐
orq  $0, %r10     ;      │  all 4 lines = NO-OP
imulq $1, %r9     ;      │
subq $0, %r8      ;  ←───┘

Hundreds of those surround every real instruction. Clearly, there was some obfuscator run on this shellcode. Rule of thumb: anything that adds/ors 0 or multiplies by 1 can be ignored.

A meaningful basic block

After deleting the junk, the verification loop becomes tiny:

; rdi → end of user input,   rcx = groups,   rbx = 0xCBF29CE484222325
.loop:
    movzx eax,  byte [rdi-1]      ; b0
    movzx edx,  byte [rdi-2]      ; b1
    movzx ebx,  byte [rdi-3]      ; b2

    ; pack little-endian
    mov   ecx, eax
    shl   edx, 8       ; b1 << 8
    or    ecx, edx
    shl   ebx, 16      ; b2 << 16
    or    ecx, ebx     ; ecx = b0 | b1<<8 | b2<<16

    imul  ecx, 0x5F356495   ; *MULT  (odd)
    xor   ecx, 0xA6C3E1D2   ; ^XOR   (hides plaintext)

    cmp   ecx, DWORD PTR [rip+triplet_tbl+rcx*4]
    jne   wrong

    xor   rbx, rcx          ; FNV-64
    imul  rbx, 0x100000001B3

    sub   rdi, 3
    dec   rcx
    jnz   .loop

    cmp   rbx, QWORD PTR [rip+final_state]
    je    success
wrong:
    ; fall-through → exit 1

The data right after final_state matches exactly this layout:

<u32 len>  <u32 0>  <u64 final_state>  <u32 groups>  <u32 triplet₀> …

Recognizing FNV-64

The constants 0xCBF29CE484222325 (offset basis) and 0x100000001B3 (prime) scream “FNV-64”. This is never even actually needed to solve the challenge.

Inverting the triplet transform

Mathematics:

val   = ((packed * MULT) ^ XOR)  mod 2³²
packed = b0 | b1<<8 | b2<<16
MULT  = 0x5F356495   (odd)
XOR   = 0xA6C3E1D2
MULT⁻¹= 0x32C446BD   (because MULT*INV ≡ 1 mod 2³²)

Therefore

packed = ((val ^ XOR) * MULT⁻¹) & 0xFFFFFFFF

Split the 24-bit packed back into b0,b1,b2. Doing this for each table entry (and reversing the order) yields every byte of the flag except for the leading zero-padding needed when the length isn’t a multiple of 3.


## One-screen solver

#!/usr/bin/env python3
import lief, struct, sys
MULT, XOR, INV, KEY = 0x5F356495, 0xA6C3E1D2, 0x32C446BD, 0x5A

bin  = lief.parse(sys.argv[1] if len(sys.argv)>1 else 'opermutated')
base = bin.get_symbol('encoded_shellcode').value
sz   = struct.unpack('<Q', bin.get_content_from_virtual_address(
        bin.get_symbol('encoded_shellcode_len').value, 8))[0]
blob = bytes(b ^ KEY for b in bin.get_content_from_virtual_address(base, sz))

# locate header (scan backwards)
for i in range(len(blob)-20, -1, -1):
    ln = struct.unpack_from('<I', blob, i)[0]
    if 1 <= ln <= 128 and blob[i+4:i+8] == b'\0\0\0\0':
        groups = struct.unpack_from('<I', blob, i+16)[0]
        if groups == (ln+2)//3:
            off=i; length=ln; break

vals=[]; p=off+20
while len(vals) < groups:
    v = struct.unpack_from('<I', blob, p)[0]
    if v or vals: vals.append(v)
    p +=4

rev=[]
for v in vals:
    pk = ((v ^ XOR) * INV) & 0xFFFFFFFF & 0xFFFFFF
    rev.extend([pk&0xFF, pk>>8 &0xFF, pk>>16 &0xFF])
core = bytes(reversed(rev))[groups*3-length:][:length].decode()
print(f'MetaCTF{{{core}}}')
MetaCTF{0bfus4t3d_4s5embl3y_bu7_b3hav10r_n3v3r_l135}