In this reverse engineering challenge, we’re given two files, one being a program and the other composed of seemingly random bytes.
Solution
Note: For the purposes of this solution, the binary is called
vm
and the bytecode is calledchal.bin
. Your files may be named differently.
Running the binary on its own gives us a usage message, indicating that we need to supply the .bin
file as a command-line argument:
┌──(kali㉿kali)-[~/Desktop/testing-grounds/rev_CestLaVMie]
└─$ ./vm
Usage: ./vm bytecode.bin
Following the usage instructions prompts us to enter the flag and tells us whether we’re correct or not.
Running file
on the binary tells us that the binary is stripped:
┌──(kali㉿kali)-[~/Desktop/testing-grounds/rev_CestLaVMie]
└─$ file vm
vm_challenge: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=b3f2ba1f6da06e57527d89b9a52a411ade894b9e, for GNU/Linux 4.4.0, stripped
Decompilation
Knowing that the binary is stripped, we can head to the entry point of the program (represented by the entry()
function) after loading it in Ghidra. Inside, we see a call to __libc_start_main()
, which is responsible for initializing the runtime environment before calling the main function. We see that FUN_00101120
is supplied as the first argument, so we can assume this is our main function and rename it accordingly. We see that if we supply the correct number of arguments, the following things happen:
- The filename of our
.bin
file gets passed as an argument toFUN_00101290()
- The function
FUN_00101340
is executed - The
free
function is called on global variableDAT_00104048

Let’s take a closer look at what the two first steps are doing.
Loading
At first glance, there appears to be a bunch of file operations taking place in FUN_00101120()
. Assuming that param_1
is chal.bin
, we can see that the contents of the file are read in and saved to DAT_00104048
, which is the global variable we saw earlier in main()
.

Execution
Moving onto the next function, FUN_00101340()
, we see that a lot more is going on here. A massive switch statement is present, with different cases for hex characters 0x01
to 0x19
. It appears that the bytes from chal.bin
file are being iterated through, with each byte corresponding to a specific set of instructions based on the switch statement. This indicates that we’re dealing with a VM-based challenge, where the chal.bin
file is actually bytecode that gets executed as instructions by the binary.
Disassembly
Following this, we can start building a disassembler to more about how the VM works, including which opcodes map to which assembly instructions. From looking at all the cases in the switch statement, we notice that many of the operations affect the global variable DAT_001041a0
. We can guess that this is probably where our registers are located.
Below is a summary of instructions, opcodes, and argument counts:
Mnemonic | Opcode | Arg Count |
---|---|---|
MOVI | 0x01 | 2 |
MOVR | 0x02 | 2 |
ADD | 0x03 | 2 |
SUB | 0x04 | 2 |
XOR | 0x05 | 2 |
CMP | 0x06 | 2 |
JZ | 0x07 | 1 |
JNZ | 0x08 | 1 |
JMP | 0x09 | 1 |
READ | 0x0A | 2 |
PRINT | 0x0B | 1 |
HLT | 0x0C | 0 |
NOP | 0x0D | 0 |
MULADD | 0x10 | 3 |
SWAP | 0x11 | 2 |
SALT | 0x12 | 1 |
ROL | 0x13 | 2 |
ODD | 0x14 | 1 |
STRPRINT | 0x15 | 2 |
STORE | 0x16 | 2 |
LOAD | 0x17 | 2 |
LOADR | 0x18 | 2 |
STORER | 0x19 | 2 |
The SALT
instruction is rather interesting, as it is not a common one. If we look back at the decompiled code, it appears to perform the following transformation on the input passed to it:
(x * 31 + 7) ^ 0xC0DE) & 0xFF
We can attempt to disassemble chal.bin
and annotate it, which results in something like the instructions below:
MOVI R0 69
STORE R0 200
MOVI R0 110
STORE R0 201
MOVI R0 116
STORE R0 202
MOVI R0 101
STORE R0 203
MOVI R0 114
STORE R0 204
MOVI R0 32
STORE R0 205
MOVI R0 102
STORE R0 206
MOVI R0 108
STORE R0 207
MOVI R0 97
STORE R0 208
MOVI R0 103
STORE R0 209
MOVI R0 58
STORE R0 210
STRPRINT 200 11
READ R0 0
STORE R0 0
READ R0 0
STORE R0 1
READ R0 0
STORE R0 2
READ R0 0
STORE R0 3
READ R0 0
STORE R0 4
READ R0 0
STORE R0 5
READ R0 0
STORE R0 6
READ R0 0
STORE R0 7
READ R0 0
STORE R0 8
READ R0 0
STORE R0 9
READ R0 0
STORE R0 10
READ R0 0
STORE R0 11
READ R0 0
STORE R0 12
READ R0 0
STORE R0 13
READ R0 0
STORE R0 14
READ R0 0
STORE R0 15
READ R0 0
STORE R0 16
READ R0 0
STORE R0 17
READ R0 0
STORE R0 18
READ R0 0
STORE R0 19
READ R0 0
STORE R0 20
READ R0 0
STORE R0 21
READ R0 0
STORE R0 22
READ R0 0
STORE R0 23
READ R0 0
STORE R0 24
READ R0 0
STORE R0 25
READ R0 0
STORE R0 26
READ R0 0
STORE R0 27
READ R0 0
STORE R0 28
READ R0 0
STORE R0 29
READ R0 0
STORE R0 30
READ R0 0
STORE R0 31
MOVI R0 45
STORE R0 100
MOVI R0 212
STORE R0 101
MOVI R0 38
STORE R0 102
MOVI R0 82
STORE R0 103
MOVI R0 119
STORE R0 104
MOVI R0 1
STORE R0 105
MOVI R0 16
STORE R0 106
MOVI R0 188
STORE R0 107
MOVI R0 65
STORE R0 108
MOVI R0 199
STORE R0 109
MOVI R0 47
STORE R0 110
MOVI R0 90
STORE R0 111
MOVI R0 66
STORE R0 112
MOVI R0 174
STORE R0 113
MOVI R0 17
STORE R0 114
MOVI R0 110
STORE R0 115
MOVI R0 126
STORE R0 116
MOVI R0 142
STORE R0 117
MOVI R0 117
STORE R0 118
MOVI R0 187
STORE R0 119
MOVI R0 225
STORE R0 120
MOVI R0 138
STORE R0 121
MOVI R0 48
STORE R0 122
MOVI R0 250
STORE R0 123
MOVI R0 155
STORE R0 124
MOVI R0 52
STORE R0 125
MOVI R0 72
STORE R0 126
MOVI R0 249
STORE R0 127
MOVI R0 120
STORE R0 128
MOVI R0 211
STORE R0 129
MOVI R0 157
STORE R0 130
MOVI R0 34
STORE R0 131
MOVI R2 0
MOVI R3 0
MOVI R4 100
MOVI R6 1
MOVI R7 0
MOVI R8 32
loop:
MOVR R0 R3
ADD R0 R2
LOADR R1 R0
MOVR R0 R2
SALT R0
XOR R1 R0
SALT R1
MOVR R0 R4
ADD R0 R2
LOADR R0 R0
CMP R1 R0
JZ match
MOVI R7 1
match:
ADD R2 R6
CMP R2 R8
JNZ loop
MOVI R0 0
CMP R7 R0
JZ success
JMP fail
success:
MOVI R0 70
STORE R0 240
MOVI R0 108
STORE R0 241
MOVI R0 97
STORE R0 242
MOVI R0 103
STORE R0 243
MOVI R0 32
STORE R0 244
MOVI R0 99
STORE R0 245
MOVI R0 111
STORE R0 246
MOVI R0 114
STORE R0 247
MOVI R0 114
STORE R0 248
MOVI R0 101
STORE R0 249
MOVI R0 99
STORE R0 250
MOVI R0 116
STORE R0 251
MOVI R0 33
STORE R0 252
MOVI R0 10
STORE R0 253
STRPRINT 240 14
HLT
fail:
MOVI R0 87
STORE R0 220
MOVI R0 114
STORE R0 221
MOVI R0 111
STORE R0 222
MOVI R0 110
STORE R0 223
MOVI R0 103
STORE R0 224
MOVI R0 32
STORE R0 225
MOVI R0 102
STORE R0 226
MOVI R0 108
STORE R0 227
MOVI R0 97
STORE R0 228
MOVI R0 103
STORE R0 229
MOVI R0 33
STORE R0 230
MOVI R0 10
STORE R0 231
STRPRINT 220 12
HLT
Recovering the Flag
Now that we have the assembly instructions, we can attempt to recover the flag. The first block of instructions, up until STRPRINT 200 11
, represent the “Enter flag:” prompt from the beginning of the program. The next block, which consists of STORE R0 xx
and READ R0 0
instructions, represents our input to the program. Following that, we know the transformed flag is stored within the program, so we can assume that the next block of MOVI
and STORE
instructions represent it.
With our knowledge of the SALT transformation, we can build a script that attempts to find the correct input bytes:
def inverse_SALT(y, index):
key_from_index = ((index * 31 + 7) ^ 0xC0DE) & 0xFF
for x in range(256):
intermediate_byte = x ^ key_from_index
if ((intermediate_byte * 31 + 7) ^ 0xC0DE) & 0xFF == y:
return x
def decrypt_string(encrypted_list):
flag = ''
for i, encrypted_byte in enumerate(encrypted_list):
flag += chr(inverse_SALT(encrypted_byte, i))
return flag
encrypted_list = [45, 212, 38, 82, 119, 1, 16, 188, 65, 199, 47, 90, 66, 174, 17, 110, 126, 142, 117, 187, 225, 138, 48, 250, 155, 52, 72, 249, 120, 211, 157, 34]
print(decrypt_string(encrypted_list))
Now that we know the input, we can confirm that it’s the flag:
MetaCTF{In5an3_1n_7he_VM3mbr4n3}