[corCTF 2023] utterly-deranged

Note

I have recently learned that my solution was not intended and only the result of a mistake the challenge author made. The actual goal was to overcome the obfuscation rather than bruteforcing the input.

Doing the things

The challenge came in the form a single binary. Loading it into IDA revealed that only a handful of used functions.

At this point im thinking this will be an easy challenge. I mean there are at max 3-4 functions to be reversed, how hard could it be.

Pressing F5 in the main function reveals what the issue with this challenge might be. We are greeted with a nice error message from Hex-Rays...

A quick google search hints to a maximum size for functions to be decompiled somewhere in the settings of the Hex-Rays decompiler. These settings can be found at Edit->Plugins->Hex-Rays Decompiler. Changing the function size limit to 512 should suffice (see screenshot below).

With that out of the way, lets dive into the main function at last! We are greeted by a whopping 1300 lines of decompiled code, most of which look like this:

Oh did i mention there are over 360 local variables? At this point i was determined to not reverse engineer the code of the main function. At this point i could have researched how to specifically defeat the obfuscation technique used, but a bit of code at the end of the function piqued my interest. To not bother you with more screenshots, i will provide it in regular code form.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

if ( &v366 >= 0 )
  {
    if ( !memcmp(input_buffer, something_else, 0x80uLL) )
    {
      puts("Well done for peeling back the layers");
    }
    else
    {
      puts("Better luck next time :(");
    }
  }

So we just need to pass this memcmp to solve the challenge. At this point i just set a breakpoint at 0x41687C and dumped the contents of both buffers for different inputs. What was noticeable is that upon a single byte of the input changing, only one byte of the encoded input would change. This suggested that the "cipher" was just applying to one byte at a time, so a bruteforce might be feasible!

In order to automate this tedious task, i put together a small script automating gdb to dump both buffers and compare them one byte at a time.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

from pwn import *
context.log_level = "error"
# Exploit configs
binary = ELF("utterlyderanged")

def parse_memdump(dump_string):
  result = []
  for line in dump_string:
    for dump_byte in line.split(b"\t")[1:]:
      result.append(dump_byte)
  return result

if __name__ == "__main__":
  flag = ['_'] * 0x80
  for i in range(0, len(flag)):
    for guess in b"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-{}_":
      io = process(["/usr/bin/gdb"])

      # i am using gef, for vanilla gdb change "gef➤" to "gdb"
      io.sendlineafter("gef➤",b"file utterlyderanged")
      io.sendlineafter("gef➤",b"handle SIGALRM ignore")
      io.sendlineafter("gef➤",b"b *0x41687C")
      io.sendlineafter("gef➤",b"r")

      flag[i] = chr(guess)
      io.sendline("".join(flag))
      # call with DEBUG to change log level
      # call with NOPTRACE to skip gdb attach
      # call with REMOTE to run against live target

      io.sendlineafter("gef➤",b"x/128c $rdi")
      dump = io.recvuntil(b"gef").split(b'\n')
      curr_dump = parse_memdump(dump)
      io.sendline(b"x/128c $rsi")
      dump = io.recvuntil(b"gef").split(b'\n')
      target_dump = parse_memdump(dump)

      if target_dump[i] == curr_dump[i]:
        print("".join(flag),end="\r")
        break

      io.close()

This script takes about 5-10 minutes to run and eventually prints the full flag corctf{s331ng_thru_my_0p4qu3_pr3d1c4t35}.