Hex-Rays Decompiler Tips and tricks Volatile memory

https://www.hex-rays.com/products/decompiler/manual/tricks.shtml

First of all, read the troubleshooting page. It explains how to deal with most decompilation problems.

Below is a mix of other useful information that did not fit into any other page:

Volatile memory

Sometimes the decompiler can be overly aggressive and optimize references to volatile memory completely away. A typical situation like the following:

                device_ready    DCD ? ; VOLATILE!

                                MOV     R0, =device_ready
                                LDR     R1, [R0]
                LOOP:
                                LDR     R2, [R0]
                                SUB     R2, R1
                                BEQ     LOOP
        

can be decompiled into

                while ( 1 )
                  ;
        

because the decompiler assumes that a variable can not change its value by itself and
it can prove that r0 continues to point to the same location during the loop.

To prevent such optimization, we need to mark the variable as volatile.
Currently the decompiler considers memory to be volatile if it belongs to a segment with one of the following names:

IO, IOPORTS, PORTS, VOLATILE.

The character case is not important.

Constant memory

Sometimes the decompiler does not optimize the code enough because it assumes that variables may change their values. For example, the following code:

                  LDR     R1, =off_45934
                  MOV     R2, #0
                  ADD     R3, SP, #0x14+var_C
                  LDR     R1, [R1]
                  LDR     R1, [R1]        ; int
                  BL      _IOServiceOpen
        

can be decompiled into

                IOServiceOpen(r0_1, *off_45934, 0)
        

but this code is much better:

                IOServiceOpen(r0_1, mach_task_self, 0)
        

because

                off_45934 DCD _mach_task_self
        

is a pointer that resides in constant memory and will never change its value.
 
The decompiler considers memory to be constant if one of the following conditions hold:

1. the segment has access permissions defined but the write permission is not in the list
   (to change the segment permissions use the SetSegmentAttr built-in function)
2. the segment type is CODE
3. the segment name is one of the following (the list may change in the future):

   .text, .rdata, .got, .got.plt, __text, __const, __const_coal, __cstring, __literal4,
   __literal8, __pointers, __nl_symbol_ptr, __la_symbol_ptr,
   __objc_protorefs, __objc_selrefs, __objc_classrefs, __objc_superrefs, __objc_const,
   __message_refs, __cls_refs, __inst_meth, __cat_inst_meth, __cat_cls_meth.

CONTAINING_RECORD macro

The decompiler knows about the CONTAINING_RECORD macro and tries to use it in the output.
However, in most cases it is impossible to create this macro automatically,
because the information about the containing record is not available.
The decompiler uses three sources of information to determine if CONTAINING_RECORD should be used:

1. If there is an assignment like this:

               v1 = (structype *)((char *)v2 - num);
           

   it can be converted into

               v1 = CONTAINING_RECORD(v2, structype, fieldname);
           

   by simply confirming the types of v1 and v2. 
   NOTE: the variables types must be specified explicitly.
   Even if the types are displayed as correct, the user should press Yfollowed by Enter to confirm the variable type.

2. Struct offsets applied to numbers in the disassembly listing are used as a hint
   to create CONTAINING_RECORD. For example, applying structure offset to 0x41C in

               sub     eax, 41Ch
           

   will have the same effect as in the previous point. Please note that it makes sense to confirm the variable types as explained earlier.

3. Struct offsets applied to numbers in the decompiler output. For example, applying _DEVICE_INFO structure offset to-131 in the following code:

               deviceInfo = (_DEVICE_INFO *)((char *)&thisEntry[-131] - 4);
           

   will convert it to:

               deviceInfo = CONTAINING_RECORD(thisEntry, _DEVICE_INFO, ListEntry);
           

   Please note that it makes sense to confirm the variable types as explained earlier.

Indirect calls

Since the arguments of indirect calls are collected before defining variables, specifying the type of the variable
that holds the function pointer may not be enough. The user have to specify the function type using other methods in this case.
The following methods exist (in the order of preference):

1. For indirect calls of this form:

               call ds:funcptr
           

   If funcptr is initialized statically and points to a valid function, just ensure a correct function prototype. The decompiler will use it.

2. For indirect calls of this form:

               call [reg+offset]
           

   If reg points to a structure with a member that is a function pointer, just convert the operand into a structure offset (hotkey T):

               call [reg+mystruct.funcptr]
           

   and ensure that the type of mystruct::funcptr is a pointer to a function of the desired type.

3. Specify the type of the called function using Edit, Operand type, Set operand type.
   If the first two methods can not be applied, this is the recommended method.
   The operand type has the highest priority, it is always used if present.
4. If the address of the called function is known, use Edit, Plugins, Change the callee address (hotkey Alt-F11).
   The decompiler will use the type of the specified callee. This method is available only for x86.
   For other processors adding a code cross reference from the call instruction to the callee will help.