Plugin for IDA Pro

Table of Contents

• About
• Installation
• Configuration
• Fork features
  • Config
  • Create netnoded virtual tables
  • Dummy structure create
  • Sync method and function names
  • Struct field recast
  • Rename a field/variable according to its type
• Features
  • Structure reconstruction
    • Structure View
    • Templated Types View
  • Disassembler code manipulations
    • Containing structures
    • Function signature manipulation
    • Recasting & Renaming
    • Name Propagation
    • Untangling 'if' statements
  • Classes
  • Structure Graph
  • API
• Presentations

Most of the changes for IDA 9 compatibility, mine and from other people, have been added to the code. Now it need to test and fish out the remaining porting issues.

It's my fork of plugin with tuning for my convenience of work. Changes:

• Main change. I added "Feature Config", which allows you to enable and disable certain elements of the pop-out menus and some another functionalities. I don't use central feature "Reconstruct type", but various auxiliary recasts functions is amazing. Config can be obtain in plugin form through Edit->Plugins->HexRaysPyTools "Configure features" button
• I added my own version of creating of type and virtual tables. In my variant, a simple structure of a certain size with fields of a given width is created. Further, the fields are detailed during analysis.
• Added new feature to pop-out menu "TakeTypeAsName" that works for variables and structure fields.
• And rewrote parsing for type recasts. I think my variant more stable and correct.

Pulled some changes from https://github.com/sabason/HexRaysPyTools, https://github.com/NyaMisty/HexRaysPyTools, https://github.com/oopsmishap/HexRaysPyTools and https://github.com/mahmoudimus/HexRaysPyTools

• Added new Templated Type view to Structure Builder
• Compatibility fixes for IDA 9
• Some fixes and upgrades

You can read about it here Fork features

=====

The plugin assists in the creation of classes/structures and detection of virtual tables. It also facilitates transforming decompiler output faster and allows to do some stuff which is otherwise impossible.

Note: The plugin supports IDA Pro 7.x with Python 2/3.

• Copy HexRaysPyTools.py file and HexRaysPyTools directory to $IDAUSR/plugins directory.

• Install the plugin requirements found in the requirements.txt file. Be sure to install these for the Python environment used by your IDA installation.

  • pip install -r requirements.txt

Can be found at $IDAUSR\cfg\HexRaysPyTools.cfg

• debug_message_level. Set 10 if you have a bug and want to show the log along with the information about how it was encountered in the issue.
• propagate_through_all_names. Set True if you want to rename not only the default variables for the Propagate Name feature.
• store_xrefs. Specifies whether to store the cross-references collected during the decompilation phase inside the database. (Default - True)
• scan_any_type. Set True if you want to apply scanning to any variable type. By default, it is possible to scan only basic types like DWORD, QWORD, void * e t.c. and pointers to non-defined structure declarations.
• templated_types_file. Set to default TOML file path for templated types view. (Default - Empty, will auto load %IDA_USER_DIR%/cfg/templated_types.toml)

Most of the features can be enabled/disabled via the added configuration window. To switch a feature, double-click on the selected one.

A vtable can be created either from pseudocode (via a pointer to a global variable) or from an address in the disassembly. It is created info in a netnode, a store inside the IDB, where, for each method in each table, the function’s offset from the table base is stored. As a result, you can always jump to a method’s code—for example, by double-clicking in pseudocode. Or via Jump to method function action in Local Types window

You can create a struct of a given size and fill it with fields of fixed widths (1, 2, 4, 8 bytes). You can set alignment. You can specify a base class. Click at number to set the field size.

Because each method entry is mapped to its function, names stay in sync. Renaming a method in a vtable renames all other entries pointing to the same function, and renames the function itself. Renaming a function renames all method entries across all vtables that reference it.

Walks the AST to determine the exact type to apply (not perfect, but better than the original). Handles cases like Byte2(struct->field).

something* → p_something

Something* → pSomething

something → o_something

Recently added

The reconstruction process usually comprises the following steps:

1. Open structure builder.
2. Find a local variable that points to the structure you would like to reconstruct.
3. Apply "Scan variable". It will collect the information about the fields that were accessed in the boundaries of one function. As an option, you can apply "Deep Scan variable", which will do the same thing but will also recursively visit other functions that has the same variable as its argument.
4. After applying steps 2 and 3 enough times, resolve conflicts in the structure builder and finalize structure creation. All the scanned variables will get a new type. Also, cross-references will be remembered and usable anytime.

Now, a few more details.

The place where all the collected information about the scanned variables can be viewed and modified. Ways of collecting information:

• Right Click on a variable -> Scan Variable. Recognizes fields usage within the current function.
• Right Click on a variable -> Deep Scan Variable. First, recursively touches functions to make Ida recognize proper arguments (it happens only once for each function during a session). Then, it recursively applies the scanner to variables and functions, which get the structure pointer as their argument.
• Right Click on a function -> Deep Scan Returned Value. If you have the singleton pattern or the constructor is called in many places, it is possible to scan all the places, where a pointer to an object was recieved or an object was created.
• API [TODO]

Structure builder stores collected information and enables interaction:

• Types with the BOLD font are virtual tables. A double click opens the list with all virtual functions, which helps to visit them. The visited functions are marked with a cross and color:

• Types with the ITALIC font have been found as passed argument. It can help in finding substructures. [TODO]
• Double click on field Name or Type to edit.
• Double click on Offset opens a window with all the places, where this field has been extracted. Click the "Ok" button to open a selected place in the pseudocode window:

Buttons serve the following purpose:

Finalize - opens a window with an editable C-like declaration and assigns new types to all scanned variables.

Disable, Enable - are used for collision resolution.

Origin - switches the base offset which is used to produce new fields to structure (this value will be added to every offset of a newly-scanned variable, default = 0).

Array - renders a selected field as an array the size of which is automatically calculated.

Pack - creates and substitutes a substructure for selected items (collisions for these items should be resolved).

Unpack - dismembers a selected structure and adds all its fields to the builder.

Load - loads a predefined type that is loaded into IDA into the structure builder.

Remove - removes the information about selected fields.

Clear - clears all.

Recognize Shape - looks for appropriates structure for selected fields.

Resolve Conflicts (new) - attempts to disable less meaningful fields in favor of more useful ones. (char > _BYTE, SOCKET > _DWORD etc). Doesn't help to find arrays.

Templated Types View - switches the templated types view

Structure View - switches to the structure builder view

With HexRaysPyTools, every time the F5 button is pressed and code is decompiled, the information about addressing to fields is stored inside cache. It can be retrieved with the "Field Xrefs" menu. So, it is better to apply reconstructed types to as many locations as possible to have more information about the way structures are used.

Note: IDA 7.4 has now an official implementation of this feature, available through Shift-X hotkey.

Warning!! Very raw feature. The idea is to help find where a variable came from so as to run Deep Scan Process at the very top level and not to skip large amounts of code.

Usage:

1. In Pseudocode viewer, right click on a number -> "Structures with this size". (hotkey "W")
2. Select a library to be looked for structures.
3. Select a structure. The Number will become sizeof(Structure Name), and type will be imported to Local Types.

Helps find a suitable structure by the information gleaned from pseudocode after variable scanning.

Usage:

• Method 1
  1. Right click on a variable with -> Select "Recognize Shape".
  2. Select Type Library.
  3. Select structure.
  4. Type of the variable will be changed automatically.
• Method 2
  1. Clear Structure Builder if it's currently used.
  2. Right click on the variables that are supposed to be the same -> "Scan Variable".
  3. Edit types (will be implemented later), disable or remove uninteresting fields, and click the "Recognize Shape" button.
  4. You can select several fields and try to recognize their shapes. If found and selected, they will be replaced with a new structure.
  5. After final structure selection, types of all scanned variables will be changed automatically.

The templated types view allows you to easily define and propagate templated types such as the data types you find within STL.

Type List - list of the loaded types, click on the type you want to set to populate the middle form.

Selected Type - each templated typename has two fields, Type & Name. The Type field is the actual type that will be defined, this has to be a real type, as it will throw an error if it is not. The Name field is used to create an unique typename for the templated type.

Creating Type - a live output of the types you will set.

Reload Templated Types TOML - if you have edited the current TOML file you will have to reload the list.

Open Templated Types TOML - open your own custom templated types TOML following the templated types structure

Before:

After

["std::vector<T>"]           
base_name = "std_vector_{1}"
types = ["T"]                
struct = """                 
struct std_vector_{1}        
{{                           
   {0} *_Myfirst;            
   {0} *_Mylast;             
   {0} *_Myend;              
}};                          
"""                          

Class Name - pretty type name, this is used as dictionary key and in the types list (["std::vector<T>"])

Base Name - - name used for IDA's structs as we cannot use ::, format identifiers for type name, these are always odd numbers (Note: this must match the base structure name within struct string)

Types - define the types here as a list of strings, each type will have 2 format specifier tokens

Struct - this is where you define the struct, format specifiers work as the following:

• 1st type:
  • {0} (actual type)
  • {1} (pretty print type, int* -> pInt)
• 2nd type:
  • {2} (actual type)
  • {3} (pretty print type, std::string* -> pStdString)
• (Note: brackets need to be doubled up {{ & }} due to str.format

Helps find containing structure and makes code prettier by replacing pointers with CONTAINING_RECORD macro

Before:

After:

Usage:

If a variable is a structure pointer and there's an access to outside of the boundaries of that structure, then:

1. Right click -> Select Containing Structure.
2. Select Type Library.
3. Select appropriate Structure and Offset.
4. If the result does not satisfy the requirements, then Right Click -> Reset Containing Structure and go back to step 1.

1. Right click first line -> "Remove Return" converts return type to void (or from void to _DWORD).
2. Right click on argument -> "Remove Argument" disposes of this argument.
3. Right click on convention -> "Convert to __usercall" switches to __usercall or __userpurge (same as __usercall but the callee cleans the stack).

Every time you have two sides in an expression, where each side may be a local or global variable, argument or return value of the function signature, it is possible to right-click or press the hotkey to give both sides of the expression similar types. Below, there is the table of possible conversions:

When you have an expression like function(..., some_good_name, ...), you can rename function parameter.

When you have an expression like function(..., v12, ...), and function has an appropriate parameter name, you can quickly apply this name to the variable.

Also possible to rename vXX = v_named into _v_named = v_named and vice versa.

And there's a feature for massive renaming functions using assert statements. If you find a function that looks like an assert, right-click the string argument with the function name and select "Rename as assert argument". All the functions where a call to assert statement has happened will be renamed (provided that there is no conflicts, either way, you'll see the warning in the output window)

This feature does the same recursive traversal over functions as the Deep Scan Variable does. But this time, all elements that have a connection with the selected one receive its name. It’s possible to rename it or use names of both local and global variables, as well as structure members. By default, the plugin propagates names only over default names like v1, a2. See Configuration in order to change that.

• Clicking if manually allows to switch then and else branches
• Automatically applies the following transformations:

Before:

...
if (condition) {
    statement_1;
    statement_2;
    ...
    return another_value;
}
return value;

After:

...
if (opposite_condition) {
    return value;
}
statement_1;
statement_2;
...
return another_value;            // if 'then' branch has no return, than `return value;`

Also, it can be found at View->Open Subview->Classes. Helps to manage classes (structures with virtual tables).

Class, virtual tables, and functions names are editable. Also a function's declaration can be edited. After editting, the altered items change font to italic. Right click opens the following menu options:

• Expand All / Collapse All.
• Refresh - clear all and rescan local types for information again.
• Rollback - undo changes.
• Commit - apply changes. Functions will be renamed and recasted both in virtual tables in Local Types and disassembly code.
• Set First Argument type - allows selecting the first argument for a function among all classes. If right click was used on class name, then its type will be automatically applied to the virtual table at offset 0.

You can also filter classes using Regexp either by class_name or by existence of specific functions. Simply input an expression in line edit for filtering by class_name or prepend it with "!" to filter by function name.

Shows relationship between structures:

Also: dark green node is union, light green - enum.

Usage:

1. Open Local Types.
2. Select structures and right click -> "Show Graph" (Hotkey "G").
3. Plugin creates a graph of all structures that have relationship with selected items.
4. Double clicking on a node recalculates the graph for it.
5. Every node has a hint message that shows C-like typedef.

Under construction

• ZeroNights 2016
• Insomni'hack 2018
• Infosec in the City 2018 (Slides)