Thursday, June 23, 2011

Hex-Rays + C++ = The Low Pass Filter

Recently doing some RE on a closed-source C++ library, and Ida alone wasn't cutting it as the C++ name-mangling, and operations can be quite 'noisy'. So I figured why not throw it through Hex-Rays' decompiler to get a look at just the C++ operations.

My favorite feature of Hex-Rays is being able to quickly recognize obfuscation operations. These are generally instruction intensive and if written in a high-level language, often times filled with extra mov's and unnecessary operations. This makes it more time-consuming to decipher. Hex-Rays generally chews this kind of stuff up and spits out the juicy bits.

Now, following the de-obfuscation was a series of C++ string-class operations that consisted of concatenating various pieces. To aid in deciphering which pieces were used when, I figured that Hex-Rays would be able to easily decipher this and provide me the same goodness as the de-obfuscation.

Anyone who's used Hex-Rays knows all about its ability (or IN-ability) to actually decompile some code and the feature of just omitting the code. In this case, I was hoping that it should at least pick up the C++ object operations and just display those to minimize the extraneous work. Turns out, Hex-Rays actually dropped most of the C++ operations that I was originally interested on the floor. BUT, what it did reveal was a key even better than what I was searching for. So in this case, Hex-Rays acted as low-pass filter to remove the noise that I thought was interesting data. Lesson learned? Just because it was omitted doesn't mean it wasn't useful, but it may make something else more obvious.