Developing PE file packer step-by-step. Step 3. Unpacking

Previous step is here.

Let’s continue! It’s time to write an unpacker, this is what we are going to do during this step. We will not process import table for now, because we have some other things to do at this lesson.

We will begin from the following thing. To operate, the unpacker definitely needs two WinAPI functions: LoadLibraryA and GetProcAddress. In my old packer (that I’ve written once) I developed unpacker stub in MASM32 without creating import table at all. I looked for these function addresses in kernel, which is rather complicated and hardcore, besides that, this may cause serious antivirus suspicions. This time, let’s create import table and make loader to tell us these function addresses. Of course, set of these two functions in import table is as suspicious as their total absence, but nothing prevents us from adding more random imports from different .dll files in future. Where will the loader store these two function addresses? It’s time to expand our packed_file_info structure!

I added three fields to the structure. The loader will add LoadLibraryA and GetProcAddress function addresses from kernel32.dll to first two fields. Last field points to the end of import address table, and we will write null to it, to let the loader know that we don’t need any more functions. I will tell you more about this later.

Now we need to create a new import table. My PE library will be very handy for this. (We will forget about old original import table for now).

The first part is clear – we created library import, added a couple of functions, created imported library list consisting of kernel32.dll only. I will explain the line, where we set RVA to IAT (kernel32.set_rva_to_iat). For each imported library the following structure is created in import table:

Loader writes imported function addresses to Import Address Table (IAT) for each imported dll, and it takes imports or imported function ordinals from Original Import Address Table (or, in other words, Import Lookup Table). We can manage without last one, for example, all Borland compilers always operate that way, they don’t care about Import Lookup Table. In this case our only Import Address Table initially contains imported function ordinals or names, and there, over these data, the loader writes imported function addresses. We will not create Original Import Address Table too, we will manage without it (import tables will take less space), so let’s turn off this option in imports rebuilder.

The settings.save_iat_and_original_iat_rvas call sets up the rebuilder in such way, that it will not create its own IAT and Original IAT, but write everything by the addresses, which are specified in each library (do you remember kernel32.set_rva_to_iat call?).

Then we just rebuild import table. Start not yet finished packer, pass its name as first parameter, and watch the result. Make sure that everything works as expected:


Now we start the resulting binary in OllyDbg and make sure, that the loader wrote the addresses of two required functions to the correct places:


As you can see, the addresses we need have been written to 0x1009 and 0x100D, this means everything was done correctly. (Entry point address is absolutely random yet, and there isn’t any unpacker, so the file still will not run, but we achieved a lot already).

Let’s go further. Now we need to prepare our sources to develop an unpacker. We move all structures from main.cpp to structs.h file, it will contain the following:

There is no need to explain anything here, we just moved the code. We will include this file to main.cpp in turn:

It’s time for hardcore! We will develop an unpacker. I thought a little and decided not to use MASM32, but develop it in C with C++ elements and inline assembler – this will increase code readability. So, we create a new project in solution and call it unpacker. We add unpacker.cpp file to it and create parameters.h. Further we set up the project settings like we did in lzo-2.06 at first step, to make the build small and base independent. Set unpacker_main as entry point (Linker – Advanced – Entry Point). Further, in Configuration Manager (see step 1) make this project to be built always in Release configuration:


Set simple_pe_packer project dependency to unpacker project (Project Dependencies, as at step 1) and add parameters.h file to packer project includes – we will write required unpacker build parameters to this file:

Now we start to develop the unpacker itself. Open unpacker.cpp…

So, I start the explanations. Firstly, we included the file, which contains packer structures declarations – we will need them in unpacker. Further we create entry point – the
unpacker_main procedure. Notice that this function is declared as naked. This tells the compiler not to create prologue and epilogue (stack frame) for this function automatically. We need to do this manually, and why – I will explain this at the next step. Now we create the same prologue and epilogue as created by MSVC++ compiler. “sub esp, 128” line allocates 128 bytes on stack – this will be enough for our needs. The packer will not do anything serious at this step. We need prologue and epilogue to let us allocate memory on stack without additional issues. At the end we write ret instruction – return to kernel. Now we will write most simple packer body. Let it just welcome us by displaying a Message Box with “Hello!” message.

Here we declared two local variables. First one contains actual image loading address, and the second one – relative first section address, in which, as you remember, we store all required unpacker information and packed data themselves. Using packer, we will save real values instead of 0x11111111 and 0x22222222.

It looks like everything is clear here. At the beginning of first packed file section there is packed_file_info structure, which is created by packer. It has three additional fields, which are filled by the loader itself – we set up import table in that way, as you remember. We get LoadLibraryA and GetProcAddress function addresses from these fields. You can also ask, why I declare all variables first, and assign their values later, although I could make this with one line. The thing is, that it is not possible to declare a variable and assign its value right away in naked functions.

And the last (for now) packer code part:

This also should be clear in general, except weird string filling. We allocated buf buffer on stack. All our strings also should be allocated on stack – we can not write anything to data section, because it will inevitably bring to relocations occurrence, and the code will become base dependent. This is why we so absurdly write strings directly to stack buffer by 4 bytes. We also need to keep in mind backward bytes order, which is used in x86 architecture, and we write code for it, so letters in 4 byte string pieces are arranged backwards.

At first we load user32.dll library, then we get MessageBoxA procedure address from it, and then we call it. That’s all with unpacker!

There is one thing left – we need to insert packer code to packed file and set it up. I decided to automate it. To do this, let’s add a new project named unpacker_converter to a solution. The purpose of this project is to open the result of unpacker compilation – the unpacker.exe file, read its only section data (actually, the code) and convert it to an .h file, which we include to simple_pe_packer project. Let’s set the same include directory in unpacker_converter project as in simple_pe_packer project, in order to include PE library .h files, then add main.cpp file to project and start development.

I will not provide detailed explanation of this code – lots of it is familiar to you already. I just say that it simply generated unpacker.h file from unpacker.exe file like this:

These data are the hexadecimal representation of first and only unpacker code section data. It is very small and simple yet. How do we make unpacker_converter to generate automatically such h-file for us while rebuilding the unpacker? We need to edit unpacker project settings (Build Events – Post-Build Event):

Why did not I use macro $(Configuration) in this setup? For unpacker project it will always resolve into “Release”, because in both debug and release configurations the project builds as release (we changed this in Configuration Manager earlier). Thus, we will just copy unpacker_converter.exe file from ITS current configuration folder to project root folder, and the unpacker project will be able to take it from there. So, the last thing we should do is to fix unpacker_converter project configuration (Build Events – Post-Build Event):

It is necessary to set Project Dependencies: unpacker on unpacker_converter (probably, this is not absolutely logical, but that’s fine). After that we will build everything in both release and debug configurations.

I will explain, what we are going to write to parameters.h file. It will contain the following:

We write offsets relative to unpacker code beginning (in built binary form) of two numbers – 0x11111111 and 0x22222222. These numbers will be rewritten by unpacker, and offsets 0xC (12) and 0x13 (19) are calculated in any HEX editor or using any autogenerated unpacker.h file. They are likely remain unchanged, because we are not going to write code before two mov commands in unpacker.

Let’s add to simple_pe_packer project includes autogenerated unpacker.h file:

The last stage of this step will be inserting unpacker body to packed file. We did this thing at the previous step:

Now we will insert unpacker code there and set it up:

That’s all! Now the unpacker will be set up and added to the packed file! Let’s test it. As always, we will pack ourselves, as a result we get packed_simple_pe_packer.exe file. We will run it and see long-expected window, for which we had to work so much!


So, the unpacker is correctly built, set up, converted and run, which is good. At the next steps we will make it to do more sensible things!

As always I share the full packer solution (except PE library): Own PE packer step 3

2 thoughts on “Developing PE file packer step-by-step. Step 3. Unpacking”

  1. Hi! I’m trying to do this development of a packer step-by-step
    and I ran into an error LNK2001: unresolved external symbol _main in project unpacker with file name “LIBCMT.lib”. I googled about this problem but couldn’t find the exact solution that helped.. so I’m writing this comment wondering if I would have an answer from you. I’m using visual studio 2015 and also I think this problem would be from the different version or something like that, I guess. Could you give me the advice to solve this problem? thx for entire reading.

    1. This probably means that you have the option Linker -> Input -> “Ignore All Default Libraries” set to “Yes”. Please, check this value.
      You can also upload your project somewhere, so I might take a look.

Leave a Reply

Your email address will not be published. Required fields are marked *