Tag Archives: Programming

Unpacking FFXi Related Files

One of the many aspects of reversing Final Fantasy XI be it statically on disk or live in memory requires that parts of the game files are unpacked. Square Enix has taken the time to create an internal packer of the games sensitive data for the PE files (.exe and .dll). In many of the game files, there is a packed section called POL1. This section contains the files actual .text section data, but is packed.

For example, this is the games FFXiMain.dll as of Jan 01, 2015:
FFXiMain Sections

In this image you’ll see something strange. The files .text section has a raw size of 0. This is because the actual data does not exist in the file until its unpacked. Instead, the files .text data is actually within the POL1 section. If we load FFXiMain.dll inside of OllyDbg, or any other debugger / disassembler, we will see this at the files current entry point:
FFXiMain Unpacking Routine

Like much of SE’s other packing / encoding methods, the encryption of the section is not very secure or unique.
Some basic shifting and such and the unpacked data is obtained fairly easily.

However, handling this unpacking externally from a static context is a bit more difficult because it involves having to rebuild the PE file on disk after it is unpacked. The .text section must be resized to its proper size containing the new raw data which in turn unaligns all the other sections. So each section must be rebuilt and realigned with the file. Afterward, we must also set the files new size and its new entry point.

Unpacking the file is fairly trivial, we can just steal the decryption method from the games file and use it locally. If we adjust the ASM as needed, we can reuse it like this:

;*******************************************************************************
; XiUnpack.asm (c) 2014 atom0s [atom0s@live.com]
;
; Unpacks a given POL1 section from a file.
; This function is taken from the FFXiMain.dll file.
;*******************************************************************************

.586
.model flat, C
option casemap :none

.code

    ;*******************************************************************************
    ; @brief Unpacks the given POL1 section.
    ;
    ; @param packed         The packed POL1 section to unpack.
    ; @param unpacked       The unpacked buffer to write the data to.
    ;*******************************************************************************
    XiUnpack PROC
        PUSHAD
        MOV EBP, ESP
        MOV ESI, DWORD PTR SS:[EBP + 024h] ; packed section
        MOV EDI, DWORD PTR SS:[EBP + 028h] ; storage
jmp_6:
        MOV ECX, 8
        MOV BL, BYTE PTR DS:[ESI]
        INC ESI
jmp_5:
        SHL BL, 1
        JNB jmp_1
        MOV AL, BYTE PTR DS:[ESI]
        MOV BYTE PTR DS:[EDI], AL
        INC ESI
        INC EDI
        JMP jmp_2
jmp_1:
        XOR EAX, EAX
        MOV AL, BYTE PTR DS:[ESI]
        INC ESI
        MOV EDX, EAX
        MOV AL, BYTE PTR DS:[ESI]
        MOV AH, DL
        AND EAX, 0FFFh
        JE jmp_3
        INC ESI
        NEG EAX
        SHR EDX, 4
        ADD EDX, 3
jmp_4:
        MOV BH, BYTE PTR DS:[EDI + EAX]
        MOV BYTE PTR DS:[EDI], BH
        INC EDI
        DEC EDX
        JNZ jmp_4
jmp_2:
        LOOP jmp_5
        JMP jmp_6
jmp_3:
        POPAD
        RETN
    XiUnpack ENDP
END

This would allow us to call XiUnpack to unpack the POL1 section like this:

XiUnpack((DWORD)ptr_to_packed_data, (DWORD)ptr_to_output_buffer);

So the last bit would be now to load the file locally, dump the PE file information including:

  • DOS Header
  • DOS Stub (If it exists.)
  • Nt Headers
  • Section Entries
  • Section Data

Here is a step by step run down of what we would need to do. I will post the source code to this on Github as well for those interested.

  1. Load the file locally into memory.
  2. Validate the file is a PE file.
  3. Validate the file has a packed POL1 section.
  4. Obtain the POL1 section for its data and raw size.
  5. Obtain the .text section for its virtual size. (This is the size of POL1 unpacked.)
  6. Invoke XiUnpack, as seen above, to unpack the data.
  7. Begin Rebuilding The Unpacked File
    1. Write the original DOS header to the new file.
    2. Write the DOS stub, if it exists, to the new file.
    3. Write the original NT headers to the new file.
    4. Process each section of the file.
      1. If .text section, set the size of raw data to its virtual size.
      2. Set the sections raw data pointer to the previous sections end. (Realigning the sections.)
      3. Realign the sections pointer to raw data and size of raw data to the files section alignment. (Required for Windows to consider the file valid!)
      4. Write the section entry to the file.
      5. Set the file pointer to the sections raw data offset.
      6. Write the sections raw data to the file.
      7. Reset the file pointer to the section table for the next section to be written.
  8. Reset the file pointer to the NT headers offset.
  9. Adjust the files SizeOfImage inside of the NT headers with the new last section information.
  10. Rewrite the NT headers to the file with the new image size.
  11. Close the new file.

At this point our new file should be all set. The result will look something like this:
Unpacked

As you can see here, our .text section is now properly sized (and aligned) to the file. Following the raw data pointer will take us to the unpacked data from POL1 as well. Each following section has had their raw address set to their new locations following the .text’s section data. Setting the raw data pointer is as simple as (rawSize + rawAddress) of the section previous. .text starts at 0x00001000. The next section will start at (.text->RawSize + .text->RawAddress). And so on for each following section.

Something to keep in mind, the POL1 section is not required after being unpacked. We have reset the entry point of the file to its new location. This allows us to completely remove the section if we wanted. Doing so will require us to fix the .rsrc and .reloc sections to properly align in the file with the POL1 section removed. For this example and source base, I just left the section in place. It does not hurt having it there, it just takes up some extra space.

Some side notes on how certain information is handled..

Aligning The Sections
According to MSDN, ‘SizeOfRawData’ and ‘PointerToRawData’ must be section aligned. It is required to be a multiple of the files ‘FileAlignment’ value found within the NT headers optional header. This is simple to do. Because it needs to be a multiple we want to make sure we are rounding up. In order to do, we would do the following:

(((in + align - 1) / align) * align)

So as an example, our .text section in FFXiMain.dll would look like this:

PointerToRawData = (((0x400 + 0x200 - 1) / 0x200) * 0x200); // would equal: 0x400
SizeOfRawData = (((0x2F4C2E + 0x200 - 1) / 0x200) * 0x200); // would equal: 0x2F4E00

Obtaining The Entry Point
With the example code base that is on Github, we will see the following for the entry point:

        auto baseAddressOffset = *(DWORD*)(((DWORD)packed + polSection.Misc.VirtualSize) - 0x51);
        auto baseAddressOriginal = ntHeaders.OptionalHeader.AddressOfEntryPoint;
        ntHeaders.OptionalHeader.AddressOfEntryPoint = (baseAddressOffset + baseAddressOriginal) + 0x9B;

Here we are reading the original code from the packed file. The first read is the jump offset to the last jump going to the actual original entry point. Next, we obtain the original entry point then we calculate the original from the given offset. The + 0x9B is the offset to the last JMP to the entry point. So we are calculating the difference from the original entry point to the real entry point based on the jump.

Github link: https://github.com/atom0s/xiunpack

Simple Memory Scanner Example

Wrote this for someone that was asking for help on Cheat Engine’s forums. This is a very very basic and light-weight memory scanner that will scan for 4 byte values.

/** 
 * Simple Memory Scanner Example 
 * (c) 2014 atom0s [atom0s@live.com] 
 */ 

#include <Windows.h> 
#include <string> 
#include <TlHelp32.h> 

/** 
 * @brief The target process to scan within. 
 */ 
#define TARGET_NAME "winmine.exe" 

/** 
 * @brief Obtains the process id of the given target. 
 * 
 * @return The process id if found, 0 otherwise. 
 */ 
unsigned int getTargetProcessId() 
{ 
    PROCESSENTRY32 pe32 = { sizeof(PROCESSENTRY32) }; 

    // Obtain a snapshot of the current process list.. 
    auto handle = ::CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0); 
    if (handle == INVALID_HANDLE_VALUE) 
        return 0; 

    // Obtain the first process.. 
    if (!::Process32First(handle, &pe32)) 
    { 
        ::CloseHandle(handle); 
        return 0; 
    } 

    // Loop each process looking for the target.. 
    do 
    { 
        if (!_stricmp(pe32.szExeFile, TARGET_NAME)) 
        { 
            ::CloseHandle(handle); 
            return pe32.th32ProcessID; 
        } 
    } while (::Process32Next(handle, &pe32)); 

    // Cleanup.. 
    ::CloseHandle(handle); 
    return 0; 
} 

/** 
 * @brief Entry point of this application. 
 * 
 * @param argc  The count of arguments passed to this application. 
 * @param argv  The array of arguments passed to this application. 
 * 
 * @return Non-important return. 
 */ 
int __cdecl main(int argc, char* argv[]) 
{ 
    // Obtain the target process id.. 
    auto processId = getTargetProcessId(); 
    if (processId == 0) 
        return 0; 

    // Open a handle to the target.. 
    auto handle = ::OpenProcess(PROCESS_VM_OPERATION | PROCESS_VM_READ | PROCESS_QUERY_INFORMATION, FALSE, processId); 
    if (handle == INVALID_HANDLE_VALUE) 
        return 0; 

    // Obtain the current system information.. 
    SYSTEM_INFO sysInfo = { 0 }; 
    ::GetSystemInfo(&sysInfo); 

    auto addr_min = (long)sysInfo.lpMinimumApplicationAddress; 
    auto addr_max = (long)sysInfo.lpMaximumApplicationAddress; 

    auto found = 0; 

    // Loop the pages of memory of the application.. 
    while (addr_min < addr_max) 
    { 
        MEMORY_BASIC_INFORMATION mbi = { 0 }; 
        if (!::VirtualQueryEx(handle, (LPCVOID)addr_min, &mbi, sizeof(mbi))) 
        { 
            printf_s("Failed to query memory.\n"); 
            break; 
        } 

        // Determine if we have access to the page.. 
        if (mbi.State == MEM_COMMIT && ((mbi.Protect & PAGE_GUARD) == 0) && ((mbi.Protect & PAGE_NOACCESS) == 0)) 
        { 
            // 
            // Below are flags about the current region of memory. If you want to specifically scan for only 
            // certain things like if the area is writable, executable, etc. you can use these flags to prevent 
            // reading non-desired protection types. 
            // 

            auto isCopyOnWrite = ((mbi.Protect & PAGE_WRITECOPY) != 0 || (mbi.Protect & PAGE_EXECUTE_WRITECOPY) != 0); 
            auto isExecutable = ((mbi.Protect & PAGE_EXECUTE) != 0 || (mbi.Protect & PAGE_EXECUTE_READ) != 0 || (mbi.Protect & PAGE_EXECUTE_READWRITE) != 0 || (mbi.Protect & PAGE_EXECUTE_WRITECOPY) != 0); 
            auto isWritable = ((mbi.Protect & PAGE_READWRITE) != 0 || (mbi.Protect & PAGE_WRITECOPY) != 0 || (mbi.Protect & PAGE_EXECUTE_READWRITE) != 0 || (mbi.Protect & PAGE_EXECUTE_WRITECOPY) != 0); 

            // Dump the region into a memory block.. 
            auto dump = new unsigned char[mbi.RegionSize + 1]; 
            memset(dump, 0x00, mbi.RegionSize + 1); 
            if (!::ReadProcessMemory(handle, mbi.BaseAddress, dump, mbi.RegionSize, NULL)) 
            { 
                printf_s("Failed to read memory of location: %08X\n", mbi.BaseAddress); 
                break; 
            } 

            // Scan for 4 byte value of 1337.. 
            for (auto x = 0; x < mbi.RegionSize - 4; x += 4) 
            { 
                if (*(DWORD*)(dump + x) == 1337) 
                    found++; 
            } 

            // Cleanup the memory dump.. 
            delete[] dump; 
        } 

        // Step the current address by this regions size.. 
        addr_min += mbi.RegionSize; 
    } 

    printf_s("Found %d results!\n", found); 

    // Cleanup.. 
    ::CloseHandle(handle); 
    return ERROR_SUCCESS; 
}

Grim Dawn File Archive / Database Extractors

I’ve recently bought Grim Dawn after wanting an offline ARPG to play while my internet is utter shit or offline. I stumbled across Grim Dawn by accident doing some Google searches looking for something that has a similar Diablo feel to it. GD happens to be by the makers of Titan Quest, which I did enjoy somewhat when I was younger, however I never really got too in depth with playing it. But from the look of the game having that Diablo 2 look and feel, I wanted to give it a try.

Fast forward to now, I’ve been personally interested in the game data files to see how things are stored, what is where, and so on as a local lookup for information about the game, such as where monsters spawn, what they drop, and such. Which lead to the creation of the two following tools.

grimarc – Grim Dawn Archive File Extractor (.arc)
This is a simple tool to extract the .arc archive files for Grim Dawn.
https://github.com/atom0s/grimarc

grimarz – Grim Dawn Database File Extractor (.arz)
This is a simple tool to extract the .arz database files for Grim Dawn.
https://github.com/atom0s/grimarz

At this time, these tools are simply for static analysis locally, they do not offer any ability to edit, read, modify or repackage the files they have since extracted. My interest is simply in seeing the content of the files, not to modify them.