WiiPax

WiiPax
General
Author(s)	fail0verflow
Type	PC utility
Version	0.2
Links
Source

WiiPax is a tool used to obfuscate the Homebrew Channel, CEIL1NG_CAT, and the HackMii Installer. The open source version uses LZMA to compress ELF files, with a loader stub added to decompress it.

Closed source unpacking algorithm

The standalone version copies itself to MEM2, and has a total of 33 functions (3662 instructions).

Key generation

4 keys are used in the final algorithm: an IV (4 words), an index key (44 words), a value key (256 words), and a finalization value key (256 bytes).

The value key is generated by first building a two-way lookup table for a function that leftshifts a byte by one bit and XORs the result with 0x1B if the removed bit is 1. Another two-way lookup table is then built where each value is obtained by finding lookup1[255 - inverseLookup1[index]] and XORing the result with the result leftrotated 1, 2, 3, and 4 bits, as well as the constant 99; the reverse lookup table is the finalization value key. Finally, each entry in the value key is the joining of 4 bytes, each of which is found by applying a function where the first argument is the respective value in [11, 13, 9, 14], and the second argument is the entry in the finalization value key. This function performs a reverse lookup in the first table on both arguments, then adds the two results together and performs a division-like operation, before looking up the result in forward table 1.

To generate the other keys, a payload key is stored for each unpackable section. Specifically, the IV is generated as {0, &payloadKey, 0, payloadKey[8]}. An intermediate key is also generated through a duplicated loop, each of which runs 0x20 times where a constant of 0x9e3779b9 (0xc480507b ^ 0x5ab729c2) is multiplied by 0x20, and 0x9e3779b9 is subtracted during each iteration. Two state variables are maintained, initialized with values from the payload key, which are the first 2 values during the loop and the last 2 values during the second loop. Operations of the form stateB = (addrShuffle[decrementedValue >> k] + decrementedValue) ^ (stateA + ((stateA << 4) ^ (stateA >> 5))) are run, where decrementedValue is the value above and addrShuffle is an array where the key address is XORed with binary specific constants (for example, the DVDLowOpenPartitionWithTmdAndTicket exploit in the HackMii Installer uses {&payloadKey ^ 0x6403dd7a, &payloadKey ^ 0xcdce1257, &payloadKey ^ 0x5142ef7f, &payloadKey ^ 0x97b67c2c}). stateA and stateB alternate between the two state variables, and k is 11 for the first state and 0 for the second state. decrementedValue is updated between the two states within one loop iteration.

The index key is generated by creating a temporary buffer with space for 44 words where the first 4 words are an input seed key that is either generated from the above algorithms, or set to 0 to clear the index key. The remaining words are initialized with tempBuf[index] = tempBuf[index-4] ^ func(tempBuf[index-1]) ^ xorList[index], where func is a rotation 8 bits to the right for indices that are multiples of 4 or the identity for other indices, and xorList is a list of results from successive computations of the first function involved in value key computation. After this, the first 4 words of the temporary buffer are copied to the last 4 words of the index key, and the remaining words are computed through a similar division process as above, with the same 4 constants being rotated. Finally, the last 4 words in the temporary buffer are copied to the beginning of the key to prevent the seed from being discovered.

Unpacking

For each 16-byte chunk of the ciphertext, the 16 bytes are XORed with the first 16 bytes of the index key. Then, a transformation is repeated 9 times where each word is mapped to indexKey[4 + 4*transformationIteration + index] ^ valueKey[preimage[4*index+3]] ^ (valueKey[preimage[4*C]] r<< 24) ^ (valueKey[preimage[4*A+2]] r<< 8) ^ (valueKey[preimage[4*B+1]] r<< 16) (the array (A,B,C) rotates between (3,2,1), (0,3,2), (1,0,3), and (2,1,0) to reduce consistency). Finally, each word in the ciphertext is mapped to indexKey[index] ^ finalizationValueKey[preimage[4*index+3]] ^ (finalizationValueKey[preimage[4*C]] << 24) ^ (finalizationValueKey[preimage[4*A+2]] << 8) ^ (finalizationValueKey[preimage[4*C+1]] << 16).

After this algorithm is applied to individual chunks, the state is XORed with the ciphertext from the previous chunk.