Earlier today I started trying to work on reverse engineering the rank for Batrider, so first thing I did was cobble the code ROMs together and setup the 68000 interrupt vectors, as usual. Unfortunately, I forgot that 68000 has A-line and F-line traps, and Yagawa uses them at length throughout the code for Garegga, Batrider, and Bakraid. These are software interrupts triggered whenever an instruction has the top 4 bits set to binary 1010 (A-line) or 1111 (F-line), and allow encoding 12 bits of trap code data so that you can use these interrupts for system calls.
These interrupts are actually extremely useful and cool, but IDA Pro's support for them is very underwhelming. A-line traps are supported, but to change what each code maps to, you need to modify a configuration file and re-load the CPU module, which is a huge hassle. F-line traps are completely unsupported, so you're left completely in the dark for those. They will ruin your time as well, since the 68000 module completely does not understand F-line traps, and treats them as unknown data. This is a particularly big problem since IDA Pro will refuse to build functions around unknown data.
As a solution, I wrote an IDAPython script/plugin that adds support for these software interrupts. This took me far longer than it should have, as there are very few examples of processor extensions for IDA in general, and there are practically none written in IDAPython. I figured the extension would be small enough that I shouldn't bother with writing it in C++ and figuring out that mess, so I was pretty determined to get it working in IDAPython. The lack of any good documentation on IDAPython's API is certainly no help here. So I'm releasing this module in the hopes that it helps somebody out if they attempt to write any of their own.
M68ktrap can be downloaded from https://bitbucket.org/trap15/m68ktrap-ida
Figured I might as well start dumping some information since it's pointless for me to just sorta hoard it for no reason. Probably useful for whenever I decide I'm tired of WS and don't even care to write anything more up.
Firstly, let me explain how the WonderWitch firmware files work. If you hold the B button while booting the WonderWitch, instead of proceeding to the launcher, you are dropped into what's known as Freya Monitor. This is a minimal bootstrapping program that is usable for recovery in the case that the main firmware is corrupted, and is located in a section of the Flash ROM that the various flashing functions on the WonderWitch BIOS don't support writing to. This lockout is simple and intentional; if you set the destination page to the page that this program is located, the function will just abort. If you additionally hold A, all on-screen text will be forwarded through the serial port, and you can use your keyboard to control the Freya Monitor cursor.
From Freya Monitor you can download the firmware (System) currently on the cart, upload System, as well as download/upload a program (Soft). Soft actually gets to run in essentially the same context as Freya Monitor, and it will run instead of the main launcher. The only way to restore access to the actual launcher after uploading Soft is to upload System, since the uploading procedure for Soft will overwrite the main firmware, and sets a few bytes differently to pick between the two.
Note however, both System and Soft images are not in plain text. They are encrypted, but the encryption is very very simple and is nearly symmetrical -- changing from encryption to decryption and vice-versa only requires flipping two operations. Here's a quick program written in C that can encrypt and decrypt these files (just compile the source file, no libraries necessary): http://daifukkat.su/blog/files/ww_codec.c, and here's pseudo-code for encrypting a binary (if you want to implement it yourself):
index = 0
foreach byte in buffer {
if index == 0, last_byte = 0xFF // reset last byte to 0xFF on every XMODEM sector
byte ^= last_byte // \_Swap these two operations to decrypt instead
buf = byte // /
last = byte
write(byte)
index = (index + 1) & 0x7F
}
Hopefully someone finds this information useful. Right now I'm taking a bit of a break from WonderSwan, but soon I intend to bust open the protection on WonderSwan Color (and some interesting WonderSwan original) carts. I've also been working on a definitive WonderSwan hardware documentation, which I will probably start publicly uploading soon, since it's almost in a presentable state (though nowhere near finished).
See you space cowboy...