Daifukkat.su

This post is the final part of a series. I highly recommend reading the previous articles, or this one may not make much sense: Part 1 Part 2 Part 3 Part 4.

The time has finally come for the release. This will be my final post on the NMK004 dump. As of this post, the patches have been added to MAME, and will definitely be included in the next stable release.

Let's start out with what everybody has been waiting for, the NMK004 internal ROM dump. You may obtain the dump here: http://daifukkat.su/files/nmk004.zip.

Next, I'm also releasing my dumping tools. This is split into two repositories, both hosted on bitbucket. OPNCAP is located at https://bitbucket.org/trap15/opncap, and nmk004-trojan is located at https://bitbucket.org/trap15/nmk004-trojan.

Lastly, I have a special secret extra bonus, but you'll have to put up with story time for a bit. A few months ago, I bought a Hacha Mecha Fighter PCB off of eBay. I knew it looked somewhat strange, being mostly EPROMs instead of mask ROMs, and lacking the NMK-110 protection MCU that's usually on the PCB. When I got it, I realized almost instantly that it had to have been a bootleg conversion from Thunder Dragon. After playing and comparing to PCB recordings of a genuine board, I knew that what I had was gold: an unprotected version of Hacha Mecha Fighter. I never bothered to dump it until I started on this project, but I felt now was the right time. So I dumped the program ROMs and sent them off to Haze to have them added to MAME. As expected, it works perfectly in MAME, and now it's possible for anyone to play it properly. In the future, I'd like to actually figure out the protection on Thunder Dragon and Hacha Mecha Fighter, but having unprotected versions works just as well.

You can get this ROM set from http://daifukkat.su/files/hachamfb.zip.

Title screen of Hacha Mecha Fighter

The following MAMETesters bugs may now be closed:

• http://mametesters.org/view.php?id=1117
• http://mametesters.org/view.php?id=3395
• http://mametesters.org/view.php?id=2422
• http://mametesters.org/view.php?id=2417

Last but not least, I'd like to give some shout outs to those who made this all possible:

• David "Haze" Haywood
• Charles MacDonald
• Jonathan "Lord Nightmare" Gevaryahu
• Nicola Salmoria
• austere
• #raidenii
• Beer and coffee

As a cute little aside, I've uploaded the work log I kept while doing this. Maybe somebody will find it interesting. There's a lot of content, lots of information on my thought process, a few diagrams and tables, stuff like that. Pretty unfiltered though, so don't be offended by anything I have written. http://daifukkat.su/files/NMK004-WORK-NOTES.txt

This is post is part of a series. I highly recommend reading the previous articles, or this one may not make much sense: Part 1 Part 2 Part 3.

Hold on to your hats, this post is gonna be a long one.

Title screen of GunNail

As mentioned in part 1, I revisited the NMK004 dumping project late last month, and made good quick progress. The first thing I did was attach my Saleae Logic logic analyzer to the OPN on my Thunder Dragon board. I started a song and used the logic analyzer to look at a few of the signals. Unfortunately, my model only has 8 probes, so I wasn't able to get a full picture of the interfacing activity. I managed to figure out how to detect a write to the chip, but the lack of probes meant I wouldn't be able to fully grasp the exact activity. Compounded with how many "dummy" writes were constantly being done, I knew I had to move onto writing FPGA code next.

After installing Quartus II, I had to spend a bit of time remembering VHDL's syntax, since I hadn't used it for a while. I spent about an hour writing the initial logging code and making it compile. I'm not as good with VHDL programming as other conventional languages, so I ended up having some broken logic that took me about an hour and a half to fix. Once it worked, I added a filter to make the FPGA only record PSG writes. It turns out that all upkeep pokes that the NMK004 does are exclusively on the FM registers; the PSG registers are only poked when their values need to be changed. This saved a lot of space on my FPGA's relatively small 512KB SRAM. Saving space allows for more bytes to be dumped at once, making the dumping process much faster.

From here, I started probing the possible attack vectors. I knew that the most likely targets would be the note frequency tables and the volume envelope tables for the PSG. I wrote some small testing ROMs to do a quick check, and was dismayed when I found that the volume envelope table could only directly dump 4 bits at a time, and the note frequency table could only directly dump 12 bits. While contemplating what to do, I realized that I could extract an extra 8 bits out of the volume envelope table by simply looking at the timing between writing the new volume. This still essentially only gives me an 8-bit dump per channel that skips every other byte, but it also gives me an additional 4 bits of verification. This particularly came in handy when I realized that the length value ends up being exactly the same for a $00 byte and a $01 byte. Using the 4 bits from the volume value lets me determine exactly which value it is. In addition, since we're just recording values being written to the sound chip, I can now multiplex the three PSG channels to get triple dump speed. Armed with this knowledge, I knew that this was going to be my attack vector, and exactly what I was going to do to make it work.

Title screen of Koutetsu Yousai Strahl

After carefully planning out my attack, I started writing the tools that I would need to make the dump process smooth and easy. With a few hours of work, I had code that would analyze the captured register activity and turn it into a log of activity over time. I also wrote a fair bit of code that would allow querying for when a register changes, finding the next register activity, and such similar things. It ended up being a nicely full featured suite for capture analysis. I still needed some more data from the NMK004 before I could make the tool actually reconstruct the capture into ROM data though. I needed to get the real-time length for the values in the volume envelope table. To accomplish this, I made a bogus ROM that had the length values $00,$01,$02,$04,$08,$10,$20,$40,$80,$FF. I figured this would give me a very good idea of how long each value takes, and it ended up being very spot on. It takes approximately 30250 × value capture ticks for each length value, where $00 is the same as $01. One thing that stood out during this test was that the first tone would last slightly shorter than the rest. 2500 ticks shorter, to be specific. This was an easy fix, but it could have definitely tripped me up if I didn't notice it.

As part of my attack strategy, I utilized an interesting concept in the conversion tool that I call a prioritized usage map. For each incoming nibble -- 2 nibbles for length, 1 nibble for volume register value -- I would store the channel it was captured from, and where that nibble came from. In the case of a volume register value, I gave it the highest priority, 3. For a length nibble, I would give it the lowest priority, 1. If the length byte was determined to be $01, I would use the high nibble and give it medium priority, and ignore the bottom nibble. If a priority value in the usage map is 0, it would mean that the corresponding nibble has not yet been defined for the dump. Because of this system, if two overlapping nibbles were ever different, the tool could print out a warning, and use the value of the higher priority nibble. With this system in place, I felt very confident that the dump would be accurate, or at the very least be extremely obvious when a dump error occurred.

With the data analysis part done, I needed to make the program actually save the results. I decided on making the conversion tool "stackable", in that it takes a working state binary, loads it, performs its task, then saves it back. This way, multiple capture files can combine to create a final ROM image. While I was working on adding this feature, I noticed an interesting behavior in the volume envelope table handling on the NMK004. I originally thought that each table would last forever, but it turns out that it will actually wrap every $100 bytes. I didn't think that this would matter much, as I decided to play it safe and only dump a spread of $C0 bytes per channel per song, but I'd later find that I was dead wrong. It only affected two of the later songs, which I of course hadn't done testing with, so I ignored the behavior.

Considering 3 channels and that each set needs to be run through twice, each song is able to dump approximately $120 full bytes. I performed a test on the first song, and received an all too familiar message: "All Music,Effect Software(C)1990 N M K Corporation". After seeing that message, I decided it was finally time for bed, and put off further work for later.

Title screen of Choujikuu Yousai Macross

After having spent so long on that one day, I took a break for about a week. When I came back, I decided I should start documenting all of my processes and software that I'd developed, as I knew I wanted to release them all along with the ROM itself. I also split up the project into two separate but related projects: OPNCAP and nmk004-trojan. OPNCAP consists of: opncap-fpga, the FPGA code for capturing OPN register activity; reccat, a PC tool to combine capture files; and opncap-pc, a PC library used to parse and use the data in a capture file. nmk004-trojan does what it says on the tin, and consists of: trogen, a tool to generate trojan ROMs; and opnrom, a tool that uses the opncap-pc library to convert a capture file to a ROM image. Once I'd finished splitting stuff out, I continued to spend a good amount of time doing documentation. Writing the documentation was fairly painless and easy, since all the procedures were still pretty ingrained from my marathon work. Doing most of the documentation in one go ended up being a great idea, as it let me work on the project in a different way that I wasn't burned out on, and let me recover from that burn out in a productive manner.

Eventually I felt satisfied with the state of the documentation, and thus decided that I should finally do the dump. I generated a set of ROMs and burned them to 27C512 EPROMs that I had lying around. On my first few tries, I spotted several bugs in opnrom that I had to fix before continuing. Thankfully, I knew I wouldn't have to redump from my PCB because of how simple and fool-proof the actual capturing is. It took me about an hour to fix the opnrom bugs, but once that was done I had absolutely no problems. After cleaning everything up, I proceeded to dump more of the ROM.

Unfortunately, not everything was well. There were two song IDs that caused massive havoc with my capture setup. On the 13th song, the song filled up the SRAM extremely quickly, and I lost all my capture progress from that dumping set. I realized that the cause of this peculiarity was that the section is mostly filled with very low values, causing the dump to write lots of data very quickly. Because each song is a fixed length and the previously mentioned looping behavior of the volume envelope table, it would loop the dump continuously. This combined behavior led to the SRAM filling and then overflowing within a few minutes. My solution to this problem was a bit inelegant, but worked well. When the SRAM was nearly full, I would flip a switch that would pause capturing activity. Then I would copy off the contents, wait for the mischievous track to end, then begin the next song. In this way, I managed to recover the data from the two troublesome tracks and not require any strange hacks in the actual project code.

Once everything was captured, I knew there was only one step left for the dump to be complete: conversion of all capture files into a single ROM image. I ran opnrom over all of the capture files, to produce the final nmk004.bin. The conversion program gave only a single nibble warning, which I have verified to be a non-issue. I was ecstatic. The dump was finally done, I finally had the binary.

Title screen of Acrobat Mission

Once I had the dump complete, I asked the fantastic David "Haze" Haywood for assistance in hooking up the ROM to MAME. Within a day, he had it hooked up and mostly working, though there were some parts that were hacked to work because they weren't understood. I helped out a bit, fixing the 16-bit timer behavior in the TMP90C840 core, and fixing NMI behavior. I had also noticed that the channel balance was very wrong for the newly emulated games, so I spent a while comparing the output from MAME to various PCB recordings, and eventually found values that sound very close. This change actually fixes sound for more than just the NMK004 games, it fixes almost every game in the nmk16.c driver.

There was also a mysterious issue where the emulated NMK004 would reset itself after a set amount of time, and kill all the sound. After tracing pins around my PCB, I deduced that it was attempting to reset the entire system when that happened, and provided Haze with the information necessary to hook that up. According to Haze, the suicide would be prevented if the NMK004 got an NMI within a set amount of time from the last NMI, so I then traced the NMI pin from the NMK004. It led to another ASIC on the board (NMK005), which is the GPIO controller. Eventually, it was deduced that the main CPU pokes the GPIO controller to give the NMK004 an NMI when it needs it. It seems like this setup was done to turn the NMK004 into a watchdog of sorts. Once all that had been hooked up, everything seemed to work very well.

Except for two specific games: Black Heart, by NMK; and Uchuu Senkan Gomorrah, by UPL. These two games would not work with this setup, and I was mystified for a while. Figuring that there must have been some interrupt that was needed that just wasn't firing, I looked at the IRQ implementations for Black Heart. Seeing that IRQ2 was exclusively dedicated to kicking the sound NMI, I realized that was probably the greater issue. Thanks to the wonderful NMK PCB manual located at upl-gravedigger, we were able to massively clean up and improve the accuracy of the IRQ behavior in the NMK driver. This change, much like the sound balance one, affects nearly all games in the nmk16.c driver, so look for that! Sadly, this fixed behavior still didn't fix Gomorrah. In the end, we settled for a game-specific hack that inverts the level of the NMI signal. The reason Gomorrah still wouldn't work is strange, it does kick the sound NMI occasionally, but it often kicks it just slightly too late, and gets reset before the kick happens. We believe this issue may be caused by inaccuracies in the TLCS-90 CPU emulation speed, but it's such a bizarre and difficult issue that we both felt that it wasn't worth trying to fix it at this point in time.

The last change we made is something that might be a bit controversial. Haze mentioned that MAME, as it currently stands, has sprite "wobble" -- indicative of missing sprite buffering -- and asked me if I knew if the sprites were buffered. According to the previously mentioned manual, sprite RAM is indeed DMA'd on each frame, adding 1 frame of sprite delay. However, as part of some investigation I've been doing the past year or so, I also knew that the sprite hardware was framebuffer-based, and double-buffered. This adds a second frame of sprite delay. Therefore, NMK PCBs have 2 frames of sprite delay. We have added this to the MAME driver to preserve accuracy.

The next and final post in the series will be coming very soon. The next post will include the NMK004 ROM, the release of all of my dumping tools, as well as a secret extra bonus that I think will make many people excited. I am also going to give Haze the go-ahead to submit the MAME changes once the next post is up, so everyone will be able to experience the joy that is NMK.

Title screen of Thunder Dragon

This is post is a sequel to a previous post. I highly recommend reading it before this article, or this one may not make much sense: http://daifukkat.su/blog/archives/2014/09/07/nmk004_rom_dumping_part_2/

In the time since the last post I have finished dumping, reconstructing, and verifying the NMK004 internal ROM. With the help of David "Haze" Haywood, it has been hooked up to MAME. As a result, all of the previously listed games now have perfectly working sound. With the sprite priority fixes I submitted a while ago, I think this pushes the emulation quality of almost all games on NMK's hardware up to "acceptably accurate" at the very least. Expect to see these changes come to MAME within a week or two.

In August 2013, I had submitted fixes to the sprite priority emulation on NMK hardware. Later in the month, I decided to probe the Emuversal shoutbox for opinions on my work. While on the subject, I also mentioned the NMK sound's poor state, and that I hoped it would be fixed soon. Haze replied, and told me that the NMK004 internal ROM was probably only extractable with a decap. There was also a mention that someone speculated that it would be possible to extract the internal ROM by playing it out as sound data, but that the setup would be complex and that nobody had tried to do it yet.

After a bit, I asked for and was given some TLCS-90 datasheets (the same CPU used in the NMK004) as it was assumed that's what it was. Unfortunately, the NMK004 ended up not being the same MCU model as were in any of those datasheets, but I eventually found the correct one much later. The actual model seems to be a TMP90C840, datasheet available at: http://raidenii.net/files/datasheets/cpu/tmp90c840.pdf

Title screen of Vandyke

The next day, I spent some more time probing things and poking people on the Emuversal shoutbox, trying to get a feel for the possibilities for dumping. I concluded that I should try to play out the internal ROM with one of the tables, record the outputs, then analyze the sound files to recover the internal ROM. Within a few hours I had cooked up a sample ROM that attempted to play things out from the internal ROM. I did a short test in MAME just to verify that it would make sound, and it did. From there I made a reference ROM that I could use to compare the various values. By this time I had decided on using PSG note length as my dumping table, due to its long spread and easy reconstruction method.

I had tried to ask people to make a reconstruction tool for me, but this never materialized. I figured it'd be trivial enough, so I wrote my own (extremely ghetto) tool that could analyze the resulting WAV files from recordings. My recording setup involved using an RCA->3.5mm adapter to connect the sound output of my supergun to my PC's microphone port. From there I would open Audacity and let that record until it stopped. When it would stop, I would save the output as a WAV file, perform the conversion, then compress the WAV file down to a good quality OGG and throw away the WAV file. Writing the tool for analysis took a good couple hours, though the tool ended up being pretty solid and worked very well.

After the tool was written, I started dumping the first 256-byte block of the internal ROM. This block was one of the quicker ones to dump, and took five and a half hours for just the first 256 bytes. The next day, I decoded the dump, and got my first validation that everything I was doing was correct and working: "All Music,Effect Software(C)1990 N M K Corporation". You wouldn't believe how excited I was when I saw this message, one that is likely to have never been seen by a single person in literally 20 years, in the converted dump. Unfortunately, the rest of the dump was far from as exciting. Waiting 8 hours for each 256 bytes to dump is absolutely not the best way to keep motivation high, especially when the ROM to dump is 16384 bytes long. Each block takes so long to dump because of the way I'm doing the dumping. Since I was using note lengths, a small 16-bit value would dump extremely quickly, and a high one would dump extremely slowly. For reference, a $FFFF value takes about 7 minutes to dump. Multiply that by 128, and a 256-byte block of $FFFF takes 15 hours to dump. Luckily nothing was that absurd in the blocks that I had dumped.

Eventually I gave in to the length. Sadly, looking back on it, I noticed I was very close to the end of actual data. My last set was $1200-$12FF, and the actual contents of the ROM end at approximately $1400. Everything from $1400 to $1FFF is just an $FF fill, which would have taken ages to dump if I had bothered to do so. It's been about a year since that last excursion, and I'm very glad that I went back and finally got it done. The previous attempt was a serious headache, and I knew that if I made the procedure less of a total headache, I could get it done.

I realize this hasn't been that technical of a write-up, but the lack of notes makes it not too easy for me to write about the technical side and what didn't work. What it essentially came down to, technically, was setting the Shared note length table's pointer to point into the area to dump, then having a bogus song that would play out a note of each length. I seem to recall I also used a padding note between each actual dump note, to help segment them up and make the conversion tool simpler, but I couldn't say for sure anymore.

The next post will be around in a few days, detailing the final attempt all the way down to the nitty gritty. I'm still undecided how I want to release the ROM, but I think I'll figure it out before publishing the next post.

Real quick post. I've added comments to nanoblogger, thanks to Miek Gieben. All comments are moderated, and require a name and email. Hopefully this will work out well. NMK004 stuff is very close to ready, I should have a blog post or two on that coming shortly.

This is post is a sequel to a previous post: http://daifukkat.su/blog/archives/2014/09/05/nmk004_rom_dumping_progress/

An interstitiary post now. I realize that I absolutely blew right through any background information and as a result that last post was probably gibberish to most people. A bit of explanation is in order then. Before continuing, I'd like to give thanks to all of the people who worked on the NMK004 simulation in MAME. Though this work could have been done without it, it definitely would have taken much much longer. I owe a great amount of gratitude to those contributors, especially Nicola Salmoria.

Title screen of Black Heart

First off, this short project's goal is to dump the internal ROM of a protected sound CPU used on many arcade PCBs made by NMK. This sound CPU is labeled NMK004, and is a TLCS-90 CPU -- essentially a Z80 on 16-bit steroids -- with an internal code ROM, and some other stuff that doesn't matter much. The internal code ROM has never been dumped, and as a result, any attempt to emulate these games has resulted in either poor sound emulation due to incomplete/wrong simulation of the behavior of the NMK004, or none due to not bothering to simulate the chip at all.

A quick list of games that utilize this sound CPU:

• USAAF Mustang
• Uchuu Senkan Gomorrah (Bio-ship Paladin)
• Vandyke
• Black Heart
• Acrobat Mission
• Koutetsu Yousai Strahl
• Thunder Dragon
• Hacha Mecha Fighter
• Choujikuu Yousai Macross
• GunNail

These games all use the same sound setup: two OKI MSM6295s (simple 4 channel ADPCM playback chips) and one Yamaha YM2203 (3 channel FM sound chip with 3 channel PSG built in, first entry in the OPN sound chip family). They also have an external EPROM used to hold the song data for each game. Different games have completely different contents here, because they have different music. An important note, and the very feature that allows my hack to work, is that the ROM's data layout is flexible. There is a header (well, footer) located at 0xEFE0 in the external ROM. This header contains pointers to a bunch of tables, containing information for music playback. These pointers point at direct addresses, so they are not transformed at all before being used.

This alone wouldn't have been enough. It would have been possible to prevent this attack in a few ways, but there didn't seem to be any attempts to secure the MCU's code. The external ROM is on the same address space as the internal ROM. The external ROM is also in an "overlay" configuration, meaning that when it's being accessed, the address is not transformed before being sent to the device. This means that I can set the pointers to point into the internal ROM instead of the external ROM with absolutely no problem. This simple attack could have been defeated if they had merely checked the bounds of the pointer before using it, but they must have felt content with just having locked down internal ROM. Admittedly, their solution worked quite well, since as far as I'm aware, there were no bootlegs of the hardware that didn't just replace the sound section wholesale or weren't just conversions of other games.

I said last post that I'd give a breakdown of the functioning of my hacks in this post, but I don't think it's time for a full breakdown yet. Instead, here's a breakdown of the contents of that external ROM, the vulnerabilities that it affords, and the pros and cons of each table to be used for an attack. Most of this information comes directly from my notes, or has been slightly modified to make it more understandable to an outsider.

Title screen of USAAF Mustang

• Sample tables for the MSM6295s.
  • Not very usable...
• Modulation tables for the OPN's FM.
  • Can dump 8 bits a time (16 bits if careful).
  • Tricky to use.
  • Has a HUGE range of entries each ROM.
  • Can be accidentally interrupted by specific values...
  • Probably not usable...
• Note frequency table for the OPN's FM.
  • Dumps 16 bits at a time.
  • Only has a range of 16 entries each ROM.
• Note frequency table for the OPN's PSG.
  • Dumps 12 bits at a time, needs two passes to get both bytes.
  • Only has a range of 16 entries each ROM.
• Volume envelope tables for the OPN's PSG.
  • Dumps 4bits (volume value) + 8bits (volume length) at a time.
  • Can extract length from the data...
    • Length 0 and 1 are the same, but we can work around this.
  • Has a large range of entries per ROM I think.
  • Might take a bit, but being 8bit lengths, shouldn't be too crazy.
• Shared note length table for the MSM6295s, and OPN's FM and PSG.
  • Dumps 16 bits at a time.
  • Has a range of 128 entries each ROM.
  • Can take FOREVER.
• Two sub-timer tables of mostly unknown usage.
  • Unknown what these really do, so not too helpful.
• Command tables for controlling the actual sequencing.
  • Obviously no good.

I think that should cover everything that I missed, and hopefully explains most of what needs to be explained. The next post should have more in-depth information on the trojan, and maybe the dump. We'll see.

Current dumping setup

The previous time that I had worked on this project, I had written up a huge WIP post, but that file no longer exists due to my recent loss of computers so I'll write something totally new. This is actually my second attempt at working on dumping the NMK004's internal ROM; I had worked on doing this dump about nine months ago. My previous effort was actually successful and worked very well, but had one major flaw: it took forever to dump. As in, 9 hours for 128 bytes. Yeah.

After dumping a bit over half of the ROM, I ended up moving my setup around for some reason (I can't recall exactly why) and didn't want to recalibrate all of my tools and set it all back up, so I just stopped bothering. After a while, Lord Nightmare gave me a suggestion to try a different method and recommended some SMD clips to me. I bought some not long after, but never got back to dumping it. I'd go on to use these clips for a while for miscellaneous projects, but didn't really want to try revisiting the NMK004 stuff for various reasons.

Fast forward to 22 August 2014. I've moved back home and am unemployed. What am I going to do now? Why not revisit an old project that still needs to be dealt with instead of being terminally bored! This seemed a great idea, so I spent a good 26 nearly-straight hours working on writing VHDL code and crafting trojan ROMs to spew out the internal ROM to the OPN's PSG registers in a peculiar way. Once I had it theoretically working, I tried dumping the first sector of the internal ROM, as I knew that it had a very distinguishable string there from the last dump attempt. To my surprise, it worked with very little debugging.

First dump

My development process was gradual and built up all the elements over time, making bug elimination a short hiccup through the steps, rather than doing large amounts of debugging at the end. I think this was essential in keeping my motivation high. After getting the sample dump working, I decided I was quite tired and needed sleep, so I postponed further efforts to another day. As I usually do, I ended up not getting back to the project for another several weeks. Typical me.

Now it's 5 September, and my motivation has been renewed. I resumed work yesterday, starting with documenting all of my tools and my hardware setup in the case that I actually have to tear my setup down again, it'll be easy to put it back together. It'll also allow others to dump their own PCBs, though I doubt this will be necessary. This is the last piece of the documentation for a while, so I will start performing the actual dump soon enough. The new method dumps quite quickly (a few minutes for about 192 bytes), so I don't believe it will take too long to actually do the dumping.

There is still a good amount of work I want to put into the code before I release them publicly. I'm not sure how I want to release the actual dumped ROM yet, or what I want to do with it, but it shouldn't be too much longer until emulation of NMK sound should no longer be an issue. It's sad that it's taken this long, and that I of all people had to do it. I don't think this solution is very difficult to have come up with or done, so I'm disappointed that it hadn't already been done. Details of how this dumping method works, and how my previous one works will be given in my next post on this subject, which will probably be around the time that the dump is released.