Making A MESS

I loved video games growing up.  Pacman and Donkey Kong reigned supreme with the general populace, but for me it was Defender that grabbed me hardest and wouldn't let go.  I remember once having a single quarter burning a hole in my pocket.  I jumped on my 10-speed and biked the couple of miles to the nearest arcade where that classic Williams' game awaited me.  I dropped in my quarter and suddenly became the pilot of that ship.  With its frantic gameplay and excellent graphics, I could have played that game for hours.

My First Love

Unfortunately for me, the game was fiendishly difficult and all I had was one quarter and no skill.  It was only a matter of moments before my butt was back on my bike seat and I was headed home.  I got a lot of exercise that way, one quarter at a time.

Today our computers have processors easily 1000x more powerful than the Motorola 6809 running at 1 MHz at the heart of Defender.  All of that processing power has been put to work in projects like MAME (Multiple Arcade Machine Emulator) and MESS (Multi Emulator Super System) to deliver cycle-accurate emulations of the processors in these old arcade wonders.  These projects do more than just the emulation of the processors: many games have perfect sound and video emulations as well.  Dig up the original ROMs that were in these games, and you aren't playing just some reasonable facsimile of the game.  You are playing THE game.

There are many things I love about the MAME and MESS projects.

• They are intended as a preservation project rather than an emulation project.  The emulation is an accurate representation of the hardware, sans a few things like emulating transistor level stuff like programmable logic chips and a couple other exceptions.
• They are open source.  As a matter of fact, MAME and MESS joined forces a short while ago and now their code lives in a common repository.  Want to know how this stuff works?  Grab a copy, poke around, and try compiling it yourself.  It isn't hard.
• One look at their SVN Repository shows the project is very active.  There are additions and fixes going in all the time.
• The scope is enormous.  There are around one hundred processors and a similar number of sound chips emulated.  Thousands of games and old computer platforms are emulated.  Thousands.

Now let's change gears.  Something like three years ago, I bought myself a Davis weather station.  The undocumented expansion port at the back of the thing nagged at me.  With some knowledge of electronics and some helpful documentation from the manufacturer, I was first able to figure out how to interface to the weather station console with nothing more than a commonly available USB to LVTTL serial adapter.  A while later I sorted out how to build my own datalogger to store weather data for periods when the console wasn't connected to a computer.  These two projects were cheap and easy to put together.  They also faithfully duplicated the expensive equivalent sold by the manufacturer.

Other people then expanded on this work with projects of their own.  One person developed a way to connect the console directly to a WRT54G router and upload weather data without a computer in the middle.  Another developed a wireless interface with commonly available XBee modules.  One or two enterprising folk actually started selling low cost third-party designs to what could only have been a fairly limited audience of enthusiasts.  This gave the enthusiast crowd a low cost option for an interface to their console for their own data while the mainstream crowd could always buy a console interface off-the-shelf if they wanted one.  Everybody seemed happy.  Or so we thought.

Apparently, Davis is putting the hammer down.  There is a new version of the console firmware that prevents the DIY interfaces from working.  This is a pretty wrong-headed move if you think about this for a bit.  They are pissing off their biggest supporters by deliberately breaking something that would amount to a small fraction of their sales of these units.  These big supporters are the same people who recommend these systems to other people.  These supporters also buy a lot of Davis hardware in the first place.  So maybe, just maybe, Davis could lose money rather than make money by doing something like this.  Also consider that what Davis can do to differentiate their logger from a DIY version is very limited.  Efforts are already underway to see if the new logger can be reverse-engineered.  Given that the SPI interface between the logger and the console is easily sniffed, I don't expect this reverse engineering effort to take very long or be difficult to emulate.  Then we'll all be back to where we started, but with incompatible hardware versions floating around.  Brilliant.

This situation doesn't make me very happy.  Not happy at all.  In fact, it makes me kind of angry.  Angry enough to wonder if there isn't something I can do about it.

Maybe there is.  Followers of this blog will know that I've built my own receiver using a child's toy and can decode all of the weather related information transmitted from the ISS (Integrated Sensor Suite) outdoor unit.  There was still some work to do though: the ISS transmits a Frequency Hopped Spread Spectrum signal, and I was able to determine the hop sequence so I could write some code to follow its transmissions continually.  However, I only figured the hop sequence out for one of the eight possible transmitter IDs, and only for the version sold in North America.  Figuring out the hop sequence for the other IDs and the other regions would be a lot of work that I didn't feel like doing.

I then considered reverse engineering the firmware by disassembling it.  This is not a lot of laughs either.  The disassembler doesn't know code from data so you've got to tell it which is which, meaning that you need to figure that part out first.  Then you aren't able to actually run the code and get an understanding of how it works.

That made me think about emulation.  I had thought about writing my own CPU emulator for the Atmel ATMEGA128 processor running the console.  I had done this kind of thing before, actually.  I wrote an 8086 emulator in x86 assembly language for the old Retrocade arcade game emulator that brought Q-Bert to the party.  I learned a ton in doing so, but one of the biggest things I learned is that writing a CPU emulator is HARD.  Very hard.  The initial coding isn't so bad.  It is debugging it that is a killer.  A very simple error in the emulator can totally screw the emulation up, and often these errors are very difficult to find.

I didn't want to spend a ton of time writing my own emulator so I started poking around.  Atmel's own AVR Studio looked promising, but it was more suitable for code development than reverse engineering.  I also couldn't see how I'd be able to hook in functionality to the processor emulator for things like the console's keyboard.  Then it occurred to me that MAME/MESS might have a CPU emulator I could mess with.  And what do you know?  Looks like a lot of work has gone into an Atmel 8-bit CPU emulator that had potential for use in reverse engineering how the Davis code works.  I checked the SVN Repository which also showed that there were a bunch of recent fixes going into this emulator since the last official release.  And finally, I was impressed by the statement at the top of the code stating that this emulator could faithfully run code written for the "demo scene".  These demos (check it out) are written to push a platform to its absolute limits using every trick in the book.  The fact that demo code could work on this emulator spoke well of its accuracy.

Check it Out

Looking at the avr code showed me that the emulator itself ties in all kinds of MAME / MESS dependencies for loading firmware, enabling a debugger, etc.  I would have to work around all of that if I wanted to use this CPU emulator in a project of my own.  Then I realized that I was looking at things backwards.  Why consider this a project of my own?  Why not see if I could make this a part of MAME / MESS itself?  I had a week off between Christmas and New Year, so why not see how far I could get?

Click To Embiggen

What you are looking at is MESS running the actual Davis firmware in its debug mode.  The window in the upper left shows a disassembly of the code at the current program counter location as well as other good stuff like CPU register values.  The right side shows a view into the firmware, RAM, and EEPROM memory spaces.  I am now able to single step through the code and see exactly what it is doing.

Getting this far wasn't all smiles and sunshine.  MAME / MESS undergoes constant churn, so an accurate tutorial on how to add a new driver to the code base doesn't seem to exist.  And while the AVR emulator seems to be quite accurate, that doesn't necessarily mean it is complete.  Last night I had to code up the emulation of the ELPM (Extended Load Program) instruction.  That ended up being pretty straightforward based on the existing infrastructure, but it was just another hoop to jump through.  Never mind that there also seem to be some compiler bug triggered by MAME that had me doing nothing but segfault when I started out.

How far will I get with this?  We'll see.  It is certainly a powerful tool for deep-diving into the belly of the console to figure out how it works.  Working against me is that I have some stuff coming up that will severely limit how much time I can spend on this stuff.

What will I do with what I find out?  That is a topic for a future post assuming all goes well.  But it is cool.