Emulator 160*128 fix attempt (DISPLAY_MODE_INDEX) and quickfix

thomasvt

5 years ago

Hi,

I have spent about 30 hours of very deep diving into the sourcecode of the gamebuino-meta library and the emulator made by aoneill to fix the issue that the DISPLAY_MODE_INDEX (160x128) is not working.

I am a professional developer but not of microcontrollers, so after a long journey of debugging many things and narrowing it down step by step, I am now too stuck in the lowlevel microcontroller issues to find the next step. 

Therefore, I decided to post my findings here because I have done a lot of work and maybe a microcontroller expert can take the next steps.

It's a complicated issue, so it has a complicated story :)

The issue it al began with:

I enabled DISPLAY_MODE_INDEX. It worked but pixels were stretched in the emulator: X axis was 160px wide but pixels would be doubled vertically. And only pixels on even rows were visible.

I program in Visual Studio, I could't get the DISPLAY_MODE_INDEX (= 160x128px) working using the config-gamebuino.h (does anyone?!), so I simply did the precompiling myself: i searched for DISPLAY_MODE_INDEX in the library sources and removed all precompiler conditions in favor of DISPLAY_MODE_INDEX.

So, I started digging. a lot...

First some technical context:

Sending pixels to the screen

In DISPLAY_MODE_INDEX, the Gamebuino sends visual data per row to the screen through SPI  because it has too little RAM for a full double buffer. Sending this data is done asynchronously, which means that the processor executing the Sketch is not occupied while the pixel data is being transfered (it is done by another part of the hardware). So, for performance reasons, the graphics pipeline already starts translating the next row of pixels while waiting for the previous transfer to finish.

It does this by using 2 small row-buffers (arrays): while 1 row is being sent using buffer 1, the cpu already fills buffer 2. When transfer of buffer 1 is done, the buffers switch places and the story continues. (remember this paragraph)

How the emulator receives pixels

I include emulator source code file+lines. They may vary because i have changed a lot in the code by now.

The emulator receives the pixel SPI streams upon executing a STRB instruction (atsamd21.ts @ line +-850). It sees the address is 0x41004840 (which means its a DMAC address + CHCTRLA_OFFSET = yay, it's the pixel stream!!) and relays the data to a PeripheralWriteHandler (dmac-registers.ts @ line 40) which decodes the data into individual pixel values that are written (through another PeripheralWriteHandler) to the virtual display (ST7735.ts @ line 48) on the web page.

The issue

As explained earlier, there are 2 row buffers = 2 pointers in RAM memory that contain pixel data which are used in an alternating fashion.

So, the emulator should be alternating between buffer 1 and 2 too, but it doesn't. It sticks to buffer 1, even though the Sketch is sending buffer 2 for every odd row. 

I have debugged this extensively and it doesn't seem to be a bug in the Gamebuino library. I have logged the pointer addresses on the Sketch side and on the receiving Emulator side. The Sketch is sending alternating pointers, the Emulator only uses the first.

Emulator log:

Early on in my research, I added browser console logging from within the Sketch code by linking the Serial.print() stream to the browser's console.debug(). No more blindness :)

Arduino Sketch logs are prefixed with "COM>" (because they arrive over serial COM). The emulator process logs start with "EMU>".

01 COM> row pxs: 65535000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
02 COM> Data address = 0x20007d9c
03 EMU> srcaddr-btcnt: 0x20007d9c
04 COM> row pxs: 0843900000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
05 COM> Data address = 0x20007c54
06 EMU> srcaddr-btcnt: 0x20007d9c

line 01: the Sketch shows us the first row of pixels it is about to send (first pixel is white, the rest is black)

line 02: the Sketch shows the pointer address where the pixel data sits

line 03: the emulator shows the pointer address where it reads the data (in dmax-register.ts @ ln55) - the addresses match = correct.

line 04: the Sketch shows us the second row of pixels it is about to send (second pixel is colored, the others are black)

line 05: the Sketch shows the pointer address of that second row of pixels

line 06: the emulator shows us the incorrect pointer address. It is still using the first pointer. (it gets this pointer from RAM in dmac-registers.ts @ line 45: var srcaddr = processor.fetchWord(channelConfigAddress + 0x04);

It is a complex problem because i have seen it sometimes getting the correct address anyway (rarily though). So, it probably has to do with timing, or overwriting data in RAM in the wrong order by other instructions (?). 

NOTE: the emulator is single process, so there is no parallel SPI, it is all synchronous, so it cannot be a threading issue.

Recreating the issue and quick fixes

Recreating the issue is easy: enable the DISPLAY_MODE_INDEX, either the official way, or mine described in the first section. Then just do some display.drawPixel() calls and you'll see that pixels on odd rows are ignored and pixels on even rows are doubled vertically. Or: odd rows are omitted, and even rows are doubled:

for (auto i = 0; i < 160; i++)
{

// every 5 = white, every 2 is blue, other is brown
const auto color = i % 5 == 0 ? ColorIndex::white : i % 2 == 0 ? ColorIndex::brown :          
ColorIndex::blue;
gb.display.drawPixel(i, i, color);
}



Note that all pixels are 2 high, we don't even see the blue pixels (because they are on odd rows), and we only see half of the white pixels (again: only the even rows). (A screenshot of a succesful render is shown at the bottom of this post.)

QuickFix 1: sending rows one by one

By removing the 2 buffer alternating, there is only 1 pointer to read from, and the emulator runs perfectly at 160x128. But of course, this slows the rendering down on the actual Gamebuino.

I did not include the solution to this fix because its a bit of code. If someone wants it, I can provide it, it's quite easy and intuitif.

QuickFix 2: weird fix (probably has to do with the core reason)

(this is the easiest fix for people only working with the emulator. The actual Gamebuino will probably not work though, explained below)

The Gamebuino lib uses up to three parallel ZeroDMA channels for sending SPI streams. The Display_ST7735::sendBuffer() method prepares a channel by filling a Descriptor structure and then sends the data over that channel. Immediately after preparing that, it points the next_channel pointer to the next descriptor (so a future call to sendBuffer will use the next). And apparently, by changing that pointer, the emulator messes things up. 

Also, because the emulator is single-process anyway, it doesn't do multiple DMA channels at the same time (the Sketch may think so, but the emulator will execute one after another anyway), so we can safely comment out the next_channel assignment in file Display-ST7735.cpp to fix rendering on the emulator:

 

And then, you get the perfect result:

Blue and brown pixels alternating, and every 5 a white one.






I expect the real hardware (that does use parallel SPI) will not work with this fix, because it does uses multiple DMA channels in parallel, which we just broke by commenting out that "hop to next channel" line.

The strangest thing here is that, apparently, changing the next_desc variable to the next Descriptor AFTER doing the current stream, influences the correct reading of that current stream. It makes no sense, and i suspect the core issue to be found here.

Quick fix summary

For people who just want to fix the emulator, you can comment out the line described in QuickFix 2. That source file is part of the Gamebuino library and is normally installed here: 

You can change it, save it and rebuild your Sketch. It will recompile the changed library too.

Note that this is a poor fix, as it will break the code for the Gamebuino hardware. You could uncomment the line again when you build for the real hardware :)




Thierry

NEW 5 years ago

I'm not sure to understand every bit you wrote, but I will read again and again.

That's a huge works and publishing it there is a wonderful idea.

jicehel

NEW 5 years ago

Yes, many thanks for your works thomasvt, amazing analysis. I'm not able to help and i regreat it, some can i think. Soru will probably be able to understand and help when she will read it and some other maybe too. So if i understand well, we have to remove comment to compile a version for emulator and then put line in comments again to compile for real hardware ? (Sorry i read it at works, so i can't test now)

thomasvt

5 years ago

No, the exact opposite. Comment OUT the line for emu, and put it back for hardware. I'll rephrase the fix a bit clearer.

thomasvt

NEW 5 years ago

jicehel jicehel

No, the exact opposite. Comment OUT the line for emu, and put it back for hardware. I'll rephrase the fix a bit clearer.

jicehel

NEW 5 years ago

Thanks Thomas for answer ;) I had the idea, only the idea but it's better to do it fine at the first try  ;)

amyjack

NEW 5 years ago

I have also found many issues in the coding but I got solution of it on the best institute in bhubaneswar so you can also find your solution there.

eriban

NEW 5 years ago

Hi Thomas,

Thanks for sharing your progress. It inspired me to investigate the issue, and I think I managed.

As you already explained, DISPLAY_MODE_INDEX uses an alternating buffer for transferring pixel data to the display. For this, the Adafruit_ZeroDMA class uses a descriptor chain consisting of two descriptors. The Display_ST7735::sendBuffer method alternates between these. However, if you inspect the Gamebuino code closely, you will see that this only impacts the next invocation of Adafruit_ZeroDMA::changeDescriptor, which in turn only changes the source address in its private data structure. The Gamebuino code does not do anything with this.

So, how does it work then? As it turns out, the DMA handler should, after it handled a given transfer request, inspect the "descaddr" of the corresponding DMA descriptor. It should then takes this as the next descriptor. This is described in a bit more detail here: https://forum.arduino.cc/index.php?topic=518461.0

The emulator code, however, was ignoring the descaddr register setting, which explains why it always read from the same buffer. With some minor changes to DmacRegisters class (https://github.com/erwinbonsma/gamebuino-emulator/commit/a23072ed788df72ffc44dafc8cea2b1d2ec65975) the emulator switches buffers as needed.

For now the fix is in my fork of the emulator code, but I will initiate a Pull Request to get my changes into Andy's repository.



aoneill

NEW 5 years ago

You guys are amazing! I figured something was wrong with the DMA emulation, but I had completely given up on it after getting nowhere. I just saw eriban's  pull request. I'll reach out to the Gamebuino team to see if they want to update the emulator embedded on the site.

eriban

5 years ago

Hi Andy. I am happy to have done my small bit, but let me repeat here that we should really thank you for making the emulator in the first place. It's a very helpful tool while developing, but also for quickly checking out games made by others.

thomasvt

5 years ago

And thank you aoneill, for making the emu in the first place. During my research I have studied your code intensely, and I was quite impressed with your knowledge of how this hardware can be emulated. Nice work! And very useful.

For debugging purposes, I relayed the arduino Serial stream to the browser's output console in your emulator. If you think this is a good thing (i found it very handy :) ), I can do a pull-request. 

If the emu works again for my game, I will probably add even more debugging features.

thomasvt

5 years ago

Hey Andy,

Could you get hold of the Gamebuino team?

thanks

eriban

NEW 5 years ago

aoneill aoneill

Hi Andy. I am happy to have done my small bit, but let me repeat here that we should really thank you for making the emulator in the first place. It's a very helpful tool while developing, but also for quickly checking out games made by others.

jicehel

NEW 5 years ago

Yes you're amazing mates. Great team works. Many thanks from all Gamebuino's users even i can't talk for them all, i'm sure they will thank you for that  ;)

Steph

NEW 5 years ago

Yes! You are doing well to remind it @jicehel!
We are all very grateful for this great quality work and whose technicality is out of reach for many of us!
A big thank you to Andy, Erwin and Thomas  <3

thomasvt

NEW 5 years ago

Aah nice... i was hoping for this to happen :)

Currently, I'm working on a game that doesn't work on the emulator either because it uses the same DMA alternating design. (the game is 160x128 full color + double buffer). So, I'm happy that it will probably work with this new version! A lot easier than flashing the hardware every 2 minutes :)

thomasvt

NEW 5 years ago

aoneill aoneill

And thank you aoneill, for making the emu in the first place. During my research I have studied your code intensely, and I was quite impressed with your knowledge of how this hardware can be emulated. Nice work! And very useful.

For debugging purposes, I relayed the arduino Serial stream to the browser's output console in your emulator. If you think this is a good thing (i found it very handy :) ), I can do a pull-request. 

If the emu works again for my game, I will probably add even more debugging features.

eriban

5 years ago

Hi Thomas, I was wondering what you did to make the Serial stream available in the browser's output console. In the end I did not need it for these changes, but it could come in handy later. So I think it would be a useful addition. Let's see what Andy thinks though. If it's not too much effort, you could create the pull request already.

eriban

NEW 5 years ago

thomasvt thomasvt

Hi Thomas, I was wondering what you did to make the Serial stream available in the browser's output console. In the end I did not need it for these changes, but it could come in handy later. So I think it would be a useful addition. Let's see what Andy thinks though. If it's not too much effort, you could create the pull request already.

thomasvt

NEW 5 years ago

aoneill aoneill

Hey Andy,

Could you get hold of the Gamebuino team?

thanks