A free Gamebuino if you compile the SD card bootloader !

Libraries, utilities, bootloaders...

Re: A free Gamebuino if you compile the SD card bootloader

Postby rodot » Sat Mar 15, 2014 3:01 pm

The LOADER is now working (except when you have hundreds of files and nested folders). There is still a lot work to be done, but now it's enough for me to be able to make a video and show you the magnificence of this bootloader \o/ (I'll do it next week)
Well done Myndale, your bootloader is pure magic :D

PS: I wait to receive the laser cut acrylic enclosure to send you your Gamebuino.
User avatar
rodot
Site Admin
 
Posts: 1290
Joined: Mon Nov 19, 2012 11:54 pm
Location: France

Re: A free Gamebuino if you compile the SD card bootloader

Postby Myndale » Sat Mar 15, 2014 8:22 pm

Awesome, thanks! I've signed up for one of the Indiegogo units as well so I'll have one for playing and one for hacking :)

My next task will probably be a demo showing how to do code hot-swapping.
Myndale
 
Posts: 507
Joined: Sat Mar 01, 2014 1:25 am

Re: A free Gamebuino if you compile the SD card bootloader

Postby Myndale » Wed Mar 26, 2014 2:52 am

I've been putting more work into the boot loader this week and I thought I should post a quick update regarding some changes. I'll put a summary at the end, so feel free to skip over all the technical stuff.

The original plan for self-patching games was to put the patch code/data in a HEX file on the SD card and use the existing game-loading code to flash it to the chip. After looking into it more closely I don't believe this is the best way to achieve this. Under normal circumstances a boot loader transfers control to the application and doesn't get control back until the next reset. In the case of Gamebuino the application can transfer control back to the boot loader to load a new game, but again it's a one-way process. Game patching involves a new scenario in which the game needs to transfer control to the boot loader but then the boot loader needs to return control back when it's done, and in the absence of an underlying OS we immediately run into the problem of memory management. Both applications have sizable memory requirements and each doesn't know how the other is managing RAM.

One way around this is for the calling application to pass the boot loader a "scratch pad" buffer to use during patching, but the size of this buffer needs to be 539 bytes exactly, or about a quarter of total available RAM. It also means that any future boot loaders cannot exceed this 539 requirement (or any other we set) for any reason whatsoever without breaking compatibility with existing games that use the game-patch feature.

To get around this I've implemented this feature as a function that simply allows the user to pass in individual 128-byte pages and the boot loader will flash it into memory for you. If you want to load code and/or data off SD card then that's fine, you just use the full SD library in your app to load and flash one page at a time. It also means the files can be in sub-folders, which the boot loader can't handle. If you only need to flash a portion of a page then that's fine too, just buffer it in RAM, make your changes and re-flash the whole thing. This ability to flash partial pages from RAM opens the door to one very nice feature indeed: self-modifying code!

It also comes with another unexpected benefit. A few days ago I posted about the possibility of storing user settings in Flash memory i.e. default user name, sound settings etc. With software flash functionality we get this feature automatically without the need for any additional boot loader code. We simply set aside a single 128-byte page in flash memory somewhere and the game library can use it however it wants.

Which leads me nicely to the final issue: if this feature were to be added then it would have to go at the end of the application space rather than inside the boot loader area. First of all we're not going to have much space in the boot loader. More seriously, Gamebuino would have to be shipped with the lock flags set to allow the boot loader to be overwritten. If the user then tries to load an application via USB that spills into boot loader memory then they will brick their Gamebuino, at least until they can get a hardware programmer on it to fix it.

Summary:
  • Game patching can currently be done by calling a function in the boot loader to flash pages
  • Self-modifying code is now possible
  • A page for user settings is possible e.g. for allowing customization of the game loader screen, but it would have to reside at the top of app space. If the user uploads a large sketch they'll overwrite it, although this could be easily detected by the library code.

Current status: all of this is implemented and working well but the boot code is currently 32 assembly instructions too long. I still haven't done any size optimization though so I'm very confident of hitting the 2KB target over the next few days.

Feedback from everyone welcome as always. Rodot, I'm going to need feedback from you regarding the user settings page because if it's something you want to officially sanction then I'll need to put code in the game loader to prevent users overwriting that page...it won't stop them trashing it themselves when programming via USB of course but at least it's something. Either way, as soon as I get the green light from you I'll start wrapping things up ready for release.
Myndale
 
Posts: 507
Joined: Sat Mar 01, 2014 1:25 am

Re: A free Gamebuino if you compile the SD card bootloader

Postby rodot » Wed Mar 26, 2014 7:26 am

Sounds good, thank you for this complete explanation :)
One of my concerns is that the SD card library takes a lot of RAM and ROM (btw I have some problems with the loader getting unstable because of that), and you'll need to #include it to be able to read the page you want to send to the bootloader... but as the bootloader is only compatible with FAT16, I guess we could use a FAT16-only library which would take less space ?
Thank you for you hard work!
User avatar
rodot
Site Admin
 
Posts: 1290
Joined: Mon Nov 19, 2012 11:54 pm
Location: France

Re: A free Gamebuino if you compile the SD card bootloader

Postby Myndale » Thu Mar 27, 2014 4:31 am

Happy to report the final version of the boot loader is built and running. C button still works, load_game still works, page flashing now works and there's 128 bytes at the end of the application space (i.e. address 0x7780 - 0x77ff) reserved for whatever rodot wants to use it for. My concerns about users overwriting the user settings page during USB programming were unfounded, the IDE's boards.txt file actually lets you specify the maximum sketch size so I've updated that accordingly. Best of all it now fits in 2K :)

I'll do a little more clean up and upload the final release version to the wiki over the next day or so along with a few demos. Beta version is attached to this post, don't forget to updates your boards.txt file as explained in README.TXT. The #define for the load_game function has changed now as well so be sure to use that one instead from now on, it won't change again in future.
Attachments
gamebuino_boot_beta.zip
(33.43 KiB) Downloaded 685 times
Myndale
 
Posts: 507
Joined: Sat Mar 01, 2014 1:25 am

Re: A free Gamebuino if you compile the SD card bootloader

Postby rodot » Thu Mar 27, 2014 6:05 am

Absolutely Marvelous. Pure Awesomeness. Thank you Myndale !
User avatar
rodot
Site Admin
 
Posts: 1290
Joined: Mon Nov 19, 2012 11:54 pm
Location: France

Re: A free Gamebuino if you compile the SD card bootloader

Postby rodot » Tue Apr 01, 2014 12:52 pm

It seems that your last version doesn't work (I triple checked). I'll stay with the r_4 (which works like a charm) until you check that you uploaded the right files. Thanks!
User avatar
rodot
Site Admin
 
Posts: 1290
Joined: Mon Nov 19, 2012 11:54 pm
Location: France

Re: A free Gamebuino if you compile the SD card bootloader

Postby Myndale » Thu Apr 03, 2014 4:45 am

I've tested and re-tested and triple-tested that code and it's worked fine on my end so the most likely explanation is that the stack in your loader application is extending down into the area of RAM that the game loader is trashing when it does its work. I'm copying the file name onto the stack to avoid this exact memory corruption from happening but if there's a lot of stuff on the stack then it's going to happen anyway.

Not to worry, it's an easy fix on my end. I've modified the boot loader to store the file name in the 9 bytes at the very top of RAM, and I managed to squeeze back an extra 56 bytes or so of Flash memory in the process :D New version is attached to this post, I believe this one should work for you but if you do have trouble then I might have to get you to send me the latest version of your loader so I can figure out what's going on.
Attachments
gamebuino_boot_r5.zip
(34.94 KiB) Downloaded 690 times
Myndale
 
Posts: 507
Joined: Sat Mar 01, 2014 1:25 am

Re: A free Gamebuino if you compile the SD card bootloader

Postby rodot » Thu Apr 03, 2014 7:42 am

Thanks, I will try that this afternoon. You can find the latest version of the loader on GitHub.
Edit: USB and SD parts of the bootloader r5 are working. I'll try the auto-programming feature asap
User avatar
rodot
Site Admin
 
Posts: 1290
Joined: Mon Nov 19, 2012 11:54 pm
Location: France

Re: A free Gamebuino if you compile the SD card bootloader

Postby Myndale » Thu Apr 03, 2014 11:48 pm

Ok hopefully this will be the last version. The load_game function in this version disables all interrupts while it's doing its work and also resets the stack to the upper end of SRAM instead of using the calling application's stack (which may overlap lower work areas). This should result in a much more stable boot loader that is more forgiving of cases where the calling application is pushing the limits of available memory and also won't crash if the user calls it with timer interrupts etc enabled.

Rodot if you're happy with this then let me know and I'll start looking into conditional compilation flags in the library.
Attachments
gamebuino_boot_r6.zip
(35.14 KiB) Downloaded 765 times
Myndale
 
Posts: 507
Joined: Sat Mar 01, 2014 1:25 am

PreviousNext

Return to Software Development

Who is online

Users browsing this forum: No registered users and 27 guests

cron