Tilemap streamed from the SD card

Advice on general approaches or feasibility and discussions about game design

Tilemap streamed from the SD card

Postby Valden » Sun Aug 16, 2015 5:09 pm

Hello everyone !

I'm starting to work on a RPG similar to the first Final Fantasy games. One of my goal is to load the data dynamically from the SD card so I wont have to worry about the available size of the HEX file.

I managed to create a functional tilemapper that read the data (the map and the sprites) from the SD card. I wrote a converter in lua that packs the data into one .DAT file that must be put in the SD card of the Gamebuino. That allow me to have a total control of the data inside the file, and I also only have to open one single file. The converter also support maps from Tiled. Here is a small gif of a test map I have created.

Image

And now with more animation !
Image
(As always with me, the speed of the gif is not representative of the performance on the real hardware.)

Here you can see a map that is 255*255 tiles on the SD card (well most of it is only water). That represents 65 025 bytes of ram, which would be much harder to do using only the flash storage of the Gamebuino, as you would need some form of compression in order to put all the map into memory.

On the technical side, the routine takes around 12 000 miliseconds to run in the worst case scenario, leaving 38 000 miliseconds for other operations like collision testing, sound and character spirtes drawing. All the data thats need to be used (the sprites and the map) is stored inside the ram in a 512 byte buffer. The map in the ram is cropped so it only uses what need to be displayed on the screen (around 96 bytes of data). The the camera moves, the map data in the ram is shifted and data from the sd card is re-loaded.

Credits for the ASM 8*8 sprites routine goes to Sorunome (The routine does not work properly at the moment , as I have to manually clip the sprites on the bottom of the screen in order to avoid crashes from the console ).

Credits for most of the sprites goes to Oryx (http://oryxdesignlab.com/). It's from an old contest on the tigsource forums. The images aren't hosted anymore, but you can still find them on the original post thanks to the Internet wayback machine : https://web.archive.org/web/20110717060157/http://forums.tigsource.com/index.php?topic=8970.0

Here is the repository of someone want to see how it's done : https://bitbucket.org/valdenthoranar/tilemaptest/src

The project uses petit_fatfs that should be installed with your gamebuino library. If you want to try the program on your gamebuino, you'll have to put the MHQ0000.DAT file in the root directory of your SD card.
Last edited by Valden on Mon Aug 17, 2015 2:07 pm, edited 3 times in total.
User avatar
Valden
 
Posts: 31
Joined: Mon Jun 08, 2015 5:33 pm

Re: Tilemap streamed from the SD card

Postby EatMyBlitch » Sun Aug 16, 2015 5:47 pm

Wait are you making FF1 !!! oh my god start the HYPE :o
EatMyBlitch
 
Posts: 76
Joined: Tue Dec 23, 2014 4:29 pm
Location: Netherlands

Re: Tilemap streamed from the SD card

Postby jonnection » Sun Aug 16, 2015 8:47 pm

@Valden

I have been waiting for a long time for this to happen. The memory barrier of the Gamebuino begins to disappear.

We need a packing tool to allow people to put all game resources (pictures, text, sound effects, videos, song patterns) in a single binary on the SD card. And an easy-to-use API to access them. After we have that, the Pokemon games and Minecraft/Terraria games that people have dreamed of will actually be doable.

Really really great. Well done !

I intend to use your tile engine for a Jagged Alliance type game.

Image
User avatar
jonnection
 
Posts: 317
Joined: Sun May 04, 2014 8:21 pm

Re: Tilemap streamed from the SD card

Postby jonnection » Sun Aug 16, 2015 9:54 pm

Valden wrote:Here is the repository of someone want to see how it's done : https://bitbucket.org/valdenthoranar/tilemaptest/src


I looked at the code and tried to understand it to the best of my abilities. Nice work.

One thing I am wondering is if it is possible/useful to implement some sort of a double-buffering scheme in SD reading. pf_seek is the operation that takes a lot of time. If 84x48 requires 11x7 tiles of 8x8 pixels in worst case (77 bytes), your "window" that is read from the SD could be 13x9 (117 bytes, one extra row of bytes on each side). While the player is moving the visible area through 8 pixel boundary of the 1st buffer, load 2nd buffer in the background that overlaps in the detected direction of movement. In this way, the lag of seeking the data could be totally hidden from the player. I know it should be possible, because this is exactly the method used to get non-stop streaming music from the sd card.

Perhaps this is not relevant for a slow-paced game. Just a thought. In case you get "jerking" movement.

Image
User avatar
jonnection
 
Posts: 317
Joined: Sun May 04, 2014 8:21 pm

Re: Tilemap streamed from the SD card

Postby Valden » Mon Aug 17, 2015 8:43 am

I'm already doing that ;). The map buffer is 14*7 tiles. Because I can read one line of data easily, I don't need to buffer the up and down part of the displayed map. But because I can't load in one pass the left and right part of the map, I have to load one tile per frame if I don't want to experience lags. That is done via the buffer_map() function, which is called automatically if the camera move along the x axis. This function "reload" one line of data (14 bytes) per pass.

I hope my explanations are understandable :| .
User avatar
Valden
 
Posts: 31
Joined: Mon Jun 08, 2015 5:33 pm

Re: Tilemap streamed from the SD card

Postby Drakker » Mon Aug 17, 2015 10:52 am

Could it be faster to store the data twice, with the second set of data rotated 90°? This would allow you to read both vertical and horizontal directions in two passes instead. We have plenty of room on the SD card anyway, so we can afford to store the data twice.
User avatar
Drakker
 
Posts: 297
Joined: Sun Mar 30, 2014 2:54 am
Location: Québec, Canada

Re: Tilemap streamed from the SD card

Postby jonnection » Mon Aug 17, 2015 11:02 am

Drakker wrote:Could it be faster to store the data twice, with the second set of data rotated 90°? This would allow you to read both vertical and horizontal directions in two passes instead. We have plenty of room on the SD card anyway, so we can afford to store the data twice.


Not feasible with petitfatfs as it can only open 1 file at a time. SDfat lib perhaps, but that uses way more ram and flash than petitfs.

Hey Valden, could you put your lua script somewhere to try as well ? I'd like to try out TilED also, and the tmx parsers I have seen have a bit too many features.

And thanks for the buffer explanation. I was thinking that perhaps its built in already (as it was). I wasn't smart enough to understand exactly what was going on in your code just by browsing it.
User avatar
jonnection
 
Posts: 317
Joined: Sun May 04, 2014 8:21 pm

Re: Tilemap streamed from the SD card

Postby Valden » Mon Aug 17, 2015 12:07 pm

There is the source for the converter. It's very dirty, as I use the LÖVE framework (normaly used as a 2d game engine) for processing the tileset.
Keep in mind that I didn't planned to release that library, so the code is quite messy.

https://bitbucket.org/valdenthoranar/tilemapconverter/src

Here a quick guide to use the converter :
The data must be put in the 'converter/data' folder (due to a limitation in the LÖVE framework that only allows access to the game folder). There is only one tileset usable at the moment. It should be named 'test.png' (it's very work in progress :P ). The dimensions of the .png shouldn't matter, but you should know that in the current state of my lib, only the 13 first sprites will be loaded, and ultimately only 32 sprites will be allowed per screen (I'll see later if I allow more).

For map creation, you can use tiled (http://www.mapeditor.org/). You should use as tilset the 'test.png' file in the 'converter/data' folder. If you don't, a warning message will pop when trying to compile the data. Then make sure your sprite size is set to 8*8px, and that your map dimensions are under 256*256 (again, a warning should appear if you don't do so). Then, when you are done, you have to export the map in the .lua format and put it in the data folder. At the moment, you can add as many maps as you want, but only the first one will be displayed.

Ignore the music part of the converter, It's a relic from previous testes with the 'Sqwash' sound library for arduino that have been unsuccessful.

Finaly, you can run the '00_RUN_CONVERTER.BAT' file in the root folder. The final file will be in the output dir and shall be named "MHQ0000.DAT"

EDIT : I'm currently trying to make the library more usable. The lib is now an object, and it almost manages animated tiles (I have some bugs I need to fix).
User avatar
Valden
 
Posts: 31
Joined: Mon Jun 08, 2015 5:33 pm

Re: Tilemap streamed from the SD card

Postby Drakker » Mon Aug 17, 2015 1:13 pm

jonnection wrote:Not feasible with petitfatfs as it can only open 1 file at a time. SDfat lib perhaps, but that uses way more ram and flash than petitfs.


I was thinking more about one file with the data for both directions appended one after the other. That would require some seeking but surely less than one pass per line.
User avatar
Drakker
 
Posts: 297
Joined: Sun Mar 30, 2014 2:54 am
Location: Québec, Canada

Re: Tilemap streamed from the SD card

Postby Valden » Mon Aug 17, 2015 1:40 pm

I pushed some changes for he library in the git. It includes the lib that works as a object and some rudimentary 2 frames animations support.

Image
User avatar
Valden
 
Posts: 31
Joined: Mon Jun 08, 2015 5:33 pm

Next

Return to Project Guidance & Game development

Who is online

Users browsing this forum: No registered users and 28 guests

cron