Guide: How to store data in Java

Imagine creating your own application or game. When you first start out, you may keep and generate your data in code. As you progress you may find how inefficient and cumbersome this turns out to be and perhaps contemplate moving all that to external files.

How do you store the growing amount of application data? What formats do you use to store them?

There are endless methods to store and retrieve data for your project. Before reinventing the wheel by creating your own parser and file format (which is an enormous effort), one must contemplate what requirements your format needs to fulfill.

For example: Does my data need to be...

• ...flexible and extendable?
• ...compatible with different versions?
• ...easily editable?
• ...easily maintainable?
• ...as compact as possible?

In Java, multiple ways of storing data for your projects exist. I want to explore some of the technology that exist to do that. A summary can be found below. Click on the format to jump to the details.

Format	Syntax	Legible	Complexity of saved Data	Extendability	Libraries		Potential Uses
CSV	Simple	Yes	Plain	Changes must be adopted manually	none		- export or import data across programs
.ini Format	Simple	Yes	Tree with depth 1	Changes must be adopted manually	ini4j		Configuration files
XML	Bloated	Yes	Tree	Changes in classes can be ignored (data won't become incompatible)	JDOM JBAX		- Configuration files - Game data
JSON	Clear	Yes	Tree	Changes in classes can be ignored (data won't become incompatible)	json-simple		- Configuration files - Game data
Binary	None	No	Tree	JDK: Changes in classes may cause saved data to become unusable Kryo: Changes in classes can be logged, preserving compability	JDK, Kryo		- Networking - Game data - Temporary data - Compressed binary files

CSV

 

Comma Separated Values. This is probably the most basic format you can use (it's also one of the oldest, being around before personal computers even existed). CSV files can be easily written programmatically. No additional parsing needed. If you need to keep it as simple as possible (e.g. you only need to store a lot of Strings) you're good with CSV. It has the advantage of being editable with programs like Excel or OpenOffice, which can be very useful tools. However, the second your data uses a tree-like structure (objects in objects) you get dangerously close to creating your own parser, eventually ending up reinventing the wheel. Don't go there ever!
CSV extremely handy for tasks like storing configs or as export format - I've seen it being used for translation files - but other than that... try to stay away from it.

Back to table

Windows .ini file

 

Before starting I want to point out that this is the only file format I haven't gotten into much detail, but want to mention it regardless.
The .ini file format is a rather ancient remnant of the past, being in use ever since Windows XP and earlier. It strikes out due to its simplicity. Because .ini files allow you to pack your name-value pairs into sections or groups, they can be very handy even to this day. Another plus: you can read them and edit them with any text editor. These kind of files still won't allow you to store an object in an object, but are easier to use than CSV. They make great configuration files - as long as you keep the data you want to store simple. Keep in mind it's likely unsuited to store larger amounts of data. You're welcome to try it out and get the ini4j library implementation for Java.

Back to table

XML

 

Due to its nature, XML is a format that supports storing of objects in other objects. Its syntax is rather bloated which results in a rather large file size. XML is readable and easily editable, and that is why it is commonly used in a variety of applications, like storing configs and also more complex data itself for games and software alike. It is supported by many frameworks. For this reason, it is also being used to export files across different programs.
When I dove into XML I was using a custom JDOM serializer. That pretty much ended up being a nightmare because I was reinventing the wheel! Since you should probably not do the same mistake, you can use Serializer like JAXB to turn your data into XML files with greater ease.

Back to table
 

JSON

 

JSON is an acronym for "Javascript Object Notation". It is a readable format that supports complex object trees. Unlike XML it's syntax is compact, so the file size is small in comparison. It also comes with a large support across different platforms and is a very flexible format to store your data in, for configs and complex data alike. This is due to the fact some JSON serializers can be configured to ignore data in the file if it could not be found in the class it is attempting to serialize. If the serializer can't find the object's field in the file, it simply leaves it to the state you declared or initialized it in the class. This makes it a very flexible and pleasant to use format, since it works without making your data itself incompatible. Some game frameworks or engines (like LibGdx) also ship with a JSON Serializer. If you're not working with such a framework, you can use json-simple (version 1.1.1).

Back to table

Binary

 

Another method to store your data is to simply store it in binary, as 1's and 0's. The upside is, your data won't be readable by anyone (well, partially). The downside is, you always have to call your binary serializer from code to store your data.
When working with Java standard serialization, you will encounter problems deserializing your objects from a file once you've changed the class. The code will terminate telling you the object cannot be serialized (because you changed, added or removed a constructor, methods or field). This can be disastrous if you are recklessly making changes to a class, just to find out you just made hundreds of bytes of binary data incompatible. Good luck redoing all of this!

To avoid this, there are libraries like Kryo that allows you to 1) create customized serializers for each class and 2) add version control for each field you are storing. Point 2) may give you some control over adding compability, but you will be left with unused code fragments you can't remove. If you do remove them, Kryo will kry that an old, deprecated field in your code is missing (excuse the pun). This allows you for some control but is still not as flexible as JSON, I've found.

However, there are some neat things you can do with binary files, for example controlled binary serialization (only store the bits of data you need using your own definition), creating your own compressed data formats, and much much more. One important thing about binary is the fact it's used to transmit data over a network.

Back to table

Summary

The most flexible format I have used for saving large amounts of data that is easy to use and maintain is without doubt JSON, followed by controlled binary serialization (Kryo). With a decent serializer, XML can also be very powerful despite of its large file size and bloated syntax. CSV and .ini are rather simple formats and very handy for simple config files that don't change frequently during development. Which of these formats you will use eventually and for what purpose - you decide!

How To Make Your Game Articles More Interesting

Browsing IndieDB and a few other games sites I have noticed many indie dev folks write very dry and boring articles about their games and development, often bloated with technical details (to be fair, it happens to me too sometimes). Being a dev myself, that doesn't bother me, but most readers will click these articles away after just a few paragraphs or even sentences. I think that's a missed opportunity to grow an audience right there.

So how can you possibly make your articles more attractive? 

How do you write an interesting and compelling article?

How do you make people stick around with you longer?

Having read and observed quite a bit about this topic, I have compiled a list that may help achieving this.

• Add all kinds of multimedia content (gifs/videos, pictures) to your article.
  If you blog about games, your pictures and gifs you post doesn't necessarily need to be about games. E.g. when talking about making music, you might write: "Don't be like a keyboard cat", followed by the appropriate cat gif and by a set of do's and don't's. You probably notice I've done this above.
  
  
  
• To allow people to quickly skim over your article, divide it into small sections with headlines.
  Some people do not want to bother reading everything, just the parts that seem interesting to them (I'm like that, too). You can keep your sections as short as possible by using pictures to split up large texts into smaller chunks that are easier to read. 


• Use an interesting format that goes along with your content.
  Important statements in bold or as headlines, whitespace and images to create reading pauses, allowing the reader to contemplate what was read. Each time you're writing an article, imagine you're orchestrating a little movie into your reader's heads.
  
  
• Use a writing style that includes humor, wit and lyrical means.
  For example, make use of metaphors, aliteration, hyperbole, rhetoric... How about adding a witty headline? The possibilities are endless. Feel free to play around and develop your own style.
  
  
• Put your own stamp on it, keep it personal and engaging!
  Add an interesting story if possible. Tell a tale or anecdote serving as a transition to what you are trying to tell and sell. Tell us how this one bug you discovered yesterday was turned into a feature!
  
  
• Less is more, keep it short.
  Don't bloat your article with too much technical details. Avoid text walls!Make your audience (including the not so tech-savvy ones) interested for more, so he or she comes back to you. You can, of course, e.g. briefly explain how your game mechanics work, but I suggest creating a separated, more technical article intended for fellow devs and technically interested people.
  
  
• Spread your content. Instead of trying to put all of your material into one post, it's often better to make two posts. Firstly, you can market those posts at different times (more people will see it) and also have more articles to spread around on your social networks. Secondly, by disclosing a bit less information readers are likelier to come back to you for more.

Feel free to add your own suggestion in the comments! Feedback is welcome and appreciated!

CamoTactics: Editor, Map Previewer, Skill System

 This is a more technical article discussing the recent features I implemented in the last three months.

I'm happy to be back, developing this game. So let's talk about some of the recent things I have been working on: editor implementation, database changes, improvements to the skill system, and other technical things, which I explain in more detail.

Editor Implementation: Create your own data!

I've been working on an editor implementation, that currently works as a proof of concept. Some objects are a pain to edit manually and having a visual and idiot-proof interface to do that is nice to have.

It's capable of fully automatically creating a display component using properties information defining how a class should be displayed (like a Weapon in the picture to the left). For example, hit points are being displayed as spinner component with a range limit from zero to infinity: that means you cannot enter values below zero. For defining projectile IDs (that specify the caliber of a weapon), an ID editor helps choosing one valid projectile by getting all projectiles currently existing in the database.

The method of fully automatically generating a display component like this can also be used to display objects in the GUI, due to its modular nature and compability between frameworks. Spares me to create info screens manually!

Database Improvements: I don't care what framework you're on!

All data is now saved in binary or JSON files. The database managing sprite sheets is now compatible across different frameworks (libgdx as used by the game, java swing as used by the editor). The game will know what type of images are needed and loads the correct image classes. What files (that means, image, binaries or JSON files) should be loaded is freely configurable, too, which will be a paradise for modders.

I am also working on controller bindings so I can implement gamepad support and changable controls.

Skill System: All your perks are belong to me!

The skill system got a redo as well, going from a purely mathematical/formula approach to a more "perk-based" or percentual approach. A test setup of this you can see below. The output consist of total experience, the current experience in this level, followed by a list of skills or properties.

The plan is to make it possible to train skills by using them and degrade them if you don't. The effectivity of all skills improve as you level up.


Let's take the first property as example:

"Hitpoints [100]: 142 - 21%" 

It's basically interpreted as:

Attribute [BaseValue]: CurrentValue - Skill%

Hit points are affected by a skill called "Hard as a Rock", which is 21%. So if you are Lieutnant rank with 21% of this skill (which isn't much at all), you will have ~142 hit points available. The BaseValue is multiplied with Skill% and then calculated together with the rank of your character. If you level it up enough by taking damage and killing enemies in melee, your hit points can increase up to 500 and more.

Map Previewer: Hello world!

A while ago I created small, tile-based engine in Java Swing out of boredom. Turns out it can be used to preview and plan out CampTactics maps!

 

It is very handy for two reasons:

1. It uses the same tile scale, so it can be directly compared with a CamoTactics map, and
2. Prototype maps are fast and easily created using a simple image editor (like MS Paint). That has its advantages, since creating prototype maps with the map editor Tiled gets tedious and time consuming after a while.

Like this, I just set up a so called color key that maps one color to a texure. Who would have thought this little engine would be so useful! :)


Next week, I'm going to work on the visual stuff a bit more and (if I have something to show) make a lot of screenshots!

Status Report #7: CamoTactics is Back! - Weapon Mods, Unit Graphics, Interface

The silence around CamoTactics was mainly caused by a switch to a much better framework (March-June), followed by profound real life changes (August-September). However, since end of October, I fortunately picked up development again.

A lot has changed during that time.

The necessity of changing frameworks arose after I figured out that the input polling system was going frenzy with updates once too many bots were on the map. In fact, it didn't make any sense at all since, what does AI have to do with the fact you're pressing a key on your mouse or keyboard...? Nothing at all. What a baffling bug that was.

So that ate up much of the performance that was needed elsewhere, e.g. rendering, making the game very slow and resource-demanding. The error likely was somewhere deep, deep down in the library I was using and therefore not in my power or knowledge to fix. Restrictions imposed by the libraries' way of handling things were another issue I had.

Thus, instead of having to deal with all these problems (which likely will magnify as the game's development progresses), I decided to switch frameworks instead:

In retrospect, that decision is probably the best thing that could have ever happened to CamoTactics.

I managed to improve the overall visual appearance, for example:

• canon turrets rotating around the correct center (no more funny looking tanks!)
  

• bullets spawning at the correct canon position (no more hitting yourself with bullets!)
  

• canons finally being positioned correctly (no more canons wobbling around next to your vehicle!)
  
• smooth camera movement
  
• inventories with better functionality (no more sluggish drag 'n' drop!)
  

As you can probably tell, there is a first implementation for language support, too.

• inventory icons grow in size when hovering over them (more interactivity!)
  
• better graphics for tanks and soldiers (more eyecandy, just in time for Christmas!)
  

And yes, those are in fact different uniforms and skin colors here.

Uniforms will eventually also affect how your avatar in-game looks, too:




...and much much more, as you can probably tell! All this goes along with a variety of things I could not do before, e.g. adding glow effects, mobile port... the list goes on!

But not only visual appearance has improved.

Since libgdx makes use of your GPU, performance also made quite a leap, see the table below.



The values are approximations, using my desktop, with approximately the same circumstances. Those are enormous differences... especially considering the advantages! It's like trading in your crappy Fiat for a fast and comfortable Mercedes - which is pretty neat!

Migrating the game to the new framework also made me realize how crappy the old one actually was. If you ever get into gamedev: spare yourself a lot of work and choose your tools wisely. Means: don't drive a Fiat if you can have a Mercedes.

Despite of not being able to run CamoTactics from the period from September till December, I was able to do some important changes to the source code that were long overdue (I was just not able to test it). But I want to discuss that and a possible next release in next week's article.

Full development can resume now, as I just got new awesome hardware!

(I can play Fallout 4 on this!)

Look forward to more regular updates! Thanks for staying with me and following CamoTactics!

Stay tuned!

CamoTactics Update: Improved Sound, UI, Plants, Chests, and more!

I made good progress on CamoTactics. I settled with a decent UI style, improved sound (you'll love this), created a ready-for-action mission system, including general polish, amongst more. The next release is planned towards end of December or start of January next year. Till then, you can download version A1B5 build here:

Sound

Sound is finally played directionally or in stereo. Different sounds are used depending on your distance to the sound played. In other words: an explosion in the distance has a more damp or dull sound as one played up close. This can be extended to all sounds, but it'd make only sense for really loud ones (gun shots/explosions). The result of this however is really amazing and has enhanced the gaming experience  A LOT.

Mission System

The mission system I prototyped before A1B4 release has been finished to a state it can be used to create mission based gameplay. As far as I'm concerned, I should be able to do most if not all functions a decent mission requires. For example, I can make enemies spawn if certain conditions are met, let a dialog pop up, start a new mission, kill off one entity, add HP to one entity, and so on. The player receives reward upon completion, and you can also fail your mission (even if you dont know you did - this still needs decent UI feedback).

 

AI Improvements & Other Additions

I've experimented with AI, improving targeting by taking sensors into account, and adding a simple zombie bot. That zombie bot is going to be the base of all AI I'm going to make , and I've prepared a solution for pathfinding already. AI is still a huge deficit in this game. To the left you see a squad of enemy soldiers being spawned once you enter a certain area (There's a missile in white shot from an friendly AI bot about to hit them and kill them all).

Other things that were added are chests, death avatars, plants that grow and are harvestable, (still buggy) melee weapons and fire modes, customizable weapons (you can use that already), a prototype skill system and a new HUD. As for general polish, the screen is shaking now if the player is close to an explosion and improvements to UI components have been made. I've also implemented mouse control to the game. You can aim and use most components, including the inventory, with it

Current Progress

I'm currently in the process of reimplementing visual effects such as explosions, muzzle flash, and other particles. Doing this I've noticed many flaws and unnecessary complexity in the visual controllers for entities. I'm in the middle of implementing a complete new, much more simple and flexible system. It allows me to do some neat things in the future as well.


Despite of some delays the game is getting forward and I will keep pushing it forward. Considering the progress, I won't be able to finish this game with complete story for the planned release in February. But I think it is realistic to say that till then, it's going to be out of alpha stage with one completed game mode (likely tower defense/zombie mode) for sure. There are still a lot of things to do in terms of AI, doing missions/levels and the story itself.

Until February next year, I expect to have a decent AI, most of the planned UI screens,  and a working skill system in place. Improvements I want to realize till then mostly surround camouflage mechanics, shaders, camera/viewport, dialogs. Also: more content.

CamoTactics Update: Pixelart, Particles & Inventories

It's been silent around CamoTactics lately.  I've been going on and off track from game development during the last two months. Part of that was caused by family meetings, the other by motivation issues.

As of recently I've been eagerly working on new features however, and there are plenty! Example: some first drafts for ranks/levels/skills/factions, size based inventories, a new way of handling graphics, customizable weapons, freshly awesome pixelart, particle effects, a new main menu and also basic in-game dialoge. All of this goes with some heavy optimization here and there too!

 The main menu is now animated, with moving tank, ground camo and particles. This screen and the UI still needs a lot of work.

 Inventories support multiple item sizes and weapons can be customized. You can also switch between the character's inventory and the inventory of the vehicle you're currently driving.

Experimental particles have been added for shooting and traveling over different terrain types. Looks promising, but that system requires better code.


Despite of the break I am very happy with the progress, and there's a good chance this game is going to get finished! The next release is planned on 19th September, so stay tuned.

Thank you for reading!

Camo Tactics Alpha 1 Build 4 released!

Alpha Release 4 is out! Get it on GameJolt or Indie DB!

 
Many features were implemented and I worked off all planned items, including some that I considered optional. Interestingly, most of this was implemented during the first two weeks after release 3, and that productivity spike sadly slacked off in the following two weeks. I am still very satisfied with the progress I made this iteration. To recap what I wanted to implement this month, here is the full list:

• [DONE] Make installation easier. There's multiple ways to do this (installer or bundled jar) and requires more research.
• [DONE] Extend UI functionality (preparation for main menus and missions)
• [DONE] Get rid of the prototype-ish graphics and replace them with better ones.
• [DONE] The early prototype of this game included a very simple "camouflage" system which gave combat an interesting twist. This system is going to be re-implemented.
• [DONE] Performance and usability improvements
• [DONE] Many bug fixes (primarily inventories, weapons, sound) and internal improvements
• [DONE] adding some first particle effects
• adding infantry
• mounting vehicles
• [DONE] more advanced AI

I have a lot of plans for the next build already, which is going to be scheduled for 16th of August. On my list are:

• "Walking" AI. So far I only have one for immobile static defenses.
• A first stub of the Tower Defense Mode, as attacker. I can possibly get that one running till then.
• A first stub of the Mission System.
• Playtesting revealed I need better possibilities to take cover. Infantry is going to be a lot more fragile than tanks, so this is VITAL. For this I'm going to use obstacles such as trenches or bunkers as "sensors", and these sensors reduce damage or even have the chance to eliminate a projectile if it touches that sensor. This can be different to each projectile type, such that Missiles aren't able to travel through a wired fence, for example.
• Reimplementation of Infantry. Infantry was not adapted to gfx changes made prior alpha 1 build 3 and thus taken out of the game, so I'm going to reimplement them next update.
• Infantry mounting tanks (later vehicles).
• Many internal improvements, cleanups and bug fixes.

Optional list is:

• Play around with armed motorcycles.
• Consider overheating as additional weapon property.
• A "size-based" Inventory, rather than grid-based: Larger items are larger in inventory than small ones.

So how is Camo Tactics going to look like when done? For some answers, please refer to this post.

I am very excited with this game, and I hope you are too. Thank you for reading!