GDScript: managing data catalogs with the Resource class
I am happy to say we're coming closer and closer to the two-year anniversary of my coding adventure on Lampyre. Working on GODOT is a joy and I feel proud to have maintained such a long daily streak of programming sessions (I code every day for a minimum amount of twenty minutes, generally more like ninety minutes and up to six hours - while skipping less than a session every month).
Like any other hobby, game development slams you with its unforeseen gap of knowledge. The difference between "Yeah, let's code a simple inventory system, I can totally see how it'll work ! I've played so many games that have one. It's a two days job - at most." and the actual "HOW DO I MANAGE SO MANY ITEM SLOTS UI INTERACTIONS WITHOUT LOSING MY MIND ??".
Even if you prepared the mechanics of your game in advance, they will need iterations - refining. Sometimes you realize an idea is not so good and you drop it altogether. And you can be sure that many good insights will surface as you work on your project. So you acknowledge your mistakes, rearrange your code. You live and learn ! There are many little things about GDScript I wish I knew from the very start of Lampyre. But hey, if I did, we'll all be native coders and there would be a storm of incredible, perfectly optimized indie games around (which sounds very awesome - please work on that).
The Resource class is one of these tricks I wish I knew from the start !
Disclaimer: while I've been working on Lampyre for more than a year, I am still a casual GDScript programmer. I am exploring my way through the code and by no means, I would claim my practices to be the best - especially depending on the type of project you work, and the number of people in your team. I still make mistakes, and will always do. Do not hesitate to investigate a subject further with the awesome GODOT community.
Instanciate everything
When you're a newbie like me and want to get into coding, you will probably start by reading the pleasant official introductions to the engine and maybe complete the 2D and 3D tutorial projects for a concrete hands-on with GDScript and nodes.
And just after, as you're happily let loose in GODOT just like a frenzied chicken is set free into a grassy field, you can begin working on (too) ambitious systems such as procedural spawning, or vegetation and natural resources management for your RPG (how hard can it be to spawn a forest and let your villagers cut their own wood, mhm ?).
You do your best by writing your own classes and giving them their own names, so they can be instantiated as nodes in your game-world. Which is great. Buildings ! Rocks ! The player character ! GODOT scene tree is there to help you handle them.
But what about abstract entities ? What do you do when you need to somehow stash all your crafting recipes, or make sure a procedural river can always be generated with the same codified informations ? A natural (beginner's) reaction would simply be to create a new class inherited from Node3D - the parent of most tree nodes he's been using so far to create objects in the game world. And then to add the different variables that could be desired to characterize the object (in a good old object-oriented fashion).
Alas, it's not the ideal way of doing.
The Node3D parent overshoot
Let's use the example of our cooking recipes, since I am currently working on crafting mechanics for Lampyre (I'm not including getters and setters here for clarity's sake).
extends Node3D
class_name FoodRecipe
## A class defining a food recipe in our game.
var difficulty:float ## The difficulty's score for cooking this recipe, ranging from 0 to 100.
var dish:Item ## The dish resulting from this recipe.
var ingredients:Dictionary ## Items required to craft the recipes as well as their quantities ([_item:Item]: {_quantity:int}).
var recipeName:String ## The formal name of our delicious recipe.
var requireFire:bool = true ## Does this recipe require an active fire to be achieved ?
# Ran when our node enters the scene tree.
func _ready() -> void:
pass
# Ran each frame.
func _process(_delta:float) -> void:
pass
This is a cute object with straightforward variables defining each cooking recipe. But alas, like a poor child in a tragic dystopian story, the sole fact it inherits from the wrong parent will guarantee it a grim fate when you'll try to make it functional.
It's never going to be in the scene tree anyway
First of all, and while that might be very obvious to a seasoned GDScript programmer - your food recipes are never going to be added to your scene trees (unless you're creating a world where the cooking book pages have little legs and run on their own ? that sounds awesome). You can forget about the _ready() and _process() functions and safely delete them.
Then comes the main issue - referring your recipes elsewhere in your game. Let's say that I have a cooking fire with a script attached. I want to press a button, and it will roast a forest potato (even potatoes appear magical when you imply they're a special kind grown in a mossy forest).
# Ran when we hit the "Cook" button on our firecamp. Produces a roasted forest potato.
func _on_cook_button_pressed() -> void:
var _roastedPotatoRecipe:= FoodRecipe.new() # create a reference to our food recipe
# -------------
# WHAT'S THE ROASTED POTATO RECIPE ??
# -------------
produce_dish(_roastedPotatoRecipe.dish) # ask the firecamp to cook the recipe's results - a delicious, grilled and mysterious forest potato
You need a way to tell your script what the recipe is so it can obtain the resulting dish to produce. Let us see three of them (spoiler: none of them is an effective one to use).
The local variable fill
Simply fill out the recipe's informations on the place. Easy.
var roastedPotato:Item
var potato:Item
# Ran when we hit the "Cook" button on our firecamp. Produces a roasted forest potato.
func _on_cook_button_pressed() -> void:
var _roastedPotatoRecipe:= FoodRecipe.new() # create a reference to our food recipe
# -------------
# WHAT'S THE ROASTED POTATO RECIPE ??
# Solution 1: fill the informations locally
_roastedPotatoRecipe.difficulty = 10.0 # it's not very hard to cook a potato
_roastedPotatoRecipe.dish = roastedPotato # an item reference declared on firecamp
var _ingredients:Dictionary
_ingredients[potato] = 1 # you need 1 potato Item to produce this dish
_roastedPotatoRecipe.ingredients # another item reference declared on firecampe
_roastedPotatoRecipe.recipeName = "Roasted forest potato"
_roastedPotatoRecipe.requireFire = true # you'd need fire to cook it
# -------------
produce_dish(_roastedPotatoRecipe.dish) # ask the firecamp to cook the recipe's results - a delicious, grilled and mysterious forest potato
This works. But as soon as you'll want to refer to roasted forest potato elsewhere in your code, you're bound to write all of its attributes again, and make sure they stay the same across all instances. Decided to change its difficulty to balance the game ? You'll have to replace it manually everywhere, which is a nightmare to maintain and highly error-prone.
The PackedScene (or loading) way
@export var potatoPreset:PackedScene ## A scene you previously saved in GODOT with all the recipe informations set with @export variables.
# Ran when we hit the "Cook" button on our firecamp. Produces a roasted forest potato.
func _on_cook_button_pressed() -> void:
var _roastedPotatoRecipe:FoodRecipe # create a reference to our food recipe
# -------------
# WHAT'S THE ROASTED POTATO RECIPE ??
# Solution 2: load a recipe scene you prepared and saved
var _preset:FoodRecipe = potatoPreset.instantiate() as FoodRecipe # load the saved preset
if _preset: # if it has been successfully loaded
_roastedPotatoRecipe = _preset # copy its informations
# -------------
produce_dish(_roastedPotatoRecipe.dish) # ask the firecamp to cook the recipe's results - a delicious, grilled and mysterious forest potato
Here we create a scene and added a FoodRecipe node inside the tree. We can then fill out the recipe's properties inside the editor (provided you add @export in front of each one so they can be directly set). You then save the scene somewhere, and refer to it in your campfire script so it can be loaded (first line above).
That's a bit better, since the preset scene you saved can be the source of all modifications for the roasted potato recipe. Change one variable value in the preset, and it will affect all instances your recipe your game. It still means you have to add the @export packed scene reference to every nook where you want to use the recipe (imagine a cooking station with a hundred dishes available - that's a lot of editor entries and saved scenes to juggle with).
Additionally, instantiating the packed scene means loading a resource pulled from your discs into your game. While it's not the end of the world, it is miles away from being as efficient and as safe as manipulating abstract data through your code.
Note that using load() or preload() with your packed scene path instead of an @export variable is the same - it just means an extra step and an extra path reference you have to manage, lest you get loading errors crashing your game.
var potatoPresetPath:String = "res://recipes/roasted_potato_recipe.tscn"
var _packedScene:PackedScene = load(potatoPresetPath)
if _packedScene != null:
var _preset:FoodRecipe = _packedScene.instantiate() as FoodRecipe # load the saved preset
if _preset: # if it has been successfully loaded
_roastedPotatoRecipe = _preset # copy its informations
The (still pretty wrong) detailed dictionary way
When I coded my first natural resources lists (such as plant essences or rock types), I though I had a genius idea. What about creating an index class ? A centralized catalog to gather all of my plant species in one neat place. So I wrote another function inside my class that looked like this.
On the FoodRecipe class:
var roastedPotato:Item
var potato:Item
# Returns a chosen recipe's characteristics.
func fetch_food_recipe(_recipeName:String) -> Dictionary:
var _recipe:Dictionary
match _recipeName:
"Roasted forest potato":
_recipe["difficulty"] = 10.0 # it's not very hard to cook a potato
_recipe["dish"] = roastedPotato # an item reference
_recipe["ingredientsType"] = [potato] # another item reference
_recipe["ingredientsQuantity"] = [1]
_recipe["recipeName"] = "Roasted forest potato"
_recipe["requireFire"] = true # you'd need fire to cook it
return _recipe
Back to the firecamp class:
# Ran when we hit the "Cook" button on our firecamp. Produces a roasted forest potato.
func _on_cook_button_pressed() -> void:
var _roastedPotatoRecipe:= FoodRecipe.new() # create a reference to our food recipe
# -------------
# WHAT'S THE ROASTED POTATO RECIPE ??
# Solution 3: load a centralized dictionary entry
var _roastedPotatoEntry:Dictionary = _roastedPotatoRecipe.fetch_food_recipe("Roasted forest potato") # get the recipe entry from the static function
var _dish:Item = _roastedPotatoEntry["dish"] # just get the dish information you want and ignore the rest
# -------------
produce_dish(_dish) # ask the firecamp to cook the recipe's results - a delicious, grilled and mysterious forest potato
This is a monstruosity in all kind of ways, and I wanted to use this example because I actually wrote things like this in my first optimizing attempts.
Sure, informations are centralized. Dictionaries seems neat to organize data in keys, but they can also crash very fast. Imagine again that I have a hundred recipes listed in my fetch_food_recipe() catalog. It would be just as many entries with no less than six lines of data keys to fill each time (you can get the key wrong - and you can get the key content wrong as it does not expect any specific format).
When the dictionary is sampled for its "dish" entry on the firecamp, there is no safety net. If the key does not exist for one reason or another, the game will throw you an error. You'd need to add a verification line each time you access the data.
var _roastedPotatoEntry:Dictionary = FoodRecipe.fetch_food_recipe("Roasted forest potato") # get the recipe entry from the FoodRecipe function
if _roastedPotatoEntry.has("dish"): # make sure the key actually exist
var _dish:Item = _roastedPotatoEntry["dish"] # just get the dish information you want and ignore the rest
Editing an entry key on the FoodRecipe class or anywhere else where you're calling fetch_food_recipe() is bound to be a terrible experience. We're also accessing the recipe entry with a String name ("Roasted forest potato"), which is asking for trouble.
This won't return anything because of a spelling issue, and your editor will NEVER warn you about it because it has no idea of the String the function expects (it will warn you of your game crashing though).
var _roastedPotatoEntry:Dictionary = FoodRecipe.fetch_food_recipe("Rosted forest potato")
One last bit: since the fetch_food_recipe() function is not static, that means we still need to create an instance of FoodRecipe just to gather the data (I did told you I took the worst example of everything I messed up in my first attempts).
Which leads me to the biggest flaw of inheriting our FoodRecipe from Node3D, and instantiating it left and right...
It's not RefCounted: You have to clean up your mess !
Memory management is a big things in programming (and its specifics are waaaay over my own skills range). Data and objects are loaded into your computer's active memory to be manipulated. More complex languages like C++ allow precise control and operations about memory allocation. Dextrous developers would (and still) manage how the variables and objects are distributed at runtime, but also (and perhaps more importantly) when they must be freed.
Memory usage today is still a crucial factor of a program's efficiency - and some softwares were infamously known for memory leaks: data that is never cleaned up after being used. It occupies your computer's working memory, uselessly eating up its resources until it caused performance issues or crash. You would then need to reboot the program to clear up the unused data (and sometimes, reboot the whole PC).
While becoming less frequent nowadays, these problems are still a reality. And they apply to our dear GODOT too !
Each time you create a new class object through .new() and .instantiate(), it gets allocated its own cozy place into your computer's memory, ready to work. And for most objects, it means that if you do not free them - they will exist idly until the game is closed - even if you never access them again.
So, back to our solutions just above - they're not only displaying obvious type-safety flaws. Each time you'll hit the "Cook" button on the fireplace, a new instance of a FoodRecipe 3DNode will be created and will chill around in your working memory until the end of times.
When I coded Lampyre's inventory system, I realized at some point I was piling up hundred of unused Item instances when moving them around (because they needed to be duplicated when splitting and merging stacks, or when shifting from one container to another). And that was on minutes-long playtest. Imagine what garbage can accumulate over the course of whole hours or more.
Just using any of the solutions I mentioned above would force you to add one or more freeing lines at the end of your _on_cook_button_pressed() function:
# free up our objects before ending the function
_preset.queue_free() # if you used a FoodRecipe preset for solution 2
_dish.queue_free() # if you created a dish Item object in solution 3
_roastedPotatoRecipe.queue_free() # our FoodRecipe holding informations
That is cumbersome and sometimes easy to forget (although you will need to free up resources if you're manipulating classes that inherit Node). Also, even if not added to a game tree, Node3D is a class that is way too bulky for our recipe use, leading to slower iterations for no particular reason.
Fortunately for us, GODOT has a great solution to help. I just listed many things you can do wrong with data management - now is the time to do things in a tidier way !
The Resource way
All classes inherited from Node must be manually freed from memory with free() or queue_free() if not used anymore. But there is another type of object that you should be fond of, and it's the Resource class.
The Resource class inherits from the lovely-named RefCounted class, itself a child of the mother of all GODOT script types: Object.
Object > RefCounted > Resource
Objects are the ultimate lightweight type in GODOT - even slimmer than its Node child. As the GODOT documentation elegantly puts it, Objects have this advantage of...
"Simplifying one's API to smaller scoped objects helps improve its accessibility and improve iteration time. Rather than working with the entire Node library, one creates an abbreviated set of Objects from which a node can generate and manage the appropriate sub-nodes."
While a tiny less optimized than Object, RefCounted have this amazing ability to automatically track their own references, freeing themselves on their own if they are not accessed or referred to by any other script part !
You can create, instantiate, duplicate them to your heart's desire without fearing any memory leak. Perfect for carrying little encapsulated data pods around your game.
Note: this automatic reference cleaning is not instantaneous if you're coding in C# instead of GDScript.
That being said, why would we be using Resource instead of its RefCounted parent ?
Mainly for two reasons. First, contrary to RefCounted, Resource is inspector compatible and lets you easily modify your object's variable inside GODOT. Let's say Node3D was always a terrible parent to our FoodRecipe class (sorry Node3D, you shine in open space but you just suck at moving my data around without making a fuss). So we're changing the parent up for the delicate Resource.
extends Resource
class_name FoodRecipe
## A class defining a food recipe in our game.
@export var difficulty:float ## The difficulty's score for cooking this recipe, ranging from 0 to 100.
var dish:Item ## The dish resulting from this recipe.
@export var ingredients:Dictionary ## Items required to craft the recipes as well as their quantities ([_item:Item]: {_quantity:int}).
@export var recipeName:String ## The formal name of our delicious recipe.
@export var requireFire:bool = true ## Does this recipe require an active fire to be achieved ?
As I added the @export keyword to our Resource variable, you'll see that they are easily modified through the inspector should you put a FoodRecipe variable on another script.
This will bring you a dedicated, encapsulated space to complete the recipe's fields directly through the GODOT editor should you need it.
@export var recipe:FoodRecipe
Notice that I did not add @export in front of our dish Item variable. Unfortunately, you can only export Node-derived classes in other Node-derived classes, and Item is not one of them. But as I'll suggest in the paragraphs just below, you can replace this Item export for an enum entry.
The second cool reason to use Resource instead of RefCounted is that it is the base class for serialized objects in GODOT. I won't get into details here, but that crudely means the engine will entirely handle the data on its own when saving and loading it in and out of your game, requiring less code logic and producing smaller files. In short, helping you code your save system with less headaches.
Note: it turns out the PackedScene class is also RefCounted and can be easily serialized - no need to free it when loading one at runtime ! But be careful. Everything else inheriting from Object and its Node child is a potential object that will pile up and suck up memory until you free it.
All of this is great, but changing up Node3D for Resource did not help in letting our food recipes be managed and distributed through the game's scripts. Let's take a look at my current favorite index setup I'm using for Lampyre, which is an hybrid solution between my old catalog idea and the Resource minimalistic approach.
My current solution for moving encapsulated data around
Here is how I would re-write the FoodRecipe class. I added two other recipes to demonstrate how you'd use the class as a self-contained catalog.
extends Resource
class_name FoodRecipe
## A class defining a food recipe in our game.
enum RecipeName { ## Our recipes names as an enum to avoid relying on arbitrary String or int.
None, # a default value if none is provided
AshenDeerVenison,
RoastedForestPotato,
SaltyChanterelles
}
var difficulty:float ## The difficulty's score for cooking this recipe, ranging from 0 to 100.
var dish:Item.ItemName ## The dish resulting from this recipe (this is an enum of all game items).
var ingredients:Dictionary ## Items required to craft the recipes as well as their quantities ([_itemName:Item.ItemName]: {_quantity:int}).
var recipeName:RecipeName ## What recipe is this ?
var requireFire:bool = true ## Does this recipe require an active fire to be achieved ?
## Returns a chosen recipe's characteristics.
static func fetch_food_recipe(_recipeName:RecipeName) -> FoodRecipe:
var _recipe:= FoodRecipe.new() # create a new food recipe template to carry informations
# list here every variable you could want to change inside any recipe - you can set default values if you want
var _difficulty:float = 0.0
var _dish:Item.ItemName # remember, this is now an enum referring to an Item entry
var _ingredients:Dictionary
# recipeName will automatically be set to this function's argument - if we're asking for the roasted potato recipe, then the recipe is roasted potato
var _requireFire:bool = true
match _recipeName: # fill out recipe's informations where the default value does not fit
RecipeName.AshenDeerVenison:
_difficulty = 60.0
_dish = Item.ItemName.AshenDeerVenison
_ingredients[Item.ItemName.AshenDeerMeat] = 1
RecipeName.RoastedForestPotato:
_difficulty = 10.0
_dish = Item.ItemName.RoastedForestPotato
_ingredients[Item.ItemName.ForestPotato] = 1
RecipeName.SaltyChanterelles:
_difficulty = 35.0
_dish = Item.ItemName.SaltyChanterelles
_ingredients[Item.ItemName.Chanterelles] = 6
_ingredients[Item.ItemName.Salt] = 1
_requireFire = false
# whatever the recipe chosen, replace all variables with the potential overrides you expressed
_recipe.difficulty = _difficulty
_recipe.dish = _dish
_recipe.ingredients = _ingredients
_recipeName = RecipeName
_recipe.requireFire = _requireFire
return _recipe
First, I now list all my different food recipes inside an enum. This is a much safer way to refer them across your game's scripts as there is no more room for typos, and your editor should help you complete your enum when you begin typing FoodRecipe.RecipeName...
I say should, because GODOT's IDE is a little finicky with enum completion and sometimes will plainly not help you fill them out - but at least you'll get a very clear error message if you mistyped your enum entry.
Then on the same model, I also changed my Item entries for dish and ingredients to a lighter Item.ItemName enum (which, you guessed it, is an enum containing all the unique items entries in my game).
fetch_food_recipe() is now a static function that can be accessed from anywhere without instantiating a particular FoodRecipe object. Its only argument is a RecipeName enum entry - no more room for typing mistakes.
Then comes the heart of the catalog. Using the efficient match keyword on our recipe enum entry, we override values for variables we'd like to customize on the recipe entry. Setting default values beforehand (such a default difficulty at 0, or that a dish requires fire to be cooked) saves you writing time inside each recipe where you don't need to replace them.
I'm still hungry for roasted FOREST potato ! How does the cook function looks like on our firecamp side ?
extends Node3D
class_name Firecamp
# Produce a specific dish on this firecamp.
func produce_dish(_dish:Item.ItemName) -> void:
# fetch the dish Item object through its entry (another catalog ! zOMG !)
# add it to the firecamp storage, or let it drop on the floor nearby...
pass
# Ran when we hit the "Cook" button on our firecamp. Produces a roasted forest potato.
func _on_cook_button_pressed() -> void:
# we directly create our FoodRecipe data capsule by calling our static catalog function
var _roastedPotatoRecipe:= FoodRecipe.fetch_food_recipe(FoodRecipe.RecipeName.RoastedForestPotato)
# -------------
# WHAT'S THE ROASTED POTATO RECIPE ??
# WELL WE ALREADY KNOW BECAUSE THE VARIABLE IS INSIDE _roastedPotatoRecipe WITHOUT FURTHER ACTION REQUIRED
# -------------
produce_dish(_roastedPotatoRecipe.dish) # ask the firecamp to cook the recipe's results - a delicious, grilled and mysterious forest potato
# _roastedPotatoRecipe does not need any freeing up, because it's a reference-counted Resource !
# _roastedPotatoRecipe.queue_free() <- useless
Two lines of safely encapsulated informations. fetch_food_recipe() will directly warn you if you're asking for a recipe that does not exist, and _roastedPotatoRecipe.dish will also throw you an error if the dish variable is invalid.
The whole thing is as light as I could manage, and have zero risk of creating orphan objects that would be forgotten into the depth of your active computer memory.
The solution is not yet perfect.
- It is still quite bothersome to completely change a recipe name (e.g RoastedForestPotato becoming RoastedMarshPotato) - you'll have to make sure all the places specifically referring to a given recipe enum are properly edited (such as RecipeName.RoastedForestPotato), whether it's in the editor or in your code.
- Adding or substracting a variable to FoodRecipe also means you have to edit all the FoodRecipe entries inside the fetch_food_recipe() - but it is a catalog, and whatever system you're using, modifying a data structure means editing the entries depending on it. At least, setting a default value can save you time - and the Resource encapsulated format will make sure your editor warns you if a variable does not exists on your object.
- Just for extra safety, I would add the following line preventing an recipe with an empty dish to be attempted on the firecamp. You can circumvent this by adding a default produced dish directly inside fetch_food_recipe(), so it never returns empty.
if _roastedPotatoRecipe.dish != null:
produce_dish(_roastedPotatoRecipe.dish) # ask the firecamp to cook the recipe's results - a delicious, grilled and mysterious forest potato
The external document way
Another appreciated way of manipulating large indexes of data through your game can be to outsource these informations inside external files - say, in CSV format. It lets you edit your entries informations through a simple text or data sheet editor. And it can then be loaded and read at runtime by GODOT.
I do not feel comfortable using this solution, but many developers successfully use it and I can see it being very helpful if your data file needs other softwares/applications reading them. An even more centralized way of managing and sharing your entries from the outside of GODOT !
See: GODOT Docs - Localization using spreadsheets
Conclusion
That's it for this little overview of the Resource class - one that I now use regularly for all my items, building, recipes or abstract entities through Lampyre.
If you're interested in learning more, check out the (much more) informative GODOT Docs pages and experiment with solutions that fit your own project.
GODOT Docs - Resource class page (4.5)
GODOT Docs - Resource tutorial page (4.5)
GODOT Docs - When and how to avoid using nodes for everything
GODOT Docs - Creating your own resources (4.5)
Happy coding, take care.