openage dev updates

Openage Development News: June 2025 (and before)

2025-07-14T00:00:00+02:00

Hello everyone and welcome to the newest update on openage development. This time we are going to take a look at all the things that happened since August and also other stuff that might be interesting. So without further ado, let's bring you up to date!

Game entity interaction

We'll start with the features that enable interaction between game entities, i.e. attacking, converting and all other things that game entities can do to each other. I'm happy to say that this milestone has now been completed (yay!) and also works pretty nicely with everything else we've implemented so far. Check out this recording to see how it looks:

To recap the initial problem description for game entity interaction: The challenge in letting game entities interact is not the triggering of a single action, e.g. an attack. That's actually the easy part. Instead, the major challenge arises from modelling the complex gameplay patterns in RTS that combine multiple actions. What do we mean by that? Well, take these types of gameplay as examples:

looping actions, e.g. attacking until a target is dead
chained actions, e.g. moving closer to another unit before attacking
combinations of the above, e.g. chasing a moving target

You can probably find more examples if you think about your last gaming session. RTS and especially AoE2 are full of those patterns.

Agent Behavior in the openage Activity System

Another way to think about how these gameplay patterns work is to see game entities as agents for the player. Unlike in a first-person game, where a player's inputs directly control the character, agent control is usually more indirect. Agents may be given commands that then trigger specific behavior and might set up a complex chain of events. They also may act completely on their own, e.g. units using the aggressive/defensive/passive stances system in AoE2.

openage implements this agent behavior in a place called the activity system, which we already talked about before. The idea is that game entity behavior is modeled via a node graph that acts like a state machine for the individual game entity. Traversing the graph happens when the game entity receives events, e.g. a player command. Visiting nodes then executes actions or checks conditions to branch on different paths in the graph. You could also say that the activity graph represents the decisions a game entity can make.

Most of the work for game entity interaction went into making the activity system fit for interactive agent behavior. For example, we implemented new decision node types to evaluate the state of the game entities that interact, e.g. for checking if one game entity is in range of the other's attack radius. We also had to design an activity graph that matches the unit behavior in games like AoE2, so that gameplay feels the same. You can see the resulting graph below:

Follow the graph from the Start node on the left. The behavior for attacking is located in the top right portion of the graph. You can see the basic loop for checking whether the target is in range as well as the application of an effect (like attacking) via the ApplyEffect node.

Additionally, we wanted to make the whole node graph configurable, ensuring that game entity behavior is changeable instead of relying on hardcoded behavior. This actually worked pretty nicely. We managed to expose all node types, decision functions, and actions via the game data files, so almost everything can be configured without changing the code. Currently, there are no failsafes for checking if a graph is "correct" (i.e. it doesn't crash the whole engine), but that will hopefully be added in the near future.

Targeting game entities (with your mouse)

So far we've only mentioned how agent behavior may be triggered by commands and conveniently left out how these commands end up in the game. We want to specifically look at the implementation of targeting game entities, i.e. finding out which game entity a player "right-clicked" on.

Player inputs like right-clicking are routed through openage's input system. A minor "problem" for interaction is that this input system knows almost nothing about what's going on in the simulation. This design allows us to run the input system in a separate thread, making the handling of inputs faster and easier, so we ideally don't want to change it. However, it raises the question: How do we get to know what game entity a player right-clicked on? After all, we should be able to give the selected game entities a hint on what they should interact with.

openage's current solution is a bit janky, but it currently gets the job done. For this, we consult the openage renderer - another engine subsystem. The idea is that the renderer creates an additional texture, a so-called ID texture, that writes the value of the game entity ID to the pixel location of sprites belonging to said game entity. Essentially, this texture lets us look up which game entity is occupying each pixel on the screen.

The ID texture is directly passed to the input system which then uses the (x, y) coordinates of the click event to look up the game entity ID at that position. Unlike the color textures created by the renderer, the ID texture is never shown on screen. Here is what it would look like if it would be drawn:

Black pixels match no game entity. Colored pixels represent locations of game entities.

Side Tangents

Aside from game entity interactions, there were a bunch of features that were implemented on the side. These features are probably only "cool" if you are a giant nerd (like @heinezen), but he writes these posts and decides what gets discussed, so...

Curve Compression

Curves are openage's internal data structure for storing value changes over time in the game simulation. In other words, they store past, present, and predicted values of game entity data used during the game, e.g. HP or unit position, as time-value keyframes. Values for the time between keyframes can also be interpolated.

In most situations in the simulation, keyframes are inserted lazily, which means that the simulation doesn't check whether the keyframe is redundant or not. This is usually fine and also desired, since this makes operating on curves much faster.

Curve without compression. Keyframes B, D, E, H, and K are redundant and don't change the interpolated values.

However, there are cases where we don't want duplicate keyframes. An example of this is the usage of curves for storing the animations of objects in the renderer. To which frame index to use for the current animation, the renderer checks the animation's keyframe time to determine when the animation started. The frame index is then roughly calculated using this formula:

1	`frame_idx = (current_time - keyframe_time) / time_per_frame`

Problems can occur when the animation is triggered by a looping action, e.g. attacking a game entity until it's dead. Currently, the animation is requested for every "iteration" of this loop, i.e. every time the action is done, a new animation keyframe is inserted. However, if the loop time for the action does not match the loop time of the animation, then the animation gets abruptly cut off whenever a new keyframe is inserted.

Curve compression solves this nicely by only adding keyframes is they don't change the interpolated value. This is what the curve looks like when compressed:

Curve with compression. Only the necessary keyframes remain.

This not only solves the jittering animations in the renderer, but it's also useful for other use case where we want the data to be compact. Examples for this are curves that are sent over the network or recordings written to disk.

Converter Cleanups

Transforming the game data from AoE2 and other games to openage formats is handled by the openage converter. The code for this conversion is actually pretty complex and rivals the actual engine code in size. However, it's also been infamous for containing a bunch of files that are pretty much unreadable as they contain up to 10,000 lines.

Having huge files is not a really a problem for running the scripts, but rather developing on them. Turns out parsing giant files doesn't make the job of IDE indexers very easy. That alone was enough motivation to finally split up the converter files more. Increasing the readability is also a nice plus and makes the converter more maintainable.

As a side goal, we've also been working on making the converter less error-prone and more forgiving in some cases. This was necessary due to the constant changes to the game data format in the new expansions for AoE2:DE. Updates to AoE2:DE should hopefully not crash the converter as much anymore, although we still have to keep pace with the data format changes.

Outside Contributions

Since our last update, openage has received a number of outside contributions that we want to honorably mention here. These add a few significant improvements to the engine :)

@jere8184 and @dmwever improved the openage pathfinder. Most notably are the increased the pairing heap performance, the introduction of the dirty flag for integration as well as the addition of an array curve datastructure for use in the field types.

@jere8184 also added fixes for the Windows builds and CI pipeline. They also introduced fused types to the SLP/SMP/SMX/SLD image conversion in our converter.

@ZzzhHe has made added several new features to the renderer: - Completeness checks for uniform initialization - Binding empty vertex array objects - Configurable shader templates

@haytham918 had added optional clang-tidy checks to our buildsystem.

@bytegrrrl added a few operations to our FixedPoint class that enable us to use pure fixed-point calculations in more places. FixedPoint can now also use an intermediary type useful for operations on large numbers, thanks to the contribution of @Eeelco.

We hope we didn't forget anybody who made important changes :) More external contributions are also open in the GitHub repository.

What's next?

A better first question might be: When does the next blogpost release after this one? Will it also take almost a year? That's a good question, although the answer is that sadly it will probably take a while to write the next update. Writing the devlogs takes a lot of time, especially when they should be understandable for newbies. Also, when the choice is between writing devlogs and coding, coding is still much more fun... So coding will take priority for a while. However, I (@heinezen) still like creating the devlogs, so maybe they will pop up more frequently again sooner or later :D

As for openage, we now have opened a loooot of possibilities with the completition of game entity interaction. We can now go the boring route and add more interaction types that are not handled fully yet (like building construction). There's also the possibility for a more exciting route which leads us even deeper into topic of state transitions, e.g. proper handling of unit deaths. Or we might do something entirely different depending on who has better ideas. I guess you'll find out next time ;)

Take care!

Questions?

Any remaining questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: August 2024

2024-09-09T00:00:00+02:00

Hello everyone and welcome to another update on openage. For this month, our priorities have mostly been getting release 0.6.0 ready to ship and thinking about something to start on for the subsequent 0.7 milestone. While release 0.6.0 is currently stuck in review for a bit, milestone 0.7 is already starting to take shape, so we can already tell you a few details about it. So without further ado, let's get started.

Game entity interaction

As our major focus for milestone 0.7, we have decided on interaction between game entities. This includes all ingame mechanics that let units do things to each other. The most common example of this would probably be attacking, although conversion, construction or resource gathering also fall under this definition.

openage (mostly) encapsulates all interactions inside one ability type: ApplyEffect. As the name suggests, ApplyEffect hold a batch of Effect objects that define what interactions are done when the ability is used. The concrete type of the Effect object determines the type of interaction. For example, attack damage is modeled by the Effect type with the (somewhat bulky) name FlatAttributeChange. Different effect types can be mixed in a batch, so an ApplyEffect ability could theoretically do damage and simultaneously try converting a unit.

Structure of ability and effect types in the API.

Our implementation is still in a very basic work in progress stage, so there are not many interesting stories to tell yet. We are currently focusing on getting attack interactions to work with the game entities in the AoE2 modpack with some example code. Here you can see part of the result:

A knight game entity, currently attacking itself due to an absence of other targets.

Implementing more interaction will get more and more complex as other systems get involved such as attack stances, line of sight, state changes, and so on. You can expect to read more on that over the next months.

What's next?

When we are satisfied with the basic interactions, we will gradually expand the capabilties of the engine to support more interaction features. An obvious choice for the next improvement would be the introduction of a collision system, which we need so that game entities can compute when they are close enough to each other for interactions.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: July 2024

2024-08-04T00:00:00+02:00

Hello everyone and welcome to another openage monthly devlog. For this month, we are looking at a new renderer feature that adds a common optimization strategy to our rendering pipeline. We will also talk a little bit about the imminent release of the openage version 0.6.0.

Frustum Culling

We are going to start with frustum culling, a feature added by outside contributor @nikhilghosh75 that has finally landed in the main repository.

A frustum in this context is referring to a camera frustum, which basically represents the view cone of our camera in the 3D scene. Objects that are outside of this view cone would not be visible in a rendered frame, even if we told the GPU to draw the object. However, requesting the GPU to draw these objects and letting it figure out where and if they would be on screen can still take a great amount of time. Only once the GPU notices that the object would be drawn outside the viewport, it can skip further calculations.

With frustum culling, we are trying to exclude the objects that are not visible as early as possible, so that they don't even result in a draw call to the GPU. Ideally, we can also skip shader uniforms updates (done on the CPU side) while the object stays invisible. The potential for time saves here can be huge. Imagine a game scene that has 10,000 objects in total but only 500 are visible in the camera view. In this case, the number of draw calls and shader updates would be reduced by 95%!

openage currently supports 2D frustums (for sprite animations) and 3D frustums (for objects like terrain). Since most of our rendered objects are sprite animations, the 2D frustum is the type that would be used most frequently. You can see how the culling works in this small demo:

Red rectangle: Frustum bounding box
Blue rectangles: Animation boundary box of an object

As you can see, objects (blue rectangles) outside the frustum boundaries (red rectangle) are not included the render pass. To make the culling effect visible in this demo, the frustum's size is smaller than the actual camera viewport. In a normal game, the frustum would be slightly larger than the camera viewport to ensure that all objects that are visible are displayed. Notice also that frustum culling is not pixel perfect for the animations (e.g. for the leftmost object). Instead, the maximum boundary of all animation sprites are used, so an object may still be drawn if one of its current animation's sprites could be visible during a rendered frame.

Preparing for the Next Release

We have decided that after the last pathfinding improvements have been merged, we want to release the latest state of the project as version 0.6. The engine has improved a lot since the last release, so in our mind it makes sense to publish a minor feature milestone to reflect what we have achieved so far.

Most additions for the next release should already be known to avid readers of our devlogs, so we won't repeat them in detail here. Our major personal highlights are the new flow field pathfinder as well as the massive performance improvements we accomplished. Of course, there are also a bunch of small and big fixes, like getting the engine to work on Windows and macOS again, that are pretty nice.

What's next?

When the release is done, we will focus more on game simulation, specifically game entity interactions. There are also long-term improvements planned for the GUI and audio systems, so maybe we also start on that in the near future.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: June 2024

2024-07-10T00:00:00+02:00

Hello and welcome to another openage devlog. This month, we have introduced a few more pathfinder updates that make it work better with the game simulation. We are now getting to the point where the pathfinding looks and performs okay enough that we can include it in the next release version!

Propagating Line-of-sight through Portals

As described in our April devlog, the pathfinder uses line-of-sight (LOS) optimization to improve pathing close to the target cell. To put it simply: When a cell on the grid is flagged as LOS, there exist a direct (non-obstructed) path from this cell to the target. In practice, this means that any game entity at the position of this cell can move to the target position in a straight line. This makes pathing look much more natural and smooth. If we would only use the direction vectors of the flow field right until the end, we would be limited to their 8 possible directions.

In our initial implementation, LOS optimization was confined to the target sector, i.e. cells outside the target sector were never flagged as LOS. However, this meant that a lot of paths that could have been a straight line but crossed a sector boundary looked noticeably weird. Instead of bee-lining straight towards the target when there are no obstructions, game entities would have to move to the edge of the target sector first before they would reach an LOS cell:

You can almost see where the sector boundary is as the game entity is turning very sharply towards the target as it reaches the first LOS flagged cell in the target sector.

This behaviour has been fixed by propagating the LOS integration through sector portals. Essentially, LOS flags are passed from one side of the portal (in the entry sector) to the other (in the exit sector). LOS integration then continues in the exit sector using the passed LOS flagged cells as the starting point. As a result, paths look much better when they cross multiple sectors:

Optimizing Field Generation

The topic of performance came up in a Reddit comment before, so we thought it might be interesting to pick it up again. Performance is big factor in the feasibility of flow fields, since pathfinding should not stall gameplay operations. Flow fields do have some benefits for the quality of paths, but the added complexity can come with a hefty performance price tag. Thus, we have to ensure that the pathfinding is still as performant as possible.

Over the last month, we have applied several optimization strategies:

Simplified Code

We removed a lot of redundant code from the design phase of the pathfinder. This includes things like redundant sanity checks, debug code, or flags that were only relevant for the pathfinding demos that you've seen in our monthly devlogs.

CPU-friendly datastructures

To increase throughput for field generation on the CPU, we replaced most occurences of datastructures that are known to be slow (e.g. std::unordered_map, std::deque, std::unordered_set) with vectorized data types (std::vector and std::array). These data types utilize the CPU's L1-L3 caches much better, which means that the CPU has to spend less time on fetching data from RAM and has more time for running calculations.

Flow Field Caching

Generated flow fields are now cached and reused for subsequent path requests if possible. In practice, chaching can be done for all flow fields on the high-level path where the target is a portal cell (i.e. any sector that is not the target sector). Since field generation is deterministic, building two flow fields with the same target cell results in them being equal. Therefore, if a high-level paths uses the same portal as a previous path request, the previously generated flow field can be reused.

The overall result of our optimizations is that pathfinding is now about 2x-4x faster than in our first iteration. Technically, there are no benchmarks yet, so you have to trust our numbers for now. On our test machines, a path request can take between 0.3ms and 2ms for paths of roughly the same length, depending on the number of fields that have to be built and how many obtructions there are per sector. Flow field and integration field generation in the low-level pathfinding stage is now so fast that the A* calculations of the high-level pathfinder are becoming the bottleneck with ~50% runtime usage.

What's next?

That was definitely enough pathfinding for a while. There is a probably still lot to improve, but other parts of the engine need attention too.

Next month, we will focus more on game entity interactions. That means we will make them do things to each other, like damage or healing or whatever we can think about that's fun (and not too complicated).

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: May 2024

2024-06-04T00:00:00+02:00

We have a few more pathfinding updates for you this month. These are mostly about integrating pathfinding into the game simulation to test it under real gameplay conditions.

Pathfinding in Game Simulation

We finally got flow field pathfinding implemented in the game simulation! You can see the result below:

In the game simulation, not that much is being done beyond initializing the flow field grids when loading a modpack (as discussed in last month's post) and sending path requests from the movement system to the pathfinding subcomponent. In the movement system, we only have to create the path request consisting of grid ID (extracted from the moving game entity's Move ability), start location (current position of the game entity), and target location (position of the movement target). The request is forwarded to the Pathfinder object referenced in the game state which acts as an interface to the pathfinding subcomponent. After the pathfinding is finished, the movement system gets back a list of waypoints which can then be inserted into the game entity's movement curve.

While all of this sounds pretty simple now, there were still a bunch of fixes necessary to the main pathfinding algorithms to make it work properly. We will quickly go over the major problems that we faced in the next sections.

Diagonal Paths Blocking

When building a flow field, we want to point the vectors of each cell in the direction of the path with the minimum movement cost. The naive way of computing a direction vector for a cell in a flow field is pretty simple: For each cell, we check the integration values for the 8 surrounding neighbor cells and let the direction vector point in the direction of the neighbor with the lowest value. Do that for all the cells and the flow field is ready.

However, this solution is flawed in that it does not account for a specific edge case concerning diagonal movement. You may be able to spot the problem by looking at the example grid below.

Hint: Check the direction vectors in the top corner.

As you can see above, the problem here is that some diagonal paths at the top should not be possible. It's as if the path would literally slips through the cracks of the impassable cells. This behavior is cause by the naive implementation considering all neighbor cells individually. However, logically (or in AoE2 at least), a diagonal path should only be able to exist if an adjacent horizontal or vertical cell are passable. If neither of them are, then the corresponding diagonal cell should be considered "blocked".

The solution to this problem is actually pretty simple. We can find out which neighbor cells should be considered "blocked" by processing the 4 cardinal neighbor cells' integration values first and add an additional check that sets a flag if they are impassable. Afterwards, we process the 4 diagonal cells where we now also check whether their adjacent horizontal/vertical cells are impassable. If they both are impassable, the diagonal cell is considered "blocked" and the integration value comparison is skipped over.

The result then looks like this:

Start/Target Sector Refinements

As previously explained in an older devlog, the pathfinder executes in three stages:

High-level search: Search for a path on the sector level with A* using the portals connecting each sector.
Low-level search: Build flow fields for all sectors in the sector path found by the high-level search.
Waypoint creation: Follow the direction vectors in the flow fields from start to target cell.

The reason we do stage 1 is to save computation time by only creating and building flow fields for the sectors that are visited by the path.

In the inital implementation of stage 1, there were a few bugs that have to be ironed out. For example, one (wrong) assumption we made was that a start or target cell would automatically have access to all portals in their sector. There are some obvious counterexamples, e.g. when the start is on an island:

Ingame example of how such an island could look like. Note that terrain is only one way of creating this situation. Sorrounding the game entity with buildings would have the same effect.

In the situation shown above, we cannot even exit the sector and are confined to the island. Another example would be a situation where only partial access to portals exist:

Start cell (green) and target cell (orange) have access to different portals, even though they are in the same sector. The only way to reach the target from the start is to path through the portals to the neighboring sector on the left.

To avoid these problems, the pathfinder is now doing a preliminary check before the high-level search that determines which portals are accessible by the start cell and the target cell, respectively. This is done by calculating their sectors' integration fields and checking which portals are passable in said integration fields¹. When the A* algorithm in the high-level search is started, we use the portals reachable from the start sector as the starting nodes. The portals reachable by the target cell become possible end nodes.

¹ We can reuse these fields in stage 2 (low-level search) when building the flow fields for these sectors. Thus, it isn't even much of an overhead.

What's next?

You can be excited for more pathfinding updates next month, but then that's probably it. We promise! Other than pathfinding, we also have updates for the renderer pipeline queues up, so tune in if you are interested in more graphics-related progress.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: April 2024

2024-05-05T00:00:00+02:00

Hello everyone, we are happy to have you here for yet another openage update. For the latter half of the month, we have taken a small break from pathfinding to work on other small internal projects that have been piling up, e.g. some leftover tasks in the renderer. That also means that, unlike last month, this update won't be 100% pathfinding exclusive, so those of you who enjoy more variety have something to look forward to!

Pathfinding in nyan API

Since the pathfinding system in the engine is mostly finished, we have been making efforts to officially use it into the movement systems of the game simulation and build it into our rudimentary map generation. The first major step in this direction is the integration of new pathfinding-related objects into our nyan modding API. This allows game objects to utilize the pathfinding functionality in several ways.

The new API object PathType can be used to define grids in the pathfinder. Usually, there exists more than one pathfinding grid per game. For example, AoE2 has two grids, one for land-based units and one for ships. If you are very generous, you could additionally consider that there is a third "air" grid as the ambient birds flying all over the map have separate pathfinding rules. A unique air grid is also used for flying units in Star Wars: Galactic Battlegrounds.

PathType objects can be referenced by other API objects to reference a specific grid, e.g. by the Move ability with its new member path_type. This tells the engine which grid to search for pathfinding when the ability is used by a game entity. For dynamically influencing the cost of the grid, there is now a Pathable ability which changes the cost of grid cells for one or multiple grids at its location when it is active. The latter ability may for example be used by buildings to make parts of the grid impassable as long as the building exists.

For map generation, the Terrain API object now allows defining the pathing costs associated with each terrain via the path_costs member. This member simply assigns a cost value to each grid defined with PathType. When the map is created, the terrain generator used these values to initialize the cost fields for the grid in the pathfinder.

Rendering Multi-textured Objects

After adding all the relevant pathfinding types to our API, we are finally able to put all pieces together and make units move with proper pathfinding. However, there was one minor problem that we wanted to resolve first: While the gamestate has supported tiling with terrains for a while, the terrain renderer couldn't display them properly yet. In fact, the renderer would only use the texture of the first tile for texturing a whole 10x10 chunk, so all chunks looked like they only contained a single terrain type. Given that this is rather distracting when we want to test whether pathfinding based on specific terrain costs works, we took a slight detour to extend the renderer first. You can see the result in the screenshot below.

Previously, each terrain chunk in the renderer was handled as a single mesh that would be drawn with one texture in the shader. In the new implementation, the chunk is split up so that all tiles with matching terrain textures get their own mesh. For example, if there are 6 different terrain textures, then there would be 6 meshes created for the chunk. All of the meshes are drawn individually, but this is practically unnoticeable as the meshes border each other seamlessly. You may have also noticed the lack of terrain blending which currently make the tile edges very distinct.

After we updated the terrain renderer, we also applied similar changes to the world renderer (which draws unit sprites). Here, we added support for rendering animations with multiple layers. In the above screenshot, you can see this in action when you look at the castle, which consists of a main layer and a shadow layer. The main layer is the building sprite, whereas the shadow layer is the semi-transparent grayscale sprite to the left of the building. Previously, the renderer would only draw whatever layer was defined first.

The principle is roughly the same as for the terrain chunks: Every layer gets its own mesh which is drawn individually. Draw order is a bit more important in this context because animations of different units are more likely to overlap each other than tiles in the terrain renderer. Therefore, we added the possibility to insert renderable objects by priority into the rendering pipeline. As a result, all animation parts should now be displayed properly in the game.

What's next?

Now that the terrain renderer actually displays what's happening in the gamestate, we can start working on integrating the pathfinder again. In theory, this is pretty trivial to add, but we'll have to see if we encounter some errors in the implementation.

The next obvious step for the terrain renderer is blending support. This could be much harder as we would have to find a strategy that can handle the blending styles of all the old and new releases (AoE1, Conquerors, and the Definitive Editions all have different approaches). We will probably try to tackle the classic Conquerors blending first and then iterate and adjust gradually for other styles.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: March 2024

2024-03-31T00:00:00+01:00

Hello everyone and welcome to another round of openage updates. If you enjoyed the pathfinding explanations from our last devlog, you can probably start getting excited because this month's post will be all about even more fun pathfinding features. However, we hope that those of you who are not avid pathfinding aficionados can have some fun too. So let's get to it.

Line of Sight Optimization

In last month's update, we showed off the flow fields that are created by our new pathfinder implementation. Essentially, the idea behind flow fields is that instead of computing a single path from start A to target B, we generate a vector field for the whole grid where every cell gets assigned a vector that points towards the next cheapest cell for reaching B. As a result, units anywhere on the grid can follow (or flow) along these vectors to eventually arrive at the target (see below).

target cell == (7,7); origin is left corner

yellow == less expensive
purple == more expensive
black == impassible

In this example, you can see that the flow field vectors only support 8 directions. A side effect of this is that paths become diamond-shaped, i.e. they have turns of at least 45 degrees. While this may not be as noticeable when you are just looking at the static flow field, it can be very annoying when observing units in motion. When controlling units, most players would expect them to go in a straight line if there are no obstacles between start and target (and not take detour into a castle's line of fire). For this reason, we have to do some low-level optimization that smoothes out short-distance paths in these situations.

One of these optimizations is a so-called line-of-sight pass. In this preliminary step, we flag every cell that can be reached from the target in a straight line with an "line-of-sight" flag. Units in these cells can then be pathed directly towards the target in a straight line without having to use the vector field. You can see the results of doing such a line-of-sight pass in the image below.

target cell == (1,4); origin is left corner

white == line-of-sight flag
light grey == blocked flag
dark grey == passable, but no line-of-sight
black == impassible

Impassable cells or cells with more than minimum cost are considered line-of-sight blockers and are assigned a separate "blocked" flag. The same goes for cells that are only partially in line-of-sight, e.g. cells that are on the line between the blocker cells' outer corners and the edge of the grid. The cells marked as "blocked" form the boundaries of a line-of-sight vision cones that span across the grid.

For cells with a "line-of-sight" flag, we can skip calculating flow field vectors as they are no longer necessary for finding a path.

Travelling with Portals

One advantage of flow fields is that the generated field may be reused for multiple path requests to the same target, e.g. group movement of units. While reusing the fields can save computation time in this context, the complexity of flow field calculations make the initial cost of building the flow field much higher than for other pathfinding methods. On larger grids, this can make individual path requests incredibly slow.

To solve this problem, we can utilize a simple trick: We split the pathfinding grid into smaller sectors, e.g. with size 16x16, and search for a high-level path through these sectors first. Afterwards, we only calculate flow fields for the sectors that are visited and ignore the rest of the grid.

To accomplish this, sectors are connected via so-called portals. Portal are created on the pathable edges between two sectors. Additionally, portals in the same sector are connected to each other if they are mutually reachable. The result is a mesh of portal nodes that can be searched with a high-level pathfinder. For the high-level pathfinder, we can use a node-based approach intead of flow fields, e.g. the A* algorithm, to search the mesh. Finding the high-level path should usually be pretty fast as it only depends on the number of portals on the grid.

white == passable
grey == portal tiles
black == impassible

What's next?

The only major step left in the pathfinder integration is to include it into the actual game simulation, i.e. building it into map/terrain generation. This should be easier than the implementation the pathfinder itself, but will take some effort to get right. If pathfinding becomes too tedious, we might switch things up and work on something else for a short while. In any case, you will find out what we decided to do in next month's update!

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: January 2024

2024-02-18T00:00:00+01:00

Hello everyone and welcome to another (very delayed) update to the current openage development progress. This time, we have a lot to talk about a new larger feature implementation (and a few other cool things that are interesting for you nerds). So without further ado, let's look at the changes.

Pathfinding with Flow Fields

In mid-January, we started working on a new pathfinding algorithm based on flow fields. It is set to enhance our previous pathfinding logic which so far is a pure A* implementation.

For those unfamiliar with flow fields, here is a quick introduction: Flow field pathfinding is a technique that's specifically intended for large crowd movements, which is exactly what we are doing in most RTS games. In these games, you often have situations where a) multiple units are controlled at the same time and b) moved as a group to the same goal location. The key idea behind flow field pathfinding is that instead of finding the best path for every individual unit, as done in A*, we calculate the best paths for the whole grid as direction vectors. These vectors then steer units around obstacles and towards the goal. As a result, all units with the same goal can "flow" across the grid using these vectors, no matter where their starting positions are.

Explaining every detail about flow fields could warrant its own blogpost, so we will stop here and direct everyone interested enough to read the article that our implementation is based on. Currently, we are still demoing our flow field pathfinder to tweak it before we build it into the actual game simulation. The demo shows just the basic flow field functionality, but you should already be able to see where we are going with it.

green == less expensive, red == more expensive, black == impassible

Every flow field pathing request starts with a cost field that assigns each cell in the grid a cost value. This cost determines how expensive it is to move to a cell on the grid. The cost field changes infrequently during gameplay, e.g. when a building is placed.

target cell == (7,7); origin is left corner

yellow == less expensive, purple == more expensive, black == impassible

When we get a pathing request to a specific goal, another field called integration field is computed. The integrated costs of each cell contain the minimum movement cost required to reach the goal from the cell. To do this, we integrate outward starting with the target cell by checking each cell's direct neighbors and setting the cells integrated cost to own_cost + cheapest_neighbor_cost.

As a final step, we create the flow field that calculates steering vectors for each cell. The steering vectors point towards the neighbor cell with the lowest integrated cost. This way, units following the vectors should always take the path with the cheapest movement cost to the goal. This in independent from where their initial position is on the grid.

Optimizations

Since the beginning of this year, we have started optimizing some parts of the code in Python and C++ that were in need of a speedup. On the C++-side, this is mostly addressed by making our internal data structures more cache-friendly. This is especially relevant for our renderer, where cache-friendly data means more throughput and, as a consequence, more objects that can be shown on screen.

In our Python code, we have added multi-threading support to the final media export step in the conversion process. Previously, conversion of graphics data took a significant amount of time, especially for the newer game releases which require processing gigabytes of graphics files. Converting this data would often take up at least 30 minutes.

The new implementation of the media converter now parallelizes the conversion of graphics data using Python's multiprocessing module. This drastically speeds up the overall conversion process by utilizing all available CPU threads. Conversion should now take no longer than 5 minutes for any AoE release.

The table below shows conversion times before and after multi-threading was introduced. As you can see, the great beneficiaries are DE1 and DE2. The other games also profit, although not as much because some files convert so fast that the converter cannot spawn threads fast enough to keep up. This is also the reason why AoE1 is slightly slower now, although the difference is negligable.

Release	Before	After
AoE1 (1997)	28.410s	33.39s
AoE2 (1999)	81.186s	51.01s
SWGB	109.620s	62.07s
HD Edition	67.717s	53.23s
DE1	216.225s	66.48s
DE2	959.706s	250.63s

What's next?

We are going to put more effort into pathfinding, so that we can use it in the actual game simulation soon. That would also require us to properly design collision detection, so the feature might stay in a "work in progress" stage for a while.

Besides the new pathfinding, we also want to integrate more GUI and HUD features as that will become more relavant once we add more gameplay features.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: December 2023

2024-01-03T00:00:00+01:00

Hello everyone and welcome to yet another openage development update. As 2023 is wrapping up, we can look back on a huge amount of features, fixes, and restructerings that made it into the engine this year. Basically, we transformed openage from an unmaintanable monolith into a usable, albeit still work-in-progress, engine prototype. All of the newly designed subsystems were finally integrated into the engine core. This means we can start 2024 without worrying as much about the engine's stability and architecture as before :)

During the holiday season, openage development also took a small break, so this blogpost is slightly more brief than usual. We will get back to the usual format next month.

Configurable Activities - Part 2

Our new activity system, which we introduced in last month's new post, has now been finalized and published in the new openage nyan API v0.4.1 specification (see PR#1608). In comparison to the progress we showed last month, there are a few minor changes. Most notably, there are now dedicated, built-in conditions for the XORGate nodes. Some of the other objects were also renamed for extra clarity:

Support for the new activity system nyan objects has been added to the engine and the converter. In the converter, there are now two default activity graphs built using the new API objects. One is a very simple graph for buildings that just consists of a basic Start -> Idle -> End flow that essentially only makes sure that the building gets properly drawn on screen. However, the activity graph for units is more complex and makee full use of the event-based node transitions. It is an extension of the hardcoded actvity graph that we previously used in examples, with the notable improvement that game entities can now "break out" of a movement action when they receive a new command.

Game entities created during modpack conversion will automatically be assigned an Activity ability that references one of the default graphs. Ingame, the end results looks like this:

The current implementation should support most of the simple action flows found in AoE games. We will keep testing and extend the nyan API definition as we add more and more features that utilizing the activity graph.

Did you do a 37C3 lightning talk?

As some of you who are following the prject for longer than a year may know, we usually try to organize a lightning talk at the annual Chaos Communication Congress (see the YouTube recordings of our previous attendences). Unfortunately, we couldn't organize an update on the big stage at the 37C3 this year due to scheduling problems, so there will be no regular annual status report video for 2023. We will try to record an alternative status update in either January or February to make up for that and show off some of the recent developments. Until then, you can find our status update for release 0.5.0 on YouTube which also covers a large portion of the significant improvements we added last year.

What's next?

There are no concrete milestone for 2024 yet, but we are still working on improving our internal demos and adding more gameplay features.

For next month, we will start implementing more complex mechanics such as collision detection and pathfinding to the engine. These will likely take more than a month to be usable, so there will also be updates in-between that add small stuff like configurable hotkeys or a better display of game entity data in the viewport.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: November 2023

2023-12-08T00:00:00+01:00

Hello and welcome to another one of our monthly openage devlogs! This month was all about adding usability features to the engine as well as making the nyan data API easier to use. Well, easier for us developers at least. Without further ado, let's start with our most interesting new feature.

Drag Selection

Selecting units with drag selection on screen is something you see in every RTS game since the 90s and I'm sure everyone reading this has used it before. For the player, the process is pretty simple: They draw a rectangle on screen by holding the mouse button down and everything visible in the rectangle is then put into their selection queue. In openage it now looks like this:

While this looks like a small feature, the implementation a bit more challenging than what we have done before. Most of this is because drag selection requires multiple subsystems to work together to get the desired result. First of all, we need the input system to handle not just one event but a sequence of three mouse actions (MouseDown -> MouseMove -> MouseUp). For the selection itself, we also need to figure out which units inside the rectangle are selectable by the player, so we don't end up with a bunch of trees in our selection queue. Last but not least, the rectangle also has to be drawn on screen, so the player can see what they are actually going to select.

In our old engine architecture, the implementation of this feature was a hot garbled mess that massively tanked performance (also visible in our YouTube demo), partly because it was basically taking control of the renderer which stalled all other draw commands. This is where our new engine architecture with decoupled subsystems finally pays off big time. In comparison to the previous implementation, the new drag selection is communicated via sane interfaces from the input system to the other subsystems of the engine.

Here is how the input events are handled now:

MouseDown: The input system detects the drag selection sequence initialization and switches into a drag selection context
MouseMove: Inputs are forwarded to two different places
1. A HUD controller that tells the renderer to draw/resize the rectangle on screen
2. A game controller for the player that keeps track of the rectangle position and size
MouseUp: The input system finalizes the selection
1. The HUD controller tells the renderer to delete the drawn rectangle
2. The game controller sends the final rectangle position size as a drag select event to the game simulation
3. The game simulation handles the drag select event by
  1. Checking which units are inside the rectangle
  2. Checking which units are selectable by the player
  3. Finally, sending the list of unit IDs back to the controller

Configurable Activities

Way back in June, we added configurable game entity behaviour in the form of the activity system into the engine. However, for the last months the "configurable" part only existed in theory, since all game entities were assigned a hardcoded activity by default. This month. we have been making efforts to add support for activities both to our nyan data API and the openage converter. This will eventually allow us to define the whole behaviour in the modpack itself, which in turn also allows modders to change game entity behaviour with a simple data mod.

Currently, the nyan API changes look like this:

Since activities are simple directed node graphs, we can model all node types as nyan objects which point to successor nodes via the next member. Internal events are handled the same way, using nyan objects that have members for the event payload. When the modpack is loaded, the engine builds the activity graph from using the node definitions.

Activities are assigned to game entities with a new ability (also called Activity) which references the node graph. This means that every game entity can potentially have its own unique behaviour (which can also be changed at runtime).

What's next?

During next month, we will put more work into the configurable activities and release a new data API specification when we are done. If there is enough time, we will also improve the HUD a bit to display more useful information for players on screen.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: October 2023

2023-11-09T00:00:00+01:00

Hello everyone and welcome to another update to the openage project. October has been a rather slow month, with us mostly focusing on getting the code base cleaned up. As part of the cleanup, we also ported a few rendering features of the old engine core to the new renderer. But more on that further below.

Legacy Code Cleanup

Our legacy code cleanup continues with the refactoring of the old GUI interface which is now completely decoupled from the game simulation. The old GUI directly interfaced with gameplay features, which was not necessarily bad but kind of slow. In comparison to this, the new GUI will not communicate with the game simulation directly, but rather send input signals through our regular input system that is also used for keyboard and mouse inputs. Communicating in the other direction (simulation to GUI) will be a little bit more complicated but that is a story for a future News update.

We've additionally removed the last remnants of SDL related code. As a result, SDL2 will be removed as a dependency in the next release as all window system functionality is now handled by Qt. Furthermore, we have also resolved a few nasty crashing issues and memory leaks in the renderer and fixed display bugs when using the Wayland compositor.

Terrain Chunks

openage can now properly handle terrain assets from modpacks and display them ingame. Our previous terrain implementation used a hardcoded test texture - which you should be familiar with from all the screenshots we've published. Not only can the renderer use "real" terrain assets now, it can also display more than one of them at once! As a result, openage looks almost like a real game:

Internally, both the terrain data in the game simulation and the terrain render objects are now organized in chunks. Chunks are collections of 16x16 tiles that can be updated individually. This allows us to utilize better runtime optimization, since we don't have to update the whole terrain grid at once if there is a change.

The current implementation is nowhere near feature complete yet. For example, there's no blending and only one texture can be displayed per chunk. Gameplay-wise, terrain has also no effect as pathfinding and collision are not implemented right now.

What's next?

Code cleanup will probably be finalized in November and published as a new release. Afterwards, we will start adding proper unit selection to the engine and find a way to display the selected units on the screen.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: September 2023

2023-10-07T00:00:00+02:00

Hello and welcome to our monthly openage recap. This month involved a significant amount of work reviewing, fixing pre-release code, and then preparing the publishing of said release. Even if there's currently not much going on feature-wise, there's a lot happening in the development process.

Version 0.5.0 Release

As some of you may have noticed, openage 0.5.0 has been released on GitHub. It's the first major release in a while and we are very proud of what we pulled off. 0.5.0 introduces the new architecture and is something that we can easily improve and build on. Avid readers of the News posts should already know what's in the release, so we'll spare you the details. For everyone else, take a look at the release highlights.

Let's just hope the next release won't take as long ;) ahem

What's coming for 0.6.0?

We are not totally sure yet where to start with 0.6.0 and what exactly will be part of the next release. Right now, the consensus is that we try to improve on 0.5.0 and add more features that directly support gameplay, e.g.:

HUD rendering (or parts of it)
Proper unit selection
Collision & Pathfinding
Configurable inputs
Audio Support

While implementing these, new gameplay features will probably be added in parallel depending on what we think is doable.

Before we start thinking about new features though, we also have some project maintainance work to take care of. On our GitHub, we've already introduced new project boards to track the development progress. We're additionally planning to add a bunch of new beginner issues for new contributors, which also have their own board. Furthermore, the should be a bugfix release coming soon that addresses a few compilation errors and engine startup problems.

Legacy Code Cleanup

While 0.5.0 in practice only uses the new architecture flow, most pre-0.5.0 legacy code has so far remained in the code base. We couldn't remove it previously because there were a giant amount of interdependencies in the legacy subsystems and removing them while also working on other parts of the engine would have been a nightmare. Now that 0.5.0 is out and running without relying on the old legacy subsystems, we can clean up what's left.

Since release 0.5.0, we have started the process of removing the legacy code subsystem by subsystem (see PR#1556). Currently, it looks like about ~40,000 lines will eventually be removed (compared to ~65,000 lines added for the new architecture). We have also identified some parts that are still salvagable and can be transferred to the new architecture such as:

Audio Manager (has to be ported from SDL to Qt)
Parts of the GUI (only needs new engine interface)
Some Game Logic (terrain generation, score calculation)

What's next?

We will focus on cleanup tasks publishing fixes for the current release first. After that is done, we will pick a new feature to work on, probably something "simple" like unit selection. As always, you will learn about it next month at the latest!

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: August 2023

2023-09-04T00:00:00+02:00

Welcome to our monthly openage update. As we wrap up for release v0.5.0, we can talk about a few quality-of-life and speed improvements that happened over the last weeks.

Tales of Renderer Troubles

In last month's update, we announced the implementation of a stresstest for the renderer. This has now been added in form of a simple render demo that incrementally adds renderable objects to the render loop and logs the current FPS. It's very bare bones, but it gets the job done:

However, while playing around with the first few stresstest runs, we noticed that the renderer performance was extremely bad (not to say abysmal). It barely managed handling 400 objects with 30 FPS and there were massive slowdowns after only 100 rendered objects. Keep in mind that the objects in question are not just units. They are everything that needs to be displayed in the game world: buildings, trees, birds, cacti... If 400 objects were the limit, we would run out of frames pretty quickly in a real game.

So we set out to investigate and fix the bottleneck. It turns out that there was a call to the Python API in the objects' frame update. Every frame the renderer would check its asset cache to see if the texture required for a render object was already loaded from disk. The cache lookup would resolve the texture path, initiating a Python call and introducing a massive overhead. We have since removed the calls to Python and the stresstest now easily manages >3500 objects at 30 FPS - almost 9x as much as before.

New Startup Flow

In the new release, we want to make the initial startup more user-friendly, so that it's easier to get the engine running. Well, maybe you should imagine large quotation marks around "user-friendly" because the engine is very much not ready to be used by the general public. However, we wanted to make sure that at least the people who get as far as successfully compiling everything are not greated with crashes.

To accomplish this, there is a new CLI startup flow that guide users through the initial setup phase and the conversion process. Common installation folders for games can be automatically searched, so that it should be easier to create usable modpacks no matter which AoE game or release you have. Even if no installation is available, the converter is now able to download the AoC Trial Version as a fallback. We also made sure that the engine works with all game releases.

What's next?

After release v0.5.0 is done, we will do a brief planning session to set the milestones for the next months. It's possible that we will prioritize removing legacy code in the next minor release and take a step back from adding new features until that's done.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: July 2023

2023-08-06T00:00:00+02:00

Welcome to another monthly update for openage. This month's progress does not involve a lot of code changes as we are preparing for the next release. Nevertheless there is a bunch of new stuff that's not code that we can tell you about.

Architecture Documentation

A lot of work has gone into cleaning up the architecture documentation to include all new/rewritten engine subsystems. If you have followed our monthly News posts in the last months, you have basically already seen an abridged version of the contents. The documentation just adds a lot more technical details and fancy diagrams. Overall, the documentation PR probably contain about 20 pages of new text.

In summary, there are completely new docs for the following subsystems: - Event System - Game simulation and all its sub-components: - Components - Systems - Event-based game logic - Time management - Curves

We also removed a bunch of old stuff for code that is now removed or deprecated and updated outdated information. The only thing remaining to be reworked is our website, although this shouldn't take much effort.

Next Release Plan

We are currently preparing for a new release v0.5.0 that will happen soon-ish, maybe even in August. This release will still not really be a "usable" release for casual users. However, it is an important milestone since it will be the first release where all new engine subsystems work together and the basic architecture outline is clear. In theory, this should also make outside contributions easier but we will have to see about that. There is a lot of legacy code that has to remain part of v0.5.0, so jumping into the code might not be that pleasent yet. We will use subsequent v0.5.X releases to refactor and clean up the legacy parts of the codebase.

Release v0.5.0 will add a few more interactable demos that show off what the engine is capable of. Basically, this will be similar to the things you've seen in previous news posts but a bit larger in scope. It should work as a minimal example on how to use the engine for someone that wants to contribute. There will also be a stresstest demo that checks overall engine performance and could help us to locate bottlenecks in the code.

What's next?

Preparartion for the next release will be our major focus. Apart from merging the docs and implementing the demos mentioned above, there also are a few minor bugs and annoyances left to fix.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: June 2023

2023-07-03T00:00:00+02:00

Welcome to another monthly openage update. This month there's been a lot of progress inside the gamestate, so there is much to show and even more to discuss.

Modpack Loading - Part 2

Since our previous update, modpack loading and game data handling inside the engine's gamestate have been greatly extended. The biggest advancement in this regard is that the engine can now create actual units - or game entities as we call them - from the information in the nyan database. Additionally, the available game data from the modpacks can now be used in the game simulation routines. Right now, game entities can do very little and therefore still use a very limited amount of the data, but it's a step forward from having hardcoded test entities.

Here you can see how different entities from the AoE1, AoE2 and SWGB modpacks are created and displayed from modpack data:

Note that this example contains both buildings and units as game entities. Internally, openage doesn't differentiate between different unit types and instead uses composition to define what each game entity can do. This means that when a game entity is spawned, the engine essentially "assembles" its capabilities by assigning properties and abilities (e.g. Move, Turn, Idle, Live, etc.) based on what is defined for the game entity in the modpack.

Rudimentary Game Logic

Now that we can get game entities into the game world, we can start playing around with them. Currently, there are only two things a game entity can do: Idle or Move. This doesn't sound like much, but more stuff will soon be added to the gamestate one by one. Most of this months work went into the architecture to support more complex action states of game entities.

In openage, the action state of a unit is not directly driven by commands from the input system but by a node-based flow graph with task nodes, branching paths, and wait states for events. Commands mainly signal where to go next in the graph. We do things this way because commands in RTS usually do not trigger one specific action but start entire action chains. For example, a Gather command in AoE2 does not just involve the process of slurping up resources; it also requires moving to the resource, automatically finding a dropoff place and actually depositing the resources in the player's storage.

Below you can see an example of what such a flow graph looks like. The example flow graph is currently hardcoded in the engine and assigned to every game entity. In the future, we want these flow graphs to be configurable per game entity (or game entity type), so that every unit could display different behaviour.

Operating on the flow graph in the engine then looks like this:

What's next?

The flow graph logic in the gamestate was the last major component missing in the new engine architecture. Since that is almost done, we will probably prepare a new release soon containing all the changes made over the last months. Before that, we have to clean up all the rough edges in the code and make the features more "presentable" to those who haven't followed the update posts.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: May 2023

2023-06-01T00:00:00+02:00

Welcome to another monthly update to the openage development. This month's progress consisted of a lot of cleanup and bug fixing tasks but there were also a variety of small features added to the engine. So without further ado, let's jump right in.

Modpack Loading

While the openage converter could already generate modpacks from the original game assets for a long time, the engine couldn't really understand them yet (except for loading/showing single animations for testing). This has been changed now with the addition of a simple internal mod manager as well as loaders for getting game data into the gamestate's nyan database. At this point, the gamestate still doesn't have any logic that can do something meaningful with the data. However, it gets us one step closer to testing features with "real" data instead of the more generic tests and demos that we've shown before. This also means that you might see more AoE2 or SWGB visuals in future news posts!

The engine's configuration is now also loaded as a modpack which is simply called engine. Currently, it contains the bindings for the openage data/modding API referenced by other modpacks. Later, it will probably include other bindings for scripting and GUI. engine is always implicitely available to other modpacks and can be used to interface with the engine API. Storing the engine confguration as a modpack could also allow basic modding of the low-level engine internals, although we'll have to see how well that works in the future.

Renderer Features

Our renderer can now properly handle drawing directions (or angles, respectively) of unit sprites. Depending on the direction a unit is facing in the scene, the renderer will now select the correct subtexture coordinates from the animation's sprite sheet and pass it the the shaders. You can see here how the result looks for one of our test assets (taken from here under CC-BY 4.0):

Of course, it also works for animations from AoE2:

What's probably less noticeable in the videos is that the renderer is additionally able to handled mirroring of sprites, i.e. flipping sprites alongside their X or Y axis. The feature is implemented directly in the display shader and using it should therefore produce very little overhead.

Mirroring is commonly used to save space since the sprite sheet only has to contain sprites for half the available directions. The sprites for the remaining directions can then be inferred from the vertically opposite direction. For example, assets in the original AoE2 release only contained sprites for 5 out of 8 directions. and mirrored the remaining 3.

What's next?

Since the engine now supports modpack loading, we can start working on the gamestate internals using actual game data from the converted assets. The first step will be to initialize the various abilities of a game entity in the engine with information from the nyan database. We will probably start with very simple abilities like Idle, Position, Live, and maybe Move and work our way to the more complex abilities later.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: April 2023

2023-05-02T00:00:00+02:00

Welcome to the April 2023 update for openage

This month we had progress in several subsystems of the engine, most of which we built in the previous months: the event system, the renderer, and the gamestate. These systems already worked together, but previously consisted more of hardcoded "duct tape" code to make it all work. In the past weeks, we replaced many of these code paths with something that is closer to the final architecture targeted by the engine.

Spawning Entities Reloaded

We already showed off a spawn event last month which created a Gaben entity on mouse click.

Previously, this just placed Gaben at a hardcoded position at the origin point of the 3D scene (0,0,0). With the new changes from this month, Gaben is able to spawn where the player clicks on the screen.

To achive this mouse spawning feature, we rewrote parts of the engine's old coordinate system and added a raycasting algorithm that can point at objects in the 3D scene.

The coordinate system is necessary to convert between the different coordinate types used by the engine's subsystems. For example, the input system mainly receives 2D pixel coordinates from the window management that represent the position of the mouse inside the window. The game world on the other hand uses 3D coordinates for positions of objects in the game world. A different kind of 3D coordinates are also used in the renderer to draw animations and terrain.

To place Gaben at the correct position inside the 3D game world, we have to convert the 2D pixel coordinates from the input system to a 3D coordinate in the scene. We achieve this with a very straightforward technique called "raycasting". Basically, we cast a ray from the position of the camera using its direction vector as the direction of the ray. Then, we check for a point where this ray intersects with an object in the 3D scene, e.g. the terrain mesh. The resulting intersection point is placement position we are looking for (as 3D scene coordinates). Since the 3D scene coordinates use a different axis orientation than 3D game world coordinates, we have to make one final conversion to get the correct values for the gamestate.

Moving Entities

Spawning entities at the mouse position is pretty cool by itself, but we are not done yet. With the power of the coordinate system, we can also make objects move!

To effectively explain how objects in the gamestate move, you have to understand curves - the data containers that openage uses to store its runtime data. They are described in more detail in an older blogpost, but here's the gist of it: Instead of calculating the current position of an object every frame (like many other engines do), curves store only changes to the position as keyframes, containing the new position and a timestamp. Basically, the keyframes represent waypoints of the objects where each waypoint also is assigned a timestamp that signifies when the object reached the waypoint. To get the position at a specific point in time t, the curve interpolate between the two keyframes before and after t. In case of positions, this means that it calculates the position between the two waypoints defined by keyframes.

Curves may sound complicated and internally they definitely are. However, they also make internal gamestate calculations and testing much easier because we can just insert a bunch of keyframes and let the curve figure out what the position at the current simulation time is (curves also allow us to easily go backwards in time, but that is a story for another month). In the current implementation, we add 4 additional waypoints to every spawned entity so that they follow a "square" path after creation.

What's next?

Spawning and moving entities is nice, but there is obviously more to do to make this feel more like an actual game. There are actually several options for what to do next. Adding more input events that do other stuff than spawning would be nice, e.g. movement or selection. On the other hand, we can also improve the internal gamestate by adding more event types that affect the simulation itself. Or maybe we do both, depending on how well each of them goes.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: March 2023

2023-04-02T00:00:00+02:00

Hello and welcome to the March 2023 update for openage.

As promised last time, this months update is mostly about inputs, managing those inputs and a tiny bit of event handling. It also contains small amounts of Gaben, so wait for the end of this post if you are into that. But first let us tell you something about the challenges of processing inputs.

Inputs in RTS

The problem with RTS is that they can allow a wide range of different mechanics and actions that have to be mapped to inputs. To complicate matters further, you usually don't control just one entity but many, which can also have their own actions available depending on ownership, state and various usage contexts. Furthermore, shortcut keys may be assigned multiple times in different contexts, so a naive mapping of key to action doesn't really work.

For openage, we have to take all this into account and additionally ensure that everything is as configurable as possible. Otherwise, support for multiple games will soon become very tricky.

openage's new input management

The solution implemented this month is to divide input processing into two stages: A low-level input system that handles/preprocesses raw inputs from the Qt window system and several high-level input handlers that translate the results into the actual (gameplay) actions or events.

Here is an overview for how that works internally:

Raw inputs from Qt (keyboard or mouse events) are first forwarded to the low-level input system (InputManager) via the window management. There, we do a little bit of preprocessing like stripping unnecessary information from the raw inputs. Context management, i.e. figuring out the currently active key binding, is also handled at this stage. If the input can be associated with an active key binding, it is sent to a high-level input system that performs an action with the input.

The high-level input systems are basically gateways to other components inside the engine, e.g. the gamestate or the renderer (Controller). Therefore, these are the places where the actual actions that have an effect on the game or visual output are performed. In the case of the engine controller - the gateway to the gamestate - the actions are functions that create an event for the gamestate. Which action is performed is decided by a simple lookup using the input event. The available actions will most likely be hardcoded until we introduce more scripting, but the current system allows them to be mapped to any key combination or mouse button.

With this feature done (or rather awaiting merging in PR #1501), the engine is now able to spawn a game entity with the gaben graphic on mouse click:

What's next?

Focus will now shift to implementing a few more input actions for creating gamestate events and implementing the gamestate internals along the way. The internal gamestate implementation for processing these events still needs a lot of work too, so we will probably try to implement one or two events at a time and see what else needs to be done along the way.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: February 2023

2023-03-01T00:00:00+01:00

February is gone, but we're here with another update on openage and what happened over the month.

Our initial February implementation goal from last month was getting the simulation running with player input. However, that input part turned out to be more complicated than at first glance, so it will take a little longer before there is anything interesting to talk about. However, there are other things about the general simulation framework and the renderer that made some progress, so you get news about that instead.

Simulation Time Shenanigans

The most precious resource of our internal simulation is time (as in actual time points provided by a clock). This has to do with how our engine calculates the current gamestate. In openage, the gamestate is event-based which means updates to anything (unit, building, or cactus) are scheduled for a specific time and then executed when the simulation time has advanced enough. Therefore, it is very important that the internal clock works correctly, since we would get all kinds of simulation deadlocks or desyncs otherwise.

Which leads us to a funny little "problem" with the clock that was merged in PR #1492 last month. Advancing the time consists of a simple diff between the current system time and the time of the last update, multiplying this value by the current simulation speed, and then adding the result to the cumulated simulation time. Additionally, this clock has no regular update intervals; it only advances time as a side effect of getting the current time. And all that works reasonably well... until the program is involuntarily stopped by a debugger or the OS going into sleep mode or your browser in the background eating too much RAM. You might have guessed it already but the problem is that the time diff between system time and last update grows larger and larger, even if the program is frozen by the OS. Technically, the simulation never stops in this case. When you close your laptop lid on a running game, you would probably be surprised that the AI won the game while you were away for 2 hours.

But openage development isn't about AIs taking over when you don't expect them to. So in the implementation, the clock has a dedicated update method, caps time advancements at a very small time value, runs in its own thread, and (hopefully) doesn't cause trouble anymore.

Rendering with the Clock

Other than the internal event-based simulation, the clock is also used in the renderer to time animation frames. Most of the functionality for that was completed in a PR this month which also added a bunch of other rendering features (mostly asset management). Playing animations based on the current simulation time also got its own demo which you can see here:

(Run with these commands:)

1	`./bin/run test --demo renderer.tests.renderer_demo 4`

The video of the demo also shows how animations can speed up, slow down and even reverse based on the simulation speed. Reversing time is something much more relevant for the gamestate than the renderer, but it's cool to show off here.

What's next?

It will be input events for the gamestate, but this time for real. There are no major features that need implementing anymore which don't involve the gamestate. Simulation, rendering and the event loop are all set up and working. For input events, we still need to rewrite the control flow of the old input handlers so that they produce events for the gamestate. Once that is finished, we can start creatng game objects that receive and act on these events.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: January 2023

2023-02-04T00:00:00+01:00

We're back with updates from December & January. Despite holiday stuff and being plagued by illnesses, we've made some progress on the codebase.

Camera

The renderer now supports a camera that acts as players' view into the rendered scene. It's pretty basic at the moment, but already supports moving around as well as zooming in and out of the scene. There's also functionality to look at a position by centering the camera view on a scene coordinate. The latter should become more useful when there is actual gameplay to center on.

Since openage implements a mixture of 2D (units/buildings) and 3D (terrain) for rendering, the camera can technically be used to display arbritrary 3D objects, i.e. calculate the necessary view and (isometric) projection matrices for 3D rendering. This is probably not interesting for classic Age of Empires gameplay, but we could use it for debug purposes in the future, e.g. to show collision boxes.

Merging Progress & Technical Demos

The current state of the renderer has matured enough that we can merge it into the main codebase now (see PR Link). There's still some things to do, but the structure of the renderer will likely stay the same for now. With the renderer "finished", this means we can focus on the gamestate part of the engine next.

The code in the PR contains a few technical demos that show off the new renderer features and their usage. You can try them yourself, if you want, by building the project and running

1	`./bin/run test --demo renderer.tests.renderer_demo X`

in the project folder and replacing X with a number between 0 and 3. For example, demo 3 allows controlling the camera in a basic test scene. That's also where the camera video comes from.

What's next?

Well, how about some gameplay? This is obviously the next step, although it could take us a while to get something playable running. The crucial step will be the implementation of the internal event simulation, e.g. getting input events and converting them into commands for the gamestate. We also need a way to time events with a simulation clock (which is already implemented in the renderer PR) and save them to an event log.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: November 2022

2022-12-02T00:00:00+01:00

Hello again with a slightly delayed November Update for openage.

Last month's post had an unfortunate lack of images, so I hope we can make up to it this time. With this in mind, we can present you a wonky, weirdly colored screenshot taken straight from the current build.

It probably doesn't look like much, but it actually already shows the usage of rendering components directly by the engine's internal gamestate (although still making heavy use of test textures and parameters). The current build also implements the major render stages for drawing the game: terrain and unit rendering.

Gamestate to Renderer

But let's backtrack a little bit and start from where we left off last month. In our last update, we talked about decoupling renderer and gamestate as much as possible, so that they don't depend on each other as much. However, the gamestate still needs to communicate with the renderer, so it can show what is happening inside the game on screen. Therefore, this month's work was focused on building a generalized pipeline from the gamestate to the renderer. Its basic workflow looks like this for unit rendering:

Left side shows the state inside the engine, right side the state inside the renderer. As you can see from the flowgraph, the gamestate never directly uses the renderer itself. Instead, it only sends information on what it wants to be drawn to the renderer via a connector object (the "render entity"). This object then converts the information from the gamestate into something the renderer can understand. For example, it may convert the position of a unit inside the game world into coordinates in the graphics scene of OpenGL.

The converted data from the render entities are then used for actual drawable objects (e.g. WorldRenderObject). These are mostly used to store the render state of the drawable, e.g. variables for the shader or texture handles for the animations that should be displayed. Every frame, the drawable objects poll the render entities for updates and are then drawn by the renderer. Actually, there are several subrenders which each represent a stage in the drawing process, e.g. terrain rendering, unit rendering, GUI rendering, etc. . In the end, the outputs of each stage are blended together and create the result shown on screen.

Here you can see how that happens behind the scenes.

Skybox Stage: Draws the background of the scene in a single color (this would be black in AoE2).

Terrain Stage: Draws the terrain. The gamestate terrain is actually converted to a 3D mesh by the renderer which makes it much easier to texture.

World Stage: Draws units, buildings and anything else that "physically" exists inside the game world. For now, it only draws dummy game objects that look like Gaben and the red X.

GUI Stage: Draws the GUI from Qt QML definitions.

What's next?

With the rendering pipeline mostly done, we will probably start shifting to more work inside the gamestate. The first task here will be to get the simulation running by setting up the event loops and implementing time management. The renderer also needs rudimentary time management for correctly playing animations. Once that's done, we can play around with a dynamic gamestate that changes based on inputs from the player.

If there's enough time, we may also get around refactoring the old coordinate system for the game world. This would also be required for reimplementing camera movement in the renderer.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: October 2022

2022-11-01T00:00:00+01:00

Goodbye SDL

As announced previous month, we've spent a lot of time removing SDL from the codebase and replacing it with Qt. Before, we used both SDL and Qt in tandem. This generally worked okay, but was always a bit weird, since both frameworks have essentially the same features. Mixing them together in our codebase also required a few workarounds, like having to convert SDL window events to Qt events (and vice versa), wrapping the Qt window inside SDL for GUI drawing (which also created problems on some OS's display servers), and numerous smaller forms of jank.

As of now, everything related to SDL graphics has been ported to Qt6. This includes these components for example: - Window Creation - Input Events (mouse clicks, key presses) - OpengL Context Management - Texture Loading - GUI Management and Drawing A lot of the glue code connecting SDL and Qt could also be removed, which reduces the code complexity quite a bit. While SDL is not part of the graphics pipeline anymore, it still remains in other parts of the code, e.g. audio management, so it's not completely removed yet. However, we will probably replace that with a Qt equivalent in the near future.

Decoupling Renderer and Engine

Another side effect of the rework of the graphics code is that the remaining code is now separated from the main engine where the gameplay calculations happen. Before, graphics and gameplay were rather closely coupled, with direct communication between GUI, renderer and gameplay. There was also no clearly defined path between the components, so keeping the complex gamestate intact when the code was changed was not easy.

We've now reworked this wild-west approach into something which is hopefully more manageable in the future. Basically, the new workflow of the engine considers the renderer and GUI components as optional, i.e. everything in the engine concerning gamepay should work on its own without access to a display. Whenever the components have to communicate, there are now clearly defined paths that operate in one direction. For example, user input events will be funnelled into the engine by pushing them into the engine's event queue where they will be delegated to the correct place. Similarly, the engine can push animation requests into the rendering queue, where the renderer decides what to do with them.

What's next?

There are still some parts in the renderer which need improvement, so work on that will continue for the next month. To support the new rendering workflow, the renderer needs to provide connector objects, so that the engine can make rendering requests. For these requests, the renderer then has to decide where the objects have to be drawn on screen, what texture to use and potentially handle animation states.

Since the engine is the main user of the renderer, we will also have to work on more basic engine stuff. This will probably involve a few rendering tests, before we actually implement "real" gameplay.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Openage Development News: September 2022

2022-10-01T00:00:00+02:00

What's new

Part 1 - SLD format documentation/parsing

In August the Definitive Edition of AoE2 received an update that changed its default graphics files from the previously used SMX format to the new SLD format. Since the AoE2 devs did not publish any information about the format, we had to reverse engineer it ourselves (together with Tevious from SLX Studio, spriteblood from Overreign and simonsan from the LibreMatch project). You can find the reversed specification in the openage repository (Link).

There are still some unknowns in the format that we don't fully understand, but the specification allows decodung of the most relevant parts: the main graphics, shadows and the player color mask. We still have to figure out how unit outlines and building damage masks are decoded exactly, so you can expect some updates to the linked spec in the future. The openage converter can already read the files and convert them to PNG with help of the singlefile converter

For openage, the change to SLD doesn't change much because we don't use the files directly and instead always convert them to PNG. However, one important thing to note is that the SLD graphics compression is lossy, so the quality of sprites is slightly worse in comparison to the lossless SMX compression. The difference is barely noticable ingame, but if you like zooming in really far, you should probably make a backup of any SMX files in the installation folder.

(SLD layers: main graphics, shadows, damages mask, playcolor in that order)

You can see the difference between SLD and SMX if you zoom in 1500%:

(Left: SLD; Right: SMX)

The SLD is more blocky because it uses a texture compression algorithm operating on 4x4 pixel blocks. Finer details are lost and there's less variety between colours.

Part 2 - GUI and Qt6

Back at the openage engine, there's also been progress, mainly focussed on the GUI framework. The old GUI is "functional" but it does not work great and needs a serious overhaul before we can start slapping new features onto it. Previously, we used a mixture of SDL2 and Qt5 for the GUI, which also required some hacky lines of code to get that working on different platforms. There's also old engine code entangled into some of the classes which makes maintance very unpleasent.

The most important change so far in the new GUI is that we ported Qt5 code to Qt6. Qt6 can handle multiple graphics backends (OpenGL or Vulkan) much better than Qt5 and also made some improvements in terms of cross-platform support. Hopefully, this means we can get rid of a lot of weird and hacky stuff. In the long run, we will probably also get rid of SDL2 (since Qt has mostly the same features) and maybe the codebase will actually be readable :D

Right now there's not much to see, unless you like empty window frames.

Next month, there should be something more interesting that we can show you.

What's next?

There is still a lot to do for the new GUI, so that will also be the focus for next month. After the GUI has been cleaned up, we have to stitch the individual components for graphics output back together (unit rendering, camera movement, terrain drawing, etc.). Once visual output works again, we can start testing the core engine.

There is a chance that we'll also get to work on the gamestate in the engine, although that would probably involve more render tests than actual gameplay. Getting something visible on the screen is more important right now.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org

Engine Core Modules

2021-12-31T00:00:00+01:00

Hello everyone,

It's time for another status report. Our blogpost from last year and the subsequent talk at rC3 dealt with the general handling of game data with nyan and our modding API. This year, we focused on implementing the parts of the engine that actually do something with the data... and also everything else necessary to make it usable. And how hard could that be, right?

Current state

While the gamestate is integral for creating the game mechanics that define the games, other core modules are equally as important for getting things to work. The gamestate module just drives the simulation of the game world. It takes events and handles them based on certain rules (i.e. it is responsible for gameplay). However, it is not concerned with user inputs or outputs. That's what the other modules are for:

Renderer: Shows what's happening in the game on your screen. It receives positional information and graphic assets from the gamestate and decides when and how to display them.
Presenter: Despite its name, it's not only used to "present" things, but also for receiving user inputs. The presenter module manages local user input and output for everything that is not graphics. For example, it translates all the key presses and mouse clicks to game events.
Network: Receives network packages for a multiplayer game and feeds them into the local simulation, i.e. the gamestate and eventsystem modules.
Eventsystem: Manages and tracks events in the engine and ensures that they are properly ordered (the latter is important for receiving network events).
WorldUpdater: Applies events to the game world.

All of these modules have to communicate and work together for creating a functional (multiplayer) game. In openage, the core modules will be loosely coupled, i.e. we try to minimize their dependencies on each other. We do this on the one hand to allow for flexibility for extensions or changes in the future, preparing the engine for low-level modding. On the other hand, we hope a loose coupling keeps the code maintainable and prevents the engine from evolving into a giant spaghetti monster.

Gamestate: How we do ingame things (Recap)

For those still unfamiliar with our game data handling, we recommend the rC3 talk again which goes much more into detail about this. For everyone else, here's a quick refresher on how the general workflow operates.

Ingame units have so-called abilities which define what they can do (e.g. move, attack, gather), what they are (e.g. selectable) or what traits they have (e.g. attributes like HP). Abilities are assigned to a unit by adding the associated Ability object to the unit definition in the modpack data. The Ability object also stores all necessary data related to that ability, e.g. a gather rate for the Gather ability.

In the engine, every Ability object is associated with a gameplay system that can be accessed by the gamestate for unit instances with the assigned ability. This system uses the data from the object, e.g. for Gather the gather rate, to execute an action. In the case of a Gather ability, the action would be that a villager's resource storage is increased by the gather rate value, while the targeted resource spot's resources are decreased.

Gamestate 2: Do many things and do them right

So far so good. In this basic model, every ability has a corresponding gameplay system which, when used, handles the necessary calculation to do the associated gameplay action. However, something that needs to be addressed is that most gameplay mechanics in AoE (and RTS in general) can often not be modelled as one single action. More precisely, gameplay mechanics often involve action routines that are executed in order. For example, an infantry attack in AoE2 actually involves at least two actions:

Move to the target
Attack the target

These routines can also be much more complex. Take the gathering mechanic in AoE2 for example, which we will break down for you.

The player commands the villager to a resource (beginning of the task)
Villager moves to the resource spot (Move action)
- this action continues until the villager arrives at the resource spot
Villager starts gathering the resource (Gather action)
- this action continues until the villager's resource storage is full
Search for a resource dropsite (technically not an action, but important for the routine)
Villager moves to a resource dropsite (Move action)
- this action continues until the villager arrives at the resource dropsite
Villager deposits the gathered resource at the resource dropsite (DropResources action)
Start again at step 2.

As you can see, these routines can get quite complex. In this example, we have actions that end on a condition (step 2, 3 and 5), intermediary steps (step 4) and we even have to integrate a loop (step 7). Mechanics might also have branching paths based on certain conditions, e.g. a farm is only reseeded if the player has enough resources. Other actions, such as health regeneration for the unit, might also happen in parallel.

The solution we chose for this problem is to use flow graphs for handling these complex mechanics (or activities as they are called internally). We take advantage of the event-driven nature of our gamestate implementation here. Starting, ending or cancelling an action always involves an event. With the help of flow graphs, we can define what events start or end actions by mapping the events to paths in the flow graph. When an event happens, the appropriate follow-up action is started. How this would look like can be seen below.

Every action can be represented as a node in the directed flow graph. When a node is visited, the associated action is taken, i.e. the gameplay system executes. Paths between the nodes are mapped to events. The unit - in this case the villager - registers and listens for events that are required for advancing from the current node to the next node. For example, if the current node is step 3, the villager would listen for the event that its resource storage is full. When the event fires, the activity advances to the next node and the next action is taken. Although not depicted here, a node may have several outgoing paths that each map to a different action.

The nice thing about this solution is that it allows for sophisticated control over the flow of actions, while also being reasonably efficient in terms of memory and computation. The flow graph only has to be defined once for every unit type. Sometimes the flow graph for an activity can be the same for an entire class of units. For example, the AoE2 attack mechanics would use the same flow across every military unit. At runtime, the unit only needs to know its current node in the activity's flow graph as well as the events it needs to listen for. Advancing to the next action is a simple lookup for the next node, using the received event.

Renderer

In theory, the renderer module has a simple task. It gets handed animation IDs and positional arguments by the gamestate and draws them on the screen. openage, like AoE2 and other Genie Engine games, uses a 2D renderer, so animations are sequences of 2D images (sprites).

However, in practice the sprites are actually more than plain images. In AoE games, some of the pixels in a sprite are "special" and are used to convey gameplay information. For example, pixels can be marked as player colors or as an outline. How these pixels are displayed depends on the associated gameplay information, e.g. the player ID. This situation imposes two questions for us that we have to address. First of all, we need a way to mark the special pixels, so that the renderer knows they exits. Secondly, the renderer needs to know how to transform the special pixel into a color using the gameplay information.

We address the first problem by encoding special pixels in the pixel data itself. Sprites for openage are stored in PNGs using the 32 Bit RGBA format, where every color channel (red, green, blue and alpha) is 8 Bit wide and thus can have 256 possible values.

PNG 32 Bit RGBA

11001100  10101010  00001111  11111111
--------  --------  --------  --------
^         ^         ^         ^
Red       Green     Blue      Alpha

To mark our special pixels, we change the interpretation of the alpha channel slightly. We reduce the alpha channel to 7 Bits and use the last bit as a marker bit instead. The marker indicates whether the pixel requires special processing.

openage 31 Bit RGBA + marker bit (color)

11001100  10101010  00001111  1111111  1
--------  --------  --------  -------  -
^         ^         ^         ^        ^
Red       Green     Blue      Alpha    marker bit

If the last bit is 1, the pixel is treated as a normal color and would be copied directly into the drawing buffer. If it is 0, then the remaining 7 bits of alpha channel are interpreted as an index for a special draw command. The 24 Bit of the RGB channels may be used as a payload to store additional information such as a palette ID. Based on the command index, the GPU can choose a corresponding code path to draw the special pixel.

openage 31 Bit RGBA + marker bit (command)

00000000  00000000  00000000  1111111  0
--------  --------  --------  -------  -
^         ^         ^         ^        ^
Red       Green     Blue      Alpha    marker bit

This effectively leaves us with 128 usable alpha values for normal colors and 128 assignable draw commands.

Now we need to tell the renderer, or rather the GPU, how to handle these commands. We do this by letting the renderer assign custom shader code chunks to each command index. At load-time, the renderer assembles the code chunks for all commands into the fragment shader that is uploaded to the GPU. When the fragment shader receives pixel data, it will first check the marker bit. If it's a command, then the shader will execute the predefined code chunk passed to the renderer at load-time. A side benefit of handling custom draw commands in shader code is that the commands can potentially be redefined by each modpack, i.e. they are not hardcoded into the engine.

For every command, we can define "associated gameplay data" that should be sent by the gamestate alongside the animation ID. For example, this can be the ID of the owner of a unit (to determine the player color). The information on how to find this data is defined in the modpack and stored in the engine's indexing system at runtime. When a system in the gamestate sends an animation request to the renderer, it will also look up all associated data using the information in the indexing system and attach it to the request.

(The final rendering result after processing all pixels. Red pixels are player color commands, green pixels are outline comands.)

Presenter: Game Control

To make games running on the engine playable by actual people, we have to translate the input events from the player's peripherals to game events usable for the game simulation. As stated in our introduction, this is part of the presenter module's responsibility. The presenter manages the peripherals as well as their configuration.

However, mapping input events to game events is not the only relevant task. Game control is also about the separation of concerns, specifically between ingame and outgame control information. An example for ingame control information is the command queue of a unit. This type of information is directly relevant for the game simulation as it affects the behaviour of units in the game world. On the other hand, outgame information is everything that mostly concerns the person in front of the screen, but does not affect the game world on its own. Example for this are the input devices of a player, the layout of the UI, but also control concepts like a selection queue for units. Outgame information can be used to influence the game simulation, but it should not be required to run it.

So, what's the point of having a separation of concerns here? Our main motivation is to generalize the interface to the game simulation, so that it is not limited to human players. In RTS games, humans are not the only actors that can exercise control. Possible actors could be an AI, a script, or a dedicated server in a multiplayer game. The outgame information (and outgame control) these actors posess can be very different, yet they should all be able to take control of ingame information in their own way. This is especially relevant to AI, since it should not be limited to human control methods.

Part of the solution to this problem is our model of "controllers". A Controller object is a link between the outgame control methods and the game simulation. The main purpose of a controller in this regard is to define the level of access control an outgame actor has over ingame information. For example, a controller may define that a human player has control over an ingame faction with a specific ID. It can also define the permissions the actor has when accessing the ingame information. A spectator may only have "view" access to an ingame faction, while the acting player of said faction has "view" and "action" access. A nice benefit of handling access control like this is that we can easily switch control between ingame factions by modifying or replacing the controller.

The second job of the controller is to map input events from the presenter to ingame events for the game simulation. For this purpose, the controller acts as an interface between the gamestate and presenter modules. The important thing is that the controller merely does the mapping and does not care about the type of input device. That way, all kinds of input events can be translated to the game simulation, whether they come from physical devices, an AI, a script, or a network socket.

Questions

Do you have something to discuss? Visit our subreddit /r/openage or pass by in our discussion board!

And you can reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

New Gamestate 2020

2020-08-15T00:00:00+02:00

Over the past weeks and months, two more important engine components of the engine have become ready to be merged. These components are the new gamestate and the new nyan converter.

The integration of both components marks the beginning of a new development phase as pretty much all major engine core parts will be set in place. We can now use this foundation to build all gameplay-relevant features around them.

What's the plan?

The first thing we will do is merge the new converter. In comparison to the older code, which mainly just dumped the assets into open formats, the new converter is able to create modpacks that use the openage nyan API. It also brings (preliminary) support for the other Genie games Age of Empires 1 and Star Wars: Galactic Battlegrounds.

Next up is the new gamestate PR. This replaces the lockstep implementation we have used in the engine until now with an event-driven system. Furthermore, it makes use of the curve-based simulation model that has been implemented for a while now. Another core part that is coupled with the new gamestate is the new low-level renderer/presenter.

With the integration of the new components, the old gamestate implementation will no longer work. However, this means that the gameplay features that were implemented along with it will no longer work! There might be a lot less visible features for a while. We have debated about leaving the old gamestate intact, but ultimately decided that having parallel structures in the engine is too much work and probably confusing to outside contributors. The old gameplay logic will also not be lost, but rather re-implemented using the paradigms of the new gamestate.

What's the next step?

Once all the code merging, refactoring and documentation updating has been solved, we will start to reimplement gameplay with the Entity-Component-System paradigm. With this approach a lot of features can be added in parallel (although we can't promise that it won't be complicated still). That also means that we do not have to tell new contributors the phrase "you can work on anything, except gameplay" anymore as implementing it is no longer blocked by missing egine core functionality.

For testing features, we will create a simple gameplay demo which will be gradually expanded with new systems. It is supposed to act as a testing bed for the implementation and will probably not model a fully-featured game at first. It's main goal is to give players and contributors an idea of what we are doing... because nobody wants to read long blogposts all the time, am I right? :D

Questions

Do you have something to discuss? Visit our subreddit /r/openage or pass by in our discussion board!

And you can reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

The openage Converter - Part IV: Conclusions

2020-06-24T00:00:00+02:00

With the general converter pipeline explained, there are not that many technical questions left to be answered except the most important ones: Does it work? and How well does it work?. You should be able to guess the answer to the first question yourself if you have read the previous bloposts (Hint: It's "Yes"). The second question is what we want to answer in this blogpost. To do that, we will take a look at the overall performance of the converter, the file size of the created modpacks it generates and how well it is equipped for future improvements.

Performance

The most time-consuming operations in the conversion process are parsing the .dat file in the Reader stage and writing media files during the Export stage. Parsing the AoC 1.0c .dat file will usually take 1-5 minutes, depending on how old your processor is. Media export for AoC is in the same range. In comparison to that, the Genie to nyan format conversion in the Processor is really fast and took a maximum of 3 seconds in my tests. Its impact on the runtime is almost negligeble. All in all, the conversion from AoC to an openage modpack should not take longer than 10 minutes.

It should be noted that the conversion time does not increase linearly with the size of the original game. AoE2: Definitive Edition, which is more than 100x bigger than the AoC 1.0c, will therefore hopefully not take 100x as long to convert. Since the converter does not fully support all media formats of the Definitive Editions yet, we cannot directly compare the runtime right now. However, we estimate that total necessary conversion time will be between 30 minutes and 2 hours.

One remaining challenge is the reduction of the converter's memory consumption. Currently the in-memory storage of the .dat file values consumes a huge amount of space (~360MB for AoC). The reason for this is that we store these values in Python objects along with contextual information necessary for the conversion process (see the first blogpost in the series). In a worst-case scenario, a boolean that has a length 1 Byte in the .dat file will use up to 100 Bytes in its Python object form, even when we employ memory optimizations like the usage of __slots__. This only becomes a real problem with the large dataset of the AoE2: Definitive Edition (~3,800MB), but it's a problem nevertheless.

The solution to the memory problem will be to offload some of the larger structures to temporary files during the Reader stage and load them on-demand during the nyan conversion. This will increase conversion time, but will be less demanding for devices with low memory.

Modpack size

Modpack size is mostly determined by the size of media files, especially the graphics. Most graphics in Genie games use palette-indexed sprites with a color depth of either 8 Bit (AoE1, AoC, SWGB) or 10 Bit (AoE2: Definitive Edition). In comparison to that, openage sprites are stored in PNG files using the 32 Bit RGBA format.

To prevent the increase in color depth from blowing up the file size, we also compress the spritesheets with standard PNG compression. Funnily enough, this can lead to situations where an openage 32 Bit spritesheet has a smaller file size than the uncompressed 8 Bit media file it was converted from. In the end, the openage modpack size will be about even with the size of the original games and could potentially be slightly smaller.

Of course, compression always comes with a price: The time to decompress the spritesheet at runtime before rendering. Unless your PC is from 1999, decompression time is so small that it should not have an impact for the spritesheets of AoC. For the Definitive Editions, which have sprites with higher resolution and frame rates, we will have to see how much decompression affects texture load times. If it is an issue, we are probably able to solve it by implementing predicted asset pre-loading or utilizing block-wise parallel decompression methods.

Mod support

Future releases of the converter should also be able to transform a mod for the original .dat file into a valid mod for the converted AoE2 openage modpack. The challenge here is that mods often not only change existing data; they can also add, remove or rearrange logical entities in the dataset. Thus converting mods will need more than just diffing values from the .dat files, since we have to figure out how the change fits into the openage data model. This could be more or less tricky depending on the features of the mod, but adding mod conversion to the converter will make transitioning to openage much easier for modders used to AoC.

What about the other games?

AoC 1.0c is only one of the games running on the Genie Engine, so what about all the other games? Over the years, there have been 6 different releases that use the engine.

Age of Empires 1 + Rise of Rome (RoR)
Age of Empires 1: Definitive Edition (DE1)
Age of Empires 2 + The Conqueror's (AoC)
Age of Empires 2: HD Edition (HD)
Age of Empires 2: Definitive Edition (DE2)
Star Wars: Galactic Battlegrounds + Clone Campaigns (SWGB)

At the time that this blogpost is published, we have already implemented partial support for 4 of these versions. The code for the AoC processor is also the converter core. The conversion processors for the other supported releases (RoR, DE2 and SWGB) reuse most of the core functionality of the AoC processor. Functions only need to be reimplemented for abilities where behaviour between AoC and the other games is different. This is where our modular approach pays off.

Release	Unique code lines
AoC (core)	38,000
RoR	4,000
DE2 (partial)	3,000 (est.)
SWGB (partial)	8,000 (est.)

(only data conversion)

The DE2 and SWGB conversion processors do not convert all features of the games so far, but already produce functional modpacks. DE2's processor is only missing some of the newer tech effects. SWGB deviates the most from AoC and has its own unique features, e.g. shields and stealth units, that are not supported yet; This is partially the cause of me not being that knowledgable about the game. So if you do know more about SWGB's inner workings, feel free to come by in our chat. We appreciate the extra help :-)

Contact

If you have any questions regarding this blogpost or you are eager to help developing openage make sure to pass by in our discussion board. You can also check the weekly development news on our subreddit!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

The openage Converter - Part III: Convert!

2020-06-16T00:00:00+02:00

In our previous blogpost we introduced the idea of API-like objects which organize logical entities from AoE2 in such a way that we can map their data to the concepts of the openage API. With all preparation done, we can now take the final step and convert the data to our nyan data format.

Iterate!

The approach we take to create the nyan objects is very straightforward: We iterate through all concept groups, check what properties they have and create the associated nyan objects. Since we made sure during the preparation phase that every concept group has information to be processed individually, the transformation to nyan is surprisingly simple. So let's do an example.

A very basic concept group conversion for a GenieUnitLineGroup would look like this:

We check for the properties the unit line could have one by one. A property (or ability as it is called in openage) represents something the unit line is or does, e.g.
- Can it attack?
- Can it build/create other game entities?
- Is it harvestable?
- ...
If a property check is successful, the GenieUnitLineGroup instance is passed to a function in a subprocessor. There, the converter creates the corresponding nyan objects and assigns their member values by mapping the unit values from the .dat file to them. All created nyan objects are stored with the GenieUnitLineGroup instance, so they can be exported to file later.
Repeat this process for every possible property a unit line can have.

The same principle is applied to concept groups for buildings, ambient objects or concept groups that are not even game entities such as techs or civs. The only difference for the latter is that techs check for effect types, not properties. However, the workflow is the same, with the tech being passed to a subprocessor that creates the nyan objects for its effect types.

A subprocessor function in the AoE2 converter are designed to be reusable for the conversion of other games running on the Genie Engine, if they implement the corresponding property in the same way. For example, basic movement properties are the same across all games. The reusability is a huge benefit as it drastically decreases code redundancy and makes implementing converters for other Genie games much simpler.

Request!

Something we didn't talk about so far is the conversion of media files. In the old converter, data and media files (mainly sounds and graphics) were exported separately. The converter would just take a media container format like graphics.drs and dump all its contents into one folder, regardless of whether they were actually used in the game.

This process has been revised for the new converter. The biggest change is that it now only exports media files that were explicitly requested during data conversion. As a result, we can be more selective about the media we want to have. For example, the "main" AoE2 modpack will only contain data relevant for skirmish and multiplayer maps which means scenario or beta units are not included. Since these units are not in the modpack, we also do not generate export requests for their graphics. This ultimately saves a lot of conversion time for players that are not interested in singleplayer.

Integrating the media conversion into the data conversion also enables us to automatically generate context-specific filenames such as idle_archer_animation.png instead of 8.slp.png. Sometimes this doesn't work as well as intended - especially when a media file is shared between units - but it's certainly better than finding volunteers to name 1000 files manually.

Export!

The export step is even more simple than conversion step as we just dump all the data we created. Nyan objects are transformed into their human-readable string representation and printed into files. Media files are converted into open image and sound formats as well as having their metadata extracted into text files. It doesn't get more complicated than that.

However, every modpack also gets a few extra meta formats for glueing the modpack together. One of these is the modpack definition file. This file stores the load configuration data for the engine initialization, e.g. the paths to the included data folders and referenced to other modpacks it may require. It also contains basic descriptive information such as the modpack's human-readable name, a (translated) description or the list of authors.

The other formats are a manifest file optionally accompanied by a cryptographic signature. Every manifest stores a list of hash-filename pairs for every file in the modpack. This can be used to check the integrity of the modpack during load time. Its main purpose is to ensure that all players in a multiplayer game use the same files and thus operate on the same data. Signatures (generated by a trusted authority) can be used to additionally verify that the signed modpack is safe to use, e.g. they don't contain malware or mine bitcoins while you play. However, this is much more relevant for user-created modpacks that use complex scripting than for the modpacks generated by the converter.

To be continued...

There will be one final blogpost in this series where we talk about future plans and considerations for the converter. Keep an eye out for that!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

The openage Converter - Part II: Preparations for Conversion

2020-05-14T00:00:00+02:00

Our last blogpost was all about parsing the AoE2 .dat file and structuring its content into objects that represent the logical entities of the original format (like in Advanced Genie Editor). However, we cannot start with the transition to nyan objects just yet. The reason for this is that the openage API has a different structure than the objects we acquired from the .dat file so far. Thus, an additional refinement is required to before we can do the transition to nyan. How this affects the converter is our topic for today.

Spotting the Differences

Attributes from the .dat file often do not have a direct 1-to-1 mapping to the nyan API. While openage and AoE2 essentially do the same thing on the surface, the underlying structure of the openage API can differ quite significantly from the structure of the logical entities in the Genie Engine. Some of the mechanics were extended or streamlined by us, other had to be altered to fit the Entity-Component System model our API uses. A few examples of different data models are listed below.

Game Mechanic	Genie Engine	openage
Unit Upgrades	Units are replaced with upgraded unit	Unit attributes are upgraded (with patches)
Creation location	Stored in connection or created unit object	Stored with creating unit object
Trebuchet (un)packing	Trebuchet is replaced with (un)packed unit	State change in state machine
HP	Hardcoded behaviour (damage, death, heal)	HP is a configurable attribute

Unit upgrades are especially challenging because AoE2 replaces the whole unit object reference, while openage upgrades only the attributes that actually change. In order to detect which attributes change we need to know the upgrade order of units (also called a unit line) and create a diff between neighboring units in that line. Replacement mechanics are also used in other ways, e.g. for the trebuchet, which requires its own special treatment.

Other mechanics, such as the creation location of buildings or units, can sometimes be annoying to detect when we process units one at a time. For example, there is no way to see which buildings a villager can create by looking at the villager object alone.

Concept Grouping

The first step we take to solve the described problems is to use concept grouping. The idea behind this method is that even though the data model of the Genie Engine and openage can be very different, the gameplay mechanics/concepts stay the same. One example of such a concept would be the aforementioned unit line, while a concept group is a specific instance of the concept, e.g. the archer unit line in AoE2. Examples for other concepts are age upgrades, civs or transform groups (trebuchets).

In the converter, the concepts are implemented as Python classes whose instances become the concept groups. The classes usually implement methods that can figure out additional context about the groups, e.g. whether a unit line is unique or what position every unit in the line has. To establish the concept groups, the converter iterates through every logical entity, finds the related concept and adds the entity to its corresponding group.

(The above image shows an UML excerpt from the Python class GenieUnitLineGroup (for - you guessed it - the unit line concept) on the left as well as an example instance in form of the archer line. The instance stores the units that are part of the line as an ordered list of GenieUnit objects which you should be familiar with from the last blogpost. Methods from the class are available for every instance and help us determine a line's properties.)

One big advantage of concept groups is that every concept does have a 1-to-1 mapping from AoE2 to openage; something that a single logical entity did not necessarily provide. The groups' Python classes provide an abstract view on the concepts in this case which translate the Genie data model to the openage data model. Hence, you can also interpret the created concept groups as API-like objects that represent an openage API concept using Genie Engine data. By operating on the groups during nyan conversion, the converter does not have to worry about the context of individual logical entities anymore and can concentrate solely on fetching the necessary data from the groups.

Linking

The purpose of linking is to add information about the relation to other groups to a concept group. We mostly do this for relations that would be hard to detect from just looking at one concept group because it is not stored as a value in its logical entities. An example for this that we mentioned before are the buildings a villager creates. The links in a concept group are technically just a bunch of concept-specific lists in the Python classes. These lists are filled by iterating through all concept groups we created, each time looking for a relation to another group and appending the group to the list asspociated with the relation type.

(This is the same GenieUnitLineGroup class as shown above, this time with linked information visible. creates stores the building lines created by the villager, garrison_locations the places where it can garrison. Note that these links are also stored as lists, but in contrast to line which stores references to GenieUnits, links will usually reference group instances.)

Linking is not as important as grouping, but it saves us a lot of runtime when a relation is relavant more than once. It also ensures that every concept group has all information it needs for conversion available in the Python object instance and (theoretically) does not need to search the rest of the dataset for information.

Now that we have prepared the data for conversion, we can finally start to translate it to the nyan API.

To be continued...

The series will continue soon with another blogpost about the converter. Next time we will start to look at the most important step in the conversion process: the nyan object creation.

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

The openage Converter - Part I: Reading Data

2020-03-13T00:00:00+01:00

In our previous blogpost series about the openage modding API, we outlined how AoE-like gameplay mechanics will be implemented in openage and how they are modeled in our dedicated database language nyan. However, one issue with creating a new modding interface from the ground up is that we cannot directly use the data files from the original games. We have to convert them to our own logic and formats for them to work with the openage engine. This process is what this new blogpost series will be about.

Today's topic will be the overall structure of the new Python converter and its data reading module. There will be at least two more blogposts in the next week, covering the transformation from AoE2 structs to nyan API objects and the export to the file formats.

Converter Workflow

Every game or game edition we convert will go through 3 different stages: Reader, Processor and Exporter.

In the Reader, we parse data or media files and put them into a structured format that we can use during conversion. The content read is passed onto the Processor where the actual conversion takes place. Here, the data are first organized into conversion objects that resemble the original games structure (AoE-like objects). These are transitioned into objects that model the openage API (API-like objects) and then put into nyan API objects that can be exported to file. An Exporter will take the converted objects and sort them into modpacks. The very last stage of the conversion is printing the objects to nyan files and other human-readable formats along with requested media files.

Every stage is modular which means it can easily be exchanged or replaced. This comes in handy when we have to consider the multiple releases of AoE1 and AoE2. For example, the Definitive Edition of AoE1 will use an extended Processor stage compared to the original version of AoE1 due to the presence of new features and different mechanics. With the modular approach we can do a lot of code sharing and do not have to build a new converter for every edition of a game.

Note that from this point onwards we will exclusively talk about data conversion, or more specifically, converting gamedata from the empires*.dat files. Media conversion is a separate topic that we will leave out for now and maybe describe in an additional blogpost in the future.

Reading and Storing

Our first goal in the conversion process is to access the data and load it into the converter in a usable format. Since AoE2's .dat file uses a binary format, we have to serialize the values before we can do anything with them. In this case, serialization means that we traverse the file byte by byte, extracting the values and storing them as Python structs. The format has been well documented over the years, so we know exactly at which offset what attribute can be found.

When we encounter an attribute value in the .dat file we have to worry about two things: How we read it and how we store it. The distinction between those two is a result of the different contraints values from the .dat file and values in our own database language nyan have. For example - depending on the attribute - integers in the .dat file can differ in:

Length: An integer may be stored with length 8 Bit (int8), 16 Bit (int16) or 32 Bit (int32).
Signing: Integers may have 1 prefix Bit to signal a negative value (signed int) or not (unsigned int).

Therefore, we have to specify the correct read type for every attribute in the .dat. Otherwise we would get the wrong values.

Additionally, even if two attributes have the same read type, we might want to use a different storage type depending on the context the attribute is used in. For example, the converter used different storage types for attributes representing unit stats (e.g. HP, movement speed, damage vaues, ...) and resource IDs (e.g. sound or animation IDs). This becomes relevant when we want to compare two units later on to calculate upgrades or when we want to determine the differences between vanilla AoE2 and a user made data mod.

In our code, the serialization of the attributes looks like this:

dataformat.extend([
    ...
    (READ, "hit_points", StorageType.INT_MEMBER, "int16_t"),
    (READ, "line_of_sight", StorageType.FLOAT_MEMBER, "float"),
    ...
    (READ, "dead_unit_id", StorageType.ID_MEMBER, "int16_t"),
    ...
    (IGNORE, "hidden_in_editor", StorageType.BOOLEAN_MEMBER, "int8_t"),
    (IGNORE, "old_portrait_icon_id", StorageType.ID_MEMBER, "int16_t"),
    (READ, "enabled", StorageType.BOOLEAN_MEMBER, "int8_t"),
    ...
    (READ, "resource_cost", StorageType.ARRAY_CONTAINER, SubdataMember(
        ref_type=ResourceCost,
        length=3,
    )),
    ...
])

For every attribute in the .dat file, the Reader module contains a 4-tuple that stores its read mode, attribute name, storage type and read type.

Read Mode

This specifies whether we want to actually read the value (READ) or skip over it (IGNORE). The latter read mode is used for values that are irrelevant for conversion. A lot of the time these are leftovers from old beta versions (e.g. old_portrait_icon_id).

Attribute Name

A human-readable name for the attribute. This name will be used in the Processor stage to address them and read their values.

Storage Type

Here we tell the Reader which storage type it should use for an attribute. Every type corresponds to a ValueMember subclass object that stores the attribute name, its value and its storage type. ValueMember also implements an auxiliary method for diffing.

Read Type

This determines what read type we have to use to extract the correct attribute value from the .dat file. We can either refer to primitive types such as integers or floats, but we are also able to use nested dataformat definitions, like ResourceCost in the code sample above. Nested definitions contain another list of 4-tuples which are read sequentially. Employing these definitions is very helpful for repeating structs as it lets us define its dataformat once and read it as many times as we want.

Organizing

After reading the .dat file, we receive a full representation of its content in an array of ValueMember instances. While the data dump we receive from the Reader already contains all the attributes and values we need later on, this output is too unstructured to be handled efficiently yet. If possible, we would like to operate on logical entities we know from the game like units, buildings, civs or terrains, instead of cherry picking from a giant array of attributes.

To do this, we transform the dumped values into converter objects that resemble the structure used in the original engine (hence AoE-like objects). These objects (GenieObjects) are very similar to what you would see in Advanced Genie Editor, just without the fancy GUI. For example, we put every ValueMember attribute belonging to an AoE2 unit into a GenieUnit object. The same process is repeated for every unit and every other logical structure that we can recognize in the game. Afterwards, each attribute will be assigned to an appropriate GenieObject subclass. Every subclass instances receives its own list in a container which can be accessed in the Processor stage.

Not only is this object-oriented approach much easier to work with, we also no longer have to worry about where exactly logical entities, e.g. units, are located in the dump. Thus we can focus purely on its content and leave the legacy of the .dat file structure behind.

To be continued...

That's it for now. We will continue next time when we transform data from the older AoE-like structure to the openage API.

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

D14: Openage modding API - Finale

2019-06-26T00:00:00+02:00

Over the last year, we have introduced a lot of features to our API and spent a lot of time polishing and improving its core concepts. But as an old Chinese saying goes: "You have to let go, otherwise you will never finish the converter and jj will be unhappy". So today, we finally present you the first release version of the openage Modding API tree and will talk a little bit about the features that come with it.

We present you the result of hours of thought and discussions (Open picture in new tab to enlarge):

Our Feature Set

As a baseline, everything that is and was possible in previous Genie Engine release versions will also be supported in the openage modding API. This includes all features from Age of Empires 1, Age of Empires 2 and Star Wars: Galactic Battlegrounds. Most of them have been reworked to provide better accessibility to modding, but the original behaviour can always be replicated.

Additionally, the API provides these new exciting possibilities:

Modular ability and bonus system for units, buildings and other game objects
Every ingame object, whether it is a unit, a building or part of the ambient scenery, is able to use any ability the API provides. This makes units much more flexible and enables modders to implement behaviour known from other games, e.g. Age of Mythology, such as mobile/movable dropoff points and military units constructing buildings. And of course other more crazy combinations are also possible:

Trading with trees or birds
Shooting houses as projectiles
Let relics research special techs and fight in combat

Nonlinear tech trees
Techs can be unlocked through alternative paths and are not necessarily bound to Age Upgrades. The upgrade path can be different for every civilization or game mode. It also allows giving players the choice to select one out of two (or more) upgrades for a unit, e.g. a decision between more attack damage and faster reload speed.

Proper inventories and item systems
Relics will no longer replace the monk unit when picked up. Instead, the API unifies garrisoning, transporting and inventories with a new container system that give units the ability to store other game entities properly. Items and garrisoned units are allowed to grant new abilities or remove existing ones.

Enable different abilities depending on the unit's state
Like the trebuchet from AoE2, all game objects can have multiple states which activate other abilities and give different boni or mali. Modders will be able to define as many transformation states as they like. Additionally, construction, damage and harvest progress are able to change the state of the unit. For example, this can be used to make a building's armor weaker when it is not fully constructed.

Combine effects such as damage, convert or heal
Effects in the API are just as modular as our ability and modifier system. Any unit can apply any effect and is allowed to combine them freely. As an example, modders will be able to create attacks that do damage and also have a chance to convert the unit.

Techs and civ bonuses affect more diplomatic stances
In AoE, your civ choice grants bonuses to the whole team. But what if your enemies would profit too? In our API, civs are allowed to define modifiers for other players that have a different stance than "friendly". Additionally, techs can influence not only yourself, but also others on the same map. Abilities and effects are also allowed to be limited to specific diplomatic stances which enables more specialized diplomacy such as overlord-vassal relationships.

These are the most prominent features that we support. There are a lot of other helpful features that were either too small or too complex to mention. Be sure to check out our previous blogposts if you are interested in more thorough explanations of the underlying mechanics.

What happens next?

We will stop thinking about new features for a while and integrate the existing API into the engine. The core engine parts that need to be worked on are the core gamestate and the converter. The gamestate needs to be able to understand the API objects and implement the behavior for abilities and modifiers. Our converter can already read the .dat files of the original game and now needs to sort the old data into our nyan API tree.

Roadmap for integration:

The converter will be changed to create nyan API objects and assemble them as a modpack
In parallel, the engine will be extended to load API objects and implement their intended behavior
Once everything works, write a modding guide and tutorials

Let's do it!

Trivia

As a bonus, here is nice collection of fun and pointless facts about the API and its development.

The initial attempt at the API (April/May 2018):

The API contains 282 individual objects of which 58 are abilities, 34 are modifiers and 13 are effects.
There are 36 passive and 22 active abilities (considering their functionality).

Member type	Amount
bool	10
int	47
float	59
text	5
file	3
set	126
orderedset	9
nyan::Object	130
Total	389

The only ability from the first draft that survived until today is Fly. All other abilities have been reworked.
ShootProjectile has the most members (18) of any API object.
The object with the longest name is RelativeProjectileAmountModifier (32 characters). In contrast, the shortest object name is Mod (3 characters).
The maximum depth of the API tree is 5 which is the longest inheritance distance between the root object Entity and the children of FlatAttributeChangeModifier.
Even though nyan supports multiple inheritance, none of the API objects currently use it (this was not always the case). The small number of dependencies makes the API more robust to changes. However, mods using the API are expected to make extensive use of multi-inheritance.
The lack of multiple inheritance means that the API is indeed a tree and not just a graph.
The number of units from AoE2 will be reduced from 58 to 28 (excluding unique and cheat units) in the API. This is because we do not store upgraded units as separate ojects, but as patches to the original unit.

Questions?

Questions? We probably know the answer! Ask us something about the blogpost by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

D13: Openage modding API - Diplomatic Stances (+ more updates)

2019-03-01T00:00:00+01:00

It's time for another update on the latest changes and this time, they primarily deal with the relationships between players. In previous blogposts, we always assumed that an ability already indirectly defines whether it can be used against friend or foe. However, this is often not as obvious as it seems, especially after the introduction of the ApplyEffect ability (see our last blogpost) which can be used for friendly actions like healing or hostile actions such as attacking. Our solution to this problem is the addition of diplomatic stances which we will now discuss in detail.

Other interesting articles in the modding API series:

View all articles

Diplomacy and abilities

The core concept behind diplomatic stances is that they decide what abilities can be used on another player's units. This is realised through the new DiplomaticAbility API object. Any ability that inherits from DiplomaticAbility can only be used with/against units of players with the corresponding diplomatic stances defined in the member stances. A really simple example is the Attack ability from the example above that can only be used against units from players where the stance is set to Enemy.

It is important to stress that the openage API only defines one built-in stance (Self). Self is a special stance that implies an abilitiy can be used on the player's own units, if added to the set of stances of a DiplomaticAbility. This can be used for active abilities like Heal, but also for more passive ones like DropSite which enables a GameEntity to be a resource dropoff point. All other stances have to be defined by the mod or game they are part of. In the case of AoE2 these stances would be Ally, Neutral, Enemy and possibly Gaia for the special relationship with wildlife.

Although players in AoE2 usually have the same stance towards each other, there is the potential to be more creative with diplomacy. For example, the stance system makes it possible to have vassal-overlord relationships where the overlord can use special abilities on its vassal, e.g. forcefully conscript the subject's units or collect tribute.

It should be noted that inheriting from DiplomaticAbility is entirely optional and by default, every ability works with all diplomatic stances.

Diplomacy elsewhere

Abilities are not the only useful application for diplomatic stances. Another use case are the Effects that we introduced last time. If you recall, effects define a form of interaction (damaging, healing, converting, etc.) between two or more game entities. In some situations, the effect should only apply to units of players with a specific stance. For example, the kamikaze attacks of the petard and the demolition ship only do damage to enemy units. Similarly, projectiles usually do not damage friendly units with the exception of onager shots.

The problem is solved by introducing a DiplomaticEffect that works in the same way as DiplomaticAbility. By making an Effect inherit from DiplomaticEffect, the effect will now only be applied to units of players that are covered by the diplomatic stances in the stances member. Effects that do not inherit from DiplomaticEffect will be applied regardless of the stance of the target.

On the side of smaller changes, there is the new DiplomaticPatch type. Before, patches in a Tech only upgraded the game entities of the player who researched the technology. DiplomaticPatch lets you define upgrades that can also apply to other players. These could be very fun to experiment with, as technologies are now able to influence enemies and allies alike.

The Civilization object previously contained a team_setup member that stored the team boni that were applied to other team members in AoE2 games. Since the API now supports diplomatic stances directly, we have opted for a more fine-grained and flexible solution. team_setup is superseded by the DiplomaticSetup objects in diplo_setup. diplo_setup can store as many objects as necessary, so it is possible to define a setup for every existing stance, if required. As an interesting side effect, a civilization could also give boni/mali to its enemies.

Side notes (Container ability)

Last but not least, there is something else to discuss which is somewhat related to diplomatic stances. In blogpost No. 7 we introduced the Inventory ability to provide a way for units to store items like the relic rather than merging with them. Back then, Reddit-user Kaligule pointed out that transport ships, rams and siege towers (and garrisoning in general) could potentially be modelled the same way. However, at the time the API could not have supported this because Inventory relied on items being neutral game entities that did not belong to a specific player.

With the addition of diplomatic stances, there are no more excuses to keep Inventory and Garrison as separate abilities. Hence, they were merged into the Storage ability. Storage is essentially the same as Inventory, so there is nothing special to tell about it that was not discussed before. A side effect of Storage being a DiplomaticAbility is that you can put other player's units in inventories now, which will make creating Pokemon game modes for AoE2 a bit easier.

So as always: Do not hesitate to suggest improvement ideas on Reddit. It's encouraged to question the devs, it just takes long to implement :)

Questions?

Do you still have questions? Then let us know and discuss them with us and the community by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

Openage modding API - Effects

2019-01-27T00:00:00+01:00

A recent discussion on how the Convert ability should work brought up the question how we want to handle interaction between different game entities. In our previous drafts, every interaction had its own ability. This aproach works, but inevitably creates redundancy. We introduced a new mechanic in form of Effect objects to address these issues and hopefully make the engine even more extensible along the way.

Other interesting articles in the modding API series:

View all articles

Many abilities, many overlaps

The main problem with our interaction abilities is the lack of consistency in their mechanics. Some -- like Heal or Repair -- are one-sided as the outcome is solely decided by the unit that executes the ability, while others such as Attack are two-sided with the final attack value depending on both the attacker and the defender. Also, there are specialized abilities for the purpose of activating a ResourceSpot like the Hunt ability. And of course there's Convert with weird and ridiculous rules (thanks go out to Jineapple who figured all of this out).

One major drawback of having different behaviors for every interaction is that they are not compatible with each other. It is not possible to have an Attack that does damage and has a chance to convert for example. The idea behind the reorganization of abilities into an effect-based system is to eliminate these incompatabilities and to make combinations of different effects possible.

Effect and Resistance

The new system draws heavy inspiration from the old attack definition described in blogpost No. 5. It is recommended that you read the whole article if you haven't already.

Every interaction is now two-sided and modeled through a pair of Effect and Resistance. The Effects are defined by the applicant of the effect, while the Resistance is always defined by the unit the effect is applied on. For convenience sake, we will call the two sides "effector" and "resistor" from now on.

The mechanics that were previously tied to abilities are now defined as generic effects and resistances, albeit with different names. For example, the ArmorAttack and ArmorDefense objects previously used for the Attack ability are now covered by FlatAttributeChange (on the left) and FlatAttributeResistance (on the right). Despite the bulky object names, attacking still works the same way as before. Newly added are ChanceEffects, which can be used for conversion, and MakeHarvestable that makes ResourceSpots accessible to villagers.

One important mechanic that was carried over from the attack system is that the application of an Effect by an effector always requires the corresponding Resistance on the resistor's side. For example, if a resistor has no ConversionResistance defined, it cannot be converted with the Conversion effect. This is slightly counterintuitive, but makes giving units immunities much easier. If you want a conversion with zero resistance, this can instead be done by assigning a ConversionResistance object with chance_resist set to 0 to the resistor.

The Effect and Resistance objects can usually be used both ways. Either for the benefit or for the disadvantage of the targeted game entity. For example, setting a positive value for change_value in FlatAttributeChange will damage a unit, while a negative value essentially be a heal. To prevent a heal from accidently damaging the resistor, modders can define minimum and maximum value limits for an Effect. This is entirely optional, so you can also choose not to and let everything go haywire.

Calculation examples can be found in the Addenndum section at the bottom of the article.

Discrete vs. Continuous

The current effects can come in two forms, discrete and continuous. DiscreteEffects are applied immediatly at a specific point in time. ContinuousEffects on the other hand happen over time at a defined rate (e.g. reduction of attribute points per second). The differentiation is necessary because an ability using DiscreteEffect needs to define the application interval which is not necessary for ContinuousEffect. Examples for the application of a ContinuousEffect in Age of Empires include healing and repairing.

The new abilities

With the introduction of effects, the interaction abilities are subject to a lot of changes, too. As we can cover most of them with an Effect, the Convert, Heal and Repair ability do not need to be part of the API anymore, indicated by their white color. Instead, they are from now on handled as derivatives of ApplyDiscreteEffect and ApplyContinuousEffect shown in the centre of the diagram. For example, implementing the old Heal ability would be realized by inheriting ApplyContinuousEffect and defining one or more continuous FlatAttributeChange effects. A dedicated Heal ability is unnecessary as the needed definitions are stored in the effect. The only specific ability remaining is DiscreteAttack because the API does not have a concept of diplomatic stances yet which is required for recognizing friendly fire (although this could change in a later API draft). Resistances are stored in the Resistance ability (shown in the upper right corner).

For every general application ability there also are two more specialized API abilities available: AreaOfXXXEffect and RangedXXXEffect. With RangedXXXEffect one can define the minimum distance to a unit for the application of effects. AreaOfXXXEffect applies the effects in an area around the effector. This was previously only modeled for attacking, but now it is available for all effects. This means that you can make super monks that convert everything around them. Neat! Because effects is a set member, all abilities are able to bundle and apply multiple effects at once.

The greatest advantage of this system is that it has eliminated the incompatibilities mentioned in the beginning. Delegating the behavior to individual effects allows us to derive abilities that combine them freely without being restricted to a specific type of effect. Theoretically, an ability can even apply both continuous and discrete effects by inheriting from ApplyContinuousEffect and ApplyDiscreteEffect simultaneously.

More to come..?

The next logical step to improve the effect system would obviously be the definition of even more effect and resistance types. Right now, the API only features 4 concrete effects and resistances and there is room for more. Extensions could be damage-over-time effects like Poison or Fire Damage as well as more creative effects like a Life Steal. Another addition would be allowing effects to attach modifiers to other units that give temporary buffs or debuffs, e.g. a Disease.

There is also the question remaining whether Gather and Build are supposed to be effects or if they should be left as separate abilities. Both of them are candidates as they define a form of interaction, but modeling them as effects might require further depature from the way they were implemented in AoE2. Therefore, we will postpone the decision to a later date.

Questions?

Hint: You can suggest your own improvement ideas on Reddit. It's not forbidden to question the devs.

Do you still have questions? Then let us know and discuss them with us and the community by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on freenode.net

Addendum: Example Calculations

Defining and calculating the result of an effect-resistance pairing is very easy and straight forward. Look at the example below for a simple interaction effect involving FlatAttributeChange.

# Effector's side
MeleeAttack(FlatAttributeChange):
    type = MeleeArmor
    min_change_value = 0
    max_change_value = None
    change_value = AttackAmount

    AttackAmount(AttributeAmount):
        type = Health
        amount = 4

    ignore_protection = {}

###################################

# Resistor's side
MeleeResistance(FlatAttributeResistance):
    type = MeleeArmor
    block_value = BlockAmount

    BlockAmount(AttributeAmount):
        type = Health
        amount = 1

The effector's attack does 4 health points of damage and the resistor blocks 1 damage, so the overall damage done is 3. Keep in mind that Effect and Resistance have to match up for this to work. For FlatAttributeChange, the match is defined by the type member of FlatAttributeChange/FlatAttributeResistance and the type member of AttributeAmount. An example where the pair doesn't match up can be seen below.

# Effector's side
MeleeAttack(FlatAttributeChange):
    type = MeleeArmor
    min_change_value = 0
    max_change_value = None
    change_value = AttackAmount

    AttackAmount(AttributeAmount):
        type = Health
        amount = 4

    ignore_protection = {}

###################################

# Resistor's side
MeleeResistance(FlatAttributeResistance):
    type = MeleeArmor
    block_value = BlockAmount

    BlockAmount(AttributeAmount):
        type = Faith                   # <-- Not a match
        amount = 1

The type of AttributeAmount in the effect does not match the type of AttributeAmount in the resistance. In this case the resistor is immune to the MeleeAttack effect, but usually effector's apply more than one effect and resistor's have more than one resistance.

# Effector's side
MeleeAttack(FlatAttributeChange):
    type = MeleeArmor
    min_change_value = 0
    max_change_value = None
    change_value = AttackAmount

    AttackAmount(AttributeAmount):
        type = Health
        amount = 4

    ignore_protection = {}

PierceAttack(FlatAttributeChange):
    type = PierceArmor
    min_change_value = 0
    max_change_value = None
    change_value = AttackAmount

    AttackAmount(AttributeAmount):
        type = Health
        amount = 2

    ignore_protection = {}

Convert(ChanceEffect):
    type = Conversion
    chance_success = 0.5
    cost_fail = None

###################################

# Resistor's side
MeleeResistance(FlatAttributeResistance):
    type = MeleeArmor
    block_value = BlockAmount

    BlockAmount(AttributeAmount):
        type = Faith # No match
        amount = 1

PierceResistance(FlatAttributeResistance):
    type = PierceArmor
    block_value = BlockAmount

    BlockAmount(AttributeAmount):
        type = Health
        amount = 5

ConversionResistance(ChanceResistance):
    type = Conversion
    chance_resist = 0.1

This time, multiple effects come into play. The effect MeleeAttack attack has no match on the resistor's side, so it is ignored.

PierceAttack does match up with PierceResistance, so the damage can be calculated. With a damage value of 2 and a block value of 5, the overall change to the resistor's health would be -3, which is equivalent a heal of 3. Because the effector has initialized min_change_value with 0, the change is rounded up to that number. As a result 0 pierce damage is done.

Convert also has a match (by type) on the resistor's side in form of ConversionResistance. The chance is calculated by substracting chance_resist from chance_success, resulting in an overall chance of 0.4 (40%) for the effector to be successful.

D11: Openage modding API - Additions and Updates

2018-12-24T00:00:00+01:00

Throughout the last months we made several corrections and additions affecting the modding API that were too small to discuss in one blogpost. Since quite a few changes accumulated though, we decided to bundle them and wrap up the year with a collection of mini blogposts.

Other articles in the modding API series:

Game entity

From all of our API objects GameEntity saw the most drastic changes. Firstly, the members for hitbox and language data were moved to abilities, while the icon is now part of the nyan UI modding API (which is under construction at the moment). Building also had unique members which were moved to the five new abilities Constructable, DamageableBuilbing, TileRequirement, Foundation and TerrainRequirement. We hope that by outsourcing these members to abilities, game entities will become even more versatile and less restrictive in their definition.

Secondly, we introduced GameEntityType referenced by the member types which is a new object that can be used for a broader classification of game entities. A game entity can be of several GameEntityTypes like Infantry or Archer or both. In comparison to the classes in AoE2, GameEntityType is not tied to armor and acts as an independent mechanism.

Last but not least, we want to welcome Projectile as the fifth GameEntity category. In the 2. blogpost we declared that Projectile should not be a GameEntity because it does not operate independently and always needs a host unit that shoots it. Howerever, we came to the conclusion that just because projectiles are dependent on their host in AoE2, this doesn't necessarily have to be the case in openage. Hence, the Projectile object has emancipated itself and can now be used for other things than attacking, e.g. converting, resource gathering, repairing and more.

Resource Contingents

Another addition specifically designed for population space is ResourceContingent. Contingents are resources with the additional feature that they can be used and provided temporarily. In context of population space, this means that units use 1 PopulationSpace resource while they are alive and free it again when they have died. Similarly, houses provide 5 PopulationSpace once they are built which is available as long as they are not destroyed.

Because ResourceContingent is of type Resource, contingents can potentially be gathered and thus permanently added to the resource pool, too. This could lead to some interesting dual-use mechanics where a resource contingent is gathered first and then temporarily used. For example, one could define a resource contingent Grain which has to be collected from farms, while the amount of grain is also used as the population limit. The player can then choose between

Keep the Grain for additional population space, thus being able to field more units
Sell Grain for Gold and therefore lose room for population, but gain the potential to buy better upgrades for their units.

Attributes

Age of Empires 2 is not known for giving its units many attributes except HP, yet many other RTS games are more generous in that regard. Examples are mana, stamina (Battle Realms), "special" power (AoM, EE), shield (SWGB) and so on and so forth. So, instead of creating an ability for every single attribute RTS designers thought of, we generalize them with the Attribute API object. The definition of attributes is very similar to that of Resource which makes sense as attributes technically are resources, although only available in the scope of the unit. Abilities that cost attribute points reference AttributeAmount which is an attribute's counterpart to ResourceAmount.

For AoE2, two attributes are needed in total: Health and Faith. Health should be self-explanatory, Faith is used by monks to convert units. Faith is also a RegeneratingAttribute that is refilled at a defined rate. In preparation for SWGB support, we also added ProtectingAttribute which shields another attribute from damage. For example, if Shield protects Health, then the attribute points of the shield will be substracted first in case of an attack. Once the attribute points of Shield reach 0, the Health points of the unit start to get subtracted. Of course, we can also build longer protection chains with 3 or more attributes.

Introducing the attribute system required reworking attacks and armor. Now ArmorDefense is always connected to a specific attribute through AttributeAmount. ArmorAttack had a similar change so that every attack value is targeted at one specfic attribute. Optionally, ArmorAttack is allowed to ignore protections from the ignore_protections set. As a result, we can now have attacks in SWGB that only affect shields or ignore the shield entirely and damage the HP directly.

Training and building

At last, we have made a small change to how the cost and creation time of units and buildings were managed. Before, cost and creation time were decided by the created unit through the Creatable ability. In our newest API draft, we delegated the decision to the creating unit or more precisely, to the Train and Build abilities. These reference a CreatableGameEntity object which stores cost, creation time, requirements and the unit/building that is created. This slight alteration should give more freedom to pricing when the same unit is produced in several different buildings.

Questions?

That's it for this time, but there will probably be more to add, change or discuss about soon.

Did we leave something out or are we forgetting something important in our API? Shout at us (and discuss your ideas) by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

D10: Openage modding API - Restocking farms

2018-11-25T00:00:00+01:00

Food takes quite an unusual role in the AoE economy as its main harvesting source from the mid-game onwards - the farm - is a renewable resource spot. Other than the natural resources that are generated with the map, the farming economy relies on the constant reconstruction of the production building by the player. In this blogpost we will explain how resource spots can be restocked in openage and also how we implement the process of queueing farms in the mill.

Other articles in the modding API series:

View all articles

Preliminary analysis

Before go into detail about the concrete API implementation, we should make clear how the farming economy from AoE2 works on an abstract level. For that purpose we prepared a simple diagram.

Note that we leave out unimportant aspects such as delivering food to a drop site or deleting the building. We focus solely on the standard life cycle of the farm. A stylized hand indicates that the task is initiated by the player through the GUI, while tasks with a stylized person are automatically executed by a villager.

As seen above, the process of harvesting a farm can be divided into these steps:

A farm's life cycle starts with the player placing and constructing it (needs 60 wood) with a villager.
One villager (more are not allowed) gathers food from the farm until it is depleted.
Then one of three scenarios can happen:
1. There is a farm queued in the mill at the moment the farm depletes. In this case, the villager automatically rebuilds it and 1 farm is removed from the queue. They automatically resume gathering. This is a feature introduced in the first expansion The Conqueror's.
2. The player rebuilds the farm manually by right-clicking on it with a villager. This will cost them 60 wood. They automatically resume gathering.
3. If 3 minutes of ingame time passed, no farm was queued and the player did not manually rebuild the farm, the depleted farm is deleted.

Streamlining the process

One problem that remains is that the process described in the previous section is still very specific to farms. As we are writing an API, we would like to generalize it as much as possible. Particularly, we would like to fulfill these requirements:

Restocking resources should be possible for every ResourceSpot, not just farms. Trees and berry bushes are candidates where renewability would make sense.
Similar to what we said in previous blogposts, the game entity containing the ResourceSpot should not be replaced when we restock it. The action should only refill the resources.
Queuing restocks should also be very easy to control through the API. Modders should have the option to enable and disable automatic restocks.

With this in mind, we can now look at the API implementation.

Restock

There are two abilities involved in the restocking process. On the left side we have the ability Harvestable of the farm, which contains the ResourceSpot that villagers can gather from. Villagers get the Restock ability with which they can fill up the resources again.

Restock targets a specilization of ResourceSpot which is called RestockableResourceSpot. A restockable resource spot works like a normal resource spot, except that it is not immediatly destructed after the resources are harvested. Instead, it will be depleted until it is either restocked or the time set in destruction_time_limit has run out. We can also decide with auto_restock whether queued/automatic restocks are allowed. The two sets of Progress objects contain graphics that are used when a certain percentage of restocking is complete. It is important to stress that normal ResourceSpots can be turned into RestockableResourceSpots through patching.

The Restock ability can be used on a restockable resource spot after its contents are depleted. Though its attributes look very similar to those of Build, the two abilities operate completely independent from each other. Restock is only concerned about the resource spot it is supposed to refill (target) and will not reconstruct the whole building. The necessary time to restock is stored in restock_time. We can also set the amount of resources that should be restocked. The costs for a restock action is split into two options: manual_cost is used when the player initiates the restock (like right-clicking on a depleted farm), while auto_cost is used when villagers automatically execute the restock, provided the resource spot allows this. Both costs can be the same, but it can be handy to set them to different values as we will see now.

Queuing farms

When we implement our Restock ability for the resource spot of AoE2 farms, we have to initialize manual_cost and auto_cost. The naïve approach would be to interpret both cases as "build farm" actions and set both values to 60 wood. But this would not suffice. As we can see in our first diagram, the automatic restock does not really cost 60 wood, it costs one farm from the queue in the mill. That begs the question of how we actually queue farms. Fortunately for us, it turns out to be quite trivial.

First of all, the farm queue does not exactly represent buildings. It is probably closer to a "I can rebuild my farm"-ticket for the villager. Secondly, these tickets are available to all villagers and are not tied to a specific mill. They can be used from everywhere on the map on a first-come, first-serve basis. If you think about it, these tickets work like the other four resources in the game, with the exception that they cannot be gathered by the villagers. Thus, we are going to interpret the elements in the queue as actual resources. We will introduce a fifth Resource called FarmRestockResource which will work exactly like Food, Wood, Gold and Stone.

Since FarmRestock will probably not be gatherable on the map (although that is possible), we have to find another way to acquire it. For that purpose, we can use the TradeResource ability which is also used for selling and buying with the market. In the context of farming, the mill will get a TradeResource ability which sells 60 units of Wood for 1 unit of FarmRestockResource. We can then set auto_cost in the villager's Restock ability to 1 unit of FarmRestockResource which is therfore consumed whenever a villager automatically restocks a farm.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

D9: Openage modding API - Civilizations and Uniqueness

2018-09-20T00:00:00+02:00

Civilizations (also called Factions or Races in other games) are a concept as old as Dune 2. Their purpose is to add strategic complexity by inserting a layer of asymmetry to a game, thus enabling a variety of unique play styles and increasing replayability. How we are handling this and how much complexity we can add with the openage modding API is our topic for today.

Other articles in the modding API series:

View all articles

Initial requirements

Throughout the history of strategy games, a lot of different ways to implement civilizations have evolved. In some games factions distinguish themselves only by a difference in unit stats, e.g. in Empire Earth, while others make them entirely unique (Starcraft being a prime example) with individual units, techs and strategies built around them. The Age of Empires series occupies some kind of middle ground in which each civilization uses a subset of a shared tech tree in addition to a few economic and military boni for each civ. Of course, other strategy games introduced many related flavors for their faction systems that come with their own quirks.

To support all these levels of uniqueness, we need a system that is flexible and easy to adjust. We would also like it to still be easily moddable, so that graphical and balance changes can be made by the community. Fortuately for us, nyan makes both of these requirements really easy to fulfill.

Upgrades and unlocks in openage

Before we begin explaining how civilizations are implemented, we should do a quick recap on how upgrades and unlocks are realized in openage. On a technical level every upgrade in a game is defined as a bundle of patches which modify the game entities that are currently available. In turn most upgrades are unlocked by other upgrades and were therefore also patched in at some point. This basically means that any uprade is the result of a series of patch bundles which can be traced back to one starting configuration. For example, the upgrade from spearman to pikeman is a result of these events:

Starting configuration: Dark Age buildings, units, techs are available
Researching FeudalAgeTech patches CastleAgeTech into Research ability of town center
Researching CastleAgeTech patches PikemanTech into Research ability of barracks
Researching PikemanTech patches graphics, name and attributes (HP (+10), attack damage (+1), etc.) of Spearman

The way we unlock PikemanTech is implicitely defined through the previous techs and the starting configuration. It is also important to stress that the upgrade path is not fixed and could just as well look like this:

Starting configuration: Dark Age buildings, units, techs are available
Researching FeudalAgeTech patches WoodenStaffTech into Research ability of blacksmith
Researching WoodenStaffTech patches PointyStickTech into Research ability of blacksmith
Researching PointyStickTech patches PikemanTech into Research ability of barracks
Researching PikemanTech patches graphics, name and attributes (HP (+10), attack damage (+1), etc.) of Spearman

This is benefical for two major reasons. For one it allows techs or units to be unlocked in different ways, which is always cool. But more importantly it shows that a tech's functionality is independent from its unlock path. Sure, PikemanTech has to be unlocked at some point, but it doesn't matter if CastleAgeTech or PointyStickTech did it. In consequence, this means that any of these five steps on the unlock path -- including the starting configuration -- can be altered individually without breaking the engine logic.

Civilizations

But what does this have to do with the uniqueness of civilizations? It turns out that any unique property of a civilization is just a change of a step on the unlock path. And because we are able to alter every individual step without dangerous consequences, we can change pretty much anything we want. The civilization setup is practically a 0th step that comes with its own changes which again are introduced by patching.

Let's say we want to have a civilization that grants every pikeman of this civ 5 more HP than the "standard" pikemans have. We do so by patching the patch from PikemanTech that upgrades HP to give another 5 points of health.

# The tech holds all the necessary patches for the upgrade
# from spearman to pikeman, including the patch for HP.
PikemanTech(engine.Tech):
    updates = o{HPforPikeman, ...}

# This is the standard HP increase for pikeman that
# patches the Live ability of Spearman units to gain
# 10 more HP.
HPforPikeman<engine.Live>(engine.Patch):
    hp += 10

# With this addition, we patch the standard HP increase patch
# to give an additional 5 HP.
CivAdditionalHP<HPforPikeman>(engine.Patch):
    hp += 5

In the series of events that leads to the research of PikemanTech, the application of CivAdditionalHP is part of the 0th step we talked about earlier.

Civ setup: CivAdditionalHP patches HPforPikeman from PikemanTech to grant an additional 5 points of health
Starting configuration: Dark Age buildings, units, techs are available
Researching FeudalAgeTech patches CastleAgeTech into Research ability of town center
Researching CastleAgeTech patches PikemanTech into Research ability of barracks
Researching PikemanTech patches graphics, name and attributes (HP (+15), attack damage (+1), etc.) of Spearman

Of course, as the unlock path of PikemanTech itself consists of patches we are able to change them, too. For example, we can have PikemanTech unlocked in Feudal instead of Castle Age.

# The standard Feudal upgrade which unlocks the tech for Castle Age
FeudalAgeTech(engine.Tech):
    updates = o{CastleAgeUnlock, ...}

# On the standard unlock path CasteAgeTech enables PikemanTech
CastleAgeTech(engine.Tech):
    updates = o{PikemanTechUnlock, ...}

# A new patch puts the unlock for pikeman into Feudal..
CivMovePikeToFeudal<FeudalAgeTech>(engine.Patch):
    updates += {PikemanTechUnlock}

# ..and removes it from Castle
CivRemovePikeFromCastle<CastleAgeTech>(engine.Patch):
    updates -= {PikemanTechUnlock}

We essentially removed one intermediate step on the unlock path.

Civ setup: CivMovePikeToFeudal patches PikemanTechUnlock into FeudalAgeTech
Starting configuration: Dark Age buildings, units, techs are available
Researching FeudalAgeTech patches PikemanTech into Research ability of barracks
Researching PikemanTech patches graphics, name and attributes (HP (+10), attack damage (+1), etc.) of Spearman

When we handle things this way, we end up with a very flexible system that allows us to uniquely define how a game is played with a civilization. One faction might unlock the pikeman by researching Feudal Age, another one by researching Imperial Age. Keep in mind that we don't even need age upgrades for unlocks, PikemanTech might as well be unlocked by Forging or another not so important tech. The starting configuration can also be changed easily because it just represents the game data before any patches have been applied. The civilization setup can alter it the same way we did with the other steps.

The question that remains is whether this system still allows for easy modding. When we look at any unlock path, we see that the 0th step is nothing special and essentially the same as the others: The use of patching to alter attributes. Similar to how we added the 0th step for "Civ Setup", we can introduce another step that comes even before the 0th step. This step would be able to change all subsequent steps, including "Civ Setup". Such a preliminary step would be exactly what a data mod is. And because mods are also a bundle of patches, we can have a step before each mod, which would also be a mod that allows a preliminary step that would again be a mod... (repeat indefinitely)

API description

All of a civilization's individual properties are handled by the Civilization API object. Civilization defines various attributes for its name and ingame help, the names of civilization leaders and its starting resources. boni is meant for civilization specific Bonus objects that are not bound to game entities or techs, such as the vietnamese bonus that reveals the starting positions of enemy players on the map. Everything concerning uniqueness of the civilization is handled by three patch bundles civ_setup, graphics_set and team_setup. They will be applied in the "0th step" we mentioned previously.

civ_setup: Contains all patches for civ specific data changes, excluding graphical setup
graphics_set: Contains all graphical changes for game entities
team_setup: These patches are applied to the player of the civilization and their allies

All patches together define the unique properties of the civilization.

Questions?

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

D8: Openage modding API - The Transformers

2018-09-13T00:00:00+02:00

Anyone familiar with The Transformers franchise by Michael Bay probably knows how the Autobots work. For those who don't: The Autobots are sentient robots from outer space that come to Earth for robot business reasons, but more interesting is that they can switch between two forms. The first one, vehicle form (a.k.a. product placement form), disguises them as cars or trucks from Earth while also allowing them to drive around at a very fast pace. When transforming to robot form, they lose both disguise and the option to move fast, but are able to shoot, punch or slice their enemies effectively. You could say that each form gives the Autobots different abilities. In a sense, this concept is very similar to a unit from AoE2 that also possesses different abilities depending on its form. The trebuchet.

Other articles in the modding API series:

View all articles

Generalizing the problem

What the little example from the Introduction shows us is that despite the uniqueness of the trebuchet in comparison to AoE2's units, the concept of units that have several forms, which give access to different abilities, is far from uncommon. This means there is a good chance that we can generalize the behavior of units that have multiple forms and use this generalization for the trebuchet, instead of having to hardcode specific behavior for just one unit.

The way AoE2 handles its precious stone thrower is by defining the two forms, packed and unpacked, as separate units in the game data. Transforming from either of those forms to the other is done by replacing the unit after the transformation is done. We've already discussed why replacement is a bad idea for villagers in our last blogpost and the same problem also applies to the trebuchet. It requires the replaced and replacing units to be in a predictable state. In openage, this is not guaranteed (by design) as individual units can go through a substantial number of changes due to patching, so we need a different approach here. Instead of two units that are switched, we would like to have a single unit where only the active abilities change.

Finite-state machines

Before we consider the openage API definition, we should discuss how we would want transformations to work in general. Fortunately, there are already existing models that we can use for that purpose in the form of finite-state machines. A finite-state machine is defined by a finite number of states, which are equal to what we have called forms until now, and the transitions - the equivalent of transformations - between them. Each state is further defined by the abilities and transitions that are available when the machine reaches it. Ideally, we would want our game entities to also be finite-state machines. We will go through an example to clarify what this means exactly. However, to explain the topic properly we need something more complex than an Autobot or a trebuchet: A vending machine.

The vending machine that is shown here is defined by a finite-state machine with the three states s_0, s_1, s_2 (depicted as nodes) and various transitions between them (depicted as arrows). s_0 (marked with a double outline) is the starting state that we assume the machine is in when a customer would approach it. Abilities available in a specific state are visible next to the corresponding node in a blue colored font. The total number of abilities the machine has can be seen in the upper left. Only a subset of them is active in each state.

Some of the abilities are transitional, for example InsertCoin, which means they trigger a transition to another state when they are executed. A transition is denoted as an arrow with the name of the transitional ability next to it. Other abilities, like ShowBeverages, do not result in a transition.

Vending machines are a pretty good example case where finite-state machines are useful because we don't want to give the customer access to all of its abilities at once. For example, using ShowSelection makes no sense before SelectBeverage was executed. Giving access to the EjectBeverage ability before InsertCoin was used definitely requires a lot of goodwill, too. With state machines we can make sure that specific actions/transitions are taken before an ability is available to users.

Note that the depicted vending machine always allows us to go back to the starting state s_0 from any other state. There are even multiple pathways of transitions that accomplish this. However, this is not necessarily the case for all finite-state machines. We can show this with a simple extension.

This new version of the vending machine offers one new ability Break and a new state s_e (changes are colored red). Break is available in every state except s_e. The practical implications of this are that if our vending machine "breaks" in any of these states, the machine will transition to the error state s_e and is only able to execute ShowError. s_e is a dead-end as there is no transition that gets us back to a previous state anymore.

Dead-end states are not limited to errors in their use cases. They could just as well be defined for a situation where the vending machine runs out of drinks. The point is that it can sometimes be beneficial to transition to a state or even a branch of states, so that it is impossible to reach the starting state again.

So far we discussed finite-state machines where every state is reachable from s_0 and the whole state graph is interconnected. This does not always have to be the case.

We again introduced new states s_3 and s_4 and some new abilities which would qualify for some kind of "maintainance mode". In our example, it would make sense for these abilities to be completely unavailable from "customer mode" (especially EjectMoney) and therefore have the state machine offer no transition to any of these states. Only an authorized person should have access to them. But how would they be able to reach these states?

The obvious answer is to use outside measures which add a transition at runtime, but are not part of the actual machine. As you might remember, game entities in openage can change at runtime. As a result, a transitional ability could be added at a later date by a patch, an item, a script or a trigger in a custom scenario. Finite-state machines in openage are not fixed as they are just as alterable as the game entities that represent them.

Going back to our vending machine, an example for such an outside measure would be a keycard that is slotted into the machine. Similar to an InventoryItem, this keycard would enable two additional abilities EnterMaint. (for s_0) and LeaveMaint. (for s_3) that transition between s_0 and s_3 (depicted as dashed red arrows).

API definition

Finally, we are going to look at the API definition of finite-state machines in openage. There are two core abilities that accomplish this, Transform and TransformTo. Transform is the ability that enables a GameEntity to be a state machine. It stores all the the possible states of the game entity and the startig state as initial_state. TransformTo on the other hand is meant as a transitional ability which can be used to get into another state. It does so by defining a target_state that must point to a state from states in Transform. It also defines the time the transition takes (transition_time) as well as the sprites used during the transition with TransformProgress objects. After the transition time has passed, the game entity will switch to the target state.

If we would model the trebuchet in our API, we would need one Transform abilitiy and two transitional TransformTo abilities for our state machine. The transitional abilities change the current state to Packed or Unpacked. Both of the states are defined in the set states from Transform as instances of TransformState.

All of the abilities referenced in the defined states are also stored in the abilities attribute of the Trebuchet game entity. This allows them to be easily accessible when we want to patch them, even if they are not enabled in the unit's current state.

Improvements?

While modelling game entities with finite-state machines can be very powerful, the current API system does not explore their full potential (yet). For example, the TransformTo API ability does not do much except for transitioning between states. Furthermore, units will not be able to use any abilities during the transformation. There is a simple reason for that: We don't need so much complexity right now as the only real practical example is the trebuchet. The priority is making sure that the system works reliably for trebuchets first and then gradually improve it with ideas from modders and developers.

For example, we could add a cost for transformations, cooldowns for transitions, states that units can only be in temporary or states that force them to transform back after a time limit. There is a lot of potential for extending the system.

Questions?

In our next iteration of the API blog series, civilizations will make an appearance. We will discuss how they work and how the openage modding API handles uniqueness.

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

D7: Openage modding API - Too many villagers!

2018-09-06T00:00:00+02:00

If we would ask AoE2 players how many villager units there are in the game files, most would probably answer one or two, depending on whether they consider female and male version as separate units. In reality, the AoE2 game data contains a total of 22 different villager units; enough for them to play their own football game. This seems a bit much, so maybe there is some redundancy that we can get rid of.

Other articles in the modding API series:

View all articles

Divide..

Before we actually start working on the redundancy problem, we have to find out why the game data defines so many villagers. Obviously, Ensemble did not intend to make their units play football matches, did they*? As it turns out, the game contains this many villagers because units in AoE2 are only able to have five animated abilities.

Action (for most units this is an attack)
Idle
Move
Die
Decay (this is not resource decay, but corpse decay)

This system works fine for military units, but creates trouble for villagers because they need more than one Action to support all the types of gathering they do. AoE2 supports 8 different forms of gathering resources which, together with attacking, building and repairing, amounts to 11 Actions a villager is required to support. So to solve this dilemma, Ensemble created a unique villager unit for every one of these Actions (twice, because villagers come in two genders) and makes the game switch between them at runtime depending on the Action that is currently needed. In addition to this, they also gave each of these villager units their own idle, move, die and decay animations that add a lot of visual variety.

When we look at the game data in more detail, we see that the villager is not the only unit that is handled like this. Monks and trade carts are also units that have more than one unit defined in the original game data to make them support more animations, although the villager is the most extreme example.

While this implementation certainly works for Age of Empires, there are several reasons why this design is not great. First of all, it adds a lot of redundancy just to change some of the animations. Values for HP, speed and other stats stay exactly the same for each unit, yet have to be redefined twenty-two times in the game data. Secondly, the ingame transitions between the villager types need to be hardcoded in the engine. This would likely mean that we would have to restrict modders that use the openage API to a specfific number of resources and gathering types, which is an unacceptable requirement for us. Finally, replacing different villager units at runtime "on-the-fly" is prone to consistency problems. In openage, units can be more individual than in the Genie Engine and it is very difficult to keep this individuality when replacing them. Ideally we would want the Actions and alternative animations to be part of a single villager unit.

.. and Conquer

Our first advantage is that game entities in the openage API are not limited to five animated abilities. Therefore, we can assign all of the 11 villager-specific actions directly to one unit without much effort. However, we also need a way to include the related move, idle, die and decay animations that the individual villager units had for each type of Action. We can do so with the CommonAnimationOverride object.

CommonAnimationOverride is an API object that any Abiliy can inherit from. By doing this, the unit's default animations from the Move, Idle, Die and Decay abilities will be temporarily overriden with the ones from CommonAnimationOverride until:

any animated ability other than Move, Idle, Die or Decay is executed or
another ability inheriting from CommonAnimationOverride is used

In practice this means an ability can indicate it wants to temporarily replace the animations of the four "common" abilities mentioned above. The example below shows how this would look for the villager ability Forage.

It is important to note that inheriting from CommonAnimationOverride is entirely optional and all abilities should generally work fine without being of this type. It is also not limited to typical villager abilities like Gather, Build or Repair. For example, another unit that can make use of it is the trade cart with its Trade ability. In fact, any ability is allowed to inherit CommonAnimationOverride, even though it doesn't make sense for all of them.

More alternative animations

Having CommonAnimationOverride is still not enough because all gathering abilities have second alternative animations for Move and Idle. These are used as carrying animations and replace their default counterparts as soon as the amount of resources carried by the villager exceeds 20% of their resource capacity. The way we can handle this is very similar in its mechanics to CommonAnimationOverride: We just inherit from a second API override object called CarryAnimationOverride.

By inheriting from CarryAnimationOverride, an ability is able to provide alternative animations for Move and Idle. The alternative animations replace the default ones when the carried resource amount passes the carry_threshold which should be a floating point number between 0 and 1, indicating the percentage at which the threshold is reached. It only works with abilities that make a game entity carry something, so it is best suited to be used with Gather, Transport, ProvideGarrison and Inventory. Even though it is not an ability, InventoryItem is be allowed to inherit from it, too. This makes it possible for items like the relic to give a unit a unique carry animation.

CarryAnimationOverride can be used in combination with CommonAnimationOverride. In this case, if the carry threshold is reached, the move and idle animations from CarryAnimationOverride always take precedence over the definitions from CommonAnimationOverride.

Variants add variety (and genders)

So far we've only seen how abilities activate alternative animations, but we haven't covered how animations change depending on a villager's gender. It turns out this actually sounds harder than it is because in AoE2 genders are decided at creation time and don't change after that. Well, at least they don't by gathering and building. This means we can simply patch in the animation set for each gender before the unit is created. We do so by utilizing a so called Variant of the unit.

A Variant object contains a number of patches that are applied to the unit on creation. Which variant is chosen depends on the type of the Variant objects. A RandomVariant has a chance to be chosen at random (which perfectly fits for our villagers), while PerspectiveVariant depends on the angle the game entity is viewed. The latter variants are particularly useful for walls which have to cover up to 8 different angles. Other variant types could be introduced by us later, if there's a demand.

Variants are not restricted to graphical changes and can patch pretty much every attribute of a game entity, e.g. add new abilities or change the attributes of existing ones. Therefore modders should be able to implement a lot more crazy ideas for unit variety than just alternative animations.

Questions?

Next time we will discuss everyone's favorite siege unit and the implications it has for the API.

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

* The sport depicted is obviously not football because the monks never use their feet. It actually is a special form of gridiron which tries to make up for the lack of feet by making the rules more complicated and running more commercials.

D6: Openage modding API - Inventory System

2018-08-30T00:00:00+02:00

Our topic for today is the Inventory ability which allows units to store items. AoE2 only offers one item, the relic, but it is very popular in custom scenarios. Considering that other strategy games implement other and more various kinds of items, it makes sense to include a fully fledged inventory system in openage, too.

Other articles in the modding API series:

View all articles

Inventory

The abilities for inventory management are PickupItem, DropItem, TranferItem and Inventory. The first three of these abilities interact with items in and out of the inventory, while the actual inventory is defined by an ability that is just called Inventory. An inventory's definition is determined by a list of InventoryItems stored in the allowed_items attribute. Note that InventoryItem is different from Item and has a different purpose:

Item is the (physical) representation of the item in the game world. It is a GameEntity and can therefore exist independently. This means it can also have its own abilities and boni.
InventoryItem defines the effect of an Item when it is stored in a unit's inventory. In contrast to Item it cannot exist independently and all of its effects are applied to the inventory owner. InventoryItem always references the Item for which it stores the effect in its item member.

The benefit of modeling the inventory system like this is that the same item can have different effects for different units. We already know a situation where this is useful because a similar behavior can be observed in AoE2. When stored in a monk's inventory, a relic provides no boni (and even takes away some of their abilities). However, when inside a monastery's inventory it provides a continuous gold bonus to the player. Another more RPG-like example would be a health potion that provides 20HP when a villager consumes it, while a mighty and experienced warrior gets 100HP from using it.

By keeping Item and InventoryItem as separate API objects, we also have more options for items that are outside of an inventory and lie on the ground. Because an Item game entity can have its own abilities assigned, we could enable it to run away or hide from the player, summon units for its protection or just have it mind its own business by chopping trees in the woods.

A closer look at InventoryItem

The effects of InventoryItem that we briefly touched a few paragraphs before are the following: First of all, InventoryItem can grant a variable number of boni to the inventory owner through the boni set attribute. In addition to this, InventoryItem is allowed to take away units' abilities (through disable_abilities) and add entirely new ones (through enable_abilities). This means that a unit's abilities - like Attack - can potentially be swapped for better (or worse) versions of the ability.

items_per_slot determines how many items of the same type can be stacked in one inventory slot. In the diagram we see that HealthPotion can be stacked 5 times before a new slot is required. In consequence, the ExampleInventory can hold up to 30 health potions, while the maximum number of relics is 6. Of course, relics and health potions can be in the inventory at the same time, but every inventory slot can only be occupied by one type of item.

An interesting attribute is conflicts. While the Item specified in item resides in the inventory, items listed in conflicts cannot be there at the same time. This is useful when items are supposed to be incompatible with each other, for example a holy relic and a satanic bible. It can also be beneficial for a rudimentary equipment system where a unit is only allowed to wear one garment for each body part. Considering the complexity, an equipment system should probably have its own ability, but this is something for a future version of the API. After all, we are still making a real-time strategy engine and not an RPG*.

Last but not least, we will show you a code example showing how storing relics from AoE2 would work with Inventory and InventoryItem.

# Monk's inventory
MonkInventory(engine.Inventory):
    items = {MonkInventoryRelic}
    slots = 1

# Monk's inventory item
MonkInventoryRelic(engine.InventoryItem):
    item = Relic
    items_per_slot = 1
    conflicts = {}
    enable_abilities = {}
    disable_abilities = {Convert, Heal}
    boni = {}

# Monastery's inventory
MonasteryInventory(engine.Inventory):
    items = {MonasteryInventoryRelic}
    slots = 10

# Monastery's inventory item
MonasteryInventoryRelic(engine.InventoryItem):
    item = Relic
    items_per_slot = 1
    conflicts = {}
    enable_abilities = {}
    disable_abilities = {}
    boni = {RelicGoldBonus}

* If you're interested in that though, check out Flare.

Questions?

Our next blogpost will be about a very special and complicated unit from AoE2 - the villager, or rather the villagers. Tune in next week when we tame this beast and adapt its quirks to our modding API.

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

D5: Openage modding API - Bonus

2018-08-23T00:00:00+02:00

So far the only customization option we discussed were abilities. However, there's a second option that we brushed before: The Bonus API objects.

Other articles in the modding API series:

View all articles

Motivation

As establihed previously, each Ability in the API is linked to an engine function where its behavior is defined. On the one hand this is a benefit because we know exactly what a unit can do when we give it the ability. On the other hand there could be an ingame situation where we want the ability to do something slightly different, an edge case so to speak.

An example for this is the elevation factor in the damage calculation in AoE2. When two units are on the same elevation level the damage they do and receive is calculated normally. But when the elevation level different, the unit on a higher elevation level will suddenly do 25% more damage, while the opponent does 25% less. The elevation factor is examplary for an edge case behavior that we want to cover in our API with the Bonus object.

Ability and Bonus relation

On an implementation level, Ability and Bonus are quite similar. Every Bonus provides a link to behavior hardcoded in the engine. The key difference however, is the purpose they are used for:

Abilities define general behavior of a GameEntity.
Boni apply under certain conditions or in certain situations. They are not limited to GameEntity.

So unlike abilities, which are usually always available to a unit, a bonus requires a condition to be fulfilled or a situation to occur. Then and only then do they have an impact on the game.

Taking the previous example about elevation the process for handling a bonus would roughly look like this: On execution of the Attack ability the engine checks if the units has any corresponding boni assigned. After that it checks whether the conditions of these boni apply. If they apply, the values in the engine function's calculation are modified accordingly.

Each GameEntity can have an unlimited number of boni assigned in the boni member. Other objects, like Civilization or Cheat, are also allowed to store boni.

Until now we've only talked about boni in the context of (damage) modifiers. In practice, a bonus can do much more, e.g. a map reveal (cheats) or providing a flow of resources (relics). A Bonus object does not necessarily have to alter an ability. This versatility is a requirement to a certin degree as Bonus objects should be able to cover a lot of different edge cases by definition.

While AoE2 has some applications for boni, it really are its successors Age of Mythology and Rise of Nations who make them an integral part of the game. In AoM relics can provide very unique boni, ranging from cheaper upgrades to automatically producing units for the player at the temple. RoN offers the player boni for building wonders and harvesting unique resources. These mechanics could act as a blueprint on what boni will be implemented into the API by us over time.

It is important to note that Bonus is not the same as what the Age of Empires community colloquially refers to as attack bonus or civilization bonus. Most of these are not conditional and only change data. For these cases of application, nyan patching is suited much better.

Unconditional Bonus

We have established before that Bonus objects are conditional, but sometimes it might be beneficial when the mere presence of the bonus is condition enough for its application. This especially concerns boni that are percentage-based.

Consider a civilization that gives cavalry 10% more health in Feudal Age, 15% more in Castle Age and 20% more in Imperial Age. We could do this through patching, but it does not look nice.

FeudalHPUpgrade<engine.Live>(engine.Patch):
    # increase HP by 10% or 110/100
    # all is still fine
    hp *= 1.1

CastleHPUpgrade<engine.Live>(engine.Patch):
    # here the problems begin. To get
    # from 10% to 15%, we need to calculate
    # 115/110 which is
    hp *= 1.045454545

ImperialHPUpgrade<engine.Live>(engine.Patch):
    # Last but not least, to get from 10%
    # to 15%, we need to calculate 120/115
    hp *= 1.043478261

Not only is this very unreadable, it's also prone to rounding errors. We can improve it, if we model it with a Bonus.

# We define bonus...
CavHPBonus(engine.Bonus):
    modifier = 1.1

# ... and add it in Feudal Age to get 10% more HP
FeudalHPUpgrade<engine.GameEntity>(engine.Patch):
    boni += {CavHPBonus}

# In Castle/Imperial Age, we increase the Bonus
CastleAndImperialHPUpgrade<CavHPBonus>(engine.Patch):
    modifier += 0.05

Much better to read for a human.

Questions?

Next time we are going to investigate abilities again by taking a look at the Inventory ability.

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

D4: Openage modding API - Archers don't kill units, projectiles do!

2018-08-17T00:00:00+02:00

In a list of essential features for strategy games, attacking would probably be sorted in first or second place. Of course something that important should not be missing from the openage API. This time we are going to look at how units damage each other with the Attack ability.

Other articles in the modding API series:

View all articles

Damage calculation

In the current API design, damage is dealt as a flat amount and is always directed against a specific type of armor. A single Attack can damage several armor types at once through the definition of ArmorAttack objects in the damage member. Every ArmorAttack stores the damage done against a type of armor. On execution of the attack, each damage value from ArmorAttack the objects of the attacker is matched against a defender's ArmorDefense object with the same armor type.

It's best if we look at a simple example before explaining the system further.

armor_type	damage	armor_value
Melee	3	0
Pierce	5	4
Crush	1	5
Fire	2	N/A

The above diagram gives us very basic definitions of one unit's Attack and another unit's Defense abilities. For convenience, a table shows the comparisons between the corresponding damage and armor values of the armor types.

The calculation for the Melee and Pierce armor types is very straight forward. armor_value in ArmorDefense is subtracted from damage in ArmorAttack which yields the following results:

1	`Damage against Melee armor: 3 - 0 = 3`

1	`Damage against Pierce armor: 5 - 4 = 1`

The situation for the Crush armor type is a bit more tricky because subtracting armor_value from damage would result in a negative value. This behavior is generally undesirable as negative damage can have weird effects. Therefore, the engine will always round negative damage against an armor type up to 0.

Damage against Crush armor: 1 - 5 = -4 --> rounded up to 0

For the last armor type we are facing another special yet common case of the damage calculation. The attacking unit has an ArmorAttack object for the Fire armor type, but the defender does not have an ArmorDefense object with matching armor type. Intuitively one might assume that the amount of fire damage dealt is 2, since the defender seems to have no armor against it. However, this is not true. The engine will only calculate damage when there is an armor type match between an ArmorAttack and an ArmorDefense object. If there is no match, the damage defaults to 0. We've already seen the correct way to give a unit no defense against an armor type when we look at MeleeDefense. Here any attack against Meelee armor does full damage because the defender has the matching armor type and armor_value is set to 0.

The behavior is a bit unintuitive because it inevitably means that an attack will deal no damage when a unit has no ArmorDefense objects defined in armors, which goes against the expectations of the majority of people. On the other hand, resistances against specific types of damage can be modelled much easier.

1	`Damage against Fire armor: 0`

In the end the individual damage values are summed up and then compared to min_damage of the Attack ability. The engine chooses the greater of the two values. In this example min_damage was 1 which is smaller than the calculated damage value 4. The overall damage is therefore 4.

1	`Overall damage: max(1 , 3 + 1 + 0 + 0) = max(1 , 4) = 4`

At runtime there can be situations where the damage is modified further, e.g. because of a height advantage. But that's a story for another time. It is also important to note that openage will likely support other attack systems that use a different damage calculation, like percentage based armor or attacks with a block chance, in the future.

Other types of Attack

In addition to the normal Attack ability, the API supports three more special versions of attacking. AreaAttack does area of effect damage with an optional damage dropoff over distance. The SelfDestruct ability is an even more special AreaAttack where the unit kills itself during the attack. With RangedAttack the attacking unit can attack from a specified distance and does not have to stand right next to the target. RangedAttack must not be confused with ProjectileAttack, since the former does not require any projectile related data.

Shooting projectiles

The first striking difference between Attack and ProjectileAttack is that they are not directly related (other than the previous "special" versions of Attack). This is rooted in the fact that ProjectileAttack merely enables a unit to fire one or more projectiles. Each projectile can have its own Attack ability that is executed on hit. In other words: The units with ProjectileAttack are not doing damage, it's the projectiles they shoot.

The data for attacking with projectiles is partioned between the ProjectileAttack ability and the Projectile objects. In the ability attack range, number of projectiles per attack and of course the projectiles themselves are defined. Inside the Projectile objects we store the actual Attack and other related properties such as accuracy, whether it is fired in an arc or not, whether it is allowed to pass through units. This is great because it allows for a lot of customization. A unit could fire 20 different projectiles, each with individual properties and Attack values.

Questions?

Although behavior for abilities like Attack is hardcoded in an engine function, the execution outcome should be allowed to deviate slightly. Preferrably we would want a way to define behavior edge cases have an influence on the calculation, e.g. when we attack with a height advantage. How that is handled is discussed next week, when we take a look at Bonus objects.

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

D3: Openage modding API - Patching

2018-08-16T00:00:00+02:00

In our last blogpost we talked about abilities, how they are used to define behavior of game entities and had a look at an initial declaration of a Swordsman unit. This time we are going to investigate how we alter the initial declaration at runtime through the use of Patch.

Other articles in the modding API series:

View all articles

Motivation

Most RTS games let players increase the stats of units in one form or another, usually by researching technologies, levelling or giving items to them. Technology heavy games like AoE2 even allow units to upgrade multiple times, e.g. the Miltia unit which can be improved up to four times. One of the easiest methods to accomplish this is by replacing the old unit with an upgraded version that has better stats. For example, the Man-At-Arms upgrade in AoE2 switches all Militia units with Man-At-Arms. While handling upgrades like this is easy, it creates some problems:

Units cannot improve individually because the individual stats cannot be carried over to the upgraded replacement unit.
The units will be very generic. A replacement unit has to be designed for every possible unit upgrade.
Changing ownership of units can create weird bugs because different civilizations could have different upgrade paths for each unit.

We would prefer a more flexible system that is also easy to understand. That's where Patch comes into play. A patch is a special object that defines attribute changes for another nyan object. Here is one simple example of a Patch:

MoreSpeed<engine.Move>(engine.Patch):
    speed += 0.5

The declaration of a Patch is very similar to the declaration of a normal object with the exception of <engine.Move>. The reference in the angled brackets is the target object of the patch. The line below shows the member that is altered by patching. In this case the current value of speed is increased by 0.5 when the patch is applied. Instead of a relational change, absolute changes are also possible.

MoreSpeedAbsolute<engine.Move>(engine.Patch):
    speed = 2.0

This alternative patch would set speed to 2.0 when it is applied.

API objects for applying Patches

As stated before, the declaration of a patch does not define in what situation it is applied. However, there are API objects that cover the most common situations, e.g. Tech and Mod.

Tech objects represent technologies that have to be researched by a GameEntity with the Research ability. The member upgrades stores a set of patches that are the effects the technology has on the player's units. Patches of Tech are applied as soon as the game entity has finished researching the technology.

Similar to Tech, Mod also holds a set of patches, but they are automatically applied right at the start of a game with no further requirements. This can be utilized by modders to get their intended alteration into the game (e.g. different animations/sounds or balance changes) without touching the original dataset.

Benefits

The great advantage of a Patch is that it only defines what is changed, but when, how often and which unit it will be applied to can be decided elsewhere. The patch MoreSpeed of our example that changes the Move ability could be applied on all units, only selected units, a single unit, a class of units, multiple times, once, every 2 minutes, and so on. Furthermore, it allows us to make more fine-grained upgrades, instead of the sledge hammer approach that the unit replacement method is.

So will every upgrade be defined as patches? The answer is yes. Every single upgrade will be done by a patch or a bundle of patches. This goes so far that in the AoE2 swordsman line (Militia, Man-At-Arms, Longswordsman, Two-Handed Swordsman, Champion), Militia will be the only unit that is initially defined. The other units will be defined as patches that upgrade animations, stats and abilities. This might be unintuitive at first, but gives considerably more options to developers and modders.

Questions?

Now that the basics of abilities and patching are established, we will dive deeper into the topic of abilities and take a look at the Attack abilitiy. See you next time!

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

D2: Openage modding API - Abilities

2018-08-02T00:00:00+02:00

Last week we learned about the GameEntity API object and how it provides a skeleton for the objects present in the game world. This time we are going to enable units to interact with each other by giving them abilities.

Other articles in the modding API series:

View all articles

The Ability API object

Each Ability API object has two main purposes.

First of all, an Ability is always linked to behavior that is defined by an engine function. By adding an Ability to a game entity, the engine knows that the unit is allowed to execute this behavior. For example, once the Move ability is added to a unit, it will automatically be able to move around. The exact behavior of an ability is always decided by the engine, hardcoded into a function.

Furthermore, an Ability object stores the attributes that are necessary for the execution of the ability. An example for Move would be the speed at which a unit moves or for Build the buildings that can be constructed. Keep in mind that nyan objects only store definition data which is different from the stats of a unit at runtime. For example, the Live ability only defines the maximum HP, while the current HP value is handled by the engine's runtime simulation system.

Complexity

Because pretty much everything is an ability, the associated properties can range in complexity. Some abilities are purely passive and don't even define members, e.g. Passable which just tells the engine to turn the collision of a game entity off. Others define stats for units, like the Live ability does with hp and line_of_sight or the Creatable ability with creation_time.

The more sophisticated abilities allow units to interact with other entities in the game world, e.g. Repair or Build. Often these complex abilities are also animated and have an execution sound. This is signified by them inheriting from SoundAbility or AnimatedAbility. Abilities usually only store one animation (with rare exceptions), but can define several sounds, one of which is played randomly on execution.

nyan example

Now (finally) we are going to have a look at how all the definition we talked about are written down in the nyan language.

Swordsman(engine.Unit):
    # Strings and translation data (not explained here)
    name = SwordsmanName
    description = SwordsmanDescription
    tech_tree_help = SwordsmanHelptext

    ...

    icon = "swordsman_icon.png"

    # Hitbox
    radius_x = 0.3
    radius_y = 0.3
    radius_z = 0.55

    variants = o{}
    abilities = {SwordsmanLive, SwordsmanMove, SwordsmanDie}
    boni = {}

    SwordsmanLive(engine.Live):
        hp = 40
        line_of_sight = 3
        pop_space = 1
        visible_in_fog = False

    SwordsmanMove(engine.Move):
        # Inherited member from AnimatedAbility
        animation = SwordsmanMoveAnimation

        SwordsmanMoveAnimation(engine.Animation):
            animation = "swordsman_move.sprite"

        # Inherited member from SoundAbility
        sounds = {SwordsmanMoveSound1, SwordsmanMoveSound2}

        SwordsmanMoveSound1(engine.Sound):
            play_delay = 0
            sounds = o{"swordsman_move.opus"}

        SwordsmanMoveSound2(engine.Sound):
            play_delay = 0
            sounds = o{"swordsman_move_alt.opus"}

        # Unique members of Move
        speed = 1.56
        turn_speed = inf # this means turns are instant


    SwordsmanDie(engine.Die):
        # Inherited member from AnimatedAbility
        animation = SwordsmanDieAnimation

        SwordsmanDieAnimation(engine.Animation):
            animation = "swordsman_die.sprite"

        # Inherited member from SoundAbility
        sounds = {SwordsmanDieSound}

        SwordsmanDieSound(engine.Sound):
            play_delay = 0
            sounds = o{"swordsman_die.opus"}

In this example we have created a simple unit Swordsman. Right from the start, we can see that Swordsman inherits from the Unit API object (referenced by engine.Unit). Aside from the values that were required for the members inherited from GameEntity there are three abilities defined: SwordsmanLive, SwordsmanMove and SwordsmanDie. As you probably already guessed, they use the API objects Live, Move and Die.

The three abilities are defined as nested objects. Nested objects work just like normal objects with the neat little benefit that we can keep the notation of the units, its abilities and other dependent objects together in one space. For the abilities we do what we always do: We assign values to the members that are required by the API. After filling everything in, we have completed the definition of Swordsman, a special case of Unit/GameEntity.

And that's really the gist of it. Using the API mostly evolves around "filling in the blanks" by assigning values to the required attributes. The above definition can be taken as is, placed as plaintext inside a .nyan file and will be understood by the engine. Of course, in a real life situation a modder would also have to create animations and sounds to be included. But we hope it was clear that these do not get magically created out of the blue.

Questions?

We have now seen the initial definition of a unit in nyan, but in real-time strategy games the intial values are often changed by upgrades, techs and other modifiers. How these changes are handled in nyan and why it is awesome will be discussed next week when we talk about Patches.

Until then feel free to ask questions and discuss the blogpost in our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

D1: Openage modding API - Units, Buildings & more

2018-07-26T00:00:00+02:00

Last week we explored the overall principles and design decision for our modding API. This time we will see how the definition of units, buildings and other visible objects in the game world will work.

Other articles in the modding API series:

View all articles

GameEntity

Everything that visibly exists and independently operates in the simulated game world can be modelled with the GameEntity API object. This classification includes almost everything that moves, stands around or generally does things on the map. Projectiles are an exception because they need a GameEntity, e.g. a unit, to be spawned, thus violating the independent property. They also have some special features that differentiate them from GameEntity. These will be discussed in another blogpost dedicated to ranged attacks.

GameEntity requires very few attributes to be defined. Only name, description, helptext, an icon and some information about the hitbox (stored in the three radius_ members) are strictly necessary. The other attributes of each game entity are defined through Ability objects which are stored in the abilities member. By adding abilities, game entities gain their main features, like animations, hp, attack damage and more. Two other set members variants and boni exist and are able to specialize a game entity even more. However, we will come back for them at a later date to not complicate this blogpost further.

Developers and modders should not let their objects inherit directly from GameEntity and instead choose one of the four child objects Unit, Ambient, Item or Building. With the exception of Building these objects are just for categorization and don't require additional attributes to be defined. An Item object could have the same abilities as a Unit object *. However the categorization helps us, the engine developers, because we are able to design abilities that are aimed at a specific category. For example, the PickupItem will first and foremost be designed to handle game entities of type Item. We'll now explain what is expected from each category.

* Fun fact: Because nyan allows for multiple inheritance an object can be in more than one of the categories. So if you think that your units should also be items, you can go for it!

Unit

Unit is probably the most generic category. In general, all game entities that move, die, decay and have HP should be units. In the current API design, units are the only types of GameEntity that can be trained with the Train ability.

Examples: military, villagers, animals

Ambient

Ambient objects usually lie or stand around on the map and look pretty. Typically, these objects never move, do not have HP and some of them are harvestable for resources. Flags and flag attachements would also be classified as Ambient. In AoE2 most of these objects would be owned by the neutral player Gaia.

Examples: cactus, tree, gold/stone pile, mountain

Item

These are very similar to Ambient objects, but can be put in an inventory. openage features an inventory system through the Inventory ability. Items in there can have special effects on the inventory owner, provide boni or grant new abilities.

Example: relic

Building

Buildings are more sophisticated than the othere three categories because they have to be constructed first and need space on the map grid. Each Building can have construction and damage stages that are defined in a Progress object. Progress objects allow buildings to display an alternative sprite after a certain percentage of something is complete. The number of stages is unlimited and theoretically a building could have 100 stages of construction that each have their own graphic.

It is also allowed to define more complex buildings that consist of multiple parts by using MultiPartBuilding. This type acts as a container that manages several subbuildings which are allowed to have individual properties, e.g. certain abilities and boni. MultiPartBuildings are also useful for modelling buildings that have passable parts like the Town Center in AoE2.

Questions?

You've chosen the perfect category for your game entities - now what? Fear no more because next time, we are finally introducing the Ability system that gives them a purpose.

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

D0: Openage modding API - Introduction

2018-07-19T00:00:00+02:00

A while ago we introduced our game configuration database language nyan. Now we present our efforts to integrate nyan into openage and to create a powerful and extensible modding API.

Other articles in the modding API series:

View all articles

What is an API?

As you probably all know, openage is not just one game. It is intended to be a fully functional game engine that provides a framework for multiple RTS games. Because any game developer can start working the engine and we cannot anticipate how they want to use it, it wouldn't be smart to have every tiny bit of game logic hardcoded. Instead, the engine should only implement general behavior and provide an interface for game developers. That's where the API comes into place.

An API, also known as an application programming interface, is a method of exposing an engine's functionality to its users (in our case: developers and modders). Engines can implement several APIs that give access to a variety of functionality, like scripting or adding GUI elements. This blogpost will solely focus on the modding API which is used for defining game data.

Developers usually take the API first and then use it to derive special cases they want for their game. You may wonder how this derivation process works exactly. We are going to look at a very simple example.

API example

In this example the engine has defined a function that lets a unit run around on a map and displays their name and HP when it is selected. The function takes a very simple API object called Unit as input. Unit objects must have the attributes name (as a text value) and hp (as an integer value), but the exact value of these attributes is up to the developer. By setting the value, the special cases are derived.

The above diagram shows how units for a medieval game could be defined. A developer derived the three units Swordsman, Spearman and Knight from the generic API object Unit by assigning specific values to the attributes. The engine will handle all special cases like it will handle Unit, but takes their specific values into account. For example, if a Knight is selected, it will display Knight as the name and 120 as the HP.

A developer with a different game in mind can also utilize the API to define completely different special cases.

This would also be an viable use for the API when a developer wants to have a more modern setting in their game.

You might wonder if the derived objects from the two games could be used simultaneously. The answer is yes, they can!

In this API design the engine essentially does not care that the theme of the medieval and the modern units is vastly different. As long as they use the same API they can be used in any combination, even if it does not make much sense. For a modding community this kind of API design is a huge benefit because it is very easy to extend, replace and combine mods.

We will later see how the openage modding API handles things in a similar manner, albeit the design is much more complex.

API Overview

The UML diagram below shows the latest draft of our API and should be very close to what will be implemented in the next months.

If you are familiar with Age of Empires 2, you most likely spotted some similarities between the game and our API objects. Although it draws inspiration from the game, modding in openage will be completely different from modding in AoE2. The openage modding API exposes much more features and should allow for very complex mods and a range of different styles of RTS gaming.

Over the next weeks, we will have a look at specific parts of the API and explain how it can be used.

General design decisions

To support our goal of making modding easy and keeping the API extensible we made a few major design decisions.

Single API tree

The whole API is part of a single object tree (or graph to be more specific). All API objects inherit implicetely or explicitely from the root object Entity. This will keep the hierarchy of objects consistent, even if a lot of mods are activated at once. It also allows every object to be added to a set with type Entity.

Define units through abilities

Units are almost entirely defined through API objects of type Ability. Abilities describe what a unit can do (e.g. Build, Move, Research) or what it is (e.g. Creatable, Selectable). Most of a unit's attributes will reside in an ability. This goes so far that Live itself will be an ability that stores the member hp and its value. Each unit stores its abilities in a set, so that they can easily be added or removed (including through patching at runtime).

Because abilities are API objects themselves, their values can be uniquely defined for each unit. For example, two different units can derive their own version of the Move ability and each specify different values for speed.

Every object that is of type GameEntity (e.g. units, buildings, items, trees, gold spots & more) can posess any ability. The system is very powerful and allows for

Trees training units
Animals converting villagers
Relics chopping wood

and other crazy stuff.

All game modifications are done only by patches

nyan allows us to dynamically change objects' members at runtime by applying Patches. Everything from upgrading stats to unlocking units will be done by patching the members of an object. Any value can be changed and adjusted by a patch. Also, individual units or game entities on the battlefield can be patched individually. This will allow developers and modders to create special units from generic units at runtime.

No replacement of units within their life cycle

In AoE and other strategy games objects are often replaced by a new object when they switch to a new state. This can for example be observed when unit lines are upgraded or when villagers are assigned to the various gather tasks.

We cannot use that strategy for openage as patching has the potential to severely alter the member values of the unit. By replacing a unit its changes would be lost. To avoid replacing objects we have implemented a variety of solutions which will enable the objects to switch states without losing their patch history.

Questions?

Next week we will explain how GameEntitys, the objects in the game world, are handled.

Any more questions? Let us know and discuss those ideas by visiting our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

Implementing Python-Style Enums in C++

2018-04-21T23:01:00+02:00

It has been proven over and over again that we suffer from a severe form of NIH. Today, we tried to reinvent the wheel once again, but with rockets attached, and tackled the beauty of C++17 by creating a usable enum implementation.

The current openage enums sometimes produce linker errors on macOS and warnings on other systems:

(`instanciation of enum required here, but no definition is available`)

...
[ 35%] Building CXX object libopenage/CMakeFiles/libopenage.dir/log/stdout_logsink.cpp.o
In file included from /home/jj/devel/openage/libopenage/log/stdout_logsink.cpp:3:
In file included from /home/jj/devel/openage/libopenage/log/stdout_logsink.h:5:
In file included from /home/jj/devel/openage/libopenage/log/logsink.h:8:
In file included from /home/jj/devel/openage/libopenage/log/level.h:8:
/home/jj/devel/openage/libopenage/log/../util/enum.h:99:17: warning: instantiation of variable 'openage::util::Enum::data' required here, but no definition is available [-Wundefined-var-template]
                return &this->data[this->id].second;
                              ^
/home/jj/devel/openage/libopenage/log/stdout_logsink.cpp:16:33: note: in instantiation of member function 'openage::util::Enum::operator->' requested here
        std::cout << "\x1b[" << msg.lvl->colorcode << "m" << std::setw(4) << msg.lvl->name << "\x1b[m" " ";
                                       ^
/home/jj/devel/openage/libopenage/log/../util/enum.h:129:19: note: forward declaration of template entity is here
        static data_type data;
                         ^
/home/jj/devel/openage/libopenage/log/../util/enum.h:99:17: note: add an explicit instantiation declaration to suppress this warning if 'openage::util::Enum::data' is explicitly instantiated in another translation unit
                return &this->data[this->id].second;
                              ^
1 warning generated.
...

We tried getting rid of that warning several times, but that always led into a deep rabbit hole of linker errors. Now we were fed up and decided to get rid of our old enum implementation.

Why?

But why this own enum implementation?

Usage exactly like the enum class
Have a string representation of each enum value
Allow member methods for the enum type
Everything at compile time and accross TUs without funny extern definitions

After some experiments, we implemented it with a wrapper class (EnumValueContainer) which has implicit conversions to the EnumValue type. The actual enum values are constexpr static members of a subclass of EnumValue. We need this in order to use LogLevel for both containing the all possible enum values, being the type for the enum like an enum class name, and using it as non-const type that references to one of the possible values.

This way, the container can store a reference to its static member, i.e. a handle to a enum value like you know from enum class!

The remaining problem was member methods, especially because the "container" class type should also be usable a enum-value type. To achieve this, we had the idea of using a mixin class LogLevelMethods with CRTP. This is what we came up with:

enum.h

#pragma once

#include <iostream>
#include <typeinfo>
#include <type_traits>
#include <cxxabi.h>

template<typename DerivedType, typename NumericType=int>
struct EnumValue {
    // enum values cannot be copied
    EnumValue(const EnumValue &other) = delete;
    EnumValue &operator =(const EnumValue &other) = delete;

    // enum values are equal if the pointers are equal.
    constexpr bool operator ==(const DerivedType &other) const {
        return (this == &other);
    }

    /* SNIP */

    friend std::ostream &operator <<(std::ostream &os, const DerivedType &arg) {
        int status;
        os << abi::__cxa_demangle(typeid(DerivedType).name(), 0, 0, &status) << "::" << arg.name;
        return os;
    }

    const char *name;
    NumericType numeric;
};

template<typename DerivedType>
struct EnumValueContainer {
    using this_type = EnumValueContainer<DerivedType>;

    const DerivedType &value;

    constexpr EnumValueContainer(const DerivedType &value) : value{value} {}

    // implicit conversion operator!
    constexpr operator const DerivedType &() const {
        return this->value;
    }

    constexpr EnumValueContainer &operator =(const DerivedType &value) {
        this->value = value;
    }

    constexpr bool operator ==(const this_type &other) const {
        return (this->value == other.value);
    }

    /* SNIP */

    friend std::ostream &operator <<(std::ostream &os, const this_type &arg) {
        os << arg.value;
        return os;
    }
};


struct NOVAL {};
struct VAL {};


template <typename T, typename novalue, typename ET=void>
struct EnumMethods {
    template <typename X=novalue>
    typename std::enable_if<std::is_same<X, NOVAL>::value, const T*>::type
    get_this() const {
        return static_cast<const T*>(this);
    };

    template <typename X=novalue>
    typename std::enable_if<std::is_same<X, VAL>::value, const T*>::type
    get_this() const {
        return static_cast<const T*>(&(static_cast<const ET*>(this))->value);
    };
};

loglevel.h

#pragma once

#include "enum.h"


template <typename T, typename novalue, typename ET=void>
struct LogLevelMethods : EnumMethods<T, novalue, ET> {
    void foo() const {
        std::cout << "foo is " << this->get_this()->name << std::endl;
    }
};


/** Here, new member values for each enum value can be added */
struct LogLevelValue : EnumValue<LogLevelValue>, LogLevelMethods<LogLevelValue, NOVAL> {
    const char *color_code;
};


/** Usage of the first design attempt for the new enum */
struct LogLevel : EnumValueContainer<LogLevelValue>, LogLevelMethods<LogLevelValue, VAL, LogLevel> {
    using EnumValueContainer<LogLevelValue>::EnumValueContainer;

    static constexpr LogLevelValue debug = {{"debug", 10}, {}, "31;1"};
    static constexpr LogLevelValue info = {{"info", 20}, {}, "32"};
};

Thus, it is now possible to do this:

LogLevel a = LogLevel::info;  // store a handle to the static constexpr member!
a.foo();                      // call the enum "member" method!
LogLevel::info.foo();         // same, but without the LogLevel wrapper!

But why does that work?

The = assignment uses the implicit conversion from LogLevelValue to LogLevel. The latter stores a reference to LogLevelValue.

The a.foo() call is even more obscure: The magic with LogLevelMethods effectively "redirects" the operator . from LogLevel to LogLevelValue through LogLevelMethods via EnumMethods.

EnumMethods adds the foo method to both the LogLevel container and each LogLevelValue directly, and can convert the this pointer accordingly to reach the per-enum-value data. If we kept this, we probably would have added macros to simplify the template shenanigans in the declarations of LogLevelValue and LogLevel.

Containing almost standard-library-levels of template magic, we didn't exactly want to commit this to the repo.

Improvements

The much easier variant, which works without a redirect/"overload" of the operator . is to just use operator -> instead, which can actually be overloaded without hacks:

struct EnumValueContainer {
    /* ... */
    constexpr const DerivedType *operator ->() const {
        return &(this->value);
    }
    /* ... */
}

struct LogLevelValue : EnumValue<LogLevelValue> {
    const char *color_code;
    // of course, more members and functions could be added here

    void foo() const {
        std::cout << "foo: " << this->name << std::endl;
    }
}

/** Usage of the "final" design for our new enum */
struct LogLevel : EnumValueContainer<LogLevelValue> {
    using EnumValueContainer<LogLevelValue>::EnumValueContainer;

    static constexpr LogLevelValue debug = {{"debug", 10}, "31;1"};
    static constexpr LogLevelValue info = {{"info", 20}, "32"};
};

Now, we can just call it with ->:

int main() {
    LogLevel lvl = LogLevel::debug;
    lvl->foo();
    return 0;
}

Mind that all this only works with C++17.

Here is the full code for our new enum (GPLv3 or later):

#### Enum definition:

// Copyright 2018 the openage authors, GPLv3 or later.
#pragma once

#include <iostream>
#include <typeinfo>
#include <cxxabi.h>


template<typename DerivedType, typename NumericType=int>
struct EnumValue {
    // enum values cannot be copied
    EnumValue(const EnumValue &other) = delete;
    EnumValue &operator =(const EnumValue &other) = delete;

    // enum values are equal if the pointers are equal.
    constexpr bool operator ==(const DerivedType &other) const {
        return (this == &other);
    }

    constexpr bool operator !=(const DerivedType &other) const {
        return !(*this == other);
    }

    constexpr bool operator <=(const DerivedType &other) const {
        return this->numeric <= other.numeric;
    }

    constexpr bool operator <(const DerivedType &other) const {
        return this->numeric < other.numeric;
    }

    constexpr bool operator >=(const DerivedType &other) const {
        return this->numeric >= other.numeric;
    }

    constexpr bool operator >(const DerivedType &other) const {
        return this->numeric > other.numeric;
    }

    friend std::ostream &operator <<(std::ostream &os, const DerivedType &arg) {
        int status;
        os << abi::__cxa_demangle(typeid(DerivedType).name(), 0, 0, &status) << "::" << arg.name;
        return os;
    }

    const char *name;
    NumericType numeric;
};


template<typename DerivedType>
struct EnumValueContainer {
    using this_type = EnumValueContainer<DerivedType>;

    const DerivedType &value;

    constexpr EnumValueContainer(const DerivedType &value) : value{value} {}

    constexpr operator const DerivedType &() const {
        return this->value;
    }

    constexpr EnumValueContainer &operator =(const DerivedType &value) {
        this->value = value;
        return *this;
    }

    constexpr const DerivedType *operator ->() const {
        return &(this->value);
    }

    constexpr bool operator ==(const this_type &other) const {
        return (this->value == other.value);
    }

    constexpr bool operator !=(const this_type &other) const {
        return (this->value != other.value);
    }

    constexpr bool operator <=(const this_type &other) const {
        return this->value <= other.value;
    }

    constexpr bool operator <(const this_type &other) const {
        return this->value < other.value;
    }

    constexpr bool operator >=(const this_type &other) const {
        return this->value >= other.value;
    }

    constexpr bool operator >(const this_type &other) const {
        return this->value > other.value;
    }

    friend std::ostream &operator <<(std::ostream &os, const this_type &arg) {
        os << arg.value;
        return os;
    }
};

#### Usage:

#pragma once

#include "enum.h"


struct LogLevelValue : EnumValue<LogLevelValue> {
    const char *color_code;

    void bar() const {
        std::cout << "bar is " << this->name << " and " << this->color_code << std::endl;
    }
};


struct LogLevel : EnumValueContainer<LogLevelValue> {
    using EnumValueContainer<LogLevelValue>::EnumValueContainer;

    static constexpr LogLevelValue debug = {{"debug", 10}, "31;1"};
    static constexpr LogLevelValue info = {{"info", 20}, "32"};
};

int main() {
    LogLevel l = LogLevel::debug;
    std::cout << l << " => " << l->bar() << std::endl;
    std::cout << (LogLevel::debug < LogLevel::info) << std::endl;

    return 0;
}

Oh C++, such an adventure game.

T2: Curve and nyan roadmap

2018-01-25T00:00:00+01:00

After a very long planning and development phase, two of our new core components are implemented: nyan and the curves.

Both are designed to work together: nyan provides the configuration building blocks, the curves enable our tickless, event-based game engine to work. Now, we only have to combine them in order to run openage.

Replacement strategy

The current simulation, which is implemented with a classic time step loop, will be replaced by the new event based game simulation system.

The replacement will be done in several steps:

Design of the game state structure
Design of the nyan model (i. e. API objects) for configuring the game state structure
Creation of a mod pack format
The convert script has to create a mod pack with assets and nyan files from the original game
The engine has to load mod packs and start a basic game with a moving unit
The new renderer has to be combined with the new simulation core

Those steps are tracked in a github project board.

The first step is the cleanup and merge of the eventsystem code. Meanwhile we think about the nyan API and the modpack format.

Once the eventsystem is merged, we create a demo, which basically is a new main(). In this, we slowly add features so game entities are instanced by nyan until we have a simple standing unit (militia, for simplicity).

In order to see this on-screen, the new renderer has to be able to display the terrain and units.

After that, we should add create the Movement ability, which allows walking around. The pathfinding can remain very stupid (like our current implementation is), but in the future should be far more sophisticated. It should be quite easy to improve and extend.

Likewise, we have to add many other abilities to the new simulation.

Once the demo is nearly as good as the current game, we just enable it as the new main function of the game.

Sounds easy, right? Then let's do it.

Questions

Do you have something to discuss? Visit our subreddit /r/openage!

And you can reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

T1: Curves logic

2017-08-22T00:00:00+02:00

The most important component of most games is the core gamestate. It contains all information relevant for the internal game logic, which the other components take into account and use for example for rendering or network transmission.

Gamestate Interpolation

The naive approach to this problem would be having a list of objects, every object has a position and hitpoints. If a object receives a move command, we can calculate a path, and move it along this path, frame by frame with deterministic lockstepping, whereby with the knowledge of how much time dt has passed since the last frame, the current position and the current speed of a unit, we can integrate the position like

o.position = o.position + speed * dt

This approach has the major benefit of being really simple, and there is not much fancyness involved. This is the most simple, the Euler integration method. There are many more, more accurate and more calculation-heavy interpolations, but they all have the same drawbacks.

It is really hard to go back in time without keeping a store of old values.
You diverge from your optimal path just because of numerical effects.
You have to lay hand on every single object to update their position on each tick, even if they are not rendered at the moment.

Curves - the answer of your dreams?

And so we came up with another idea - the Curves, inspired by Planetary Annihilation.

A curve is a collection of

#  time    value
0: (0,     (50, 50)),
1: (150,   (75, 0)),
2: (450,   (0, 30)),
3: (750,   (75, 30)),
...

The curve is visualized in the below image, whereby tha ball is flying in different directions.

containing for example the path of an object on the map over the time from 0 to 750ms. What we see there is the list of keyframes to describe the position of an object, going from middle to right to left and back playing ping-pong.

To access for example the position of the object at now=100ms we can calulate between two keyframes: (t0, k0) and (t1, k1). We hereby use element-wise addition and substraction of vectors and multiplication of a vector with a scalar.

o.position['100ms'] = k0 + (k0 - k1) * (now - t0) / (t1 - t0)
= (50, 50) +  ((75, 0) - (50, 50)) * (100 - 0) / (150 - 0)
= (50, 50) + (25, -50) * 100/150
= (50 + 25 * 0.66, 50 - 50 * 0.66)
= (66.5, 17)

Which is right between the two key-frames - the red dot in the above image.

It is awesome!

Now to move from a ball playing ping pong to units. As you see - it is slightly more expensive to calulate the exact position of one unit, but this does not need to happen every frame for every unit, but instead only for those that are currently on screen. So huge armies, that are larger than a screen do not have to be interpolated as whole all the time.

Another thing that comes free house is that it is very simple to go back to any point in time, since the old keyframes are not overwritten for new values. And for RAM saving purposes, we can always remove keyframes that are not relevant anymore and lay some time in the past.

This approach also does not accumulate floating point errors as fast as frame-by-frame interpolation, since key-frames lay further appart, and are not calculated as often. The constant floating point error that occurs within the interpolation is negligible, since it is not integrated over.

Or maybe not?

Of course this approach has major drawbacks, especially if the future of a object is changing very often or is not predictable at all. This is the case for any user-controlled unit, because users are the major source of randomness, and curves do not like users. But for command-driven genres like adventure-games, puzzle-games or realtime strategy games this comes in handy. We only have to calculate the path of a unit once, and as long as we do not look, we do not care where the unit is - except if it collides with something else on the path. But that is a story for another time.

So can everything be represented as a curve?

There is a short answer to that, and a long one. the short answer is: Yes!. And not can. Must!

The long answer is: to be able to fully integrate the curve logic into the game logic, it is not sufficient to only use positions of objects. it is neccessary to track the hitpoints as well - because the path of the unit depends on the hitpoints of the building the unit is currently avoiding. And the hitpoints of the unit depend on how many enemies are currenytly attacking it. How many enemies are attacking it depends on when the unit was built in the barracks. When the unit was built in the barracks depends on how many ressources are there ... gathering ... villager producing ... existence of town center ... it would go on all night. So every value in the gamestate has to be time-dependent - a curve.

The smallest common datatypes that can be used here are linear interpolated curves, as seen above, and discrete curves, that hold a value until the next keyframe. Discrete curves do not need to be interpolated, and so they can contain data, that does not define "-" or "+", like strings and objects and nyan references.

Integration

Now you might wonder how curves works together with the rest of the engine? We've got many ideas there and are experimenting how the simulation could be done best, but the overall structure boils down to this:

client:
[nyan] <-> simulation playback/prediction with curves <-> [presenter: gui, renderer, audio]
           ^
           | network
server:    v
[nyan] <-> authoritative simulation with curves

This architecture allows us to have one dedicated game server (which can be run by any player) and users can still do client-side modding.

Once the plan is more clear we'll explain the inner workings of the simulation and prediction.

Questions

Wanna discuss those ideas? Visit our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

T0: nyan API integration

2017-08-13T10:20:00+02:00

This is probably our zeroth technical information post, it may be informative to you if you want to follow development more closely. We'll post more things on an irregular basis :)

What's nyan?

nyan is our new database for storing the game configuration: It provides which units are available, what they can do, cultures, technologies, i.e. everything that makes openage behave and look like age of empires.

nyan also functions as mod-API, as it is designed to be easily usable and extensible for content creators and modders. After some minor changes and improvements in nyan, we start to integrate it into the engine and use it as data source.

For this, we need to create the "openage API", i.e. everything the engine can do. The API is defined in nyan-files:

The engine accesses content data by names and structures it defined in the API
The game content is created and set up with this API

API Example

For example this could be a simplified version of the openage API, defined in valid nyan code:

Entity():
    name : text

Unit():
    hp : int
    abilities : set(Ability)
    # probably much more is missing here :)

Resource():
    name : text
    icon : file

DropSite():
    accepted_resources : set(Resource)

ResourceProvider():
    resource_type : Resource
    amount : int

ResourceSpot():
    provided_resources : set(ResourceProvider)

Animation():
    image : file
    frames : int = 1
    loop : bool = True
    speed : float = 15.0
    # ^ the above information could also be in
    #   some info file that accompanies the image

Ability():
    animation : Animation

Movement(Ability):
    speed : float
    instant : bool = False
    range : float = inf

HarvestResource(Movement):
    target : Resource
    harvest_animation : Animation

Using that API, we can now create content for a game that is running on the engine, e.g. our AoE2 implementation:

Wood(Resource):
    name = "chop chop"
    icon = "wood.svg"

Tree(Entity, ResourceSpot):
    name = "big ol' oak"

    TreeWood(ResourceProvider):
        resource_type = Wood
        amount = 150

    provided_resources = {TreeWood}


Villager(Unit):

    Walking(Movement):
        WalkingAnim(Animation):
            image = "walking_villager.png"
            frames = 18

        animation = WalkingAnim
        speed = 15.0

    HarvestWood(HarvestResource):
        Transport(Animation):
            image = "wood_transport.png"
            frames = 40

        Chop(Animation):
            image = "wood_cutting.png"
            frames = 20

        target = Wood
        animation = Transport
        harvest_animation = Chop
        speed = 12.0

    name = "Villager"
    hp = 25
    abilities += {Walking, HarvestWood}

If we now assume the engine properly implements the expected behavior of the API, this could be an intuitive definition of a villager that can do wood cutting.

The content for AoE1 can be described just as easily. Modders will have a great time because they can add any object and action on the fly, as long as it uses the API.

You want to add a new resource? Define it, add resource spots and the harvesting action and it's in the game.
You want "special moves" or hero units like in Warcraft 3 or AoM? Just define a new ability, add it to the units that shall have it and you're good to go.

You may now ask yourself "why the data is described this way and not another?"

The majority of errors are checked at load-time through the nyan type system.
nyan allows to change the data at run time with patches:
- A patch object defines changes to member values of a target object.
- This means that every possible change is already stored in the database.
- When mods e.g. add new abilities to a unit, this is done by a patch activated when your mod is activated.
- When you click a button for technology research, a patch is applied (villager now has +10 HP for example).

In essence: all game modifications are done only by patches. Be it technology research, development testing, mods or total gameplay overhauls.

Here, have an overly-simplified example with the loom technology, the first research usually available in the town center.

Loom<Villager>():
    hp += 15

TownCenter(Unit):
    researches = {Loom}

The openage engine implements the feature "research buttons", which will activate the patch if a user clicked on it. Missing in the example is of course configuration of the button look, delay, cost for the research etc, but you get the point.

Integration

Now you might wonder how nyan works together with the rest of the engine? We've got many ideas there and are experimenting how the simulation could be done best, but the overall structure boils down to this:

client:
nyan <-> [simulation playback/prediction with curves] <-> [presenter: gui, renderer, audio]
           ^
           | network
server:    v
nyan <-> [authoritative simulation with curves]

This architecture allows us to have one dedicated game server (which can be run by any player) and users can still do client-side modding.

Once the plan is more clear we'll explain the inner workings of the simulation and prediction.

Questions

Wanna discuss those ideas? Visit our subreddit /r/openage!

As always, if you want to reach us directly in the dev chatroom:

Matrix: #sfttech:matrix.org
IRC: #sfttech on libera.chat

Stupid Test

2017-08-12T00:01:00+02:00

This test is totally meaningless as it just is a test. It contains literally no information. Go away. No reason to continue reading.

wtf = "You can try as hard as you like, no riddle is hidden here."
print(wtf)

As you can see, boring things like code highlighting works. This is all defined in funny markdown files (may actually not be funny).

class TotallyUseless {
    struct its_very_boring {
        int stupid;
        std::string content;
    };

    using yes_even_here = std::vector<its_very_boring>;

public:
    TotallyUseless()
        :
        seriously{
            {13, "it's"},
            {37, "not"},
            {42, "getting"},
            {235, "better"}
        } {

        this->do_nothing();
    }

    const yes_even_here &do_nothing() const {
        return this->seriously;
    }

protected:
    yes_even_here seriously;
};

After some extensive suffering, it's also possible to draw tables:

This is	such a table
it also	contains no content
you may	have guessed this.
but it	looks pretty.

And because sometimes you have non-code preformatted text, we have a <pre> feature.

This allows drawing
░█░█░█▀▀░█▀▀░█░░░█▀▀░█▀▀░█▀▀░░░█▀█░█▀▀░█▀▀░▀█▀░▀█▀░░░░░█▀█░█▀▄░▀█▀░█
░█░█░▀▀█░█▀▀░█░░░█▀▀░▀▀█░▀▀█░░░█▀█░▀▀█░█░░░░█░░░█░░▄▄▄░█▀█░█▀▄░░█░░▀
░▀▀▀░▀▀▀░▀▀▀░▀▀▀░▀▀▀░▀▀▀░▀▀▀░░░▀░▀░▀▀▀░▀▀▀░▀▀▀░▀▀▀░░░░░▀░▀░▀░▀░░▀░░▀

I now congratulate you warmly for standing through reading all this nonsense.

You may now report your fabulous reading experience on /r/openage.

Landing Page

2013-08-21T00:00:00+02:00

Curve Logic

Curves Logic Introduction