AVA

A few days ago someone asked me why I was teaching students the use of smart pointers. The code of the students was considered "bad" because they had used smart pointers. In that same week, other students asked me what they should use: smart pointers or raw pointers.

I defended my point of view on smart pointers, but my modus vivendi is to always question yourself. So am I right? Are smart pointers a good tool in our toolbox as game programmers or should they be avoided at all times? When we see them, should we run away in terror or should we consider it good design (if well used)?

The community's opinion

There's definitely been a shift in the mindset of the gamedev community. Googling for "smart pointers in game development" brought me upon several threads at gamedev.net on the topic, in chronological order:

In the 2012 thread, it's clear that C++11 is just new and people are still questioning the standard. But in the 2016 thread people actually defend the standard.

The standard

Recently there's something known as the "C++ Core Guidelines", a collection of code guidelines for C++ written by people that are knowledgeable about C++, edited by Herb Sutter and Bjarne Stroustrup. When we look at the chapter about resource management, we see that the use of smart pointers is encouraged.

R.20: "Use unique_ptr or shared_ptr to represent ownership". Ok that's clear, when we want to think about ownership, use those pointer types.
R.10 and R.11 are clear: avoid using malloc/free and new/delete. Why? Because we all know that we should use RAII, and memory is a resource, so it should be wrapped in a RAII wrapper. Enter the smart pointers.
R.3: "A raw pointer (a T*) is non-owning"

Why is it up for discussion?

In game development we deeply care for memory access patterns because it can heavily impact the performance of our algorithms, as I have shown in this previous blog post. The fear of smart pointers is mostly about the shared_ptr, because that one needs to keep a reference counter in memory somewhere. When an extra shared pointer to a given object is created, the reference counter should be increased (and when the pointer is released, decreased) which causes a memory access we don't want. Indeed, from Scott Meyers' Effective Modern C++:

std::shared_ptrs are twice the size of a raw pointer.
Memory for the reference count must be dynamically allocated.
Increments and decrements of the reference count must be atomic.

But, as Scott Meyers points out, most of the time we use move construction when creating a shared pointer (a c++11 feature), thus removing overhead 3. Creating the control block can be considered free as long as you use "make_shared". Dereferencing a shared_ptr is the same as dereferencing a raw pointer (so use that).

The exact ins and outs of smart (shared) pointers and the possible performance impact they have is discussed in this very detailed talk on smart pointers by Nicolai M. Josuttis at NDC 2015. He describes in detail what exactly the cost is. There is a memory overhead of 12-20 bytes, depending on the usage, and in multi threaded applications there is an overhead in updating the reference counter. Updating the reference counter must happen atomic as Scott Meyers writes, but that can introduce stalls in the CPU's store buffer. Nicolai illustrates this in his talk and the impact is astonishing. However as long as you don't copy the smart pointers by value, there is no noteworthy cost.

In GotW #91 Herb Sutter gives these two guidelines:

Guideline: Don’t pass a smart pointer as a function parameter unless you want to use or manipulate the smart pointer itself, such as to share or transfer ownership.
Guideline: Prefer passing objects by value, *, or &, not by smart pointer.

This translates in the Core guidelines as

R.30: Take smart pointers as parameters only to explicitly express lifetime semantics

My conclusion

Is often this one: use the right tool for the right job (and know how to use it!). Indeed there are potential costs to std::shared_ptr, ones we don't like in game development. As seen in the tests, this happens most often when shared pointers are copied by value. That's why we teach our students to pass these by reference, just like strings. What I learned here is R.30, only pass these smart pointers when you're manipulating their lifetime. Raw pointers and references can and should be used when lifetime is not an issue.

If we're not working on the hot code path, we want the safety and correctness these smart pointers give. And yet again: profile, before you optimize. Be sure that there is a performance impact before you start to remove features in favor of "optimization".

Know the difference between the various types of pointers and use them for their intended purpose. I hope this post gives you some links to resources that help you with just that.

04 August 2018

More foggy adventures

It seems I was too eager with my stylized fog effect in my previous post. When I added a fly-through script on my camera (which I hadn't done before writing the previous post) it quickly showed something was wrong:

As you can see in the above image, the gradient moves on the terrain as you turn around. This is a typical side-effect of a depth based fog (which is standard).

Flat depth vs distance

Normally with a single color and a good fog distance this isn't too bad, but because we introduced the gradient this side-effect becomes really apparent. In a side scrolling context this is not much of an issue, since you don't turn around. But often you're turning your head so we desire distance based fog. The above image and the ins and outs of fog come from this excellent tutorial so check that out for more details.

I got a lot of inspiration from the now deprecated "Global Fog" post process effect from Unity. Both that script and the tutorial from catlike coding explain how to implement distance based fog. So we implement this with the PostFX v2 system. First, pass the frustum corners to the shader:

public override void Render(PostProcessRenderContext context)
{
 //...
 
 Camera cam = context.camera;
 Transform camtr = cam.transform;

 Vector3[] frustumCorners = new Vector3[4];
 cam.CalculateFrustumCorners(new Rect(0, 0, 1, 1), 
  cam.farClipPlane, cam.stereoActiveEye, frustumCorners);

 Matrix4x4 frustumVectorsArray = Matrix4x4.identity;
 frustumVectorsArray.SetRow(0, frustumCorners[0]);
 frustumVectorsArray.SetRow(1, frustumCorners[3]);
 frustumVectorsArray.SetRow(2, frustumCorners[1]);
 frustumVectorsArray.SetRow(3, frustumCorners[2]);

 sheet.properties.SetMatrix("_FrustumCorners", frustumVectorsArray);
 sheet.properties.SetVector("_CameraWS", camtr.position);

 //...
}

In the vertex program select the correct corner to have it interpolated:

struct v2f
{
 float4 vertex : SV_POSITION;
 float2 texcoord : TEXCOORD0;
 float2 texcoordStereo : TEXCOORD1;
 float4 ray : TEXCOORD2;
};

v2f Vert(AttributesDefault v)
{
 v2f o;
 
 // ...
 
 i.ray = _FrustumCorners[o.texcoord.x + 2 * o.texcoord.y];
 
 return o;
}

And then use that in the fragment shader:

float4 Frag(v2f i) : SV_Target
{
 half4 color = SAMPLE_TEXTURE2D(_MainTex, sampler_MainTex, i.texcoordStereo);
 float depth = SAMPLE_DEPTH_TEXTURE(_CameraDepthTexture, sampler_CameraDepthTexture, i.texcoordStereo);
 depth = Linear01Depth(depth);

 //float dist = ComputeFogDistance(depth);
 float dist = length(depth * i.ray);

 half fog = 1.0 - ComputeFog(dist);
 half gradientSample = 1.0 - ComputeFog(dist * _Spread);
 half4 fogColor = SAMPLE_TEXTURE2D(_FogGradient, sampler_FogGradient, gradientSample);
 return lerp(color, fogColor, fog * fogColor.a);
}

Easy enough, right? Or so I thought. This did not work at all! For some reason the interpolated rays were incorrect in the fragment shader. I spent the rest of the day with debug rendering, comparing results between the GlobalFog effect and mine, but I failed to find why interpolation seemed broken.

After a good night's sleep (solutions are always found after a good night's sleep) I decided to dig into the source code of the PostFX system. It never occurred to me before that at the end of the Render call in my effect it says "BlitFullscreenTriangle", where in all other legacy post fx examples it says "Blit". In the source code it literally says:

// Use a custom blit method to draw a fullscreen triangle instead of a fullscreen quad
// https://michaldrobot.com/2014/04/01/gcn-execution-patterns-in-full-screen-passes/

Right, ok, that explains a lot, we're not interpolating a quad but a triangle that covers the entire viewport, which is apparently more cache friendly and thus faster. The coordinates look like this:

Where we used to have four vertices between -1 and 1 on both axes we now have a triangle between -1 and 3. Thus we change the provided corners:

public override void Render(PostProcessRenderContext context)
{
 //...
 
 Camera cam = context.camera;
 Transform camtr = cam.transform;

 Vector3[] frustumCorners = new Vector3[4];
 cam.CalculateFrustumCorners(new Rect(0, 0, 1, 1), 
  cam.farClipPlane, cam.stereoActiveEye, frustumCorners);
 var bottomLeft = camtr.TransformVector(frustumCorners[1]);
 var topLeft = camtr.TransformVector(frustumCorners[0]);
 var bottomRight = camtr.TransformVector(frustumCorners[2]);

 Matrix4x4 frustumVectorsArray = Matrix4x4.identity;
 frustumVectorsArray.SetRow(0, bottomLeft);
 frustumVectorsArray.SetRow(1, bottomLeft + (bottomRight - bottomLeft) * 2);
 frustumVectorsArray.SetRow(2, bottomLeft + (topLeft - bottomLeft) * 2);

 sheet.properties.SetMatrix("_FrustumVectorsWS", frustumVectorsArray);

 //...
}

We select the correct corner via the vertex coordinates:

v2f Vert(AttributesDefault v)
{
 v2f o;
 //...
 int index = (o.texcoord.x / 2) + o.texcoord.y;
 o.ray = _FrustumVectorsWS[index];
 //...
 return o;
}

And done! When we now look around us the fog stays the same:

If you looked closely you've also seen some height fog in the gifs, I'm still working on that but expect another update soon on that topic :)

02 August 2018

Stylistic fog from Firewatch with Unity's PostFX v2

A friend of mine (Kenny Guillaume) asked me if it would be possible to implement a fog effect as in Firewatch:

The picture above is taken from this video of the GDC 2015 talk on the art of Firewatch, where they explain how they implemented it.

The effect is simple enough: apply fog as a post process effect and for each sample fetch the fog color from a gradient texture. I even copy pasted the gradients from the same video:

My first take on this was a MonoBehaviour where we apply this effect in the OnRenderImage override. While this yielded good results, this is not how things should be done nowadays.

No sir, now we have Unity's PostFX V2 which you can enable via the package manager. I've seen this new library at Unite Berlin and was really impressed by it. It is a well designed system if you ask me!

So the challenge was to incorporate this "stylistic fog" effect (as they called in the Firewatch video, I rather call it "Stylized Fog") into the PostFX V2 system. In a Post Process Profile the settings look like this:

I managed to get this result by consulting the other effects that are available on github (since PostFX v2 is completely open source) and this very nice tutorial on custom effects. Definitely check this manual on the new PostFX system too.

The cool part is that there are only three code files required: a shader, a script and an editor script - wonderful! They really made it super user-friendly to extend their system with custom effects. This is a screenshot of my programmer-art terrain + Stylized Fog with the firewatch gradient applied:

Just imagine what an artist could do with this!

If you plan to use this, be aware that this effect replaces the regular fog in Unity. In other words you need to disable fog in the lighting settings:

If you enable fog in the lighting settings, the post process effect will be disabled and vice versa.

Be aware that, since this effect needs the depth buffer, you should not use MSAA, since the depth buffer will have no AA. Instead enable a screen space AA effect to fix this. This is the setting I used for the above screenshot:

Of course I added all this to my Unity Toolset repo, so have fun! I'm eager to receive any feedback on this!

[Edit] I put some extra work into this, as it turned out to be not completely ready, read about it here.

29 July 2018

Take a screenshot in Unity

I am working on a mobile app for the home poker tournament manager (HPTM) and while preparing the google play store for release I had to create some screenshots to display in the store. In the editor I can easily select the different resolutions and aspect ratios that are supported by the app, so I googled for the best way to take a screenshot from within the editor. It surprised me how many different solutions exist for this mundane task.

A lot of people write scripts that take a camera in the scene and render a screenshot from that. There are basically three methods that these scripts can use.

Texture2D.ReadPixels can even capture a part of the screen.
A slight different way is using a RenderTexture but that is almost the same as using ReadPixels, since at some point you'll need that call anyway. Instead of using the default rendertarget you're reading from a custom render texture.
ScreenCapture.CaptureScreenshot which replaces the deprecated Application.CaptureScreenshot. The main advantage is that this method also captures the UI that is rendered as "Screenspace - Overlay". The HPTM is almost entirely rendered in overlay, so this proved to be the only viable option.

A possible implementation of all three these methods can be found here.

I read some post where a user complained that this should be a default thing to do in the editor, supported by Unity itself. But reading all the different posts and comments on the forums and answers, it seems to me that there are a lot of different use cases, making it impossible for Unity to come with a standard default way to capture screenshots that would be useful to all developers (and simple to configure and use).

There are also some plugins available on the asset store that promise to be awesome tools. I tried only this one and it did not fit my needs. Perhaps there are paid solutions that cover it all, but since it is so easy to write it yourself, why bother?

Another "disadvantage" of all these scripts is that you need to attach them to some gameobject in your scene, while all I want is some editor code that has no impact on the game at all. It shouldn't even be compiled together with the game code! So I quickly wrote my own version of a screen capture script, adding yet another one to all the others than can be found on the internet :)

I added it to my UnityTools repo. Main advantages of this script? It uses the ScreenCapture functions, generates filenames with increasing numbers and a very, very, simple UI.

Always happy to receive feedback!

10 July 2018

Bend the world with Unity's Shader Graph

Hurray! Unity 2018.2 is out and that means: vertex positions are now accessible in graph shaders!

This means that the world bend shader from this previous post can now be implemented as a node! The cool part is: I only added one .cs file containing a custom node and it works with the existing code.

Add the "World Bend" node into the graph, and define 3 properties "Horizon", "Spread" and "Attenuation" (check my previous post to see what they are for). Connect the output as input for the position of your Master Node. Do not expose the three properties, but give them these reference values respectively: "_HORIZON", "_SPREAD", "_ATTENUATE". That way they receive the global values set by the World Bender script on the camera. And that's it! There is only one extra file:

using System.Reflection;
using UnityEditor.ShaderGraph;
using UnityEngine;

[Title("Custom", "World Bend")]
public class BendNode : CodeFunctionNode
{
 public BendNode()
 {
  name = "World Bend";
 }

 protected override MethodInfo GetFunctionToConvert()
 {
  return GetType().GetMethod("Bend",
   BindingFlags.Static | BindingFlags.NonPublic);
 }

 static string Bend(
  [Slot(0, Binding.ObjectSpacePosition)] Vector3 v,
  [Slot(1, Binding.None)] Vector1 Horizon,
  [Slot(2, Binding.None)] Vector1 Spread,
  [Slot(3, Binding.None)] Vector1 Attenuate,
  [Slot(4, Binding.None)] out Vector3 Out)
 {
  Out = Vector3.zero;
  return @"
{
 Out = mul (unity_ObjectToWorld, v);
 float dist = max(0.0, abs(Horizon.x - Out.z) - Spread);
 Out.y -= dist * dist * Attenuate;
 Out = mul(unity_WorldToObject, Out);
}";
 }
}

As always, you can find a small demo on my unity toolset repo. I'm open for feedback!

06 April 2018

Trashing the cache: C++ vs C#

In my previous post I showed how data locality can improve your program's performance. Since then I have been wondering whether these same optimizations are valid in C# too. (If you haven't read the previous post you'd better do that first, or this won't make much sense)

Integer array

Thus again, we loop over an array of integers with increasing step size, inspired by this blogpost. The goal is to illustrate the impact of cache-fetches, where we need to fetch data from memory into the cache instead of having a cache-hit. In C# we have almost identical code to the C++ version:

const int length = 64 * 1024 * 1024;
var arr = new int[length];
for (int k = 1; k <= 1024; k = k * 2)
{
  for (int i = 0; i < length; i += k)
  arr[i] = i;
}

We measure how long it takes to go through the array and perform an operation on each value, with an increasing step size of powers of two. That gave me these results:

That's quasi identical to the C++ version:

In other words, we're just as memory-fetch-time bound in C# as we are in C++. This should not come as a surprise.

GameObjects

We create a similar GameObject class as in the C++ version:

class GameObject3D
{
  public float[] transform = {
    1,0,0,0,
    0,1,0,0,
    0,0,1,0,
    0,0,0,1
  };
  public int ID;
}

Here we stumble upon a difference between C++ and C#: creating an array of these objects in C++ allocates the memory and initializes it with the default constructor on every instance. In C# we must write this ourselves:

GameObject3D* arr = new GameObject3D[length];

vs

GameObject3D[] arr = new GameObject3D[length];
for (int i = 0; i < length; i++)
  arr[i] = new GameObject3D();

The timings are very similar. Again, we are bound by cache speed:

Here's my alternative again:

class Transform
{
  public float[] values = {
    1,0,0,0,
    0,1,0,0,
    0,0,1,0,
    0,0,0,1
  };
}

class GameObject3DAlt
{
  public Transform transform;
  public int ID;
};

Which yields this graphic:

Wait what? What happens here? The chart has a different shape than the C++ version and is almost twice as high? C# clearly introduces some overhead here. The talk of Joachim Ante on the new Entity-Component system of Unity gave me the idea to try a struct version of our classes; because we are creating the game object instances one by one they're not guaranteed to be located next to each other in memory. So I tried this instead:

struct SGameObject3DAlt
{
  public Transform transform;
  public int ID;
};

Unlike arrays of classes, arrays of structs are co-allocated in memory and initialized, causing the same speeds as we had in C++ (even a little bit faster!):

Ok, that's it. We re-created all tests in C# and we achieved quasi identical results. In conclusion: C# is just as memory-fetch-bound as C++, and in C# you carefully need to think whether or not you want to use structs instead of classes. But the fact that we achieve at least the same timings also shows that C# is as fast as C++ in these rather contrived tests.

24 February 2018

Trashing the cache

Recently I switched jobs. I now pretend to know a thing or two about game development as a lecturer at Howest DAE.

One of the courses I'm teaching has the beautiful name "Programming 4". The topics are software design patterns, memory management, threading and networking.

The first week we talked about the cache and the CPU pipeline. I have always known about these things, but it wasn't until I had to explain the concepts to students that I really took an in-depth look at it.

Integer array

When memory is used in your code, it gets fetched from main RAM memory into the cache of your CPU. When you need a single integer it wont go fetching that single value, instead it will fetch an entire cache line, containing - among other things - the integer you requested. If the data next to the integer is used next by your program, it won't have to be fetched anymore, you can use it immediately. That's a cache-hit. Instead when the data is not there and you need another fetch from memory, that's a cache miss. Check out the data locality chapter from Game Programming Patterns by Robert Nystrom for more details.

The first assignment was to loop over an array of integers with increasing step size, inspired by this blogpost. The goal is to illustrate the impact of cache-misses, where we need to fetch data from memory into the cache instead of having a cache-hit, where we use the data already in the cache.

Consider this loop:

const unsigned long length = 64 * 1024 * 1024;
const auto arr = new int[length];
for (auto k = 1; k <= 1024; k = k * 2)
{
 for (auto i = 0; i < length; i += k)
  arr[i] = i;
}

We measure how long it takes to go through the array and perform an operation on each value, with an increasing step size of powers of two. That gave me these results:

The vertical axis is the time it took to go through the array, the horizontal axis is the step value. The red line is what we would expect: every time we multiply the step size with two. Thus, every step we only do only half as much multiplications as before, we would expect the time to be cut in half too. But this is not the case.

Instead, we decline to some kind of minimum value, and only once the step size is over 16 we start to cut the time in half with every step. What happens?

My cache line size is 64 bytes, so we can fit 16 integers in one cache line. When we loop over the array 16 integers will be loaded at once in the cache. Consider this image:

With step size one, we use every integer in the cache line. With step size 2 we use every other integer, step size 4 every fourth integer and so on. With step size 32 however we skip an entire cache line fetch that we needed before, with step size 64 we'll skip three cache fetches etc. It's only when we use fewer catch fetches that our timings lower!

In other words, although we did less and less computations with every step size, we could not get faster than the time it takes to fetch the array from memory into cache. We were "cache-bound".

GameObjects

So instead of integers, let's have an array of these GameObjects and perform an operation on the ID field:

struct Transform
{
 float matrix[16] = { 
  1,0,0,0,
  0,1,0,0,
  0,0,1,0,
  0,0,0,1 };
};

class GameObject3D
{
public:
 Transform transform;
 int ID;
};

That results in this graphic:

This time it almost follows the expected line. It confused the students: why is there no bump now? The reason is this: the GameObject3D is 68 bytes large - we need to fetch at least two cache lines from memory to get a single GameObject3D in the cache. This means that when we process half the number of GameObjects, we also need half the number of cache lines filled, thus it will take half as much time.

Then consider this alternative:

class GameObject3DAlt
{
public:
 Transform * transform;
 int ID;
};

We now have a pointer to the transform instead of the entire struct. This results in this graphic:

Ah, there is our bump again! But isn't this worse than the previous one? Not really - compare them:

The alternative game object is processed way faster now, because we only need one memory fetch to process 8 game objects (the object is 8 bytes large). By fitting more objects into one cache line we gain more speed - we are definitely cache-bound here.

This only shows that it's important to consider the layout of your game object data in memory. The smaller they are, the more objects fit in a single cache line.

It also means that when your object is (roughly) the same size as your cache line size, it does not really matter anymore where it's located, you will have to fetch it from memory every time anyway and there's nothing else that can come along.

I thought this was cool ...

07 January 2018

Home Poker Tournament Manager

Again I participated in the Finally Finish Something Jam, as I did last year with Hopper.

This year I re-made the pokerroom home game organizer, which was a very nice and beautiful program to host a poker tournament at home with. The visuals where nice, the setup was easy. This is how it looked:

It had only two flaws: it was made in Flash and it only worked on a 800x600 screen resolution. I did use it for years however, every time setting my screen resolution to 800x600 during the tournament. Annoying if you had to do anything else on the pc in the meantime.

Last summer I decided to start on a remake with Unity, and today I finished it, kicked in the butt by the Finally Finish Something Jam 2018.

This time I made the UI with the Kenney Game Assets, a very nice comprehensive library of game assets, the backgrounds I just googled, the same for the sounds.

So with a little bit of pride I announce the Home Poker Tournament Manager, let me know what you think!

HPTM in action:

26 October 2017

Review of our code review process

August 19th 2016, twelve days before the deadline, the release of another expansion of the game world of Kweetet. I read this article online that absolutely convinced me that we had to start with code reviews, and we had to start now. The stats spoke for themselves. 80% of the bugs are found during code review? Sign me up!

Did you read the above line? “80% of the bugs are found during code review”. I’m still baffled by that.

When you’re rushing for a deadline that’s when the most mistakes are introduced, so I wanted to start with code review immediately, to capture as much of them as we can. It turned out this was very successful, we did find bugs that would have been very embarrassing in a release.

There are a lot of options in setting up a code review process, for example as described in this article, which is a pre-merge review process with pull requests and is by far the most common model, certainly when using git.

At die Keure we use perforce for the development of Kweetet, which does not know the concept of pull requests. This forced us to use a post-check-in process. More specifically, we used a MOB code review process (how cool a name is that?).

Now, a good year later, we’re still doing code reviews and didn’t change much to our initial process.

Context

Maybe I should start with a little context, because although our process works well in our team, there’s no guarantee it would work in another. Actually, it turned out it didn’t work for another team - more on that later.

Kweetet is played at school and at home during the six years that a child attends primary school. It is a story driven adventure game, where children complete quests, interact with numerous NPC’s, collect stuff, play puzzles and minigames and a lot more. A child starts playing the game in the first grade and only finishes it in the sixth. Every school year they start a new chapter in the story. And every chapter is played in another world, which means a lot of content. And a lot of code.

We have apps for Windows, Mac, iOS and Android plus a “lite” version for Android and iOS. Kweetet can't be called a small project; I didn’t count the number of lines, but we have a good 7000+ files with code, not counting the server code. This code has been written by a team of 4-5 people (seniors, juniors and interns) over the course of four years. In total, 10 programmers worked on the code.

We could have done this faster, but we faced a major technical hiccup in the middle of our development: the game used to be played directly in the browser via the Unity webplayer plugin, but alas, browsers decided they didn’t want any plugins anymore forcing the webplayer to retire. We tried to convert the game to WebGL, but we learned the hard way that this technology, promising as it is, has no way near the performance (yet) as the webplayer had. This forced us to convert the game to installed versions on several platforms. We lost a lot of time on that.

So in short the context is: lots of code, a small team with interns coming and going, over four years. Only the last 1.5 years we had code reviews. By now it’s safe to say: we should’ve done this by the start, we would’ve been ready perhaps a year earlier.

As said before we don’t use git, which is a common tool in code review processes, but Perforce. Completely unlike git-flow, the whole team works on the trunk (and for good reasons, but not in the scope of this post to elaborate on). So we’re actually reviewing commits rather than pull requests, if put in a git context. There is collective code ownership.

Our process for MOB code reviews

Right after lunch, we sit down with some team members for maximum 45 minutes (it’s often shorter) and review all commits since the last review session. Right after lunch, because then we don’t interrupt anyone’s “zone”.
Anyone who wants and/or has time can join, but you’re not obliged. In general we’re doing these reviews with 3-4 people; with more, the discussions can become lengthy.
We do this reviews on a projector, with someone behind the keyboard going through the files. There’s no designated driver, anyone who wants can take the wheel.
We don’t exceed the 45 minutes limit, anything that isn’t reviewed we review later. Programmers note the bugs that are found and fix them afterwards before continuing on what they were doing.

These are not hard rules, if no one is free, or there haven’t been many commits, the review is postponed. Sometimes we skip to important parts first and then review the other things later.

We try to keep to a daily review, but at least once a week it happens that we skip one.

The good

Less bugs

Obviously, we have less bugs. We don’t measure anything, but we do find mistakes regularly, so these are all bugs that are squashed. This is actually the least important advantage (but an advantage nonetheless).

Better code

The more important advantage is not so much that there are less bugs, but that the code is just better. It becomes more readable, it is more performant, it is better structured.

The above situation is one that could occur in the past, but doesn’t anymore.

Because we talk with each other on a daily basis about our code, we discuss the chosen solutions on problems. We share a lot of views and opinions with each other, we discuss why we prefer certain approaches, we share ideas, we tell each other what we perceive as readable code, we have many good laughs.

Thus the code became very consistent, all members of the team write code in pretty much the same way now, and we all changed our ways to get to that point. Voluntarily.

Better developers

This made us all better developers, we learned solutions to problems we would’ve never thought of, we write better and more performant code, we all know and understand almost the entire code base.

Better team

It made us a better team. We have our share of heated discussions, certainly at the start, but it all happens in mutual respect and it made a stronger team. There was also a lot of humor. A lot of it :)

Respect and team spirit among team members can really improve the quality of the code, since everyone is open for improvement and feedback. These code reviews introduce that.

More resilient

We are more resilient to changes, by which I mean members of the team entering or leaving. When a team member joined us (often interns) they quickly learned the codebase and also learned our approaches. On the other hand we were challenged by these new team members; we had to defend to them why we did the things the way we did them.This brought new insights to the team and to the new member.

When we weren’t doing these code reviews, we often ended up with very alien pieces of code in our codebase written by interns who didn’t know any better. We still suffer from that.

And when a team member left, everyone knew the code he/she worked on so there is little to no need of knowledge transfer sessions.

Collective code ownership

Because everyone has seen the code grow, has discussed the changes, had the chance to contribute (even without writing any code), there is a stronger sense of code ownership among the team over the complete codebase.

We talk to each other

Instead of writing our remarks behind a computer, we sit together and talk to each other (Of course, this is only possible when your team is not spread over different locations in the world.). Differences in opinion are voiced immediately and can be discussed in an open manner. Misunderstandings are detected early. It creates a stronger team spirit.

It’s fast

Typing a sentence is not as fast as saying it. And when you’re typing, you really need to weigh your words, lest they are not misinterpreted. Discussions in text can generate a lot of back and forth messages, costing more time.

Less delays

The review of pull request can take time before someone had time to do them, and by the time you get the feedback you’re already on working on something else, as happened with the author in this article. By reviewing commits rather than pull requests, we review code very early, even while it’s still in development. This can seem obsolete, since the programmer might detect his own errors before he’s done, but more often the programmer is just helped by this. Even better: if we notice that a programmer is implementing a system fundamentally wrong, we detect it before much work has been wasted. This is especially the case when interns join the fray.

The not so good

It didn’t work for the webdev team

We also have a team of four web developers who tried to implement the same process, but they stopped after a few months. They considered it a waste of time and instead started to review pull requests in git from time to time via the computer. Discussions happen via slack.

Perhaps important to note is that they used to work via a weak code ownership model (each programmer is responsible for a separate part of the code or a separate project), this makes the open code review process not such a good fit.

We mostly review code

Since we review the commits one by one, file by file, we’re mostly considering small local changes to files and new code files. We’re not looking at the big picture in a structural manner. However this is not such a bad downside; although we don’t review systems in a consequent process, we often discuss the architecture of the game when a related issue pops up

We can get behind

Sometimes we get behind with our reviews, when most of us can’t attend the review, we skip it. We try to keep this to a minimum, but sometimes we skip a few days, and then it’s possible we have a lot on our plate. Nothing to do about that, we just do the work and sometimes extend the duration of the review session a bit. The problem is that this can become a drag and that causes us to be less attentive. We try to avoid this problem as much as possible, by having review sessions nearly every day.

No history

An advantage of written code reviews is that you have a history of them. You can refer to previous reviews for motivation of arguments. You can cross check how you dealt with similar issues in the past. This is something we don’t have in the process. Can’t say I miss it though.

On the spot

In the proposed process, we need to review and spot defects on the spot. This can be hard sometimes. It’s a big advantage of reviewing pull requests on the computer: you can take all the time you need to review a file/codebase and you can even take a complete checklist and go over the code at length.

Conclusion

The team and myself all feel that this MOB code review process improved our code, our product and ourselves. For a small co-located team, working on the same project, I think this is a good approach.

Feel free to share your own code review experiences below! I'm starting as a lector in game programming shortly and I'm playing with the idea to incorporate code reviews in the lessons, so any feedback is more than welcome!

24 May 2017

Nordic Game 2017 takeaways - part 2

(Read part 1 here)

The second day was even hotter, the talks as well!

Analytics

The first talk of the day was about an algorithm for fun and profit, turned out it wasn't really an algorithm but rather a (very) quick overview of analytics concepts. All in all it was just promoting their analytics platform.

The slide here illustrates the metrics that drive revenue and this was the algorithm. I missed the fun though.

Design patterns for distributed game servers

Say "design patterns" and you have my attention :). This was also a very quick and short talk about three aspects of game servers. Again it was a bit promotion for their own server software Nakama, which is freely available on Github.

It was about session management and JWT tokens, which is a en encoder format for session data that can detect tampering. If you put non-identifiable user data into tokens and after authentication, you cache the token on the device. That way you have some info about the user without having to authenticate again, saving on database queries.
Have a field to track when the token is expired.

About databases the tip was to understand the database engine you use and how it behaves for each kind of query you need. Postgres has the most options when it comes to indexing. In general, use key-value tables as much as possible.

The talk made the software look good, if I ever have the time, I would love to port Kweetet over to this software.

Mundaun

I had some time left so I started to check out some of the games on display. It was my intent to check as much of these games as possible, but I didn't get very far. Anyway, I started talking with Michel Ziegler, the developer of one of the Swiss games, "Mundaun".

The game was still a bit rough and needs some more visual love, but it already had a very beautiful aesthetic that made it look promising. What was really impressive is that he made this game all by himself and was now promoting it at this event! Good luck to him.

Soft launch

Next up was "How to soft launch successfully" by Paul Conway.

There are three phases to a soft launch:

Planning (smart!)
Retention soft launch, where you try to optimize your retention and daily active users (DAU)
Monetization soft launch

Two to three months is enough for the retention soft launch, as long as you can update easily so you can perform a/b testing on all the important features. Good retention rates are 40% for day 1, 20% for day 7 and 10% for day 30 retention.

The slide on the right says it all to get these retention: "Cater for different skill sets", make sure your game doesn't get boring for skilled players and too hard for less skilled players, in other words: adapt.

Denmark, for some reason, is a land with high retention rates, so you can start there for the soft launch.

Consider returning players, when a player returns to your game, make sure he has a pleasant experience: make the game a bit easier for players who have been away for quite some time, make it challenging for players who return regularly.

And then follows the monetization soft launch, you can start experimenting with in app purchases. The conversion rate of your game will (or should) be around 2% of the total number of players, in other words: 2% of the players make an in-app purchase in your game. The best purchase that you can offer your players is the starter bundle. Make sure that the starter bundle has clear value.

In the end you must ensure that the LTV is greater than the CPI. (Life Time Value and Cost Per Install)

Quite some stats came from this blog and the gamesparks blog itself has also some interesting topics.

Hitman

Senses of the Hitman AI was a cool talk! They showed how the line of sight system works for the AI, how their current state influences the view, how they respond on what they see, and how they hear. I didn't write as much down as I should have, I think I was too busy looking at the visual candy :)

Some of the content was similar to this GDC '13 talk, but they've improved a lot since then.

Maybe trivial, but a very important idea is how they clearly differentiate between what the state of an AI object is and what other AI's know about that object. By seeing or hearing things, what they know about an object changes.

PBR on mobile

We're not there yet when it comes to high end graphics on mobile devices, but we're getting there. It's only a matter of time. In the meantime I had to put on my thinking cap because the session "Bringing physically based shading to mobile" was a very technical and deep session. It's a repeat from his session at GDC '16 and this one I think, but the topics differed somewhat.

Full PBR on mobile devices is not possible yet, but this talk explained in detail how they achieved triple-A visuals (not graphics) on a mobile in CSR2. They started with a full PBR rendered concept and given the game's characteristics skipped or optimized where possible.

For example they don't do dynamic shadows, since the game goes to fast to notice them. Also they made sure that light always remained static, from one direction, enabling to use pre baked image based lighting in cubemaps, or rather prefiltered mipmaped radiance environment maps. To generate these cubemaps they used this adapted version of AMD's cubemapgen tool. A detailed explanation can be found in that article.

These cubemaps can best be sampled with the texCUBElod shader instruction, however not many mobile devices support it, they had to use texCUBEbias instead. It's supported, but heavier on the shader.

For AAA visuals you need HDR, but mobile devices don't support these kind of textures. Therefor they changed the cubemaps to contain the low range values in the RGB channels and the high range values in the alpha channel and change the appropriate shader code accordingly.

For the cars they could get the albedo values from the manufacturers (obviously), but for all the other scenery they needed to get the values elsewhere. You can try and capture values yourselves, but it turns out that photographers often provide these values with their work and they were able to use these. I haven't been able to find a site like this, so if anyone knows such a link please do share in the comments!

More mobile graphics

Right after this heavy session, I went to "Optimizing our renderer for Metal on iOS" by Timo Heinäpurola. He shared his slides for this talk here.

The concept of triple buffering here is interesting and there are lots of good optimization tips in the slides. I for one will try what the effect is of using smaller texture types in Kweetet.

Unboxing Unreal 4

Last talk of the day, by a bearded lady. The team started a new project with Unreal and the talk was a collection of lessons learned.

Unreal is C+++blueprints, you can never have one without the other.
Always overload CharacterActor and MovementComponent
Use the store! The game they showed was almost completely made with store assets
But put them in a separate project and import only the content you need in your own game. This is actually a tip that is true for Unity too.
They gave the grass more depth with contact shadows.
They had a cool post process effect for their fog of war
The camera transition system was custom made and very cool

Awards

Then came the Nordic Game Awards, which was a very weird show. The host Niels Forsberg was great! Apparently he's a Danish geek/comedian bit like our own Lieven Scheire.

The award winners were just weird, it seemed as if they were not so happy about it, saying just "thank you" when they received a reward or nothing at all. One said: "Backup your files, eat your vegetables, visit your parents". Another one was very emotional and asked all the other nominees also on stage. Very weird, but good fun :). One winner didn't show up, so some random guy came up the stage and took the bottle of champagne :)

Booze

I couldn't afford to be tired in the weekend so I skipped the Nordic Party, nevertheless there was good food and good beer and good company!