So we have our semi-working webgl build. I still had to re-enable parts of the code that I commented out, so the adventure isn't over yet :)
A bug with RawImage
In our dialog system we use images to represent the player and the NPC's talking to each other:
But on the webgl build the scale of the images were completely wrong. I made a separate project and could reproduce the problem, resulting in a bug report that the QA people of Unity could confirm:
In the code we assign a texture to a RawImage object and then call "SetNativeSize()", but as you can see this doesn't happen correctly in a WebGL build. See here for the development on that. In the meantime I changed the code to use an Image instead of a RawImage. Instead of a texture for the RawImage, we now need to create a sprite from the texture and assign that to the Image which works, so I'm in the clear for now.
The battle with memory
Our game is a memory hungry monster, when we load the first scene Firefox takes +/- 1.8GB of memory! And that's without the asset bundle memory leak problem from my previous post. That problem has been solved since 5.2.3p1, but it appears that there are still some issues, the memory doesn't leak anymore, but it still takes a lot; Firefox goes to 3GB when we load the assetbundles in the cache. In other words, not usable yet.
So obviously a lot of memory is going to the assetbundles, but how much? Why? And where does the rest go to? I asked on the Unity forum for some leads on this. I learned that surfing to "about:memory" in Firefox gives you a complete breakdown of its memory use, cool! But apart from "a lot of memory goes to javascript handling" I learned nothing useful from that.
To limit the javascript impact on memory it is key to have as little code as possible, and certainly to have very few to no plugins, since they can drag in a lot of .Net code that you don't even use. We removed the UniWeb plugin from our game, since we don't use it in the webgl build and the uncompressed development build size went from 195mb to 119mb! That didn't translate to a big difference in Firefox' memory abuse though.
Audio
Audio seems to be an issue by itself on WebGL. As the manual clearly states, audio is handled completely different compared to the other platforms, and unfortunately it shows. In your project, if you use wav, mp3 or ogg exports of the same audio file, they should all sound the same, which they do in the editor, windows standalone and webplayer builds, but in WebGL there is a difference between the three! I found that mp3 files are best when targeting WebGL, but it still strikes me as odd why there would be a difference since they are all converted to AAC.
5.3.x headaches
And then finally Unity 5.3.0f4 was released! Hurray! Lots of fixes and improvements!
Or at least that is what was promised, but it was the opposite.
The memory (ab)use of firefox went from 1.8GB to 2.4GB!
There are numerous reports on performance issues, like this one and this one by myself
Audio does not work anymore on the Firefox developer build, and although Unity QA claims it does work, I have 5 pc's at work where it doesn't, hopefully that gets fixed before it goes live (March 2016 to be precise).
So that's it. I got until the end of the year to create a decent WebGL version, but because of all the bugs in Unity (most not even WebGL related) I cannot in good conscience declare that I have a WebGL build that is remotely ready to go to the production branch. I'll have to abandon this endeavor and instead we'll create a standalone build with an installer and hope that users will not complain too much about that.
I hope I find time here and there to continue because I still believe it should be possible, but first Unity will have to get their shit together. Where Unity 4 was a stable engine, Unity 5 is biting your nails every time there is an update and hoping nothing obvious went broken.
So now that we have a +/- working version running with Unity5 it was time for the next step: making a webgl build. That wasn't hard, getting the build to run on the other hand was.
1. Communicating with the api
You can't start Kweetet unless you're logged in, so one thing that we needed to get working first was the communication with the api. We used sockets in the webplayer with the UniWeb plugin, but as you might know sockets aren't supported in webgl, so we had to look for other solutions. Our website had a javascript class that already communicated with the api via CORS for other purposes than the game, so we wrote a javascript plugin that uses this class. The documentation on how to do that can be found here, but what's missing is how to use callbacks. Our function needs to call the api and trigger a succes or error callback. It ended up looking like this:
Just today I stumbled upon a subtle problem with the code from the manual: to create the buffer to write a javascript string into, the manual uses the length of the string to create a buffer of that many bytes + 1. But if the string contains multibyte characters as in for example "Dierië" then you need to allocate more bytes than characters, or you have a buffer overflow!
The javascript function writeStringToMemory does not look at the size of the buffer so if the string is larger than the buffer it simply writes outside of it. This of course caused many seemingly random crashes in our game, which made it difficult to find the cause at first. The solution is using the function that counts the bytes in the string, and thus enables us to create a buffer of the correct size. I mentioned this on the forum and they're updating this.
2. Memory leak
So with the api in place we can start the game, but even before the first loadscreen was done the browser ran out of memory. I had this in a latest version of Firefox 32bit and it was even worse in Chrome (Chrome has proven to be very bad at most of the things I tried). I then installed a 64bit developer edition, since that one would be able to go beyond 3GB memory and it did: the memory consumption went up to 10GB (!) and kept rising.
So we temporarily use the regular WWW download and now the memory leak issue is gone. It's supposed to be fixed in 5.3 so I'll try again then.
3. "Could not produce class..."
Another hard to solve problem was that our game crashed when loading a new scene with LoadLevelAsync. Apparently (thanks Marco Trivellato) this is caused by the stripping of classes you never reference in your code but do use in your scenes that you then load via assetbundles. You know you have this issue when you start seeing log entries that look like "Could not produce class with ID xxx". This gives you an ID which you can use to lookup here. For example, we had a few WindZones with ID 182 in some scenes that caused this issue.
4. Development vs Release build
With a development build it's easier to read stacktraces which you need to fix bugs, but most browsers aren't capable to handle the large output of that build (a .js file of 192MB!). Firefox 32 bit tries and succeeds most of the time, Chrome gives up before it executed anything. So I really recommend using the 64bit developer edition of Firefox, that one can handle any build, no matter what I enable or disable.
Luckily, chrome can run a release build, so we're in the clear. But it's still not as fast as Firefox.
Once these issues were solved we finally have a running build! Hurray! I commented out some parts of the game however to get this far, so I have to re-enable those en see what errors I need to fix there. But I'm cautiously confident that we'll succeed to port our game to webgl, how cool is that!
Because of the deprecation of plugins on first Chrome, then Edge and now even Firefox we are forced to convert Kweetet to Unity 5 so we'd be able to create a WebGL build. In the hope that a WebGL build will even work, that is, because WebGL isn't nearly as powerful as a plugin can be (yet - That's why I don't understand why browsers deprecate plugins without providing a decent alternative. Sure, in the future WebGL will be brilliant, I believe that too, but what about the present?).
To start we had to do some preparations, first one: convert from NGUI to the new Unity UI. The reason is simple: the less code there is in the webgl build, the smaller it will be and the less errors there can be. so we're working to ditch all big third party libraries and use the "native Unity" ones.
This step went fairly easy; NGUI is very similar to the Unity UI so the conversion was easy. There are a lot of improvements that made our life even easier than before.
The second step then was to actually make sure the project works in Unity 5 (the NGUI conversion was done in 4.6.x). This conversion was, in short, "less smooth".
Don't get me wrong, I'm a big fan of Unity and I understand most of the decisions they've made when changing Unity so I'm with them all the way, but I feel that they are rushing Unity 5 a bit too hard. The features they introduce or change have many bugs and are sometimes badly documented. After a month's worth of work we didn't have a stable version of the game at all. We still have the 4.6.8 branch though, so we can continue to develop, but the 5.2.x version has many issues. Positive is that forum posts get answers quickly from the Unity crew and things are getting fixed.
My intent is to document as much as possible all issues I encounter and what I did to solve them, in the hope that this is helpful for others. I want to do this as I go along, since the issues remain fresh in memory when I write this, so this will be a post in multiple parts.
1. Perforce
We use the P4Connect plugin from perforce itself, version 2.6. This is a 32-bit version we needed to upgrade, the latest version is 64bit compatible but doesn't work at all; it keeps losing the perforce connection when pressing play or other actions in the editor, rendering it completely useless.
This is not a Unity feature you say? I agree, but the only reason we use the P4Connect plugin is because the default perforce integration in Unity isn't working either, or at least never did for me. For example, P4Connect checks out every file you youch, together with the meta file and it even moves files in perforce when you drag them in the editor. The native perforce integration did not do this. At the moment we got it up and running with the 2015.2 version. The currently latest 2015.3 version however is not capable to create a correct connection with the server, so don't get that one.
2. VSTU integration is/was broken.
Microsoft and Unity announced together that Unity has now VSTU integrated. I was very happy to hear that because I'm an avid user of that plugin. But I was very disappointed of how the result turned out to be. There are numerous bug reports on this matter, for example here and here. Since the 5.2.2 version things stabilized a bit, but it's not as good yet as it used to be. For some reason the project needs to reload very often, setting up a new perforce connection every time that happens and resharper re-parsing all files anew. This wasn't the case with the old VSTU.
3. Asset bundles have changed - a lot
The folks at Unity clearly saw that many people created complex assetbundle creation setups for their games, so they came to help and created one of their own that would cover most use cases. It took me quite some time to convert our system to theirs. They did similar things as we did, but at first the new system was badly documented. Now they created a magnificent sample project and extensive documentation. Things would have been easier if I had it at the start :)
It surprised me a bit how much the system had changed, because I didn't feel the need. It is now supported to define assetbundles in the editor, but for a big production I cannot see that as feasible. Manually assigning the correct assetbundles for each asset in the game sounds like a tedious and very error prone work.
At Unite Europe 2015 there was a presentation about asset bundles, very interesting and very similar to our own system, but alas only applicable to Unity 4.x. A bit weird to see that presentation there while everything else was about the new Unity 5.
We're currently in the progress of converting our game to Unity5. Not as easy as I had hoped it would be, but that's for another post :)
One of the things that changed in Unity5 is that concave triggers are not possible anymore. They used to be, but they triggered enter/exit events every time a collider passed through the surface of the trigger. Now you just get an error that you can't have concave triggers.
Too bad for us, because we used a lot of them via my Area Trigger system. The code in that post is obsolete in Unity 5 (as are many of my other Unity-related posts). So I've updated the code on bitbucket.
To recap: the idea of the area trigger concept is that it exists of a height and a 2d polygon, that can be concave. This defines a concave volume. We can then add a mesh trigger and flag it as convex. We now receive events from the convex volume, we check these against the concave volume and if the collider is in the concave volume we pass an area trigger event.
The only downside now is that we check whether the position of the collider is in the area trigger instead of the whole volume (which would be far more difficult and obviously the reason why even PhysX doesn't do it). For our purposes however this is enough; we use these area triggers to establish whether the player or an npc walks into a zone (or area, hence the name) in the scene.
A colleague of mine added a cool feature that was requested by our artists: you can now also use the system to create area colliders. With it, you can quickly generate simple mesh colliders as blockers in Unity, instead of having to create them in a modelling tool, where this doesn't make much sense. (I hate FBX files where visuals and physics are mixed).
I could have titled this post "Even more games in Belgium", but the event that led me to writing this post is the sad news that one of the once bigger game companies in our country has filed for bankruptcy. GriN is no more.
This is sad news, for numerous reasons
Woolfe was a promising game. I've never got around to play it but I have seen a lot of footage and gameplay - there was a lot of love and dedication in it.
My kids love the game "Forest Pals", and I mean love! They know the song in the game by heart.
Wim Wouters, the CEO of GriN is a very enthusiastic and driven game developer, who has done a lot for the Belgian Game Development scene, which makes this news extra harsh.
I wish him all the luck in the world, and lots of time to recharge his batteries. Because I know we'll see more of him in the Belgian game dev scene, he'll be back :)
In previous posts about games in Belgium I wrote that the business was expanding, rapidly. I have the impression that we're not expanding that fast anymore, but we are by no means shrinking either. Last gamescom there was a strong representation of Belgian games joined in a Belgian Booth (I should have gone this year). DAE grows in number of students (could use two of them on Kweetet, hint, hint), and news of Belgian games is popping up more and more. There are also a few entries of Belgian games in the Dutch Game Awards, curious to see if we'll win one! (And by 'we' I mean both Belgians in general and ourselves at the Kweetet team ;) )
Of course there is Larian, they have set records with Original Sin and now they're bringing an even better version to consoles. They're definitely growing strong and I'm curious to what Swen is tweeting about.
The Flemish games association is also growing stronger, I find them quite active. I really should go to one of the café events they hold, but somehow there was always something that took priority. Their list of game companies in Belgium has grown longer and I have the impression that some of them are growing well in team size. Definitely follow the Flega news and the Belgian Games Industry group on LinkedIn.
So there was sad news today - but the future still looks bright :)
I am a big fan of Deathspank. I love the art style, I like the humor, the gameplay, the combat, the inventory, the start menu, all of it. You can see that in the game I currently work on (Kweetet): some of the art style has a resemblance, the dialog system is the same and: we've got a rolling world too. I really like the rolling world effect in Deathspank, it creates a cartoony feel for the world in general. So I said the concept artist that I really wanted it in the game, he liked it too, so we implemented it. In this post I'd like to explain how we did it, what problems were encountered and how we solved them or didn't solve them.
Here is a movie that illustrates the effect in the game:
The shader
The idea behind it is actually very simple, we just displace the vertices of any mesh in the vertex shader according to a parabola. The vertex shader of the main material in Kweetet looks like this:
As you see at the top, we have two multi compile directives. The first one is used to enable bending on a global level via a shader keyword. That way all materials bend if it's enabled in the current scene. Sometimes however you do want to disable the bending, even if the world bends, for example objects that are shown in the UI instead of the scene. To do that enable the BEND_OFF keyword via the material. We use these two because we can't disable the BEND_ON keyword via the material if it was enabled via a global shader keyword.
If BEND_OFF is disabled and BEND_ON is enabled, you see that the function DSEffect is called before we multiply the vertex with the modelviewprojection matrix (DSEffect as short for "DeathSpankEffect" - I called it like that when we were still experimenting and never took the time to refactor). That function looks like this:
There are three global variables, that I'll explain further. First we put the given vertex in world space. Then we calculate it's z depth (more on that line later). Then we apply the parabola formula:
y = -a * z^2
where a is the attenuation of the parabola. It's a negative attenuation so the world goes down. (If it were positive the world would bend upwards. Can be cool too). We add this result to the y value of the vertex. Then we put the vertex back in object space and return the result. In Unity 4.x you need to multiply with unity_Scale.w, in Unity 5 this is no longer needed.
For the characteristics of our game we only need bending in the z direction, and only with a parabolic function. But off course we can use other functions that alter the y value in function of both the z and the x coordinates.
The bendcontroller
In a scene that needs to bend we place a gameobject with a bend controller script attached, that looks something like this:
We need the transform of the avatar of the player (or some other reference), the z position of the avatar needs to be the top of the parabola, so this is the offset we have to apply on the vertices of the other meshes. This is what I called the _HORIZON variable in the shader. It's the z coordinate in world space where the top of the parabola will be. It looks something like this:
In this case the avatar is walking on top of the world, but most of the time we want the horizon to be a bit higher, so that's why we have the horizonOffset value in the controller. This is the same scene, but with an offset of 3:
And that's it. There's nothing more to it than this to achieve the bending world effect. Unfortunately the effect introduces some consequences that need to be addressed, if the game is impacted by them.
Raycasting
The effect is done in the vertex shader, not on the actual vertices. This means that the meshes in the game do not bend, and thus the physics world doesn't bend. This is a good thing, because if the physics world would bend, everything would start to fall off. The same goes for pathfinding meshes; they are still valid. The raycasts that may be used in pathfinding are valid too, so everything remains ok.
But when we perform a raycast from the camera into the scene, to select objects in the world or to select the next position that the avatar needs to walk to, things are more difficult. The user will click on the visual position of an object but if the object is bended the actual location in the physics world is different.
The following image illustrates this: the user wants to click above the cube in the scene on the floor (the red ray) but will actually click on the cube. If the user would have clicked on the bottom of the cube (to select it for example) he would have clicked on the floor instead.
My first idea was to determine the position that the ray hits in the visual world by calculating the intersection of the ray with the parabola. I projected that position back to its original height and then I did a raycast to that position instead. This worked quite perfectly for situations similar to the one in the above image, however when the ray is above the horizon we don't have an intersection point, so this solution doesn't cover all cases.
The best solution of course is not to use these raycasts at all of course. This is what Deathspank does; navigation is not done by clicking on the next spot where Deathspank needs to go but with the keys or the gamepad. Interaction with objects is done when you're close to them with keys or the gamepad. In the pc version of DeathSpank you can use the mouse to navigate, so I don't know what solution they've come up with.
Animal Crossing uses a rolling world as well. I've never played it but since it's a 3DS game I don't think there's ever any raycasts needed from the camera, so they won't have this issue either.
With Kweetet we are targeting children in primary school with possibly limited gaming skills playing on a pc in the browser, so they need to be able to do everything with a single mouse click. So I had no choice but to find a solution, or we would have to abandon the idea entirely.
In the game, objects can be interacted with once they are within a certain distance to the avatar. That distance is mostly between 3 and 5 units. This means that if we create an area of +/- 5 units around the avatar where the world doesn't bend, all relevant raycasts will be correct. This is what the _SPREAD variable does in the vertex shader. it flattens the top of the parabola, creating an area around the avatar where physics and visuals match.
This turned out to be the perfect solution, raycasts work where they should and the flat area around the player is small enough to still enjoy the effect of the world bending.
Frustum culling
Frustum culling is done before any vertex shaders are executed (obviously, or it wouldn't be culling). Because of that, objects that are bended into the view of the camera but are actually outside of it get culled. The result is that these objects pop in and out when they shouldn't.
In the above image the red line and circle is the non bended world, the black line and circle is the bended world. The blue lines depict the camera frustum. As you can see, the circle will be culled, while it should be visible.
These objects are typical small and highly located in the world. Ideally, we want to disable frustum culling for these objects, but that is not possible in Unity.
[Start edit]
The first comment on this post pointed me in another direction to solve this issue, so I changed this part. In Unity 4 it wasn't possible to change the camera used for culling without affecting the camera used for rendering. It turns it that in Unity 5 this has been made possible!
This totally removes the need to, as I used to do, enlarge bounding boxes of meshes or combine them into larger objects. Instead, you can add a script to the camera that looks somewhat like this:
This adds a bit on the top and bottom of the culling camera, as you can see in this screenshot (the grey lines are the view frustum of the render camera, the colored lines are the view frustum of the cull camera):
This solution solves the culling issue completely, without any manual intervention other than tweaking the extra cull height value.
[End edit]
Particles
Another thing to consider are particles. With particles you can choose between meshes or billboards. With meshes nothing changes so you can use the same vertex shader, but billboards are passed in camera space instead of object space to the vertex shader. Thus we need a camera to world space matrix to achieve the bending effect on billboard particles. I have this in a script on my camera:
Which is basically the same one, but without the unity_Scale and using the other matrices. We override the built-in particle materials in Unity to use this bending function when appropriate. In other words, this situation is easily fixed, but it requires some careful handling of the shaders.
Conclusion
This is in short how we've implemented this technique in our game and how we solved certain problems. I'm by no means sure that these are good solutions, so I'm happy to receive feedback on all of this. I'm planning to set up a Unity project on github where I can place all the scripts I describe on this blog, with demo scenes of every technique. And if this post helps the reader to apply this technique in another game then I'm pleased to be able to contribute.
[Edit]The unity project with a demo is now available here (it's a bitbucket repo instead of github). Look for the world bender scene in the project.[/Edit]
[Edit]Since 2018.2 is out, I also have a node that can be used in a shader graph, read all about it here.
Unity's UI system used to be very limited, so most people turned to the asset store for UI plugins. One of the most popular ones is NGUI. I've been using NGUI for more than two years now. Needless to say that I was happy to hear that the developer of NGUI was going to work for Unity on a new UI system. Apparently he did that for only a year but the result I hoped for has been achieved.
Unity 4.6 and 5 have a new UI system and it is very resemblant to NGUI. I could convert my NGUI version to the unity version with minimal changes in the code. But with all new components of course.
But there was one capital feature missing: NGUI has an easy text localization system and Unity hasn't build an alternative for that. So I rewrote the localization code from NGUI so it would work with the new unity UI. I also changed a few things so it would fit my needs better, it is not a one to one conversion. I have asked permission to post it on this blog.
NGUI expects a Localization.csv file somewhere in a Resources folder. In my version you'll see there's a Resources folder containing a single asset. In the files collection of that asset you can assign any csv anywhere in your project. That way you can have multiple csv files containing localization.
The reason to do this was that the excel sheet containing the localization became very large and hard to maintain, so I wanted the ability to split it up into multiple sheets, resulting in multiple csv's.
To be able to export these sheets easily in multiple csv's I found this page which I used to write the program below. It takes an xlsx filename and an output path as input and will generate a csv for each sheet in the excel file. In my unity project I have this excel file and my build process uses this program to generate these csv's that are linked in the localization asset. That way I always have the latest version of the localization in the excel file build into the game.
[Edit]For the program below to work you need to have Excel installed and the correct oledb drivers. If you have a 64bit version of excel make sure to compile the program as 64bit or 32bit if Excel is 32bit.[/Edit]
As always this software is free to use at will, enjoy!
usingSystem;usingSystem.Data;usingSystem.Data.OleDb;usingSystem.IO;namespaceXls2CsvConverter{classProgram{staticvoidMain(string[]args){Console.Out.WriteLine("Source file: "+args[0]);Console.Out.WriteLine("Target path: "+args[1]);stringexcelFile=Path.GetFullPath(args[0]);stringcsvPath=Path.GetFullPath(args[1]);ConvertExcelToCsv(excelFile,csvPath);Console.Out.WriteLine("Done");}staticvoidConvertExcelToCsv(stringexcelFilePath,stringcsvOutputFile){if(!File.Exists(excelFilePath))thrownewFileNotFoundException(excelFilePath);//if (File.Exists(csvOutputFile)) throw new ArgumentException("File exists: " + csvOutputFile);// connection stringvarconnectionString=String.Format("Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0};Extended Properties='Excel 8.0;HDR=No'",excelFilePath);varoleDbConnection=newOleDbConnection(connectionString);// get schema, then datatry{oleDbConnection.Open();varschemaTable=oleDbConnection.GetOleDbSchemaTable(OleDbSchemaGuid.Tables,null);for(inti=0;i<schemaTable.Rows.Count;++i){stringworksheet=schemaTable.Rows[i]["table_name"].ToString().Replace("'","");stringsql=String.Format("select * from [{0}]",worksheet);vardataTable=newDataTable();vardataAdapter=newOleDbDataAdapter(sql,oleDbConnection);dataAdapter.Fill(dataTable);stringcsvFile=Path.Combine(csvOutputFile,worksheet.Replace("$","")+".csv");MakeWritable(csvFile);using(varwtr=newStreamWriter(csvFile)){foreach(DataRowrowindataTable.Rows){boolfirstLine=true;foreach(DataColumncolindataTable.Columns){if(!firstLine){wtr.Write(",");}else{firstLine=false;}vardata=row[col.ColumnName].ToString().Replace("\"","\"\"");wtr.Write("\"{0}\"",data);}wtr.WriteLine();}}}}finally{// free resourcesoleDbConnection.Close();}}privatestaticvoidMakeWritable(stringpath){if(!File.Exists(path))return;FileAttributesattributes=File.GetAttributes(path);if((attributes&FileAttributes.ReadOnly)==FileAttributes.ReadOnly){// Make the file RWattributes=RemoveAttribute(attributes,FileAttributes.ReadOnly);File.SetAttributes(path,attributes);}}privatestaticFileAttributesRemoveAttribute(FileAttributesattributes,FileAttributesattributesToRemove){returnattributes&~attributesToRemove;}}}
Edit: As per request I added a demo project here. You'll see I have a GameController class who calls Init on the text localization class and selects the correct language. In the localization asset in the Resources folder I added a demonstration CSV file.
