Over the course of the last few months, as I was reading forums and talking to people on IRC, I couldn't help but notice the staggering amount of suggestions and fairly simple requests that we've been offered. While this would be by default a fantastic thing, it triggers a returning problem that most players are blissfully unaware of: the complexity of our architecture, and our dedication to keep the architecture as robust as possible. What this essentially means is that sometimes the most trivial minuscule feature has to go through a rather arduous chain of procedures to be finally implemented on the client side. My attempt in this post today is to present you with a case study of how this happens in case of something most people would consider a single line of code and 20 seconds of work. But first, let's talk about concepts.

The principle of our system

The idea of the Perpetuum infrastructure has been engineered to be robust from pretty much Day 1, although admittedly we kept expanding on several things over time. The concept was the following: Since we're a very very small group of developers, we cannot afford turnaround time and taking time away from each other when it comes to fixing things, so the optimal solution is to allow anyone to fix anything they can. Of course, this doesn't mean that people with no interest in coding can happily add mov ax, 1 into random places into our source tree, but it's important to realize that in a project like Perpetuum (which is a rather strongly data-driven game), we have several times more the data than we have actual code. This means that as long as the artists and content folks can change the data, programmers can work independently of them, so essentially what it boils down to is exposing as much data as we can. (If you started giggling, do a few push-ups. It helps.)

The way we do it is manifold: For artists, who mostly work with standard format data (bitmaps, meshes, sound files), we have a file server where they can upload new versions of resources, and they get downloaded the next time someone starts the game. Of course this can result in some problems when we change some internal file formats, but as long as the client is able to do an update, this shouldn't be an issue. (We also have revision control, so reverting to an earlier version is easy.) On the other side, we have the content guys who work on managing the actual game logic data (i.e. item data, balancing, that sort of stuff) - this isn't as trivial because this data needs to run through both the server and the client: the server needs this data for calculating the game logic, the client needs this for displaying data and doing client-side prediction. This also needs to be both persistent and easily alterable, so for this, we need a strictly database-driven environment.

So here's how it works.

A simple example

The most trivial example I already noted when we were working on it was related to sparks: The spark GUI was pretty much finished codewise, when Zoom noted that the order of the sparks doesn't make much sense because the syndicate ones are on the bottom while the ones that are harder to access are on the top. The reason for that, obviously, was that Alf simply added them in the order he felt like at the time, assuming that we can just sort it on the GUI-level later. He was obviously right about that, except we didn't really think of a method.

The idea looks simple on the surface, but there are several solutions to fix it and all of them are either laborious or asinine:

  • We could leave them unsorted. That doesn't fly because it's annoying.
  • We could sort them on a GUI level, by hardcoding weight values. This is just a big no-no considering that any change would need to go through the client coder. (=me)
  • We could create a resource file in the resource repo that describes an order, and thus any non-programmer could edit it. Sounds good on paper, but sparks can be added or removed at any time, which means that if the resources are out of sync, the whole thing becomes a mess, and you're still working with two things at the same time. So no, this is just shooting ourselves in the foot. (Update - I might have forgotten to mention the most important part: non-programmers and markup languages rarely mix well.)
  • We could sort them on the server, by hardcoding etc etc see above yeah no.
  • We could extend the database records for sparks with a sort order value. Shiny clean solution, but with the most work to be done. Why? See below.

Implementation

So here's the entire process of how a single integer value traverses through our infrastructure. As you may see these aren't particularly complicated steps by themselves, but the amount of steps and their versatility eventually amounts up into a surprising amount of time.

First, we take the database table for sparks, and add a field. This is done on the database server, usually by Crm.

SQL design phase

Second, we extend the administrative web-based back-end (what we lovingly christened "the MCP") with the field so that we can have data to work with. This can get tricky if the field isn't just a standard number (but e.g. a new item type or an extension ID), but in this case this is just a simple text field. This is done by me.

That isn't going to do you any good, Flynn.

Let's remind ourselves of something at this point: Because of our rather rigorous security-principles, no game-related data will be kept on client side, everything comes through the server. Now because sparks are very much a core gameplay element, everything related to them will also come through the server as well. So our next step is to add server code that loads the new value from the database, and hands it to the client. This is done by Crm again, and usually involves a server restart, much to the chagrin of whoever is working on the development server at the time.

Server phase

So now our value is en route to the client, all is left for me is to add fields to the client cache and use it to actually sort the sparks in the correct order, i.e. the aforementioned "single line of code". Of course, the actual sorting algorithm code is already there, I just have to build a new comparison function, test it, deploy it as the newest client update, and ultimately the feature is done easily.

Client phase

Simple enough, but if you look back, the ricochet performed by that one measly integer value is pretty staggering.

What you have to understand here is that this is the best solution, as strenuous as it seems:

  • It's robust (no hardcoding involved, extending doesn't involve breaking anything previous)
  • It's completely "user"-driven, anyone can use the web-interface to change the values without having to worry about breaking something in the code level (mostly); database-constraints and form-validation can take care of values we don't want in there.
  • Most of the code-changes needed are just passing around values without thinking of what it really is.
  • Once the code works, it'll always work as long as the data itself is solid.

But of course, as mentioned, the downside is that it needs four different levels of changes (database, back-end, server code, client code), which is only done by two people because of the involuntary competence we acquired over the years - in a "larger" project, this could possibly take a lot of cross-department management, while here it's usually just strong language and the occasional "Eh, I'll just do it myself". However, once we're done, we can wash our hands and never think of this pipeline again, because the content will go in on one end and the code will take care of it in the other. (On a tangential note, this applies to blogposts too: This post will probably be / was proofread and reviewed by several of us.)

Exercise for the reader

Now, to make sure that you understand the gravity of this stuff, here's a little thought-experiment for you: When you right-click an item and select "Information", there's a lot of parametric data that ends up on the screen, each with their specific measurement unit and number format. Now remind yourself that some of these values may look completely different internally: percentage values that show as 5% are obviously stored as 0.05, bonuses that show up as -25% may be stored as 0.75, some values need two decimal digits while others need five, and so on.

So here's the challenge: knowing the above principle (i.e. be data-driven), what kind of system do you build to make sure that each of those values end up being displayed correctly for the player? (No prize for the correct answer, although we do offer occasional trauma counselling.)

In a later post, I will go into a deeper insight about how an actual feature (case in point the whole spark-doohickey) goes from an idea to design to concept to task to implementation, down to the level of "BUG #4170 - please move that text one pixel to the left".

One of the most frequent complaints we get from new players regarding the graphics is the quality of robot animations. This has posed a big problem for a long time as creating a more complex animation system would require a lot of time, resources and client processing power we don't have. We finally came up with a workable solution for the issue and will gradually be rolling out new walk animations for the smaller robots, starting with the very first bot players encounter: the Arkhe.

Once we're done with the small robots we'll be looking at the big ones of course, which will need to be rolled out in a single, big patch - but that's still some time away.

And how does it compare? Let's have a look:

New week, new patch - seems like we’re on a roll. As promised a month ago, we’ll deploy the spark change and extension system upgrade patch this Friday (10-28). If you haven’t read the previous blog about the topic, I’d recommend checking it out, because I won’t repeat everything there, but rather focus on the new stuff.

Sparks 2.0

Asintec industrial spark

Have you ever wondered what the tiny nanobots we call “sparks” look like? Well, now you’ll know :)

In order to give sparks a gameplay role, we converted them into passive module-like items, which provide small permanent bonuses and are interchangeable. The first pack of available sparks is already quite sizeable, with different types in terms of level, faction, specialization and price, but we intend to introduce new ones in the future too.

To get hold of a spark, however, you need proper authorization to use them (basically “unlock” them). This authorization process can either mean a one-time NIC payment, or reaching a certain level of relation with a corporation. Later on we’ll also introduce new sparks unlockable for Energy Credits (when that system rolls out), or a combination of these.

Sparkling sparks

Once you’re authorized to use a spark, you have to install it, which costs varying amounts of money. Unlike authorization, which you only have to do once, changing between sparks always has an installation cost, even if you change back to one that you already had installed. There is also a 1-hour integration period, during which you can’t install another spark. Standard sparks which you choose during character creation are all authorized from the start, so you’ll only have to pay a small installation fee if you want to change to another standard type.

Extension downgrading

Again, pardon me if I won’t repeat all the details and reasons about attribute removal and the new calculation method of extensions. Please see the previous devblog for reference.

In short, extension downgrading will be the ability to downgrade extension levels (duh), and to get back the EP and NIC you have spent on them. This feature is mainly intended for new players, to give them a chance to correct their mistakes while they are learning the game, so downgrading will only be available for 30 days after you create your first character. Since this is a new feature, though, we’re also providing all veteran players a 30-day grace period. This grace period starts from the patch on Friday.

There are a few limitations to downgrading:

Extension downgrading
  • It’s not possible to downgrade below your initial extensions.
  • It’s not possible to downgrade an extension level which is a prerequisite for another extension that you have.
  • There is a global freeze limit, which means if you go over it, you won’t be able to downgrade anymore (this also applies to old characters). This limit has been set between level 6/7, so if you have something on level 6, you'll still be able to downgrade it, but not level 7 and up. There will be a popup warning if you’re about to cross the freeze limit with an upgrade.
  • There will be a rolling limit which will only allow you to downgrade a sum of 5 levels of any extension in the last 12 hours. (We call these downgrade tokens, and they are account-based, just like available EP.)

As you can see on the screenshot, the downgrade button will tell you how much EP/NIC you would get back if you decide to do it. The lower part of the window has also been extended with more info about your spent EP and on downgrading.

Last chance to do an account reset

As announced in the previous blog, we’re saying farewell to the account reset feature in this patch. If you have any reason to perform an account reset, you can still do so until Friday! You can do it without any repercussions, since all the EP penalty will be reimbursed during the patch, regardless how many resets you did so far. This is also your only chance to get rid of an unwanted extension that you upgraded beyond the above mentioned freeze level of 6.

Closing

As much as we would like to, we can’t promise that we’ll have a patch every week now. These patches are merely a culmination of our development work during the past few months, so these are special times :) Of course, we’re doing our best to finish some features that are still in the works as soon as possible. November will be a very exciting month. Two words: intrusion and anniversary.

The weird concept of “NPC eggs” has been floating around in random forum topics for a while now. We’re of course not planning to introduce robot chickens, though I’m sure this is sad news for some.

The egg system, or how it should be called from now on, distress beacons, is a new feature that is intended for quick “on-demand” PvE battles. It also kindof replaces the roaming Observers that have been removed from alpha islands in the last patch.

The pranksters of Nia

So what are distress beacons? They are basically a secure communication device and a teleport endpoint node, which the alien factions are using to call in reinforcements. However, if you get your hands on one of these beacons, you can use them to your favor: upon activating these beacons, the alien forces will instantly teleport there, where you can easily ambush them if you are properly prepared. It is really much like doing false emergency phone calls.

Some more details:

Various types of distress beacons
  • There are two main types of beacons: the ones that get in normal multi-wave squads, and the ones that call single wave high ranked officers and commanders with shiny loot.
  • 4 factions (the industrial federation being the 4th), 3 different levels and 2 different size classes for each. This results in a total of 48 different distress beacons.
  • Depending on level and type, you might need multiple Agents to activate them. (You can find the number in their info.)
  • You can only deploy them while in a squad, even if it needs only one man to activate.
  • The NPCs brought in this way will retreat if they are not attacked for a minute.
  • Much like mobile teleports, they can’t be deployed near terminals.
  • Some of the high ranked NPCs drop observer kernels.

Squad-type beacons and the lowest level commander beacons can be acquired from artifacts (with a higher chance on beta islands) or by looting certain elite NPCs. The higher level commander-type beacons are dropped by the robots you called in with your first beacons. It works like a cascade, if you get a really lucky streak, you can get from the lowest level officer to the highest commander.

We’re also introducing two brand new tiers of modules, which can be looted from the officers and commanders. I don’t want to spoil here what these are exactly, find out for yourselves! :)

Just to get you in picture on how all this works in practice, some of us got together for an afternoon brawl with the aliens:

GIEF NAO!

We’re still working hard on some of the details, but it should be out within a month or so...

Just kidding, you’re getting it tomorrow in a patch :)

We’ll be also having a little pop-quiz event tomorrow (10.21.), some hours after the patch at 22:00 servertime (link for local times), where you can be the among the first ones to get your hands on a distress beacon! More info here. The patch itself will be at 13:00 CEST!

A more free trial experience

Well, of course trials are free, but so far you had to deal with quite a lot of trial limitations. Originally we introduced them as a precaution, but we decided now to lighten it up a bit. So after tomorrow’s patch, the only trial limitations that remain will be that you can’t send money to other players (but you can receive), and that you can’t found a corporation (and of course the 15 days of game time). All the other restrictions regarding market access, trading, chat, field containers and joining a corporation will be removed.

To make the life of corporate spies a bit harder, you will be able to see whether an Agent is on a trial or not if you check their profile information.

Of course we reserve the right to reinstate certain trial limitations if they are deemed necessary again in the future.

So, that’s it for now, see you tomorrow after the patch, and we’ll return next week with the final details on the new spark system!

As some of you might know Perpetuum currently is running completely on shader model 2.0 shaders. This decision has been made a long while ago, when we only started out on the game, for several reasons:

  • We wanted to target as many people as we can.
  • We wanted to keep the required PC specs as low as we can.
  • We didn't have much experience with DirectX and wanted to start working in a smaller sandbox first.
  • With some work we managed to put everything we wanted visually into shader model 2.0 in the end.

We've reached the limits of shader model 2.0 a while ago, and the game really looks better than we would have ever thought. However at this point there's no real option for real visual upgrades without either switching to a higher shader model or going through significant hacks that reduce client performance and mess up the codebase.

This post is advance warning that we'll be changing the minimum requirement for Perpetuum from shader model 2.0 to shader model 3*. As the vast majority of you have hardware that supports SM3, the change will only affect a handful of players.

Do note that changing shader models isn't a silver bullet that will make the game look better magically overnight. This change is necessary in order for us to move the 3D engine forward to more current visual techniques and will take effect when we roll out the first such change, about a month from now to give proper warning. Initially the changes will focus on client performance as opposed to visual upgrades, and we'll gradually move to making things look better when we feel the client is performing well enough.

* For those who are not that tech-savvy, shader model 3 is supported by GPUs manufactured since around 2005, specifically the AMD/ATI X1000 series and above, and the nVidia GeForce 6 series and above. While the Intel GMA series GPUs do have SM3.0 to some extent, they are not officially supported.