Tuesday, 23 February 2010

STL and project development speed

Hidden Costs:

I have been a fan of trying to figure out what makes development pace faster, quickening the development of any project and quickening the development of all future ones too. I tended to look for solutions that made development less about getting it done and more about reducing the amount of work to get it done. This was probably a rebellion against tendencies in games to just get it done.

Recently, looking at how my brain works with relation to really good high level languages such as python and C#, I can see why I made some huge mistakes with my first true foray into the world of STL. The idea of things being objects in the high level languages makes the idea of containers equally simple to understand. STL is not written in a high level language, it's written for a low level language. There is a big problem with this for my way of thinking about problems in high level languages, I think of containers as being able to be passed around as easily as any other types. This lead to quite a large inefficiency in my code. I was returning containers as answers to queries.

This is how you do it in high level languages because of the nice object oriented approach to containers and arguments and everything, but in C++ this is a hugely damaging thing that I overlooked simply because it's sensible in other languages. I should have known better, I think I even did, but tried it anyway just to be sure.

Problem I'm having now is, how many junior games coders that have come from learning C# or java are going to be aware of these kinds of problems with performance. How many non C++ coders are going to be aware of the effect of copying a container? Also, how many are not going to know of an alternative to using the built in sort? I've got to write a better sort (problem domain specific) for my container as the STL sort is missing a trick or ten thousand as it's a lot slower than I was expecting for such a small amount of data.

What's actually dangerous about STL?

There are ideas that are easy to write out in STL, and there are ones that are hard to write out. Some of the ideas in STL are great, but things like functors are insane. I want to write less code, not more. Especially not more if it turns out it's actually slower to write it all nice and STL conformant too.
STL does have a lot of nice ideas, but they should be my coders idea toolbox, not actual coherent and coupled templated functions and classes that implement those ideas. There are too many situations where STL provides a mechanism by which the problem can be solved, but can't be solved simply, which adds lots of typing without much benefit. Also, when you start using STL, there is a tendency to not roll your own solutions, which can cause you to forget that things could be better.
However, the most dangerous thing about STL from my personal experience is the compilation time problem. This helper library really, really shouldn't come at such a price.

Monday, 22 February 2010


Some of you may already be aware of the quote:

"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian Kernighan

I came across this recently at work when I worked on a particularly evil algorithm for visibility checking. The algorithm as a whole is bound by three states, have-recursed, incoming-state, current-state. This, coupled with the fact that the algorithm changes based on each of these states means that the function that handles it looks really large as it has to have six separate code paths. The fact is, it is actually a really big problem space, not as large a problem space as when I wrote my own version of Judy Tables, but big enough. It's the combination of problem space complexity and the verbosity of each of the techniques for finding answers to questions that arise due to the current problem space configuration that means this function bloats into quite a scary pile of ifs and math functions.

I will be attempting to make it readable soon, as a co-worker has pointed out "it doesn't make sense", and that is going to make it impossible to maintain later on. The main thing I have to worry about is that the algorithm seems "okay" to me, but I think half of that is that the maths and the logic flow do make sense to me. But for how long? I'm worried that I might not be able to tell when the code is simple enough. First attempt is going to be trying to function-off the questions and states as much as possible.

Has anyone else had any experience of this? How do you tell when your code is good enough for public consumption?

Friday, 5 February 2010

Database of Flow

I was talking with my friend last night on the way back from London about my editor and how I implemented undos and redos. My initial quote was "use memento pattern", but I'm not quite sure this is right now, as the technique I used extended the idea a little and touched the "command" pattern too.
What I've done is simply base class all model operations with a command class and made all commands push themselves onto a done stack. Undo commits the undo method of the top of the undo stack into the model, then pushes that item onto a redo stack, and pops the undo stack.

Redo just reverses the operation and commits the redo method on the model instead. The command is it's own memento.

This way of working was important for me because I wanted (eventually) to be able to save commands to disk as I was working. This was to be my uncrashable editor. Or at least very easily debuggable editor.

Having recently played with SQLlite, I'm thinking of using a database like this to store my commands as well as the data. It's well known to be safe from crashes and general corruption problems so should be the basis of a very safe editor system.

If a crash happens, I should be able to just load up the last session, and replay the commands issued up to the point of the crash, which should allow me to debug even final release code by having repeatability.

But the thing that came to me last night was more useful than just that. Being able to replay commands is useful in at least two other ways. Firstly, we have a database of the commands that are used to do jobs. We can replay the actions of a user (even the undos and redos if that's part of the saved structure), and learn about what actions could be done quicker if there were macros or combination actions that did more than what's currently available.

Think of it like sample profiling, find out what the user is spending most of their time on and optimise the operation so it takes less time, or can be automated.

The other thing, by adding other metrics to the commands such as when they were done/called, we can find out which users are most productive, and learn techniques from them. We can learn how to shortcut (minimal actions or time to do a task), and learn about how they get into flow (time between commands). Although this would probably feel like an invasion of privacy to some people in a professional environment, I'm actually a bit fired up at the idea of "time and motion" being done inside the tool you're using. There's nothing wrong with making your job easier!