« MIGS: Every Click Counts | Main | Unite '09 »
Thursday
Nov052009

Making Games Faster

At last week's Unity engine conference, Flashbang Studios gave a presentation about how to make games faster. To me, it was a surprising and great presentation. Surprising because went against my expectations of what a talk at the Unity conference would be about. Great because it summarized much of the best advice I've heard about project management in one place with nothing extra.

Flashbang decided to make games on an 8-week cycle (!!) and post them at blurst.com. They made 8 games this way. They've now decided to change their model and work on polishing up just one of those 8 games into a full version, but that's really beside the point. The particulars of their situation aren't important because their message is appropriate to a very broad audience: not just Unity engine users, not just people who want to make 8-week games--or games at all--but really anyone who is working on a software project.

Reframing the Problem

Matthew Wegner (founder and CEO) and Steve Swink (game designer and kindred sprit) presented. They started by explaining the importance of questioning the framing of a problem, as opposed to trying to find a solution for a problem. A problem you might have is, "It takes too long to place enemy AI waypoints in each level of my game." You might optimize that process by finding a way to reduce the number of clicks the designer needs to place any given waypoint. But if you use a bit of lateral thinking, you might instead design levels that don't need so many waypoints. Or much better than that, design the AI system so that it doesn't need waypoints at all! What they're getting at is that if you take the problem as given, you will probably limit your solution space--the set of things you would even possibly consider as potential solutions.

What if your problem is, "how do we make games faster?" I think they might have given another presentation about how to optimize various things inside the Unity engine (or at least I think they said they did), but here they were saying that that sort of thing isn't *really* the problem. I mean, when they try to ship games in 8 weeks, is it really the inefficiency of waypoints or whatever that threatens the schedule? There are much, much bigger problems. In fact, the question should really be, "how can I get more things done?" (Not their words, but here's a link to the seminal book on Getting Things Done.)

Two Amazing Hours

The first part of their theory is that we really only get about 2 hours of seriously focused, amazing-quality work per day--if we're lucky. Maybe you can get 2.5 or 3 sometimes, but that's pushing it. There are so many distractions and blockers, so many times when you're too tired or hungry or upset about something, or whatever. Flashbang is saying just be real here: accept that you're only going to be able to do amazing work for a short time each day. Knowledge work as it's called, is the type of thing where you could spend 20 hours on a problem and not solve it, but just *one* hour of your fully charged genius-time could solve it.

Ok, so how do we make sure we get that super-charged-time each day? And how do we maybe get a little bit more of it than usual? Flashbang's answer is that the WORST thing you could do is work really long hours as is common in so many game companies. If you're spending all your time at work, tired, fatigued, probably malnourished, how are you going to have any of that time be the amazing 2 hours? Factor in that you probably had no time for laundry, a haircut, your dentist appointment, or your relationships, and it sounds like you're going to be pretty miserable. Do you think being miserable is a good way to increase the number of super-productive hours you have?

Flashbang tried an experiment. For two weeks, they REDUCED their work hours to 10am to 3:30 pm. The idea is that everyone knew they had only limited time to get things done, and they had plenty of time to live a good life outside of work. At first, they actually kicked people out at 3:30 and turned off the power to make sure people left and didn't stay out of some strange guilt. They measured (though didn't give the details) their productivity before and after this change. If it turned out they got less work done than usual during that 2 weeks, they would cancel the plan and go back to regular work hours. The thing is, they found that productivity really did go up. They kept these reduced work hours for the rest of their projects. It's more informal now and sometimes people do stay longer, but they said "if we're still at work and it's time for dinner, we usually say 'hey, that's pretty weird! This is like crunch-mode day!'"

They said that if you see your time as an unending ocean ("hey, I'll be here for another 15 hours anyway today, and again tomorrow"), then it doesn't even phase you if the solution to a problem would take 5 hours of menial labor from you. But when your time is limited, you think about alternate solutions, or question the framing of the problem that is leading to the tedious work. This is one area where I'm not sure I agree though. I do often see my own time as an unending ocean, and if a task takes me 10 hours, that's what it takes. I asked them about this afterward and Matthew said "yeah, when you work from home, it feels like every moment is a moment that you 'should' or at least could be working." I guess spending long hours of my own time on what I choose to spend it on at home isn't quite the same thing as a game company creating a corporate culture where you must be there 15 hours per day to fit in.

48 Minutes of Flow

This concept of high productivity time is really called Flow. (Here's the seminal work on Flow.) Steve Swink gave further advice on how to actually get down to work. He said what usually happens is that you sit down to do something, but then you get an e-mail, answer that, see something on facebook, write some comment there, and the next thing you know, you're looking at cat videos. (See, I got you.) His prescription is to pick a number less than 60, he usually uses 48, 50, or 52. Decide that you will work uninterrupted for that number of minutes, then you'll goof off the remainder of the hour. But during those minutes, you close everything else that could be a distraction. No e-mail for sure, no chat, no forums, no web surfing. [Editor's note: at this very moment as I typed that last line, someone instant messaged me. I said I would talk to them later!] Matthew Wegner put in, "And if you feel you can't really go 50 minutes, then you should probably think, 'Really? I mean REALLY? I can't go fifty damn minutes without goofing off? It's really not a lot to ask.'" Steve said that when he does this, he often finds that he reaches the end of the time and thinks, "hmm, I got a lot done here but if only I did just this one more thing...or two..." and sometimes he can go a while longer and maintain the high rate of productivity.

No Interruptions

Next they explained that another consequence of this theory that you only have a very limited amount of genius time per day (aka flow), is that your time is very, very valuable when you're in the zone. When someone asks for help, your natural reaction is "oh don't worry about it, sure I'll answer your question" because you don't want to be a jerk. But actually that is undervaluing your own time. You need some way of actually saying no to that interaction if it's at the wrong time for you.

At first, Flashbang tried a Quiet Hour where no one could talk between 2pm and 3pm. It went horribly, as you might imagine. No one wants The Man telling them they can't talk. So they scrapped that and instead tried a system where if you are wearing headphones, that is your signal that no one should bother you. I've seen that same system spring up naturally in other companies, and it works well. If you have a stupid question for someone and you see they are wearing headphones, you just wait until later or figure it out yourself. This doesn't work for me personally though because I never use headphones and don't really listen to music, even. Maybe I would get a hat as my signal.

Improving Communication

I was surprised by this one too. Matthew asked "How can we improve the team's communication?" I thought his answer might be something about the water cooler or watching movies together or something. Nope! His answer, in its entirety, was their use of Growl. Growl is a system-wide notification tool for in Mac OS X. (You are using Macs right?) You can customize how big or small Growl messages are (they appear as dark gray boxes with text in the corner of your screen by default), whether you have to click to dismiss them or whether they fade after a couple seconds, and so on. You can choose which applications send these messages, and even customize which type of messages or displayed which way.

At Flashbang, they all have Growl hooked up to their version control system. Whenever anyone modifies a file, everyone gets a Growl notification that it changed. Now, it's easy to see people's complaints with this, but I think it's exactly what's needed. I'm used to using Growl for other stuff anyway, so it's not some strange concept. Don't you get notifications all the time? Yeah, though again you can set which projects you get them from (or don't).

The point of it all is that you get something really valuable that you can't even quantify: you get situational awareness of the whole project. There's all sorts of confusion that never happens in the first place because everyone just knows, deep in their bones, what everyone is doing. Even though you "aren't paying attention" to those messages, you still kind of are. When you play the game and notice that some level is kind of broken, you don't even think about WHY you know exactly what caused it. I mean of course it was something Bob did because (subconsciously) you noticed Bob checking in stuff 3 minutes ago. And so you know to tell Bob he broke the level. And then Bob will say "Oh! It must have been the taco-physics-whirly-gig!" That's because it's still in Bob's short term memory what he did. Without the Growl system keeping everyone aware of everyone else's work, it's more normal to discover this hours later. By that time, even if you know it was Bob that broke things (which might be harder to know), Bob will say "hmm, I don't really remember what I changed. There was a lot going on."

What's kind of funny is that I accidentally stumbled into this exact same concept myself. I've been using Dropbox for filesharing (greatest thing ever, please signup using my referral code, thanks) and somehow it's automatically configured to give me Growl messages whenever files my Dropbox get changed...even when it's the guy 3000 miles from me in Canada who is changing them. So really, it's exactly the same situation as Flashbang described where I'm getting Growl popups whenever something changes, and I immediately saw the value in it. (And this was before the Flashbang talk, so I was already sold on this idea.)

The Best ROI?

Matthew said that looking back at their projects, what gave the best ROI (return on investment) when it came to getting their games done faster? Of all the experiments they tried or things they bought to help, what really mattered? He says that in all fairness, the #1 thing was probably having computers. Their 27" iMacs are a pretty damn good ROI. And after that, the Unity engine itself allows them to do so much for such a small amount of money. But he means *besides* those things. His answer is that he thinks what mattered the most as gym memberships. He gave free CrossFit memberships to everyone in the company, and noted that for people to get to the 5pm sessions on time, they pretty much had to leave work by 4pm, which he also fully endorsed. My takeaway there is not so much that specifically the CrossFit workout program mattered, but that being generally healthy helped the whole team. More energy and fewer health problems make for good work.

I would expand his philosophy to even more mundane healthy things. Often in my work I've had to come up with ideas for how things will work. It's really no easier to do that sitting at a computer as it is walking around a park, so I would often leave work and walk down the block to do that. That's usually frowned on because it creates the *appearance* of goofing off, but if you're concerned with appearances as opposed to actual substance, then you're pretty far on the wrong track already. Likewise, sleeping on the job is seen as bad. I've never taken a nap in an office, but as a consultant who usually works from home though, it's laughable to think that sleeping during the day is bad. If I am to spend X hours helping a client with something (that can be done at almost any hour), and I am feeling tired, why would I run up their clock then? It makes a lot more sense to take a nap, then only be on the clock when I'm as productive and alert as I can be. Naps in an office setting are pretty much always frowned on, but the alternative is spending hours of non-productive time being tired. I'm not sure why our culture prefers that outcome.

Standup Meetings

Flashbang's next topic really boils down to "how much management do we need?" That's a hotly debated topic in the business world, but the smartest people seem to all say similar stuff. In my words, I would summarize their answer as "A bit more than none." To have actually NO project management oversight in place is not a good idea. Programmers will (rightly) tell you that meetings break up their day into unusable chunks, that reporting what they did or what they will do is annoying, and so on, but I think it's not too hard to show that NO management at all leads to everyone going in different directions, or some other disaster.

Matthew said that from the producer's point of view, you wish that you could get a super detailed report of everything that everyone did and will do, and all the things that might block them or that have dependencies, but that it would take workers like 30 minutes a day to provide such a report. That is infeasibly expensive, and a really terrible ROI. Instead, the producer should settle for much less, and try put the minimum number of project management interruptions that are necessary.

First, they do daily standup meetings at 10:05 am every day. When they say "standup" they mean it literally. You stand, rather than sit. Flashbang holds these in their server room which is hot and near a smelly garbage can. I've seen these types of meetings go wrong before. I know that even the programmers I'm working with right now will be against them, but the key to success are these three things: 1) they have to be short, 2) no seriously, I mean SHORT, and 3) SERIOUSLY, they really, really, have to be short. Short, short, short.

At these meetings, each person says what they will do today. I've seen other versions where you also say what you did yesterday, which is fine too. It's important that you don't ramble. A 3d artist might say "yesterday, I modelled a car. Today I'll model a different car." No one wants to hear about the boring details of it, just the quick explanation. If you're taking more than 1 minute, you are royally messing up here. Hopefully you take between 5 and 30 seconds.

The entire purpose of these meetings is dependency checking. Inevitably, someone will say "wait, what? You're working on that? No, you really can't work on that yet." Or, "wait what? You're working on that? Oh in that case I could work on this other thing instead that will directly help you on that." If that dependency conversation takes more than 1 minute, it needs to be discussed offline by the people involved, and everyone else needs to get the damn meeting over with. Discuss your dependency al you want with the appropriate people, but not while sucking away the time of innocent bystanders.

I think Flashbang's system is entirely reasonable there. The benefit of that meeting is great compared to the cost. You just have to make sure the cost stays small and the meeting stays SHORT, which many meeting leaders are bad at enforcing. Matthew also commented that this approach is kind of a capitalist instead of communist approach to meetings. The "communist" way is that the government mandates X meeting will happen. "Tomorrow at 3pm, we will talk about the AI system." And some people kind of don't want to be there or aren't really ready to talk about that yet, but it's this thing handed down from above. Their meetings are capitalist though, in that they happen naturally according to supply and demand. If someone says they will work on the AI system today, and someone else says "wait a minute, that impacts my work" then those people naturally have a meeting about right after. And they want to be there and chose to be there.

The 1-Minute List

Matthew mentioned a couple more project management things he does. One is that everyone submits daily accomplishments and goals at the end of the day. I think this is intended to take less than one minute to write. Steve Swink said he was initially against that (The Man is making me do what?) but he now concedes that it does fall within "a bit more than nothing"-type management. He said the ability for the project leader to understand the project is increased enough by this that it's actually worth it. I think an important distinction here is that the purpose of that 1 minute list is not to check if you're doing enough work or slacking off. If that's the case, you kind of resent writing it. It really is, straight-forwardly and honestly, a way for the producer to know wtf is going on in the first place.

Pivotal Tracker

Finally, Flashbang uses the free online tool Pivotal Tracker. There are some really in-depth project management tools out there, but I've found that a management tool has to be really, really simple and take almost no time to use, or it won't be useful at all. Fewer features and less powerful tracking is completely fine, when the alternative is "will not be used."

With Pivotal Tracker, anyone can create a "story." That's something like "I shouldn't have to click so many times to get to the game settings menu. Someone streamline that please." You can assign a point value from 1 to 4 of how big of a task that is. Anyone on the team can click "start" to take responsibility for a task, then they click done when they are done. The main useful thing here is that it's easy to drag and drop these stories into the various bins. The most important bin is the "current" bin, which means someone should really work on this today or has already decided to. When something is done, it automatically goes to the "done" bin. If you put too many things in the current bin, more than your team can reasonably do, then the last ones on the list get automatically put in the backlog bin. That's handy to help you realize you're trying to do way too much in one day. Oh, and Pivotal Tracker knows how many points (that's the 1 - 4 rating of how big a task is) your team finishes each week (the total is your "velocity") and it turns out that teams are pretty consistent at how much they accomplish each week. So Pivotal Tracker's guess at how much you can do in a given day is usually not bad.

If all that sounds complicated, the main takeaway there is that if you think of something you want done, you make a story for it, and that's it. If you see something in there that you have the power or ability to do, you can claim it as a task for yourself. Half the value here is that it's nice and AJAXy, so it's quick to go through those steps. And if you're worried about feature creep (endless new features that prevent you from ever finishing), I'd say that it's better to have a way to capture everyone's wishlists and leave them in the "someday" bin than to not even know what would make your product better.

Flashbang doesn't use Microsoft Project or any other project tool outside of Pivotal Tracker, not even a bug database.

Ok, that's about it. Incidentally, this is all from memory with no notes about a presentation I saw over 10 days ago. I'm not into taking notes. ;)

--Sirlin

Reader Comments

There are no comments for this journal entry. To create a new comment, use the form below.
Comment in the forums
You can post about this article at www.fantasystrike.com.