Before Civilization 4 came out, we were kind of on our own in terms of turn based user interface.  I don't want to say that no turn based strategy games came out since GalCiv I back in 2003 but I hadn't played any extensively.

So when Civilization 4 came out, there were a bunch of very innovative user interface tweaks.  I also really liked the Civpedia.  The Civpedia stores a lot of information that is presented to the user at the time of execution. But there's a lot of good stuff in there that I'm sure users would like to have access to from a single place.

But it always brings up the issue -- if we borrow too much from Civilization 4, would users complain?  What many gamers don't realize (and in fact most non-gamers as well) is that the game industry isn't like other industries.  Game developers don't *generally* see other games as "competitors".  We're all part of the same team -- trying to make good games.

I talked to Soren Johnson about this very issue (designer of Civilization 4).  One of the features we wanted to put in was something to indicate that there were no more units to move.  In Civilization 4, when you finish moving everything that can be moved, the turn button changes color letting you know that you could press turn and not miss a unit getting its move in.  Soren's response was "You shouldn't be afraid to borrow interface elements. If you don't people will just complain!" 

I think something as obviously good and logical as color-coding the turn button would be one of those things.  It was one of the things Soren suggested we changed based on his beta experience with the game.  There were numerous other UI tweaks like this.

In the real time arena, there are certain user interface conventions that have become standard.  When games come out and don't make use of them, many gamers (myself included) become frustrated.  As much as I enjoy Age of Empires 3, there are a number of UI conventions from Rise of Nations I wish they had borrowed. It would have made Age of Empires 3 a better game.

But I suspect many game designers feel trepidation about borrowing ideas and suggestions from other games.  I know I do.  I don't have a lot of shame. I believe in the "Great artists steal" but I do feel some shame.  I love how you can look at a city in Civilization 4 on the map and see what it's building.  We actually tried to do something like that but it didn't work out as well with our UI.  After release, I'd like to come up with something like it still for the bonuspak.

Another thing from Civilization 4 I liked (and Soren brought up) was how you can look at a unit and tell if it has moves left.  A little green dot appears if the unit has all its moves left. Yellow if it has some moves. And red if it's out of moves.  This was another one of those things that I'd like to put in as a bonus pak option.   The issue is how to do it cosmetically.  In Civ4, you have a fixed camera (unless there's an option I'm not aware of).  In GalCiv II, the camera is free form. So we want to make sure what we put in looks okay regardless of the angle and direction. 

I think in time many of the innovations in Civ4 will become "Standard" in Turn-based strategy games.  If you don't have Civ 4, you can get it here: http://www.civilization4.com.

The GalCiv II campaign has been a lot tougher to do than we had thought it would be.  A couple years ago, we whipped up a campaign for the GalCiv expansion pack called Altarian Prophecy and so this time around we thought it would be pretty much the same except we'd make it more dynamic where you could lose missions and go to alternative missions instead.

We were wrong.

It's all in the balancing.  First of all, the main feature of GalCiv II is the stand alone "sand box" games.  That is, the randomly generated games where the player is making up the story.  For me, as a player that's the whole point of the game -- to have a strategy game where I'm making my own mythology as I play along.  To "role play" if you will.  So one of the things we quickly changed was how long the campaign took to play.  Originally it was going to be an ungodly # of missions.  But then that puts most of the game play (and reviewer's time) put into the campaign because reviewers will feel they need to "finish" the campaign in order to review the game adequately.  But we don't want the campaign to be the main part of the game.

At the same time, we do want the campaign to be awesome.  And having done a lot of campaigns over the years, I know which types of missions I like and which ones I don't like.  Escort missions, protect the item, etc.:Yuck.  Blow up and kill things: Good.

Another thing I noticed is that whenever you have an impossible to defeat enemy that you've built up, sci-fi tends to end up with the good guys winning by some trick.  In Star Trek, best of both worlds, the unstoppable Borg were only stopped due to the Enterprise being able to put the Borg to sleep. No way. 

The Dread Lords gotta be tough. But they have to have a very obvious and reasonable weakness to exploit. One that makes sense to the player and can be represented in-game.  We came up with that weakness: They're so technologically advanced that it takes them a very long time to actually build things. 

Let's face it, if the US military conquered an ancient city, it's not like they could quickly convert ancient Rome into cranking out new MFVs overnight.  The Dread Lords suffer the same problem except they're millions of years more advanced.  So when they arrive, they have this issue -- they have a handful of ships which are deadlier than anything else out there.  But these planets -- so primitive.  Look at these humans and Arceans and what not, still use non-organic ships.  Still throwing trivial bits of energy about.  Still using their people to do labor.

I've zoomed out into strategic mode to show the area in question.  The blue ships are my fleet that I'm sending in. The red thing at the upper left is a single Dread Lord Dreadnought.  That single ship will wipe out my fleet but it'll hurt it good and there's no quick replacements for the Dread Lords.

The other issue to have them be super powerful but not unstoppable are raw numbers.  There just aren't very many of them.  So when they do finally invade a planet, there's only a a dozen of them versus 5 million troops. Obviously we had to put together a special invasion effect to convey 10 guys wiping out 5 million troops.  But each of these Dread Lords is as powerful as Sauron.  You can take them out, it's just hard.

So I spent the entire weekend tweaking the Dread Lord AI to make them tough but not unstoppable.  This is harder than one might think.

In the campaign mission, Apocalypse, it's basically everyone versus the Dread Lords.  So here's the design challenge: You want to make a mission in which IF the player doesn't do anything that the Dread Lords will win. BUT if the human plays decently the Dread Lords will lose. 

So you can picture playing this mission over..and..over...and over.. tuning it.  For some hours, the Dread Lords were too nerfed and eventually, just by hitting the turn button, the Dread Lords would be taken out by someone else.   So then I build them up and pretty soon they're conquering the universe and there's no stopping them. The human player, even me, gets wiped out.  So all kinds of tweaks were made until I finally started homing in on the Dread Lord weakness and making sure that was carried to its logical end -- they produce ships slowly and their ships are not as powerful as they might be (early on, that Dreadnought has a 200 attack -- if you get far enough in the tech tree, you can build incredibly powerful ships).    So I have to restrict how much they can put on their ships (that's where a lot of time went -- just how powerful are their ships).

Similarly, you have to decide how powerful their troops are.  You want to make it reasonably possible to invade their worlds if you get through their defenses but you still want their forces to be immensely tough when they do the invading. But not too tough mind you.  10 guys wiping out 5 million troops is fine. 10 guys wiping out 10 million becomes imbalancing.

|
I apologize for the blocky graphics, I have all the effects/smoothing turned-off for debug.

By 2231 in the campaign mission, which starts you out with a lot of techs (one other thing we've done a lot of work on is making is to that players get increasing amounts of tech each mission - no one wants to have to keep researching "space militarization over and over").  The idea is you "keep" some of the techs you get from the previous mission.  But even with a lot of effort, my top of the line ship is 16 missile attack and 2 armor.  Which is another thing I did to the Dread Lords.  They only get to "Adapt" ONE time.  That is, they don't get to change their ship designs like normal players.  Their infrastructure barely supports creating their advanced organic ships to begin with let alone switching from Doom Rays to Mass Drivers later on.  So they will adapt one time and that's it -- though they get to pick when.  Still my pitiful 16 missile attack is not going to stand up to a Dread Lord when they're doing 52 attack with beam. 

They already did their adaption and made useless my entire line of beam-based ships.  My "EarthForce" heavy fighters were all based on beam weapons and the Dread Lords now equip their ships with a standard 20unit shield defense making my plasma beams pointless.  So I had to start my way up the tech tree for missiles and it's been a real race.  One of the recent (i.e. last week) changes is that we made it so that each new tech you get makes the next tech a bit more expensive. Not massively mind you but later on in the game when you've built up a massive technology industry it's not right that a player could just whiz through the missile techs (like I'm trying to).  So instead of it taking 1 week to get Stinger II missile tech it's taking me 3 weeks. Not  a huge deal but it definitely puts some meat into the "You pick your weapons and defenses and you live with it." 

So you can (like I'm doing) switch to another weapon technology but you're not going to whiz through in 5 turns. It's going to take a bit of time.  We did give something in return though -- no research wastage.  When you research a tech, the excess technology production is automatically put into the next tech in the line.  So when I first started up on the really early weapons stuff, I was getting 2 or 3 techs in a single turn which was nice.

The other penalty I put on the Dread Lords is that they have a relatively low logistics ability. They can't coordinate their ships.  That way, the player isn't faced with a fleet of Dreadnoughts. It's usually a single ship or a couple ships at most.    But if you research enough logistics, you can put together a death fleet which is what I've done.

My 6 ships include 3 Battleship-level ships. Enough to kill off any Dread Lord.

Which it was. But in rare incident, the Dread Lords had two ships together and one of those Dread Lords was destroyed but the rest of my fleet was wiped out.  It was quite a battle though.  I have to say, there's something humbling about seeing not just one but three of your state of the art, large hulled, armed to the teeth designs being destroyed.

This brings me to map design. At the end of this, the player will win if they know what they're doing (and each mission has its own difficulty slider).  But part of the key was making sure the player wasn't in the action.  We have, literally "human shields".  Other civilizations who are in the way.  Namely, these guys:

By the time I'm done, they're gone.  I wish I could show you the Dread Lords battle but the PR people want me to wait and as a way of thanking IGN for their really cool coverage we're going to have the first shots be in there when we talk about the Dread Lords.  They have ridiculously cool ships that make the most of 1000 polygon models, bump mapping, and normal mapping and high definition textures plus they have the high end weapons (the weapon effects get better as you go up the tech tree).

The other "timer" is that the Drengin and Yor are on this map. They are at war with the Dread Lords too but the Dread Lords aren't as interested in them as they are you and your allies.  So while you and your friends are dealing with the Dread Lords, the Drengin/Yor alliance is slowly encroaching in.

In my mind, this one is the "toughest" mission in the campaign.  But on the other hand, it should be the toughest. This is the pay off, to match yourself to the Dread Lords.  There's no "trick" to it, it's just good old fashioned strategy gaming.

I'm glad I'm not doing translation. There's a lot of it. For a strategy game, GalCiv has a lot of text.  GalCiv II may have more text in it than most RPGs.  This really stems from the decision to be a single player game.  By focusing on the single player game, we can sit down and put in per-race dialog on a per situation basis.  Most people won't see a fraction of it.  Play as the Torians and you'll get a different set of dialog and reactions than if you played as a human or a Drengin.

The tool we made is really pretty neat.  I can type in a # and type in alternative text.  When the parser sees a # it knows that there's another randomly available bit of dialog to select or in other cases another bit of text based on a slightly different situation.

I don't know how many people even care about that kind of thing.  Strategy games usually have a pretty narrow set of text. Part of that is to ease translation and part of it is the consensus that it adds little or at least not enough to justify the effort.  But for me, I enjoy seeing players that say a lot of different things.  And plus, I type 120wpm so it's not a big deal to type a 10x10 grid.  We also have a "generic" category so that if it can't find what it needs specifically, we can wimp out and just fill in the generic stuff.

Translation was an after-thought in GalCiv I.  It was amazing it even worked as the fonts in GalCiv I were bitmapped. That means every character -- every letter -- was literally an individually encoded picture. 

Translation to a non-indoEuropean dialect would have been out of the question.  But this time, we used the True Type Engine that is part of DirectX 9c (this is actually the feature that breaks compatibility with really old Intel integrated graphics -- not the fancy 3D engine). 

So we literally now have directories such as:

\galciv2\data\english
\galciv2\data\german
\galciv2\data\russian

and so forth.

But there's a lot to it.    There's also the question on updating.  We want to make it so that when you install the game, it will put your country code in the registry. And then when a player updates the game from Stardock Central, it'll download all the different language files (they're text and compressed so we're talking tiny) and then when you run the game you will have it in your own language.

The nice thing about Stardock Central is that it enables us to ship the game with zero copy protection.  No CD in the drive required. No Internet access needed. While some people will still pirate it, we hope those numbers are low -- there's no excuse for it. The retail game is cheaper than any other AAA retail title that's come out recently. And there's no inconvenience factor. Just install it forget. And when you update it, you use the serial # with the game and it will auto-generate an account for you. So even if you get a new computer 3 years from now, lost your CDs, lost your serial #, no problem. Just use Stardock Central and type in your email address that you used (and if you lost/changed that too you can always contact us to retrieve that) and it'll set you up with a link to validate and voila download the latest/greatest version. 

We think in the long-term, there's a good reason to keep the hassle down -- international sales.  As countries that are currently piracy heavy start having more people who buy these games, the convenience of just being able to type in your UserID and password and be able to download the full game will be a big deal especially if you get any language you desire.  It's a lot like iTunes except you can re-download your game as many times as you need.

In the meantime, Paradox is in charge of foreign translation.  I am glad they're in Europe because once they see how much dialog there is to translate I think they'd want to reach over and whack us.  There's hundreds of pages of dialog.  I'll talk about that next.

We're in the final stages of Galactic Civilizations II's development cycle.  If you want to see some of the fine tuning, check out the Developer Journals.

Tonight we log how we fine tune the computer AI in various ways.

If the Stardock crew that hangs out on WinCustomize seems quiet lately, it's because it's been consumed entirely by Galactic Civilizations II.  It's a new strategy game Stardock has been developing for the past two years and it is scheduled to go gold in the next week and a half so everybody is working on those final tweaks.

Big thanks to Jafo who sent us a Kangaroo! (not a real one) it's in my office now.

For those of you interested in what we're doing, here's a link to our game development log where I talk about some of the stuff we've been doing.

Match #5:

The Settings

The Drengin and I are playing on a tiny galaxy. Just them and me. They are set to "intelligent" which means they play their best game without any advantages.  The goal is to keep tuning the AI so that at map setting that at intelligent they can play a very tough game against experienced players without cheating.

The War has begun..

While I could use a pre-made map, I am also doing this testing to make sure and tune the random map generator so that it creates good maps for all players. This map starts off slightly in favor of the human player.

I concentrate purely on research.  I also put my effort into making my ships faster by getting impulse power and then warp drive.  I used to go for military power early on but found that this didn't really buy me much because I didn't actually get to use that military power until later so better to get faster ships.

One of the things I notice is that the AI is not good at adapting its build strategy.  That is, now that it has to explore the galaxy, it is building things like constructors when it should build colony ships.  It should be able to figure out that there are some good colonies out there it just needs to build colony ships sometimes.

First Fleet Battle

The early fleet battles between the Drengin and Terrans have an interesting design strategy.  I focused purely on offense. The Drengin modified their ships to have countermeasures to my mass-drivers.  But which way is the way to go?  Since I attacked them, I get the initial advantage (it's important to be the attacker).  Net result, I win the first engagement with only 1 ship badly damaged.  But as the M0 indicates, this is the AI's first go at it.

The human race's state of the art ship for the ship at first and what would be the main ship of the line was delicately called "The Clubber"

The Clubber has a crew of 5 and is armed with 3 OTO Smithon Industries Standard Rail Guns that were the result of Mass Driver II technology giving it an attack rating of 3 in the mass-driver category.

It was 15.5 meters in length and 25.5 span-wise.    Engines were standard Hyperdrive issue.

Nothing fancy but it did the job.

The Drengin Reaction

The Drengin reacted slowly but surely to the Clubber.  After some fits and starts, their third generation heavy-fighter class called the "Dra'Hak" which roughly translates to "Human Death Giver" was the ship they hoped to clubber mine with.

Armed with two first generation plasma beam systems of unknown configuration and a 5th generation laser system, it had quite a punch. It also had armor. It was a Clubber-killer in design.

Ironically, the first Clubber vs. Dra'Hak battle took place in a one-on one battle.  The Clubber was our flag ship Clubber, it was rated at level 10 in experience.

The result proves what pilots have said for years -- experience can trump technology -- to a point.

But it was clearly time for a new ship.  Mass Driver IV technology with enhanced miniaturization would bring about a revolutionary new design.

The new ship, the Banger:

The "Banger" was the result of new miniaturization techniques pioneered by top scientists on Earth. 

The Banger has a crew of 6 and is armed with 4 Hyperion Railguns also from Smithon Industries. It has a damage rating of 8 from mass-attack.  Drengin Armor continues to improve but not fast enough to counter it.  The only problem is that the Drengin have a lot more Dra'Haks to throw at me.

While I had the initial advantage in planets, I had not been adapting my own strategy.  For most games I had played so far, the AI was slow to get transports.  This was one of the flaws in its strategies I had tried to rectify -- to have it "think" about what techs it should get a bit more and away from randomness.

So the result was that my strategy largely depended on me rush claiming a bunch of worlds without worrying about defense.  But this time, the Drengin conquered the jewel of the Terran Empire -- Markus V, a class 17 planet.  Worlds of class greater than 10 are rare to begin with, class 17s are incredibly rare. And foolishly I left it undefended.  The Drengin attacked with mass-drivers reducing it to a class 15.  A true pity but a class 15 is still impressive.  Losing such a valuable planet early on would have very negative consequences.  

By the skin of my teeth and thanks to having research shock troops, I was able to recapture Markus V without any further damage to the planet's environment.  By the dawn of 2230, the galaxy was split in half.

The Drengin aren't winning as much as they've claimed more territories thanks to starbases.  They were able to do that because I had to spend so much industrial output making up for my blunder on Markus V.  I have 6 worlds, 5 of which are truly good worlds.  The Drengin also have 6 worlds. But only 3 of them are "good".

But it's not quite as good for the humans as the raw stats imply because the Drengin population is fully developed while I've been struggling to take planets 4 through 6 at the bottom left there.  So the Drengin economy is a bit head of mine because of their population.  With the galaxy split in two the war would change because our ship designs had been short-range.  Now there was some distance between us.

Moreover, new adaptions from the last generation AI had taken effect -- all but one of the main Drengin planets had orbital fleet managers. That means their ships in orbit fight as a single fleet which is much tougher and expensive to take out.  Luckily Xasica is out in front.  And my spies tell me they were building a orbital fleet manager so time is short.

BTW, if my graphics seem blocky, I have it running in the debugger with anti-aliasing off.

The Drengin soldiers are able to withstand my first invasion attempt.  But I have it blockaded with a fleet of Bangers. 

My second invasion was successful and I was able to discover some weak spots in the Drengin AI.

First off, this planet was late colonized but should have been further along.  The AI needs to be able to better focus on planets. On a planet like this that had so many incomplete social projects it should have put more focus onto getting them built.  If Xasica V had gotten its orbital fleet manager built just a few weeks earlier, I may not have been able to conquer it. 

At this point, the balance of the war is in my favor.  And to verify that, the Drengin finally decided I was worth talking to for the first time.

and several months later..

Another issue with the AI is in defense of its planets. Unless they have an orbital fleet manager, having a bunch of ships in orbit doesn't really help it unless those ships are really tough.  Right now, it tends to aggressively defend its planets with 3 or 4 ships.  It would really be better off putting resources into military action and not beefing up defenses to that level until it has an orbital fleet manager.

However, Xasica would change hands a couple of times.  When I sent my main fleet elsewhere, the Drengin retaliated and retook the planet.  I ended up retaking it but it was a sign not to underestimate the Drengin Empire's desire for victory.

Wow. In an amazing turn of events, and something I've not actually seen during the beta:

The Drengin have had an internal revolution in which they have gone from being evil and blood thirsty to kind and gentle.  This is a good opportunity to test out the dialog code for the Drengin..

The battle of the line

The Drengin had gotten better in recent AI updates at putting together fleets big enough to tackle my fleets.  This was to be a 6 on 6 match but the Drengin got the drop on me which is a big advantage. Also, their armor had really advanced so my ships were not nearly as tough as I'd like. The result was a serious defeat with the loss of all 6 of my ships versus only 3 of theirs.

The difference: THIS SHIP:

The brand new K'Zalan (Virtuous Justice -- no doubt based on their new alignment change).  It's the first Frigate class starship we've come across.  Theorized specs:

Level 2 Plasma beam of an attack rating of 10 with Duranthium based armor of level 8.  Crew complement of 35 Drengin.  Length approximately 100 meters by 60 meters.  It can really take a beating while giving one out too.  It was time to look at either producing our own frigates or to upgrade our heavy fighters.

Luckily, the Drengin fleets are still hampered by short-range.  The Drengin have not been research life support systems.  So at the close of 2130, the humans have the long-term edge, but the Drengin have the military might but not the range to make use of it (something to tweak -- AI needs to evaluate its range situation better). 

Speaking of range, I embark on a cross course to improve my range with new life support systems.  Life support technologies improve your existing base range plus give you extended range modules for your ships.  Soon the core Drengin worlds would be in range.

As 2231ended, the galactic balance we pretty even.  But I was preparing my final offensive. I've won the game so you can quit reading at this point.  The AI, while improved, still needs some work.  Its military is too undirected, too unfocused still.  

That means it's time to go into the debugger and figure out what the deal is. 

(2 hours pass..)

The problem is more like a bug.  Two issues in particular.  First, someone changed the way that colonies were being ranked in such a way that the AI no longer had access to the data.  So it was only getting its home planet as a key world.  This meant that it wasn't protecting its key worlds very well -- i.e. it didn't know to send reinforcements to planets under attack.  Secondly, there was an issue where the AI would focus on getting to rally points even if it was plenty tough to do the job.  I.e. it was too conservative in building itself up.

So now to go fix that.

(another hour passes)

Let's see how the AI can recover.  First thing I notice is that the AI has started sending massive forces at the newly identified threat.  My transport has already wiped out most of the inhabitants but it'll take a second one to finish the job.

The numbers look more impressive than they are.  These are units that had been cooped up around their home world (Because they thought only Drengi was worth really protecting).  So these are a bunch of mark 0 (first generation) small fighters. I wipe them out.

But there is wave after wave of these things and my bangers are finally taken out. But not before I take Martzia.  But unfortunately, it's undefended.

The Drengin take back Martzia and are now back at Xasica.    Time to regroup.  The Drengin are using military forces they've had all along but were using them poorly.  The changes made and brought into the saved game have caused them to focus this pent up military power very specifically. Xasica has an Orbital Fleet Manager however.  So a lone mark 6 isn't powerful enough to take out the defending fleet.

A huge battle eventually does take place in which we both militarily take out the trash.  Our offensive capacities have been restricted for a bit.  Still, the AI hasn't sent ships to follow up its attack. 

(back to the debugger).

Fixed another issue with it not going after planets that are outside a few key sectors. Or I should say, not focusing.  Though I am still pretty safe, the AI's range is still terrible and while I've fixed it, it won't be updated in this game.

I on the other hand don't have this problem.  All my fleets go to Martzia. For a huge battle:

After several back and forth battles, Martzia is mine again. Drengin, the capital of the Drengin Empire, is in range now.

Still, the Drengin have rolled out the 6th generation of their fighter:

Advanced Plasma weapons coupled with Duranthium armor give it a beam attack of 7 and an armor rating optimized against my weapons of 5.  In head to head, it is superior to the Banger.  It single combat, it will destroy a Banger and still keep 40% of its hull integrity.  The Banger is getting old.

In the meantime, Drengi has been conquered and the Drengin ambassador, now under the rule of the reformed Drengin ways of niceness (remember earlier event) speaks differently:

Oh boo-hoo.  Sure, I'm evil but whatcha going to do about it?  Another thing I modify is that AI ships need to worry less about maximizing their fleet sizes and go for targets of opportunity more.  They need to THINK MORE and follow scripted directives less.  So this change is in though a bit too late for the Drengin this time.

Which brings me to my war finishing design: Code-named "Pounder".

Armed with 5 twin-Singularity based cannons from Smithon Industries. The projectiles are incredibly high density and travel at near relativistic speeds.  The fighter is 18 meters long with a 12 foot wing span.  It has a crew of just 4.  It is relatively short-range and not as fast as it could be, but it won't need to. This is the mop up.

The knife tip though is the new Frigate called the Crusher.  It's the first Frigate-class ship in the Terran Empire.

Same technology level as the Pounder. But it is equipped with the next generation Warp Drive giving it 3 more parsecs/week in terms of speed.  It has a crew of 50 and is 150 meters in length. It's fairly large, fast, and tough. An attack rating of 12 with a shield defense of 3 should keep the Drengin down for the count. 

The only thing saving the Drengin is that they found a Precursor ship that is of Ranger class. It's tough but it's only one ship. It takes two full fleets of Bangers to take it out.  At this stage, the Drengin are down to 2 minor colonies.

But I don't have to mop them up:

They surrender.

And so ends another episode of Human Drengin wars.  The AI has been improved, tuned up and next time they will probably be even more challenging. 

T-minus 10 days until code-freeze. There is still time to keep improving the AI.  Even though I thoroughly defeated the Drengin (again), it was tougher than previous and it was far far better at playing than GalCiv I was.

While beta testers seem pretty pleased with the performance, there is always room for improvement...A LOT OF ROOM.  Users with beta 5 will be astounded by the improvements in performance.  Game loading speeds cut by 90% over beta 5. Save game speeds cut to 30% the time of beta 5.  But there's still more. 

The key is profiling.  That is, you use tools that literally tell you how many ms the game spends in different areas.  The Green Reaper finally graduated this past year and has joined Stardock as a full time developer.  Having him is like having the first round draft choice of all developers worldwide for a given year.  As a teen, in his spare time, he reverse engineered the multiplayer server for The Corporate Machine and made a new one (That we still use today).  His primary project is working on our killer app for Windows Vista (which I can't talk about).  But for this month, I've tasked him with figuring out what is slow in GalCiv and how to fix it.  He is also doing the galactic empire map for the Metaverse which you'll hear more about soon.

Here are his findings (unedited):

Seemingly a big problem (this is where the std::tree stuff was coming on that I found with CodeAnalyst):

AIPlanningStrategy (60 million mcs total):
-classCivilization::AIActivateDomesticPolicy (17%, 392 calls)
--classCivilization::AIDesignShips (85%, 122 calls)
---classCivilization::AIDesignShipType (99%, 2,196 calls)
----CShipDesigner::LoadAvailableComponentsForPlayer (99%, 1,586 calls)
-----GetComponentDefsOfClass (79%, 14,274 calls)
------CPropertyBucket::GetIntProperty (47%, 4,140,224 calls) [due to: if(pComponent->GetIntProperty(COMPONENTCLASS, -1) == cc)]
-------CPropertyBucket::GetValue (53%, 4,207,204 calls)
--------std::tree::find
---------std::tree::_Lbound
---------std::basic_string::compare
--------std::tree::assign
-------_tstoi (30%, 4,182,617 calls)
------std::sort components (46%, 14,289 calls) [due to using CompDefGreater, which has a string comparison on the name]
-----classCivilization::MeetsTechRequirements (20%, 459,940 clalls)
------classGalaxy::FindTechByName (almost as many calls, making it an expensive function)
------CPropertyBucket::GetStringProperties (also lots of calls, and twice as expensive due to its resulting callees)
-classAICivilization_Diplomat::AISetSpendRatios (7%, but only because of the DebugMessage call)

I think four million calls anything is probably a bad idea.

LoadAvalableComponentsForPlayer looks like it needs to be cached, or one of the other methods above it needs to be trimmed down.

MeetsTechRequirements has similar issues, due in part to the cost of its callee methods (but also because it was called half a million times)

GetValue caused significant time use elsewhere, too. Perhaps there a more performant data type that could be used than the current one, or maybe values need to be cached.

Other biggies

RenderNode (63 million mcs total)
-CStarbox.Render (42%, 7,199 calls)

CShipGraphic::LoadConfiguration (43 million mcs)
-CShipGraphic::AddComponent (76%)
--CShipGraphic::LoadBumpAndLights (84%)
---CResourceManager::GetTexture
----LoadTexture (most?)
----GetFilename might be worth looking at, as it is called 150,000 times overall
-CShipGraphic::SetShipFile

On a second run:

UIUpdateGraphicMsg used significant CPU time in:

classGalaxy::FillMovableShipList (used by ::OnUpdate of CBegin[Player|AIPlayer]TurnProcess)
--classCivilization::AIAddShipsToFleets
---classGalaxy::AINewFleetFromStackedShips
----claasGalaxy::AddStackedShips
-----CFleet::UpdateFleet
------CFleet::GetTopShips
-------UIUpdateGraphicMsg (21 calls, comment for use: "update the fleet unit graphic" - resulted in almost all of the CPU use of the higher five methods)

..

Easy fruit: CMouseCursor::UpdateImage (59 million mcs, 13,636 calls) - the call to SetCursorProperties at the end of this method (that occurs in every pass through GameLoop) is not free! It results in constant calls to CreateDIBitmap, DestoryCursor and CreateIconIndirect, as well as critical section usage (and entering a critical section can be expensive on multiprocessor machines, I think including hyperthreaded ones). If it is possible to detect when it is and is not necessary to update the cursor then that would be good. SetCursorPosition should be used to move the cursor, not SetCursorProperties. See help for IDirect3DDevice9::SetCursorProperties.

AIDesignShips and (to a lesser extent) AICivilization_Diplomat::SetSpendRatios are using a bit on startup, the latter mostly because of a DebugMessage

CMidpointDisplacementHF::GenerateField uses up 27 million mcs, in part via this call path:

InitializeHiddenPlanetSurfaces
-classPalent::InitiPlanetSurfaces
--CPlanetSurface::InitSurface (252 calls, 40.9 million mcs)
---CPlanetSurface::generateHeightField (252 calls)
----CMidpointDisplacementHF::GenerateField (39 calls, self time 6.5 million mcs, total time 27.3 million

THis is due mainly to calls to CCoreRandomBase::RandFloat (6.8 million of them) and possibly double->float conversions relating to that. If this is being done on a background thread, it may not be that much of a problem.

EnterCriticalSection/LeaveCriticalSection can be a significant part of the screen update in GameLoop (10 million mcs verses the 42 million mcs of the Draw3D itself)

..

This particular performance run was done in a large galaxy with lots of planets. The previous notes about performance apply here as well.

CPlanetTerrainGraphic::calcTerrainColor used in CPlanetTerrainGraphic::CreateTexture was a significant use of time. It got called 2.3 million times and used up over the vast majority of the texture generation time, and seemingly around pmethe CPU cycles used to load the game in total. 21% of its use was encapsulated in CPlanetTerrainGraphic::GetNeightborBleed (which was also called in CreateTexture).

UIBeginGame (100%, 2 calls)
-CD3DWindow::InitGalaxyGfx (58.2%,, 2 calls)
--CD3DWindow::CreatePlanetGfx (86.0%, 84 calls)
---CPlanet3DWindow::CreatePlanetTexture (98.5%, 24 calls)
----CPlanetTerrainGraphic::CreateTexture (98.6%, 1,728 calls)
-----CPlanetTerrainGraphic::genTextures (99.9%, 1,728 calls)
------calcTerrainColor (69.2%, 1,769,472 calls)
------CPlanetTerrainGraphic::getNeighborBleed (22.7%, 1,916,928 calls)
-------calcTerrainColor (84.3%, 532,890 callls)
-------CPlanetSurface::GetTerrainType (12.1%, 532,890 calls)

--CD3DWindow::CreateShipGfx (6.3%)
--CD3DWindow::CreateAnomolyGfx (5.6%)
--CD3DWindow::CreateStarGfx (1.3%)

While the entire calcTerrainColour was expensive, particularly expensive was:
* Indirection of the CPlanetTerrainGraphic::tColorDefList::iterator defIt
* Comparison against colorSet.mDefaultColorList.end

Sections of interest were:
* The do loop following "// There's no override for this terrain type, so use the default straight up".
** Here the repeated indirection (*defIt) was costly, as was the call to lerp_color and the comparison "while ( defIt != colorSet.mDefaultColorList.end() );"

* the do loop following "// We may need to interpolate between the default and override schemes."
**Again, indirection caused the most expense here
**The code following "// See if we ran off the end of the list" was not executed. This *may* not be needed.

* The contents of the else labelled "// The texel falls within the override range"
** More indirection, more comparison against end

* The last line (call to lerp_color) was slightly expensive, but little compared to the first two

It would probably be worth trying to cache (*overIt) at the start of each loop iteration and use that within the loop. Of course, the compiler might be doing this. There appears to be an average of 40 indirections per call to calcTerrainColor

It might also be possible to cache some of the things like .begin and .end in calcTerrainColor.

Another big CPU user was classGalaxy::LoadBlock, which came back to the generation of large number of random numbers:

classGalaxy::LoadBlock (40.6 mcs, 2 calls)
-classPlanet::FromBlock (89%, 910 calls)
--CPlanetSurface::InitSurface (97.9%, 506 calls)
---CPlanetSurface::SeedSurfaceFromTerrain (98.3%, 506 calls)
----CPlanetSurface::generateHeightField (89%, 506 calls)
-----CMidpointDisplacementHF::GenerateField (97.3%, 70 calls)
------CCoreRandomBase::RandFloat (75.8%, 36,700,020 calls)
-------CCoreRandomMT::Rand32 (69.2%, 36,700,020 calls)
----CPlanetSurface::smoothSector (6.2%, 7,492 calls)

In generateHeightField (30 mcs including calls, 7mcs self time), the key lines were:

* The inner statement of the for loops *before* "Square step -", beginning "mpHeightField[mi+mj*rectWidth] (7.7 mcs taken by mRandGen.RandFloat)
* The line following "Calculate the square value for the top side of the rectangle" (10.4 mcs taken by mRandGen.RandFloat)
* The line right after that, "Calculate the square value for the left side of the rectangle" (4.9 mcs taken by mRandGen.RandFloat)

For loading, the main Lib3D usage (Which is not *too* big) appears to be:

CResourceManager::GetRootFrame
-CResourceManager::ComputeMeshTargets
--MeshMender::Mend
---MeshMender:rocessBinomials
---MeshMender:rocessTangents
----MeshMender::BuildGroups (used by both the above)
-----MeshMender::FindNeighbors

There is a thread that appears to spend a lot of its time looking at the foreground window. It sets an event. The call GetForegroundWindow in this thread is somewhat expensive, certainly in relation to the other calls in that thread procedure (like GetParent, IsWindow, GetWindowPlacement And GetWindowThreadProcID). If it's on a timer it might be good to reduce its frequency (if possible).

Analyis of long-running game. This game was one where I just stayed in the corner and did nothing, so it's mostly measuring the background processes like AI and ship building updates that always occur in a turn.

----

Large rendering costs:
CScene::Render (2,810 mcs, 19,151 calls)
-CShipGraphic::Render (1,169 mcs [89.6 mcs wait time], 84,874 calls)
-CPlanetoidGraphic::Render (1,028 mcs [873 mcs wait], 83,257 calls)
-COrbitingGraphic::Render (408 mcs [23.5 mcs wait], 16,594 calls)
-CStarbox::Render (113 mcs [36.1 mcs wait], 37,045 calls)
-CRingGRaphic::Render (20.8 mcs [1.26 mcs wait], 16,594 calls)
-CZOCGraphic::Render (16.8 mcs [3.01 mcs wait], 48,845 calls)
-CStarGraphic::Render (9.86 mcs [1.35 mcs wait], 16,726 calls)
-CFOWGraphic::Render (7.42 mcs [0.47 mcs wait], 61,069 calls)

* The large wait time on CPlanetoidGraphic is odd. Perhaps a large amount of processing is being done elsewhere each time it is called? IF so, perhaps this could be reduced somehow?

----

I think I've mentioned this one before, but UIUpdateGraphicMsg used up 424 mcs. 78.6% of the CPU time used by it was caused by classColony:;CompleteShip, a method which was responsible for 98% of the time used by classGalaxy::BeginTurn (this happens every turn via UpdateProcesses -> OnUpdate -> BeginTurn). Another 4.5% was caused by classStarShip::CompleteTradeRoute, 3.5% by GCObject::SetPosition and 3.3% by both classStarShip::UpdateFOW and CFleet::GetTopShips.

I think the large usage in CompleteShip is caused by:

// Joe 04/07/05 - There is a crash when an orbited ship is attacked.
//                In the case of a new ship, this was caused by
//                the ship not having a graphic node and not being added
//                to the sector node.
//                To fix the problem, I create the graphic and immediately
//                hide it. This ensures that all ships have graphics when
//                they are needed.
UIUpdateGraphicMsg(GCMSG_UPDATE_GFX, pshipBuilding->GetID(), pshipBuilding->GetType());
UIUpdateGraphicMsg(GCMSG_HIDE_GFX, pshipBuilding->GetID(), pshipBuilding->GetType());

* How many ships are created but just stay in orbit, or otherwise out of view? I wasn't exploring anywhere in this game - most things were hidden by the fog of war. It seems silly to have to spend time and memory making images for things that are never seen.

* It could well be worth seeing if there is another method of avoiding the crash that performs better and doesn't have side effects.

* I'm wondering if this is part of what is taking up memory later on in the game - how much memory is associated with a "graphic node"?

---

NtCreateFile was called 1,822,520 times, resulting in 246 mcs. I don't think GalCiv II has nearly 2 million files. Opening a file has significant cost even if the file data is in cache due to security checks etc.

Almost all of these calls were made by CResourceManager::GetTexture (via D3DXCreateTextureFromFile), which was itself called 3,250,089 times. CShipGraphic::LoadBumpAndLights took up this time, due to CShipGraphic::AddComponent.

of time taken by CShipGraphic::AddComponent ultimately almost all was used bysubmethods of CD3DWindow::CreateShipGfx in CD3DWindow::SceneGraphCallback:
-52.9% (155mcs, 45,784 calls) was caused by CShipGraphic::LoadConfiguration
-32.9% (96.4mcs, 7,220 calls) was caused by CShipGraphic::AutoPlaceModule
-9.5% (27.7mcs, 1,715 calls) was caused by CShipGraphic::AutoPlaceEngine
-4.6% (13.5mcs, 2,096 calls) was caused by CShipGraphic::AutoPlaceWeapon

Each call to GetTexture also caused a call to GetFilename, which is expensive (16.7 mcs) due to _splitpath/string assign/_makepath etc.

* Perhaps some kind of texture cache?
* Convert to using D3DXCreateTextureFromFileInMemory, perhaps with memory-mapped IO? That way the file handle creation is under our control. I am sure the vast majority of the calls were to the same files.

---

Saving a game could probably do with better memory allocation methods.

When saving the game 4 times:

classGalaxy::SaveBlock

MemBuffer::AddElement called MemBuffer::GrowIncrementally 1,273,720 times.

The largest causes of time were saving of planets and (especially) ships - I'm guessing ships are big?

MemBuffer::Grow had a large self time (probably due to copying data between allocated and freed blocks, and spent even more time in MemManager::Allocate and MemManager::Free themselves.

This can get a little annoying when you're playing the game and you have to wait for it to autosave. Only 6 saves occurred in the game but in total they took 67 mcs)

----

More fun with huge trees to iterate through (and do repeated string comparisons of elements).

CPlanetSurface::FindFirstImprovements was an expensive call, using 32.8 mcs. 80% of this was in CPlanetImprovement::GetInternalName, which suggests that improvements there would be a help. Perhaps the real issue was that it had to iterate over every improvement item and then get the string and do a case-insensitive compare on it. That's gotta cost.

It was used most by classStarShip::GetMaxHitPoints (54%) and classColony::CalcMorale (46%), 0.3% by classGalaxy::NewShip

* Checking for the OmegaDefenseSystem in classStarShip::GetMaxHitPoints(VOID) caused 18 mcs on its own:

IPlanetImprovement* pImp = pPlanet->GetPlanetSurface()->FindFirstImprovementByName( "OmegaDefenseSystem", IPlanetImprovement::kOperational );
This line was called 750,000 times. It found the defense system 23,562 times.

GetMaxHitPoints itself was called by a lot of things, most notably classStarShip::RepairShip (42.6%) but also classStarShip;:GetCurrHitPoints (18%), classCivilization::CalcMilitaryMight (12%) and UpdateAITurn (8)

* Similarly, checking for SecretPoliceCenter in classColony::CalcMorale caused 15 mcs:

classCivilization::CalculateMoraleMight was the most expensive by far of all the historical calculations. It was expensive because classCivilization::CalcApproval was expensive, due to classColony::CalcMorale . . . due to CPlanetSurface::FindFirstImprovement.

---

What takes time for the AI?

classStarShip::UpdateTurn was most used by UpdateAITurn (98%)

classStarShip::UpdateTurn (109,600 calls)
-classStarShip::RepairShip (48%, 108,577 calls)
--CShipDamageIndicator::RepairDamage (14.3%, 108,577 calls) - but almost all the time used by this is in getting the max hit points
--classStarShip::GetMaxHitPoints (84.7%, 688,745 calls)
---CPlanetSurface::FindFirstImprovementByName (94.8%, 373,090 calls)
-classStarShip::CalcTotalMoves (39.5%, 108,577 calls)
--classCivilization::GetTradeGood (82.8%, 97,264 calls)
---classGalaxy::findImprovementInLocatorMap (96%, 97,264 calls)
-classStarShip::GetMaxHitPoints (6.5%, 110,482 calls)
-classStarShip::CalculateSensorRange (4.1%, 108,577 calls)

The reason for GetMaxHitPoints using lots of time due to the check for the Omega Defense System has been covered above.

* The CPU usage in CalcTotalMoves is mostly due to checking for the trade good GravityAccelerators, which costs 6 mcs, although calculating the starbase speed bonus and penalty takes up a small amount of time.

---

Pathing and moving ships around takes a long time.

Profile of classGalaxy::MoveShip

classGalaxy::MoveShip (128.6 mcs total)
-classStarShip::QuickMove (55.6%, 117,918 calls)
--classStarShip::Teleport (46.6%, 84,954 calls)
---classStarShip::CheckForBlockedGoal (56.9%, 19,712 calls)
----classStarShip::CheckForBlockedGoal (97.8%, 87,016 calls)
-----CSectorMapper::HitCheckPlanets (65.9%, 171,970 calls)
-----CSectorMapper::HitCheckShips (28.9%, 42,353 calls)
-classStarShip::FindPath (41.6%, 118,397 calls)
--classStarShip::CalAStarPathToDest (99.8%, 81,055 calls)
---classStarShip::CalcAStarPath (68.9%, 80,588 calls)    - both of these
---classStarShip::CalcAStarTradeRoute (31%, 6,194 calls) - go to the folowing:
----classStarShip::CalcAStarPath (96.5%, 80,588 calls)
-----AStarShipMoveCostCallback (95.8%, 2,801,622 calls)
------classStarShip::CalcAStarMoveCost (100%, 2,801,622 calls)
-------CSectorMapper::HitCheckShips (67.1%, 2,594,057 calls)
-------CSectorMapper::HitCheckPlanets (21.7%, 2,714,842 calls)
-------CSectorMapper::HitCheckStars (5.8%, 2,529,019 calls)

For classStarShip::HitCheckShips
-GetDistanceInTiles (53.4%, 55,934,764 calls)
--GCObject::GetTileX (23.6%, 77,861,854 calls)
---CSectorMapper::WorldToTileX (31.5%, 77,861,854 calls)
--GCObject::GetTileY (23.0%, 77,861,854 calls)
---CSectorMapper::WorldToTileY (31.1%, 77,861,854 calls)
--CSectorMapper::WorldToTileX (7.9%, 77,861,854 calls)
--CSectorMapper::WorldToTileY (7.3%, 77,861,854 calls)
-ShipHit (21.8%, 364,205 calls)
-equal_range (4.0%, 2,638,080 calls)

CSectorMapper::HitCheckPlanets takes up 23.8mcs in classStarShip::CheckForBlockedGoal, and there is a comment there that suggests that it might not be necessary (don't know enough about it to be sure):

// I'm going to try to try checking planets first because there should be no ships under planets
// but if they are, chances are we want to be attacking the planet
pSectorMapper->HitCheckPlanets (this, v3Destination);

----

StaticText::RenderText takes 67 mcs and PushButton::RenderText takes 33mcs. They both create a new text sprite each time, which costs most of the time.

StaticText::RenderText (67 mcs, 333,484 calls)
-CreateClippedTextSprite (45.3%, 309,778 calls)
-DdEntry5 (18.5%, 8,346 calls)

* Could text sprites on static text and buttons be cached to avoid costs with sizing and rendering text, without going overboard on memory use?

----

The UIUpdateGraphicMsg at the end of classStarShip::AddStarbaseModel is expensive, costing 6.7mcs for just 193 calls. Added to the DestroyShip it means that starbase additions cost almost 10mcs.

----

And the part I was particularly interested -- AI performance.  The AI in GalCiv II (post beta 5) is radically improved but much more intensive since it's doing a LOT of analysis.  Here's some of the calls and what they're using.

AI planning focus.

Obvious issues:

* Biggest cost (as previously identified) is ship design - classCivilization::AIDesignShipType - due mostly to loading the available components for each player. There is a huge cost in iterating through the component hash_map and (to a lesser extent) matching tech requiements. It also sorts the hash_map in GetComponentDefsOfClass - is this even needed? It takes literally over half the time.

* In classCivilization::AIActivateMilitary, classAICivilization_Diplomat::AIFindConstructorDestination is the main expense due to calling classCivilization::AIFindStarbaseThatNeedsUpgrades, which calls classStarbase::GetModulesAvailable. In fact, looking at the code, it calls it three times in if()s just to check different return values! (eg: if(pShip->pStarbase->GetModulesAvailable() > 20) . . .). Iterating through the entire list of available modules and matching them to the current starbase state and available techs is very expensive.

* classAICivilization_Diplomat::AISetColonySocialProject could do with a less expensive way of figuring out what social projects it can/should build.

* classShipTypes::GetMaxCapacity has pCiv->CalcAbility( ABILITY_MINIATURIZATION );, which really seems too slow to be called so often.

* classCivilization::CalcApproval calls classColony::CalcMorale which spends a heck of a lot of time looking for the SecretPoliceCenter on every colony.

* There's various DebugMessages that need to be weeded out eventually, particularly the SetSpendRatios ones.

* Use of classCivilization::CalcAbility in classShipTypes::GetMaxCapacity == slowdown

* Generally speaking, CPropertyBucket methods are used in loops that end up being called often and deeply.

Planning call tree hotspots (methods not contributing significantly not included):

AIPlanningStrategy (577 mcs)
-AIActivateDomesticPolicy (75.8%, 437 mcs, 2,873 calls)
--classCivilization::AIDesignShips (88.9%, 390 mcs, 687 calls)
---classCivilization::AIDesignShipType (100%, 390 mcs, 12,364 calls)
----CShipDesigner::LoadAvailableComponentsForPlayer (96.9%, 377 mcs, 8,930 calls)
-----GetComponentDefsOfClass (82.8%, 312 mcs, 80,368 calls)
------std::sort of ComponentDefMap (52.0%, 162 mcs, 80,368 calls)
------CPropertyBucket::GetIntProperty (43.2%, 135 mcs, 31,775,109 calls)
-----classCivilization::MeetsTechRequirements (16.3%, 61.3 mcs)
------CPropertyBucket::GetStringProperties (59.6%, 36.7 mcs, 3,976,506 calls)
------classGalaxy::FindTechByName (22.5%, 13.8 mcs, 1,157,569 calls)
----CShipDesigner:esignShipType (2.7%, 10.5 mcs, 8,390 calls)
-----classShipTypes::AddComponent (51.0%, 5.4 mcs, 36,081 calls)
------classShipTypes::GetMaxCapacity (53.9%, 2.9 mcs, 36,081 calls)
-----classShipTypes::GetMaxCapacity (27.9%, 2.9 mcs, 59,224 calls)
------classCivilization::CalcAbility (97.7%, 6.0 mcs total, 97,314 calls)
--ClassCivilization::AISetSpendRatios (9.5%, 24 mcs, 1,400 calls)
---DebugMessage (53.8%, 10 mcs)
---classCivilization::CalcResearchMightRating (21.1%, 3.9 mcs, 461,686 calls)
---classCivilization::CalcTotalSpending (11.4%, 2 mcs, 201,548 calls)
---classCivilization::CalcApproval (6.8%, 1.3 mcs, 136,144 calls)
---classCivilization::CalcRevenueFromTrade (5.5%, 1.0 mcs, 132,695 calls)
--class[AI]Civilization::AIResearchTech (1.3%, 4 mcs)
---DebugMessage (90%, 3.5 mcs)
--classCivilization::AIBookeeping (0.1%, 0.5 mcs)
--classCivilization::CalcShipMaint (67.6%, 0.36 mcs)
--classCivilization::AIChooseGovernment (0.1%, 0.4 mcs, 41,032 calls)
---classCivilization::CalcApproval (92.0%, 0.4 mcs, 687 calls)
----classColony::CalcMorale (92.9%, 0.4 mcs, 15,071 calls)
-----CPlanetSurface::FindFirstImprovementByName
-classCivilization::AIActivateGovernors (12.2%, 70.2 mcs, 2,873 calls)
--classAICivilization_Diplomat::AISetColonySocialProject (70.5%, 49.5 mcs, 1,400 calls)
---CPlanetImpDBRecord::IsAvailable (57.8%, 28.6 mcs, 356,940 calls)
----CPlanetImpDBRecord::GetStringField (24.1%, 8.3 mcs, 481,131 calls)
----classCivilization::IsTechKnown (23.6%, 8.1 mcs, 423,300 calls)
----CPlanetImpDBRecord::GetImprovementType (17.8%, 6.1 mcs, 459,000 calls)
----classCivilization::GetSuperProject (10.8%, 3.7 mcs, 22,131 calls)
----CPlaentImpDBRecord::GetIntField (8.9%, 3.1 mcs, 918,000 calls)
----CPlanetSurface::IsImprovementRestricted (3.8%, 1.3 mcs, 63,546 calls)
----CPlanetImpDBRecord::GetField (3.2%, 1.1 mcs, 84,279 calls)
---classColony::GetNumberOfImpType (13.1%, 6.5 mcs, 42,245 calls)
----CPlanetImprovement::GetIntValue (71%, 5 mcs, ~1,000,000 calls)
-----CCorePropertyHolder::HasProperty (53.3%, 3 mcs)
-----CCorePropertyHolder::GetProperty (37.0%, 2 mcs)
----CPlanetSurface::GetCompletedImprovements (23%, 2.0 mcs, ~43,000 calls)
---classColony::CalcTurnsRequired (8.5%, 4.2 mcs, 42,245 calls)
---CPlanetSurface::CalcFoodProduction (3.8%, 1.9 mcs, 42,647 calls)
----CPlanetSurface::calcOperationalTotal (98.6%, 1.8 mcs, 42,647 calls)
-----CPlanetImprovement::GetIntValue (81.7%, 1.5 mcs, ~380,000 calls)
--classAICivilization_Drengin::AISetcolonySocialProject (21.4%, 15 mcs, 400 calls) [details as above]
--classAICivilization_General::AISetColonyMilitaryProject (7.3%, 5.1 mcs, 1,800 calls)
---classAICivilization_Diplomat::AISetColonyMilitaryProject (87.8%, 4.5 mcs, 2,222 calls)
----classAICivilization_Diplomat::AIIsColonyAdequatelyDefended (46.6%, 2.1 mcs, 2,222 calls)
-----classGalaxy::GetMostPowerfulOtherPlayer (50.0%, 1.0 mcs, 2,222 calls)
-----classCivilization::GetToughestShip (49.5%, 1.0 mcs, 2,222 calls)
----classCivilization::CalcColonyMaint (36.5%, 1.6 mcs, 2,222 calls)
----classColony::CalcColonyMaint (87.7%, 1.5 mcs, ~15,000 calls)
-----CPlanetImprovement::GetMaintenanceCost (58.4%, 0.9 mcs, ~100,000 calls)
-----CPlanetSurface::GetCompletedImprovements (33.2%, 0.5 mcs, ~16,000 calls)
---classAICivilization_Drengin::AISetColonyMilitaryProject (10.5%, 0.5 mcs, 661 calls)
-classGalaxy::NextTurnRequested (7.0%, 40.7 mcs [39 mcs wait], 193 calls) [I don't think this is real performance loss]
--UIHideTurnButton (70.5%, 28.7 mcs, 193 calls)
--UIRefresh (29.5%, 12.0 mcs, 193 calls)
-classCivilization::AIActivateMilitary (3.5%, 20.4 mcs, 2,873 calls)
--classAICivilization_General::AIExecuteShipAssignments (93.0%, 19.0 mcs, 2,672 calls)
---classAICivilization_Diplomat::AIFindConstructorDestination (54%, 10.2 mcs, 16,810 calls)
----classCivilization::AIFindStarbaseThatNeedsUpgrades (98%, 10.4 mcs, 581 calls)
-----classStarbase::GetModulesAvailable (97.2%, 10.0 mcs, ~20,000 calls)
------classCivilization::MeetsTechRequirements (63.8%, 6.5 mcs, ~600,000 calls)
------FindStarbaseModuleByID (30.5%, 2.9 mcs, ~2,400,000 calls)
---classCivilization::AIFindConstructorDestination (11.3%, 2.1 mcs, 2,959 calls)
----classCivilization::AIFindStarbaseThatNeedsUpgrades (98%, 2.0 mcs, 187 calls) [as above]
---classCivilization::AIFindMilitaryDestination (7.0%, 1.3 mcs, 46,005 calls)
----classCivilization::FindNearestRallyPoint (30.9%, 0.4 mcs, 64,750 calls)
---classCivilization::AIFindTransportDestination (5.4%, 1.0 mcs, 4,228 calls)
----classStarShip::FindClosestUndefendedEnemyPlanet (61.2%, 0.6 mcs, 3,695 calls)
----classCivilization::AIHostilesInSector (34.7%, 0.4 mcs, 3,680 calls)
---classCivilization::AIFindScoutDestination (4.9%, 0.9 mcs, 4,169 calls)
--classCivilization::AIBuildFleets (6.1%, 1.2 mcs, 2,873 calls)
---classStarShip::IsStarbase (30.2%, 0.4 mcs, 1,496,390 calls)
-classCivilization::AIActivateForeignPolicy (1.4%, 8.3 mcs, 2,873 calls)
--classCivilization::AIMananangeForeignRelations (88.3%, 7.3 mcs, 1,001 calls)
---classCivilization::AICalculatePrimaryEnemy (29.6%, 2.2 mcs, 1,001 calls)
----classCivilization::AIEvaluateProximityRelations (66.2%, 1.4 mcs, 2,817 calls)
-----classCivilization::GetToughestShip (65.5%)
-----classCivilization::GetRankedPlanet (29.8%)
----classCivilization::AIEvaluateTradeRelations (18.3%, 0.4 mcs, 2,817 calls)
---classCivilization::AICalculateRelationsWith (26.4%, 1.9 mcs, 1,704 calls)
----classCivilization::AIEvaluateProximityRelations (65.0%, 1.3 mcs, 1,790 calls)
---classCivilization::AITradeTechnologies (25.4%, 1.9 mcs, 3,020 calls)
----classCivilization::IsTechKnown (58.6%, 1.1 mcs, 352,445 calls)
----DebugMessage (37.6%, 0.7 mcs, 92 calls)
---classCivilization::AIMonitorGalaxy (11.4%, 0.8 mcs, 3,020 calls)
----classGalaxy::GetMostPowerfulPlayer (96.3%, 0.8 mcs, 1,608 calls)
--classCivilization::AIEvaluateOpponentDefenses (11.7%, 1.0 mcs, 1,001 calls)
---classCivilization::GetToughestShip (49.1%, 0.5 mcs, 780 calls)
---classGalaxy::GetMostPowerfulOtherPlayer (32.9%, 0.3 mcs, 702 calls)
---classCivilization::FidnMostLethalEnemy (16.5%, 0.2 mcs, 925 calls)

Just a head's up, Stardock will soon be putting into beta a new Object Desktop program called SoundPackager (or some name that isn't already trademarked <g>). 

It will do for system sounds what IconPackager did for icons.  It will add a host of new sounds for all kinds of events on the system that will help spruce up your environment and give you greater control over it.  We will have a WinCustomize section for users to submit their sound packages as well.

People with fancy new monitors make me mad. That's because I don't have a fancy new monitor.  But Paul Boyer, the UI designer behind GalCiv II has a new one.  See the screenshot there? That's from his monitor.

There's been so much going on.  I didn't get to bed until 5am last night.  The number of touches, tweaks, and enhancements to make the game "fun" has been immense.

For me, I'm been working on balance. Making sure the game is fun and challenging.  This has mainly meant changes to the economics, weapons, costs, and most importantly, the computer AI.

I have to say, GalCiv II's AI is the finest AI that I have ever developed in my career.  I've done a lot of AI over the years and a lot of it has really culminated here.  For me, computer AI is not about making the AI win. That's easy.  What I want is an AI that plays like a human being would - minus the swearing, disconnects, etc.  The AI isn't quite human yet, there's coordination stuff that can be improved.  But it's leaps and bounders better than anything we've done before.  Given that our games are best known for computer AI, I think that says a lot.

The test this week has been what we're calling the Drengin/Human wars.  Duels.  One on One.  Can the AI adapt from dealing with a large galaxy with 9 other players to a tiny galaxy with only one player that wants to kill them?  The answer for the first GalCiv was no.  We never even tested dueling. Not enough time.  This time, the AI not only has multiple C++ personalities (our AIs aren't scripted).  They have multiple random sub-classes to call upon in order to deal with different situations.

The AI in GalCiv scales based on its intelligence.  As you increase the intelligence, new algorithms get unlocked.  At Beginner, the AI is really crippled (but fast).  At "Intelligent" it is playing its best game with the same stuff you've got.  My goal in testing it is to be able to have a hard time beating the AI at Intelligent. 

For players who have Beta 5, the AI metric we had would be what we'd call around a 20%.  That is, around 40% of players would be able to beat the AI at "Intelligent".  When we redid the AI during the holidays, we brought that number down to 20%.  At this point, I think we're getting down to around 5%.  And best of all, as we've improved the AI, we've been able to take away crutches.  In GalCiv I, the AI knew where the good planets were.  In GalCiv II, they don't. They have to scout them just like everyone else.

At first, I smoked the Drengin on intelligent.  Raising the difficulty to "Genius" made is to that I could only win if I got very good starting conditions.  But as I debugged and debugged I was able to make tweaks.  The hardest part is having the AI research the right technologies.  A good human strategy gamer often has a "build order".  But we don't want to get down with "Research this, then this, then this".  That's not AI. That' s a script.  We want the AI to choose good technologies based on analysis of the map, the situation its in, and then conclude that they need X.  So it's easy to tell the AI "Get weapons, then get invasion, then build transport and take human out." That might work for the particular situation, but it's not useful for the thousands of other scenarios out there.  That's why most single player games get boring -- scripted AIs. You figure out their trick eventually and counter them. 

If anyone is interested, I can write up the results of one of my recent tests.