Match #5:

The Settings

The Drengin and I are playing on a tiny galaxy. Just them and me. They are set to "intelligent" which means they play their best game without any advantages.  The goal is to keep tuning the AI so that at map setting that at intelligent they can play a very tough game against experienced players without cheating.

The War has begun..

While I could use a pre-made map, I am also doing this testing to make sure and tune the random map generator so that it creates good maps for all players. This map starts off slightly in favor of the human player.

I concentrate purely on research.  I also put my effort into making my ships faster by getting impulse power and then warp drive.  I used to go for military power early on but found that this didn't really buy me much because I didn't actually get to use that military power until later so better to get faster ships.

One of the things I notice is that the AI is not good at adapting its build strategy.  That is, now that it has to explore the galaxy, it is building things like constructors when it should build colony ships.  It should be able to figure out that there are some good colonies out there it just needs to build colony ships sometimes.

First Fleet Battle

The early fleet battles between the Drengin and Terrans have an interesting design strategy.  I focused purely on offense. The Drengin modified their ships to have countermeasures to my mass-drivers.  But which way is the way to go?  Since I attacked them, I get the initial advantage (it's important to be the attacker).  Net result, I win the first engagement with only 1 ship badly damaged.  But as the M0 indicates, this is the AI's first go at it.

The human race's state of the art ship for the ship at first and what would be the main ship of the line was delicately called "The Clubber"

The Clubber has a crew of 5 and is armed with 3 OTO Smithon Industries Standard Rail Guns that were the result of Mass Driver II technology giving it an attack rating of 3 in the mass-driver category.

It was 15.5 meters in length and 25.5 span-wise.    Engines were standard Hyperdrive issue.

Nothing fancy but it did the job.

The Drengin Reaction

The Drengin reacted slowly but surely to the Clubber.  After some fits and starts, their third generation heavy-fighter class called the "Dra'Hak" which roughly translates to "Human Death Giver" was the ship they hoped to clubber mine with.

Armed with two first generation plasma beam systems of unknown configuration and a 5th generation laser system, it had quite a punch. It also had armor. It was a Clubber-killer in design.

Ironically, the first Clubber vs. Dra'Hak battle took place in a one-on one battle.  The Clubber was our flag ship Clubber, it was rated at level 10 in experience.

The result proves what pilots have said for years -- experience can trump technology -- to a point.

But it was clearly time for a new ship.  Mass Driver IV technology with enhanced miniaturization would bring about a revolutionary new design.

The new ship, the Banger:

   

The "Banger" was the result of new miniaturization techniques pioneered by top scientists on Earth. 

The Banger has a crew of 6 and is armed with 4 Hyperion Railguns also from Smithon Industries. It has a damage rating of 8 from mass-attack.  Drengin Armor continues to improve but not fast enough to counter it.  The only problem is that the Drengin have a lot more Dra'Haks to throw at me.

While I had the initial advantage in planets, I had not been adapting my own strategy.  For most games I had played so far, the AI was slow to get transports.  This was one of the flaws in its strategies I had tried to rectify -- to have it "think" about what techs it should get a bit more and away from randomness.

So the result was that my strategy largely depended on me rush claiming a bunch of worlds without worrying about defense.  But this time, the Drengin conquered the jewel of the Terran Empire -- Markus V, a class 17 planet.  Worlds of class greater than 10 are rare to begin with, class 17s are incredibly rare. And foolishly I left it undefended.  The Drengin attacked with mass-drivers reducing it to a class 15.  A true pity but a class 15 is still impressive.  Losing such a valuable planet early on would have very negative consequences.  

By the skin of my teeth and thanks to having research shock troops, I was able to recapture Markus V without any further damage to the planet's environment.  By the dawn of 2230, the galaxy was split in half.

The Drengin aren't winning as much as they've claimed more territories thanks to starbases.  They were able to do that because I had to spend so much industrial output making up for my blunder on Markus V.  I have 6 worlds, 5 of which are truly good worlds.  The Drengin also have 6 worlds. But only 3 of them are "good".

But it's not quite as good for the humans as the raw stats imply because the Drengin population is fully developed while I've been struggling to take planets 4 through 6 at the bottom left there.  So the Drengin economy is a bit head of mine because of their population.  With the galaxy split in two the war would change because our ship designs had been short-range.  Now there was some distance between us.

Moreover, new adaptions from the last generation AI had taken effect -- all but one of the main Drengin planets had orbital fleet managers. That means their ships in orbit fight as a single fleet which is much tougher and expensive to take out.  Luckily Xasica is out in front.  And my spies tell me they were building a orbital fleet manager so time is short.

BTW, if my graphics seem blocky, I have it running in the debugger with anti-aliasing off.

The Drengin soldiers are able to withstand my first invasion attempt.  But I have it blockaded with a fleet of Bangers. 

My second invasion was successful and I was able to discover some weak spots in the Drengin AI.

First off, this planet was late colonized but should have been further along.  The AI needs to be able to better focus on planets. On a planet like this that had so many incomplete social projects it should have put more focus onto getting them built.  If Xasica V had gotten its orbital fleet manager built just a few weeks earlier, I may not have been able to conquer it. 

At this point, the balance of the war is in my favor.  And to verify that, the Drengin finally decided I was worth talking to for the first time.

and several months later..

 

Another issue with the AI is in defense of its planets. Unless they have an orbital fleet manager, having a bunch of ships in orbit doesn't really help it unless those ships are really tough.  Right now, it tends to aggressively defend its planets with 3 or 4 ships.  It would really be better off putting resources into military action and not beefing up defenses to that level until it has an orbital fleet manager.

However, Xasica would change hands a couple of times.  When I sent my main fleet elsewhere, the Drengin retaliated and retook the planet.  I ended up retaking it but it was a sign not to underestimate the Drengin Empire's desire for victory.

Wow. In an amazing turn of events, and something I've not actually seen during the beta:

The Drengin have had an internal revolution in which they have gone from being evil and blood thirsty to kind and gentle.  This is a good opportunity to test out the dialog code for the Drengin..

The battle of the line

The Drengin had gotten better in recent AI updates at putting together fleets big enough to tackle my fleets.  This was to be a 6 on 6 match but the Drengin got the drop on me which is a big advantage. Also, their armor had really advanced so my ships were not nearly as tough as I'd like. The result was a serious defeat with the loss of all 6 of my ships versus only 3 of theirs.

The difference: THIS SHIP:

   

The brand new K'Zalan (Virtuous Justice -- no doubt based on their new alignment change).  It's the first Frigate class starship we've come across.  Theorized specs:

Level 2 Plasma beam of an attack rating of 10 with Duranthium based armor of level 8.  Crew complement of 35 Drengin.  Length approximately 100 meters by 60 meters.  It can really take a beating while giving one out too.  It was time to look at either producing our own frigates or to upgrade our heavy fighters.

Luckily, the Drengin fleets are still hampered by short-range.  The Drengin have not been research life support systems.  So at the close of 2130, the humans have the long-term edge, but the Drengin have the military might but not the range to make use of it (something to tweak -- AI needs to evaluate its range situation better). 

Speaking of range, I embark on a cross course to improve my range with new life support systems.  Life support technologies improve your existing base range plus give you extended range modules for your ships.  Soon the core Drengin worlds would be in range.

As 2231ended, the galactic balance we pretty even.  But I was preparing my final offensive. I've won the game so you can quit reading at this point.  The AI, while improved, still needs some work.  Its military is too undirected, too unfocused still.  

That means it's time to go into the debugger and figure out what the deal is. 

(2 hours pass..)

The problem is more like a bug.  Two issues in particular.  First, someone changed the way that colonies were being ranked in such a way that the AI no longer had access to the data.  So it was only getting its home planet as a key world.  This meant that it wasn't protecting its key worlds very well -- i.e. it didn't know to send reinforcements to planets under attack.  Secondly, there was an issue where the AI would focus on getting to rally points even if it was plenty tough to do the job.  I.e. it was too conservative in building itself up.

So now to go fix that.

(another hour passes)

Let's see how the AI can recover.  First thing I notice is that the AI has started sending massive forces at the newly identified threat.  My transport has already wiped out most of the inhabitants but it'll take a second one to finish the job.

The numbers look more impressive than they are.  These are units that had been cooped up around their home world (Because they thought only Drengi was worth really protecting).  So these are a bunch of mark 0 (first generation) small fighters. I wipe them out.

But there is wave after wave of these things and my bangers are finally taken out. But not before I take Martzia.  But unfortunately, it's undefended.

The Drengin take back Martzia and are now back at Xasica.    Time to regroup.  The Drengin are using military forces they've had all along but were using them poorly.  The changes made and brought into the saved game have caused them to focus this pent up military power very specifically. Xasica has an Orbital Fleet Manager however.  So a lone mark 6 isn't powerful enough to take out the defending fleet.

A huge battle eventually does take place in which we both militarily take out the trash.  Our offensive capacities have been restricted for a bit.  Still, the AI hasn't sent ships to follow up its attack. 

(back to the debugger).

Fixed another issue with it not going after planets that are outside a few key sectors. Or I should say, not focusing.  Though I am still pretty safe, the AI's range is still terrible and while I've fixed it, it won't be updated in this game.

I on the other hand don't have this problem.  All my fleets go to Martzia. For a huge battle:

After several back and forth battles, Martzia is mine again. Drengin, the capital of the Drengin Empire, is in range now.

Still, the Drengin have rolled out the 6th generation of their fighter:

   

Advanced Plasma weapons coupled with Duranthium armor give it a beam attack of 7 and an armor rating optimized against my weapons of 5.  In head to head, it is superior to the Banger.  It single combat, it will destroy a Banger and still keep 40% of its hull integrity.  The Banger is getting old.

In the meantime, Drengi has been conquered and the Drengin ambassador, now under the rule of the reformed Drengin ways of niceness (remember earlier event) speaks differently:

Oh boo-hoo.  Sure, I'm evil but whatcha going to do about it?  Another thing I modify is that AI ships need to worry less about maximizing their fleet sizes and go for targets of opportunity more.  They need to THINK MORE and follow scripted directives less.  So this change is in though a bit too late for the Drengin this time.

Which brings me to my war finishing design: Code-named "Pounder".

Armed with 5 twin-Singularity based cannons from Smithon Industries. The projectiles are incredibly high density and travel at near relativistic speeds.  The fighter is 18 meters long with a 12 foot wing span.  It has a crew of just 4.  It is relatively short-range and not as fast as it could be, but it won't need to. This is the mop up.

The knife tip though is the new Frigate called the Crusher.  It's the first Frigate-class ship in the Terran Empire.

Same technology level as the Pounder. But it is equipped with the next generation Warp Drive giving it 3 more parsecs/week in terms of speed.  It has a crew of 50 and is 150 meters in length. It's fairly large, fast, and tough. An attack rating of 12 with a shield defense of 3 should keep the Drengin down for the count. 

The only thing saving the Drengin is that they found a Precursor ship that is of Ranger class. It's tough but it's only one ship. It takes two full fleets of Bangers to take it out.  At this stage, the Drengin are down to 2 minor colonies.

But I don't have to mop them up:

They surrender.

And so ends another episode of Human Drengin wars.  The AI has been improved, tuned up and next time they will probably be even more challenging. 

T-minus 10 days until code-freeze. There is still time to keep improving the AI.  Even though I thoroughly defeated the Drengin (again), it was tougher than previous and it was far far better at playing than GalCiv I was.

While beta testers seem pretty pleased with the performance, there is always room for improvement...A LOT OF ROOM.  Users with beta 5 will be astounded by the improvements in performance.  Game loading speeds cut by 90% over beta 5. Save game speeds cut to 30% the time of beta 5.  But there's still more. 

The key is profiling.  That is, you use tools that literally tell you how many ms the game spends in different areas.  The Green Reaper finally graduated this past year and has joined Stardock as a full time developer.  Having him is like having the first round draft choice of all developers worldwide for a given year.  As a teen, in his spare time, he reverse engineered the multiplayer server for The Corporate Machine and made a new one (That we still use today).  His primary project is working on our killer app for Windows Vista (which I can't talk about).  But for this month, I've tasked him with figuring out what is slow in GalCiv and how to fix it.  He is also doing the galactic empire map for the Metaverse which you'll hear more about soon.

Here are his findings (unedited):

Seemingly a big problem (this is where the std::tree stuff was coming on that I found with CodeAnalyst):

AIPlanningStrategy (60 million mcs total):
-classCivilization::AIActivateDomesticPolicy (17%, 392 calls)
--classCivilization::AIDesignShips (85%, 122 calls)
---classCivilization::AIDesignShipType (99%, 2,196 calls)
----CShipDesigner::LoadAvailableComponentsForPlayer (99%, 1,586 calls)
-----GetComponentDefsOfClass (79%, 14,274 calls)
------CPropertyBucket::GetIntProperty (47%, 4,140,224 calls) [due to: if(pComponent->GetIntProperty(COMPONENTCLASS, -1) == cc)]
-------CPropertyBucket::GetValue (53%, 4,207,204 calls)
--------std::tree::find
---------std::tree::_Lbound
---------std::basic_string::compare
--------std::tree::assign
-------_tstoi (30%, 4,182,617 calls)
------std::sort components (46%, 14,289 calls) [due to using CompDefGreater, which has a string comparison on the name]
-----classCivilization::MeetsTechRequirements (20%, 459,940 clalls)
------classGalaxy::FindTechByName (almost as many calls, making it an expensive function)
------CPropertyBucket::GetStringProperties (also lots of calls, and twice as expensive due to its resulting callees)
-classAICivilization_Diplomat::AISetSpendRatios (7%, but only because of the DebugMessage call)

I think four million calls anything is probably a bad idea.

LoadAvalableComponentsForPlayer looks like it needs to be cached, or one of the other methods above it needs to be trimmed down.

MeetsTechRequirements has similar issues, due in part to the cost of its callee methods (but also because it was called half a million times)

GetValue caused significant time use elsewhere, too. Perhaps there a more performant data type that could be used than the current one, or maybe values need to be cached.

Other biggies

RenderNode (63 million mcs total)
-CStarbox.Render (42%, 7,199 calls)

CShipGraphic::LoadConfiguration (43 million mcs)
-CShipGraphic::AddComponent (76%)
--CShipGraphic::LoadBumpAndLights (84%)
---CResourceManager::GetTexture
----LoadTexture (most?)
----GetFilename might be worth looking at, as it is called 150,000 times overall
-CShipGraphic::SetShipFile

On a second run:

UIUpdateGraphicMsg used significant CPU time in:

classGalaxy::FillMovableShipList (used by ::OnUpdate of CBegin[Player|AIPlayer]TurnProcess)
--classCivilization::AIAddShipsToFleets
---classGalaxy::AINewFleetFromStackedShips
----claasGalaxy::AddStackedShips
-----CFleet::UpdateFleet
------CFleet::GetTopShips
-------UIUpdateGraphicMsg (21 calls, comment for use: "update the fleet unit graphic" - resulted in almost all of the CPU use of the higher five methods)

..

Easy fruit: CMouseCursor::UpdateImage (59 million mcs, 13,636 calls) - the call to SetCursorProperties at the end of this method (that occurs in every pass through GameLoop) is not free! It results in constant calls to CreateDIBitmap, DestoryCursor and CreateIconIndirect, as well as critical section usage (and entering a critical section can be expensive on multiprocessor machines, I think including hyperthreaded ones). If it is possible to detect when it is and is not necessary to update the cursor then that would be good. SetCursorPosition should be used to move the cursor, not SetCursorProperties. See help for IDirect3DDevice9::SetCursorProperties.

AIDesignShips and (to a lesser extent) AICivilization_Diplomat::SetSpendRatios are using a bit on startup, the latter mostly because of a DebugMessage

CMidpointDisplacementHF::GenerateField uses up 27 million mcs, in part via this call path:

InitializeHiddenPlanetSurfaces
-classPalent::InitiPlanetSurfaces
--CPlanetSurface::InitSurface (252 calls, 40.9 million mcs)
---CPlanetSurface::generateHeightField (252 calls)
----CMidpointDisplacementHF::GenerateField (39 calls, self time 6.5 million mcs, total time 27.3 million

THis is due mainly to calls to CCoreRandomBase::RandFloat (6.8 million of them) and possibly double->float conversions relating to that. If this is being done on a background thread, it may not be that much of a problem.

EnterCriticalSection/LeaveCriticalSection can be a significant part of the screen update in GameLoop (10 million mcs verses the 42 million mcs of the Draw3D itself)

..

This particular performance run was done in a large galaxy with lots of planets. The previous notes about performance apply here as well.

CPlanetTerrainGraphic::calcTerrainColor used in CPlanetTerrainGraphic::CreateTexture was a significant use of time. It got called 2.3 million times and used up over the vast majority of the texture generation time, and seemingly around pmethe CPU cycles used to load the game in total. 21% of its use was encapsulated in CPlanetTerrainGraphic::GetNeightborBleed (which was also called in CreateTexture).

UIBeginGame (100%, 2 calls)
-CD3DWindow::InitGalaxyGfx (58.2%,, 2 calls)
--CD3DWindow::CreatePlanetGfx (86.0%, 84 calls)
---CPlanet3DWindow::CreatePlanetTexture (98.5%, 24 calls)
----CPlanetTerrainGraphic::CreateTexture (98.6%, 1,728 calls)
-----CPlanetTerrainGraphic::genTextures (99.9%, 1,728 calls)
------calcTerrainColor (69.2%, 1,769,472 calls)
------CPlanetTerrainGraphic::getNeighborBleed (22.7%, 1,916,928 calls)
-------calcTerrainColor (84.3%, 532,890 callls)
-------CPlanetSurface::GetTerrainType (12.1%, 532,890 calls)

--CD3DWindow::CreateShipGfx (6.3%)
--CD3DWindow::CreateAnomolyGfx (5.6%)
--CD3DWindow::CreateStarGfx (1.3%)

While the entire calcTerrainColour was expensive, particularly expensive was:
* Indirection of the CPlanetTerrainGraphic::tColorDefList::iterator defIt
* Comparison against colorSet.mDefaultColorList.end

Sections of interest were:
* The do loop following "// There's no override for this terrain type, so use the default straight up".
** Here the repeated indirection (*defIt) was costly, as was the call to lerp_color and the comparison "while ( defIt != colorSet.mDefaultColorList.end() );"

* the do loop following "// We may need to interpolate between the default and override schemes."
**Again, indirection caused the most expense here
**The code following "// See if we ran off the end of the list" was not executed. This *may* not be needed.

* The contents of the else labelled "// The texel falls within the override range"
** More indirection, more comparison against end

* The last line (call to lerp_color) was slightly expensive, but little compared to the first two

It would probably be worth trying to cache (*overIt) at the start of each loop iteration and use that within the loop. Of course, the compiler might be doing this. There appears to be an average of 40 indirections per call to calcTerrainColor

It might also be possible to cache some of the things like .begin and .end in calcTerrainColor.

Another big CPU user was classGalaxy::LoadBlock, which came back to the generation of large number of random numbers:

classGalaxy::LoadBlock (40.6 mcs, 2 calls)
-classPlanet::FromBlock (89%, 910 calls)
--CPlanetSurface::InitSurface (97.9%, 506 calls)
---CPlanetSurface::SeedSurfaceFromTerrain (98.3%, 506 calls)
----CPlanetSurface::generateHeightField (89%, 506 calls)
-----CMidpointDisplacementHF::GenerateField (97.3%, 70 calls)
------CCoreRandomBase::RandFloat (75.8%, 36,700,020 calls)
-------CCoreRandomMT::Rand32 (69.2%, 36,700,020 calls)
----CPlanetSurface::smoothSector (6.2%, 7,492 calls)

In generateHeightField (30 mcs including calls, 7mcs self time), the key lines were:

* The inner statement of the for loops *before* "Square step -", beginning "mpHeightField[mi+mj*rectWidth] (7.7 mcs taken by mRandGen.RandFloat)
* The line following "Calculate the square value for the top side of the rectangle" (10.4 mcs taken by mRandGen.RandFloat)
* The line right after that, "Calculate the square value for the left side of the rectangle" (4.9 mcs taken by mRandGen.RandFloat)

For loading, the main Lib3D usage (Which is not *too* big) appears to be:

CResourceManager::GetRootFrame
-CResourceManager::ComputeMeshTargets
--MeshMender::Mend
---MeshMender:rocessBinomials
---MeshMender:rocessTangents
----MeshMender::BuildGroups (used by both the above)
-----MeshMender::FindNeighbors

There is a thread that appears to spend a lot of its time looking at the foreground window. It sets an event. The call GetForegroundWindow in this thread is somewhat expensive, certainly in relation to the other calls in that thread procedure (like GetParent, IsWindow, GetWindowPlacement And GetWindowThreadProcID). If it's on a timer it might be good to reduce its frequency (if possible).

Analyis of long-running game. This game was one where I just stayed in the corner and did nothing, so it's mostly measuring the background processes like AI and ship building updates that always occur in a turn.

----

Large rendering costs:
CScene::Render (2,810 mcs, 19,151 calls)
-CShipGraphic::Render (1,169 mcs [89.6 mcs wait time], 84,874 calls)
-CPlanetoidGraphic::Render (1,028 mcs [873 mcs wait], 83,257 calls)
-COrbitingGraphic::Render (408 mcs [23.5 mcs wait], 16,594 calls)
-CStarbox::Render (113 mcs [36.1 mcs wait], 37,045 calls)
-CRingGRaphic::Render (20.8 mcs [1.26 mcs wait], 16,594 calls)
-CZOCGraphic::Render (16.8 mcs [3.01 mcs wait], 48,845 calls)
-CStarGraphic::Render (9.86 mcs [1.35 mcs wait], 16,726 calls)
-CFOWGraphic::Render (7.42 mcs [0.47 mcs wait], 61,069 calls)

* The large wait time on CPlanetoidGraphic is odd. Perhaps a large amount of processing is being done elsewhere each time it is called? IF so, perhaps this could be reduced somehow?

----

I think I've mentioned this one before, but UIUpdateGraphicMsg used up 424 mcs. 78.6% of the CPU time used by it was caused by classColony:;CompleteShip, a method which was responsible for 98% of the time used by classGalaxy::BeginTurn (this happens every turn via UpdateProcesses -> OnUpdate -> BeginTurn). Another 4.5% was caused by classStarShip::CompleteTradeRoute, 3.5% by GCObject::SetPosition and 3.3% by both classStarShip::UpdateFOW and CFleet::GetTopShips.

I think the large usage in CompleteShip is caused by:

// Joe 04/07/05 - There is a crash when an orbited ship is attacked.
//                In the case of a new ship, this was caused by
//                the ship not having a graphic node and not being added
//                to the sector node.
//                To fix the problem, I create the graphic and immediately
//                hide it. This ensures that all ships have graphics when
//                they are needed.
UIUpdateGraphicMsg(GCMSG_UPDATE_GFX, pshipBuilding->GetID(), pshipBuilding->GetType());
UIUpdateGraphicMsg(GCMSG_HIDE_GFX, pshipBuilding->GetID(), pshipBuilding->GetType());

* How many ships are created but just stay in orbit, or otherwise out of view? I wasn't exploring anywhere in this game - most things were hidden by the fog of war. It seems silly to have to spend time and memory making images for things that are never seen.

* It could well be worth seeing if there is another method of avoiding the crash that performs better and doesn't have side effects.

* I'm wondering if this is part of what is taking up memory later on in the game - how much memory is associated with a "graphic node"?

---

NtCreateFile was called 1,822,520 times, resulting in 246 mcs. I don't think GalCiv II has nearly 2 million files. Opening a file has significant cost even if the file data is in cache due to security checks etc.

Almost all of these calls were made by CResourceManager::GetTexture (via D3DXCreateTextureFromFile), which was itself called 3,250,089 times. CShipGraphic::LoadBumpAndLights took up this time, due to CShipGraphic::AddComponent.

of time taken by CShipGraphic::AddComponent ultimately almost all was used bysubmethods of CD3DWindow::CreateShipGfx in CD3DWindow::SceneGraphCallback:
-52.9% (155mcs, 45,784 calls) was caused by CShipGraphic::LoadConfiguration
-32.9% (96.4mcs, 7,220 calls) was caused by CShipGraphic::AutoPlaceModule
-9.5% (27.7mcs, 1,715 calls) was caused by CShipGraphic::AutoPlaceEngine
-4.6% (13.5mcs, 2,096 calls) was caused by CShipGraphic::AutoPlaceWeapon

Each call to GetTexture also caused a call to GetFilename, which is expensive (16.7 mcs) due to _splitpath/string assign/_makepath etc.

* Perhaps some kind of texture cache?
* Convert to using D3DXCreateTextureFromFileInMemory, perhaps with memory-mapped IO? That way the file handle creation is under our control. I am sure the vast majority of the calls were to the same files.

---

Saving a game could probably do with better memory allocation methods.

When saving the game 4 times:

classGalaxy::SaveBlock

MemBuffer::AddElement called MemBuffer::GrowIncrementally 1,273,720 times.

The largest causes of time were saving of planets and (especially) ships - I'm guessing ships are big?

MemBuffer::Grow had a large self time (probably due to copying data between allocated and freed blocks, and spent even more time in MemManager::Allocate and MemManager::Free themselves.

This can get a little annoying when you're playing the game and you have to wait for it to autosave. Only 6 saves occurred in the game but in total they took 67 mcs)

----

More fun with huge trees to iterate through (and do repeated string comparisons of elements).

CPlanetSurface::FindFirstImprovements was an expensive call, using 32.8 mcs. 80% of this was in CPlanetImprovement::GetInternalName, which suggests that improvements there would be a help. Perhaps the real issue was that it had to iterate over every improvement item and then get the string and do a case-insensitive compare on it. That's gotta cost.

It was used most by classStarShip::GetMaxHitPoints (54%) and classColony::CalcMorale (46%), 0.3% by classGalaxy::NewShip

* Checking for the OmegaDefenseSystem in classStarShip::GetMaxHitPoints(VOID) caused 18 mcs on its own:

IPlanetImprovement* pImp = pPlanet->GetPlanetSurface()->FindFirstImprovementByName( "OmegaDefenseSystem", IPlanetImprovement::kOperational );
This line was called 750,000 times. It found the defense system 23,562 times.

GetMaxHitPoints itself was called by a lot of things, most notably classStarShip::RepairShip (42.6%) but also classStarShip;:GetCurrHitPoints (18%), classCivilization::CalcMilitaryMight (12%) and UpdateAITurn (8)

* Similarly, checking for SecretPoliceCenter in classColony::CalcMorale caused 15 mcs:

classCivilization::CalculateMoraleMight was the most expensive by far of all the historical calculations. It was expensive because classCivilization::CalcApproval was expensive, due to classColony::CalcMorale . . . due to CPlanetSurface::FindFirstImprovement.

---

What takes time for the AI?

classStarShip::UpdateTurn was most used by UpdateAITurn (98%)

classStarShip::UpdateTurn (109,600 calls)
-classStarShip::RepairShip (48%, 108,577 calls)
--CShipDamageIndicator::RepairDamage (14.3%, 108,577 calls) - but almost all the time used by this is in getting the max hit points
--classStarShip::GetMaxHitPoints (84.7%, 688,745 calls)
---CPlanetSurface::FindFirstImprovementByName (94.8%, 373,090 calls)
-classStarShip::CalcTotalMoves (39.5%, 108,577 calls)
--classCivilization::GetTradeGood (82.8%, 97,264 calls)
---classGalaxy::findImprovementInLocatorMap (96%, 97,264 calls)
-classStarShip::GetMaxHitPoints (6.5%, 110,482 calls)
-classStarShip::CalculateSensorRange (4.1%, 108,577 calls)

The reason for GetMaxHitPoints using lots of time due to the check for the Omega Defense System has been covered above.

* The CPU usage in CalcTotalMoves is mostly due to checking for the trade good GravityAccelerators, which costs 6 mcs, although calculating the starbase speed bonus and penalty takes up a small amount of time.

---

Pathing and moving ships around takes a long time.

Profile of classGalaxy::MoveShip

classGalaxy::MoveShip (128.6 mcs total)
-classStarShip::QuickMove (55.6%, 117,918 calls)
--classStarShip::Teleport (46.6%, 84,954 calls)
---classStarShip::CheckForBlockedGoal (56.9%, 19,712 calls)
----classStarShip::CheckForBlockedGoal (97.8%, 87,016 calls)
-----CSectorMapper::HitCheckPlanets (65.9%, 171,970 calls)
-----CSectorMapper::HitCheckShips (28.9%, 42,353 calls)
-classStarShip::FindPath (41.6%, 118,397 calls)
--classStarShip::CalAStarPathToDest (99.8%, 81,055 calls)
---classStarShip::CalcAStarPath (68.9%, 80,588 calls)    - both of these
---classStarShip::CalcAStarTradeRoute (31%, 6,194 calls) - go to the folowing:
----classStarShip::CalcAStarPath (96.5%, 80,588 calls)
-----AStarShipMoveCostCallback (95.8%, 2,801,622 calls)
------classStarShip::CalcAStarMoveCost (100%, 2,801,622 calls)
-------CSectorMapper::HitCheckShips (67.1%, 2,594,057 calls)
-------CSectorMapper::HitCheckPlanets (21.7%, 2,714,842 calls)
-------CSectorMapper::HitCheckStars (5.8%, 2,529,019 calls)

For classStarShip::HitCheckShips
-GetDistanceInTiles (53.4%, 55,934,764 calls)
--GCObject::GetTileX (23.6%, 77,861,854 calls)
---CSectorMapper::WorldToTileX (31.5%, 77,861,854 calls)
--GCObject::GetTileY (23.0%, 77,861,854 calls)
---CSectorMapper::WorldToTileY (31.1%, 77,861,854 calls)
--CSectorMapper::WorldToTileX (7.9%, 77,861,854 calls)
--CSectorMapper::WorldToTileY (7.3%, 77,861,854 calls)
-ShipHit (21.8%, 364,205 calls)
-equal_range (4.0%, 2,638,080 calls)

CSectorMapper::HitCheckPlanets takes up 23.8mcs in classStarShip::CheckForBlockedGoal, and there is a comment there that suggests that it might not be necessary (don't know enough about it to be sure):

// I'm going to try to try checking planets first because there should be no ships under planets
// but if they are, chances are we want to be attacking the planet
pSectorMapper->HitCheckPlanets (this, v3Destination);

----

StaticText::RenderText takes 67 mcs and PushButton::RenderText takes 33mcs. They both create a new text sprite each time, which costs most of the time.

StaticText::RenderText (67 mcs, 333,484 calls)
-CreateClippedTextSprite (45.3%, 309,778 calls)
-DdEntry5 (18.5%, 8,346 calls)

* Could text sprites on static text and buttons be cached to avoid costs with sizing and rendering text, without going overboard on memory use?

----

The UIUpdateGraphicMsg at the end of classStarShip::AddStarbaseModel is expensive, costing 6.7mcs for just 193 calls. Added to the DestroyShip it means that starbase additions cost almost 10mcs.

----

And the part I was particularly interested -- AI performance.  The AI in GalCiv II (post beta 5) is radically improved but much more intensive since it's doing a LOT of analysis.  Here's some of the calls and what they're using.

AI planning focus.

Obvious issues:

* Biggest cost (as previously identified) is ship design - classCivilization::AIDesignShipType - due mostly to loading the available components for each player. There is a huge cost in iterating through the component hash_map and (to a lesser extent) matching tech requiements. It also sorts the hash_map in GetComponentDefsOfClass - is this even needed? It takes literally over half the time.

* In classCivilization::AIActivateMilitary, classAICivilization_Diplomat::AIFindConstructorDestination is the main expense due to calling classCivilization::AIFindStarbaseThatNeedsUpgrades, which calls classStarbase::GetModulesAvailable. In fact, looking at the code, it calls it three times in if()s just to check different return values! (eg: if(pShip->pStarbase->GetModulesAvailable() > 20) . . .). Iterating through the entire list of available modules and matching them to the current starbase state and available techs is very expensive.

* classAICivilization_Diplomat::AISetColonySocialProject could do with a less expensive way of figuring out what social projects it can/should build.

* classShipTypes::GetMaxCapacity has pCiv->CalcAbility( ABILITY_MINIATURIZATION );, which really seems too slow to be called so often.

* classCivilization::CalcApproval calls classColony::CalcMorale which spends a heck of a lot of time looking for the SecretPoliceCenter on every colony.

* There's various DebugMessages that need to be weeded out eventually, particularly the SetSpendRatios ones.

* Use of classCivilization::CalcAbility in classShipTypes::GetMaxCapacity == slowdown

* Generally speaking, CPropertyBucket methods are used in loops that end up being called often and deeply.

Planning call tree hotspots (methods not contributing significantly not included):

AIPlanningStrategy (577 mcs)
-AIActivateDomesticPolicy (75.8%, 437 mcs, 2,873 calls)
--classCivilization::AIDesignShips (88.9%, 390 mcs, 687 calls)
---classCivilization::AIDesignShipType (100%, 390 mcs, 12,364 calls)
----CShipDesigner::LoadAvailableComponentsForPlayer (96.9%, 377 mcs, 8,930 calls)
-----GetComponentDefsOfClass (82.8%, 312 mcs, 80,368 calls)
------std::sort of ComponentDefMap (52.0%, 162 mcs, 80,368 calls)
------CPropertyBucket::GetIntProperty (43.2%, 135 mcs, 31,775,109 calls)
-----classCivilization::MeetsTechRequirements (16.3%, 61.3 mcs)
------CPropertyBucket::GetStringProperties (59.6%, 36.7 mcs, 3,976,506 calls)
------classGalaxy::FindTechByName (22.5%, 13.8 mcs, 1,157,569 calls)
----CShipDesigner:esignShipType (2.7%, 10.5 mcs, 8,390 calls)
-----classShipTypes::AddComponent (51.0%, 5.4 mcs, 36,081 calls)
------classShipTypes::GetMaxCapacity (53.9%, 2.9 mcs, 36,081 calls)
-----classShipTypes::GetMaxCapacity (27.9%, 2.9 mcs, 59,224 calls)
------classCivilization::CalcAbility (97.7%, 6.0 mcs total, 97,314 calls)
--ClassCivilization::AISetSpendRatios (9.5%, 24 mcs, 1,400 calls)
---DebugMessage (53.8%, 10 mcs)
---classCivilization::CalcResearchMightRating (21.1%, 3.9 mcs, 461,686 calls)
---classCivilization::CalcTotalSpending (11.4%, 2 mcs, 201,548 calls)
---classCivilization::CalcApproval (6.8%, 1.3 mcs, 136,144 calls)
---classCivilization::CalcRevenueFromTrade (5.5%, 1.0 mcs, 132,695 calls)
--class[AI]Civilization::AIResearchTech (1.3%, 4 mcs)
---DebugMessage (90%, 3.5 mcs)
--classCivilization::AIBookeeping (0.1%, 0.5 mcs)
--classCivilization::CalcShipMaint (67.6%, 0.36 mcs)
--classCivilization::AIChooseGovernment (0.1%, 0.4 mcs, 41,032 calls)
---classCivilization::CalcApproval (92.0%, 0.4 mcs, 687 calls)
----classColony::CalcMorale (92.9%, 0.4 mcs, 15,071 calls)
-----CPlanetSurface::FindFirstImprovementByName
-classCivilization::AIActivateGovernors (12.2%, 70.2 mcs, 2,873 calls)
--classAICivilization_Diplomat::AISetColonySocialProject (70.5%, 49.5 mcs, 1,400 calls)
---CPlanetImpDBRecord::IsAvailable (57.8%, 28.6 mcs, 356,940 calls)
----CPlanetImpDBRecord::GetStringField (24.1%, 8.3 mcs, 481,131 calls)
----classCivilization::IsTechKnown (23.6%, 8.1 mcs, 423,300 calls)
----CPlanetImpDBRecord::GetImprovementType (17.8%, 6.1 mcs, 459,000 calls)
----classCivilization::GetSuperProject (10.8%, 3.7 mcs, 22,131 calls)
----CPlaentImpDBRecord::GetIntField (8.9%, 3.1 mcs, 918,000 calls)
----CPlanetSurface::IsImprovementRestricted (3.8%, 1.3 mcs, 63,546 calls)
----CPlanetImpDBRecord::GetField (3.2%, 1.1 mcs, 84,279 calls)
---classColony::GetNumberOfImpType (13.1%, 6.5 mcs, 42,245 calls)
----CPlanetImprovement::GetIntValue (71%, 5 mcs, ~1,000,000 calls)
-----CCorePropertyHolder::HasProperty (53.3%, 3 mcs)
-----CCorePropertyHolder::GetProperty (37.0%, 2 mcs)
----CPlanetSurface::GetCompletedImprovements (23%, 2.0 mcs, ~43,000 calls)
---classColony::CalcTurnsRequired (8.5%, 4.2 mcs, 42,245 calls)
---CPlanetSurface::CalcFoodProduction (3.8%, 1.9 mcs, 42,647 calls)
----CPlanetSurface::calcOperationalTotal (98.6%, 1.8 mcs, 42,647 calls)
-----CPlanetImprovement::GetIntValue (81.7%, 1.5 mcs, ~380,000 calls)
--classAICivilization_Drengin::AISetcolonySocialProject (21.4%, 15 mcs, 400 calls) [details as above]
--classAICivilization_General::AISetColonyMilitaryProject (7.3%, 5.1 mcs, 1,800 calls)
---classAICivilization_Diplomat::AISetColonyMilitaryProject (87.8%, 4.5 mcs, 2,222 calls)
----classAICivilization_Diplomat::AIIsColonyAdequatelyDefended (46.6%, 2.1 mcs, 2,222 calls)
-----classGalaxy::GetMostPowerfulOtherPlayer (50.0%, 1.0 mcs, 2,222 calls)
-----classCivilization::GetToughestShip (49.5%, 1.0 mcs, 2,222 calls)
----classCivilization::CalcColonyMaint (36.5%, 1.6 mcs, 2,222 calls)
----classColony::CalcColonyMaint (87.7%, 1.5 mcs, ~15,000 calls)
-----CPlanetImprovement::GetMaintenanceCost (58.4%, 0.9 mcs, ~100,000 calls)
-----CPlanetSurface::GetCompletedImprovements (33.2%, 0.5 mcs, ~16,000 calls)
---classAICivilization_Drengin::AISetColonyMilitaryProject (10.5%, 0.5 mcs, 661 calls)
-classGalaxy::NextTurnRequested (7.0%, 40.7 mcs [39 mcs wait], 193 calls) [I don't think this is real performance loss]
--UIHideTurnButton (70.5%, 28.7 mcs, 193 calls)
--UIRefresh (29.5%, 12.0 mcs, 193 calls)
-classCivilization::AIActivateMilitary (3.5%, 20.4 mcs, 2,873 calls)
--classAICivilization_General::AIExecuteShipAssignments (93.0%, 19.0 mcs, 2,672 calls)
---classAICivilization_Diplomat::AIFindConstructorDestination (54%, 10.2 mcs, 16,810 calls)
----classCivilization::AIFindStarbaseThatNeedsUpgrades (98%, 10.4 mcs, 581 calls)
-----classStarbase::GetModulesAvailable (97.2%, 10.0 mcs, ~20,000 calls)
------classCivilization::MeetsTechRequirements (63.8%, 6.5 mcs, ~600,000 calls)
------FindStarbaseModuleByID (30.5%, 2.9 mcs, ~2,400,000 calls)
---classCivilization::AIFindConstructorDestination (11.3%, 2.1 mcs, 2,959 calls)
----classCivilization::AIFindStarbaseThatNeedsUpgrades (98%, 2.0 mcs, 187 calls) [as above]
---classCivilization::AIFindMilitaryDestination (7.0%, 1.3 mcs, 46,005 calls)
----classCivilization::FindNearestRallyPoint (30.9%, 0.4 mcs, 64,750 calls)
---classCivilization::AIFindTransportDestination (5.4%, 1.0 mcs, 4,228 calls)
----classStarShip::FindClosestUndefendedEnemyPlanet (61.2%, 0.6 mcs, 3,695 calls)
----classCivilization::AIHostilesInSector (34.7%, 0.4 mcs, 3,680 calls)
---classCivilization::AIFindScoutDestination (4.9%, 0.9 mcs, 4,169 calls)
--classCivilization::AIBuildFleets (6.1%, 1.2 mcs, 2,873 calls)
---classStarShip::IsStarbase (30.2%, 0.4 mcs, 1,496,390 calls)
-classCivilization::AIActivateForeignPolicy (1.4%, 8.3 mcs, 2,873 calls)
--classCivilization::AIMananangeForeignRelations (88.3%, 7.3 mcs, 1,001 calls)
---classCivilization::AICalculatePrimaryEnemy (29.6%, 2.2 mcs, 1,001 calls)
----classCivilization::AIEvaluateProximityRelations (66.2%, 1.4 mcs, 2,817 calls)
-----classCivilization::GetToughestShip (65.5%)
-----classCivilization::GetRankedPlanet (29.8%)
----classCivilization::AIEvaluateTradeRelations (18.3%, 0.4 mcs, 2,817 calls)
---classCivilization::AICalculateRelationsWith (26.4%, 1.9 mcs, 1,704 calls)
----classCivilization::AIEvaluateProximityRelations (65.0%, 1.3 mcs, 1,790 calls)
---classCivilization::AITradeTechnologies (25.4%, 1.9 mcs, 3,020 calls)
----classCivilization::IsTechKnown (58.6%, 1.1 mcs, 352,445 calls)
----DebugMessage (37.6%, 0.7 mcs, 92 calls)
---classCivilization::AIMonitorGalaxy (11.4%, 0.8 mcs, 3,020 calls)
----classGalaxy::GetMostPowerfulPlayer (96.3%, 0.8 mcs, 1,608 calls)
--classCivilization::AIEvaluateOpponentDefenses (11.7%, 1.0 mcs, 1,001 calls)
---classCivilization::GetToughestShip (49.1%, 0.5 mcs, 780 calls)
---classGalaxy::GetMostPowerfulOtherPlayer (32.9%, 0.3 mcs, 702 calls)
---classCivilization::FidnMostLethalEnemy (16.5%, 0.2 mcs, 925 calls)

"

Just a head's up, Stardock will soon be putting into beta a new Object Desktop program called SoundPackager (or some name that isn't already trademarked <g>). 

It will do for system sounds what IconPackager did for icons.  It will add a host of new sounds for all kinds of events on the system that will help spruce up your environment and give you greater control over it.  We will have a WinCustomize section for users to submit their sound packages as well.

People with fancy new monitors make me mad. That's because I don't have a fancy new monitor.  But Paul Boyer, the UI designer behind GalCiv II has a new one.  See the screenshot there? That's from his monitor.

There's been so much going on.  I didn't get to bed until 5am last night.  The number of touches, tweaks, and enhancements to make the game "fun" has been immense.

For me, I'm been working on balance. Making sure the game is fun and challenging.  This has mainly meant changes to the economics, weapons, costs, and most importantly, the computer AI.

I have to say, GalCiv II's AI is the finest AI that I have ever developed in my career.  I've done a lot of AI over the years and a lot of it has really culminated here.  For me, computer AI is not about making the AI win. That's easy.  What I want is an AI that plays like a human being would - minus the swearing, disconnects, etc.  The AI isn't quite human yet, there's coordination stuff that can be improved.  But it's leaps and bounders better than anything we've done before.  Given that our games are best known for computer AI, I think that says a lot.

The test this week has been what we're calling the Drengin/Human wars.  Duels.  One on One.  Can the AI adapt from dealing with a large galaxy with 9 other players to a tiny galaxy with only one player that wants to kill them?  The answer for the first GalCiv was no.  We never even tested dueling. Not enough time.  This time, the AI not only has multiple C++ personalities (our AIs aren't scripted).  They have multiple random sub-classes to call upon in order to deal with different situations.

The AI in GalCiv scales based on its intelligence.  As you increase the intelligence, new algorithms get unlocked.  At Beginner, the AI is really crippled (but fast).  At "Intelligent" it is playing its best game with the same stuff you've got.  My goal in testing it is to be able to have a hard time beating the AI at Intelligent. 

For players who have Beta 5, the AI metric we had would be what we'd call around a 20%.  That is, around 40% of players would be able to beat the AI at "Intelligent".  When we redid the AI during the holidays, we brought that number down to 20%.  At this point, I think we're getting down to around 5%.  And best of all, as we've improved the AI, we've been able to take away crutches.  In GalCiv I, the AI knew where the good planets were.  In GalCiv II, they don't. They have to scout them just like everyone else.

At first, I smoked the Drengin on intelligent.  Raising the difficulty to "Genius" made is to that I could only win if I got very good starting conditions.  But as I debugged and debugged I was able to make tweaks.  The hardest part is having the AI research the right technologies.  A good human strategy gamer often has a "build order".  But we don't want to get down with "Research this, then this, then this".  That's not AI. That' s a script.  We want the AI to choose good technologies based on analysis of the map, the situation its in, and then conclude that they need X.  So it's easy to tell the AI "Get weapons, then get invasion, then build transport and take human out." That might work for the particular situation, but it's not useful for the thousands of other scenarios out there.  That's why most single player games get boring -- scripted AIs. You figure out their trick eventually and counter them. 

If anyone is interested, I can write up the results of one of my recent tests. 

Get your software, technology, and Windows customization news to the world.  To submit news, go to the forums and then click on "Create News Item" on the left. 

Galactic Civilizations is the only Windows game I've ever worked on that isn't multiplayer. 

The first Windows game I developed, Entrepreneur, had multiplayer. It included a built-in chat area and match-making.  Stellar Frontier also had multiplayer -- up to 64 players on a persistent world.  The Corporate Machine had multiplayer.  The Political Machine had multiplayer.

In short, I've worked on a lot of multiplayer games.  Moreover, I play multiplayer games. As was pointed out on-line, I played Total Annihilation, Starcraft, Warcraft 3, and several other strategy games a great deal on-line in a "Ranked" capacity.  I've tried out nearly every major strategy game from Master of Orion 2 to HOMM 3 to Civ 3:PTW/Civ 4 multiplayer.

From this, I've concluded two things:

1) Some strategy games really benefit from a multiplayer component. Multiplayer extends their "fun" lifespan.

2) Some strategy games don't benefit from multiplayer and the sacrifices made for multiplayer lessened the overall experience.

To people who don't develop games, multiplayer may seem like a simple checkbox feature.  Indeed, many developers I've spoken to feel pressured to put multiplayer in because some reviewers will give the game a lower score if it lacks it despite the fact that for most strategy games, the percentage of players playing on-line is very low.

But multiplayer brings sacrifices that many people may not be aware of. Galactic Civilizations II was developed so that multiplayer could be added later (i.e. it passes messages back and forth).  But the gameplay was not.  We were not willing to sacrifice the single-player experience for multiplayer.

I'm going to give three reasons why multiplayer does not make sense in Galactic Civilizations as part of the base game.

#1 It sacrifices single player features.  Ask any game developer whether they be at Ensemble, Paradox, Firaxis, or Big Huge Games, most people play strategy games by themselves on a single computer.  What % that is depends on the game. But on a TBS game, I would wager that greater than 95% of players never play a single game on-line even if the option is available -- that includes Civilization IV.

But developing multiplayer is incredibly time-consuming and expensive. In our last game, The Political Machine, a full third of the budget was for multiplayer. The game was ideally suited for multiplayer, published by Ubi Soft it would sell a ton of copies.  The game came out and sure enough, only a tiny percent of people played the game multiplayer.  That tiny percent didn't justify the 33% budget dedicated to them.

My favorite game of 2005 was Civilization IV. It has multiplayer in it that is as good if not better than any implementation in the history of turn-based games.  But what was sacrificed in exchange?  There's no campaign.  There's no in-depth scenarios.  No in-depth random events.  You can only trade certain items and techs back and forth no matter what.  Do you think this is a coincidence?  No random civil wars based on certain criteria?  No war-causing assasinations? No crusades? Not even once in a long while?  I suspect that there were a lot of concepts and features that Civ IV would have had if it didn't have multiplayer.

When we were making Galactic Civilizations II, we took a poll on multiplayer. Only a small percentage of GalCiv I players cared about it.  We took additional polls since multiplayer advocates were so vocal.  Same thing.

So instead, Galactic Civilizations II got ship design and a campaign.   I think most people would agree that we could have taken GalCiv I, slapped a 3D engine on it, given it multiplayer and been in good shape.  But can anyone who's played the beta imagine the game without ship design?  And when you play the campaign, I think you'll find that was worth it too.

Moreover, players get a lot more single player experience.  There are rare events that players may only see once in a great while but they're worth it -- a religions war that breaks empires in pieces. New republics formed from remnants of shattered civilizations.  Civil wars. Precursor ships found on worlds. Powerful artifacts that slowly increase the power of a given civilization so that everyone has to team up on them. Terrorists. On and on.  In multiplayer, this would all have to be turned off, but then again, if there had been multiplayer, whey develop any of this at all if it wasn't always going to be used?

Similarly, there's diplomacy.  Last night, I played as the galactic arms dealer.  The Drengin and Torians were at war and I was supplying both sides with ships for money. I then took that money and slowly bought up the worlds of dying civilizations.  That kind of flexibility in diplomacy would be a nightmare in multiplayer, you'd have to put all kinds of restrictions in the name of balance.

#2 The majority ends up subsidizing the minority. Outside some game reviewers and people who have friends who are really into this stuff, most people don't know other people on-line to play these games with.  And let's face it, playing a turn-based strategy game with strangers is an excercise in frustration (I am not sure I've ever actually managed to complete a TBS on-line without the other player either dropping or quitting prematurely).

Galactic Civilizations II is $40. Not $50.  That $10 may not seem like a lot to some people but to many gamers it makes a difference.  Check out the prices on the latest multiplayer strategy games -- they're $49.99.  Part of that price is to subsidize the multiplayer component that only a tiny percentage of users will play.

If there's sufficient demand for multiplayer, we'll do it -- but as an expansion.  Those who want multiplayer can then buy it and those who don't aren't forced to pay for it.  And everyone wins because they saved $10 in the first place.

Because of the Metaverse, GalCiv II already has multiplayer plumbing.  We even have a multiplayer design.  But it'll cost money and time to implement it. So if there's demand, we'll do it. But it has to be demand in raw numbers, not just vocalness of the people who want it.

#3 It would have changed the design priorities.

When you design from the start to be a good multiplayer experience you have to make sure the game is streamlined -- particular the interface.  So things that might slow the multplayer pacing tend to come out.

Galactic Civilizations has lots of mini-cut scenes in it.  Things to help the player enjoy and savor the civilization they've created. The technology tree is huge and designed to linger through and pick just the right one.  The ship design is full of extras that are there so that players can make cool looking ships.  The battle screen was implemented to be not just functional but fun to watch. The planetary details screens include quotes from random citizens and there's flavor text all over.  Each civilization has its own vocabulary based on who it is talking to (i.e. how a Torian talks to a Terran is different than how they would talk to a Drengin).

But a good multiplayer game has to be far more streamlined. You don't want to have core features that encourage users to do anything but move their units and make their decisions efficiently.  Take a look carefully at any decent multiplayer games recently and notice how efficient they are.  Efficient is great in a multiplayer game.  But in a single player game, there is something to be said I think for inefficiency -- for fluff.

And because we designed the game from the ground up to be a single player experience, Galactic Civilizations II has a LOT of fluff:

The screenshot below, I designed all the ships in this particular game. Every new game I make new ships. Why? Because it's fun. It's not efficient though from a sheer "get to the next turn quickly" point of view:

And what's the point of sitting there watching your ships battle it out?

In fact, there's a ton of things that involve reading quite a bit of text. There's a lot of customization within the game that has no "point" other than to let players indulge in the civilization they've created.

It's not that a game with multiplayer can't have these things. Civilization IV has the Civpedia for instance.  But the tendancy in a good multiplayer game is to move the text and other stuff out of the way during gameplay and out to a place where it's looked at at ones leisure.  And that's a good idea in a multiplayer designed game.

But these days, nearly all games are designed with multiplayer in mind.  And for people who have no shortage of choices on multiplayer, it's nice to have a good old fashioned single player experience where you can sit down and indulge yourself to the full experience. 

I remember a game called Master of Magic back in the early 90s. It was a great game. But it was only a great game because it was single-player.  The things that made it really neat would have been a disaster in multiplayer. 

And that's really the point -- multiplayer is a feature. Sometimes it makes sense and sometimes it doesn't.  In some RPGs sometimes they're better being single player (Knights of the Old Republic) and sometimes they're better being multiplayer (Neverwinter Nights).

For those who think multiplayer is a must in strategy games I hope this post has at least made the case that maybe sometimes multiplayer doesn't belong in the base game. 

We know how to make multiplayer games. We had the budget to make GalCiv II a multiplayer game if we wanted. We like multiplayer games.  But we felt that GalCiv II would be a better game if multiplayer was not part of the base game. 

 

For those of you who are interested in what we're currently working on, check out the latest dev journal.

So much of a strategy game, particularly a single player one comes down to little things like pacing. If things cost too much, it's too slow and boring.  If things cost too little, then things move too fast and nothing has value.

That also holds true to the difficulty levels. How hard should it be? How aggressive should the computer players be?  For me, what I've tried to do is make it so that I can beat the game at "Intelligent" (for the AI players) but just barely. At higher levels it becomes very difficult to win unless I have good starting conditions.  Since I've been playing this game in one form or other for millions of years now then I hopefully represent a fairly expert player at the game.

At lower levels, the AI shouldn't be very challenging to experts but to new users present a challenge still.

In game design one of the goals is to provide X minutes of raw fun. Stuff that players find extremely fun.  Many fun things may only last 2 seconds, others 2 minutes.  There is some metric that says that a good game should provide X total minutes of fun.  Designing ships we hope is one of those areas of fun in the game.  I have to admit, I really enjoy designing ships, far more than I ever thought I would. 

When I'm feeling funny I'll name the ship classes after my wife and kids.  In a test game today, I played with my two sons Alex and Ryan (and in fact, my son Alex Wardell was the actual "leader" of my civilization).  Being able to name everything in the game -- your civ, your leader your ship classes your individual ships your planets your starbases, etc. I think helps add a little extra fun.  I know my kids kick a kick out of some alien saying "I don't think so, Alex Wardell! We're not afraid of you!". 

So in this screenshot, these ships are named after my wife (Debbie) and two sons.  The detail you can go into with your ship designs is one of those "fun" things we hope people really get a kick out of.  The size of ship makes no difference.

There is no limit to the # of little jewelry pieces you can add. Obviously this would never work in an RTS since time matters. But in turn-based you have all the time in the world.  Where it would work in RTS is the save function -- if you could do it off-line and bring in your units that would be pretty cool in RTS.  You can save your designs here too and use them next game.

Another area that can be problematic is end-game build up.  That is, where you've gotten to the end of the tech tree or at least pretty far and things no longer make any sense numerically.

This is something we've been spending a lot of time on this month.  You don't want players to eventually have infinite money or infinite resources, it takes away the challenge.

In this screenshot, this is my main research planet.  I have my spending 33/33/34 and as you can see I do 101 research per turn here.  You can also see a bug (where we're missing a thumbnail for one of the improvements.).

At the same time, with my tax rate at 33%, I about break even.  But my approval rating is 85%.  The whole approval rate thing kind of bugs me because I feel like population and approval should do more.  For example, I'd like to see that if my approval rating is >67% (for instance) that I get some sort of "bonus".  I don't know what, maybe my influence on that planet is increased by 15% or something.  Or if my population is greater than 10B that I start to get production bonuses.  These things won't go into the final game, but it leaves us room to think about it for an update after release. 

In this game I wiped out the AI pretty good but the difficulty was only at "Normal".  The AI is so far behind that my fighter level ships completely outmatch their cruisers.

Boom.

Their battleships are a different matter.

For me, one of the "fun" things is that I can pause the battle and rotate around. It's like doing the Matrix type thing when I spin the thing around to do the screenshot. It's totally cool freezing a ship at its moment of death.

Two ships named after my sons.

Another area of tweaking comes in the form of the ethical alignment and government.  How much should Federation give you? What goodies should each ethical alignment give you?

In the current build, you get free maint on the initial colony for your top 5 worlds. But we've been wimping out the cost in recent builds because new users tend to run out of money.  So we may have to bump up the power of this.

Another area of challenge has to do with the starbases, especially since we ended up letting their powers "stack".  This opens the door to abuse but we wanted to let players go nuts with them if they can.  I built up my influence abilities as far as I could and have influencer starbases that improve cultural influence by 322%.

My view is that if the AI lets you get away with it, then that's the AI's fault, not the player's.

But in many ways, nothing is taking up as much time as getting the diplomacy just right.  When should the AI ask for tribute? When and how often should the AI give in to tribute to you? Moreover, we want different civilizations to have very different reactions and text.

They rejected my tribute demand and they are the Altarians and they don't like me because I was recently at war with them.

By contrast:

The Arceans like me and I am more powerful than they are and I've never asked for anything before.

Or how about making the distinction between weapon trades and non-weapons:

If you ask for tribute too often:

The number of different scenarios is pretty big.  And you want to make sure it's realistic and reasonable.  What would a decent player on-line do?

There's also sensitivity to what the player is doing.  Should the AI react to when the player does an obvious build up? If so, at what difficulty levels? Right now, I have it so that only higher difficulty levels do detailed analysis of human strategies and react accordingly (it's fairly CPU intensive).  But on the other hand, will we end up with some review in some magazine where the reviewer played it at easy level and walks over the AI and complains that the AI had bad strategy? This actually happened in a couple of reviews of The Political Machine.  Some reviewers think that difficulty levels just mean how much free money we give to the AI.  In GalCiv, the difficulty levels determine what algorithms get used.

A decent AI would recognize this for what it is and react.

At lower difficulty levels if the player is being really blatant the AI will say "We know what you're doing".  But at higher levels, it will rarely provide clues or be vauge.

BTW, doesn't that invasion ship look cool?

So that's what this weekend was spent doing. Balancing balancing balancing.  Next week we have to finish up the Dread Lords campaign...

 

Gameshark has one of the best previews of Galactic Civilizations II I've seen yet.

It goes into extensive detail on the new game elements and also talks about an interesting issue with regards to expansion packs.

One thing that many gamers new to Stardock's software may not be aware of is that Stardock doesn't just release software and call it a day with a couple of patches.  They get updated (for free) long after release.  These updates aren't just bug fixes but add new elements to the game.

Galactic Civilizations I, for instance, received countless updates after release that added things like Rally-Points, the Galactic Encyclopedia, and a ton of tweaks and enhancements that helped keep the game interesting long-after it was originally released.  Of course, the downside is that sometimes expansion packs don't have quite the same impact since players were getting so many free updates.

Still, in the long haul, we think that it benefits gamers and Stardock to release free updates that add new functionality to the game. 

For example, Stardock plans to once again release a free BonusPak that will be available for download upon the game's release.  We did this with Galactic Civilizations I and it did very well.  The BonusPak is essentially a series of goodies that beta testers came up with during development that weren't part of the game design but that we thought would be neat things to throw in later.

Because of the 3D engine, there's more opportunity to do more updates too.  For example, in most 3D games, the textures are bitmapped based.  As a result, if you go back and play one of those 3D games a couple years later, the graphcs look muddy compared to the "latest" 3D games.  In Galactic Civilizations II, our textures are vector-based which means that we will be able to keep updating them easily to higher and higher resolution. (see this screenshot as an example).

The idea being that even years from now, Galactic Civilizations II should continue to look as good as the newest games.  The 3D engine also doesn't limit the # of polygons used in a model. Modders can make .X models with as many polygons as they want.  And since the game is resolution independent (we've played it at 1920x1200 for instance) it isn't tied down by today's display limitations.

Now someone might make the argument "Well aren't you hurting sales of a future GalCiv? Why buy GalCiv 10 when GalCiv II looks fine?" but that brings up one of the key parts of Gameshark's previews: NEW VERSIONS SHOULD BE NEW VERSIONS.  A sequel to a game should be a real sequel, not a glorified expansion.  Gameplay, not graphics, should be what drives a new version of a game.

Read the whole article, it's a great read and highly informative.

The hours are starting to get to us. 

Stardock, as a company, is 7/8's a Windows desktop software developer and 1/8th game developer/publisher. At least, that's how revenue in 4th quarter came out to be.  It's not that the games do so badly but rather our other products are so popular.

But right now, nearly everyone in the office is on Galactic Civilizations II.  One thing that really hurt us is turn-over.  In all the years we've done games, we've never had anyone leave a game project before it was done.  This time, we had two people leave.  They left on good terms but the fact is, despite what Game Developer Magazine occasionally writes, game development doesn't pay nearly as well as regular software development and in both cases, they left game development for "Regular" development jobs.  One was an engine developer and one was a key animator.  And on a project like this, it's hard to absorb those kinds of losses.

And so we do our best with what we've got.  We've put more people on the project from other areas so that the core team can focus on the things that only they can do and the parts that others can do can be done by newcomers to the project.

It's times like these that I can understand why many developers find console development so appealing.  Why does configuration X have giant stars? Dunno.  Why are the textures getting funny on configuration Y? Dunno.  Why are the overlays disappearing on configuration Z? I dunno.  I can definitely see why Civilization 4 went Windows XP-only.  But we want to make sure Windows 98, ME, and 2000 gamers can play our game.  Luckily, our non-game testing center is now on GalCiv II so all those "I dunnos" are quickly becoming knowns.  But time grows short and the hours grow long.

We also have a lot of "It would be nice to have".  Soren from Firaxis (Designer of Civilization IV) has been on the beta and has been providing some really useful suggestions.  For instance, now when there are no more units to move, the turn button glows green a bit to indicate to the player that it's okay to hit the turn button.  A number of other tweaks and changes have been made based on his feedback.  Since Civilization IV came out, we've been trying to avoid "stealing" too many ideas from that game.  But I love Civ 4 so much. It's how game design should be done.  GalCiv II is very different in most ways but there are enough 4X user interface similarities that there are some nice touches such as the visual turn button cue that we can add in.

The cut-scenes are most worrisome.  We have a lot of video in this game -- far more than any other 4X strategy game probably ever.  The dynamic campaign is meant to convey the story of the Dread Lords and so we want to present that story.  The screenshot you see on the left is a single frame from a cut-scene from the campaign.

The team has been putting in 12 to 14 hour days now since we got back from Christmas break.  We've got another two weeks to go before we have to wrap it up and send it to manufacturing so that we're at stores on February 25th.

At least I can honestly say that the game is turning out very well.  Everyone we have had play these last few builds have remarked at how fun it is and how much better than the first one it is. 

A few stats:

Music: GalCiv I had about 10 minutes of music.  GalCiv II have over an hour of music.  Video: GalCiv I had a handful of cutscenes of various quality.  GalCiv II has a LOT of cut-scenes.  And I don't think the gameplay is even remotely comparable now.  Going back and playing GalCiv I is painful -- no zoom out, weird interface, limited strategic options, ugly graphics, inconsistent pacing.  GalCiv I was a good game -- for an indie developer.  GalCiv II takes it to a whole new level (I think anyway).