We’ve internally staged the new proxy server update for Demigod.  It’s going through QA right now but it’s looking like it won’t pass QA tonight due to just sheer lack of time.  Last week we had a lot of people working late into the night in QA to get builds but I’m trying to avoid having people working crazy hours now that the MP works for most people pretty flawlessly.

Here are the things that the update is supposed to do:

1. If a user failed direct connect and then fails to connect via NAT it will send them over to a proxy server that will host the game for them. In theory, it should be pretty bullet proof but you’d be surprised how hard it is to test since, not surprisingly, “direct connect and NAT work great here!”.
2. Supposedly the favor points and favor items work properly. But I’ve heard this before so I’ll believe it when I see it.
3. Supposedly, and I haven’t verified this personally yet, the /noAI option is in which will prevent AI players from being in pantheon or skirmish games.
4. When looking to join pantheon or skirmish games, Demigod will tell you once it starts polling users, how many people are looking for games. By doing so, we can increase the match making max time from 2 minutes to 5 minutes so that players can get better quality matches.

But it’s not looking like it’ll go up tonight for the public.

Some comments from “What the hell happened?”

I get around the net too and the article I wrote that outlined specifically what happened with Demigod got out there. There have been lots of interesting comments on the net and I thought I’d take this opportunity to respond to some of them.

How did they not perform any simulations or trial runs with players connecting out over the internet in order to stress test this? If they did, how could their testing not notice that routers in general don't like large numbers of listening sockets on it's clients? This is a serious amateur mistake.

Very few routers have a problem with this.  The problem was not the # of sockets but the precise timing needed in handing off the sockets. We didn’t realize until 2 weeks in that the scalability problem in Demigod’s MP matchmaking was due to the third party network library we used having up to 30 to 40 second delays once there were thousands of players on.  During the beta, users ran into this problem too and the solution was to add more servers and increase the timing threshold from <1 second (in beta 2) to around 5 seconds for beta 3f and release).  But the delays we saw weren’t 3 or 4 seconds but 30, 40, 60 seconds. It increased exponentially not linearly.

Minus an open public MP beta, this would never have been found.  In hindsight, that’s what we should have done, clearly.  But neither Stardock or Gas Powered Games, both with considerable experience in this area, foresaw a problem. Clearly we blew it.  The ultimate answer is that neither of us have ever released a game that was primarily played multiplayer and so our assumptions on what to expect were simply wrong.

I thought it was so distasteful how this team tried to blame Gamestop for releasing the game early... but then the big problem with the game really ended up being shitty coding on their part. I kind of think Stardock is full of whiny bitches now.

The Gamestop early release didn’t help matters but it did expose that the MP system didn’t scale well.  And while Stardock takes full responsibility for the decisions it made. The issues involved don’t involve Stardock “code”.  We are the publisher of the game and licensed third-party tech.  The third party tech works very well under normal circumstances but simply had not been used in a P2P game like this.  As the publisher, we made the final call on what tech was used for this so it’s ultimately our fault.

Soooooooooooo they are going to do it right this time with their Single Player [Elemental] game, but not fix Demigod so it's server hosted?

We can only do things “right” on games we code.  Moreover, Elemental is a turn-based game and thus a client/server game is less sensitive to latency than a RTS like Demigod.  For the record, I support GPG’s MP design given what was known at the time.  Every RTS I know of, other than Sins of a Solar Empire, is P2P.

What is going on these days with company reps feeling obligated to explain the intricate deatils of the inner-workings of their, engine, business strategies, technical issues.

Here's news to anyone in the future who feels compelled to offer an explanation for whatever bad befalls their company.

No one cares why. Notice in sports how there isn't a "Loseres circle" where the losers get to rationalize and explain why they lost. Sadly reporters force the loser coaches to speak up these days but in general we do not celebrate the loser.

So buck up kiddies. This is a tough inductry, not for the faint of heart. When you fail you (should) fail alone. Don't bring the rest of us down with your miserable stories of failure.

This comment is particularly hard to address.  Demigod is certainly not a “failure” by any standard.  While I might focus on the negative (clearly, I’m a glass is half empty kind of guy), Demgod’s reviews put it well in the upper list of games released this year (particularly if you remove the first week reviews).  Its sales are quite strong, particularly digitally.  That doesn’t mean the multiplayer launch wasn’t a total cluster. It was. But I don’t think that makes Demigod anything remotely approaching a failure. In the long-run the game’s constant updating and growing community will ensure it has a strong long-term future.

That said, I also believe strongly that when a product or service has problems in some area that the company who produces (or publishes) it should be very up front with their customers.  Maybe I’m wrong and people are sick of these journal entries but that’s my view on it.

We were able to get the updated network update out yesterday and we turned the Pantheon and Skirmish to try to do 3 on 3 matches. Since so many people seem anti-bot, we are looking for ways to make that happen but the problem is, if people insist on human only then we need to provide more feedback to the user. We already get complaints from people saying it takes “too long” to get into a skirmish game (a whopping 2 minutes).  So for people who insist on human only, we need ways to make them aware that the system is working but it might take several minutes.

Skirmish and Pantheon games aren’t just about finding some random group to have you play against. It uses your ELO (experience) rating to help match along with your location to try to put together the best match possible.

One of our frustrations (and this is one of the reasons I’m not a fan of MP games in the first place) is how inconsiderate some people are.  We spent a lot of time trying to figure out why people were getting “disconnected” right when a game starts.  Well, it turns out, from looking at the logs, is that some people, having only 2 on 2 or 3 vs. 3 where an AI player had to be substituted to fill in the gap that some people just quit right at the start.  I understand the rationale but it’s still objectifying other people’s time. I won’t debate people on the issue as I think it’s a matter of agreeing to disagree.

But bottom line is that if people want to have an idealized match, it’s likely going to take more than 2 minutes and the only way to make it take more than 2 minutes is to provide more feedback so that’s what the team is working on today.

Proxy tomorrow

So besides having more feedback telling you how many people are looking for games which lets us increase the time to find an ideal match, we also plan to have tomorrow’s update have the proxy server. This should eliminate the last problems for connectivity except for the case where people have multiple IP addresses which is still being worked on.

More info please

Some people have said “I used to be able to connect fine, now I can’t.”  We really  need data from those people in the form of their impulsereactor.log files (located in my documents\my games\gpg\demigod) which you can send to [email protected] with (ideally) a description of what happened and whether you used to be able to connect so we can see.  At this point, people shouldn’t be having any problems connecting other than those with router issues.

The Documentary

Once we concluded that we were going to have to write our own multiplayer system for Demigod and that it would have to be done not in a matter of months but a matter of days or a week or two we knew that crazy hours were going to be involved. So we video’d it.  We hope to release in the next week or so a documentary video so you can see the innards of Stardock and what was involved.

Here are some images from it:

Marketing meeting

Management meeting in late April when it was decided to assign the Impulse team to develop a new system.

 
My daughter Ashley visits me at work

Our chief architect, Jeff Bargmann and his Impulse team merges with the Elemental development leads to create a new network system.  The application team had already gotten somewhat involved by creating the “Impulse Overlay” that allowed us to inject new screens into Demigod.

Elemental team can’t resist but show off what they’re doing.

Found a baby turtle while taking a walk outside

As Elemental players know, I had been on Elemental since early this year. Then…Demigod was released and the online experience for that game has been a total cluster.  So I had to “assign myself” to Demigod to figure out what the deal was and make sure it got fixed.

In my view, too many moving parts. I’m not a huge fan of multiplayer-centric games in the first place. I’m particularly not a fan of peer to peer multiplayer-centric games on the PC because on the PC, unlike on the console, the developer is stuck writing this stuff – or in Demigod’s case, the publisher had to go out and license stuff which turned out to be a disaster. 

But now my time on Demigod is starting to come to an end – at least in terms of it dominating my life.  I’m looking forward to working on a game that we’re both developing and publishing so I don’t have to take a beating for things I can’t really do anything about.

Ramifications

I had to assign both Cari and Jesse (our two top game developers) to Demigod to largely rewrite the multiplayer system. This will have an impact on Elemental’s schedule. Sorry. People paid good money for Demigod and you know me, I am not about assigning blame or caring whose fault something is so my attitude is that the problem had to be solved quickly.

They’re still on Demigod this week working with developer Gas Powered Games on a slew of new APIs that will let the game be expanded well into the future. Then they can get back to Elemental next week.

Demigod’s impact on Elemental

The good news is that because we ended up having to develop our own multiplayer module to help with Demigod, we can use that on Elemental.  Now, to be crystal clear: Elemental is a single player centric game. It will have multiplayer but to be honest, I’m not willing to sacrifice a single feature of the game for multiplayer.  So if multiplayer is your main thing, you might as well stay away from Elemental.

That said, here are a few things that multiplayer in Elemental will have:

1. Multiplayer games will be hosted by us. Period. No peer to peer. Not even hosted on the user’s box. Our servers. No ports, no proxies, nothing. We’re hosting it.

2. I’m killing off the bots concept. People hate them. I thought they were cool but they’re too much work only to be hated. So there won’t be artificial players.

3. We will support empires (clans), scheduled games, and group join from the start since those features will get added (by us) to Demigod.

Fewer moving parts

We are revisiting the way the economy works in Elemental to simplify it.  There’s been some positive developments that I can’t talk about yet. The short version though is there will be likely be a lot lot lot more story to Elemental than anything we’ve done before.  Each sand box game should feel like an epic story if we do it right. We’ll see.

Release dates

I can say that Beta 0 is not going to happen in June now.  We might be able to do an Alpha then but that will only go out to a very tiny number of people (maybe 100 tops).  I’m reserving 10 places and the rest will come from the pre-order pool as a lottery with points given to those who have GalCiv II + Sins + Demigod.  Beta 0 would likely be July at this point but I’ll know more at the end of the week.

I’ll be writing a lot more on this particular issue in the coming weeks as I’ve had more time to review internal reports.

For those of you just joining us let me bring you up to speed.

Our story so far…

Demigod, a high profile, AAA action-strategy-role playing game was released on April 14th. Well, it was supposed to be released on April 14th but actually got released at Gamestop stores early due to a…miscommunication between their corporate HQ and their brick and mortar outlets.  This wouldn’t normally have been that big of a deal except this happened to be over Easter weekend and the release servers for the game weren’t yet up. Moreover, it also caused the “warez” version (i.e. there’s no copy protection on the game so the warez version meant someone bravely zipping it up and putting it up on a torrent) resulting in over 100,000 people using it – at once – before we were even back from Easter break. Suffice to say, it wasn’t a pretty picture.

For the first few days, we struggled to migrate people to a different set of servers that only legitimate users had access to. This took about 48 hours.  But during this brief window, the game was basically unplayable because you couldn’t even get online – at all.  We got whacked with some pretty negative first week reviews not surprisingly.

But our woes weren’t over yet. It became pretty clear that the NAT servers (the servers that negotiate the connection between player A and player couldn’t handle the # of users on the game resulting in a horrible online experience.  As other people have pointed out, this sort of thing isn’t unique to Demigod (i.e. plenty of other games have had rough online launches) but the big difference is that those other games had a lot more single player content whereas Demigod relies more on its multiplayer experience than most games so it was a much bigger problem.

Like most games, Demigod uses a lot of licensed code. Demigod’s awesome 3D models are powered party by Granny 3D. The videos in the game are powered by Bink. The sound is powered by Fmod. And the network connectivity was powered by Raknet.  These are all very good libraries and used by major publishers. 

But Demigod’s network requirements are somewhat unusual and demanding. First, Demigod is peer-to-peer and not client server. Everyone connects to everyone. Second, the number of people playing is unusual. Yes, some people do play 4 on 4 games of Supreme Commander or Company of Heroes but typically they’re 1 on 1 or 2 on 2. The more connections, the more complex.

The result was that it was a nightmare to get games going online.

The problems

Demigod’s connectivity problems have basically boiled down to 1 bad design decision and 1 architectural limitation.  The bad design decision was made in December of 2008 when it was decided to have the network library hand off sockets to Demigod proper.  In most games, the connection between players is handled purely by one source. For instance, in Supreme Commander, GPGNet handled the entire connection.

So in Demigod, on launch day, Alice would host a game. Tom would be connected to Alice by the network library and then that socket would be handed to Demigod.  Then, Alice and Tom would open a new socket to listen for more players to join in.  As a result, a user might end up using a half dozen ports and sockets which some routers didn’t like and it just made things incredibly complex to connect people and put a lot of strain on the servers to manage all those connections.

Now, the architectural limitation came from the way the network library’s database handled things.  We still don’t have a clear idea on why it was so limited but this was the overwhelming problem that only got resolved late last week. Here’s how it works:

Alice hosts a game. In doing so, she sends a message to the NAT server (as well as our servers). Tom wants to join so Tom clicks join and it tells the NAT server to begin connecting them.  But, it turned out that a relatively small number of people online at once would quickly result in a huge delay in messages being sent back and forth. For instance, when Tom clicks join it sends a message to the server to tell it to start connecting Tom and Alice.  But Alice might not get that message for 30 or 40 seconds. That means, for that entire time, Tom and Alice are “attempting to connect” but haven’t even really started because Alice hasn’t even gotten the message.  As more people tried to join the game, that delay could get worse and worse. If someone left the game, it could take that amount of time for the server to realize that player had left (meanwhile it was trying to connect them).

Why did it take 3 weeks to fix?

It took us a solid week to realize that this was the problem because we assumed the issue was compatibility with routers or ISPs. We worked with people to set up their port forwarding, etc.  Eventually we started looking at the time stamps but even then, when we saw the 30 to 40 second delays there was an assumption that that difference in time was only because the servers and our machines weren’t time synced.  On one of the late nights here, I began insisting on having the machines all synced via one of the atomic clock services and then we became horrified at what we found: A 30 to 40 second delay between the time the server processed a message and actually sending a response.  That time delay during peek hours could get over a minute. 

Luckily, the developer of Raknet was here last week and was able to help us dig into the bowels of the code and we were able to find ways to mitigate this problem.  Much of this is simply because Demigod is peer to peer and so it means there’s a lot of messages being sent back and forth.  A lot of our work has been in reducing the # of messages needed to connect people and writing a new network layer that was designed for P2P specifically.

PLUS, we had to write something that network gurus can tell you is pretty damn cool: Proxy sockets. So that we wouldn’t have to hand off sockets, the Impulse team was recruited onto the project and they came up with a way to not have to hand over sockets to Demigod without having to change Demigod.

Now, with that much new code, it wasn’t perfect. Friday’s update resolved things for most people but it also created problems for some people who hadn’t problems before and it caused some grief for people who were trying to play Pantheon games. Plus, you still have people who simply have difficult Internet connectivity that still have to be resolved but now you’re down to like 5% of the base.

Today and later

Right now, we’re staging an update that addresses what was mentioned above. It’ll either get released this evening or tomorrow.

On Wednesday, the plan (not promise) is to release the update that supports Proxies.  That update will also support some new command line switches:

/proxyonly (if you know you won’t be able to connect normally you can just have us host your traffic)

/noAI (will tell Pantheon and Skirmish to not allow AI players in even if you have to wait longer) – this may not make it into Wednesday but I’m hoping.

/localproxy (if you have a monster machine with a really really good network connection you can let people who can’t connect route through you which lowers lag – only do this if you have a very fast UP speed).

What else?

There are still some other things on our short list of things we need addressed and developer Gas Powered Games is working on them.

These include:

1. When people quit (for whatever reason) it looks like they’re disconnecting (complete with annoying 30 second disconnect dialog)
2. We need a concede option.
3. We need favor points and favor items properly stored.
4. We need to make sure that the stats recorded are more accurate (it’s been flakey).
5. After using an ability, some Demigods stop moving until manually given another command.

GPG is working on those 4 items and more but I don’t have an ETA yet but I’m pushing for this week.

What about the future? An FAQ

Q: What is publisher Stardock planning to do for Demigod players?

A: The plan is to send out an email this week to users who purchased Demigod prior to today with a coupon for 50% off of Demigod that they can give to their friends.   In addition, next week we will begin sending coupons for other things on Impulse to active Demigod players to help ensure a vibrant multiplayer community. We plan to keep doing that periodically.

Q: What about my pantheon stats?

A: As soon as we feel comfortable with the robustness of the stats, we are going to launch the second Epoch which will see the stats reset (the previous stats will be archived under the 1st Epoch).

As those who visit the Pantheon area know, we are taking this stuff pretty seriously.  We are looking at this from the long-term.

Q: When will more Demigods be released?

A: 2 Demigods are in development. I will have a better idea later this week when I meet with GPG on that.

Q: What about modding?

A: We have discussed this issue with GPG and I can’t make promises on this but I can tell you that what we would like to allow is for people to mod AI players (LUA) and then submit them online and let them compete to see how they do. This would encourage good AI modding.  We would also like to support mod support in Demigod proper where the Pantheon would support players downloading officially “blessed” mods and be able to play them in game.  I don’t have any ETA on this yet.

Q: What about a demo?

A: We’d probably already have a demo out if we hadn’t been messing with this.  But yes, there will be a demo.  In all likelyhood, it will probably be a multiplayer only demo since we want to reassure people when the demo comes out that connectivity is totally nailed and bullet proof.  This is different than our original plan which would have been a single player only demo with 2 demigods and 1 map. So we’re still thinking about how to do this in a way that has the most benefit to us and potential customers.

Q: How are Demigod’s sales?

A: They’re considerably better than Galactic Civilizations II’s but slightly less than Sins of a Solar Empire at the same time.  However, there are a lot of variables. Besides the online MP debacle, you have an April release versus a February release, you have a MP-centric game versus a SP-centric game (The most players I’ve ever seen online with Sins is around 500 whereas right now, mid afternoon on a Monday there’s 2065 players playing online). You also have the review difference: Sins has a review average of 88. Demigod’s average is 78 (still pretty good). If you took out the first week reviews the average jumps to 84.  Overall though, it looks like Demigod will hit 100,000 units sold before the official European release.

Q: What has Stardock learned from the Demigod release?

A: We’ve learned that you can’t treat networking as just another thing to plug in like you would a sound library or even a 3D engine. It’s a whole different animal.  With Elemental (our next game), it’s single-player focused but its MP will be server based (and I mean we literally host the game). After Demigod, I don’t ever want to hear the words “socket” or “port” again.

Q: What is your honest outlook for Demigod?

A: You know the expected marketing answer.  But my view is, the difficult launch definitely hurt the game. There’s no way around it.  Besides upsetting a lot of people anxious for a good multiplayer game, you also have the fact that those early negative reviews are going to linger.  1UP promised to re-review the game when this mess is straightened out and so I’ll be talking to them this week about that.  But still, Gamespot (6.5) and IGN (7.5) are going to linger. That’s the breaks.  One could argue we released a game that wasn’t done (we thought it was done) and that’s what you get.

Now, that said, I do think long-term the game is in good shape.  First, we will be putting a lot of effort to build the online community.  Scheduled games, strong team and clan support, pro tournaments, matchmaker filtering (let people filter out people they don’t want to play) are all coming sooner rather than later.  Those things will help immensely. 

A lot of Demigod’s long-term success depends, in my opinion, on whether we’re able to “perfect” Demigod’s online experience before a viable alternative shows up.  Right now, Demigod has the luxury of being one of the few modern RTS games that is so well suited to playing online.  It also lucked out in that Atari doesn’t release it internationally until later this month which means a huge influx of new players.

The last few weeks have been pretty intense. Demigod has been unlike anything we’ve released before. The good news is that its sales have been really good and despite that first week which saw our network brought to its knees, the game’s average review scores have been 4 out of 5 (well over 4 out of 5 after the first week).

Our chief technology architect, Jeff Bargmann, who I brought onto Demigod a couple of weeks ago when it became clear that we were going to need to develop our own network solution for the game is clearly obsessed based on the 4am emails he sent me over the weekend.  Jeff is also the lead developer on Impulse, ObjectDock, ObjectBar, Fences, and numerous other projects. When I absolutely positively must have something amazing done right and quickly, he is my go-to guy.  So this weekend he’s been working on various proxy solutions.  Cari, our lead game developer also found ways to solve some of the Pantheon and Skirmish issues that were reported over the weekend.

I don’t want to make any promises but given the success of Friday’s update and the likelyhood that the proxies will go online this week, we should be nearing the point of diminishing returns for getting people connected on the multiplayer match making side.  I also know that Gas Powered Games is looking seriously at in-game connectivity.

Personally, I think the long-term solution is for us just to host the games entirely.  That’s what we’re doing in Elemental for its multiplayer – our servers host the entire thing from the start. 

But with the connectivity issues in MP likely to substantially diminish this week, it’s time to start thinking about what we should do next.

Demigod v1.1

So on the horizon there’s Demigod v1.1.  Now, question is, what should be in it? Now, these aren’t expansion packs but I can think of a few things on my personal list that may or may not agree with what you guys want so let’s talk about that.

Here’s my personal preference list for v1.1 in brief:

1. Favor points increased by a factor of 10.
2. Option to Concede. Would let players quit a game they feel they’re losing. Their Demigod is removed. If all players from that team concede, the game is over.
3. Smart Quitting. If someone exits within the first 3 minutes, there’s no harm, no foul. It counts as a disconnect but not a loss. Sometimes there’s lag or someone has a slow machine or whatever.
4. Random Demigod.  Players can choose a Random Demigod to play.
5. Updated AI. Smarter computer AI.
6. Various balance tweaking.

Now, this doesn’t include DLC content such as Clan Wars, Scheduled Games, Group Join, etc. This is just v1.1.  But it does give you an idea of some of the things we’re thinking of. 

What would you like to see in terms of new features?

One of the things that I have felt really bad about is that players are made to feel like they can't concede a game lest they be called "rage quitters".

What GPG and us are talking about is adding a concede option so that people can exit the game and help move the game along. 

My proposal is that when someone concedes, they leave the game, get their favor points in progress and that Demigod is GONE (not replaced by AI).

What do you guys think?

Ok guys. Let us have it.  We’ve largely rewritten the way users get connected for custom games.

So…

Is it better?

Let us know.

Update:

We are asking about the initial connectivity - getting connected to other players.  Not your experiences with other people once in the game. That is a seperate discussion.

I remember when Impulse first came out, some people didn’t like it.  But you know one fringe benefit of Impulse? Very very easy to release updates to lots of people in short amounts of time.

Anyway…

We released an update today and we’re extremely pleased with how it’s gone. We can actually go into the chat area without getting flamed! 

In early blogs, I’ve mentioned that developer Gas Powered Games and publisher Stardock both have multiple sub-teams working on various things in Demigod. 

Or more specifically, this past Monday, I mentioned that there would be a Monday update and a Thursday update.  Well, the Monday update was something of a disaster.  Yea, it worked better than what was up but we were extremely disappointed. Those of you who know me know I was extremely frustrated online about it in the chat area because I had such high hopes for it.  So today we released the update that fully implemented the changes needed to make Monday’s update be what it was supposed to be and that’s why we’re in such good moods.

Anyway, I also had said there would be a Thursday update that would have a lot of good stuff in it.  Well, it’s Thursday and the Impulse team now has their update in.  I think I referred to it as the “end all, be all” update to some people online.  Well, we’re staging it right now.

After this update, the only thing we’ll really have left to do is to turn on the proxy servers for those people who absolutely, positively cannot connect to people along with making the connectivity more robust for people with multiple network cards in their computer (you can do yourself a favor if you get stuck and if you’re like me with 3 network cards and wireless, turn some of them off when playing online until we come up with a good way to bind it).  But those solutions are much easier than what the team has had to do which is major surgery on the network libraries. 

To use an analogy, imaging if a game was released with a new version of a popular 3D engine and it turned out that 3D engine wouldn’t work for people with a popular model of ATI cards and that the only way to get it to work was to largely rewrite the 3D engine in house.  And even though it normally would take months and months to write a 3D engine, you really only have 3 weeks because people are really ticked off.  That’s essentially what we’ve been doing with the MP match making network stuff. 

UPDATE: 

It's here. People seem happy.  

If you have a low end machine and run into any error messages such as "failed to connect to NAT" try this:

download http://draginol.stardock.net/files/impulsereactor-0514.zip 

Unzip it into your Demigod\bin directory (program files\stardock games\demigod\bin).  If you're not having problems don't bother. But we found a Raknet bug that this works around for now on lower end CPU systems.

It’s been another crazy week of re-architecting the way multiplayer match-making works to handle the number of users. 

Today we are pleased to announce the new update:

Here’s what’s new for this week:

• Additional connectivity optimizations and improvements
• Lord Erebus: Bite's Mana Cost increased, Damage decreased, Armor Reduction decreased
• Fix for looping audio track after leaving character select screens
• Fix for audio volumes not being reset on restart
• Adding /skipintro command line to bypass intro movies
• Players will remain connected to facilitator if they fail to connect to other players, provided they can connect to the host. The host ultimately can kick people with problematic connections to others.
• Favor Items fixes, but still some assorted issues that should be addressed in the next update.
• Major optimizations to audio on low fidelity settings
• Fixed bug that occurred if you used an ability after issuing an attack command. This is the "won't chase the demigod I clicked" bug.
• Sending deltas for partially completed achievements.
• Each player now saves their own achievements and favor items
• Fixed partial Achievement saves bug.
• Fixed a bug that occurred if you accomplished an achievement in game.
• Fixed a bug in disconnect dialog that would try to update it even if we had determined we did not want to show it
• Changed resample rate to 24k in Low settings
• Delayed FMOD initialization until audio LOD is set via options
• Disabled hitbox interference from announcer messages.
• Life's Child no longer removes Inner Grace (Sedna fix)

Now, that said, what’s NOT in this update?

• Robust multiple IP support.  If you are talking to the Internet with more than 1 IP, you may have a harder time connecting. This is because you might connect to 1 player with 1 IP and to another player with a different IP.  We have gotten new code from Raknet to address this but it will take time to integrate and test.
• Better NAT. We have a slew of improvements to the NAT thanks to the Raknet update we got this week but again, it will take time to integrate and test.
• Route through host.  This works and we came “this” close to putting it in.  The problem is, not all players are really in a position to be routed through.  Now, my machine could do it. But we don’t have a way for Billy Bob with his DSL line from trying to host a game with 10 players. 
• Route through dedicated proxy.  This too works but we do not have the code in that lets it intelligently pick which proxy (i.e. someone in Japan is not going to want to use a proxy in Virginia). 

And lastly, let’s talk about 5 on 5.

There have been other peer to peer games.  But the connectivity complexity is C = D * N*(N-1) where N is the number of players and D is the raw data being exchanged.   While Demigod supports 5 on 5 multiplayer in custom, LAN, and single player, I have to tell you, unless you know who you are playing with, you’re asking for trouble if you try to do this with strangers.  Even if you manage to find 10 random people to play with online and they all manage to connect to each other, when you get in game, because the game is synced, you’re going to likely have a non-ideal experience. 

Let’s work through the formula where let’s assume D = 10.

With a 3 on 3 game, which, according to what I’ve seen in Supreme Commander, is the typical largest game people get into, C = 300.  On Battle.net with Starcraft, the max number of players is 8.  C = 560.  But when you go to 10 players, C = 900, nearly double the complexity of trying to do an 8 player game.

What our work has been these last 2 weeks has been to re-architect the system so that we can bring down that D multiplier. That’s the only part we can control.

When trying to connect players together, each player, based on their internet connection and PC performance has their own threshold of what variable C can reach before things fall apart.

In the beta, we were able to bring our variable D down by adding more servers.  And most beta testers had high end connections and PCs so they had relatively high thresholds before variable C had problems.

But clearly, in release, that threshold turned out to be a lot less.  The first inkling that our D variable was way too high was that the pirates who were pounding the server that first week were able to overwhelm our servers.  To put things in perspective, Impulse typically gets around 300,000 users using it each day and it doesn’t even break a sweat.  But a mere 140,000 connections that first week brought Demigod’s online experience to its knees.

We were able to get a reprieve that first week by shuffling users off to a new set of servers to get them away from the warez users. But as the game has continued to sell (and it looks like it’s going to break the 100k milestone shortly).

So the thing we continue to do is try to lower that D variable so that more and more people with marginal systems can get in.  There will likely be lots more updates as we find new ways to squeeze it. In the meantime, your best hope is to try not to go nuts with the # of players unless you know them.

We will probably have another small update tomorrow as we continue to refine this.

To see where we’re at as of 12:26am I recorded part of a beta where we got a 5 on 5 game going. Slowly but surely we’re replacing third-party code with Stardock code.

In this demo, there’s no port-forwarding going on. Port forwarding is a good idea if you can do it but it’s not necessary.

Problems I’m seeing:

1) If I have multiple IP addresses (I have a fiber line and a T1 connection and a Comcast connect – 3 NICs) it has a real hard time connecting me to other people. So that has to be addressed.

2) We have turned off routing through the host. Connections are great but if the host has a low end connection it slows things down. Will have to look to sending people to the dedicated servers but that’s for another night.

But hopefully you can see it’s sped up a lot. On Friday, the international release of Demigod goes out in Europe and Asia so we’re working to make sure Europeans get a good experience by this weekend (along with everyone else).

By the time we’re done, there will have been nearly a 100% transfusion of Stardock Impulse code into this connectivity system (so those of you at Stardock on the Impulse team reading this yes, you’re going to get the credit or blame next week <g>).

Stay tuned.