Brad Wardell's Blog


New screenshots at Worth Playing!

Published on Saturday, December 31, 2005 By Brad Wardell In GalCiv II News

WorthPlaying is the latest site to post a GalCiv2 preview.

The article is quite positive throughout, with the editor concluding:

"Overall, Galactic Civilizations 2: Dread Lords is looking to be a major entry into the Turn-Based Strategy genre. Already it sports beautiful graphics and solid audio, and has so many options for customization that nearly any type of player should be able to find a balance that best suits their style of play."

Gamma Releases in January

Published on Thursday, December 29, 2005 By Brad Wardell In Galactic Civilizations II

There will be a small number of people who will have access to the gammas come January.

Users with Diplomat access or higher will be able to report feedback in the private Council forum.

The access classes are Citizen->Veteran->Diplomat->Ambassador.

How do we determine who gets access?  Essentially it boils down to how effectively someone can report bugs and get us crash files so that we can figure out what went wrong and be available on #galciv on our IRC network to interact with directly.

So when you run into a crash in beta 5 of GalCiv II, look at the sticky posts at the top of the main GalCiv II forum and see how a proper report is.  We'll be adding a few other people to the list before the second week of January when the first gamma is released.

The AI in GalCiv II Part 5

Published on Wednesday, December 28, 2005 By Brad Wardell In GalCiv Journals

It's been a productive holiday season.   The AI that existed in Beta 5 (public) is dead. It's gone now. And it won't be remembered fondly. Not by me at least.  It functioned.  But the fact is, in a game that has no multiplayer the standard of quality on AI should be very very high. 

It isn't enough for a game AI to be challenging.  The AI needs to be more than that.  It needs to play like a human being.  Much of what I've done this holiday is go through and play the game over and over and think of different ways human beings would play the game.  Then I've tried to replicate that.  We want a player to feel like they're playing someone else on-line (minus the disconnects and swearing).

Here's what I've checked in so far:

There are a bunch of important changes in this that will have a significant impact on gameplay.

 

Here is what has been changed:

 

+ Various bug fixes

+ Various new helper functions

+ New function: CalculatePrimarySectorToConquer(). AI creates a set of sectors that it wants to conquer. It also creates a set of sectors it wants to protect as a priority.

+ Modified base ship design to put more effort into range.

+ New AI ship class: Scout

+ AI designs scouts

+ AI evaluates whether it can “trust” another player based no their past behaviors.

+ AI looks to see if someone is culturally attacking them.

+ AI looks for military threats (like someone massing forces).

+ AI looks to see if there are hostiles in sector and if so, avoids sending transports there IF it knows about the ships.

+ AI no longer “knows” where good planets are. Non-Terran AI knows where planets are and is better at scouting. Terran AI has to send scouts blindly.

+ AIIsPlanetKnown() determines whether the AI knows what’s going on on a given planet.

+ AIIsShipKnown() determines if the AI knows about the existence of a given ship.

+ AIFindUndefendedPlanet() gets a sector ID and looks to see if it knows about any undefended planets.

+ AIIsSectorFocus() checks to see if a given sector is of interest to the AI.

+ Continued (and failing) attempt to get the AI to build key planetary improvements on its planets.

+ AI builds scouts

+ Tweaks to what kinds of ships the AI will build to be “smarter”

+ THE END OF THE COLONY RUSH. Colony rushes were wrecking the AI economy because colonies now have a significant upfront cost.  Now, the AI will take a break from colonizing in order to build up what they have so that their economies stay strong.

+ Smarter choices when building ships.  AI was going broke in cases due to buying ships.

+ AI temporarily no longer leases any ships

+ AISetDestination() modified to use new SDS/AI classes.

+ SDS/AI uses new AI functionality (discussed below)

+ AIFindColonyDestination()

+ AI can only colonize planets it knows about.

+ AIFindConstructorDestination()

+ AI will try to bail out colonies that are under alien influence through its own influence starbases.

+ Various fixes so that the AI will intelligently find a new destination after its first destination changes (or is destroyed) mid-turn.

+ AI designs escort ships

+ AI now builds escorts

+ AIFindEscortDestination()  will take escorts and look for transports to escort so that they have some defense.

+ Transports are now designed with more speed in mind.

+ Transports will run away from hostile ships.

+ AIFindTransportDestination() looks for planets to conquer. Totally new tactics than old system.

+ AIFindScoutDestination() looks for destination for scouts. Terran AI has to send it to stars, non-Terran AI will send it to planets since it can see planets thanks to starting with Stellar Cartography.

+ AIFIndMilitaryDestination() totally new way of deciding which ships to go after. It’s MUCH smarter.

+ AI at higher levels uses rally points, sends ships to rally points and then builds fleets there.

+AIFindMyColonyUnderAlienINnfunce() finds a colony that is under serious risk of alien influence and returns it so that it can be countered.

+ Fixed some bugs with AI accidentally looking at Local Player (the human player) instead of the player passed in.

+ Changed the way approval rating on colony is calculated so that tax rate isn’t tied to population anymore.  Population causes unhappiness and tax rate does but they are not intermixed.

+ Fixed bug where population would grow until approval was at 30% but population would start shrinking at 40% so you would have weird jumping in population.

+ Changed (now that the manual is printed) how much money different government types give: 10% for Republic, 20% for Democracy, 30% for Federation.  Before you just got ridiculous amounts of money.

+ BUG ALERT: There are cases where ships are not being destroyed so you end up with ships on the map that have 0 hitpoints. Same as in planets. Not sure what the cause is but it was wreaking havoc so I put in a TEMPORARY hack fix into RemoveAIShipsFromPLanet() or whatever it’s called so that if a ship has no hitpoints and the ship is not marked as destroyed that II destroy the ship. This needs to be fixed correctly.

+ Lowered cost of upgrading ships.

+ Made whether a given player can see an object part of the base game object class.

+FindClosestLocalTransport(Parameters).  Depending on the parameters, will find a transport in the current sector.

+FindCLosestUndefendedEnemeyPlanet(Parameters). Looks throughout the galaxy for an undefended planet based on the parameters you give it.

+ScanSector() Tells the AI to scan around them to catalog what planets and ships they currently know about and put it together into a list.

+ Tons of tweaks to make the AI better at targeting specific ships and focusing its military might effectively.

+ AI won’t just willy nilly send its ships off to the next battle, they’ll stick around even after your defenses are gone until the transports have come and conquered. Only applicable at higher difficulty levels. Naturally with our luck all this work (which was significant) will get ignored in reviews since many reviewers play the game at beginner and don’t’ see the AI really in action.

+ Tons of tweaks and adjustments to how the AI plays out its strategies.

+ AI will sometimes report to the player that it knows what the player is up to. I.e. you mass up ships next to its planets and the report window may say “We know what you’re doing”.  IF there’s time, I’d like to have it come up with a snarky “Well duh, we play strategy games too” type dialog.

+ AI detects ship massing and will silently try to prepare itself for attack. It pains me to have all this counter-reaction going in and the human player may not ever even be aware that the AI is aware of the old “I’ll stick a bunch of big ships and transports next to their planets/cities/whatever and then suddenly attack” trick.

+ Engines use less size

+ Life support uses less size

+ Initial colony maint cost lowered from 25 to 20.

+ Cleaned up starbase modules so that they follow more logically.  Also, battle stations is required to get any of the other starbase defense ones (was getting annoyed wading through all the modules all the time).

There's still more to do, but not that much more.  The key modules are now in.

Fun from last night..

Published on Wednesday, December 28, 2005 By Brad Wardell In Ideas

The AI difference from Beta 5 to now is..well just night and day.  Beta 5  as far as I'm concerned had no AI.  Sure, the ships moved around and such but that was about it.

Now the AI reacts to things like influence starbases.  It'll try to counter your influence star bases with its own.  Some races will try to do a culture attack on you. 

And when it comes to combat, no more willy nilly sending ships around ineffectively.  They focus on specific goals and mass up for battles to take those goals.

They will identify a threat, determine what htey need to kill it and then put together the appropriate force.  No more pathetic "Look, it's a lone fighter attacking my mega fleet".

In this example, I had a huge fleet massed for attack.  I was pretty impressed with it.  But then the AI massed an even bigger fleet.  Some AI players go with fighters, others with capital ships, depends on the AI module. 

One late game battle was pretty painful, my brand new "Debbie" (my wife's name) class battle ship was alongside a bunch of "Victory" (older model) class battle ship and the Drengin's fighters were powerful enough to take out the Victory class ships on their own. (speaking of which, I significantly decreased the upgrade cost on ships as a result).

Here's some cool battle scenes from last night:

If you look closely, you can see the Drengin fighters, they're itty pitty but they're nasty little buggers.

Also, the AI is MUUUUCH better at adapting its weapons.  In Beta 5 the weapon analysis went like this:

IF (I am at war) THEN (Find most lethal enemy and counter their weapons).

Sounds good on the surface. But it's stupid.  The Drengin and Altarians had been battling it out all game and the Drengin were finishing them off while I had built a huge fleet of ships that were perfectly suited for whacking the Drengin.

So the algorithm changed:

IF (I am at war) THEN (If my most lethal enemy is much weaker then me) THEN (Look at the most powerful military in the galaxy and factor their weapons and defenses into my counter measures)

Saber Cherry AI suggestions

Published on Tuesday, December 27, 2005 By Brad Wardell In Galactic Civilizations II

Note, this was posted to the dev journal area which I removed since only GalCiv developers are supposed to be able to post to that area.

--

Artificial Intelligence, Tree Pruning, and Saving vs. Regenerating Trees


This is a suggestion about how to prune search trees in order to avoid doing AI calculations at execution time. It makes some assumptions, e.g. you are using branch-and-evaluate or A* to search a dynamically-generated tree when determining the next move at execution time (between AI ship moves).


The analogy: How do tell a very stupid, very nearsighted climber the best route to the top of 1000-branch redwood tree? Assume that any branch he tries to use might break off, forcing him to try a different approach.

GalCiv: Spend months evaluating possible paths, tell him which branch to start with, and let him feel his way from there.

GalCiv2: Give him a talking monkey who can explore the local area and tell him which branch to try next, at any time.

My suggestion: Spend months evaluating possible paths and branch-breaking probabilities. Clone the tree, and grow it in a cute little bonsai dish, trimming all the branches that are weak or inconvenient, until you are left with a tree that has only the branches for the best route, and the branches needed for contingencies (so he can still climb if a branch on the best route breaks). Give him this simplified bonsai model, with the correct routes marked, and tell him to follow it exactly (use only tree branches present in the bonsai, as indicated by the path) unless the main and nearby contingency branches all break. Then he can throw away the bonsai and use his talking monkey.


Technical stuff:

Since the (low priority) strategy thread has plenty of time to churn in the player phase, prior to execution, it can obviously make a very good initial move. However, in order to evaluate the move, I imagine you are making a tree deeper than the move of interest (as in chess AI routines) rather than simply acting on the scores of the leaves in a complete 1-deep tree. If you are indeed building a tree more than one level deep in the idle time (past the first ship move), why not save the highest-probability path(s) through the tree so it does not have to be recalculated at execution time?

For example, these ships are all within reach of each other:

Drengin: battleship, cruiser, destroyer, troop transport.
Human: destroyer, corvette, freighter, freighter.

Result: Idle thread builds a large tree and decides the best and most probable path is this:

D.Battleship attacks H.Destroyer (wins)
D.Cruiser attacks H.Corvette (wins)
D.Destroyer attacks H.Freighter (wins)
D.Transport moves toward planet

(human turn)

H.Freighter flees (but not far enough!)

(drengin turn)

D.Destoyer attacks H.Freighter
D.Transport captures planet
...Sector victory!


However, when only first move is saved, it might go like this:

D.Battleship attacks H.Destroyer (wins)
D.Cruiser attacks closest, weakest enemy, H.Freighter (wins)
D.Destroyer attacks closest, weakest enemy, H.Freighter (wins)
D.Transport moves toward planet

(human turn)

H.Corvette attacks D.Transport (wins)

(drengin turn)

D.Battleship attacks closest, weakest enemy, H.Corvette (wins)
...sector clear of enemy ships, but the invasion was thwarted!


Now, if I understand correctly, the proposed solution is to do more calculations (build deeper trees) at execution time. This is possible due to handling graphics on the GPU, and people upgrading CPUs over the last two years. However, these trees explode fast! Calculating the same tree 20 times for 20 ships in a sector will invariably limit the tree depth, and thus quality; if you want the AI to move all the ships in a sector in under 2 seconds, this would only give 100ms to generate each tree and evaluate the best path. HOWEVER - if the tree is only calculated once, using the idle time in a player's turn, a full 5 minutes might go into a tree, making it 3000x better! With ideal dynamic pruning, of course.


The problem is that such an immense tree cannot be saved, because it might take 200 GB of memory. Yet the "desired" path through the tree (which might be 2-100 levels deep, depending on how it is pruned, how many turns ahead it looks, and how many ships there are) can certainly be saved in a trivial amount of memory. Furthermore, the "second most probable" paths can ALSO be saved, since most battles are highly predictable ("biased"). For example, in the situation discussed, the entire tree-path of D.Battleship vs. H.Destroyer, D.Cruiser vs. H.Corvette, etc. is the ideal Drengin strategy and the desired outcome of each battle ("node") is perhaps 98% assured, giving a cumulative probability of 94% that the first three rounds will go as planned, meaning that the tree needs to be recalculated only 6% of the time.

What if something goes wrong? Well… recalculate the tree… or save backup paths! Let’s say that a node requires 32 bytes. If there are 16 possible moves per ship, 4 friendly and 3 enemy ships, and you look 2 turns ahead (your turn, enemy turn, your turn), and you cycle through ships in a fixed order, that makes a tree of maximal size ((2^4)^4)*((2^4)^3)*((2^4)^4) = 2^44 leaves or 2^49 bytes (more than an Opteron can address), compared to 2^24 bytes (16 MB) that I imagine Stardock has allocated to temporary AI storage. By "maximal" size, I mean that this assumes no combat and so ships are not dying each turn, which would make the tree smaller; the exact size of a tree that involves combat and movement on a 2D map cannot be determined without generating the tree. Comparatively, saving only the desired path in this 7-level tree would require 224 bytes, and not need recalculation 94% of the time (estimated, as the humans are hopelessly outgunned). To improve this accuracy, there could be a "tree trunk" - the desired and expected outcome - in which the most highly biased events (like battleship versus destroyer) are near the root, such that if reality deviates from expectations, it will do it later rather than sooner.

Furthermore, rather than just storing the trunk, every branch off the trunk can also be stored - you end up with something that looks more like a bunch of stacked octopi than a tree, in that octopus tentacles never branch after leaving the main body. This way, the semi-complete tree that you made for the sector would be reduced down to the "desired" path and "contingency" paths, such that there is a contingency plan for every unexpected event (like if the battleship loses to the destroyer). Note that there are no contingencies for a sequence of two unexpected events, so if the battleship lost and then the cruiser lost, a new tree would need to be created at execution time (smaller, though, due to having 2 fewer ships). Memory? Why, this would only require 224 bytes for the trunk nodes, and an additional 15*6*32+15*5*32... = 15*32*((6+1)*6)/2 =10080 bytes, for 10304 bytes to store the desired plan and a contingency for every unexpected outcome! If you limit unexpected outcomes to combat results (you shouldn’t have an unexpected result when moving a ship according to a plan, unless there are mines) then you only need 1 contingency per node, not 15; the resulting plans are 896 bytes (trunk and contingencies).

In the generic case, it seems wise to allocate a specific amount of memory - maybe 16KB - to the master battle plan for a sector; store the primary trunk (most probable and desirable outcome); and then store as many branches as possible (prioritizing the nodes that are most probable and closest to the root) until 16KB is exceeded. Once it is generate, and the AI still has time during the player’s turn, deeper exploration of the search-space is possible, which may lead to contingency nodes being added or removed from the plan, replacement of the trunk with a better one, or making the trunk longer. Alternately, generate the subtrees to the limits of the time allocated, and trim each one back to 16KB by truncating the lowest-probability, least desirable nodes (you end up with a "fan coral" or "fern leaf" shape, radiating out from the root).

I’m eager to hear whether such a system is already in place, or if it would not be applicable to your AI methodologies. I am currently doing a lot of programming related to NP-complete problems, computer branch prediction, and search, so I think about this sort of thing constantly; but as a result, sometimes I make assumptions that my listener(s) are immersed in the same world, and I skip parts of explanations. If my wording was confusing, I’d be happy to clarify.

-Cherry

The AI in GalCiv II Part 4

Published on Tuesday, December 27, 2005 By Brad Wardell In GalCiv Journals

When I play strategy games against the computer I am always amazed that no one does a simple for loop through the enemy units to see if they're literally just outside my city, base, planet, whatever ready to attack.

That's what we humans tend to do.  The most lethal strategy in these kinds of games is usually to line up our units, ready to attack and then WHAM, knock down the AI in a couple of lethal turns.

In GalCiv, there is also the influence attack -- building influence base after influence base and gobble up the galaxy.

In GalCiv I, the AI did monitor this stuff and would come up with some funny responses like "Just so you know, we know what you're doing.."  In the original OS/2 version, the AI would surround your ships with stealth ships (the OS/2 verison had cloaked ships but they weren't very fun which is why they're not in now) then it would determine when it had enough and then preemptively attack you.

I don't know if I'll have enough time to put in cool stuff like having AI ships surround your ships.  That's the goal.  But the AI does get a pretty good picture of what the human player is up to (at higher difficulty levels) and start to put together a response to it even if it's not apparent. 

The AI in GalCiv II Part 3

Published on Tuesday, December 27, 2005 By Brad Wardell In GalCiv Journals

One of my biggest disappointments in the GalCiv I AI was how it fought its wars militarily.

The computer players in GalCiv I were given a lot of kudos but that mainly was because they were so efficient in how they handled their resources and on-the-fly military decision making.

But when it came to the overall strategy of war, the AI was disappointing.  It wasn't supposed to be that way, it's just that the best laid plans can fall apart when things get complicated and there's not much CPU time involved. 

In GalCiv I, all the graphics were handled by the CPU.  This meant that the background threads in GalCiv which handled the AI could still effectively slow down animation and graphcis.  So I was always painfully aware that any time I wanted the AI to do a complex calculation that I might be making some animation stutter due to slowing things down.

In GalCiv II, it's all 3D and all hardware accelerated.  So my background threads for running the AI are honkey dorey. This has let me have the AI build much more complex strategies.

The GalCiv I military doctrine was code-named "Sector Domination".  That is, the AI would look at what sectors it needed to control, assemble forces, and then try to dominate that sector.  On small maps, it worked pretty well.  On larger maps, it fell apart due to the fact that it is CPU intensive to try to coordinate lots of units every turn.  So what happened is that the forces would come to the sector in dribs and drobs.  Or put another way, players would get a death train of piddly units.

The other big CPU issue was in how the AI would react AFTER it succeeded in a given turn.  For instance, a ship attacks a given ship and destroys it. The decision on which ship to attack was originally calculated in a thread during the player's turn.  But now it's destroyed that ship during the move phase.  That move phase is in the main thread.  We were very hesitant for the AI to use up a lot of CPU calculating where it should go next.  So the result was, the computer player ship would make ONE smart move and then a quick-non-optimal secondary move.  So when it destroyed that first ship, what it did after that for the rest of its turn was pretty basic.  It typically involved the ship looking to see if there was another ship in the sector and go towards it.  It was "good" enough for many players.

But the real goal would be something a lot more complex: Ship attacks planet and destroys defenses. It then does a full calculation of what it should do next AND sends a signal to ALL the other ships in the sector to do a full calculation of what they should do.  That way, if there were transports nearby, they could capitalize on the opportunities.

In GalCiv II, the AI also can make use of rallypoints -- the same kinds the human can use.  So it will, at higher difficulty levels, organize ships via rally points and build up impressive fleets and then intelligent figure out where to send those ships.  They'll tend to only go after ships or fleets that they think they can probably kill or at least make a significant dent (i.e. at higher levels, the AI won't send some fighter to attack a battle ship). 

The handling of transports has changed as well.  They and their escorts and supporting military ships will "stick around" in a given sector until it's been taken.  So even if the defenses on a planet are temporarily taken down, the various ships will hang around until every planet in the sector that belongs to an enemy has been conquered.  And if enemy ships start appearing, the transports will run away from them.

The intelligence levels players set the AI to will be key in all of this.  Because they're CPU intensive, the easier levels simply don't use them at all.  What I'd like to do in some future update after release is allow players to set AI intelligence AND Advantage Factor.  Right now the two are rolled into one.  If you play at "Intelligent" on a given player they're playing the same resources at maximum intelligence.  But I'd like to let people with lower end rigs to be able to "get a challenge" by having the AI play at say "Normal" where some of the stuff is still turned off but crank up the advantage factor (advantage factor being "here's free money, loser." for the AI) to get the challenge they want if they don't care if the AI is "cheating" or not.  But that's in the future.

Hopefully as new players make their way up from beginning to normal to tough they'll see the AI not just be harder to play against but see it play more intelligently.

The AI in GalCiv II Part 2

Published on Tuesday, December 27, 2005 By Brad Wardell In GalCiv Journals

GalCiv II has a new diplomacy AI.  It's a complete rewrite from what was in Galciv I.  The AI literally calculates how much it should like a given player based on a broad range of factors.  Some of these factors are displayed on the "report" dialog on the foreign policy window.

During the course of the betas, the kinks in the new system have shown up and this past week I've been going through and addressing them.  For example, the AI tended to go to war with way too many players.  The cause of that was that each "Factor" was independent of others. 

That is, the AI WOULD take into account that it was at war with multiple players. But it didn't do enough countering of that if it didn't like a given player.  So you ended up in a sort of banzai type mentality in the game.

What I've done this time around is to have a "worst enemy" concept.  If the AI is at war with its worst enemy, it will really try to stay out of war with other players.  If it is at war with someone else who is not their worst enemy, it'll potentially make room to go after the worst enemy if it ismilitary strong enough but generally it'll try to stay out of war with more than one player at a time if it can.

This brings us to one of the goofiest parts of GalCiv I that needs to be addressed -- alliances.  In GalCiv I, alliances were great for the weak player but horrible for the strong player because the weak player could run amok because if they got into a war, even if the weak player started it, the strong player got sucked in.

This time, Alliances will be turned into a defense package.  That is, if you attack someone else, your ally isn't bound to help you.  ONLY if you are attacked first will your ally come to your aid. 

So if you are at war with the Drengin and Yor and they are both allied and you make peace with the Drengin, that peace will remain because the Drengin won't be honorbound to re-declare war with you UNLESS you make peace with the Yor and then later START a new war with the Yor.

The AI in GalCiv II Part 1

Published on Monday, December 26, 2005 By Brad Wardell In GalCiv Journals

"So what did you do over your Christmas break?"

"Rewrote several key modules of the artificial intelligence engine in Galactic Civilizations..."

Galactic Civilizations has always been known for its strong artificial intelligence.  Since the game is designed to be a single player game, it bloody better have good computer players. 

Most people reading this don't know me very well but I'm actually a multiplayer fanatic.  I was one of the top ranked Starcraft players on Battle.net during its beta and was the top player on the "Free For All" part of Cavedog's Boneyards service (Total Annihilation multiplayer).  In short, I like to play multiplayer.

But I only play multiplayer because most games do not have very good single player experiences for strategy.  You quickly find their weaknesses and predictability and end up having to go on-line to get a challenge.  Turn-based games are too eradically paced for most people (certainly me) to get into playing them multiplayer. 

So when it comes to playing Galactic Civilizations II, I want the computer players to be able to play intelligently.  That means that the difficulty level shouldn't just be how much they cheat or how much free gold they get or whatever.  The difficulty level should determine HOW they play. 

In Galactic Civilizations I, the AI and the human played by the same rules up until Genius level at which point we cranked up the abilities of the computer players (gave them free money and such).  Playing by the same rules is much more CPU intensive and putting together a good strategy is also time intensive as well both in terms of CPU power and coding time. 

One of the most CPU intensive things in GalCiv AI used to be scanning through planets and exploring things.  So to avoid forcing the AI to do that, we made it so that the computer players "knew" where the good planets were. We made it part of the plot -- the aliens had long since explored the galaxy and the humans could not.

But in GalCiv II, you don't necessarily play as the humans.  So to be true to the story, the humans, when controlled by the AI, have to actually explore out the galaxy with scouts and what not.  In the process of doing this, we discovered that it doesn't make as big of a difference as we thought to the effectiveness of the computer player.  Moreover, on a modern machine (i.e. Pentium IV processor) it's not a major factor either.

Because in GalCiv I the AI got to know where the good planets were, some people would say "Well sure the AI in GalCiv is lethal -- it gets to know where the good planets are!".  So we've decided to eliminate this.  The AI colony ships no longer know where the good planets are. They have to go out and explore the galaxy just like a human player would. 

The ultimate goal is to make the AI in GalCiv II play virtually undistinguishable from a human player.