Actually I remember an idea I had once about an AI system for Pokemon games that was more like an AI creator program. You would create the team yourself and then you would be able to select the behaviours for each Pokemon, so it would be tailored to the team and even could have a play style, so in-game for example, most trainers would have basic AI, wheras the Elite Four and Champion would have special AIs created for their team.
How feasible is this idea, and how would it compare to your AI?
I was talking to [I believe] Wichu about something like this. He was wondering whether a modified version of the AI could be used as part of
Project Amethyst for the in-game AI. It can with very little modification.
The way I have it written, the teams are just team files in "any" simulator format (currently only Pokemon Online or Pokemon Lab... Pokemon Lab can open Shoddy Battle team files and you can just re-save those as Pokemon Lab files so that covers just about every Gen 4 team file format). I plan on having my AI load teams files at random, possibly with some sort of weight depending on conditions, but it currently just has the name of the team file in a text file that's loaded at start up. With a line or two of code, you could add a way to give each in-game trainer (or AI, in your case) a particular team file to load as their own.
The other part of the AI that would be customizable, other than the teams, is the evaluation function. Wichu's plan was to have wild Pokemon just use moves randomly (meaning he wouldn't need my AI at all for that), but he wanted a way to make trainers be smarter. There are two ways to do that with my program, and you can actually do much more than just making one AI "better" than another.
The most obvious way to make differences in different instances of the AI would be to increase the depth of search. Foes like Youngsters could search ahead only to a depth of 1, for instance, while a Cooltrainer might be able to think to a depth of 2, gym leaders a depth of 3, Elite 4 a depth of 4, and your rival a depth of 5 (my program can currently evaluate a position about 3 turns ahead in about 15 seconds and a depth of 2 almost instantly, but I have a lot of room for cutting that down... my goal is 5 turns ahead in under 30 seconds). This is a change that would be used to just make one trainer better than another (a greater depth equates to stronger play, almost regardless of the position and the evaluation function used).
The other way (and probably the much more exciting way, to Project Amethyst people) actually changes the type of player the AI is, and that is changing the evaluation constants. My program uses a plain text file that determines the value of various things. Altering this table of values allows you to change the behavior of the program quite drastically without having to know anything about C++ or recompiling the AI. For instance, in my evaluation function, I had the following values:
Stealth Rock -300
Members 1024
HP 1024
What that means is that Stealth Rock has a value of -300 * effectiveness for each Pokemon on the team. So a Pokemon that's 4x weak to Stealth Rock loses 1200 points (a hefty penalty!) if Stealth Rock is down on its side of the field. Simply having a Pokemon alive is worth 1024 point per Pokemon (so a full team of Pokemon gives you 6144 points in addition to any other boosts). What HP means is that a Pokemon at 100% HP gets an additional bonus of 1024 points, and that goes down linearly with HP. In other words, a Pokemon at 50% HP only gets 50% of the HP bonus.
With those values, and Technical Machine's team being Jirachi @ Leftovers with Iron Head and Stealth Rock, Celebi, and Subroost Moltres (plus some others), and the foe's predicted team being Hippowdon with EQ, Slack Off, Stealth Rock, Roar, and Tentacruel with Rapid Spin, this is how Technical Machine played while searching to a depth of 2:
First turn, Stealth Rock. After that continue to Iron Head Hippowdon until one of them (usually Jirachi) dies. The reason for this behavior is that Technical Machine's team doesn't have a Ghost, so it's afraid of Tentacruel's Rapid Spin. Because of Iron Head's flinch chance and the fact that EQ isn't a OHKO, Jirachi would much rather risk dying to a CH EQ, but probably flinch Hippowdon or take a normal EQ than let Tentacruel get in for free and Rapid Spin. In other words, valuing SR at -300 with Pokemon on the team weak to SR (Moltres) makes Technical Machine play with a suicide lead to a greater extreme than almost any player.
If I alter those values so that Stealth Rock is only worth -100, the play is radically different. Now Technical Machine's first move is to just switch straight to Moltres. Moltres is the chosen switch because Hippowdon has no moves that can harm it, and Moltres can fire off either a Flamethrower or an Air Slash against the foe's team, which it decided was a better course of action than risking Hippowdon using Earthquake. In other words, when it values HP much more than Stealth Rock, it plays very conservatively in this situation.
Changing the value once again to -200 gives yet another radical play style (it plays exactly like I would for the first 4-5 turns). It uses Stealth Rock first turn, and when the foe uses Stealth Rock, it switches to Celebi. Celebi gives it a solid match-up against Hippowdon, and if Tentacruel comes out, Celebi has Psychic. Sure, Celebi will take a beating from Sludge Bomb, but it has Recover too, so Tentacruel is in a tight position. It correctly recognizes that the foe will switch to Skarmory and start using Spikes, but the team it was using is incredibly weak to Skarmory, so the foe going to Skarmory puts Technical Machine in a bad position regardless of what it does.
With a little bit of programming knowledge (not much, a smart non-programmer could figure this part out by reading the C++ of my evaluation function code), they could even make a 'suicide bomber' AI, that is more concerned with hurting you than winning. They could make it so the AI doesn't care about their own Pokemon at all, they just want to do massive damage against the foe (so they could use Explosion with their last Pokemon, just so you have to go back to a Pokemon Center instead of continuing on your challenge).
To recap: Changing the depth of search, which is done by just passing a single number to the program at the beginning of the battle (I actually have plans to make it so you can change the depth of search on a per-turn basis, or even within a turn, so AI-makers could take advantage of this if they want to change the quality of play of a player over time to simulate distraction / renewed focus) changes the strength of play. Changing the evaluation function changes the type of play. Pokemon AI-makers like Project Amethyst can use this their advantage by loading a new team, evaluation constants, and depth for each trainer / type of trainer.
Also, more on the topic of your AI, I know that now chess computers have advanced so much, there are players that have developed specific play styles to beat computers. Surely this AI, especially its team and moveset prediction, would be especially vulnerable to players using unusual sets and teams. Would it be possible for your AI to have methods of combating this, or would it become too complicated or perhaps introduce worse flaws?
Well, there is a reason that certain things are standard, and that is because they are generally superior to other strategies. A player can use a strange team to throw off my prediction function, but if that weakens the team, it might not increase the odds of winning (the general problem with a gimmick team). And if it doesn't weaken the team, then that should eventually be discovered by Technical Machine and it would cease to be a surprise. The only idea I've had to try to counteract this is to try to keep taps on individual players. If Technical Machine has played a certain user before, it can assume certain things about them that could be more accurate than just the general stats. However, that's a much more advanced concept, and I'm not sure how much of a real increase in strength that would give.
I wouldn't be worried about unusual sets or teams so much. Sure, that will throw its team prediction out of whack, but that doesn't put it at any serious disadvantage compared to a normal player, unless its programmed to rely on its predictions as definite. I suppose things like Cosmic Power Jirachi could cause major concerns though, as it would take an enormous amount of time to see the inevitable win if its allowed to boost endlessly.
It currently does rely on its predictions as definite, but only within a single turn. The reason is that if it tried to predict 7 moves instead of 4, for instance, then it would also have to evaluate those 7 moves, which would slow it down for not as much gain. However, once it sees Jirachi using Cosmic Power, it immediately knows Jirachi has Cosmic Power and will take steps to stop it.
One thing; if the program will always do the "best" move in any circumstances, it does make it very easy to predict. If the opponent knows which move covers all options the best they can choose their move accordingly. Obviously I can't see a way around this though :)
My ultimate goal is to make Technical Machine select from all of its moves with a certain (possibly 0) probability. So it would select Stealth Rock 70% of the time, Iron Head 20% of the time, and Body Slam 10% of the time, and switch / use Wish 0% of the time, as an example. This would make it only predictable on average, but it's hard to fight against "On average, it will use the best move, but sometimes it will use the best counter to my best counter to that generally best move". And of course, to find out exactly what Technical Machine would do, the foe would have to be running Technical Machine themselves, but they don't have Technical Machine's exact team, so it won't play exactly the same.