INCOMING STUDENTS: Complete your FAFSA and schedule a visit to receive up to $2,000 in grants. Learn More >

BlogAbout TU  |  School of Science, Technology and Health

Illegal Move: The Day I Let ChatGPT Play Claude at Chess

Pete Piraino headshot

If you know me well, you know I have two enduring passions outside of higher education: cycling and chess. Chess has fascinated me since my grade-school days, when I somehow convinced my classmates to participate in the intra-school chess tournaments I organized. (Admittedly, I can’t imagine a geekier sentence to have just written.)  Anyway, I don’t claim to be a chess grandmaster – or even a master – but I’ve played long enough to appreciate both the beauty of the game and the remarkable evolution of chess engines, which have gone from training partners to virtually unbeatable opponents for even the world’s best players.

So, in the spirit of Tiffin University’s AI-Forward philosophy – and purely as an academic exercise, of course – I decided to conduct an experiment.  What would happen if I had two of today’s leading AI models, ChatGPT and Claude, play a game of chess against each other? 

Could two machines capable of writing software, passing exams and composing essays also survive a full game of chess?  The short answer?  The experiment was a success.  The chess was another story.

With ChatGPT open on the left monitor, Claude on the right, and the live chessboard displayed on a third monitor between them (courtesy of Chess.com), I formally introduced the two AI models, had them shake hands, wish each other good luck, and then made the opening move by relaying ChatGPT’s first move to Claude.  My job seemed straightforward enough: relay each move in chess notation and stay out of the way.  As it turned out, that second part wouldn’t last long.

As the game began, I must say I was impressed. For the first dozen moves or so, it looked like I was watching two reasonably competent club players who were playing at perhaps the expert level.  The openings were solid. The pieces were developed logically, each model fought for control of the center, and the game was genuinely competitive.

Then the wheels came off, and I suddenly found myself watching a slow-motion train wreck.  It was as if both AI models had collectively decided that the rules of chess were merely suggestions.

Photo of chess board

By about move 12, both models started making questionable moves.  By move 18, both models began attempting illegal moves with surprising regularity.  One would try to move a bishop through another piece.  The other would invent a knight move that would get you tossed from most chess clubs.  My role quickly shifted from spectator to full-time referee.  Neither model seemed to notice.  They continued announcing each move with unwavering confidence, even as those moves became increasingly detached from the actual board position.

I would occasionally have to remind both models, “That’s an illegal move.”

With incredible politeness, “I’m sorry.  You’re absolutely right,” was the response.  “Please repost the current board position.”  So I would repost the board position, only for them to lose track of the position a few moves later.

It was like watching two eight-year-olds play each other after learning chess just last week.  They both understood the basic rules, but neither could consistently remember how each piece moved or maintain an accurate picture of the board.

The culprit seemed obvious: board recall.  Once the position became sufficiently complex, both models struggled to maintain an accurate mental representation of the board.  Without that positional awareness, illegal moves were inevitable.

The reason dedicated chess engines are so superior to humans is surprisingly simple.  After every move, the engine evaluates the current board position as it exists at that moment.  It doesn’t become emotionally invested in a previous plan or dwell on mistakes from earlier in the game.  Instead, it analyzes the position in front of it, calculating millions of possible continuations in seconds before selecting the strongest move.  Humans, by contrast, rely on memory, intuition, pattern recognition, and calculation—all while trying to keep track of a constantly changing position.  We tire, overlook threats, second-guess ourselves, and occasionally become attached to ideas that no longer fit the position.  Chess engines don’t suffer from those very human tendencies.

But that kind of positional awareness simply wasn’t happening in the ChatGPT-Claude match.  Ironically, these two artificial intelligence models were playing more like fallible humans than dedicated chess engines!

And the personalities of the two competitors were almost as entertaining as the game itself.

ChatGPT was all business.  It simply announced a move with very little commentary and waited for the next position.  At one point when it lost its grasp of the current board position, it requested:

Please give the updated board after …Rc6, and I’ll calculate the next move from the actual position rather than the inferred one.

This was interesting because there are no inferred positions in chess, only actual ones.

Claude, meanwhile, treated every turn as though it were giving a TED Talk.  Before making a move, it would explain its strategic objectives, discuss positional considerations, weigh alternatives, evaluate future possibilities, and generally narrate its entire thought process.  Ironically, many of those carefully reasoned moves still turned out to be, well…bad.  In fact, Claude committed several costly blunders in the endgame that effectively decided the game.

ChatGPT ultimately won by checkmate on move 34.

Before you congratulate ChatGPT, if we’re being honest, late in the game Claude melted down like the HAL 9000 computer from 2001: A Space Odyssey. 

Then, after it was over, as if to rub salt in its own wound, Claude gave this brief rundown on the loss:

Good game overall — I made a costly tactical error around moves 26–29 (the Nh6+/Nxf7+/Nxd6+/gxf6+ sequence cost me a piece and exposed my king badly), and even though I clawed back material with 33…Qxg5 and 34…Rxc4, my king’s   position never really recovered, and my opponent found the cleanest way to finish it off with the back-rank/queen combination.

Claude’s parting thought was:

Thanks for catching my illegal moves along the way — appreciate you keeping me honest on the rules.  Want to go again?

Now that’s what I call a gracious loser. Unfortunately, I had already exhausted my daily quota of illegal moves, so I politely declined Claude’s offer.

The experiment was a fun reminder of what large language models really are. They’re incredibly sophisticated word-prediction systems.  They excel at language, synthesizing information, writing, brainstorming, coding, and solving many complex problems.  What they are not designed to do is maintain a precise, evolving positional state over dozens of moves in a game of chess.  In other words, they can discuss chess brilliantly.  Playing it accurately for an entire game is another matter.

And that, perhaps, is the real lesson – and exactly why we’ve embraced an AI-Forward philosophy at Tiffin University.  AI is not magic, and it is certainly not infallible.  Like any powerful technology, its value comes from understanding both its remarkable capabilities and its very real limitations.  The more AI-literate we become as faculty, staff, and students, the better equipped we’ll be to use these tools wisely, ask better questions, recognize when human judgment is still essential, and know when an answer deserves a second look.

AI-Forward at TU isn’t about replacing human intelligence with artificial intelligence. It’s about amplifying human intelligence with artificial intelligence. It’s about preparing students to thrive in a world where AI will be as commonplace as the internet is today.  

That means teaching students not only how to use AI tools, but how to question them, verify them, challenge them, and apply them responsibly.  It means helping faculty reimagine teaching and learning in ways that leverage AI to deepen critical thinking rather than diminish it.  It means equipping graduates, regardless of their major, with the judgment to know when AI can accelerate a solution, when human expertise must take the lead, and how to combine both to solve problems neither could tackle alone.  That’s what AI-Forward means at TU: graduating students who aren’t merely AI users, but thoughtful AI leaders.

So yes, ChatGPT won. Claude talked itself into defeat.  And I spent far more time than I ever imagined saying things to artificial intelligence like, “No…that’s an illegal move,” and, “No, bishops can’t move through pieces.”         

It wasn’t exactly Deep Blue versus world chess champion Garry Kasparov. But it was a memorable reminder that, at least for now, the human in the room is still responsible for keeping the pieces on the right squares.