ChatGPT corrects itself when you tell it off

The internet has been abuzz around playing with ChatGPT, a new chat AI thing. Putting biases and unfortunate responses aside, curiosity got the better of me and I started to fool around with it.

One of the surprising interactions was, I told it (assistant? eliza? alexa? mega-clippy?) off for getting something wrong, and it corrected itself.

You are correct, the matrix in my previous response was incorrect.

Like a teenager half arsing their homework, I managed to get them to actually make an effort.

The conversation

I’m doing advent of code again this year, a task I heartily start but eventually give up after day 15 or so, but the early days are always fun. Day 2 starts off talking about the game Rock, Paper, Scissors and computing some scoring logic based on some simple rules.

Being lazy and tired after work I was having trouble picturing all the outcomes of the game to represent them in my program, so I asked the tool to print a truth table. Not a very good question I’ll admit, a truth table usually refers to boolean outcomes not the win/lose/draw states, but they ran with it anyway.

At first glance the output was pretty cool, but I found it hard to read. The labels for the rows were not printed so you couldn’t determine who was player 1 and player 2.

So I asked them to print me a matrix.


But something was still off, the text talks about the rules of the game but the matrix looks completely wrong. You can’t tie if both players present different objects.

So I gave them a sharp reply - you are wrong!

To my surprise, they corrected themselves and printed the corrected matrix. Even admitting the previous answer was wrong!

In this case I knew the rules of the game, I just wanted something quick to reference while writing code, but it makes me wonder whether accepting the first response from these AI tools might not always be the best idea.