What AI gets wrong

This morning, I asked Google Gemini to: ‘Write me a 12-line poem about what AI gets wrong.’

This is what it came up with:

It counts the leaves, but cannot feel the breeze,
And parses laughter down to code and key,
It mimics tears we shed upon our knees,
Yet never knows what aching ought to be.

It builds a song without a beating heart,
And paints a world it cannot truly see,
Confusing clever patterns for an art,
And flawless grammar for sincerity.

It answers quick, though wisdom stays untaught,
And mistakes the words for all that lies within,
For human truth is lived, not merely processed,
And soul is where its algorithms end.

I made five changes.

1. I decapitalised the line beginnings, for modernity.
2. I replaced the commas with semi-colons in lines two and ten.  A little sloppy, I thought, given the reference to ‘flawless grammar’.
3. I took the ‘And’ away from the beginning of the third-to-last line, because the extra syllable jolted.
4. In the second-last line, AI had rhymed ‘untaught’ with ‘processed’.  I preferred ‘thought’ instead of ‘processed’, because it rhymes, and because it challenges humans too.  As I changed it, I reflected that AI may have been being ironic with its mismatch.
5. I changed the last line.  AI had offered ‘and soul is where its algorithms end’.  I strengthened the close by changing this to ‘and soul is where all algorithms end’.  I felt this made it more portentous and far-reaching.  I left the rhyme mismatch in the last line, because for me it made the word ‘end’ even more isolated and resonant. I also thought of changing the word ‘soul’ to ‘life’, but felt the world wasn’t yet ready for the provocation.

I thought about correcting ‘Confusing…for’ to the more technically correct ‘Mistaking…for’ or ‘Confusing…with’.  But I decided to leave it as a neologism, partly because it has richer connotations, and partly because there is some irony in the fact that AI chose it in the context of so-called correct grammar.

This is my amended version:

WHAT AI GETS WRONG

It counts the leaves, but cannot feel the breeze,
and parses laughter down to code and key;
it mimics tears we shed upon our knees,
yet never knows what aching ought to be.

It builds a song without a beating heart,
and paints a world it cannot truly see,
confusing clever patterns for an art,
and flawless grammar for sincerity.

It answers quick, though wisdom stays untaught,
mistakes the words for all that lies within;
for human truth is lived, not merely thought,
and soul is where all algorithms end.

As an afterthought, I asked Google Gemini: ‘What do you think of your own poem?’  The AI said it found it ‘honest'(!)  And it pointed out the flaw in its own second-last line, noting the rhyme and rhythm failure, suggesting that:

‘in a meta way, that subtle imperfection mirrors what the poem is talking about: an algorithm trying to structure human expression, but missing the exact touch.’

Touché.

Unable to resist, I finished our writing session by asking: ‘How can a poem written by AI be considered honest?’

Gemini thought for a moment, and said: ‘It’s a fascinating paradox, because AI doesn’t experience “honesty” as an internal moral standard or an authentic emotional truth.’

It then did what intellectuals do, and got buried in discussions of technique.

‘In short,’ it said, ‘AI cannot be morally or emotionally honest, because it lacks intention and subjective awareness. However, a poem generated by AI can be factually and conceptually accurate about its own nature—and when those words resonate with human experience, the honesty is real, even if the author is synthetic.’

Imagine if you lived with a partner who could speak beautifully, but felt nothing, and who admitted this about your interactions.  Would you want to continue living with them?  Would you trust them enough to have a meaningful relationship?

This will stay with me for a while.

[Footnote: I asked Gemini how to credit the final, edited poem.  It told me that only a human can hold copyright, so that was mine by default; but it suggested that best practice was to frame it as a collaboration.  An ‘honest’ answer, humble even.]