A/B Test Shanty
A/B Test Shanty lyrics
A song about waiting forever for an A/B test to mean something
Lyrics
We set sail in the first of the year
With a hypothesis and a deck of slides clear
We charted a course through the traffic and tide
And we split it down the middle, fifty-fifty inside
The captain said orange, the first mate said blue
We gave it two weeks and we prayed it came through
But the confidence interval held at eighty-three
And neither shore was visible, neither shore was free
Haul away, haul away
Which variant do we take today?
Haul away, my weary crew
A was broken - so was B - and so were you
Haul away, haul away
The data drifts but the deadline won't stay
Heave her down, boys, heave her low
Neither wind nor p-value tells us where to go
Three months we sailed on an ocean of clicks
Ninety thousand sessions and none of 'em fixed
The uplift was phantom, it moved like a ghost
It'd peak on a Tuesday and leave us at most
A 1.2% lift with a margin of three
Which means in plain language it's telling us: nothing, you see
But the client was watching and the client won't wait
So we called it a winner and we sealed its fate
Haul away, haul away
Which variant do we take today?
Haul away, my weary crew
A was broken - so was B - and so were you
Haul away, haul away
The data drifts but the deadline won't stay
Heave her down, boys, heave her low
Neither wind nor p-value tells us where to go
[Lead] Have you ever run a test that you knew was too small?
[Crew] Aye, we've run it
[Lead] Have you ever watched the confidence bar hit the wall?
[Crew] Aye, we've watched it
[Lead] Have you ever named a winner that you knew wasn't right?
[Crew] God forgive us
[Lead] And shipped it to production in the dead of the night?
[Crew] God forgive us
God forgive us
God... forgive us
We were not the first crew to sail without knowing
We were not the last to read the charts wrong
The sea doesn't care about your confidence level
The sea only cares that you've been out here too long
And maybe A was better
And maybe B was true
But we were running out of runway
And the board was overdue
So we picked the one that felt less like losing
And we called it optimised, and we called it done
And we told ourselves the data was directional
And we sailed it toward a mediocre sun
Haul away, haul away
Which variant do we take today?
Haul away, my weary crew
A was broken - so was B - and so were you
Haul away, haul away
The data drifts but the deadline won't stay
Heave her down, boys, heave her low
Neither wind nor p-value tells us where to go
Haul away, haul away,
HAUL AWAY
We set sail in the first of the year
With a hypothesis and a deck of slides clear...
Behind the Lyrics
I started as a trainee on a barquentine and went on to become a navigational officer, taking merchant ships through the Panama and Suez canals more than once. Somewhere in all of that you learn to read two things above everything else: the instruments and the sea. The instruments tell you what's measurable. The sea tells you what's true. The job is knowing which one to trust when they disagree, and they disagree far more often than anyone at the academy admitted. The first A/B test I ran ashore, I recognised the feeling immediately. Chart the course, split the traffic clean, give it time. Then the data comes back sitting exactly between two shores, and the deadline is already in the room asking why you haven't picked one. At sea, an inconclusive reading isn't called "directional". It's called uncertain, and you either wait for clarity or make a judgement call and own it completely. Nobody on a bridge says "the radar data is directional". You'd be laughed off the watch. In a marketing meeting, with a confidence interval topping out at 83%, I have said "directional data" with a straight face. More than once. And the officer I used to be died a little each time. It took me about five years to understand that the deck officer and the marketer were the same person doing the same job in different weather. A shanty wasn't a style choice. Shanties are work songs, sung because the job is too heavy, too long and too uncertain to do in silence. You don't sing one because things are going well. You sing because the rope needs hauling and you're the ones hauling it. "We sailed it toward a mediocre sun" is the line I couldn't have written before living both lives. Not a failure, not a win: arriving somewhere technically fine because the deadline required it. The outro cuts off mid-line because the test never really ends. You just stop checking.
LSLachezar, writing songs so you don't have to cry alone
Production credits
- Text
- Lachezar Simeonov
- Audio
- Lachezar Simeonov, AI-assisted production
- Voices
- AI personas built for Clicks & Swagger, some inspired by Lachezar's real voice
- Art
- Lachezar Simeonov, AI-assisted production
Frequently Asked Questions
Up next
A song about vanity metrics that look green while revenue doesn’t
Get the new song every week
One new track every week. Confirm your email and you're on the list.
We respect your privacy. Your email is safe with us.
Continue exploring
Keep exploring or help choose the next marketing pain anthem.