How Much Better?

How much better does a putter (the club) need to be, for a player to make the decision to change?

Good question. But it needs to be seen from several perspectives…

In the past, how much better was even possible? Well, all putters were within just a few percentage points of each other in make probability. In fact, no putter in the world was even 5% more likely to make a putt (in the scoring zone, from 12 to 36 feet) than the average putter.

Now, how would you even know if a putter was really even 5% better? If a Tour professional makes just 18% from 18 feet, then making 19% would be just more than 5% better… (19/18 = 1.0555 = 5.55% better).

One big problem though… to see something as “true” at that level, you need a mountain of data.

If a Tour pro was on a putting green with the Dean of Mathematics (Professor) at a top University, and said, “Hey Professor, I just made 16, 18, 20, 14, and 22 of 100 from 18 feet with my putter (500 putts), and then made 17, 19, 21, 15, and 23 (500 putts) with this new “Offset White Coated CNC Milled German Rusty Steel Fanged Flange Blade Belly Hole Hunting Incredible Iomic Insert Heel & Toe MOI Super Stroker Tour Special… with a Compass in the Grip… so I’m always one putt better, and I’m too tired to hit another 1000 putts. What should I do?”

What should the Professor say in reply? (Besides the obvious… shades of Ralphie…)

He should say… “So Pro, you made 90 of 500 with one model, and made 95 of 500 with the other. If I follow the formulas that were established by the smart guys several centuries ago, that deal with this issue, here’s what they tell me.”

He’d whip out his calculator, and run the numbers in his pre-programmed Binomial Probability Distribution Application. Plugging in probability of make being 0.18, trials at 500, and successful makes at 95, the data spits out the numbers…

Before replying, the wise Professor asks… “So Pro, how sure do you want to be, before you decide to switch?”

The Pro, ever mindful that his reputation, livelihood, and house payment are all on the line with this decision, says… “99%. I need to be 99% sure.”

The Professor simply nods no… “Sorry Pro, you’re not even close to collecting enough data for this level of certainty. In other words, if it stays like this… 18 to 19 score, you’d be putting for a while.”

“Well, it’s my life on the line here…how long would I have to stand here to get to 99.95% sure? Twice as long?” says the Tour Pro.

The Professor looks at the Pro, then back in the calculator, and thinks… well, at 500 trials, he’s only at about 70% certain that the new BB Gun is better. Time for some iterative calculations… “No Pro, it’ll be a lot higher… maybe 10x the test might do it.”.

The Professor plugs in 5000 putts with each… 900 makes vs. 950… but… “Not quite there yet Pro… that would only get you a bit higher than 96% sure.”

“Shut the Front Door!” yells the pro. “I gotta stand here and hit 10,000 total putts, at a 5% differential, and I’m still only just 96% sure?”

“Yup”…

“Wow. I’ll be here forever.”

“Right, says the Professor… and the other variables involved will dominate, rendering the whole exercise questionable.”

“Forget the other variables Prof, just tell me how long at 5% better would I need to test… I’ll do it on my indoor putting carpet.”

“OK Pro… lets try this… 6,000 putts with each… no… 7,000… no… 8000… no… 9000… no… 10,000 putts with each…

“How about this pro… at 10,000 putts each, you’re up to 99.5% sure. How’s that?”

“Not enough… I need 99.95% sure!”

“Sorry Pro, you’ll never get there. You would need to hit 10,000 putts with each, and be at 18% with one, and 19% with the other, in a fair trial, to have enough differential that one putter was actually superior to the other, by the 99.5% certainty level… it’s not some small margin to go from 99.5% to 99.95%, it’s huge.”

“That’s OK Professor… 5% more one-putts is worth about a Million Bucks to me… I’m doing it. I’ll hit a Million putts if I have to. I want to be 99.95% sure it’s better.”

The Professor warns… “Well pro, even if you do it, there are some problems when testing for that long… one thing is, your own performance variable will probably dwarf the putter differential. I advise instead you consult with our Physical Sciences Department… but good luck anyway.”

Now… what did the above teach us? With putters in the past, it was basically impossible to determine which was better, in actuality. So, it largely WAS a “feel and look” exercise!

But not any more.

We actually tested one future Hall of Fame Tour Pro. He doubled his makes (100% better) with the Black Swan in a test, over his own putter. He even said “I don’t want your putter to win, David”. That was fine with us… and when he made 26 of 120 (22%) with our putter from 30 and 26 feet, and 13 of 120 (11%) with his own putter… we gave him the Professor’s numbers… they weren’t understood, obviously.

He was at the 99.95% confidence level that the Black Swan was superior. How much superior? Clearly not 100%. But at a 99% level, how much better would he have to have been? If he had made only 22 instead of 26 with the Black Swan, we could easily state that he would have a 99% certainty that the root cause of the increased makes were the putter. All the same… he did make 26. He doubled his makes. 11% from that distance was good… about Tour averages. So he didn’t putt poorly with his favorite putter. He just blew it into the weeds with ours. All while he was breaking his routine every time he made two in a row with the Black Swan. He really didn’t want it to win, you needed to be there to see it.

Then, he walked over to another Tour Pro who was using it, hitting nine-footers and making them all. He says… “so, it works on long ones, what about those short ones?”

“Better!” came the answer from the player who debuted the putter right after this occurred.

If the Tour Pro understood the math, he’d use it and have an edge on the field. Some do (and use it), some don’t (even if they proved it to themself)… until the rest of the world starts beating them with it. It will happen. For as little as it’s been tried, it already has.

Was the PGA Tour Sony Open debut of the Black Hawk a fluke? Not likely.

Was the PGA Senior debut of the Black Swan a fluke? Not a chance.

We did test with 60 top players (the 100% better example above was the 60th). The average of all 60?

43% more one-putts.

The certainty our putter is better? Well, let’s put it this way… it’s 5 billion to 1 odds in our favor… and Your’s… if you put the Black Hawk or Black Swan in your bag.