Performance Measures that Worked – A Case Study From Major League Baseball

Would you rather watch the video than read?  If so, just click here.  4 mins with captions.

Managers introduce measures because they’re paid to get results…and often it goes nowhere.  This is a case study of where it worked, and it comes from Major League Baseball.

Baseball – Balls and Strikes

First – the game of baseball.  Robin Williams said…

…so baseball is like test cricket on speed.

While you might know baseball as hitting, fielding and running around the bases to score runs, the key moment happens over 300 times a game when the pitchers throws (‘pitches’) the ball to the hitter.  If not put into play, each pitch is a ball or a strike.  Three strikes and you’re out, four balls mean you get to go to first base.

Strikes come from swinging and not putting the ball in play…or are ‘called’.  A called strike is when the batter doesn’t swing, and the ball passes over what’s called the home plate and is between the bottom of the knees, and the halfway point between the shoulders and the top of the pants.

That is, it goes through this box here:

Now keep this in mind – major league fastballs can now exceed 100mph or 160kmh, with other pitches that break sharply in the air moving at above 80mph or 130kmh.  This is what umpires, seen in the picture above squatting behind the catcher, have to judge, and they have less than 0.5 of a second.

Tough gig!

Technology

Since the 2000s, pitch-tracking technology has come into existence, and it’s now possible to see with extreme accuracy the trajectory, spin-rate and speed of every pitch, and whether it passed through that 3D box called the strike zone.    

This isn’t the official thing…but it gives you the idea…

…so in 2009, after the technology had been used for a few years, Major League Baseball made the decision to start using it to improve the performance of umpires in calling balls and strikes.

Who Likes Having Their Performance Measured?

What’s the usual response to having performance measured?  Remember, in most situations, those deciding that performance is now going to be measured don’t have to change much.  To them, it’s a ‘change’ issue.  For those that will now have their performance measured, however, the change can be huge.

It doesn’t matter whether the previous performance level was sufficient or not, what matters is that people are being asked to change, by other people who don’t have to suffer the same amount of change.

Doesn’t make bringing in performance measures wrong, or insurmountable…but we can’t live under the illusion that they will be welcomed.

It Worked!

So, how did Major League Baseball do?  Working from a couple of brilliant papers by Brian Mills of the University of Florida at the time, here’s the graph:

 The top line shows percentage of balls that were correctly called…and the lower line is the percentage of strikes that were correctly called.  Notice the steady progression in strike accuracy from 2009?  That’s a result.  Our question is….why did this work?

The Factors In Play

These are the particular factors I noted in terms of how Major League Baseball went about doing this.

A Standard Was Set

Major League Baseball did set a required performance standard.  An expectation on percentage of correct calls.  And they did tie some rewards to this, a salary one, and that performance in this area would be an input into who would get the sought-after playoff games.

A key point here, however, was the umpire’s union negotiating that results from the technology tracking would only be an input into judgement of performance.  The start of a conversation, not the end.  As Sarah Connor taught us, no one wants their future decided by a computer.

The above is not particularly new, however.  These next two are crucial:

Feedback Was Immediate and Personal

Umpires started receiving feedback immediately after the game.  Not at the end of the month, or even worse, at the end of the year.  Straight after the game.  The raw data.  This has the significant effect of moving the measure from what I call an ‘Accountability Hammer’ to being a feedback loop on how we’re doing.

This change is massive.  It moves the measure from something to be avoided or justified, into being more like checking lap times to see how we’re going.  We can see movement in directions quickly.  And we become more open to adjustments.

Plus…the feedback was given personally.  A conversation was held.  It wasn’t a cold readout on a screen coming in way too late.  An actual human connection.

Training Was Available but Not Mandated

Major League umpires are generally not a young crew, and even less so 10 years ago.  And like umpires in all sports, they are proud of their profession, and they work in a job that is pilloried and generally treated without the utmost of respect.  So, a new training program from head office is not going to be top of their list of ways to spend their time.

Yet…training is often how people get better.  So how was this handled?

By making decent training available but not mandated.  Which treated them like professional adults.  Not children being sent to class.

No compulsory program.  No ‘roll-out’ of the latest thing that had grabbed management’s attention.  Just decent quality resources that would help performance of the actual job, available for those who wanted or needed to put in the work to improve. 

And they did.

(You might be interested to know that the performance improvements have plateaued over the last five years.  Not gone backwards, but have levelled off.  This perhaps suggests a limit to the current level of human performance in the way thing are currently done, requiring a version of breakthrough to find a new level).

Bringing it Home

Notice the 1-2-3 in play here.  There was a standard.  The measure created a feedback loop.  An avenue to improve was provided but not mandated.  All three are needed for the bringing in of any measure/KPI/target to work.

Take a quick look at how your current measures work – in other words, what you are currently subjecting your people to.  See if you can make some adjustments using the experience of the umpires in Major League Baseball.

We don’t like being under the Accountability Hammer.  Being treated like kids being sent to detention class triggers natural aversion.

But we generally don’t mind a bit of info letting us know how we’re doing and having a way to improve.

And that’s what measures are really for.

Batter-up!