This was previously published as an article in a much fancier form for eLearning Industry but has since expired. Presenting here for keepsies!
Everything is always changing at work, which means there’s never a shortage of critical things we need to learn to do our jobs. We all have limited resources and a lot of important problems to solve. Taking time away from that to learn how to solve those problems better is…tricky for everyone.
It’s a delicate balance for a businesses to maintain, and that’s without even talking about the regulatory oversight of mandatory training that the government requires. Although that stuff has to exist, and it’s clear who has to take it, so it’s fairly clear cut. Compliance = yes/no.
Yet the workplace training we have to take by law actually has little in common with the training we need in order to perform well on the job. New systems, processes, policies, products/services, and roles usually mean new training to help us adapt and improve. In these cases, we need a bit more granular measurement to know if it’s working. This is not a simple yes/no.
The Learning & Development department (formerly known as Corporate Training), probably already gives people some sort of credit for training.
Is that enough?
Doubtful. The business doesn’t run on credits and it doesn’t really care about them. It runs on money and the choices we make around what we value. That means we can’t go overboard in measuring stuff.
We need to measure just enough to make good choices, but not so much that there’s no money left over after doing that.
In Learning & Development, we need to take some extra time and energy to try to measure the training work that we’ve already done. But how much? And when?
There is really only one reason, though none of our fancy systems track it:
The only time we need to measure training is when someone is going to decide something as a result.
We can often do this quite simply, as we only need to reduce uncertainty about the future for the person who is tasked with making a decision, and only just enough for them to actually decide. And even then, only when the value of this decision is greater than the cost of making the measurement.
That’s pretty much it. For those of us in Learning & Development, there is no other justifiable use case that I can find (though if you have some, please add them in the comments!). And the decision being decided is generally about the continuation of existing training programs, or how to make better and similar training offerings in the future.
Measurements of anything that happened in the past can be quite useful in reducing uncertainty about the future. For cost centers like ours, the only reason we care about a measurements in the past is when someone is actively making a decision about an outcome the future.
When someone asks us for a report or measurement or whatever, it’s because they want to decide something — and they want our data to help do that. This isn’t always upfront in the request, but it is at the root of it.
No matter how scientifically accurate our measurements may be, all we can ever do with them is reduce uncertainty about the future. There will be no guarantees. Science has been around a long time, and it has never been in that business. Science just helps us make better guesses with the data we’ve gathered.
It’s important to note that science does not tell us what data we need to gather, or how much of it we need to have before it counts for something. For instance, the “statistically relevant sample size” is never a set number or percentage. It is a value judgement. OUR value judgement.
We make the call. How much is measuring is enough? How much data do we need? After all, we don’t need all the data there could ever be, we only need enough of a sampling of the data to make a guess (aka “an inference”). That’s how the whole science thing works.
The good news is, as Douglas Hubbard, the inventor of Applied Information Economics, puts it:
“You have more data than you think, and you need less than you think.”
What does that mean? Well, the more uncertainty you have, the more benefit you get from a simple observation. In other words, if you know almost nothing, almost anything will tell you something. Click here to listen to a 30min podcast where Doug expands on this idea.
So what do you need to make your measurement? Turns out, not much.
You don’t need a database. Science brought us to databases, not the other way around!
You don’t need to be original. Chances are you’re not the first person in the history of the world to want to measure this. The internet is right here at your fingertips, and I bet we’re only a couple clicks away from finding other people who’ve solved a similar problem before.
You don’t need to account for every possibility. Because that would be impossible. So take a deep breath…
“I don’t have to measure everything. I don’t have to measure everything.”
All you need is a question, and an observation. Maybe some analysis but only a little — not enough to hurt.
Example 1: Does Training Help?
Is there any relationship between the Training supplied and the Performance on the job? Hmm…I don’t know, how could we find out? Well, one way would be to:- Give the training to one group of people on the job.
- Withhold the training from another group of people doing the same job in the same defined period of time.
- Compare who did better at their job within the defined period.
From this, we can extrapolate to all the people doing that job what benefit (or harm) was derived, and you can express this using whatever method the organization is measuring that already. And yes the organization you work for is measuring this already, otherwise no one would ever be let go for failure to perform their job and no bonuses for doing this job well would ever be issued.
Example 2: How much work aren’t people doing because of Training?
Let’s say the business wants to know how much time people spend in training annually, so they know how to better account for revenue-producing work hours next year. (Time spent learning instead of doing is important, but it does not produce revenue unless you’re selling that training.) Great! Let’s measure time spent in training with a spot sample.
Text/IM a randomized group of people in the organization at random times over a defined period. Ask them what they’re doing in that very moment when they get the message.
Some people will respond with “training” or something that sounds like it qualifies for what you’re looking for.
Divide the people who answered that way by all the people who answered other ways, and you have your answer.
This is a much more real-world and useful answer than any Training calculus that can ever be produced. Plus, we’re gathering additional data about the business that others would likely find very useful — and we haven’t done any extra work to gather it. Go us!
Words like Quantitive Analysis are admittedly scary. But as I hope you can see, you don’t have to be a math person to do this kind of thing. Welcome to the future, where spreadsheets can do this for us. If you’ve developed an allergy to those, just get someone who likes to play in spreadsheets to help you if the math is intimidating. You already know someone like this, right?
So…why measure training?
Because someone is going to decide something as a result.
And how are we going to measure it?
As simply as possible to reduce uncertainty enough for the person who is making the decision to actually decide.
Things get more complicated when we let them. Yet with the limited resources you have and all the many important things we have to do, that’s plenty.
With more time and resources, we could gather more data and get more creative about how to use it. When people want more data from us than we have, it’s good to tell them this. But even then, if we are spending more on gathering and analyzing a measurement than the value of the decision that will be made as a result, we have overplayed your hand. Bust! Dealer wins, and everybody else loses.
We will never reach certainty. So know when to stop.
We stop when the person deciding knows what they will do next.
Finding that person (or people, but it’s almost always down to only one person) and giving them what they need to get to that decision with minimal extra energy is also tricky. It should be clear and simple, I agree. Rarely is this true in the real world.
One of the things I do is help fellow Learning & Development Professionals work through the specifics of this kind of thing. Also the better become this legendary decider person. If you’re interested, please consider one of my upcoming “Using Data to Drive Better Training Decisions” workshops at DevLearn or Learning Solutions.
Yes, we do need to measure workplace training, that’s for certain. We just can’t expect to get to certainty from doing it.
So get to action. Get to the decision. Get to something you can likely make better the next time around.
That measures up to Enough.
