Posted

This was previously published as an article in a much fancier form for eLearning Industry but has since expired. Presenting here for keepsies!

Everything is always changing at work, which means there’s never a shortage of critical things we need to learn to do our jobs. We all have limited resources and a lot of important problems to solve. Taking time away from that to learn how to solve those problems better is…tricky for everyone.

It’s a delicate balance for a businesses to maintain, and that’s without even talking about the regulatory oversight of mandatory training that the government requires. Although that stuff has to exist, and it’s clear who has to take it, so it’s fairly clear cut. Compliance = yes/no.
Yet the workplace training we have to take by law actually has little in common with the training we need in order to perform well on the job. New systems, processes, policies, products/services, and roles usually mean new training to help us adapt and improve. In these cases, we need a bit more granular measurement to know if it’s working. This is not a simple yes/no.

The Learning & Development department (formerly known as Corporate Training), probably already gives people some sort of credit for training.

Is that enough?

Doubtful. The business doesn’t run on credits and it doesn’t really care about them. It runs on money and the choices we make around what we value. That means we can’t go overboard in measuring stuff.
We need to measure just enough to make good choices, but not so much that there’s no money left over after doing that.

In Learning & Development, we need to take some extra time and energy to try to measure the training work that we’ve already done. But how much? And when?

There is really only one reason, though none of our fancy systems track it:
The only time we need to measure training is when someone is going to decide something as a result.

We can often do this quite simply, as we only need to reduce uncertainty about the future for the person who is tasked with making a decision, and only just enough for them to actually decide. And even then, only when the value of this decision is greater than the cost of making the measurement.

That’s pretty much it. For those of us in Learning & Development, there is no other justifiable use case that I can find (though if you have some, please add them in the comments!). And the decision being decided is generally about the continuation of existing training programs, or how to make better and similar training offerings in the future.

Measurements of anything that happened in the past can be quite useful in reducing uncertainty about the future. For cost centers like ours, the only reason we care about a measurements in the past is when someone is actively making a decision about an outcome the future.

When someone asks us for a report or measurement or whatever, it’s because they want to decide something — and they want our data to help do that. This isn’t always upfront in the request, but it is at the root of it.

No matter how scientifically accurate our measurements may be, all we can ever do with them is reduce uncertainty about the future. There will be no guarantees. Science has been around a long time, and it has never been in that business. Science just helps us make better guesses with the data we’ve gathered.

It’s important to note that science does not tell us what data we need to gather, or how much of it we need to have before it counts for something. For instance, the “statistically relevant sample size” is never a set number or percentage. It is a value judgement. OUR value judgement.

We make the call. How much is measuring is enough? How much data do we need? After all, we don’t need all the data there could ever be, we only need enough of a sampling of the data to make a guess (aka “an inference”). That’s how the whole science thing works.

The good news is, as Douglas Hubbard, the inventor of Applied Information Economics, puts it:
“You have more data than you think, and you need less than you think.”

What does that mean? Well, the more uncertainty you have, the more benefit you get from a simple observation. In other words, if you know almost nothing, almost anything will tell you something. Click here to listen to a 30min podcast where Doug expands on this idea.

So what do you need to make your measurement? Turns out, not much.

You don’t need a database. Science brought us to databases, not the other way around!
You don’t need to be original. Chances are you’re not the first person in the history of the world to want to measure this. The internet is right here at your fingertips, and I bet we’re only a couple clicks away from finding other people who’ve solved a similar problem before.

You don’t need to account for every possibility. Because that would be impossible. So take a deep breath…

“I don’t have to measure everything. I don’t have to measure everything.”

All you need is a question, and an observation. Maybe some analysis but only a little — not enough to hurt.

Example 1: Does Training Help?

Is there any relationship between the Training supplied and the Performance on the job? Hmm…I don’t know, how could we find out? Well, one way would be to:
  1. Give the training to one group of people on the job.
  2. Withhold the training from another group of people doing the same job in the same defined period of time.
  3. Compare who did better at their job within the defined period.

From this, we can extrapolate to all the people doing that job what benefit (or harm) was derived, and you can express this using whatever method the organization is measuring that already. And yes the organization you work for is measuring this already, otherwise no one would ever be let go for failure to perform their job and no bonuses for doing this job well would ever be issued.

Example 2: How much work aren’t people doing because of Training?

Let’s say the business wants to know how much time people spend in training annually, so they know how to better account for revenue-producing work hours next year. (Time spent learning instead of doing is important, but it does not produce revenue unless you’re selling that training.) Great! Let’s measure time spent in training with a spot sample.

Text/IM a randomized group of people in the organization at random times over a defined period. Ask them what they’re doing in that very moment when they get the message.

Some people will respond with “training” or something that sounds like it qualifies for what you’re looking for.

Divide the people who answered that way by all the people who answered other ways, and you have your answer.

This is a much more real-world and useful answer than any Training calculus that can ever be produced. Plus, we’re gathering additional data about the business that others would likely find very useful — and we haven’t done any extra work to gather it. Go us!

Words like Quantitive Analysis are admittedly scary. But as I hope you can see, you don’t have to be a math person to do this kind of thing. Welcome to the future, where spreadsheets can do this for us. If you’ve developed an allergy to those, just get someone who likes to play in spreadsheets to help you if the math is intimidating. You already know someone like this, right?

So…why measure training?
Because someone is going to decide something as a result.

And how are we going to measure it?
As simply as possible to reduce uncertainty enough for the person who is making the decision to actually decide.

Things get more complicated when we let them. Yet with the limited resources you have and all the many important things we have to do, that’s plenty.

With more time and resources, we could gather more data and get more creative about how to use it. When people want more data from us than we have, it’s good to tell them this. But even then, if we are spending more on gathering and analyzing a measurement than the value of the decision that will be made as a result, we have overplayed your hand. Bust! Dealer wins, and everybody else loses.

We will never reach certainty. So know when to stop.
We stop when the person deciding knows what they will do next.

Finding that person (or people, but it’s almost always down to only one person) and giving them what they need to get to that decision with minimal extra energy is also tricky. It should be clear and simple, I agree. Rarely is this true in the real world.

One of the things I do is help fellow Learning & Development Professionals work through the specifics of this kind of thing. Also the better become this legendary decider person. If you’re interested, please consider one of my upcoming “Using Data to Drive Better Training Decisions” workshops at DevLearn or Learning Solutions.

Yes, we do need to measure workplace training, that’s for certain. We just can’t expect to get to certainty from doing it.

So get to action. Get to the decision. Get to something you can likely make better the next time around.

That measures up to Enough.

Author
Categories ,

Posted
Comments 0

Our universe is a squishy one
It acts in probabilities over absolutes
“How much of the time?” we wonder
“Exactly” it jokes “Now you see!”

Reality is all gradients
Drill down on any one thing
And soon you’ll find
It isn’t even there
Things dissolve to into relationships
Tendencies are all we have
All we are

But we count discretely
Orienting to objects
Seeing entities
Expressing data
As if it were there

Representations work well enough
Except for the times they don’t
Whenever they don’t
We may humbly retrace to the question
To finally reveal our answer
It was there all along

When it comes to data
Encoding is foreboding
How we decide what counts
Determines what is counted

As we define, so do we create
Revealing patterns in data
To help us better pretend
That we understand
What truly is

Author
Categories ,

Posted
Comments 0

I’ve discovered that 10:00 UTC is The Best Time In The World.

Given that there are 37 different time zones ranging from UTC-12 to UTC+14 (many of whom observe daylight savings). And given the distributions of the world’s population being smallest on either side of the International Date Line. 10:00 UTC is the one time that will occur on the same date for the largest possible population of people on Earth.

If you happen to live between Hawaii and New Zealand you might still be on a different date depending on who’s doing daylight savings and if they’re in the Northern or Southern Hemisphere (sorry I-Kiribati). For everyone else, 10:00 UTC will occur on the same calendar date.

Why does this matter?

Ok, maybe it doesn’t for you. But it matters to me because I only get one time that I can specify for all our automated messages to fire for learning assignments. Our LMS won’t let us do a relative time based on when any given person starts their day. In order to say something like “This training launches on August 1st” and actually launch it on August 1st for everyone getting that message from Auckland to Hawaii the long way around, it has to be 10:00 UTC. No other time will always work.

It took more work than I thought to figure this out. You’re welcome ;)

(This was posted at 10:00 UTC)

Author
Categories ,

Posted
Comments 0

It took me a lot longer than usual to type this simple sentence, but it also took a lot less movement.

Why? Colemak, that’s why.

It’s an alternate keyboard typing layout. One of those “go slow to go fast” things. This is my first time typing like this in a long while so it is taking me a long while ;)

Here’s what the Colemak website has to say about the advantages of it:
  • Ergonomic and comfortable – Your fingers on QWERTY move 2.2x more than on Colemak. QWERTY has 16x more same hand row jumping than Colemak. There are 35x more words you can type using only the home row on Colemak.
  • Easy to learn – Allows easy transition from QWERTY. Only 2 keys move between hands. Many common shortcuts (including Ctrl+Z/X/C/V) remain the same. Typing lessons available.
  • Fast – Most of the typing is done on the strongest and fastest fingers. Low same-finger ratio.
  • Multilingual – Allows to type in over 40 languages and to type various symbols, e.g. “pâté”, “mañana”, €, em-dash, non-breaking space.
  • Free – Free software released under the public domain. You don’t have to buy a new keyboard, just install a program.

My recent Keyboardio post inspired giving this a shot again. Try it for yourself!

Author
Categories ,

Posted
Comments 0

Whether or not a particular English Lord, Judge, and Historian said the following around 200 years ago, it’s still worth considering the cycle linked to the name of Sir Alexander Fraser Tytler.

A democracy is always temporary in nature; it simply cannot exist as a permanent form of government. A democracy will continue to exist up until the time that voters discover that they can vote themselves generous gifts from the public treasury. From that moment on, the majority always votes for the candidates who promise the most benefits from the public treasury, with the result that every democracy will finally collapse due to loose fiscal policy, which is always followed by a dictatorship.

The average age of the worlds greatest civilizations from the beginning of history, has been about 200 years. During those 200 years, these nations always progressed through the following sequence:

From Bondage to spiritual faith; From spiritual faith to great courage; From courage to liberty; From liberty to abundance; From abundance to complacency; From complacency to apathy; From apathy to dependence; From dependence back into bondage.

I am not a Historian by trade or training, but informally I’ve always been very interested and well-researched in history. I spend a lot of time learning from it, and interpreting what has come before. From this I’ve come to understand that we’re not so different from our fellow humans at any point previous to this era. Our context has changed, our culture has evolved, and we as individuals can and do grow a great deal — but we as a species have not (and likely won’t have the chance for that same reason).

When I look at this lifetimes-long cycle, it certainly makes sense to my reading of history as well. I can’t seem to find any exceptions yet. And though the details are different, the core structures we create are not so new as we like to think.

Perhaps it’ll be different this time. Maybe we really are exceptional?

Believing this makes us sound really a lot like those who came before, though. It doesn’t make us sound at all like a people who actually are different, or really would be exceptional.

I believe we’re repeating this cycle. And it’s really easy to spot where were are on it too. I’d say we’re at about 8.5 on this 9pt scale. I believe I’ll live to see us pass the Point of No Return.

What do you think? The comments are here for you, my friends.

Author
Categories ,