New technology accelerates modern learning so that learners are able to collaborate, share, practice, and reflect on what they are learning and how they are applying new skills and knowledge on the job. But how do you measure modern learning so that you can prove and improve the impact of these increasingly common experiences?
Truly Determine “Success”
One best practice is taking the time to determine what success is for the modern learning experience. Too often, we jump right into what we want the learner to experience and the content for the program. We need to set aside time to determine what we would like people to do differently after the experience. This can take place in a hackathon or design session where you partner with stakeholders to further explore and define success. The success criteria are often behaviors that will change after this experience and could also be outcomes. Defining success will help you design the program as well as help determine what to track after completion.
In addition, you can take the time upfront to define engagement and participation in the modern learning experience. Is logging into the learning platform once a week good enough? What does a learner need to complete so we can say that they are engaged? Once you have an idea of what success is, you can then choose the data collection instruments to see how learners are moving toward the goal.
You can also use the gamification element, or leaderboard, to measure success. Rather than using just the top 10 points winners, I recommend using the average number of points per learner, or percentage of learners above a threshold. When thinking about what to measure, consider how you would share the results with a stakeholder. For example, you could share that 95% of the learners earned 250 points in the experience, which is better than sharing a leaderboard which shows 10 people earned over 1,000 points.
Share Learner Insights With Stakeholders
Another best practice is to share the comments generated by the learners in the modern learning experience with stakeholders. The comments help tell the story of how learners are applying their learning experience on the job. Comments, which can be collected within the discussion boards, through an online survey, or with a focus group, can also help you market to skeptical managers and learners who still want classroom learning.
Analyze the Laggards
We should also take a closer look at the learners who have signed up for the modern learning experience, but who are not participating and engaging as frequently as the rest of the group. We need to improve the program so it's effective for most—if not all—of the learners. I would recommend looking for common characteristics of the non-engaged. We are looking for barriers that keep them from participating, such as:
- Are they involved in other learning initiatives? It could be hard to participate in more than one at a time.
- Do they all have the same manager? It could be that the manager is not holding them accountable, or does not see the value in the learning experience.
- Were they assigned the program or did they choose to participate? If they were assigned, then perhaps we can communicate the benefits of participating.
In addition to sharing the success of the program, we can improve modern learning experiences by identifying obstacles and marketing the benefits.
The final best practice is just getting started with measuring your modern learning experiences. Determine things you would want to know about the program, then find the data to answer your questions. The technology and learner activity probably produces the data you need to answer key questions and make improvements.
Scott Weersing is Director of Learning Analytics at GP Strategies.