Exploring The Strategic Application of xAPI Data to Learning Design

by | Jun 14, 2017

Three years ago Sean Putman and I were hanging out at a state park in North Carolina, with a group of really innovative L&D professionals, and one of the major topics of conversation was xAPI.

It was very early in the life of the spec, so there was a lot of discussion of use cases, how to encourage adoption as well as technical and security concerns. 

Many of the ideas tossed around that weekend went to the core of why xAPI was so exciting: connecting the big picture and the personal aspects of learning is a compelling proposition.

That ability to look at learning offerings in meaningful contextual ways for organizations and in terms of individual impacts, is also a big part of the motivation behind our new book, Investigating Performance: Design and Outcomes with xAPI.

If you were at LSCon back in March of this year, then you may recall our conference session in which we explored some of the basic principles of effective data use and how to provide meaningful feedback, covered in the book:

View on SlideShare to see the additional slide notes

Connecting the Big Picture and Personal Aspects of Learning

The promise for both big picture and individual benefits was compelling to me. Despite my interest and research around how people learn, I am a data-person primarily, and keenly aware of the potential impacts of having the right data readily available.

From work in competitive intelligence, I’d seen the huge information disconnects which exist within businesses; both individuals and entire organizations are hurt by the lack of easily accessed and analyzed information.

Interviewing people across five roles in a company can end up feeling like one is speaking to five different businesses, not one. This has tremendous costs in terms of employee growth and satisfaction as well as organizational success.

I saw xAPI as a way for organizations to gain deeper internal understanding – by seeing what aspects of learning actually impact performance, there is the potential to understand what the real learning needs are to support success for employee and employer alike.

This ties into the more obvious potential for xAPI. We can cite cognitive science until we are blue in the face, talking about how to best support meaningful learning.

At the end of the day there is the issue of accountability: we need to provide some form of assessment.

A Better Means of Assessment

It’s nearly impossible to implement new approaches to learning design if we are stuck with the same old metrics of test scores and completions. We can’t offer a better learning experience until we have better means of assessment.

xAPI was the key that could open the door to more effective learning.

To build better assessment and to increase internal organizational insight, I thought a lot about how to design courses, data streams, and analysis methods to get meaningful data.  At the same time, my co-author, Sean, was looking at xAPI as a starting point to do data-driven course design — the path to a powerful feedback loop.

I was looking at how to design for data; Sean was looking at how to design from data.

We realized that we were working on the same problems from different directions and, for both of us, xAPI was the starting point to support better learning opportunities.

Going Beyond the Promise of xAPI

During that weekend, conversation went beyond the promise of xAPI and moved into explorations of the potential obstacles and difficulties involved in getting started.

Like any technological innovation, the mechanics of implementation are the easy part; it’s the strategic plan and subsequent activities which determine success or failure.

From those conversations, the outline of Investigating Performance emerged.

The initial concepts and content evolved over the next two years as we gained experience working with xAPI. In that time, we’ve had valuable discussions with other early adopters about what works, what doesn’t, and what we wished we’d known when we got started.

While the technical aspects of an xAPI implementation are not unduly complicated, and L&D professionals do not need developer level skills, understanding the fundamentals of the spec matter, it is important to understand the basics.  

Sean does an excellent job of translating the spec into plain English so that non-developers can use it effectively.

He looks into the important topic of profiles (the rules and documentation needed to implement xAPI for one’s given use case), and the associated vocabulary needed to assure that the data you collect meets your analysis needs.

Designing for Data

Designing for data starts with the big picture: evaluating business goals which underlie instructional design goals; looking at the needs of one’s learning customers and data customers to understand how to measure success.

Designing for data needs one to be forward-thinking, concerned with what actions and decisions the data will inform, not just concerned with documenting what has already happened.

Designing from Data

Designing from data is the other side of that coin. xAPI data can provide a wealth of information that supports nuanced evaluation of not just the users, but of the course itself.

What elements of a course resonate with users? Are there design flaws or technical issues that impede user success? What elements of the course can be tied to improved performance?

Exploring xAPI’s Strategic Opportunities

Writing Investigating Performance turned out to be a powerful learning experience. It pushed us to explore questions more deeply than we would have otherwise; and really dig into the challenges and benefits of xAPI; not in a theoretical way, but from wrestling with data in messy real-world situations, and finding unexpected insights.

Those insights came not just from our data but from the work or others, as is shared in case studies included in the book.

Getting started with learning data is a lot easier when it is not a solo effort. We benefitted greatly from conversations with colleagues about real world concerns regarding data stewardship, and the value of data for improved course design.

Writing the book was a constant reminder of the intersection of the big picture and the personal.

Using data to understand both learner performance and course performance fits into a larger ecosystem that is bigger than any learner or any course; and yet it is the awareness of that big picture that allows us to better serve each individual, and improve each course.

At the end of the day, the point of the book is not so much to be a technical manual, as it is a starting point to explore the strategic opportunities which xAPI offers to L&D professionals.

If that sounds like something you’d benefit from, then you can purchase the book – in paperback or digital print – and start exploring those opportunities today!

And of course, if you find you need need a little more help getting to grips, myself and the rest of the HT2 Labs team are just an email away!

About the Author

Janet Laane-Effron

Janet Laane-Effron

Data Scientist

Janet is our in-house data scientist. She has a penchant for applying diverse fields such as cognitive science and competitive intelligence to develop best practices in learning design & developing meaningful metrics for performance analysis. Basically, if you want some clever insight into your organisation’s learning, Janet’s your go-to person! Although not based in the UK, Janet makes up for it with some virtual cycling around rainy London each morning.

View all posts by Janet











Pin It on Pinterest

Thanks for reading!

Help us spread the love and share this post with your network