If you’ve ever hosted a party, you know that things go better with a bit of planning.  Not just the obvious planning such as having enough food and drink on hand, and a selecting reasonably compatible guest list; it’s also essential to have your place set up so that guests can enjoy what you’ve provided and find space to socialize.  If they can’t get to the drinks table, or can’t mingle, it’s not going to be a very successful evening.

It’s pretty much the same thing with online learning…

In a Curatr course we can have our guests (users) and we can have our materials planned out, but those two elements are not sufficient to support an effective social learning environment.

Over the past couple of years, we’ve spent a lot of time looking not only at what goes into courses, but how to set them up, so that the physical design of the course helps the party really take off.  We’re looking beyond how to set up our virtual buffet table (the course objects) well laid out: we’re also wondering do we, in effect, have the furnishings set up so that folks can take a plate of what we have on offer and easily go mingle with the other guests?

Or in plain English: does the placement of the comment section have an effect on the amount of conversation within a course?

How Can Page Design Best Support Social Learning?

We decided to take a look at a couple of our recent MOOCs and see if there was any variation in the amount of conversation depending on whether the comment section was placed below the content or next to it.

Commentst positioned below the content

Comments were positioned under the content in the Learn xAPI MOOC

Comments to the right of the content

Comments were positioned to the right of the content in the Outstanding MOOC

We started by simply looking at the average (mean) and median number of comments per user; this was done both including and excluding data for administrators and tutors in order to gauge actual participation.

Comments Beside Content
(OutstandingMOOC)
Comments Below Content
(Learn xAPI)
Mean = 9.5 Mean = 9.0
Median = 4 Median = 5

Which, especially given the sample size and the variables at play, was not a significant difference.  

However, when we started looking at the the rate of replies, the actual amount of conversation going on, that’s when things got interesting…  

Reply Rates

The percentage of comments which were replies was 46% for the course with comment section next to the content (Outstanding MOOC), and 36% for the course with comments below the content (Learn xAPI).  

When we removed the comments from the Admins and Tutors, and looked only at participant comments, 22% of comments were replies in Outstanding MOOC, compared with 12.5% for Learn xAPI).   

So, in comparing these courses, it appears that although the placement of the comments did not significantly impact the number of comments, it did have an effect on the amount of interaction within the comments; in the context of social learning, this was particularly interesting.  

It may be that people seeing conversation going on (clearly in view on the side bar) felt more inclined to join in discussion rather than simply post a comment on an object; it’s definitely something we’ll continue to study.

Our data says…

If you’re looking to stimulate conversation, taking a little time to investigate how your design affects participation can pay off big dividends.

There’s no one-size-fits-all answer, and a single change on a page has cascading effects (for these two layouts, the change in comment placement affected the UI of the whole page); but reviewing your data gives the opportunity to optimize for your particular scenario.  

Comments and Completion Rates

We also took a look at comment rates and course completion, where there were some notable differences between the two courses.

For the course How to Create an Outstanding MOOC, there is a significant amount of participation even by those who do not go on to complete a given Level of the course.

Outstanding MOOC average comments per user by level

For the course Learn xAPI, we see a rather different behaviour.  In this case, and particularly in the 2Tech Track” of the course, those who were actively commenting were those who went on to complete levels; it was more of an all-or-nothing scenario.

Learn xAPI Strategy Track average comments per user by level

For Learn xAPI, it seemed people were either fully engaged (commenting and completing) or not engaged (few comments, not completing), whereas for Outstanding MOOC, even those who were not completing levels were actively commenting.   

Learn xAPI Tech Track average comments per user by level

In some situations, course completion is an important goal for the designer.  But often in MOOCs and in Social Learning situations in general, participation is more driven by the individual needs or goals of the learner; they engage with the areas that interest them or answer their questions at that time.  

These results are giving us some useful indications of the ways that page layout can support social learning, and learner engagement.

Our data says…

Although some of this variation may have also been related to subject matter it was interesting to observe the differences in participation and completion rates between the two courses.  It’s worth investigating comment placement may encourage higher rates of engagement even for those who do not have time or inclination (or some other reason) to complete the course.

If users who join the course with the goal of meeting an information need (rather than for course completion) stay to engage in reflection and conversation in the comments, section, that can deliver longer term benefits in understanding, encoding and retention.

What ‘Success’ Looks Like In Social Learning Situations

This brings us back around to the notion of hosting a party – we don’t declare it a successful function only if everyone talks with everyone else, and samples everything on the buffet table.  It’s successful if people enjoy a few of the offerings and find some good conversations. 

With MOOCs and social learning, we have the chance to throw a really good party – one where people leave with new ideas, new understanding, and new motivations.  Conversations are an important factor in real learning, and that’s strong incentive to keep learning how to support those through both instructional design, and page design.

Find out more about social learning – download our free Social Learning Playbook.