Dan's Blog

March 4, 2010

Discussion Boards in elearning.

Filed under: education,elearning — ddswanson01 @ 7:47 am

A few years ago, I participated in an online program, given by the University of Wisconsin, Madison, on ‘How to Teach Online’. We used one of the dominant online learning platforms in the market at the time, so the participants got to find out what it was like to be a student in an online course, in a distance learning environment. One of the biggest time-sinks was the discussion board. I tried to read every post my fellow students made, and I had to log on several times a day to keep current, or I could easily get behind to the point that it took me 45 minutes or longer to catch up.

Participation in the discussions was part of the grade; the instructors had some rubrics for judging both quantity and quality of posts. In discussions, the instructors indicated that evaluating discussion responses took more time than even preparation for the course. At the time, there weren’t many tools included to help in this evaluation. I’ve unfortunately not familiar with the cutting edge improvements in online learning tools, so I hope this post covers new ground – it’s new to me, at least.

After my first couple of days reading the discussion board, I asked my fellow learners if perhaps we could institute a rule that said ‘If all you are going to say to a comment is “I agree” please don’t post.’ There was a lot of pushback on this; for some people, agreeing with what was said was their major contribution to a discussion. I didn’t think of it at the time, but another reason for these short comments is affirmation. Some people are shy about posting (not me!), and seeing several people agreeing with their posts made it easier for these folks to post subsequent messages. But to me, it wasted a lot of screen space, plus my time, having to scroll through all these short messages with virtually no content.

Facebook has addressed this nicely with the “Like” button. This is a neat way for those whose only response to a post is “I agree” to provide that response without taking up screen space. It does more than that; it counts the number of people who agree.  This might be something instructors are interested in.  Combined with other information, this tool might help evaluate student participation in discussions.

The discussion board in this particular platform came with only two tools that I knew about to help evaluate discussions; they counted the number of threads a particular student started, and they counted the number of times a student posted a comment. These didn’t help much in performing a qualitative analysis, though. I don’t know if there was a tool that counted the number of responses a thread got, but a tool that counts ‘significant’ responses to a thread and gives the instructor a way to define ‘significant’ might be useful in the qualtitative analysis of discussion board postings.

What about qualitative analysis, assisted by the computer? It could be useful to see how many keywords each student used in a particular thread, but defining keywords in advance takes a lot of time and thought. How about a tool that reads through a thread when it is finished (or even still in progress), and provides the instructor with a list of words used in that thread?  The list would be appropriately filtered, to keep out ‘a’ and ‘the’, etc, and sorted by frequency of use.  The instructor would select keywords from the list appropriate to the thread and save them in a file associated with that thread.

The instructor can certainly ‘prime’ this list by posting comments that use some of the desired keywords for a particular topic, but this kind of tool would allow students to bring their own ingenuity into the keyword analysis as well. If thread topics and the associated keyword files can be saved in a library separate from any individual course, they can easily be reused in the future.

If I were being interviewed by a Product Manager, I would request this functionality:

  • can run tool on a thread topic which already has a keyword file to add new words to the file
  • keywords already in the file are distinguished from words not in the file
  • can associate an existing keyword list with a new or existing thread topic
  • add keywords words directly to any existing list
  • delete words from a list
  • merge two existing lists

Finally, build another tool that evaluates the keyword density by each participant in a thread. I wouldn’t use this tool on its own, but it could be used as an indicator, particularly for finding students who don’t add much value to a thread, or maybe go off topic a lot.

I know that CollegeBoard has built a similar but much more complex tool for evaluating essays. You feed it several hundred essays on a particular topic, already graded by humans, and it learns the characteristics for good, average, and poor essays on this topic. You can then feed it ungraded essays, and it grades them. The tool is only as good as the sample, and it should never be used as the sole source of a grade, but I understand that the correlation between the grade given by the tool when it is well-trained and the grades given by well-trained human readers is very good, and often better than the correlation between two different human readers.

Of course, a good Product Manager doesn’t just add ‘nifty’ features to a product but instead uncovers the needs of the target audience. In this case, I’m talking as part of the target audience instead of as a Product Manager.

Blog at WordPress.com.