Dan's Blog

March 4, 2010

Discussion Boards in elearning.

Filed under: education,elearning — ddswanson01 @ 7:47 am

A few years ago, I participated in an online program, given by the University of Wisconsin, Madison, on ‘How to Teach Online’. We used one of the dominant online learning platforms in the market at the time, so the participants got to find out what it was like to be a student in an online course, in a distance learning environment. One of the biggest time-sinks was the discussion board. I tried to read every post my fellow students made, and I had to log on several times a day to keep current, or I could easily get behind to the point that it took me 45 minutes or longer to catch up.

Participation in the discussions was part of the grade; the instructors had some rubrics for judging both quantity and quality of posts. In discussions, the instructors indicated that evaluating discussion responses took more time than even preparation for the course. At the time, there weren’t many tools included to help in this evaluation. I’ve unfortunately not familiar with the cutting edge improvements in online learning tools, so I hope this post covers new ground – it’s new to me, at least.

After my first couple of days reading the discussion board, I asked my fellow learners if perhaps we could institute a rule that said ‘If all you are going to say to a comment is “I agree” please don’t post.’ There was a lot of pushback on this; for some people, agreeing with what was said was their major contribution to a discussion. I didn’t think of it at the time, but another reason for these short comments is affirmation. Some people are shy about posting (not me!), and seeing several people agreeing with their posts made it easier for these folks to post subsequent messages. But to me, it wasted a lot of screen space, plus my time, having to scroll through all these short messages with virtually no content.

Facebook has addressed this nicely with the “Like” button. This is a neat way for those whose only response to a post is “I agree” to provide that response without taking up screen space. It does more than that; it counts the number of people who agree.  This might be something instructors are interested in.  Combined with other information, this tool might help evaluate student participation in discussions.

The discussion board in this particular platform came with only two tools that I knew about to help evaluate discussions; they counted the number of threads a particular student started, and they counted the number of times a student posted a comment. These didn’t help much in performing a qualitative analysis, though. I don’t know if there was a tool that counted the number of responses a thread got, but a tool that counts ‘significant’ responses to a thread and gives the instructor a way to define ‘significant’ might be useful in the qualtitative analysis of discussion board postings.

What about qualitative analysis, assisted by the computer? It could be useful to see how many keywords each student used in a particular thread, but defining keywords in advance takes a lot of time and thought. How about a tool that reads through a thread when it is finished (or even still in progress), and provides the instructor with a list of words used in that thread?  The list would be appropriately filtered, to keep out ‘a’ and ‘the’, etc, and sorted by frequency of use.  The instructor would select keywords from the list appropriate to the thread and save them in a file associated with that thread.

The instructor can certainly ‘prime’ this list by posting comments that use some of the desired keywords for a particular topic, but this kind of tool would allow students to bring their own ingenuity into the keyword analysis as well. If thread topics and the associated keyword files can be saved in a library separate from any individual course, they can easily be reused in the future.

If I were being interviewed by a Product Manager, I would request this functionality:

  • can run tool on a thread topic which already has a keyword file to add new words to the file
  • keywords already in the file are distinguished from words not in the file
  • can associate an existing keyword list with a new or existing thread topic
  • add keywords words directly to any existing list
  • delete words from a list
  • merge two existing lists

Finally, build another tool that evaluates the keyword density by each participant in a thread. I wouldn’t use this tool on its own, but it could be used as an indicator, particularly for finding students who don’t add much value to a thread, or maybe go off topic a lot.

I know that CollegeBoard has built a similar but much more complex tool for evaluating essays. You feed it several hundred essays on a particular topic, already graded by humans, and it learns the characteristics for good, average, and poor essays on this topic. You can then feed it ungraded essays, and it grades them. The tool is only as good as the sample, and it should never be used as the sole source of a grade, but I understand that the correlation between the grade given by the tool when it is well-trained and the grades given by well-trained human readers is very good, and often better than the correlation between two different human readers.

Of course, a good Product Manager doesn’t just add ‘nifty’ features to a product but instead uncovers the needs of the target audience. In this case, I’m talking as part of the target audience instead of as a Product Manager.


February 8, 2010

Grammatical errors on web sites

Filed under: education,publishing — ddswanson01 @ 3:35 pm

I worked for an Educational Publisher for years, and would enjoy doing so again; I like contributing to books, eBooks and helping people learn. This morning something caused me to recall an incident that happened some time ago, and it sounded to me like a good topic to write about.

We used to have self-tests on our textbook web sites, with True/False, Multiple Choice, or Short Answer questions that were automatically graded. These were for review purposes only, since clever students could easily take them over and over again until the had all the right answers, and they were one of the most used features on the sites. If one of these stopped working, we could easily get hundreds of support calls before we got it fixed.

One day, one of the Tier 1 agents forwarded an angry student email to me. We had a chemistry test with fill in the blank including some definitions that looked something like this:

“__________ is the combination of a substance with oxygen.”

The proper answer to this is “Oxidation”. This student had entered “oxidation” and it was marked wrong. The student called Tech Support to complain; when the support agent tried to explain, the student used offensive language and then hung up, and then wrote an email. The email was offensive and contained obscene language; you would have thought our Web site had caused this student to fail the course. Remember, this test can be taken multiple times until you get all the questions right, if that’s what you want.  It would have required much less energy to do it over than it did to write an email.

The reason for the complaint? We were enforcing the rules of English grammar on a CHEMISTRY test. How simply terrible and impudent of us! How dare we?

As a representative of an educational publisher, I always felt it was our duty and obligation to get things right on our Web sites. To me, this includes using proper grammar and proper spelling, regardless of the discipline involved.  If for no other reason that just to set a good example. It doesn’t matter that we could easily have programmed this test to accept ‘oxidation’ as well as ‘Oxidation’. I know this seems picayune to some people, but it is important to me.

Your website is marketing collateral, often the first and sometimes the only marketing collateral your customer will see. When the web site is the source of a customer’s first impression, I want my Web site to say ‘The employees of this company are erudite and pay attention to detail, and believe that education is a holistic process. They believe that your English skills are important in everything you do in life.’

If I see a Web site that is poorly written, full of spelling mistakes and grammatical errors, I tend to think that whoever built and/or owns that site is careless and doesn’t really care about the impression the site gives to potential customers.  Personally,  I don’t really want to deal with companies like that. My grammar isn’t perfect, but I think it’s pretty darn good.  If I see an educational site with that kind of error, I usually send an email to the Webmaster, because I hope other people in the Education industry have a similar attitude. Sometimes I’m right, sometimes I’m disappointed. It’s still worth doing.

Blog at WordPress.com.