You are browsing the archive for 2012 October.

Moderate, monitor or leave well alone?

October 22, 2012 in Talking point

Talking point: When you’re using a PeerWise activity in your course, what do you do in terms of intervening in the students’ questions, answers and discussions?

When I tell someone about PeerWise for the first time, more often than not their first question is “But what if the students submit something that’s wrong?” My answer? “I don’t worry about it.”

As instructors, it seems to me that there are broadly three approaches we can take to administering our PeerWise repositories: you can moderate them, checking every question for correctness; you can monitor them, not explicitly checking everything but keeping a close eye out and intervening if you spot something egregiously incorrect; or you can leave well alone, and let the students look after themselves. My personal preference is for the last of these options, but plenty of people seem to recoil in horror when I tell them that. However, I would contend that not only can we legitimately not intervene at all, we definitely shouldn’t. Here’s why:

An instructor-free PeerWise space explicitly belongs to the students, and they have full responsibility for its contents. If they do spot an error, it’s up to them to resolve it: the teacher isn’t going to come along and fix it for them. I think this gives a much greater sense of ownership to the students, with corresponding greater commitment. Plus, deciding whether something really is an error or not, and why, can spawn some great, in-depth discussions in the comments threads, which I would argue are some of the most potent learning opportunities offered by PeerWise. This would be lost if we swept in, helicopter-like, to rescue the students all the time.

My experience in courses I have run is that less than a tenth of the submitted material has obvious errors, and from what has been reported this ratio seems to be broadly similar in other courses elsewhere. A good number of these problems do get identified and fixed by the students. “But not all,” I hear you say. True, not all. A small proportion of incorrect content does persist. But I’ve made my peace with that: students are smart people, and they understand the context very well – they know it’s a student-generated resource, and not everything in it is guaranteed to be correct. Besides, it’s not like PeerWise is the only place students might come across incorrect ideas: informal study groups in the library, the cafe or on Wikipedia are widespread, and no-one is moderating those…

I believe that as instructors we should be more relaxed about this sort of thing: any potential disadvantages of student mistakes are outweighed by the intellectual challenges of self and peer assessment, and taking responsibility for their own learning. PeerWise is a great space for students to make their own contributions to their course, and some of them really push the boundaries of creativity and sophistication. Don’t inhibit them by peering over their shoulder, virtual ‘red pen’ at the ready.

So, I’ve set my stall out. Are you nodding in agreement, or pounding the keyboard in rage? Vote in the poll (in the sidebar on the right), and use the comments below to tell me why I’m right or wrong!

So…what do you do in your courses? Have you changed what you do as a result of something that has happened? Would you prefer to do something different but have concerns or reservations? Join the debate!

What does a typical PeerWise course look like?

October 12, 2012 in Uncategorized

If you have ever wondered whether your class is too small (or too big) to use a tool like PeerWise, you may be interested in the following data.  To get a sense for both the typical size of a class on PeerWise, and the typical number of contributions made by students in each class, data from the last 1000 courses was examined.

While there are many examples of very large classes (>300 students), and even a few extremely large ones (>800 students), the majority of classes have fewer than 50 students.  The breakdown is given in the chart below.

In terms of student participation, it is quite common for instructors to reward a small fraction of course credit to students who contribute at least a certain number of questions and answers (for example, in the most recent class I taught, students were required to author 2 and answer 20 questions).

The table below gives the average overall participation (in terms of questions and answers) for classes of different sizes.  It also shows the average contributions per student in each of those classes.

The rightmost columns of this table are perhaps the most interesting – the chart below plots the figures from these rightmost columns – that is, the average number of questions authored and the average number of questions answered, by students in classes of various sizes.

If you ignore the very small (<20) and very large (>800) classes, the average number of answers submitted by students in all courses falls in quite a narrow range – from just below 30 to just above 40 answers per student.  Unsurprisingly, the average number of answers drops in very small courses – as students in these courses are unlikely to have a very large number of questions available to them to answer.  Conversely, very large courses sometimes end up with very large banks of questions (often many thousands of questions), enabling the very enthusastic students to answer (almost) as many questions as they would like.

So, where does your class fit in, and could the contributions of your students be described as “typical”?

Just how big is this thing…?

October 11, 2012 in Uncategorized

Immersed in numbers

From time to time, I email Paul Denny (he’s here – @paul), creator of PeerWise, to ask for up to date usage figures for the student-facing PeerWise website, to put into a talk or presentation that I am giving. I am giving one of these early next week to the assembled crew of the Carl Wieman Science Education Initiative here at UBC, so thought I would share some figures that Paul sent over…. some of them may surprise you!

  • Institutions:  308
  • Creators:  1796
  • Courses: 1905
  • Users: 94961
  • Questions: 379464
  • Answers: 8172405

Yes, you did read that correctly – getting on for half a million questions and ten million answers !!

Small print on the data:
“Institutions” only counts institutions for which there has been at least some activity, “creators” are instructors/teachers with the ability to create new courses.  ”Courses” includes all repositories created (even those for which is there is no associated content). “Questions” only includes active questions (i.e. not deleted or archived versions).

Sample Coursework Instructions & Marking Regime

October 10, 2012 in Announcements, Use cases

This year I’m on my 3rd year of using Peerwise to teach & evaluate students taking my Underground Construction course.  The students have to set 5 questions for their peers and have to solve 25 set by their peers.  I award 5% of the module assessment for setting (good!) questions and 5% for answering questions.  Last year our Malaysia campus joined in with our UK campus with a common Peerwise coursework (so the students couldn’t easily tell whether they were answering questions from another continent).  This year our China campus should join in.  The big benefit for me is speed of assessment – I can mark about 60 students an hour (provided I choose my on-line time OK !).  The big benefit for the students is that they enjoy it!

I have posted sample coursework instructions and my notes to markers in the Resources section of this site.  If you have any queries, I’ll do my best to answer them.

Using PeerWise to assign homework

October 10, 2012 in Use cases

This is a cross-post from my blog:

This past quarter I used a new tool called PeerWise in a physical chemistry course I co-taught.  The main idea behind PeerWise, which is a free web-based service developed by Paul Denny at the University of Auckland, is that students write multiple choice questions for each other.  I didn’t quite have the courage to say “Want homework? Write your own” so instead I used it as a supplement to the normal homework assignments, i.e. I reformulated most of the homework assignments as multiple choice questions and posted them on PeerWise, in addition to the usual homework assignment.

Some information about the course
The course is attended by about 170 chemistry and biochemistry majors and covers thermodynamics and kinetics – more specifically chapters 13-20 in Atkin’s Quanta Matter and Change.  It’s taught in a 9-week quarter and the students take one other course at the same time.  There are four other faculty members involved who also pick homework problems associated with their lectures.  The students were divided into six teams of roughly 30, and each team has six contact hours with TAs who can help them with the assignments.

Advantages of PeerWise
1. Feedback to the students.  PeerWise allows you to give extensive descriptions (you can even use videos: example 1 and example 2) on how to solve the problem immediately after the student has picked an answer.   The quality of the help provided by the TAs tend to vary greatly, and I hoped this would help even things out a bit.
2. Breaking complex problems up into more manageable pieces.  A standard question usually asks you to compute x given some conditions.  In PeerWise this can be broken up into, for example, a) which of the following equations would you use to compute x? and then b) what is the value x given these conditions.  Answering the first question lets people now whether they are on the right track before proceeding.
3. Feedback to me.  Students can leave questions or comments for each problem, which often helps me improve the explanation or identify errors in the problem.  Students can also rate the quality and difficulty of each question, which is very valuable.  I also get all sorts of detailed data including how many students answered the question correctly (more on that below).

Some data
I put 120 questions up on PeerWise and 119 students answered at least one question.  From what I understand many students worked in pairs and only one would answer on PeerWise.  39 students answered 100 questions or more.  For almost all problems, the majority of students picked the right answer, so they are not just clicking randomly to get the explanation as one could have feared.  For example, for the question with the most (86) answers was answered correctly by 57% while the question voted the hardest was answered correctly by 47% of the 76 students who answered it.

While I encouraged students to write their own questions, only 9 questions were written.  One of these questions was “Do you find PeerWise helpful?”  41 students answered, 38 said yes.  Also several students commented positively on PeerWise with one caveat (see below).

Clearly, many students used PeerWise as a study aid before the exam as seen on this figure.

One main problem: errors
The main complaint by the students was the many errors that crept in to the questions, multiple choice answers, and solutions.  The multiple choice format exacerbates the problem, since many students will work for hours trying to reproduce one of the answers.  This is of course impossible if there is an error and this is very frustrating to the student!

There are several sources of errors.
1. Several errors were introduced by me when I typed in the multiple choice answers (for example, using Joules when I meant kiloJoules) and explanations.
2. Several errors were already in the problems and was copied into PeerWise.
3. Several errors were in the answers provided in the solution manual of the text book and were copied into PeerWise by me.  Often the you would simply get a different numerical answer based on the numbers provided.

The error rate was about 1-2 questions per week and all (hopefully) will be fixed next year.

1. By default the questions are listed in the reverse order they are created (i.e. newest one first), so in cases where the order was important I would write the first question last, e.g. exercise 1b and then exercise 1a.
2. Each question comes with a one line preview, so the first sentence in the question should be the name of the exercise, e.g. Exercise 1a.
3. Each question can also be labelled by subject, so I would use labels like “week 1″.  With one click one can then display the questions on for a single week only.
4. Many of the comments left by the students said “I punch the solution into my calculator and get a different answer”.  It’s of course hard to know what the problem is, but in these cases I leave a link to the worked out solution on Wolfram|Alpha (example), which is basically on on-line calculator.
5. You add equation through one of several editors  (I recommend using the Latex editor) that create and image of the equation, which you can’t edit. If you view the page in HTML format there is a Latex code you can copy and paste into the editor and change, rather than typing everything in from scratch.

Changes for next year (When was the last time you learned anything by doing it once?)
Overall PeerWise was a hit, and I will use it next year for this course.  I plan to make the following changes:
1. I will add 10-20 math and conceptual training questions every week.  Questions one should be able to answer in 1 second: What is e−0?  If ΔG⊖<1 what can you say about the equilibrium constant?  Some of these will show up every week!
2. I will add 1-2 “regular” questions based on topics from previous weeks.
3. I will try to have the explanations include a screenshot of the solution done in Maple.  For three reasons: a) it will make me double check the answer in the solution manual, b) it will remove one source of error if I don’t have to retype something, and c) it will encourage the students to use Maple (students who use Maple efficiently are done in a fraction of the time compared to others).

Finally, I am racking my brain on how to get more students to write their own questions.


October 2, 2012 in Announcements

Welcome to this online community for instructors using PeerWise in support of their teaching!

Please take some time to explore what we’ve assembled here: you’ll find a space for discussion of ideas or issues in the Forum section, a chance to (virtually!) meet the other members of the community, a selection of resource videos about PeerWise that you may find useful. Additionally, there’s a collection of links to published material that either discuss implementations of PeerWise in various classes, or are relevant to the pedagogy of student-generated content.

Finally, probably the most important thing that you can do is… contribute! To post to the community and to be able to add resources, you’ll need to be registered, but that only takes a minute or two of your time.We’ve all kinds of plans for how we can get the community discussing various aspects of their use of PeerWise, and we’ll post details of these in due course. For now, come on in, have a look around and make yourself at home!

Welcome again

PeerWise-Community admins

Paul Denny, University of Auckland

Ross Galloway, Judy Hardy, Keith Brunton, University of Edinburgh

Simon Bates, University of British Columbia