What Happened in My Open-Laptop Exam Class? (Part Two)

Learning About Collective Intelligence From the start, the group activities were framed in terms of notions of collective intelligence and participatory culture, themes which had been central to the first part of the semester. By the time they got to the group activities, students would have done papers exploring how Wikipedia works, would have participated in lectures and discussions explaining some of the core findings from MacArthur Foundation’s Digital Media and Learning initiatives, and would have looked at a range of social media and media sharing platforms and their dynamics. We had prepared for the problem sets by having earlier inquiry based activities in discussion which were organized around groups at a variety of different scales but which were ungraded (except in terms of attendance) Students had been given a set of exam questions about a week prior to the midterm, with a subset of the questions appearing on the exam. Students could bring their notes and other materials into the exam and consult them as they filled in their blue books.  Students had the option of sharing information or pooling insights with other students on the midterm, as long as they disclosed who they worked with. Most of the students seemed to work with at least one other person on the exam.

In one case, a team of students formed and posted online their collective responses to each question on the class mailing list the morning the midterm was to be given. This unanticipated situation posed a last minute challenge to the class instructors: we decided to write to the class, warning them that not all of the information contained in the posted answers was accurate, that they should use the material at their own risk and that they should disclose whether they had consulted these responses in preparing their answers. It turned out that one of the students had taken the liberty to posting the work of the other group members and some of them were not happy being placed in that situation. Other students said that they were afraid to even read the posted answers, but for the most part, the class took the situation in stride, there was still a great deal of diversity in the quality and content of the midterm answers. Whatever was going on behind the scenes, students did not mindlessly copy down the information that had been posted.

Taking the Final

The team’s performance on the final exam was uneven, but generally, the groups succeeded in creating richer, more fully documented responses than they would have been able to do individually. Some of the responses felt fragmented and contradictory, as if the teams had not been able to fully smooth out differences between members about the best way to approach a question; some of the responses included too much information, including much that was not pertinent to answering the question.  We had tried to break each question down into a series of steps, much like the weekly problem sets, so that students had a good way to structure their problem solving activities. In general, students did best where the questions were concrete and pointed to specific readings or topics from the class; they had more difficulty abstracting from the information provided, speculating about its future implications, or evaluating real world phenomenon based on proposed criteria. The collective process brought forward a strong tendency towards synthesis but set clear limits on their capacity to produce shared critiques. While some of the questions explicitly called on them to bring in their own examples, they tended to still operate within the borders of the class materials rather than going outside in search of new information. These later insights might be consistent with what we know about Wikipedia for example: that participants are often guided by a shared understanding of what an encylopedia entry looks like, that the community’s norms value “neturality” over critique and that there is a ban on publishing “original research.”  Success here rests, then, on correctly calibrating our expectations to value what works well in a collaborative context.

Student Criticisms

For those students who found this process frustrating, the largest single factor identified was a sense of loss of control over their own classroom performance.  One put it simply, “I have more control of my grade the first half of the semester and less control of my grade the second half of the semester.” Many of the USC students are very good at playing the traditional classroom game, calculating how many points they needed to get their desired grades, and giving the teachers what they wanted. If they grew up in a networked culture, they also grew up in a culture based on standardized exams, and so there was a certain degree of discomfort, among many of the students, with a more open-ended process which did not tell them what they needed to know and with a structure which meant that they were dependent on others for their mutual success. As one student explained:

"I preferred doing things on my own because I got stuff done much faster and more efficiently. I did not like relying on my other group members to do readings because I never knew if they had done them properly or not, and some of my team members did not even show up to a single class. That meant that they were going to receive the participation in lecture points based on my participation, and that does not seem fair to me at all.”

Others felt bruised by the lack of respect and trust shown them by other team members: "In order to work in a group, people have to understand the strengths and weaknesses of their group members and they have to be flexible.  When there are group members that don't trust other group members and want to constantly be in control, the group fails." One student described the final exam as a "debacle" because the group could not agree on strategies or criteria for producing a solid answer, while another complained about harsh treatment from classmates who did not value each other's contributions: "I have never felt so disrespected in my entire life. Some of the other group members made me feel like dirt, just because they thought that they were better than two of us.”

Many of the frustrations centered around unequal sets of expectations between team members, including a different sense of how well they wanted or needed to do in the class. Here, for example, was a student who compared negatively the experience of working with an assigned group in a required class and the processes which made collective intelligence work outside the classroom:

“I did not particularly enjoy the group portion of the class because I did not trust certain members of my group to complete the work and to do it well. Although the group portion theoretically could have stimulated more conversation about the topics and inspired people to participate in their learning, a couple of members in my group seemed very uninterested and content to skate by on the work my other group members and I did...  I fully understand the value of learning how to work in groups, especially given our shift toward participatory culture; but I assume in participatory culture, the participants actually have some glimmer of interest in the content they are creating."

Student Enthusiasm For those who had a more successful experience, they felt supported by their teams and energized by the shared responsibility over the material:

“It was nice to have other people to help with the assignments. Our team worked very well together, and I think learning how to work in teams is an important skill to have. In the second half of the semester, I was pushed to do my assignments because I knew that the team relied on me. Compared to having to do assignments alone, it was nice knowing that if there was a reading that I didn't understand, then there was somebody in the group that could help contribute.”

*********************************** "Honestly, I was a little skeptical as to how group work would ultimately play out and whether it would be successful, but to my pleasant surprise it was a great success. Just as the class was intended, different teammates were responsible for different materials and therefore were able to master different contents of the class and teach them to their team members. While I felt that the first half of the class was also well done, I had an even better learning experience in the second half of the class. While there was some participatory activity going on in the first half of the class, I believe there was a well-working participatory culture in the second half. The professor and the TA's structured the discussions very strategically to be able to push the students to work quickly and efficiently in their teams by grouping their knowledge into a collective product. I genuinely feel that this made the team much greater than the sum of its parts.”

*********************************** “Group work is definitely more challenging. However it challenged me to practice better negotiation and communication skills. I would consider the second half a practical application for all the communication theories learned in past years”

************************************ "I really liked having the groups for the readings and in-class discussions. I felt that I was able to cover so much more material (even if only through the short-hand of my teammates) by examining the notes for ALL of the readings on our Google Doc. I felt that I was more informed coming in to lecture. The first half of the semester, it was often difficult for me to get all of the assigned readings done. But with only one reading per night, it was a lot easier. Plus, I had the weight of my team to encourage me to actually get it done on time."

In many cases, they were thrilled not to have to go it alone, to be able to turn easily to someone else on the team who understood a particular chunk of course material better than they did. And even some who did not have a perfect group experience saw the value in the end of the process:

“If anything it made me realize that we all have limitations. One person can not carry a group. I feel that it all worked out in the end . I wish we had better communication within our group though.”

Some of the teams clearly acquired new techniques for coordinating and collaborating within a network: “Working on assignments together via Google Docs was very helpful because we each knew our roles and could quickly add to each other's work if needed.”

Assessing the Experience

For all of the frustrations expressed by some students about students getting equal points despite not doing equal work, a review of the grades by group suggests there was significant variation in their final performance in the class within each team in part because of their individual performance in the first half of the semester and in part because the mechanism of rewarding those who attended and participated in sections worked as it was designed to do.

Overall, students seemed to have reflected deeply about the advantages and disadvantages of the collaborative production of knowledge, a theme which recurred throughout the class, and in the process, they developed a stronger appreciation of  research as a process rather than imaging knowledge as a contained body of information. There's still a lot we all have to learn about making these kinds of group processes function, especially given the degree to which they fly in the face of the ways students have been socialized throughout their formal education to think of themselves as autonomous learners. Clearly, I am troubled by the reports of some of the destructive experiences which occurred within some of the more disfunctional groups, yet, over all, many more students expressed enthusiasm for the process than shared frustrations.

Interestingly, when I taught the subject two years ago with a much more conventional grading scheme, the average GPA for the class was 3.14, while the average GPA for the class with the collective experiment was 3.21, well within the average variation from one semester to the next.

NEXT TIME: THE DISCUSSION SECTION ACTIVITIES COMING SOON: THE EXAM