Tag Archives: peer-assessment

Blogging Academe

LibriVox founder and Montreal geek Hugh McGuire recently posted a blog entry in which he gave a series of nine arguments for academics to blog:

Why Academics Should Blog

Hugh’s post reminded me of one of my favourite blogposts by an academic, a pointed defence of blogging by Mark Liberman, of Language Log fame.
Raising standards –by lowering them

While I do agree with Hugh’s points, I would like to reframe and rephrase them.

Clearly, I’m enthusiastic about blogging. Not that I think every academic should, needs to, ought to blog. But I do see clear benefits of blogging in academic contexts.

Academics do a number of different things, from search committees to academic advising. Here, I focus on three main dimensions of an academic’s life: research, teaching, and community outreach. Other items in a professor’s job description may benefit from blogging but these three main components tend to be rather prominent in terms of PTR (promotion, tenure, reappointment). What’s more, blogging can help integrate these dimensions of academic life in a single set of activities.

Impact

In relation to scholarship, the term “impact” often refers to the measurable effects of a scholar’s publication through a specific field. “Citation impact,” for instance, refers to the number of times a given journal article has been cited by other scholars. This kind of measurement is directly linked to Google’s PageRank algorithm which is used to assess the relevance of their search results. The very concept of “citation impact” relates very directly to the “publish or perish” system which, I would argue, does more to increase stress levels among full-time academic than to enhance scholarship. As such, it may need some rethinking. What does “citation impact” really measure? Is the most frequently cited text on a given subject necessarily the most relevant? Isn’t there a clustering effect, with some small groups of well-known scholars citing one another without paying attention to whatever else may happen in their field, especially in other languages?

An advantage of blogging is that this type of impact is easy to monitor. Most blogging platforms have specific features for “statistics,” which let bloggers see which of their posts have been visited (“hit”) most frequently. More sophisticated analysis is available on some blogging platforms, especially on paid ones. These are meant to help bloggers monetize their blogs through advertising. But the same features can be quite useful to an academic who wants to see which blog entries seem to attract the most traffic.

Closer to “citation impact” is the fact that links to a given post are visible within that post through the ping and trackback systems. If another blogger links to this very blogpost, a link to that second blogger’s post will appear under mine as a link. In other words, a blogpost can embed future references.

In terms of teaching, thinking about impact through blogging can also have interesting effects. If students are blogging, they can cite and link to diverse items and these connections can serve as a representation of the constructive character of learning. But even if students don’t blog, a teacher blogging course-related material can increase the visibility of that course. In some cases, this visibility may lead to inter-institutional collaboration or increased enrollment.

Transparency

While secrecy may be essential in some academic projects, most academics tend to adopt a favourable attitude toward transparency. Academia is about sharing information and spreading knowledge, not about protecting information or about limiting knowledge to a select few.

Bloggers typically value transparency.

There are several ethical issues which relate to transparency. Some ethical principles prevent transparency (for instance, most research projects involving “human subjects” require anonymity). But academic ethics typically go with increased transparency on the part of the researcher. For instance, informed consent by a “human subject” requires complete disclosure of how the data will be used and protected. There are usually requirements for the primary investigator to be reachable during the research project.

Transparency is also valuable in teaching. While some things should probably remain secret (say, answers to exam questions), easy access to a number of documents makes a lot of sense in learning contexts.

Public Intellectuals

It seems that the term “intellectual” gained currency as a label for individuals engaged in public debates. While public engagement has taken a different type of significance, over the years, but the responsibility for intellectuals to communicate publicly is still a matter of interest.

Through blogging, anyone can engage in public debate, discourse, or dialogue.

Reciprocity

Scholars working with “human subjects” often think about reciprocity. While remuneration may be the primary mode of retribution for participation in a research project, a broader concept of reciprocity is often at stake. Those who participated in the project usually have a “right to know” about the results of that study. Even when it isn’t the case and the results of the study remain secret, the asymmetry of human subjects revealing something about themselves to scholars who reveal nothing seems to clash with fundamental principles in contemporary academia.

Reciprocity in teaching can lead directly to some important constructivist principles. The roles of learners and teachers, while not completely interchangeable, are reciprocal. A teacher may learn and a learner may teach.

Playing with Concepts

Blogging makes it easy to try concepts out. More than “thinking out loud,” the type of blogging activity I’m thinking about can serve as a way to “put ideas on paper” (without actual paper) and eventually get feedback on those ideas.

In my experience, microblogging (Identi.ca, Twitter…) has been more efficient than extended blogging in terms of getting conceptual feedback. In fact, social networks (Facebook, more specifically) have been even more conducive to hashing out concepts.

Many academics do hash concepts out with students, especially with graduate students. The advantage is that students are likely to understand concepts quickly as they already share some of the same references as the academic who is playing with those concepts. There’s already a context for mutual understanding. The disadvantage is that a classroom context is fairly narrow to really try out the implications of a concept.

A method I like to use is to use fairly catchy phrases and leave concepts fairly raw, at first. I then try the same concept in diverse contexts, on my blogs or off.

The main example I have in mind is the “social butterfly effect.” It may sound silly at first but I find it can be a basis for discussion, especially if it spreads a bit.

A subpoint, here, is that this method allows for “gauging interest” in new concepts and it can often lead one in completely new directions. By blogging about concepts, an academic can tell if this concept has a chance to stick in a broad frame (outside the Ivory Tower) and may be given insight from outside disciplines.

Playing with Writing

This one probably applies more to “junior academics” (including students) but it can also work with established academics who enjoy diversifying their writing styles. Simply put: blogwriting is writing practise.

A common idea, in cognitive research on expertise, is that it takes about ten thousand hours to become an expert. For better or worse, academics are experts at writing. And we gain that expertise through practise. In this context, it’s easy to see blogging as a “writing exercise.” At least, that would be a perspective to which I can relate.

My impression is that writing skills are most efficiently acquired through practise. The type of practise I have in mind is “low-stakes,” in the sense that the outcomes of a writing exercise are relatively inconsequential. The basis for this perspective is that self-consciousness, inhibition, and self-censorship tend to get in the way of fluid writing. High-stakes writing (such as graded assignments) can make a lot of sense at several stages in the learning process, but overemphasis on evaluating someone’s writing skills will likely stress out the writer more than make her/him motivated to write.

This impression is to a large extent personal. I readily notice that when I get too self-conscious about my own writing (self-unconscious, even), my writing becomes much less fluid. In fact, because writing about writing tends to make one self-conscious, my writing this post is much less efficient than my usual writing sessions.

In my mind, there’s a cognitive basis to this form of low-stakes, casual writing. As with language acquisition, learning occurs whether or not we’re corrected. According to most research in language acquisition, children acquire their native languages through exposure, not through a formal learning process. My guess is that the same apply to writing.

In some ways, this is a defence of drafts. “Draft out your ideas without overthinking what might be wrong about your writing.” Useful advice, at least in my experience. The further point is to do something with those drafts, the basis for the RERO principle: “release your text in the wild, even if it may not correspond to your standards.” Every text is a work in progress. Especially in a context where you’re likely to get feedback (i.e., blogging). Trial and error, with a feedback mechanism. In my experience, feedback on writing tends to be given in a thoughtful and subtle fashion while feedback on ideas can be quite harsh.

The notion of writing styles is relevant, here. Some of Hugh’s arguments about the need for blogging in academia revolve around the notion that “academics are bad writers.” My position is that academics are expert writers but that academic writing is a very specific beast. Hugh’s writing standards might clash with typical writing habits among academics (which often include neologisms and convoluted metaphors). Are Hugh’s standards appropriate in terms of academic writing? Possibly, but why then are academic texts rating so low on writing standards after having been reviewed by peers and heavily edited? The relativist’s answer is, to me, much more convincing: academic texts are typically judged through standards which are context-specific. Judging academic writing with outside standards is like judging French writing with English standards (or judging prose through the standards of classic poetry).

Still, there’s something to be said about readability. Especially when these texts are to be used outside academia. Much academic writing is meant to remain within the walls of the Ivory Tower yet most academic disciplines benefit from some interaction with “the general public.” Though it may not be taught in universities and colleges, the skill of writing for a broader public is quite valuable. In fact, it may easily be transferable to teaching, especially if students come from other disciplines. Furthermore, writing outside one’s discipline is required in any type of interdisciplinary context, including project proposals for funding agencies.

No specific writing style is implied in blogging. A blogger can use whatever style she/he chooses for her/his posts. At the same time, blogging tends to encourage writing which is broadly readable and makes regular use of hyperlinks to connect to further information. In my opinion, this type of writing is a quite appropriate one in which academics can extend their skills.

“Public Review”

Much of the preceding connects with peer review, which was the basis of Mark Liberman’s post.

In academia’s recent history, “peer reviewed publications” have become the hallmark of scholarly writing. Yet, as Steve McIntyre claims, the current state of academic peer review may not be as efficient at ensuring scholarly quality as its proponents claim it to be. As opposed to financial auditing, for instance, peer review implies very limited assessment based on data. And I would add that the very notion of “peer” could be assessed more carefully in such a context.

Overall, peer review seems to be relatively inefficient as a “reality check.” This might sound like a bold claim and I should provide data to support it. But I mostly want to provoke some thought as to what the peer review process really implies. This is not about reinventing the wheel but it is about making sure we question assumptions about the process.

Blogging implies public scrutiny. This directly relates to transparency, discussed above. But there is also the notion of giving the public the chance to engage with the outcomes of academic research. Sure, the general public sounds like a dangerous place to propose some ideas (especially if they have to do with health or national security). But we may give some thought to Linus’s law and think about the value of “crowdsourcing” academic falsification.

Food for Thought

There’s a lot more I want to add but I should heed my call to RERO. Otherwise, this post will remain in my draft posts for an indefinite period of time, gathering dust and not allowing any timely discussion. Perhaps more than at any other point, I would be grateful for any thoughtful comment about academic blogging.

In fact, I will post this blog entry “as is,” without careful proofreading. Hopefully, it will be the start of a discussion.

I will “send you off” with a few links related to blogging in academic contexts, followed by Hugh’s list of arguments.

Links on Academic Blogging

(With an Anthropological emphasis)

Hugh’s List

  1. You need to improve your writing
  2. Some of your ideas are dumb
  3. The point of academia is to expand knowledge
  4. Blogging expands your readership
  5. Blogging protects and promotes your ideas
  6. Blogging is Reputation
  7. Linking is better than footnotes
  8. Journals and blogs can (and should) coexist
  9. What have journals done for you lately?

Moodle and Collaborative Learning

Something I just posted on a forum about the Moodle course management system.

Using Moodle: Thinking Through Groups

Here are some comments and observations about the “Groupsinterface (where an instructor can put participants in distinct groups) and other group-related features in Moodle.
I’m currently teaching a smallish ethnomusicology seminar and a large (170 students) introductory course in cultural anthropology at Concordia University in Montreal. I decided to get my intro students to work as teams on an ethnography project. It’s the first time in my (still relatively young) career that I’m getting students to do teamwork. Yes, it’s a challenge. Moodle has made it both easier and more difficult, IMHO.
Several of these are probably common feature requests from Moodle users and I’m not enough of a coder to implement any of those ideas. These comments also include “pie in the sky,” wacky, wishful thinking, “you gotta be kidding” thoughts about the potential of Moodle’s group-related features. Please excuse the craziness but don’t worry, it’s not contagious.
I’m using “instructor” for my role as the course creator and “participants” or “students” to refer to the people the instructor is putting in groups.

Observations, Comments

  • Listing participants by first name is inconvenient for large university classes. I would like to be able to sort students as I wish, as in the Participants list.
  • In large courses, it’s difficult to select participants who aren’t in any group yet. I understand that the interface is meant to make it possible for participants to be in multiple groups. But I believe it’s common for the instructor to be putting all students in separate groups. In such a case, it’d be so much easier to have the left-hand list of participants hide the ones which are already in a group and only show participants who still need to be put in groups. With 250 participants, scrolling that list back and forth has been very inconvenient.
  • The Participants and Groups sections overlap in function, IMHO. Maybe they could be merged. This would be especially useful in terms of messages. While searching for participants by group, selecting them, and adding them as recipients for a message works, it becomes quite cumbersome after a while.
  • When I click on a participant’s name in the left-hand list, I expect to be able to see to which team(s) this participant belongs.
  • I can select multiple participants in the left and right columns but I can’t select multiple groups to temporarily merge teams. This could be useful, especially while sending messages.
  • Several students seemed a bit puzzled about finding their groupmates. There could be a “group” section for students where they could not only see links to their groupmates’ profiles but also manage a kind of group profile.
  • It’s still somewhat unclear to me how Moodle handles groups. For instance, what does group visibility (separate or visible) mean for journal entries?
  • Maybe they can but I haven’t noticed how group participants may change the group’s name. That would be useful. Especially if they can add some information (available to the rest of the class or only to the instructor) about their group. Something like a group profile. In fact, it could summarize the profiles from all of the group’s members in one page (visibility to students as an option).

Feature Requests

  • In a way, it would be possible to work with groups as if they were individual participants. For instance, we could give grades to a group as a whole and have those grades show up in the group participants’ grade list. Or we could have one-click messaging for a group as a whole, directly from the Participants list.
  • It would be useful to be able to create a new group with selected students instead of having to prepare the groups in advance.
  • It could be neat to have both a group name and a unique group ID, especially with relatively large numbers of groups (I have about 40).
  • The number of participants in a team is very useful data and it helped me rebuild teams which had lost members during “drop and add.” Such data could be put in the interface so that the instructor can sort groups by numbers of participants.
  • Drag-and-drop (through AJAX) would be much more convenient than the current method for adding participants to groups. I guess this one is in the official plans but I want to voice my support for it! wink
  • It could be useful to be able to upload and download CSV or tab-delimited files with all the team information. The data might be available with grades or some such but it’d be very useful to download a grouped list of participants directly from the group interface. It would also be quite efficient to create groups in, say, Excel and be able to implement those groups in Moodle with a simple upload.
  • There might be a group building tutorial but I haven’t seen it in obvious places. Given the fact that the Moodle community is full of experienced instructors, that tutorial could have some advice about good grouping practices, maybe with some links to pedagogical issues.
  • There might be a group building tutorial but I haven’t seen it in obvious places. Given the fact that the Moodle community is full of experienced instructors, that tutorial could have some advice about good grouping practices, maybe with some links to pedagogical issues.
  • I haven’t checked if it might be available already but it’d be useful to have grouped Reports. I don’t want to monitor the activities of most of my students but it’d be useful to know if at least one group member is accessing Moodle frequently.
  • According to many people, it’s usually best for the instructor to create the groups, and it’s what I did. Yet, I wonder if there’s a way for students to create their own groups. If there is, I haven’t noticed it and my students haven’t either. (Maybe it’s a setting…)

Would These Work?

  • There could be a feature which would divide the course up into randomized teams automatically. I eventually used Lab Partners to create random teams that I then grouped in Moodle. It didn’t take me that long but it’s a bit error-prone and cumbersome. Fortunately, my teams will remain stable during the semester.
  • This one may seem like a far-fetched idea but it would be great to have more information about participants while we’re forming the teams. For instance, there could be a database field for majors or even MBTI results. Then, one could combine teams based on theavailable data. Of course, it’s beyond the purpose of Moodle and can probably be done in Excel, but it’s much easier to have everything in the same place.
  • I will have students assess the participation of their teammates. For a while, I was looking at the Workshop module as a way to implement this in Moodle. I ended up deciding on the use of a custom-made peer-assessment system (built at my university) but it could be an interesting feature of Moodle groups.
  • This might sound crazy but I imagine a way for groups to have their own Moodle subsection. We keep talking about peer-teaching and such and I can’t imagine a better than to have students create and manage their own mini-course. One major benefit would be to improve the interface, IMHO. The main Moodle section for the course would contain all the public information and activities. All the “separate groups” activities and material would appear in “group mode.” Students could then understand very clearly what is visible to everyone in the course and what is meant for their subsection only. In separate sections of a course taught by the same instructor (or, in fact, by different instructors) it could also have amazing benefits. I seem to recall something like this instructor-section idea being discussed for a future version of Moodle. But the Moodle take could also have a student-focused structure. Of course, this should not have to go all the way to the Moodle administrator and instructors should be able to create these subsections themselves. But, if at all doable, it would help Moodle leapfrog Sakai (which does handle course sections).
  • I pretty much like the notion of a “session” or “workspace,” which might be the reason why I tend to separate a student’s participation in the course as a whole (through the main Moodle interface for a course) from a student’s participation in a specific team (through a subsection of the Moodle site for the course). So this might be idiosyncratic (and lunatic) but I’m getting a very clear idea of how this might all work. After all, the granularity of “a course” is both too large (“coarse?” wink ) and too fine for many of our needs. Any “course” could become something of a “metacourse” and the structure could be somewhat recursive.
  • Participants could have profiles to be shared only with their groupmates. As it stands, I think the scope of Moodle profiles is system-wide (students have the same profile for all of the courses they take at the same institution, but not for courses they might take on other Moodle installations). Having group-only profiles would be interesting as students manage their relationship with teammates.
  • Another crazy idea: groups working a bit like social networking sites (e.g. Facebook). You get “friends” with whom you can share “stuff” (images, comments, chats, etc.). Those groups can go beyond the limits of a single course so that you would use it as a way to communicate with people at school. The group could even have a public persona beyond the school and publish some information about itself and its projects. Moodle could then serve as a website-creator for students. To make it wackier, students could even maintain some of these contacts after they leave the school.
  • Or Moodle could somehow have links to Facebook profiles.

Ok, I’m really going overboard. It’s just that I really love Moodle and want it to do everything at the same time. Using groups has opened up a whole new side of Moodle for me and I find myself thinking out loud a lot.