Study Activities: Course Assessments
The study activities and course assessments were one and the same. Students produced the data for this project by completing a series of course activities on Twitter and a class blog. The Twitter activities included Primary Source Tweets, Entry Tweets, Exit Tweets, and General Tweets (detailed below). The blogging project consisted of two blog posts, completed in a small team of 4-6 people, and two rounds of comments from students, submitted individually.
Other assessments in the course included attendance, participation, discussion reflections, and peer evaluations, but these assessments are not analyzed in the dissertation. I excluded attendance and participation because I did not keep field notes during the semester. Students submitted their discussion reflections and peer evaluations privately; using these course assessments as data required students’ explicit consent and therefore were not included in the final dataset of publicly-produced content.
Twitter Activities
Tweeting was both an assigned activity and a form of general participation in the course. I graded tweets either for completion or as a contribution to students’ participation score. Grading completion rather than quality provided students with low-stakes opportunities to test their ideas about the course. Students completed four types of Twitter activities in the class:
- Primary Source Tweets (PST, graded for on-time completion): Students completed the Primary Source Tweets (PST) prior to class by writing two tweets related to the primary source reading assigned for the upcoming class. The content of their tweets was flexible. I offered broad suggestions: “ask a question, tell me you’re confused about something, summarize a key idea, share a quote, connect this text to other things you’ve read.” Students also had the option to retweet, quote, favorite, and reply to each other’s tweets, though only quotes and replies counted for credit. They completed these tweets on a rotating schedule. Half of the students (Group A) completed PST for one class and then the other half of students completed PST for the next class (Group B).
- Entry Tweets (graded as part of participation): Students completed the Entry Tweets at the beginning of the class. The tweets served as a warm-up or opportunity for review. The questions varied, but I typically asked students to briefly share an experience or an opinion or requested they share something they thought was important from the previous class meeting.
- Exit Tweets (graded as part of participation): At the end of each class, students completed an Exit Tweet by responding to the question: “What did you think was most significant in today’s class?” The question was the same for each class.
- General Tweets (graded as part of participation): Before, during, or after class, students could spontaneously tweet responses to the class content. In these General Tweets, students joked around, asked follow-up questions, shared links and videos, and posted GIFs to express their response to the class. There was no formal structure to these tweets and students could choose how, when, and whether they wished to engage with the class in this manner.
At the beginning of the semester, students also completed a set of tweets called “
In addition to the text of their tweets, I asked students to include hashtags specific to the course in each of their tweets. The use of a general class hashtag (#hwc111) and a class-meeting hashtag consisting of “c” (class) and the number of the meeting (e.g., #c04) made it easier to grade the tweets and, later, collect and organize the research data. Students sometimes also chose to include images, GIFs, and links in their tweets. I almost never required the inclusion of media but didn’t discourage it either. Students had free reign over the inclusion of media in their tweets.
Blogging Activities
Over the course of the semester, students worked in small groups of three or four people to create two blog posts. Each group met with me early in the semester to choose a civilization (e.g., Rome) and to propose tentative topics related to that society for their two posts. The first blog post (Post 1) was a fairly traditional, text-based post, approximately 1,000 words in length. I encouraged students to include media in order to engage their audience, but the media was not the centerpiece. The second post (Post 2) was a creative post. Groups could choose to compose creative fiction, videos, songs, social media projects, or other multimedia posts. They were still required to use credible sources and provide accurate information, but they could present their posts in a variety of forms.
Students submitted a draft and a final version of each post. I did not grade the draft but provided comments and suggestions for improvement. The final version of each post was worth 75 points; the two posts together represented a total of 150 (out of 521) points toward a student’s final score. Each member of the group received the same grade for the post and I scored the posts using the Blogging Project rubrics.
As a form of peer review, students also left comments on the drafts of their peers’ blog posts. I required each student to leave three comments on Post 1’s and three comments on Post 2’s. Students could comment on any post and any element of a post. At the beginning of the semester, the students and I collaborated to set requirements for the content and tone of each comment. Comments needed to be respectful and include something positive as well as something that could be improved to earn full marks. Each comment that met these criteria was worth 10 points. Altogether, the comments counted for a total of 60 points (out of 521) toward the students’ grade for the semester.
Twitter Data Collection
I collected students’ tweets and accompanying media using DocNow, a web scraping program. Digital humanist Ian Milligan generously set up and hosted DocNow on my behalf, which allowed me to consistently harvest content containing the class hashtag, #hwc111. I used DocNow to collect usernames, timestamps, hashtags, the text of tweets, links to accompanying media, and records of favorites and retweets. The program also harvested static copies of images and GIFs included in students’ tweets. Each week, I ran a search of the hashtag and then downloaded .csv files of the tweets, metadata, and media related to #hwc111. I then stored the files in a secure Google Drive folder until I began compiling and cleaning the data. All told, I collected 11,454 tweets, retweets, and replies plus accompanying media over the course of the semester.
Blogging Data Collection
Most web scraping programs excel at collecting data from tables but are more difficult to manipulate into recognizing patterns in large blocks of text. Consequently, I used a combination of web scraping with OutWit Hub and simply copying and pasting content to collect data from the class blog. I harvested the comments using OutWit Hub and copied and pasted text-only versions of the posts into separate Google Docs to create documents that could be more easily read or manipulated for text analysis. Archived versions of the original posts are still available on helloworldciv.squarespace.com/blog. The links for each blog post can be accessed in the Blog Post Master List.