Designing and testing a web-based board game for teaching information literacy skills and concepts
The Authors
Karen Markey, School of Information, University of Michigan, Ann Arbor, Michigan, USA
Fritz Swanson, English Department, University of Michigan, Ann Arbor, Michigan, USA
Andrea Jenkins, School of Education and Human Services, University of Michigan-Flint, Flint, Michigan, USA
Brian J. Jennings, School of Information, University of Michigan, Ann Arbor, Michigan, USA
Beth St. Jean, School of Information, University of Michigan, Ann Arbor, Michigan, USA
Victor Rosenberg, School of Information, University of Michigan, Ann Arbor, Michigan, USA
Xingxing Yao, School of Information, University of Michigan, Ann Arbor, Michigan, USA
Robert L. Frost, School of Information, University of Michigan, Ann Arbor, Michigan, USA
Acknowledgements
The authors thank the Gladys Krieble Delmas Foundation and Trustee David H. Stam who provided the support that enabled them to develop the Defense of Hidgeon game, sponsor and evaluate game play, and give monetary awards to game winners.
Abstract
Purpose – This paper seeks to focus on the design and testing of a web-based online board game for teaching undergraduate students information literacy skills and concepts.
Design/methodology/approach – Project team members with expertise in game play, creative writing, programming, library research, graphic design and information seeking developed a web-based board game in which students used digital library resources to answer substantive questions on a scholarly topic. The project team hosted game play in a class of 75 undergraduate students. The instructor offered an extra-credit incentive to boost participation resulting in 49 students on 13 teams playing the game. Post-game focus group interviews revealed problematic features and redesign priorities.
Findings – A total of six teams were successful meeting the criteria for the instructor's grade incentive achieving a 53.1 percent accuracy rate on their answers to substantive questions about the black death; 35.7 percent was the accuracy rate for the seven unsuccessful teams. Discussed in detail are needed improvements to problematic game features such as offline tasks, feedback, challenge functionality, and the game's black death theme.
Originality/value – Information literacy games test what players already know. Because this project's successful teams answered substantive questions about the black death at accuracy rates 20 points higher than the estimated probability of guessing, students did the research during game play which demonstrates that games have merit for teaching students information literacy skills and concepts.
Article Type:
Research paper
Keyword(s):
Reference services; Library instruction; Video games; Design and development; Information literacy.
Journal:
Library Hi Tech
Volume:
26
Number:
4
Year:
2008
pp:
663-681
Copyright ©
Emerald Group Publishing Limited
ISSN:
0737-8831
Information
Undergraduates are rewarded when they master the core concepts of the academic disciplines. Simultaneously, they must master a series of skills that includes studying, time management, test taking, oral presentation, information literacy, and more. While these skills cross all of the disciplines, the skills themselves are not the focus of learning and rewards. As a consequence, students tend to ignore the academy's informal opportunities for skill-set development and suffer the consequences. The purpose of the Storygame Project was to design, develop, and evaluate the potential of computer-based gaming for teaching incoming undergraduate students skills-set knowledge, specifically library research skills that are a key component of the information literacy skills-set.
This paper focuses on the design and testing of the web-based board game called Defense of Hidgeon: The Plague Years. The game teaches incoming undergraduate students information literacy skills, specifically, the general-to-specific model for conducting library research (Kirk, 1974). The project team recruited a class of 75 undergraduate students to play the game, interviewed them at the end of the competition, and analyzed game activity logs to find out how often they played the game, what game features they used, what they learned about library research, and what improvements they would suggest for this game and information literacy games generally. Based on student game play and post-game interviews, four premises for the design of the Hidgeon and information literacy games are offered.
Literature review
When students arrive at the academy, they are operating for the first time in the same rich, deep, diverse information environment that faculty use to extend the discipline's frontiers of knowledge. Knowing little about the breadth, depth, and authority of college and university library collections, new students cling to the convenience and familiarity of Google, unaware of its many limitations such as its lack of comprehensiveness, credibility, and objectivity (Fast and Campbell, 2004; Head, 2007). Relying on the web, students are not reaping the direct benefits of producing better scholarship and the indirect benefits of learning how to become experts in their chosen domains.
Students are expected to conduct research, but faculty are reluctant to cede class time over to librarians teaching research skills (Hardesty, 1995; Breivik, 1998; Hrycaj and Russo, 2007). Librarians offer many non-credit venues – workshops, short courses, virtual reference assistance, web-based instruction pages, walk-up service at the information desk – to teach students about conducting research (Cox and Lindsay, 2008); unfortunately, most students avoid these approaches because they want customized, just-in-time assistance (Tiefel, 1995; Markey et al., 2005).
Traditional approaches to teaching incoming students information literacy concepts are not reaching them. Our solution is to design, test, and evaluate games that students play while using domain-expert tools for resource discovery and management. We choose games because what people are doing when they are playing good games is good learning (Gee, 2003, p. 199). Johnson (2006, p. 31) praises games for their ability to help us “find order and meaning in the world and make decisions that create order.” Squire and Jenkins (2003, p. 29) promote games for good learning because they “encourage collaboration among players and thus provide a context for peer-to-peer teaching and for the emergence of learning communities.” Gee (2003, p. 26) argues that “games are potentially particularly good places where people can learn to situate meanings through embodied experiences in a complex semiotic domain and meditate on the process” and presents three dozen learning principles that are built into good games (pp. 207-212).
Recognizing the popularity and potential of games for good learning, librarians have begun their experimentation with games by adding them to library collections and hosting game play competitions (Levine, 2006; Nieburger, 2007). There are a handful of efforts focused on building and deploying custom games to teach players information literacy concepts (ACCD, 2006; UNC, 2007; Poynter, 2008). While these games enlist familiar features such as boards, dice, teams, turns, prizes, etc., they merely test players about what they already know. This paper describes a research effort to design, develop, and test a new online game genre in which students do library research to make progress in the game.
Information literacy concepts
Teaching incoming undergraduate students how to navigate and choose from the maze of print and online resources in research libraries was the information literacy problem we wanted the game to solve. Knowing it would be difficult to break students of their traditional patterns of searching Google and the web, the research team sought a model that expected students to get a basic understanding of their topics from the web, put their attention on library-supplied finding tools, familiarized them with tool names, scope, and functionality, and gave them practice choosing tools that enabled them to achieve increasingly deeper levels of understanding on a topic of interest.
For inspiration, we turned to the general-to-specific (GenSpec) model and updated it with electronic resources and the web (Kirk, 1974). The updated GenSpec model advised students to start their research with broad overview tools such as general and discipline-specific encyclopedias, handbooks, and histories so they develop a general understanding of their chosen topics. Next, the GenSpec model advanced students to finding tools – bibliographies, abstracting and indexing databases, and catalogs – for specific information on their topics upon which they can build a solid foundation of understanding. Finally, the GenSpec model advanced the few students who want to specialize and achieve depth in their chosen topics to forward-chaining tools – citation indexes – to find the latest cutting-edge research. A pervasive presence in the GenSpec model was ready reference sources such as dictionaries, glossaries, and biographies to enable them to answer fact-based questions that occurred to them throughout the research process.
Designing an information literacy game
The team's initial efforts benefited from an anonymous donor who wanted to support initiatives that promote learning, discovery, and practice in venues where students live, play, and socialize. After we gained confidence in our design, we proposed the Storygame research project to the Gladys Krieble Delmas Foundation that funded game development, play, and evaluation, and modest awards to student game winners.
Making the much-needed breakthrough in game design
Initially design progress was slow and tedious because team members with one type of expertise expected members with a different type of expertise to take the lead. The project team eventually arrived at two promising ideas: game play in which students engage in bibliography building and game play in which students use digital library resources to answer substantive questions on a scholarly topic. The project team pursued the latter because it was doable given the project's limited time and staff resources.
To break the logjam, project principals drafted a vision statement for the question-and-answer idea that situated game play on a web-based game board and formulated game rules and objectives. Game objectives were few and simple – avoid “intellectual bankruptcy” by maintaining a positive portfolio of game assets and be the first team to answer a certain number of substantive questions for the six resource types in the GenSpec model, i.e. web, encyclopedias, books, edited works, journal articles from online databases, and published research from the ISI Web of Science. Game development involved building on the strengths and reducing weaknesses of the original vision.
Student members of the project team chose the game's theme because they were similar in age, mindset, preferences, and sensibility to the game's prospective players. They chose the “black death” because this time period was the setting for popular commercial games and several feature films, and the ideas of morbidity, violence, and mystery underlying this topic were likely to appeal to a college-aged audience. The principals followed up with a draft game board. The writer drafted a back story that put game play in the era of the black death and located game action in the medieval European Duchy of Hidgeon. The programmers improved the game board adding an electronic die that controlled game-piece travel around the Duchy and game spaces that issued players questions, collected their answers, and rewarded them for their correct answers.
No longer logjammed, the Storygame Project team accomplished game design and development work simultaneously. Game design was truly a project team effort. Team members proposed a wide variety of new ideas that built on the principal's original vision statement, arrived at a consensus about how features would work, and worked together to incorporate them into the game.
Playing the web-based board game Defense of Hidgeon
The game's back story places players in the middle of the fourteenth century at the height of the black death. Playing in two- to four-person teams, students roll a digital die, and travel around the medieval Duchy of Hidgeon where they are tasked with scrutinizing information in Duchy libraries about plagues past, present, and future in order to help the Duchy's ruler develop a plan of action. To win the game, students must prove they are the Duchy's richest, fastest, and most efficient team of researchers.
Game board navigation
Student teams play game pieces that travel the web-based game board (see Figure 1) in a clockwise direction on a path that looks like the letter “B.” Teams click on the electronic die (top left of the board) that moves their game piece forward 1 to 6 spaces. On the board are these 34 game spaces: 17 monastery library spaces, four sage advice spaces, two garrison spaces, two library study spaces, two oracle spaces, and one each of the following spaces: fox hunt, manor house, tavern, well, public house, hospital, and castle of the Duke. After a roll of the die moves their game piece forward, the game issues players instructions about how to proceed.
Landing on monastery library spaces
Teams reap the greatest benefit from landing on monastery library spaces. These spaces issue questions that include navigational hints so teams can find the right resources to do the library research that results in correct answers. Teams must give correct answers at least three times at each of the six different monastery library types. Whether players search the web, online encyclopedias, online library catalog, subject-oriented databases, or citation indexes to answers the question depends on the monastery library space on which their game piece resides. The game rewards teams for their correct answers by giving them a golden scroll and the opportunity to purchase an exclusive license to the library or challenge an opposing team for its exclusive license. The game's scoring algorithm penalizes teams for incorrect answers.
In Figure 2, a roll of the die propels the game piece onto the monastery library space called the White Library of St Thomas. The game displays a question that gives hints about searching online encyclopedias to find the answer (Figure 3).
Filling in a radio button (for one-answer questions) or checking a box (for two- or three-answer questions) was the way in which players submitted their answers to the game. Instead of the game giving feedback to players who gave incorrect answers, it gave a confirmation to players who submitted correct answers (Figure 4).
Because no opponent owns the license to this library, the game prompts the team to purchase an exclusive license before continuing the game with a roll of the die (Figure 5). Exclusive licenses figure positively into the game's scoring algorithm and earn gold (the game's currency) for owning teams every time an opponent's game piece lands on them.
To enable game players to learn more about library research, the game featured sage advice and library study spaces. Players encountered the same question-and-answer routines on these spaces. Game players, however, had to use online ready reference sources to answer sage advice questions and web, library catalog, database, or citation-index search engines to answer library study questions. Incorrect answers adversely affected a team's score and gold but they did not affect a team's golden scroll count.
Special game features
To make the make the game-play experience more varied, the game featured additional spaces that had positive and negative impacts on the scores of teams whose game pieces landed on them. Table I lists these game space types and scenarios that could happen to game pieces landing on them.
The hospital was especially unique. It required players to complete a task at campus libraries to earn their release. An example of a Hospital task was:Q Go to any campus library and ask the reference librarian to show you how to find and use a unique library resource in their library's collection that pertains to the black death.A team's most potent offensive strategy was to engage the game's challenge feature. Landing on a monastery library owned by an opponent, the team could challenge the opponent for ownership. The game gave a scenario to the two teams that described information needed to keep the duchy's population from panicking over the encroachment of the black death. Instead of answering the question, teams selected the three best citations from their respective bibliographies that would prepare the duchy's rulers for solving the problem. Citations were scored according to their level of credibility and how well they matched the scenario's discipline and audience level. If the challenger won, the game transferred the opponent's exclusive license to the challenger, thus, increasing the challenging team's assets and total score.
Tabs for displaying a team's backpack and dashboard were a pervasive presence adjacent to the game board (Figure 1). On the dashboard was all essential information pertaining to a team's game piece specifically and team progress generally. In the backpack were game assets that the team had earned to date. At any time, teams could check their score vis-a-vis opponents' scores on the leader board. Although game assets such as gold, exclusive licenses, and golden scrolls were important, a team's accuracy answering questions was weighed heavily in the game's complex scoring algorithm.
A YouTube movie demonstrates game play (Anon., 2007). The research team's final report describes and illustrates the game's full functionality and explains its complex scoring algorithm (Markey et al., 2008).
Game play
This section tells how the research team recruited undergraduate students to play the game, monitored game play, and collected data during the game.
Recruiting students to play the Defense of Hidgeon
The research team recruited students from SI 110, “Introduction to Information Studies” to play the game. SI 110 is taught by this paper's eighth author, and a total of 75 undergraduate students at all levels from a wide range of majors enroll in the course.
Only in passing did Frost mention game play to SI 110 students. During class on October 30, 2007, this paper's first author gave students an introduction to the game. Her remarks about the game were brief because she did not want to predispose students to thinking about the game in a particular way, instead, she wanted students to develop their own ideas about what the game was teaching them. Her introduction included a demonstration of game play, a summary of monetary prizes to first ($400 per team), second ($200 per team), and third place ($100 per team) winners, encouragement to sign up on teams to play the game, and notice that she would ask them questions about their game-play experiences after the game ended. Game play began on November 3, and ended on November 29, 2007.
Monitoring game play activity
Of the 75 students enrolled in SI 110, 29 students signed up on eight teams that ranged from two to four players. After game play began, the research team monitored logs of game play activity. Only one team played the game over the first weekend answering 12 of 14 questions correctly and acquiring eight of the game board's 17 exclusive licenses to monastery libraries. To encourage game play, SI 110's instructor gave this incentive to students: receive a half-letter grade increase for answering 40 percent or more questions correctly in the course of collecting all 18 golden scrolls. In response, an additional 20 students signed up on five new teams to play the game. Overall, 49 (65 percent) of the 75 students in the class signed up on 13 teams to play the game. By the following Monday morning (November 12), six teams had signed onto the game and attempted an answer to at least one question.
The Storygame Project team knew in advance that the Defense of Hidgeon might have to support up to 18 teams to accommodate SI 110's 75 students. The project team's solution was to distribute teams onto game boards to give teams the opportunity to acquire a critical mass of game assets and foster concentrated game play such as license purchasing and challenging. The 13 teams were distributed onto two boards named “Hidgeon” and “Plague” and bearing seven and six teams, respectively.
Collecting data about game play
While SI 110 students played the game, the project team logged selected transaction data about game play such as questions attempted and answered correctly by type, licenses purchased by type, and time elapsed since the start of the game. When the game ended, we transferred logged data to an Excel workbook for data analysis. Additionally, project team members attended SI 110's three regularly-scheduled weekly discussion groups on November 27 and 28, 2007 and conducted focused group interviews with students enrolled in the class. Students discussed whether their teams played together or individually, what they learned from playing the game, what improvements they would make to this game, and whether they would like to learn academic subjects by playing games.
The game play of successful and unsuccessful teams
The project team considered “successful teams” to be the 6 teams that met the criteria for the instructor's incentive, that is, answering the 18-question quota with 40 percent accuracy rate, and “unsuccessful teams” to be the seven teams that failed to meet the incentive.
Table II ranks the 13 teams that signed up to play the Defense of Hidgeon based on the game's scoring algorithm which took these factors of a team's game play into account:
- accuracy rate to questions;
- the value of gold collected;
- number of golden scrolls earned;
- number of exclusive library licenses owned; and
- the length of time taken to collect all 18 scrolls.
Roundtrips are estimates because the game log did not record every game space on which teams landed. Teams marked with a superscript letter met the instructor's criteria for the incentive.
Thanks to funds from the Delmas Foundation, the Storygame Project team gave monetary awards to teams ranking 1 ($400), 2 ($200), and 3 ($100).
The average accuracy rate of unsuccessful teams (35.7 percent) was not much greater than the estimated probability of guessing right answers to questions (0.30). Disregarding the performance of the unsuccessful warriors plunges the accuracy rate of unsuccessful teams to 21.4 percent. The seriousness of the warriors' game play is discussed below.
Successful teams achieved an average accuracy rate of 53.1 percent. Accuracy rates that successful teams achieved for online materials such as the web (67 percent), encyclopedias (62 percent), journal-article databases (62 percent), and ready-reference sources (52.7 percent) were considerably greater than those for print materials such as books (43 percent) and edited works (39 percent).
Playing on the “Hidgeon” game board, the InfoHunters team won the game because they were instant starters, beginning game play immediately after the game's start and, in the absence of sustained competition from opposing teams, collecting 18 scrolls and all monastery library licenses within a week of the game's start. The victors were the only serious competition to the InfoHunters on the “Hidgeon” board, challenging the InfoHunters three times and losing every time.
Game play by the Heroes team on the “Plague” game board was lacksidaisical. Starting on day five of game play, the Heroes earned three scrolls and purchased exclusive licenses to all three of them. A week later, the Heroes played for several days in a row, earning all but one golden scroll and occasionally failing to heed the game's prompts to purchase monastery libraries. The Heroes failed to respond to an opponent's challenge within the 48-hour deadline and forfeited a license to the challenger. Had the Heroes played with more intensity, the team might have overtaken InfoHunters for the lead. Perhaps the lack of competition on the “Plague” board where only two of six teams were successful made the Heroes lackluster participants. Only the unsuccessful warriors team gave Heroes competition initiating several challenges and winning one of them.
The Authorities, Maize, and Valiant teams used the game's basic functionality in a last-minute rush to meet the criteria for the instructor's incentive. Two teams signed up but failed to play the game. Four teams tested the waters, i.e. answering very few questions or failing to do the research to achieve respectable accuracy rates. Of the seven unsuccessful teams, only the Warriors played with a sense of conviction, attempting 28 questions and answering half of them correctly. Most likely, the Warriors had every intention of meeting the instructor's incentive but a combination of competing priorities and technical problems that suspended game play during Thanksgiving break prevented them from doing so.
An analysis of game features and functionality
SI 110 students' game play and post-game interview remarks enabled the Storygame Project team to learn about the game's shortcomings. As a result, they learned how they could improve Hidgeon and the design and development of future information-literacy games. This section highlights troublesome game features and functionality.
The hospital: a real show-stopper
Being sent to the hospital meant stopping online game play to visit a particular campus library to earn a code that released game pieces from the hospital. Our intentions were good: hospital tasks introduced players to campus libraries where they could learn how librarians and specific library collections could help them now and in the future.
Unfortunately, we did not expect teams would be sent to the Hospital so many times, and how disruptive hospital tasks would be to the overall flow of the game. Here is what students said about the disruptive nature of the hospital:
People on [my] team … sat in a room one day and tried to play but we ran into issues … like we all had one computer on the game and the rest were doing research … like when you got put in the Hospital and we kept getting in the hospital and it's really like a big pain to go get yourself out of the hospital. We tried to play … but two turns in we ended up in the hospital again and it just ruined that whole session.
I signed up and I played twice. I did one question and the other time when I came around I was in the hospital where I'd have to go to the libraries, I just gave up.
Going to the hospital was a pain.
If we were to redesign the game, we would put an upper limit on the requirement for teams to perform the hospital task. For example, a team's third time in the hospital would earn them a permanent release.
If it's in print, do not bother
In focus groups, the Storygame Project team sought explanations for the remarkably higher accuracy rates for online resources than for print resources. Game players told us that they guessed at the answers to books and edited works questions, that is, ones requiring them to go to the library, fetch these monographs, and examine their texts for answers. Despite our efforts to simplify the task by placing all monographs on reserve, students told us that going to the library was disruptive. Here is what students said about books and edited works questions:
I was doing other work, and I just didn't feel like going to the library.
It's just having to get up and go somewhere it's like … I didn't have enoughtime [to go to the library].
I thought it was disruptive. You kind of like had to go to the library to get a source and it's like, oh, I'm not going to the library, and it kind of puts a lag on it because you just kind of hope somebody on your team does it but it goes on for a couple days and then occasionally somebody will figure it out and then you'll get another one to go the library and it just kind of spreads it out and it takes more time to get it done.
During interviews, project staff asked students directly whether they went to the library to answer questions pertaining to books and edited works. Students answered in a chorus of “noes” and shook their heads “no.” A handful confessed that they guessed at the answers. The evidence is overwhelming – game players guessed at answers requiring them to go to the library. Students wanted game play that was entirely online and considered any deviation from an online administration as bothersome.
Feedback on incorrect answers to questions
The game gave players feedback for correct answers only because we wanted to avoid giving teams who answered incorrectly the first time an unfair advantage the second time. Furthermore, we doubted teams would ever encounter the same question twice because the game was seeded with a sufficient number of questions. Here is what game players said about the need for feedback to their incorrect answers:
If there was more than one [answer] and you got it wrong, there were a lot of times I was kicking myself because I didn't know why [I got it wrong] so a better explanation of why you got it wrong.
A lot of the times I was convinced that I had the right answer, and it was not the right answer, it was wrong … but I would have liked to … know why I was wrong and which answer was the right answer just because I was kind of left in the dark … Oh well, I was wrong, go to the next question.
In retrospect, our decision about feedback was not a wise one. Game players would have been receptive to feedback after submitting an incorrect answer because they wanted to know why their answer was incorrect. Game players might have learned something new that they could have used to answer subsequent questions or to apply to real information-seeking episodes.
Students liked the search hints that accompanied questions that gave them instructions for navigating to the right web site, catalog, or database. One player suggested that the game's feedback be designed like these search hints:
The game is not very difficult … It's … like here are your instructions, go to this website, type this, everything is very clearly explained to you so you don't really have to be very good at research or very intelligent to be honest to get the answers right, you just have to follow the instructions, but I don't think eliminating those instructions would be very good either because you need to know what to do so I think you have to find a median for it between that but I don't know how you would do that honestly.
More feedback [that is] like tips [that tell you] what to do.
Team versus individual game play
To share the workload, the project team urged students to sign up in teams of four to play the game and issued one password and one game piece per student team. How we envisioned game play is exactly how this team played:
Three of our team members got together to play the game … we had like three laptops ready and we were going to send people to different databases [because] there's only person who can be in the game at one time. The person that was in the game at the time would have to basically tell the other people with the laptops how to get to the websites to put in the question to the databases.
When students played the game in teams, landing in the hospital disrupted team play:
There was one point where three of our team members got together to play the game and it lasted about 20 minutes before we got sent to the hospital and no one wanted to go to the library but when we were together we had three laptops ready.
The suggestions students made about how to improve the game included capabilities for giving one player control while team members observed his or her actions and allowing the controlling player to pass control to fellow team members. Such a capability was not possible within our time and budget constraints but, given more time and financial resources, it would be a top priority during game development.
Challenge functionality
Despite the hard work the project team put into design and development of the game's challenge functionality, game players pretty much ignored it. Only 1 of 13 challenges was a complete challenge in which both challenger and owning team submitted bibliography entries to the game. The other 12 challenges were incomplete with either challenger or owning team or both failing to submit bibliography entries within the 96-hour deadline.
During interviews, the challenge was a rare topic of conversation. Here is one player's comment that underlines the need for teamwork to complete challenges:
We never really did the challenges but I think if a team was to do the challenges that would be a value to do as a team because there was so much homework involved.
Several factors probably contributed to the challenge's marginalization during game play. First, the game added a link to a team's backpack notifying them of an opponent's challenge. Because teams did not sign onto the game for several days in a row, they did not know about the challenges awaiting their attention. Second, the link was inconspicuous so it might not have drawn attention to itself. Third, game players might have disregarded challenge functionality because it was not necessary for meeting the criteria for the instructor's incentive. Fourth, responding to a challenge was different from the routine of game play. Some game players might not have been willing to invest the extra effort, study, and deliberation that the challenge required of them.
A redesign of the Defense of Hidgeon would have to include automatic email notification to make challengers and opponents aware of a pending challenge. Making the challenge more automatic would be another option. For example, the game could choose what it felt were the two best sources in a team's bibliography and invite game players to choose a third one. In this way, the game would shoulder some of the burden of the response and demonstrate to players the desirable characteristics of sources that were likely to score highly in the challenge. Such a semi-automatic approach might be an incentive for game players to follow-through on challenges.
The game's black death topic
Some students thought game play was meant to teach them about the black death. Because they were not interested in the black death, they were reluctant to play the game:
For me it was also the time issue and the appeal of it. I've done a lot of research through JSTOR, Lexis-Nexis, and everything, and so I didn't feel like this was an exercise that I would have gotten much from especially because the black plague is such an obscure topic, and I feel like doing research on it wouldn't have given me as much of a learning experience as I wanted to garner from it.
No, I don't think doing research on the Bubonic Plague is like what I do in my spare time.
I would like not play at all just cause like I felt like the topic to me was very boring to me.
That was another very small reason why I didn't play the game was just because I kind of had to study the black death for a really big project during my high school years, and I was really tired of the topic. Like I just didn't want to hear about it again.
During focus groups, students suggested alternative topics that we categorized as current events, major issues that would affect the world throughout their lifetimes, and topics that pertained to their chosen major. Table III lists examples of each.
Premises for the design of information literacy games
Based on an analysis of game play and evaluation data, the research team generated premises for the development of information literacy games. Four premises are featured here in a discussion that tells how the research team would improve the Defense of Hidgeon or design a new game to be in sync with each premise.
Premise 1: game play that gives players mastery over one key concept at a time
The initial design of the Defense of Hidgeon targeted the GenSpec model. Because we wanted to teach students as much as possible while we had their attention on game play, we yielded to the temptation to add complexity to the game. In addition to the GenSpec model, students got hands-on experience using several different types of information retrieval systems, exposure to a wide variety of information sources, and put in situations in which they made decisions about credibility, audience level, and discipline. Asked what they learned, students hardly mentioned the GenSpec model. Based on our experience, we recommend that future information literacy games focus players' attention on mastering one key concept, task, or procedure.
If we were to redesign the Defense of Hidgeon, we would streamline the game, eliminating game spaces or rethinking them so that they reinforced the general-to-specific model. We would also rethink the challenge so it helped players with citation selection. Alternatively, the challenge could be eliminated entirely and replaced by a “synthesis” game space at the end of the board that would ask players questions that would reveal the GenSpec model to them.
Game designers should consider how new information literacy games could be a pervasive and unobtrusive presence beside the online tools and collections students already use to research, write and document a writing assignment. For example, a new game could be made out of a collection of narrowly-focused mini-games, sensing active players who are using research tools online and challenging them to play various mini-games based on student contributions to a shared bibliography on an instructor-assigned topic in an online citation manager. A credibility mini-game would extract a citation from a shared bibliography, present it to one or two active players, and give them a limited amount of time to rate the citation according to its credibility. The game could calculate the closeness of their respective ratings and award points based on the closeness of their respective ratings. Feedback would be immediate, that is, the game would report immediately to players how closely they matched the ratings of their peers so that players could make adjustments in the future to effect better ratings. Occasionally, the game could ask players questions about why they should be concerned with credibility and provide feedback to players who give incorrect answers to such questions. Not only would students benefit from learning about source credibility by doing it during game play, they would leave behind a trail of credibility ratings that they could use to decide whether to use a particular source in a shared bibliography to complete a course assignment.
Premise 2: games that give players feedback to improve their performance
Students want positive and negative feedback from games to improve their performance. The Storygame Project team's decision to give game players feedback for their correct answers and omit it for their incorrect answers was ill-advised. In interviews, game players told us they would have been receptive to feedback after submitting an incorrect answer because they wanted to know why their answers were incorrect. Game players who studied feedback might have learned something new that they could have used to answer subsequent questions or to apply to future information-seeking episodes.
Switching feedback from Hidgeon's correct to incorrect answer messages would require a significant development effort at this time. Thus, we acknowledge our mistake and hope others learn from it.
Feedback would be an integral part of a new information literacy game that is a pervasive and unobtrusive presence beside the online tools and collections students use to research, write and document a writing assignment. A relevance rating mini-game that presents citations to two players simultaneously and asks them to rate their relevance vis-à-vis bibliography topics would give players immediate feedback with regard to each player's ratings. This mini-game would also follow up reporting to these players relevance ratings from all players to date so that they would know how their ratings fare vis-à-vis their classmates generally. A single-student version of the relevance mini-game might match the student's relevance assessments with those that a domain expert gives to citations. Instructors who discuss game play during class could ask students why there might be differences between ratings of peers and domain experts and what the consequences of such differences means with respect to conducting research, synthesizing what they read, and reporting it in their papers.
Premise 3: games that put players in control
Although students want to be in control during game play, they will collaborate with their peers when the collaboration furthers what they want to accomplish. A significant development effort would be necessary to enhance the Defense of Hidgeon for team play in which team members interacted online at the same time. Another costly alternative would be to build a capability for the controlling player to pass control to a fellow team member.
Future information literacy games should put players into situations that further their own research and, at the same time, leverage individual efforts for the benefit of all game players. Let's consider the task of building bibliographies on a particular topic. A game player would add citations and online full texts to a shared citation manager that includes the relevance assessments, credibility ratings, tags, and audience-level designations that game players add during mini-game play.
One can envision a mini-game that presents citations to two game players and challenges them to rate them using the five audience level designations we used in the Defense of Hidgeon, i.e. from fourth grade up, from ninth grade up, from college up, from college majors up, and scholars and scientists talking primarily to their peers. When the players finish rating citations, the game gives the players feedback telling how closely they match each other's ratings and uses their audience-level ratings to compute an average rating for each citation. When players are ready to write their papers on a particular topic, they could search the citation-manager database, limiting retrieved citations to audience-level ratings that match their experience and knowledge of the topic. Students may be able to apply their understanding of audience levels from playing the audience-level mini-game to the limiting of search results using audience-level and other limiting criteria. Mini-games could also be enlisted to transform other aspects of bibliography-building into collaborative activities so that the results of these activities would benefit all students who would eventually use the citation-manager database to choose citations for their papers.
Premise 4: a payoff for leaving the computer behind
Students must have concrete evidence that leaving their computer to do research will have a payoff in terms of improving their research or affecting their grades. Because Hidgeon teaches game players about the GenSpec model that includes the printed books and edited works in a library's physical collections, it would be difficult to conceive of this game without requiring players to go to the library. Perhaps the game could be redesigned to allow teams to return a game asset in exchange for passing on the performance of an offline task or for postponing the task for a limited time period. Enhancing Hidgeon's interface to allow players to pass game-board control might also make the task of visiting libraries less burdensome for them. However, convincing game players that the books and edited works in a library's physical collections are still relevant now that so many other items are online will not be an easy task.
If the design of a new information literacy game that is a pervasive and unobtrusive presence beside the online tools and collections students use to research, write and document a writing assignment involves bibliography-building, it is conceivable that students will exhaust the web and online databases searching for citations and online texts on particular topics and eventually give into searching the online library catalog that retrieves items in physical collections. To encourage players to contribute citations to books and edited works, mini-games could be programmed to award bonuses to players who contribute items that come from the library's physical collection. Playing mini-games that require them to analyze citations with regard to audience level and credibility, players might realize gaps that make it difficult for them to scaffold from the less credible information on the web to the more technical and scholarly information in online journals. This is a gap that books can fill because books synthesize human knowledge about particular phenomena in and across disciplines, they span large intellectual spaces, tackle mammoth problems, make more intensive cases than all other literary genres, and undergo rigorous editorial review.
Conclusion
The Storygame Project team wanted to build a game that did much more than merely test players about what they already know with regard to information literacy concepts. Because successful teams achieved accuracy rates between 40 and 60 percent for their answers to substantive questions about the black death, we are confident the Defense of Hidgeon fulfilled our design objective in that regard. However, the game was not entirely problem-free. For example, students resented the hospital task because it disrupted game play making them sign off the game to visit a campus library, and students guessed at most books and edited works questions rather than visit the library to examine books and edited works on reserve. For these and several other problems, the project team describes how they would improve the Defense of Hidgeon. The paper concludes with four premises for the design of information literacy games and tells how the project team would improve the Defense of Hidgeon and design a new game to be in sync with each of the premises.
Figure 1Defense of Hidgeon's game board
Figure 2Game piece at rest on the White Library of St Thomas
Figure 3Question from the White Library of St Thomas
Figure 4Confirmation of a correct answer
Figure 5Prompting teams to purchase an exclusive license
Table IOther game spaces
Table IIGame results
Table IIITopics for future games
References
Anon. (2007), "Navigating the Defense of Hidgeon: the plague years", available at: www.youtube.com/watch?v=u76tW-ne-yY (accessed 30 July 2008), .
Austin Community College District (ACCD) (2006), "The info game", available at: http://library.austincc.edu/help/infogame/starthere.htm (accessed 15 July 2008), .
Breivik, P.S. (1998), Student Learning in the Information Age, Oryx Press, Phoenix, AZ, .
Cox, C.N., Lindsay, E.B. (2008), Information Literacy Instruction Handbook, Association of College and Research Libraries, Chicago, IL, .
Fast, K.V., Campbell, D.G. (2004), "‘I still like Google’: university student perceptions of searching OPACs and the web", Proceedings of the ASIS Annual Meeting 2004, Information Today, Medford, NJ, Vol. Vol. 41 pp.138-46.
Gee, J.P. (2003), What Video Games Have to Teach Us about Learning and Literacy, Palgrave Macmillan, New York, NY, .
Hardesty, L.L. (1995), "Faculty culture and bibliographic instruction: an exploratory analysis", Library Trends, Vol. 44 No.2, pp.39-67.
Head, A.J. (2007), "Beyond Google: how do students conduct academic research?", First Monday, available at: http://firstmonday.org/issues/issue12_8/head/index.html (accessed 30 July 2008), Vol. 12 No.8, .
Hrycaj, P., Russo, M. (2007), "Reflections on surveys of faculty attitudes toward collaboration with librarians", Journal of Academic Librarianship, Vol. 33 No.6, pp.692-6.
Johnson, S. (2006), Everything Bad Is Good for You: How Today's Culture Is Actually Making Us Smarter, Riverhead Books, New York, NY, .
Kirk, T. (1974), "Problems in library instruction in four-year colleges", in Lubans, J.J. (Eds),Educating the Library User, R.R. Bowker, New York, NY, pp.83-103.
Levine, J. (2006), "Gaming and libraries: intersection of services", Library Technology Reports, Vol. 42 No.5, pp.31-59.
Markey, K. (2005), "Testing the effectiveness of multimedia for library-user education", Portal, Vol. 5 No.4, pp.527-44.
Markey, K. (2008), "Engaging undergraduates in research through a storytelling and gaming strategy: final report to the Delmas Foundation", School of Information, University of Michigan, Ann Arbor, MI, available at: http://hdl.handle.net/2027.42/58630 (accessed 15 July 2008), .
Nieburger, E. (2007), Gamers – in the Library?!: The Why, What, and How of Videogame Tournaments for all Ages, American Library Association, Chicago, IL, .
Poynter, D.S. (2008), "Digital information literacy contest", available at: www.digitalliteracycontest.org/ (accessed 15 July 2008), .
Squire, K., Jenkins, H. (2003), "Harnessing the power of games in education", Insight, available at: www.iaete.org/insight/articles.cfm?&id=26 (accessed 15 July 2008), Vol. 3 pp.5-30.
Tiefel, V. (1995), "Library user education: examining its past, projecting its future", Library Trends, Vol. 44 No.2, pp.318-38.
University of North Carolina (UNC) (2007), "The information literacy game", available at: http://library.uncg.edu/game/index.asp (accessed 15 July 2008), .
Corresponding author
Karen Markey can be contacted at: ylime@umich.edu