Company Services Portfolio Research Contact
Research Logo: eduweb

Other evaluation reports about our work:

Summative Evaluation of Power Up! A Mobile Game for Maryland Science Center's Power Up! Exhibit Comparison evaluation of children visiting a museum exhibit; half played a companion mobile game and half did not. Overall, using the game within the exhibit had clear value in promoting appealing engagement and learning.

Summative Evaluation of "PlanetMania" Mobile App in Maryland Science Center's "Life Beyond Earth" Exhibit The evaluation found that PlanetMania creates a new opportunity for youth to engage with a variety of museum exhibits, adding value both in terms of deepening exhibit engagement and increasing knowledge.

WolfQuest Summative Evaluation Report. This evaluation of the WolfQuest learning game, conducted by the Institute for Learning Innovation, found the game to be highly effective in achieving its goals, providing a rich and rewarding learning experience for players.

A Rolling Evaluation Gathers No Moss. Continuous, formative evaluation with rapid revisions was key to the successful development of Kids Design Network (KDN) by the DuPage Children's Museum and Eduweb. Paper presented at Museums & the Web .

Summative Evaluation of the Dakota Experience Pilot Module
At Eduweb, we think a lot about our practice. We evaluate our projects with users. And we conduct research into games, players, and learning — all to improve our methods and bring us ever closer to our goal of developing the most engaging and effective digital learning experiences possible.


Here are the products of these efforts, often done in collaboration
with top researchers and evaluators in the museum field.
Since the cabinets of curiosity of yore, museums have collected widely and deeply, building collections that they now draw upon to interpret the diversity of the natural world and the heights of human achievement and expression. A game, in contrast, maintains a singular focus on a system of rules that ties together game objects and player actions. How, then, can a game maintain that strict focus while doing justice to the rich, heterogenous collections and content of a museum? This paper explores the nature of this challenge and examines how a handful of museum games respond to it, using either extrinsic or intrinsic gameplay. Extrinsic games adopt existing game mechanics as generic containers for content. Intrinsic games devise novel gameplay that integrates content into the game choices and consequences. Museum game developers must understand the benefits and risks of these contrasting approaches, as well as the implications for development effort, playtesting, and learning outcomes, so they can make the best design choices for their project. Read More

Playtesting PlanetMania: a Mobile Game for Museum Exhibits
David T. Schaller, Eduweb, and Barbara Flagg, Multimedia Research, 2013.
PlanetMania is an iOS and Android mobile game designed to be played with the Maryland Science Center's new Life Beyond Earth exhibit. Intended for preteens, the card-based gameplay expands upon exhibit content and encourages interaction with the physical exhibit. Through extensive paper prototyping and iterative development, the project team revised and simplified the game content and interactivity, striving for intuitive game rules, age-appropriate scientific content, and engaging game play and learning outcomes   all in a museum environment where players have plenty of distractions. Read More

In the late 20th century, museums moved from traditional methods of knowledge transmission to constructivist interpretive methods such as narrative, a transition that many found challenging. Today, museum educators wishing to adopt game-based learning methods face a similar challenge: to move from now-familiar narrative methods to a systems-based approach in which rulesets define player choices and subsequent consequences. These rules represent some aspect of the subject matter, while also revealing the designer's perspective on it. Rules create the space for the players to author their own experience, made more meaningful because it was shaped by their own choices, actions, and struggles within the system. Read More

The Meaning Makes It Fun
David T. Schaller, Eduweb. 2011.
Museum games can be a powerful meaning-making experience for players, but only if we understand that what makes games fun is also what makes them meaningful. Renowned game designer Sid Meier (Civilization, Railroad Tycoon, Pirates) famously defined a good game as "a series of interesting choices." What makes choices interesting? The same thing that makes them meaningful: consequences, context, and a savvy appeal to human psychology. When designed well, these choices can make even mundane content meaningful. So imagine the possibilities when we use the stuff of museums — art, science, history and technology — as our content and context. Whether creating a game or a simpler activity, we can draw on principles of game design to weave a series of interesting choices that honors our real-world content while engaging visitors in thoughtful ways. This article considers the types of choices offered by that most common type of "game" — the quiz — before analyzing the board game Monopoly for attributes of interesting choices, and finally discusses how those attributes are built into several games for cultural institutions. Read More

The Player's Voice: Using evaluation to bring the player into the development process
David T. Schaller, Eduweb
Kate Haley Goldman, National Center for Interactive Learning, Space Science Institute. 2011.
The state of learning game evaluation currently resembles that of museum website evaluation a decade ago: designers recognize the value of evaluation but struggle to find appropriate questions, methods, and strategies to incorporate evaluation into the development process. In recent years, much academic research has been conducted on the learning affordances of commercial games such as World of Warcraft, and on in-school and afterschool based games. But evaluations of learning games played at home and in other free-choice environments are far less common. In this chapter from Museums at Play: Games, Interaction and Learning we draw on our experiences as a developer (Schaller) and an evaluator (Haley Goldman) of museum learning games to explore the challenges and rewards that arise when incorporating the player's voice into the game development process. Read More

Learning in the Wild: What WolfQuest taught developers and game players
David T. Schaller, Eduweb
Kate Haley Goldman, Institute for Learning Innovation
Grant Spickelmier, Minnesota Zoo
Steven Allison-Bunnell, Eduweb
Jessica Koepfler, Institute for Learning Innovation. 2009.
Summative evaluation of our WolfQuest wildlife simulation game finds that players report knowledge gain, stronger emotional attachment to wolves, and significant behavioral outcomes, with large percentages of players following their game sessions with other wolf-related activities, including such further explorations of wolves on the internet, in books and on television. This paper details these evaluation results from the summative evaluation, discusses the theory behind the project, and reflects on our experience developing the game. Read More

The Name of the Game
Susan Edwards, J. Paul Getty Museum
David T. Schaller, Eduweb. 2007
This chapter from The Digital Museum: A Think Guide discusses ways that digital learning games offer museums new opportunities to engage youth and adult audiences in compelling and meaningful ways. Read an excerpt and order the book! Read More

One Size Does Not Fit All: Learning Style, Play, and Online Interactives
David T. Schaller and Steven Allison-Bunnell, Eduweb
Minda Borun, The Franklin Institute
Margaret Chambers, Consultant. 2007
In creating educational experiences, developers often target audience segments based on demographic groups. However, we all know that people vary in other significant ways. Particularly with regard to learning styles, one size does not fit all. This paper presents research findings from our study, funded by the National Science Foundation, of the effect of learning style on user preferences for different types of online learning activities, ranging from deductive puzzles to open-ended creative design. Read More

What Makes a Learning Game?
David T. Schaller, Eduweb. 2005
What are key characteristics and challenges of an effective learning game? Adapted from a presentation at the Web Designs for Interactive Learning conference in Ithaca, June 2005. Read More

Learning Styles and On-line Interactives
David T. Schaller and Steven Allison-Bunnell, Eduweb
Minda Borun, Franklin Institute Science Museum. 2005
Learning styles may give us insight into the diverse ways that people view and interact with on-line learning interactives. This paper provides an introduction to our NSF-funded research study. Read More

From the Physical to the Virtual: Bringing Free-Choice Science Education Online
Steven Allison-Bunnell and David T. Schaller, Eduweb. 2005
This chapter from E-Learning and Virtual Science Centers proposes a series of strategies for reconceptualizing science center exhibits online, in order to take online users deeper into the scientific concepts underlying the physical phenomena on exhibit in the physical galleries. Read More

To Flash or Not To Flash? Usability and User Engagement of HTML vs. Flash
David T. Schaller and Steven Allison-Bunnell, Eduweb
Anthony Chow, Paul Marty and Misook Heo, Florida State University. 2004
Macromedia Flash as a useful tool that allows greater interactivity and multimedia compared to HTML pages, but how does it affect usability and user engagement? This paper reports on a comparative evaluation of Flash and HTML versions of a single Web site, focusing on user goals, behavior, and responses. Read More

Exploring Motivational Factors and Visitor Satisfaction in On-line Museum Visits
Kate Haley Goldman, Institute for Learning Innovation
David T. Schaller, Eduweb. 2003
Why do people visit a museum web site, and how do these motivations affect their experience with the site and the learning or meaning-making that may happen as a result of their visit? This paper builds on past research by analyzing an online survey of visitors to four museum Web sites. Read More

Transplanting learning theory from the classroom or museum environment to the Web poses unique challenges. In this paper, we review several theories of learning and explore ways that we have tried to incorporate them into our development and design process for interactive Web sites. Read More

How Do You Like To Learn? Comparing User Preferences and Visit Length of Educational Web Sites
David T. Schaller and Steven Allison-Bunnell, Eduweb
Minda Borun and Margaret B. Chambers, Museum Solutions. 2002
Developing effective public education sites for the World Wide Web requires an understanding of both learning theory and what appeals to learners. How can Web developers create sound educational activities that attract and appeal to a broad audience? Do adults prefer different types of online learning experiences than children? Read the report! Read More

Developing Goal-Based Scenarios for Web Education
David T. Schaller, Steven Allison-Bunnell and Susan Nagel, Eduweb. 2001
The theory and research underlying our "How do you like to learn?" study. Based on a paper for the National Association of Interpretation 2001 Conference. Read More