Company Services Portfolio Research Contact
 
Research Logo: eduweb
 

How Do You Like To Learn?
Comparing User Preferences and Visit Length of Educational Web Sites

David T. Schaller and Steven Allison-Bunnell, Educational Web Adventures
Minda Borun and Margaret B. Chambers, Museum Solutions

Presented at Museums and the Web 2002

Abstract

Developing effective public education sites for the World Wide Web requires an understanding of both learning theory and what appeals to learners. A recent study commissioned by IBM found that Web learners prefer passive entertainment experiences to more demanding interactive experiences (Karat et al, 2001). If people learn best in active modes, but prefer passive Web experiences, how can we develop sound educational activities that attract and appeal to a broad audience?

This paper reports results of a study designed to determine people's preferences for different types of Web-based educational activity. The primary research question was: How do people's preferences vary among types of Web-based learning activity? We identified six activity types for comparison: Creative Play, Guided Tour, Interactive Reference, Puzzle/Interactive Mystery, Role-playing Story, and Simulation.

A team of Web developers who work with museums and other learning sites collaborated with a team of educational researchers who work primarily with museums to conduct a survey of visitors to five different types of educational Web site. Two kinds of data were collected:

  • User exit surveys, eliciting an evaluation of the study site and preferred genre or type of learning activity.
  • Server statistics indicating the duration of stay.

Results indicate that there are clear differences in the type of Web-based learning activity preferred by adults and children. Adults are more likely to select Interactive Reference or Simulation whereas children prefer Creative Play and Role-playing Stories. The adult sites yield more straightforward cognitive information while the sites preferred by children allow more personal choice and interaction. Apparently, adults bring an intrinsic motivation to the learning experience. They know what they want to learn and they want to learn it in the most direct way. Children, on the other hand, need to be motivated. They respond positively to the opportunity for interaction and choice within a goal-based environment that offers them an extrinsic purpose.

Keywords: learning preferences, learning theory, Web-based education, goal-based scenario, intrinsic motivation.

Applying Learning Theory to Interactive Media

In the past decade, the World Wide Web has grown from a text-only tool of academia to a dazzling universe of ideas, community, commerce, and vanity, with a corresponding increase in its multimedia capabilities. How can the Web best be used for education? Applying learning theory to an immature medium like the Web is challenging, but several basic criteria for learning can safely be applied:

  • "Education is not an affair of 'telling' and being told, but an active construction process"(Dewey, 1923).
  • "Learners do not learn directly from technology [or teachers, or books]; they learn from thinking about what they are doing" (Jonassen, 1999).
  • Learners must be motivated, which requires an "emotional connection, challenge, and payoff" (Healy, 1994).

Somewhat more controversial is a key tenet of constructivism: "A range of results are possible and acceptable" (Hein, 1998). This tolerance for divergent outcomes distinguishes constructivism from discovery learning, in which "by engaging learners in activity...they will arrive at the correct conclusions" (Hein, 1998). Constructivism suggests that learning activities should allow multiple outcomes, each of which need only "'make sense' within the constructed reality of the learner" (Hein, 1998). (For additional analysis of theories of learning applied to the Web, see Schaller and Allison-Bunnell, 2001)

Beyond pedagogical approach, museums and other organizations devoted to leisure learning must decide on the desired type of educational experience. Gammon (2001) offers a useful typology:

  • Cognitive: Acquire and assimilate new knowledge into existing schemas, apply existing knowledge, connect concepts, draw analogies.
  • Affective: Challenge beliefs and values, appreciate view-points in other people, inspire interest, curiosity, awe and wonder, associate curiosity and thinking with enjoyable experiences.
  • Social: Develop skills of co-operation and communication.
  • Developing skills (mental and physical): Prediction, deduction, problem-solving, investigation, observation, measuring, classification, testing theories, making and telling stories, decision-making, manual dexterity, craft skills, etc.
  • Personal : Increasing self-confidence and self-efficacy; motivating to investigate further.

With these issues in mind, how has the Web fared? The Web is a form of interactive multimedia, or IMM. Some educational researchers and practitioners praise IMM's ability to use audio, video, text, and immersive environments to appeal to multiple intelligences (Veenema and Gardner, 1996). Others see in IMM the chance to move beyond passive learning modes and engage students in more active learning experiences (Prensky, 2001; Crawford, 1982; Viadero, 1996; Tipping and Graesser, 1996; Bearman, 1997; Plowman, 1996b).

However, actual evaluation of IMM products has shown that logistical problems often get in the way of fully realizing IMM's potential. Many novice learners have found the navigational choices offered in IMM programs bewildering. In studies of classroom use of IMM, students have needed considerable teacher assistance to make use of the programs. (Veenema and Gardner, 1996; Plowman, 1996b; Bearman, 1997). The problem lies in the very freedom afforded by IMM's non-linear structure: "Being a user-controlled medium, the learner expects to have control, and yet a learner does not know enough to be given full control" (Laurillard, 1996). Novice learners need more guidance and structure to ensure that they find content that is both engaging and appropriate to their knowledge level.

While classroom teachers with sufficient time, skill, and motivation can overcome these difficulties and provide the necessary guidance to make use of the experience offered by IMM, this is not an option for Web-based leisure learning experiences. Web sites must attract an audience and create a self-contained experience that is satisfying and hopefully educational. Thus we must account for what people want as well as how they might learn.

A recent study conducted by IBM suggests that, given a choice, leisure learners seek relief from bewildering interactive software. This formative research revealed that "most participants did not express interest in Web sites that involved active interaction with the content or other people." They strongly preferred being "guided through an experience or discovery process" (Karat et. al, 2001). Some participants in the IBM study "viewed the more interactive design concepts and existing Web sites as work, not entertainment" (Karat et. al, 2001). Indeed, the learning modes that IBM researchers offered participants were either quite passive (Guided Tour) or quite active (a searchable database of images and information, a chat room, and an online journal). The latter may engage the devotee or a student doing a research report, but can easily overwhelm those who lack an existing interest in the subject and the intrinsic motivation to explore it.

Based on the results of their study, IBM developed a site featuring online tours, hosted by curators and other experts, and delivered via streaming video" essentially a TV-like experience with links to additional information. Summative evaluation of the site found that "users interacted relatively infrequently with the [online] tours, and the less they interacted, the more they reported feeling engaged and entertained by the experience" (Karat et. al, 2001). These results are disconcerting in that they contradict accepted learning theories that support the value of active involvement (Dewey, 1916).

A Research Study of Web-users Preferences

If people learn best in active modes but prefer passive Web experiences, how can we develop sound educational Web activities that attract and appeal to a broad audience? We decided to develop a detailed and focused pilot study of user preferences to shed more light on this complex and important issue. The primary research question was: How do people's preferences vary among types of Web-based learning activity?

We identified six activity types based on our previous Web development experience and a review of the literature (Gogg and Mott, 1993; Karat et al, 2001; Plowman, 1996b; Sumption, 2001). The six types, as described in the survey instrument, were:

  • Creative Play.Draw a picture, write a story, make a movie, etc. Create something original based on the things you learn along the way.
  • Guided Tour.Join an expert to explore a topic that he or she knows and loves. The guide leads you on the path they choose through the topic.
  • Interactive Reference. Explore a topic on your own, through informative words and pictures. Choose the links that interest you to find out what you want to know.
  • Puzzle/Interactive Mystery. Put on your thinking cap and solve a puzzle or mystery. Put the clues together to discover the right answer.
  • Role-playing Story. Choose your own adventure" pick a character, play a role, make decisions, and see what happens. You choose your path through the story.
  • Simulation.Run a model of the real world and see what happens when you change things. The choices you make determine the results.

Referring back to Gammon's typology of learning, Interactive Reference and Guided Tour lend themselves primarily to cognitive learning. Creative Play, Puzzle/Mystery, Role-playing Story, and Simulation support both affective learning and developing skills.  Creative Play will help learners with skills such as storytelling and art making; Puzzle/Mystery and Simulations with prediction, deduction, and other problem-solving skills. Role-playing Stories can challenge beliefs and values and help learners appreciate other people's points of view.

Methodology

In November-December 2001, we conducted a series of pilot studies with visitors to one site in order to test various versions of the exit questionnaire. The challenge was to describe the types of learning activity in such a way that preference for type of learning activity was not confounded by preference for the subject matter of the particular site or its visual appearance. We tried and eliminated screenshots of sample sites, since respondents were found to cue to content and aesthetics more than the general activity type. Long Likert scales (5- and 7- point) seemed to confuse respondents, who often indicated contradictory preferences over a series of questions.

Once we finalized the twelve-question survey, five activity sites previously developed by eduweb alone or in collaboration with its clients, were selected for this initial study to represent five of the six types of Web learning activity. No site exemplifying the Guided Tour was represented in the eduweb portfolio. However, it remained in the list of types about which visitors were queried. The exit questionnaire (Appendix A) was placed on each of five educational Web sites.

In addition to the exit survey, server statistics were used to determine the duration of stay. Summary, a log analyzer (www.summary.net), generated duration charts that give a clear picture of how long users spend at each site.

A pop-up window displayed the survey on each activity site; it appeared when visitors came to the initial page, and remained behind the main browser window until the visitor clicked to leave the site. Then the survey returned to the foreground. The surveys were posted on the activity sites for 10-20 days, until 50 responses from each site were collected. Table 1 outlines the sites, types of activity, and sample size.

A control group consisting of 299 visitors to the eduweb Web site filled out the first part of the questionnaire, which dealt with learning in general and did not reference a particular activity. Members of the control group did not engage in any of the Web activities selected for the exit survey. The purpose of the control group was to provide a measure of user preferences independent of a specific learning activity for comparison to questionnaires filled out at the activity sites.


Table 1. Experimental Design

 

Type

Name

Host Site

Number

Activity Sites

     
 

Creative Play

A Brush with Wildlife

Nat. Museum of Wildlife Art

50

 

Interactive Reference

Study Art

Sanford

50

 

Puzzle/Mystery

Leonardo's Workshop

Sanford

50

 

Role-playing Story

In Search of the Ways of Knowing Trail

Brookfield Zoo

50

 

Simulation

Modeling Marine Ecosystems (subscription site)

JASON Project

50

Control Site

     
 

Control

Web Adventure directory page

eduweb

299

In the following discussion, Control group results are compared to the Treatment population. The Treatment group consists of visitors to the five different Activity Sites. It is important to note that Treatment and Control are not used in the conventional way. We are not looking for post-treatment learning effects. Rather, we are comparing the preferences of users who have and have not experienced a particular Web activity. The purpose of the comparison is to be certain that observed user preferences are not solely a function of the activity in which they have just engaged.


Results

Table 2. Demographic Characteristics of Treatment/Control

 

Percents

Demographics

Activity Sites

Control

Adults

31

49

         Adult Males

11

12

         Adult Females

21

37

Children

69

51

         Boys

26

22

         Girls

42

30

The Activity sites had significantly more children than the Control site ( X 2p = .0001). An unexpectedly large number of adult females visited the control site.

Table 3. Demographic Characteristics of Visitors by Site

 

Sites (Percents)

 

Creative Play

Interactive Reference

Puzzle/ Mystery

Role-playing

Simulation

Control

Adults

25

56

32

32

13

49

      Males

6

8

14

20

7

12

      Females

18

48

18

12

7

37

Children

75

44

68

68

87

51

      Boys

35

10

16

32

37

22

      Girls

41

34

52

36

50

30

The most important differences ( X 2p = <.0001) are the large percent of adult females at both the Control and the Interactive Reference sites and the large percent of children at the Simulation site. Also the number of adult males at the Role-playing site was greater than expected, as was the number of girls at the Puzzle site.


Table 4. Favorite Type of Computer Learning Activity

 

Percents

Program Type

Activity Sites

Control

Creative Play

24

18

Guided Tour

8

9

Interactive Reference

23

18

Puzzle/ Mystery

15

19

Role-playing Story

21

18

Simulation

9

18

Totals 

100

100

The Activity sites differ significantly from the Control site ( X 2, p = .03). The respondents from Activity sites chose Creative Play and Interactive Reference more than expected. The Control site chose Puzzle/Mystery and Simulation more than expected. As will be seen below, the differences are probably due to the unexpectedly high percentage of adult females at the Control site and a significantly higher percentage of children at the Activity sites.

We see immediately that, contrary to the findings of the IBM study, Guided Tour was the least preferred type of Web activity for both the Treatment (Activity) and Control sites. The reasons for the difference between this and the IBM study are due to differences in both sample and methodology. IBM used adult subjects, ages 21-55. The subjects, employees and interns at an IBM research facility, were recruited to evaluate selected Web sites. IBM's sample sites dealt with the subject of music. In the study described here, children are a significant segment of the population, the user group at each site is voluntary and self-selected, and subject matter varies from site to site.

With the exception of the low scores for Guided Tour, the other types of learning activity seem to be about equal in user preference at the Control site. However, significant differences emerge when the user group is subdivided by generation and gender.

Table 5. Favorite Learning Activity by Age and Gender: Control Site

 

Percents

Percents

Program Type

Adults

Children

Adult Males

Adult Females

Boys

Girls

Creative Play

9

27

10

9

41

15

Guided Tour

10

7

10

10

4

10

Interactive Reference

26

11

20

29

10

13

Puzzle/ Mystery

21

18

30

19

12

24

Role-playing Story

13

23

10

14

16

28

Simulation

21

14

20

20

18

11

Totals 

100

100

100

100

100

100

There are a number of differences between generations ( X 2p = .0001). Adults preferred Interactive Reference and Simulation and children preferred Creative Play and Role-playing.

Gender differences between adults were not significant. However, differences between boys and girls were significant ( X 2p = .01). Boys showed a preference for Creative Play whereas girls favored Role-playing Story and Puzzle/Mystery.


Table 6. Favorite Learning Activity by Age and Gender: Activity Sites

 

Percents

Percents

Program Type

Adults

Children

Adult Males

Adult Females

Boys

Girls

Creative Play

9

31

10

8

33

31

Guided Tour

15

5

15

15

4

6

Interactive Reference

40

14

25

46

11

16

Puzzle/Mystery

13

17

20

10

14

19

Role-playing Story

16

23

25

13

29

19

Simulation

7

10

5

8

10

10

Totals 

100

100

100

100

100

100

Differences in program type preferences at the Activity sites are similar to those found at the Control site. Again there are significant differences between generations ( X 2p = <.0001). As at the Control site, adults prefer Interactive Reference and children prefer Creative Play and Role-playing. At the Activity sites, gender differences were not statistically significant.

Table 7. Favorite Learning Activity by Respondent's Site

 

Sites (Percents)

Program Type

Creative Play

Interactive Reference

Puzzle/ Mystery

Role-playing

Simulation

Control

Creative Play

45

10

28

16

23

18

Guided Tour

10

6

3

14

6

9

Interactive Reference

13

65

15

4

12

18

Puzzle/Mystery

10

8

21

25

12

19

Role-playing Story

18

6

18

35

32

18

Simulation

5

4

15

6

15

18

Totals 

100

100

100

100

100

100

There are significant differences in program type preferences among the Activity sites ( X 2p = <.0001). As mentioned above, with the exception of a lack of interest in Guided Tour, respondents at the Control site were almost evenly divided in their preference for program type. The Activity site respondents, on the other hand, tended to prefer the type of program they were using. This is notably the case with Creative Play, Interactive Reference and Role-playing Story.

It is important to remember that the Control site data was collected as a check on the tendency of Activity site users to prefer the type of program they are using. However, the similarity in preferences between Control and Activity sites (when subdivided by generation and gender) suggests that there are patterns in preferences that transcend the particular site.

Table 8. Location of Respondents

 

Percents

Locations

Activity Sites

Control

Friend's House

2

1

Home

40

40

Library

1

2

School

48

42

Work

7

11

Other*

2

4

Totals 

100

100

* Home school (3), Internet café (2) Work at Home (1), Grandparent's house (1), No answer (9). Numbers in parentheses () indicate number of responses.

There were no significant differences between the Activity sites and the Control site in terms of where the respondents were when using the computer. Home and School were the most frequent location for both Activity and Control.

The remainder of the questionnaire dealt with users" responses to the Web activity in which they had engaged, consequently, these questions were not asked at the Control site. The tables below show results from the five Activity sites.

Table 9. Location of Respondents by Site

 

Percents

 

Creative Play

Interactive Reference

Puzzle/ Mystery

Role-playing

Simulation

Friend's House

2

2

2

2

2

Home

13

76

59

40

10

Library

0

0

2

0

2

School

73

12

35

44

79

Work

13

6

2

12

2

Other*

0

4

0

2

4

Totals 

100

100

100

100

100

The differences are significant ( X 2p = <.0001). Interactive Reference and Puzzle/Mystery are accessed most from home while Creative Play and Simulation are often school activities.

Table 10. Previous Knowledge of Topic

 

Percents

 

Creative Play

(Art)

Interactive Reference

(Art)

Puzzle/ Mystery

(History)

Role-playing

(Natural History)

Simulation

(Ecology)

Little

40

18

48

41

28

Medium

38

43

24

45

50

Lots

23

39

28

14

22

Totals 

100

100

100

100

100

More people than expected indicated they knew "lots" for the Interactive Reference site and more indicated "little" for the Puzzle/Mystery site ( X 2p = .01).


Table 11. Why Doing This Web Activity?

 

Percents

 

Creative Play

Interactive Reference

Puzzle/ Mystery

Role-playing

Simulation

Assigned by teacher

69

16

26

28

77

Personal interest

6

20

28

38

5

Professional interest

10

8

6

6

2

Recommended by friend

2

0

2

2

0

Recommended by parent

0

0

4

4

2

To use in a lesson

10

36

17

9

5

Other*

2

20

17

13

9

Totals 

100

100

100

100

100

* Creative Play: to show son (1).

* Interactive Reference: for a project (4), info about artist (1), daughter's homework (1), find a picture (1).

* Puzzle/Mystery: fun (1), no answer (6).

* Role-playing Story: fun (3), exploring the Web (2) recommended by teacher (1).

* Simulation: no answer (5).

Teacher assignments are responsible for a high percentage of Creative Play and Simulation usage ( X 2p = < .0001). This corresponds with Table 9 that shows that these activity sites are most often used from school. Personal interest was unexpectedly high for Role-playing and the Interactive Reference site was most often accessed to use in a lesson.


Table 12. Has your Enthusiasm for this Topic Changed?

 

Percents

Decreased

12

Remained the same

46

Increased

43

Totals 

100

Most people reported that their interest in the subject matter had stayed the same or increased after the Web-based learning activity. Results were not significantly different from site to site.

Table 13. Why Respondents Left the Site

 

Percents

 

Creative Play

Interactive Reference

Puzzle/ Mystery

Role-playing

Simulation

Finished it

60

54

48

75

45

Got bored

9

4

21

9

23

Got confused

2

0

5

2

10

Ran out of Time

24

20

18

9

10

Other*

4

22

9

6

13

Totals 

100

100

100

100

100

* Creative Play: don't have Java (1), no answer (1).

* Interactive Reference: got what was wanted(4), didn't get what was wanted (3), Knew everything (1), going to play art game (1), no answer (3).

* Puzzle/Mystery: couldn't find answer (1), done with it (1), just got started (1), text too big (1).

* Role-playing Story: slow (1), not what I was looking for (1) finishing school work (1).

* Simulation: no answer (6).

The differences here are significant ( X 2p = .007). More respondents than expected left the Simulation site because they were confused. Respondent's tended to finish the Role-playing Story more often than expected and respondents left the Interactive Reference site for "other" reasons. The reasons were likely to be that the information sought was found or not found. Actually "found what was wanted" might also be interpreted as "finished", which if coded that way would increase the percent in Interactive Reference who felt they had "finished it".

The two sites with the highest percentage of users who "finished" the experience are Creative Play and Role-playing story. These are also the sites that are most preferred by children. Perhaps the goal-based structure with a clear ending is part of the appeal of the two sites.

Table 14. How does this Activity Compare to Others of Its Type?

 

Percents

Worse

15

Equal

39

Better

45

Totals 

100

Most people felt that the Activity site was as good or better than other similar sites. There were no significant differences from site to site in users' comparison of the quality of the site.

Additional data was gathered in the form of server logs, which were analyzed for duration information.

Table 15. Time at Site

Program Type

Ideal Time

Mean Time

Mean Time 2* (minus outliers)

Percent of Ideal (Mean 2/Ideal)

Creative Play

21 min

6 min 19 sec

7 min 48 sec

37

Interactive Reference

 

7 min 25 sec

7 min 54 sec

--

Puzzle/Mystery

20 min

5 min 29 sec

7 min 42 sec

38

Role-playing Story

19 min

7 min 46 sec

12 min 24 sec

65

Simulation

25 min

16 min 40 sec

14 min 42 sec

59**

*Outliers (under 8 seconds or an hour or more) are eliminated.
** This activity required written journal entries.

We made a crude measure of the "Ideal Time" it takes a person to go through the activity by timing an adult clicking on and reading all the available content. We electronically collected actual time spent at the site. The mean is given first for the whole group and then minus the outliers, excluding those who left immediately (presumably because the site did not offer what they were looking for or did not have the required technology) and those who stayed more than an hour (possibly because they left their browser on the site after finishing the activity).

A comparison of mean time to Ideal Time shows that unusually long times were spent at the Role-playing Story and Simulation. However, many children were engaged in the Simulation Activity because their teacher assigned it to them (see Table 11); they were required to complete the Simulation and submit journal entries with their conclusions.

Apart from the Role-playing Story and Simulation, the mean times are very similar, indicating that these activity types have comparable holding power. The duration charts that follow reveal in more detail the differences in holding power of the various activity types.

Table 16: Duration of Visit

Table 16
[Click to Enlarge]

At most sites, with the exception of Creative Play and Simulation, a sizeable number of visitors leave within the first seven seconds. The explanation for this finding may lie in the specifics of the individual sites rather than in their Activity type.

The Marine Ecosystems Simulation was assigned as schoolwork for three quarters of the users of the site. The Creative Play activity starts with a series of animations that may hook visitors more effectively than the text introductions to the Interactive Reference and Puzzle/Mystery sites. The Role-playing site begins with a splash/Flash plug-in detection page that turns away a sizeable percentage of visitors apparently due to the wait involved, rather than lack of the Flash plug-in (only 16% of visitors did not have the plug-in).

The remainder of each chart is more revealing. Interactive Reference shows a bell curve. The other sites have skewed distributions, indicating greater holding power, after the initial drop-off. Most striking is the curve for Role-playing Story. Half of the visitors who got past the splash page (and 25% of all visitors) stayed for at least fifteen minutes, approaching and even surpassing the ideal time (19 minutes) for that site. The curves for Creative Play and Puzzle/Mystery are less dramatic, but suggest a similar pattern.

Conclusions

Comparisons of users' responses to an exit questionnaire posted on five Web-based learning activity sites and at a control site answer some fundamental questions about users preferences for different types of Web-based learning. An earlier study of this subject conducted by IBM found that adult users prefer Guided Tours or non-interactive Web experiences. In the current study, using self-selected subjects and including a large proportion of children, Guided Tour was the least preferred type of Web activity for both the Treatment (Activity) and Control sites.

With the exception of the low scores for Guided Tour, the other types of learning activity seem, at first, to be about equal in user preference at the Control site. However, when the user group is subdivided by generation and gender, significant differences emerge having to do with the user's age, location and purpose for engaging in the activity.

There are significant age differences in preferences. At both the Activity and Control sites, adults prefer Interactive Reference while children prefer Creative Play and Role-playing Stories. Adult females are over-represented at the Control site. Judging by the Activity sites, many of these adult females may be teachers who use Web sites for their lessons. The Creative Play and Simulation sites used here are often school activities and are usually assigned to the children by a teacher.

Activity site respondents tended to prefer the type of program they were using. The Control site data was collected to counter this tendency. The similarity in preferences between Control and Activity sites (apart from differences in demographic composition) suggests that the observed preference patterns transcend particular sites.

Most people reported that their interest in the subject matter had stayed the same or increased after the Web-based learning activity. Also, most users felt that the Activity site was as good as or better than other similar sites. This suggests that production values did not heavily influence preference for activity type.

Implications of the Web Learning Preference Study

There are clear differences in the type of Web-based learning activity that adults prefer in comparison to children. Adults prefer the information-based activities of Interactive Reference and Simulation, whereas children, not surprisingly, are more inclined to prefer the exploratory experiences of Role-playing Story and Creative Play.  The adult sites yield more straight-forward cognitive information while the sites preferred by children have strong affective components and allow more personal choice and interaction, but can lead to "dead ends" or less utilitarian solutions. Apparently, adults bring an intrinsic motivation to the learning experience. They know what they want to learn and they want to learn it in the most direct way. Children, on the other hand, need to be motivated. They respond positively to the opportunity for interaction and choice within a goal-based environment that offers them an extrinsic purpose.

Goal-based environments are advocated by Roger Shank, director of the Institute for the Learning Sciences at Northwestern University. He describes them as "Goal-Based Scenarios" (GBS)—structured learning programs that can be successful in both physical and virtual environments. The goals in these scenarios are not arbitrary extrinsic motivations, such as a good test score, prize, or reward. Rather, they stem from the activity itself—solve a crime, reach a destination, create an original artwork" which reinforces the cognitive goals of the activity. Thus, GBSs " provide motivation, a sense of accomplishment, a support system, and a focus on skills rather than facts" (Schank, 1992). In this way, they meet the basic criteria for learning of Dewey, Jonassen, and Healy cited above. They create an environment for doing and thinking, and provide both a challenge and a payoff. If designed properly, they can also connect with pre-existing knowledge and help forge an emotional connection with the subject matter.

Of the six types of Web-based learning activities explored here, two (Puzzle/Mystery and Role-playing Story) are naturally suited to the GBS approach; they inherently provide a motivation to reach a solution. Creative Play and Simulation, on the other hand, may or may not establish a clear goal. If designed as a GBS, each of these four activity types offers a goal or challenge, a payoff, structure and guidance, and some degree of interactivity. Young or novice learners who are unfamiliar with a particular learning domain need such guidance and structure to attract and hold their attention. It is interesting to note that in the Creative Play, Role-playing Story and Puzzle/Mystery activities, a plurality of users indicated they knew "little" about the subject to begin with (Table 10).

In contrast, Guided Tour and Interactive Reference are not goal-based scenarios. There is no payoff or achievement for completing the activity; learners must bring their own intrinsic motivation to the task. Users of these Web sites may have more expertise in the subject as well. On the Interactive Reference site, more people than expected indicated they knew "lots" about the subject (Table 10).

Within the structure and guidance provided by GBSs, young learners prefer some degree of freedom. Creative Play and Role-playing Stories, both preferred by children, offer a series of choices in the path of the activity and some control over the outcome. Puzzle/Mystery, which was less favored, and Guided Tour, the least popular type, offer only one outcome and less opportunity for personal involvement.

This brings us to another important dimension: whether the outcome is determined (created by the site developers) or user-created. Different pedagogies underlie the two types of outcome. Discovery learning lends itself to puzzles and mysteries, with their single correct solution (determined), while constructivism supports user-created outcomes that allow more personal choice and involvement. Most of the activities rely on determined outcomes, but Creative Play, and to a lesser extent Simulation, permit learners to create their own outcomes, be it a picture, a story, or a unique configuration of the variables in the simulation.

Putting the Results into Practice

First, developers of educational Web activities must decide whether their primary audience is adults or children, since the two groups have different learning preferences. Adults prefer reference sites. Children prefer goal-based scenarios, particularly Creative Play and Role-playing Stories. These activity types offer an appealing middle ground between the highly structured and constrained Guided Tour approach, and the "explore according to your own interests" reference format typical of much interactive multimedia. If a site must appeal to both children and adults, developers should consider a dual approach, combining reference and play (as in Leonardo's Workshop and Study Art).

Second, developers must decide on a pedagogical approach. Guided Tour and Interactive Reference sites typically provide a traditional expository/didactic approach to learning. Puzzle/Mystery activities reflect a discovery learning orientation, in which a single correct solution or conclusion is the goal. Creative Play takes a constructivist approach, encouraging open-ended experience. Simulations and Role-playing Stories can take a variety of forms, from discovery to constructivism.

Developers should also consider their audience's expertise in the subject. Expert learners with existing interest in the domain are more likely to favor interactive reference sites. Novice learners, regardless of age, are more likely to need and prefer a guided experience to introduce them to the subject and motivate them to learn more about it.

The audience's degree of expertise also affects the learning goals of a Web activity. For novices, affective learning experiences can inspire interest and curiosity in a subject, while skill-building activities help them to develop the ability to pursue further understanding. For experts who already have the knowledge base and skills to tackle the subject, interactive reference sites can provide a satisfying cognitive learning experience. For novices, goal-based scenarios can combine affective and skill building learning experiences with cognitive learning.

Of course, just because a Web activity attracts and holds users' interest doesn't mean it is achieving its educational goals. Evaluating learning outcomes is no easy task, and a matter for another study. However, a Web activity or any other learning activity must first attract and hold the interest of learners in order to have the opportunity to achieve its learning objectives.


Appendix A: Web Activity Preference Survey

When you are done exploring the Study Art section of this Web site, we would like to ask you a few questions. Your responses will help us develop better Web sites in the future. Your responses are anonymous and will be kept confidential.

Please complete this survey when you are leaving Study Art.


Are you:

Male

Female


How old are you?

5-8

9-10

11-13

14-18

19-22

23-35

36-49

50+


Where are you now?

Home

School

Work

Library

Friend's house

or Other: [enter text]


How much did you know about art before you came to this Web site?

Very little

Medium

A lot


Why were you looking at this Web site?

Assigned by teacher

Recommended by a friend

Recommended by parent

To use in a lesson

Personal interest

Professional interest

 or Other: [enter text]


Why are you leaving this Web site?

Found what I was looking for

Ran out of time

Got bored

Got confused

 or Other: [enter text]


Study Art is an Interactive Reference/Encyclopedia Web site. How does it compare to other such sites you've seen on the Web?

Much worse

Equal to

Much better


What would you say Study Art is about?

[enter text]


What is one thing you learned that you didn't know before?

[enter text]


Now that you're leaving Study Art, has your enthusiasm for art changed?

Decreased

Stayed the same

Increased


What is your favorite type of computer learning activity? There are six types of activities below. Please choose one:

Guided tour. Join an expert to explore a topic that he or she knows and loves. The guide leads you on the path they chose through the topic.

Interactive reference/encyclopedia. Explore a topic on your own, through informative words and pictures. Choose the links that interest you to find out what you want to know.

Role-playing story. Choose your own adventure, pick a character, play a role, make decisions, and see what happens. You choose your path through the story.

Creative play. Draw a picture, write a story, make a movie, etc. Create something original based on the things you learn along the way.

Simulations. Run a model of the real world and see what happens when you change things. The choices you make determine the results.

Puzzle or interactive mystery. Put on your thinking cap and solve a puzzle or mystery. Put the clues together to discover the right answer.


If you have any comments about this survey or Study Art, please write them here:

[enter text]



References

Crawford, C. (1982). The art of computer game design. Reprinted online at: www.vancouver.wsu.edu/fac/peabody/game-book/Coverpage.html

Dewey, J. (1916). Democracy and education: An introduction to the philosophy of education.

Gammon, Ben. (2001). Assessing learning in museum environments: A practical guide for museum evaluators. Unpublished Science Museum (London) report.

Gogg, T. J., Mott, J. R. A. (1993). Introduction to Simulation. Proceedings of the 1993 Simulation Conference, ed. G. W. Evans, M. Mollaghasemi, E. C. Russell, W. E. Biles.

Healy, Jane M. (1994). Your Child's Growing Mind: A practical guide to brain development and learning from birth to adolescence. New York: Doubleday.

Hein, G. (1998). Learning in the museum. London: Routledge.

Jonassen, David H., Kyle L. Peck, Brent G. Wilson. (1999). Learning with technology: A constructivist approach. Upper Saddle River: Merrill.

Karat, C., Karat, J., Claudio Pinhanez, Mark Podlaseck, John Vergo, Doug Riecken, Renee Arora, Thomas Cofino. (2001). Less clicking, more watching: Results from the user-centered design of a multi-institutional web site for art and culture. Museums and the Web 2001: Selected Papers from an International Conference, ed. D. Bearman, J. Trant. Pittsburgh: Archives & Museum Informatics.

Karat, C., Claudio Pinhanez, John Karat, Renee Arora, & John Vergo. (2001b). Less clicking, more watching: results of the iterative design and evaluation of entertaining web experiences. Interact '01.

Murray, J. H. (1997). Hamlet on the holodeck: The future of narrative in cyberspace. Cambridge: MIT Press.

Laurillard, D.M. (1996). Multimedia and the learner's experience of narrative. Keynote speech for Ed-Media, Boston 1996. meno.open.ac.uk/homerpub.html.

Loving, C. C. (1997). From the summit of truth to its slippery slopes: Science education's journey through positivist-postmodern territory. American Educational Research Journal 34(3): 421-452.

National Research Council. (2000). Inquiry and the national science education standards: A guide for teaching and learning. Washington: National Academy Press.

Plowman, L. (1996). Designing interactive media for schools: A review based on contextual observation. Information Design Journal 8 (3): 258-266. meno.open.ac.uk/meno-rev-idj.html.

Plowman, L. (1996b). Narrative, linearity and interactivity: making sense of interactive multimedia. British Journal of Educational Technology 27 (2): 92-105.

Prensky, M. (2001). Digital game-based learning. New York: McGraw Hill.

Schaller, David T., and Steven Allison-Bunnell (2001). Developing Goal-Based Scenarios for Web Education. www.eduweb.com/goalbasedscenarios.html

Schank, R. 1(992). Goal-based scenarios. Chicago: Northwestern University Institute for the Learning Sciences. cogprints.soton.ac.uk/documents/disk0/ 00/00/06/24/cog00000624-00/V11ANSEK.html.

Spectre, M. and M. Prensky. (ND). Theoretical underpinnings of games2train's approach. www.games2train.com/site/html/theory.html.

Sumption, K. (2001). Beyond museum walls -- A critical analysis of emerging approaches to museum web-based education. Museums & the Web 2001 Proceedings, ed. D. Bearman, J. Trant. Pittsburgh: Archives & Museum Informatics. www.archimuse.com/mw2001/papers/sumption/sumption.html.

Tipping, P.D. & Graesser, A.C. (1996). Active learning, passive learning, software, and texts: Does it really make a difference? Paper presented at the Sixth Annual Meetings of the Society for Text and Discourse, San Diego, CA.

Turkle, S. (1997). Seeing through computers: Education in a culture of Simulation. American Prospect. 8 (31). www.prospect.org/print/V8/31/turkle-s.html.

Veenema, S. and H. Gardner. (1996). Multimedia and multiple intelligences. American Prospect, 7(29). www.prospect.org/print/V7/29/veenema-s.html.

Weatherford, J. ND. Instructional Simulations: an overview. (ND). San Diego State University Dept. of Educational Technology. coe.sdsu.edu/eet/Articles/instrucsimu/start.htm

 

Back to top

 

eduweb