Skip to main menu Skip to main content

Evaluation of the Techniquest Web Terminals

Communicating Science through New Learning Environments, April 2001, Module 6

Contents

Aim of evaluation

The aim of my study is to evaluate the effectiveness of the Internet terminals currently used in the HuB as a learning resource for the public.

Introduction

Learning with the Internet in science centres

With the introduction of interactive multimedia terminals into the science centre environment, we allow visitors the opportunity to move from a glancing experience into a deeper involvement with the topic.

People’s interactions with multimedia systems like the Internet terminals in the Techniquest setting will be very different from those of the school or office. The users of the system (as is said by R Semper & K H Woolsey) will often be standing, operating in groups consisting of mixed ages and may be passing by with little time to stop and learn how to operate the controls.

With the recent developments in technology and the explosion of personal use of multimedia computing, its educational potential is now being exploited:

“Access to ICT (Information and Communication Technology) is a vital part of young people’s education. We are working hard to connect every school to the Internet, and to provide the equipment necessary to help schools make the most of the opportunities it offers.” (Rt Hon Tony Blair MP, from DGfL press release)

Many of the newly opened science centres across the country are seeing the potential of web based exhibits, with screens allowing the user to chose from many different web cams all over the world and terminals which allow them access to chosen web sites. As a science centre Techniquest is now almost expected to provide such services. (For description of the Internet and WWW see appendix 1)

According to the CSUP Report 1992 development of understanding involves a series of learning phases:

  • Preparing to tackle the material
  • Acquiring the relevant material
  • Interpreting the material
  • And so develop personal understanding

So for the terminals it would mean the user sitting down at the machine wanting to look up some of the information, being focused on what they are going to do i.e. sit down and have a learning experience. Next is using the navigation set up to follow the links to the relevant information. Here we are part guiding them to the information by providing the direct links to what we think is relevant. Next the user must remain and read the information or look at the images. Finally the developing an understanding is helped or hindered by the quality of the material of each site.

So if these stages are thought to be what people require for an effective educational experience are they actually interacting this way with the equipment? Are the users making use of the links and support around them, or are they merely ‘playing’ on the machines?

The HuB

As an addition to the Techniquest facilities the old cyber library has recently been converted into the newly christened ‘HuB’.

The Hub is a circular room that is part of the first floor of the Techniquest exhibition. Where as the old cyber library was a small self-enclosed room, now one of the walls has now been demolished to openly connect the room with the rest of the floor. It is no longer a separate room but appears part of the exhibition space.

The HuB has three main components:

  1. A plasma screen
  2. Modular work benches
  3. And 4 Internet terminals

The area is used to provide back up information to the public programs shows that are running in the science theatre and planetarium at the time. At the time of study the area was being used in relation to the planetarium show ‘Lights in the sky’. The room was decorated with posters about astronomy and the plasma screen was running a power point presentation with information about what was said during the show. Three workbenches were set up with exhibits relating to stars and observing. Then finally we see the four Internet stations that had been set up with links to topic related web sites.

The room itself was very light, with half of the room open to the exhibition floor and the majority of the rest was a glass wall with only one area closed to house the plasma screen. It had a very open feeling, with light coming in from all sides and lots of soft lighting in the outer ring of the circular roof. The glass windows look out over the rest of the exhibition, providing a great vantage point from which to take in most of the floor.

The area is open to visitors all the time, not just on the weekends. Allowing it to be used by both weekend visitors and weekday school groups.

The web terminals

The terminals themselves consisted of a computer screen, a selection button, a roller ball and an instruction sheet. The instructions are very basic. They explain that you can use the roller ball to move the arrow around the screen and then you can use the button to select an item. Around the back of the terminal we can only see two leads, a power lead and a phone wire. They are placed around the room with large gaps between each terminal. There are stools available for the users to sit on if required.

The starting home page screen on the terminals looks like this:

It appears like a basic Internet explorer but with only forward, back and home buttons present. We then see the graphic showing which public programs show the information was themed to, in this case ‘lights in the sky’ and then below that we have the pre-set options of sites to visit.

To look at each of the web sites the user has only to click on the link. Each of these 6 sites then opens in a window with the same functions bar running across the top of the screen. So you can always return to the starting page by clicking on ‘Home’

Methodology of evaluation

From the beginning it was clear that due to time restraints the sample size for my evaluation was going to be small. So I decided to make the most out of the available resources and combine all of the interactions which I had on the subject instead of merely limiting myself to just one of them.

The first thing which I did was to sit down and spent time using the equipment myself. I felt that this would give me a better understanding of the system and its uses.

I then carried out the study over a half term week and part of a school week in the HuB area.

From what I had already observed of the area I knew that the turn over of people using the terminals would be very high, but that their interactions with the equipment often would be short. So I decided observation of visitors and their interactions would provide a large amount of information. The area around the terminals is very open and so provides many places from which to unobtrusively observe.

Due to the time constraints of the study the number of interactions with visitors was expected to be very low. So it was judged that the most efficient means of finding out what people had to say about the terminals would be to ask visitors orally rather than getting them to fill in a questionnaire. I did consider a written questionnaire but felt that this was not in line with the feeling of the HuB area. The informal interviews were most carried out once the person or group had moved away from the terminals, so to allow other users to take over the terminal. But in some cases I was called over by the person/group and asked a question about the terminals, in which case I extended my interaction with them into the interview.

Thirdly certain members of staff appear to send large amounts of time in the area helping people use the equipment. So they seemed to provide a large amount of experience in visitors use of the terminals. I approached the staff and asked them for their views about the HuB, this could be done individually or by going to the front desk where I could talk with a group of the staff members.

Results

My own use

When I first sat down and starting using them myself the main things that I notice about the terminals are that there are no keyboards around and secondly that most of the links are actually quite quick.

But most interestingly I also noted that the information and labelling on the terminals does not mention the fact that they are linked to the Internet. So unless you notice for yourself and you stay within the 6 pre-set options then the user may think that they are just looking at information on a CD-ROM or saved on the computer system.

Who uses the terminals?

During the study it appeared that the terminals are used by people of all ages.

Older visitors:

The stools provide a seat for the more aged visitors to rest on whilst their children or grand children run around other parts of the exhibition floor, at which point they start using the terminals which they have found in front of them almost by chance.

Some older visitors take up the opportunity to find out more about the theme behind the show they have or are about to see and spend extended periods of time reading and looking at many of the sites.

Often people who walk past and realise that the terminals offer an Internet connection people will attempt to use the terminal to access whatever they want. They appear to show very little interest in the information that they have been set up to provide, wanting more to check their e-mail, the football scores or the weather forecast. Not the most productive use of the terminals for a science centre. During many of my conversations with visitors I had complaint after complaint about there being no keyboards. They were getting very worked up over this because it meant that they couldn’t check their e-mail. People seemed to take it that it was their right to use the terminals to do whatever they wanted to and we should provide for that.

Parents:

Often adults accompanying children appear to try and take over the terminal useage from their children, trying to point out the educational information that they provide. Many of the adults would see that the child was not interested and begin reading the text sections out load to their children. Often the children would fight away from their parent’s grip and run away, often leaving the interested adult reading on a little time before heading off in pursuit.

Children without an adult:

Children are drawn to the terminals due to the bright colourful images and the prospect of playing on the computers. Many attempt to find games on other sites, or if looking through the site provided many only spend a very short amount of time looking at the images (not reading the text) before they run off to the next exhibit.

They were excited about using the computers and were skilled in there use. The terminals were enjoyed until an adult came along, at which point the fun aspect continued until the adult tried to take over, when the child would become bored and move on.

School group uses:

It was observed that the computer terminals were not being used by the majority of the school groups visiting. On many occasions the teachers were discouraging their children from using the area. On conversation with some of the teachers it turned out that many of them were attempting to guide their children to exhibits which were relevant to the theme of their visit, none of which were astronomy or space based at the time.

Other more general behaviours were met crossing all of the age groups:

i.) Many people would simple look at the terminals as they walked around the HuB, either looking at the terminals and the home page image or would simple watch the people who were currently using the terminals from a slight distance before moving on. When approached many of these people said that they thought the area was a good idea but that they either weren’t interested in astronomy or didn’t like using computers themselves.

ii.) Then next group includes many of the children who used the terminals. They sat down and appeared to ‘play’ with the terminals rather than take a real interest in the material.

iii.) In some cases a person, both child and adult, would use the area before running off and returning with a group of people wanting to shoe them the terminals. Often they were exited by what they had seen and wanted to share it.

iv.) Finally we have the group who actually sat and made their way through a site actually making an effort to read and understand the information that was presented to them. This group mainly consisted of lone users who sat quietly for up to about 10 minutes before leaving the area.

Staff comments

As part of their job the floor staff spend large amounts of time helping visitors use the HuB area, so that they would have picked up on many of the recurring feeling about the area.

The main comments made were:

When using their computers at home people mostly will use a mouse to navigate around their computer but when they sit down at the terminals they are suddenly faced by the roller ball and push button controls. This appears to cause quite a lot of difficulty in using the system. They seem to try and use the controls with just one hand instead of using both.

Also quite common is that the arrow on the screen can often disappear of the side of the moniters. This appears to be due to the fitting of the housing around the screen. So if someone has tried just spinning the roller ball for fun as they walk past when the next person come to use the system them can be easily put of by not being able to find the control pointer. They think that something is wrong and move on.

The systems can quite often be a pain for the staff due to the fact of how often they have to be reset once someone has clicked on a link which removes the browsers controls, taking the ‘Back’ button with them. Once this has happen the staff must turn the power off and then back on again to re-set the system to the home page.

The staff all seemed to thing that the terminals were on the whole a good addition to the floor. But I believe this may be due to the fact that they sit and work out how to get on to the search engine pages and then from there find links to what every they want. This means that during the quiet periods of the day they can entertain themselves.


Discussion

The information of most use turned out to be the observational data, as it appeared to give the best picture of how people were using the area.

The conversations which I had with people most often turned into them complaining about not having a keyboard or simply saying what a good idea the area was. Many people were also vocal about the problems of the links taking them to dead ends from which they needed help to get out from. Even with my ID badge on many people were nervous to talk to me in much depth about their thoughts on the area.

Web Concerns and Keyboards

Over 20% of children using the Internet say they have found something that has “upset” or “embarrassed” them according to the NOP research group. Of these, two-fifths said that they found something “rude”, a quarter something that they thought would “get them into trouble” and one in seven something which “frightens them.” (Supanet Internet user’s guide, 2000)

Most perfectly normal children will search using a swear word the first chance that they get and if they are at all curious about sex, it won’t be long before they come across something which we will not want them viewing within Techniquest, if at all. There are two methods of dealing with the problem. Trying to prepare them for what they may find or simply shield them from it. As the technology of the Internet and the Web browsers increase we can now attempt to censor sites which include sex or violence.

By disabling the normal browser functions Techniquest has attempted to limit users to looking at only pages to which direct links are provided. A lot of time and effort is put into trying to ensure that all the sites used will not give access to unwanted materials. Also by allowing people to use the Web in general people may hog the machines looking up non scientific information while there are people waiting to use the terminals to look at the ‘Lights in the sky’ material for which they are provided.

The main attempt to curb the searching for unwanted materials is the removal of any keyboards, thereby not allowing directly typed searches. Keyboards were originally fitted to the terminals but they were removed after an incident when Colin (TQ Director) walked by one evening and found 3 of the 4 terminal the displays showing inappropriate materials.

Another reason for not having Keyboards is the large amount of wear and tear that they had experienced previously. It was found that they had to be repaired on a regular basis, most keyboards are not designed with the battering taken by exhibits on the floor in mind. Keyboards are provided under supervised use when the terminals are moved into the Lab as part of a workshop.

Even without Keyboards there are still risks of people accessing material we do not want them to within the centre. Firstly nearly all Web sites have links to other pages. If people follow these it is only a matter of time before they find their way on to a search engine site from which they can just point and click to find what they want. Secondly some links produce pop up windows, some of which can contain all the normal browser functions allowing access to the history files. This allowed people to bring up any pages that may have been called up by members of staff on there own computer in the offices of the centre, some of which was material which was not wanted on the floor by some of the parents (such as the ‘South Park’ web site).

Computer speed and Advertisements

The speed of the load time for the pages seemed to depend on which site they were looking at. Some are nearly instant while others can take up to a couple minutes.

The effect such a long wait had on the user was either that they would just stand up and leave, or they would use the ‘Back’ or ‘Home’ button to continue on with another topic.

Just as advertising on the web has been accused of slowing the system down it too causes problems on the terminals. Clicking on any of the adverts on the science pages will often take the users to a screen from which they have no way back without the aid of a keyboard or the full browser functions, both of which are not present on the terminals. This can often lead to a dead end and requiring the user to ask a member of staff for help, which is something that many people will not do. Instead they simply will get up and walk away. The adverts can cause real problems especially when so many are disguised as useful links. When a user ends up at one of these links they do not what to ask for help. Maybe they fear that they have done something which they shouldn’t of done.

Instructions

The instructions on the terminals for their use are very straight forward. They are divided into 4 bullet points, which state how to use the controls. What was missing was an explanation for the terminal’s presence. Labeling, clearly setting out for what purpose the terminals and the linked pages are for would be useful for focusing the attention of the user to what they can look up.

“exhibit designers struggle with the issues of making systems simple enough for brief encounters with first time users” (Semper & Woolsey 1989)

In this case the system seems to have been made in such a way that everyone observed could use, if not in the most coordinated of ways. Everyone appeared to understand what they had to do even without reading the labels.

The room itself

One interesting question which I find I have to consider is just how the set up and design of the HuB itself should affect the information content of the web pages used as links.

Now with its open and light feel, connecting almost seamlessly with the exhibition floor I feel that the area has a feel of light and quick information. The area does not lend itself well to an extended period of time sitting using the terminal, with all the distractions of the surrounding noise and exhibits I believe it is not the place for lots of information to be given in the form of long essay type pages. By far the majority of the users spend only a small amount of time on the terminals, during my observations the longest was about 10 minutes (that is of those who were looking at the intended information pages). So the terminals seem to lend themselves to short, sharp and punchy information bites, but still with the option for links to deeper investigation if required, but not as the norm.

Conclusions

There is much theory concerning the way in which visitors move around exhibits, dictating how they should be constructed and positioned to provide the most stimulating experience. I believe that the same should be true for interactive multimedia exhibits such as the Internet terminals. We need to look at ways of creating a navigational tool which allows a more constructed way for the visitor to ‘surf the web’ while still feeling the freedom to look up information, but only on subjects which we feel are relevant. The main sites need to be very carefully chosen so that they are as self contained as possible.

I still find it interesting that a user may not even realise that they are on the web. I think a decision needs to be made over whether the terminals are being set up as Internet terminals or simply as back up information terminals. If it is the latter do they actually need to be connected to the web at all, cannot the sites be downloaded onto the computer network. Working in a system which closes off the adverts and links to other sites, leaving just the information that we have chosen.

As an interface the terminals need very little in the way of previous experience on the part of the user, which makes the terminals open to all.

As to is the experience educational? My answer is that is can be just in the same way as all of the other exhibits of Techniquest, only if used in a constructive manner by the user. Does the visitor experience of the terminals need to be totally educational. I believe that a science centre like Techniquest should be about not just teaching but about inspiring. If the terminals have inspired someone to go and look for more information about a topic on the web at home then is that not the first step to learning. For the child, uses the terminals definitely appear to be a fun experience.

The HuB will be an area of Techniquest which I’m sure will continue to evolve with time. I believe the main points to be looked at are:

  • Ways of cutting out the links that lead to dead ends on the terminal browser.
  • A way of focusing more the attention of the user onto the intended use of the system.
  • And a study carried out of which type of site (essay style, practical or theory based, images based or video/audio pages etc…) that are most used. Currently the sites with the astronomy pictures were clearly favoured over the text based.

Bibliography

CSUP REPORT. 1992. Teaching and Learning in an Expanding Higher Education System Committee of Scottish University Principals Working Party.

Department for Education and Employment. 1997. Excellence in schools.[WWW]
<URL: http://www.dfee.gov.uk/wpaper/mindx.htm>

Dudley Metropolitan Borough, Education service. 2000.
“The Dudley Grid for Learning”. [WWW]
<URL: http://www.dudley.gov.uk/council/educate/dgfl/briefcover.htm>

Guillermo Pedroni. 1996.”The Importance of The World Wide Web in Education K-12.” [WWW]
<URL: http://www.geocites.com/Athens/5461/paper_1.html>

Roberts E. 1999. An Evaluation of the Cyber Library at Techniquest.
MSc Dissertation, University of Glamorgan

Semper R. & Woolsey K.H. 1989. “The Intersection of Public Space and Computing”

Introduction: Multimedia in Public space, Technical report.

Supanet. 2000. Supanet Internet user’s guide. P45

The 6 web sites that were used on the terminal:

2001. Heavens Above. [WWW] <URL: www.heavens-above.com >

2001. Science Net. [WWW] <URL: www.sciencenet.org.uk >

2001. Imagine the universe. [WWW] <URL: http://image.gsfc.nasa.gov >

2001. Why is the sky blue? [WWW] <URL: www.sciencemadesimple.com/sky_blue.html >

2001. Astronomy Picture of the Day. [WWW] <URL: http://antwrp.gsfc.nasa.gov/apod/archivepix.htm >

2001. Steve Albers’ Weather Photographs. [WWW] <URL: http://laps.fsl.noaa.gov/frd-bin/albers.wxphotos.cgi >

Appendices

The World Wide Web

Today the Internet is a network of more than 120 million users worldwide. Using telephone lines, cables or satellites to transmit information from computer to computer, the Internet provides almost instantaneous communication. It can be used to send messages, share information, learn, buy and sell goods, play games and generally just have fun.
The Internet began as an experiment in the 1960’s by the United States Department of Defence. Gradually the Internet moved towards being a communications network for scientists. In recent years businesses began to use the Internet as it worked its way into family households.
Many people think of the Internet and the World Wide Web as being the same thing but the Web is only one aspect to the Internet. The World Wide Web is a network of computerized information that can include text, photographs, video clips, and sound, which can be accessed through the Internet. It has become a worldwide phenomenon, leading to an explosion of information. The Web allows you to navigate by pointing and clicking on words or pictures to get to where you want to.

“This is what makes the Web easy for anyone to browse, roam, and make contributions, as well as transferring (placing copies on their computers) of multimedia-including sound and video. Most of the educational institutions, government agencies, commercial companies, and other organisations around the world are moving into the WWW.” (Flake, 1995/ in Guillermo E. Pedroni)

The Internet also consists of Email, FTP (File Transfer Protocol), Gophers, Internet Relay Chat, Telnet, and Usenet (or Newsgroups).