An Evaluation of Hertz’ NeverLost

A five-pronged approach to system evaluation

by

Erik Olsen

&

Tim Locklear

Presented to Dr. Woodrow Barfield

For

Human Factors Design I (ISE 5605)

16 December 1998

Contents

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Evaluation preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Evaluation sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

System overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . 3

Norman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Function/task analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Link analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Evaluation and operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Limitations and assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Usability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

User comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Telephone interview input. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Error analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Overall summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Lessons learned from multiple analyses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

NeverLost

Introduction

An in-depth review of the NeverLost route guidance system was performed using a series of methodologies common to human factors research and development. For this project, five methods were used including a “Don Norman” analysis, function/task analysis, link analysis, usability evaluation, and an (informal) error analysis. After an overview of the system, results from each of these analyses are summarized. Following this, discussion about the five-method approach is included. Finally, system improvements are suggested and future research is recommended. Let us first begin with an overview of NeverLost, background of the project, and an overview of the system we evaluated.

Overview

Computer technology has recently made its way into the automobile that provides turn-by-turn route guidance to drivers. The NeverLost system knows exactly where the vehicle is at all times and can calculate the most efficient route to reach any destination that is entered into the system. After a route has been calculated, the system guides the driver by visual displays (maps and arrows) and auditory output (voice messages and beep tones).

Background

Before deciding to evaluate NeverLost, a review of existing systems was conducted. A plethora of systems exist such as Oldsmobile’s Guidestar, the Acura Navigation System, BMW’s On Board Navigation System, GM’s OnStar System, the Lincoln Continental RESCU, Magellan’s PathMaster vehicle route guidance & driver information system, as well as dash or free-mounted systems offered by Alpine Electronics, Philips Magnavox, Delphi Delco, Sony, and Clarion (the AutoPC) (Cerullo, 1998). During this review, it dawned upon us that car rental agencies offer cars with such systems. Soon thereafter, both Avis and Hertz were contacted. Both of these companies offer cars with Magellan’s PathMaster system (which was sold by Rockwell International to Magellan in 1997). It was discovered that Avis has about 500 cars in their fleet with the system, while Hertz has 8,000 cars nationwide. For this reason, we selected Hertz and began correspondence with both Hertz and Magellan about our project.

The Hertz system (known as NeverLost) is available exclusively in Ford Automobiles. The city manager at the Ronald Reagan Washington National Airport (Washington, D.C.) was contacted, whom arranged for a visit and discounted rental of a Ford Contour with the NeverLost system.

Method

The methodology of this analysis is discussed throughout this report. As an overview, the evaluation was prepared by acquiring information about the system, evaluation sessions were conducted, and a brief overview of the system is presented. These items are discussed below and in-depth within each section of the analysis.

Evaluation Preparation

Before the trip was made, information about system operation was acquired. A customer service representative from Magellan, who used the system in her own car, was interviewed on the phone and provided a system review and basic operating instructions. Following this, the Internet was searched and various URLs were located (listed in the references section), describing the features of NeverLost as well as illustrating the screens. Finally, the operating brochure (Hertz, 1998), was reviewed (Appendix A).

Evaluation Sessions

Reviews over a 2-day period took place in Washington, D.C. and the Fairfax, VA area. Videotape was used to record a 20-minute demonstration of the system by a Hertz staff member, a 1-hour session of driving by an investigator, and two half-hour observation sessions of a driver who was unfamiliar with the system. The half-hour observations took place 1) in the evening (dark) on residential Fairfax roads, and 2) during the daytime en route to the National Airport (about 40 miles).

The 10 minute orientation included a brief review of the system components, screens, and included an example of how to program a route. The experimenter simply asked the employee to “show me how it works,” and video taped the screen and button pushes as the session continued

The 1-hour session consisted of the experimenter using the system, including his initial interactions in the parking lot and various interactions while driving throughout Washington D.C., video recorded for evaluation at a later time. Verbal protocol was used throughout the session so that the experimenter would have an audio account of what was going on, and so that impressions could be evaluated without the need of taking notes while driving. This also allowed the experimenter the ability to focus on “driving naturally” with less distractions than would occur had videotaping not been available.

The 2 half-hour observation sessions allowed the experimenter to video tape a person’s initial interactions with the system in a night time environment (typical of some rental situations) and a session during the daytime. Each of these sessions included route-entering interactions (e.g., in a parking lot, before driving) and on-road sessions (residential and freeway). For each session, minimal instructions were given to the participant, in order to see the natural progression of learning with the system. The instructions that were given included having the participant use a verbal protocol while driving (for the initial session only) and where to go (“OK, we are going to 942 April Street, in Fairfax”).

System Overview

The NeverLost system consists of four major components (Navigation Technologies, 1998). A Global Positioning System (GPS) Antenna is mounted on the vehicle’s trunk lid or roof. A computer with an electric direction sensor to track vehicle turns and logic circuits to process vehicle and cartridge data is mounted in the trunk area or beneath the seat. A Region Database Cartridge contains operating software and information (mapping) data. Finally, the Control/Display Unit is stalk-mounted in the driver/passenger compartment within easy reach and viewing.

Only the control/display unit is the component discussed in this analysis. It consists of a small 4"x4"color LCD display (234x479 pixels) as illustrated in Figure 1, and includes input controls, located below the display, as illustrated in Figure 2. The unit displays visual information such as maps and turn-by-turn directions, and also gives auditory output to the driver in the form of voice commands and beep tones. Figure 3 shows an illustration of a user with the system.

Analysis

As previously mentioned, five methods were used to analyze this system.

Norman Analysis

We began with an analysis based upon The Design of Everyday Things by Don Norman. This addresses many issues concerning the proper design of systems for understandability and usability. These ideas are easily transferable to the use of a computer-based system such as NeverLost. Specifically, Norman's ideas are well suited as tools for an initial "informal" analysis of the Neverlost navigation system.

For ease of understanding, Table 1 presents Norman’s Design Guidelines such as affordances, constraints, conceptual model, making visible what is invisible, mapping, feedback, and design for error, that were relevant in evaluating the NeverLost system.

Overall, this is a very good design. The designers quite obviously tried to present a system image that is similar to the user's image of a "traditional" paper map. With a display that is less "busy" with impertinent information, the system presents only the information that is most relevant to the task at hand - arriving at the desired destination. With minimal knowledge (in the head) of computers or even of an ATM, how to actually select the desired program choices is quite intuitive. The choices are simple, including entering street address, intersection, points of interest, and freeway/entrance/exit which makes what to choose very clear (only the choice of "guidance history'" is perhaps not immediately discernable). After the choices have been programmed and the trip is begun, the role of the NeverLost becomes that of a navigator. What to do (i.e., what directions to take) is usually very clear. As a preview, the display screen presents the calculated route by a highlighted route, and then turn-by-turn guidance information is presented in a redundant manner: 1) by the appearance of large turn arrows, and 2) by displaying the name and distance to the next turn shown. In addition, this guidance information is further reinforced by auditory voice commands and double beeps, and finally, a shrinking distance bar. Both the auditory output and the visually displayed distance bar alert the driver that a turn is soon coming up (usually 0.2 miles).

A preview (an advanced feature) of turns to be made, or a review of turns already made, is available, though not readily apparent. The only other major criticism of the system would be a need for the voice commands to be louder at times, the need for an explanation of the beeps to be given on one of the screens (in addition to the brochure) or even available as audio instructions, and a less confusing way to cancel or reset the system (as opposed to entering “cancel”). In addition, a “repeat last command” might be a valuable feature.

Norman Design Guideline (Table 1)	Neverlost Function	Function Analysis	Conclusions/Suggestions
Affordances	Buttons	Afford pushing	Good logical and physical cues; and good mappings
	Arrows (on screen)	Scrolling	" "
	Menus	Afford selecting	Based on computer (or ATM) experience (can be good or bad)
	Screens	Afford visuals: text for reading, graphics for icons, maps for directing Also affords watching	Monitor like TV (standardization) New users feel compelled to watch
	Sounds - beeps	Afford listening Affords re-enforcement	Immediate attention alerts to change in current dynamic Reminder of visual info
	Sounds - Voice	Affords commands (directions) and listening instead of looking at screen Also affords responding	Like having your own navigator (sort of!) so meets expectations. Allows more concentration on task of driving; but want to talk back to it. Audio information easier for drivers to attend to
Constraints - Artificial	Volume	Constrained by speaker and hardware	Could be louder, especially when there is more road noise
	Scale	Change in visual size	Allows more specific information
	Scrolling	Only 4 or 5 choices available on screen at time	Becomes tedious, but alphabetization facilitates
	Fixed Unit	Only rotatable on axis	Necessitates change of focus from road (to unit at right)

Norman Design Guideline (Table 1)	Neverlost Function	Function Analysis	Conclusions/Suggestions
Conceptual Model	Display	North up Heading up	Good - based on common reference frame of paper road map Allows change of perspective when needed/desired
	Operation	Menus and scrolling	Good labels and mapping, so required knowledge in head (of computers or ATMs) is minimal. Bad - knowledge in head needed to interpret beeps and access preview/review of route
	Use - Gives direction commands	Following direction commands Interpreting given commands	Good feature: knowledge in head of maps, etc. not even needed - just follow commands to "turn right," etc.
Make Visible What is Invisible
Structure	Menu options	Structure basically shallow. Deep, narrow structure of remaining choices - must be scrolled	Should make choices (options) more visible - maybe initial (default) screen with all of options, icons and labels on same screen
Gulf of Execution	Input information	Must enter something to proceed	Pretty well "forced" design for error
	Auditory commands	Difficult to hear at times w/ cabin and highway noise, etc.	Need control for repeating and more volume
	Cancel or reset	How to accomplish not readily apparent	Decrease in gulf of execution by clearer prompts or "master" reset switch
Gulf of Evaluation	Directions given (by NeverLost)	May be reviewed	Not apparent (not in brochure)
	Information(already) input	No apparent way to review	May not be necessary in light of available review of directions given
Norman Design Guideline (Table 1)	Neverlost Function	Function Analysis	Conclusions/Suggestions
	Beeps	Interpret meaning of beeps	Gulf of evaluation could be decreased by on screen tutorial/review for meaning of various beeps
Knowledge in Head vs. Knowledge in world	Beeps, preview/review of route	Only explanation of beeps is in brochure. No mention of screen preview/review route	Knowledge in head needed of brochure explanation. Main screen should "link” to potentially needed explanations and reviews
Mapping	Up/down buttons	Used to scroll through menu choices	Good- naturally mapped
	Right/left buttons	Use to scroll through entire sub- categories (e.g., entire letter of alphabet)	Not as good - not naturally mapped to choices. But (world) knowledge given on screen
	Map displayed on screen	Choice of "north up" or "heading up"	"North up" artificial mapping, but paradoxically more natural (very common, i.e., paper map reference frame) "heading up" more natural, good design to offer as alternative
Feedback
Visual	Menu Displays	Hertz opening screen, warning screen, screen showing choices, 5 input screens, plus specific direction screens	Text and graphics large, contrast generally good: makes for easy reading
	Input Screens	Scrolling up and down and moving between categories; choices highlighted; verified	Scrolling up and down intuitive, though moving between categories is not. Highlighted choice is obvious, (then confirmed) – Good design, though how to review choices made not apparent.
Norman Design Guideline (Table 1)	Neverlost Function	Function Analysis	Conclusions/Suggestions
	Readout - textual	“Calculating a route to…” destination shown in text	Good feedback of choice(s) made & overall sum of choices made
	Readout – graphics	Turn by turn graphics given: large icons and symbols	Easy to read and interpret
	Readout - maps	Includes vehicle icon, compass heading, miles to go, suggested route, map scale, street names, and GPS status	Has same elements and colors of paper map, so easy to read and interpret; but minimal information (only pertaining to specific destination) makes even easier to attend to
	Readout - other	System setup options screen: manipulation of options of map scale, orientation, voice volume, and system data	Well arranged and easy to read; but difficult to review specific choices
Auditory	Voice commands	“Proceed to highlighted route,” “right turn ahead,” etc.	Could be louder. Not always enough lead time
Design for Error	Beeps	3 beeps – alerts driver of off-route situation 2 beeps – alerts driver that a turn/action is coming up (with and without voice command)	Not clear at first. May startle some drivers Good reminder and design for error as an alert. Unique – immediately processed, and does not distract from visual tasks
Forcing functions	Programming	Must enter something before proceeding to next screen	Good design that programming is not yet finished
	Route selection	Verification of route to take	Good design for error made during selection process; can become annoying to expert user
Norman Design Guideline (Table 1)	Neverlost Function	Function Analysis	Conclusions/Suggestions
	Cancel	"Press Cancel to Quit" or "Do You Want to Cancel?" on each screen	After chosen, steps to be followed can be confusing
Other (feedback)	Navigation (giving directions to be followed)	Textual information on screen gives name of road to turn onto and how far. Large visual graphic (arrow for turn, etc) also given, plus audio command (e.g., "turn right ahead"). Then alerted within 100-200 feet with 2 beeps and a shrinking distance bar.	Good design for preventing driving error (i.e., missed turns). Visual information always complemented with audio info for easier attendance, plus several measures of redundancy.
Standardization	Menu system (for entering information)	Emulates ATMs	Presuming ATM experience more common to all drivers than computer experience. Design based on research.
	Possible future features	Keep simple for simple task - avoid creeping featurism.	Inevitable.

Table 1. Norman’s Design Guidelines

Function/Task Analysis

This section provides an in-depth view of this system by means of both a function analysis and a task analysis, including information flow analysis, hierarchical goal, sub-goal, and task and sub-task analysis. Three users were observed, in tasks ranging from a short 20 minute programming demonstration to programming and navigation tasks over 1 hour in length. From this analysis an in-depth understanding of the system emerged, and some recommendations for ease of use and improvements to the system are made. Let us first begin with a definition of functional analysis and an in-depth review of the system components.

Function analysis defined: for the purposes of this paper, a function analysis is associated with understanding aspects of systems with an analysis of relationships between components, or sub-components, that make up the overall system. A function analysis flows in a top-down hierarchical manner with emphasis on the information progression throughout the systems. Figure 4 illustrates a flow-analysis diagram of the NeverLost-driver system.

System Sub-components of the unit include the display screen, which consists of screens, menus, maps, turn-by-turn components, and other advisory displays, as well as an auditory output in the form of spoken voice prompts and beeps. Screens exist that independently show menus, maps, turn-by-turn-direction, and error messages.