Chapter I

Introduction

This project is a usability evaluation for inserting and modifying WordArt. WordArt is a feature included in Microsoft Word 97 and 2000. The following introductory sections describe the problem to be investigated, goals to be achieved, and participant selection criteria. The introduction also provides the plan and approach of the project along with a timeline of milestones.

Problem Statement and Goal

Microcomputers are being produced at a rate that exceeds more than one million per day (Birnbaum, 1985, November). In fact, there are more computers than there are people in the United States. Although computers are part of the daily lives of many, their user interface is still a major concern, especially for the software industry (Beaudouin-Lafon, 1988, November 28-30).

One of the main concerns of developing a high quality user interface is the generality and adaptability of user interfaces to a wide range of users with different tastes. Thus, building an interface that can be easily adapted to a user, no matter what the knowledge base or experience of that user, can be quite challenging (Beaudouin-Lafon, 1988, November 28-30).

Microsoft has probably tackled more personal computer (PC) software projects than any other company in the industry (Cusumano, 1997, June). Their main philosophy in developing the best user interface software is not to adopt too many of the structured software-engineering practices. Microsoft generally follows a process that iterates among design, building components, and testing. All of which contain interaction with customers during development (Cusumano, 1997, June).

Microsoft has many great products with a wide range of users. For example, many users use Microsoft Word 97. This is a word processing software package. The purpose of the software is to improve the performance of typing letters, papers, presentations, etc. Software programs, such as this one, are adapted largely to automate those tasks that have always been done (Birnbaum, 1985, November).

This project report was written to detail a WordArt usability evaluation. Three participants were selected for this evaluation. The usability evaluation assessed the potential for errors and difficulties involved in using the WordArt feature. The goals of the usability evaluation were the following:

¨ Participants will be able to insert and modify the WordArt feature in five minutes or less with no assistance.

¨ Participants will be able to begin using the WordArt feature without the use of documentation.

¨ Participants will be able to rotate the WordArt with less than one false attempt.

¨ Participants will be able to change the color characteristics of the WordArt with less than one false attempt.

¨ Participants will be able to change the spacing characteristics of the WordArt with two or less false attempts.

¨ Participants will find that the pictures used as icons are recognizable and do facilitate system use and understanding.

Relevance

This project report is beneficial to the study of Human Computer Interaction and to the future of users who use Microsoft Word 97 for word processing and presentations. The usability evaluation of inserting and modifying WordArt was chosen because it represents one of the best graphic features offered in Microsoft Word 97.

Microsoft Word 97 comes with many features – features that even experienced users are unaware of (Campbell, 1999, September 23). However, these hidden features save serious time and energy if one knew about them. One of the features that users do not know about but should is WordArt (Campbell, 1999, September 23). WordArt is a graphics tool to let users twist and turn pieces of text into colorful and lively graphics. It can be utilized to highlight any area of a document, worksheet, or presentation (Campbell, 1999, September 23).

Participant Selection Criteria

For this evaluation, one must determine whom the intended user for the product (Mayhew, 1999). The evaluation is only valid if the participants assessed are typical end users of the product, or as close to a selected set of characteristics as possible (Pieratti, 1995, July).

The following list shows the essential characteristics of the end users likely to use the WordArt feature. These characteristics are the basis for participant selection for the usability evaluation.

¨ Job Function – manager, technical (i.e. engineering, software development, product development, manufacturing), administration

¨ Computer literacy – medium to high

¨ Microsoft Word experience – more than 12 months

Plan and Approach

This project report is a usability evaluation. It is formatted in five chapters with the first chapter covering the project's problem statement, goal, relevance, participant selection criteria, plan and approach, and milestones. The second chapter provides a detail review of the literature relevant to the usability evaluation process. The third chapter describes the method (test plan) employed in completing the WordArt usability evaluation.

The fourth chapter details the results of the usability evaluation. The user profile, error occurrence, time needed, and any comments/observations are presented. These results are discussed as it relates to the goals of the usability evaluation. Recommendations and the conclusion are covered in chapter five.

Milestones

The following is a summary of the milestones for the usability evaluation along with significant dates. The first milestone was deciding on what usability evaluation to perform. The idea of inserting and modifying WordArt was decided on Wednesday, December 15, 1999. Gathering research materials and writing chapter one was completed on Monday, December 20, 1999. Chapters two and three were completed on December 23, 1999. The usability evaluation was conducted on Friday, December 24, 1999. Chapter four was completed on Thursday, December 30, 1999. Chapter five was completed on January 2, 2000. The usability project report was proofread and submitted on Monday, January 2, 2000.

Summary

In summary, the introduction given above described the problem to be investigated, goals to be achieved, and participant selection criteria. Also included were the plan and approach for the project along with a timeline of milestones. In the next chapter, this report provides a thorough review of literature relevant to the usability evaluation process.

Chapter II

Review of the Literature

The review of literature provides a brief overview of the resources used to complete this usability evaluation project. Included in this review is how these resources help contribute to this project. Subject headings organize the review of literature.

Usability Testing (Evaluating) Briefly Defined

Usability testing (evaluating) is the process that uses participants, who are representative of the target population to evaluate the degree to which a product meets specific usability criteria (Rubin, 1994). There are six basic elements to usability testing. They are the following:

¨ Development of test goals

¨ Use of representative participants

¨ Representation of work environment

¨ Observation of participants

¨ Collection of quantitative and qualitative performance and preference measures

¨ Recommendation of improvements (Rubin, 1994)

Task Analysis (Contextual)

The type of usability evaluation utilized for this project is task analysis (contextual). The purpose of this test is to understand how users currently think about, talk about, and do their work in their actual work environment (Mayhew, 1999). The method of performing a function within a product is broken down into a series of tasks. Based on these tasks, it can be used to predict how difficult or easy the tasks will be to perform and how much effort is required (Jordan, 1998).

Quantitative Data

Quantitative data gives a clear indicator by which the effect of particular design decisions can be judged (Jordan, 1998). Typical quantitative data consists of the following: time users take to complete the task, ratio between successful interactions and errors, time spent recovering from errors, number of user errors, number of commands never used, etc. (Nielson, 1993). The types of quantitative data collected for the WordArt usability evaluation is the time it takes the user to complete the task and the number of errors. Only two of these were selected because there is only one investigator. The investigator can only concentrate on a few quantitative and qualitative pieces of data at one time.

Qualitative Data

Qualitative data is essential to a usability evaluation. It provides rich descriptive data, which can be used to diagnose usability faults and prescribe design solutions (Jordan, 1998). It can also be used as an approximation to quantitative data in order to make the “first pass” at addressing an issue (Jordan, 1998). There are two methods employed for the WordArt usability evaluation. They are the think aloud method and the post-test questionnaire.

Think Aloud Method.

The think aloud technique is intended to capture what the participants are thinking while working (Rubin, 1994). The participant is free to express his/her confusion, frustration, or even delight. It is used to expose the participant’s preconceptions and expectations of how the product works (Rubin, 1994). This method is selected for the WordArt usability evaluation to capture the implicit information when the participant is performing the tasks.

Post-test Questionnaire.

The purpose of a post-test questionnaire is to gather information from the participant in order to clarify and deepen the understanding of the product’s strength and weakness (Rubin, 1994). Information from this questionnaire is based on the opinion and feelings of the participant. In Appendix A, a copy of the post-test questionnaire for the WordArt usability evaluation can be seen.

Chapter III

Methodology

Method (Test Plan)

The main purpose of this test is to see if the text descriptions and icon displays are easily learned for inserting and modifying WordArt. The goals of the usability evaluation are detailed in Chapter one under problem statement and goals.

After determining the type of participant selection criteria, three family members were chosen. First, a debriefing session was held. Included as part of this debriefing session were the following:

¨ Emphasized to the participants that WordArt was being tested, not them.

¨ They were allowed to stop at any time.

¨ Explained what data was collected (i.e. time, comments, and errors).

¨ Explained the steps of the evaluation process.

¨ Explained that the results were only for DISS 720 Human-Computer Interaction class.

¨ Any questions the participants had were answered.

¨ The participant helped fill out the user profile.

Upon completing the debriefing session, the participant was given the “Overview of the Tasks” page. The investigator walked through this information with the participant. When the participant said he/she was ready to start, task one was given to him/her. Each task had its own separate page. While the participant was performing the task, he/she commented about the things he/she did/did not like. No help was given to the participant on how to complete each task. The investigator wrote these comments down. In addition, the investigator timed and watched for the number of errors that the participant had in order to complete the task.

After all ten tasks were completed; the participant was asked to fill out a post-test questionnaire. Upon finishing the post-test questionnaire, another debriefing session was held. During this debriefing session, any additional questions were answered as well as thanking the participant.

Consent Letter

The purpose of a consent letter is to have approval from the participants that they understand the conditions and instructions of the usability evaluation (Nielson, 1993). A consent letter was not used for this usability evaluation. A consent letter can increase the participant’s anxiety level by making the test seem more foreboding than it really is (Nielson, 1993). Therefore, since the investigator is using these results for a class assignment only, a formal consent letter was not given to the participants.

Role of the Investigator

The role of the investigator was to sit in the room with the participant while conducting the usability evaluation. The investigator was to initiate the tasks, record timings, errors, and observations. The investigator was not allowed to help the participant with any of the tasks. The only time the investigator was allowed to answer was when a question arose about the usability evaluation process.

Conducting the Pilot Test

It is important to have a pilot test. The pilot test is a way of “debugging” the test procedure and all supporting materials (Mayhew, 1999). A pilot test was conducted a few days before the actual evaluation date. The task lists and materials were updated to include a “results” section to each task as well as a graphic on each task procedure.

Task List

The task list was developed with each task being listed separately on its own page. This was done because Xerox Corporation used this type of presentation for their sample Xerox Management Model evaluation (Naughton, 1995, July). In addition, it was done so as not to overwhelm the participant.

In addition to the task being listed, a “results” section was provided next to the task. Including successful completion criteria (SCC) with the task description added precision to what was being measured and what results were expected (Rubin, 1994). Lastly, graphics were added to make each task page aesthetically pleasing. A copy of the task pages can be seen in Appendix B.

Test Environment/Equipment

An environment that is representative of an office with a computer, printer, desk, and chair was needed. Therefore, a home office area was selected. A burgundy executive style chair was used for sitting. An Epson Stylus Color 500 was sitting next to a NEC 17 inch monitor. The computer was a Hewlett Packard Pavilion 4530. The specifics of the computer were the following: 350 MHz, 28 MB RAM, 4 GB hard drive, 32x CD-ROM, 56k V.90 modem, keyboard, and mouse.

The temperature of this area was set for 70 degrees. Lighting was approximately that seen in an office environment. The desk was somewhat cluttered. There was a phone to the left and some papers. The mouse pad and mouse were on the right side of the desk.

Evaluation Measures

The following quantitative measures were used for the WordArt usability evaluation.

¨ Time to complete each task

¨ Number of errors that occurred during completion of each task

The following qualitative measures were used.

¨ Think Aloud – comments/observations

¨ Post-test Questionnaire

Chapter IV

Results

Narration of Results

Participant 1.

Participant 1 has approximately 10 years computer experience. She is the Director of Medical Records at Columbia/HCA (Hospital Corporation of America). Her word processing experience includes Publisher, WordPerfect, and Microsoft Word. Overall, she has 10 years of word processing experience.

Participant 1 did not run into major problems until task seven. The first six tasks were relatively easy for her. She did not have any errors, however, 80 percent of the time she took the longest to complete each task. She had read the instructions aloud, performed them, and then verified with the results that was listed on each task sheet.

When the toolbar appeared and she was asked to rotate her WordArt, she had a hard time figuring out where to click the rotate option. She had realized the pop-up tool bar was for modifying the WordArt since it was titled “WordArt”. However, she had continued to have problems with the icon displays as she was having problems trying to figure out which one of them would allow her to rotate her WordArt. After going over each icon and waiting for the text description to appear, she had clicked on the rotate option. Then, she had encountered another problem; she had no idea how to get her WordArt to rotate. Microsoft Word had provided text instructions in the lower left corner, but she did not see them. It took some mouse clicking and she finally had it. Throughout this whole process, she would refer back to the task instruction sheet.

When asked to change character spacing, she had still encountered recognition problems with the icons. Again, she had to wait for the text description to appear. By tasks nine and ten, she had familiarized herself with the pop-up toolbar with its associated icons and was able to complete those tasks with relative ease.

Summaries of all the results are included in this chapter after the narration of participant 3. There are five tables summarizing the following in relation to each of the participants. They are the following:

¨ Table 1.0. – User Profile

¨ Table 2.0 – Completion Times

¨ Table 3.0 – Error Occurrence

¨ Table 4.0 – Comments/Observations

¨ Table 5.0 – Questionnaire Results

Participant 2.

Participant 2 has 18 years of computer experience with nine years of word processing experience. He is the Assistant Vice President of Marketing for Unitrin Insurance. Microsoft Word is his main word processing software of choice.

Participant 2 started with not understanding the first task instruction. When asked to click on the Word icon “W”, there was some confusion. He had not ever associated the “W” icon with Microsoft Word before. He had to re-read the instruction task and finally understood. At that time, he had commented that the instruction of the task should have included MS in front of Word.

He had not encountered any major problems until task eight. Nevertheless, what was ironic, he had no problems with task seven – rotating his WordArt. However, he had not understood how he had rotated his WordArt. When asked how he knew to rotate his WordArt he did not make any reference to the little green circles that appeared on the four corners of his WordArt. He stated he just went to the edge of the WordArt as that seemed logical to him.

Task eight had given him some problems. He was not able to easily recognize the character spacing icon. He also had to wait for the text description to appear for each icon. Task nine was a misunderstanding of the task instruction. He had not selected a new fill color when performing the task the first time. However, he had easily recovered his error and had known where to click to change the fill color. Task 10 did give him some problems as he did not associate “change the size to 54” with font size. In the WordArt pop-up toolbar, it had only listed “size” with a down arrow to change the font size.

¨ Table 1.0. – User Profile

¨ Table 2.0 – Completion Times

¨ Table 3.0 – Error Occurrence

¨ Table 4.0 – Comments/Observations

¨ Table 5.0 – Questionnaire Results

Participant 3.

Participant 3 has 18 years of computer experience as well as word processing experience. She is the Webmaster/Records Manager for the City of Longmont located in Colorado. She is familiar with word processing programs such as Word, WordPerfect, and Pagemaker.

Here again, this participant had no major problems until task seven. She had found the rotate icon easily but what had given her the most problems was trying to rotate her WordArt. She had double clicked on her WordArt resulting in it being unselected. She again had to go through the steps on clicking on her WordArt and clicking on the rotate option. It had taken some time to figure out that she was supposed to click on the little green circles. However, since she had past experience with other graphical software, she was able to recover quickly. She had not noticed the text instructions in the lower left corner either.

Tasks eight and nine had given her some problems because she had not associated the graphic icon to what was being asked of her. Like the other participants, she had to go over each icon and wait for the text description to appear.

¨ Table 1.0. – User Profile

¨ Table 2.0 – Completion Times

¨ Table 3.0 – Error Occurrence

¨ Table 4.0 – Comments/Observations

¨ Table 5.0 – Questionnaire Results

User Profile

Table 1.0.

User Profile

Characteristic	Participant 1	Participant 2	Participant 3

Computer Experience	10 years	18 years	18 years
Education Level	2 years – ART – Accredited Record Technician	4 years – Business Administration	Specific computer classes - no formal college education
Age	47	48	37
Gender	Female	Male	Female
Job Function	Director of Medical Records	Assistant Vice President of Marketing	Webmaster/Records Manager
Company	Columbia/HCA (Hospital Corp. of America)	Unitrin Insurance	City of Longmont
Word Processing Experience	Publisher, Wordperfect, Word 97	Word	Word, Wordperfect, Pagemaker
Years of Word Processing Experience	10 years	9 years	18 years

Completion Times

Table 2.0.

Completion Times (in seconds)

Task #	Participant 1	Participant 2	Participant 3	Avg. Time to Complete Task
1	18.21	15.49	15.41	16.37
2	4.36	1.38	3.05	2.93
3	4.87	6.06	3.02	4.65
4	3.0	2.16	4.38	3.18
5	11.87	4.92	5.83	7.54
6	28.69	25.04	8.77	20.83
7	1.27.42	16.91	24.45	43.8
8	1.27.53	19.69	17.03	42.20
9	23.53	36.33	19.94	26.6
10	17.97	18.71	12.77	16.46
Total Time to Complete all tasks	5 min. 31 sec.	2 min. 44 sec.	2 min. 52 sec.	3 min. 8 sec.

Error Occurrence

Table 3.0.

Error Occurrence

Task #	Participant 1	Participant 2	Participant 3
1	0	0	0
2	0	0	0
3	0	1	0
4	0	0	0
5	0	0	0
6	0	0	0
7	6	0	2
8	2	3	2
9	0	1	4
10	0	1	0
Total Errors	8	6	8

Think Aloud Results

Table 4.0.

Comments/Observations

Task #	Participant 1	Participant 2	Participant 3
1	Started with the start menu even though the Word icon was on the desktop. Stated “was thinking of where it was located when she is at work.”	Had some trouble with the instructions on the task list – was looking for the “W”. Felt that the instruction should say click on MS Word.	Knew exactly what to click on.
2	No problems encountered – very familiar with the top menu layout of Word.	No problems encountered – very familiar with the top menu layout of Word.	No problems encountered – very familiar with the top menu layout of Word.
3	No problems encountered – very familiar with menu layout.	Went past the “picture” selection as per the task instruction sheet.	No problems encountered – very familiar with the top menu layout of Word.
4	No problems encountered.	No problems encountered.	No problems encountered.
5	Thought the word art selection style graphics was cool.	No problems encountered.	No problems encountered.
6	Instead of just typing over the highlighted “type text here”, she deleted first and then typed her name.	He typed over the highlighted “type text here”.	She typed over the highlighted “type text here”.

Table 4.0 (continued).

Comments/Observations (continued)

Task #	Participant 1	Participant 2	Participant 3
7	Had a hard time finding the rotate feature. Did not see the pop-up toolbar. Went to top menu to find rotate feature. Had to go over each icon and wait for text description to appear. Also did not know how to rotate her WordArt. Did not notice the instructions in the left corner.	No problems encountered but he did not notice the green circles at the edges of his WordArt. He just picked the edge of his name to rotate. Found the icon easy to understand.	She had problems rotating her WordArt. She did not know to click on the green circles. She did not notice the instructions in left corner either. Good error recovery – from past graphic software experience.
8	Originally clicked the “WordArt gallery” icon. Had to go through each icon again and wait for text description.	Had to wait for text descriptions to display – did not understand icon for character spacing.	Had to wait for text descriptions to display – did not understand icon for character spacing.
9	No problems encountered.	Found the fill color easily but did not change color as per task. He was asked to go over task instruction again. Then he changed the fill color.	No problems encountered.
10	No problems encountered. Found that this participant read the “result” section on every task.	Did not understand to change size – would rather it said font size.	No problems encountered.

Post-Test Questionnaire

Table 5.0.

Questionnaire Results

Question	Participant 1	Participant 2	Participant 3
1. Overall, I am satisfied with the ease of completing the tasks in this scenario.	Agree	Agree	Strongly Agree
2. Overall, I am satisfied with the amount of time it took to complete the tasks.	Agree	Agree	Strongly Agree
3. I can easily locate WordArt in Microsoft Word 97 again.	Strongly Agree	Agree	Strongly Agree
4. I was already aware that the WordArt feature existed in Microsoft Word 97.	Disagree	Disagree	Disagree
5. I will probably use the WordArt feature for future documents, presentations, etc.	Strongly Agree	Neither Agree/Disagree	Disagree
6. The pop-up toolbar with its associated icons and text messages are easy to understand.	Agree – once I figured how to use the rotate.	Agree	Agree
7. I found the icons were representative of the characteristics to modifying WordArt (i.e. AV with the arrow underneath it for character spacing).	Agree	Agree	Neither Agree/Disagree
8. Is the ordering of the pop-up toolbar options logical?	Neither Agree/Disagree	Neither Agree/Disagree	Neither Agree/Disagree
9. I am happy with WordArt under the Insert/Picture command.	Agree	Agree	Neither Agree/Disagree

Table 5.0. (continued)

Questionnaire Results (continued)

Question	Participant 1	Participant 2	Participant 3
10. I like the pop-up toolbar when I inserted WordArt.	Agree	Agree	Neither Agree/Disagree
11. I like how the program modifies the WordArt as I am performing the tasks. I am able to see the effect instantaneously.	Agree	Agree	Agree
12. I understood how to rotate the WordArt after I selected the rotate option.	Disagree	Agree	Disagree
13. Do you have any comments about how the usability evaluation was conducted – likes or dislikes?	The evaluation was well presented.	None	Should have combined the tasks together rather than one at a time.
14. Do you have any comments about how easy it was to insert and modify WordArt – likes or dislikes?	It was relatively easy.	None	No – I normally do not use this type of program to do any graphics.

Discussion of Results

Based on this usability evaluation, the graphical icons presented in the pop-up toolbar did not provide ease of recognition or learnability. All three participants had to wait for the text descriptions to appear. Therefore, the text descriptions provided the most benefit when the participant tried to modify the WordArt characteristics.

As for the other goals, the participants could insert and modify WordArt within five minutes. Participant 1 was just a tad over the five minutes by 31 seconds. Participants 2 and 3 were able to accomplish all the tasks in approximately half the time.

None of the participants decided to use the help feature or any related Word documentation. All of them just referred to the task sheet. Based on this usability evaluation, any user should be able to insert and modify WordArt without any documentation. However, there is still a problem since these three participants did not even know the WordArt feature existed.

The rotating option provided some cause for concern. There were many errors in trying to rotate the WordArt. Two of the participants did not even recognize the graphical icon for rotating but also did not have a clue as how to rotate the WordArt once the option was selected. All three participants did not even see the text instructions in the lower left corner, which told them what to do.

Even with the other two tasks of changing the fill color and character spacing, there were still usability problems. Participants were able to change the character spacing with two or less false attempts. However, this was because of the false attempts that occurred on the previous tasks. Overall, this usability evaluation concluded that the participants did not find the pictures used as icons recognizable or facilitate system use and understanding for inserting and modifying WordArt. The next chapter provides recommendations based on this usability evaluation.

Chapter V

Recommendations & Conclusion

Recommendations

First, there is the problem of not knowing that WordArt even exists within Microsoft Word. Since it is a feature that saves serious time and energy for the user, Microsoft Word may want show the options when the user selects the insert and picture command (i.e. clipart, autoshapes, file, WordArt, etc.). A pop-up window or help screen could appear to show and select the options with examples. Based on this usability evaluation, the participants did not understand what WordArt was in the first place let alone associate it with the insert and picture command.

Second, the graphical icons do not facilitate ease of recognition or understanding. Most users appear to select either the position or shape to associate an icon with a command, but not both. Which one a user selects depends on whether the icons’ shapes are easy or hard to learn (Moyes, 1994, April 24-28). It is apparent that the participants in this usability evaluation relied on position. All three of them waited for the text descriptions to appear and then memorized the icons’ position in the WordArt pop-up toolbar.

When using icons, the question “Does a given picture convey the same thousand words to all viewers,” should be answered (Petre, 1995, June). There are two alternatives to solving this icon issue. First, Microsoft could perform additional usability testing with non-experience computer users to see if the icon representation is logical. Second, Microsoft could detect that a user is having difficulty with the icons and allow the user to select the best graphical picture to represent that command. In essence, let the user tailor the program to himself/herself. This could be done by either providing a new selection of graphical icons or allowing the user to create their own.

Lastly, the text instructions provided in the lower left corner are not noticed. A help system is usually a last resort. Many users try to avoid rather than employ it (Randall & Pedersen, 1998, September 24-26). This type of behavior was evident in the WordArt usability evaluation. A solution to this might be to have an embedded help system. This is where the software is intelligent enough to know the user is having difficulty and suggest answers. “Because the help is not called, but rather just appears as part of the user’s activities, perceptually it is not a separate entity, and thus there is no evidence of the bodily attraction or repulsion that comes with interacting with other people or personifications” (Randall & Pedersen, 1998, September 24-26). This type of embedded help system would be more beneficial than text instruction in the lower left corner of Microsoft Word.

Conclusion

Based on this usability evaluation, it is apparent that we have a ways to go in making the software interface intuitive and easy to learn for a variety of users. Usability testing has been recognized as one of the great tools in achieving a more easier to understand software interface. However, “there is only one valid way to gather usability data: observe real users as they use your site/program to accomplish real tasks. This is actually the simplest of all the methods: just see what happens!” (Nielsen, 1999).

This usability evaluation can lead to other research ideas that are beneficial to the Human Computer Interaction area. First, additional research is needed for help systems whether they are embedding or some other form of help systems. Second, graphical representation of commands needs further investigation as it relates to users and their understanding of them. Lastly, making users aware of features within a software application should be examined more closely. It seems that manuals, help buttons, and tutorials upon installing software programs are not enough. A more usable approach is needed.

References

Beaudouin-Lafon, M. (1988, November 28-30). User interface support for the integration of software tools: An iconic model of interaction. Paper presented at the Proceedings of the ACM SigSoft/SigPlan Software Engineering Symposium on practical software.

Birnbaum, J. (1985, November). Toward the domestication of microelectronics. Communications of the ACM, 28(11), 1225-1235.

Campbell, G. (1999, September 23). Top ten office features you don't use (but probably should). PC World [Online]. Available: http://wwww.pcworld.com/heres_how/article/0,1400,12897,00.html [1999, December 21].

Cusumano, M. (1997, June). How microsoft builds software. Communications of the ACM, 40(53-61).

Jordan, P. (1998). An introduction to usability. Bristol, Pennsylvania: Taylor & Francis.

Mayhew, D. (1999). The usability engineering lifecycle. San Francisco, CA: Morgan Kaufmann Publishers.

Moyes, J. (1994, April 24-28). When users do and don't rely on icon shape. Paper presented at the Conference Companion CHI '94, Boston, Massachusetts.

Naughton, W. (1995, July). Welcome to the usability evaluation of Xerox Management Model. Paper presented at the UPA Conference.

Nielsen, J. (1999). Alertbox: Current issues in Web usability [Online]. Available: http://www.useit.com/alertbox [1999, November 20].

Nielson, J. (1993). Usability engineering. San Francisco, CA: Morgan Kaufmann Publishers.

Petre, M. (1995, June). Why looking isn't always seeing: Readship skills and graphical programming. Communications of the ACM, 38(6), 33-44.

Pieratti, D. (1995, July). Usability evaluation plan. Paper presented at the UPA Conference.

Randall, N., & Pedersen, I. (1998, September 24-26). Who exactly is trying to help us? The ethos of help systems in popular computer applications. Paper presented at the Proceedings in the sixteenth annual international conference on computer documentation, Quebec, Canada.

Rubin, J. (1994). Handbook of usability testing. New York: John Wiley & Sons, Inc.

Appendix A

Post-test Questionnaire

WordArt Usability Evaluation

Post-test Questionnaire

Name:__________________________ Date:_____________________

Please answer the following questions based on your experience of inserting and modifying the WordArt feature that is part of Microsoft Word 97. Your help in this usability study is greatly appreciated.

1. Overall, I am satisfied with the ease of completing the tasks in this scenario.

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

2. Overall, I am satisfied with the amount of time it took to complete the tasks.

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

3. I can easily locate WordArt in Microsoft Word 97 again.

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

4. I was already aware that the WordArt feature existed in Microsoft Word 97.

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

5. I will probably use the WordArt feature for future documents, presentations, etc.

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

6. The pop-up toolbar with its associated icons and text messages are easy to understand.

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

7. I found the icons were representative of the characteristics to modifying WordArt (i.e. the AV with the arrow underneath it for character spacing).

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

8. Is the ordering of the pop-up toolbar options logical?

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

9. I am happy with WordArt under the Insert/Picture command.

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

10. I like the pop-up toolbar when I inserted the WordArt.

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

11. I like how the program modifies the WordArt as I am performing the tasks. I am able to see the effect instantaneously.

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

12. I understood how to rotate the WordArt after I selected the rotate option.

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

13. Do you have any comments about how the usability evaluation was conducted – likes or dislikes?

________________________________________________________________________________________________________________________________________________

14. Do you have any comments about how easy it was to insert and modify WordArt – likes or dislikes?

________________________________________________________________________________________________________________________________________________

Thank you!

Appendix B

Task List

Usability Test

Overview of the Tasks

Purpose: To see how easy it is for users to insert and modify “Word Art” in a Microsoft Word document.

Task: There are ten tasks. The tasks involve inserting “word art” into a Microsoft Word document as well as modifying the characteristics of it. Each task is given to you one at a time.

Format: For each task that needs to be performed, a systematic instruction is provided with the result described.

Your role: To perform each task. Participation in this test is voluntary and you may stop at any time.

If you are ready, the usability test will start.

Task One