Chapter 1

Introduction

This preliminary proposal is submitted to satisfy the initial requirements of DISS 880 – Multimedia Systems. The following introduction describes the problem to be investigated, the goal to be achieved, and the final deliverable of the project. In addition, the introduction provides an analysis of the relevance of the research and the potential barriers and issues expected during the completion of the project paper. Finally, the introduction details the plan and approach of the project, justifies the legitimacy of the proposed solution, and provides a timeline of project milestones.

Problem Statement and Goal

Virtual reality (VR) applications are used on the Internet to create the illusion of reality. Their objective is to provide as realistic an environment for the user as possible (Tannenbaum, 1998). They are often applied in the areas of entertainment, training, medicine, manufacturing, virtual communities, and e-commerce. As VR applications become more common on the Internet, Web designers are presented with the problem of understanding how and in which areas VR applications are best applied (Powell, 1999).

The goal of this project was to identify which VR technologies and applications were currently in use and to determine in which areas VR applications are best applied.

Final Deliverable

The final deliverable of the project was a ten-minute interactive multimedia presentation titled Virtual Reality on the Internet. The presentation was developed and viewed using Microsoft PowerPoint 2000. The content of the multimedia product introduced current Internet VR technologies: Virtual reality modeling language (VRML), panoramic images, and proprietary VR. In addition, the presentation showed examples of applications developed with each of these technologies. The presentation could be marketed as an interactive training aid for novice Web developers seeking to understand the use and proper application of virtual reality on the Internet.

Relevance

This research was relevant to the study of multimedia systems because of the increasing use of VR applications and the enhanced experience that VR provides viewers of multimedia productions delivered over the Internet (Regan, 1998). The research was significant because a better understanding of the current state of VR applications benefits Web developers as they apply the technology.

Barriers and Issues

The primary barrier to the successful completion of this project was the vast quantity of research material related to VR technologies on the Internet. This material had to be gathered, compiled, filtered, and evaluated to determine its appropriateness to the project.

Successful project completion was also complicated by the requirement for the multimedia product to run smoothly on a variety of personal computers and Internet connection speeds. Incompatibilities between the various VR application drivers, along with PowerPoint imperfections added to the challenge.

Plan and Approach

The project report was a descriptive study formatted in five chapters. The first chapter covered the project’s problem statement and goal, final deliverable, relevance, barriers and issues, plan and approach, product development process, justification, and milestones. The second chapter provided a review of the literature relevant to the problem. The third chapter described the research methods and online tools and resources employed during the completion of the project. In addition, the criteria used to develop the multimedia product and its marketing plan were discussed. Included was a list of desirable product features, along with a description of the processes used to develop and validate that list.

The fourth chapter of the project report documented the product development cycle. Topics to be included were:

Information gathering
Strategy
Prototyping
Implementation
Usability testing and product evaluation

The fifth chapter concluded with a summary of the entire project paper.

Justification

The multimedia product delivered as part of this project was a legitimate solution to the problem of helping Web developers better understand the application of VR technology. Its legitimacy stemmed from the ability of such presentations to effectively instruct through the combination of multiple multimedia elements into one integrated, coordinated, and synchronized work (McKeown, Feiner, Mukesh, & Chang, 1998). Since each of the multimedia elements of the presentation delivered information about the same topic, the individual elements were not generated in isolation and were therefore a more effective instructional medium.

Milestones

The scope of the project report was manageable and lent itself to investigation within the given time period. The following is a summary of the milestones for the project along with significant dates. The first milestone, the submission of the project proposal, was competed and submitted for review on or before October 20, 2000. After approval of the project proposal, chapter four, the final deliverable, and chapter five were the last milestones before submission. Chapter four and the final deliverable were completed on January 28, 2001 and chapter five on February 3, 2001. The final deliverable was mailed on January 29, 2001. After extensive proofreading and testing, the project report was submitted through ESET on January 30, 2001.

Summary

In summary, the section above introduced the problem to be investigated, the goal to be achieved, and the final deliverable of the project. Also included were an analysis of the relevance of the research and the potential barriers and issues expected during the completion of the project paper. Finally, the introduction detailed the plan and approach of the project, justified the legitimacy of the proposed solution, and provided a timeline of project milestones. In the next chapter, this project report provides a review of literature relevant to the problem of understanding how and in which areas VR applications are best applied.

Chapter 2

Review of the Literature

The literature review that follows is organized by subject heading. Those subjects include: bandwidth and QoS considerations, panoramic images and video, proprietary VR, usability testing, and VRML. A review of the literature pertinent to these subjects was critical to achieving the project’s goal to identify which VR technologies and applications were currently in use. In addition to subjects directly related to VR technology, usability testing, and bandwidth and QoS considerations are covered due to their importance in delivering a high quality multimedia product.

Bandwidth and QoS Considerations

Vernick, Venkatramani, and Chiueh (1996) discussed bandwidth and QoS considerations. To successfully distribute a networked version of a multimedia presentation, new issues need to be addressed. Two of these are the need for large aggregate I/O bandwidth and the guarantee of real-time performance to ensure reliable and synchronized playback at the client.

Currently the bandwidth and QoS requirements to deploy the presentation "as is" are not available over the Internet. These limitations can be addressed by either stripping the presentation of most of its multimedia content or by delivering the presentation in a manner that compensates for the limitations of the Internet. The latter is the most acceptable. Therefore the deliverable will be a stand-alone presentation delivered on a compact disc.

Panoramic Images and Video

Panoramic images are often described as “poor man’s” VR. Powell (1999) discussed how faking the real world with panoramic images is sometimes better than trying to model it. Many times a full screen picture of a 3D object will suffice. Panoramic images can be used to create the illusion that a user is in 360-degree rotation. When the capability to click on the image and zoom in and out is added, the image seems more like virtual reality than VRML.

Examples of this technology include IPIX – PhotoBubbles, QuickTime VR, QuickStitch, and RealPlayer VR Video. Powell added to the project by categorizing the various types of VR applications to be investigated and presented.

Sellers (2000) described the continued growth of Apple’s QuickTime VR technology. Recently, Corbis began offering the first 360-degree panoramic catalog of stock photos in the industry. The images are typically used for multimedia applications presented at trade shows and in retail displays. The panoramics are viewed using the QuickTime VR engine. Sellers contributed to the project by pointing out one possible source of content to include in the final deliverable.

Proprietary VR

Bourdakis (1997) compared proprietary VR systems with VRML applications in another article. The paper discussed the advantages of VRML compared with proprietary VR systems. VRML systems tended to be more expensive and lacked a number of advanced features such as model creation tools.

In addition, many VRML applications required a new hardware/software platform for development. The paper contributed to the project by pointing out differences between the two technologies.

Usability Testing

Usability testing focuses on whether a user interface is easy to learn, satisfying to use, and has the functionality that users want (Branaghan, 1999). Its goal is to show the designer how a product might be improved. In traditional usability tests, the tester observes users as they use a product to perform tasks (e.g. locate information on the Internet). During the testing, the evaluator collects both quantitative and qualitative data describing user performance and satisfaction. Usability testing also provides the researcher the opportunity to “pick the user’s brain” by asking follow-up questions. In a recent text, Jordan (1998) described how to conduct a usability evaluation. According to Jordan, evaluating an existing product has advantages. Products that have been on the market for a while have an experienced user base. These users are able to report on the positive and negative aspects of using a product in its real context of use.

Jordan continued with a description of the two types of data that can be elicited from a usability evaluation: quantitative data and qualitative data. Quantitative data is useful in situations where a design decision has to be made and a number of possible solutions are being considered. Qualitative data is also useful for a number of reasons. First, it can be used as an approximation of quantitative data when making a "first pass" at addressing an issue. Next and more important, qualitative data can be used to diagnose usability faults and prescribe solutions.

In another text, Rubin (1994) defines usability testing as a process that employs participants who are representative of the target population to evaluate the degree to which a product meets specific usability criteria. Rubin's text emphasized the more informal, less complex testing methods designed for quick turnaround of results in industrial product development environments.

Rubin described the test plan as the foundation for the entire test. It addresses, the how, when, where, who, why, and what of a usability test. The test plan serves as the blueprint for the test. It is also the main communication vehicle between the developer, test monitor, and the rest of the development team.

VRML

VRML is a 3-D graphics language used on the Web. VRML pages can be viewed and rotated. Lawton, Li, Lien, Chiu, and Yu, and Matsuba and Roehl contributed to the project as follows:

Lawton (1999) described a number of VRML-based projects recently completed for clients that included Platinum technologies and CNN Interactive. The application developed for Platinum was a 3D network visualization that allowed nontechnical users to easily navigate through and diagnose problems with a network. The CCN project used a large 3D polygonal object mapped with a high-resolution image of a hurricane.

One roadblock encountered by the author during the course of each of these projects was the lack of one standard VRML browser. Key to the success of future VRML applications is this standard. The author’s real life examples of VRML applications on the Internet aid in the development of the final deliverable by pointing out a number of possible roadblocks.

Li, Lien, Chiu, and Yu (1999) investigated the problem of navigating 3D landscapes with 2D user input devices. Their paper proposed an alternative navigation metaphor in which users specify locations of interest on a 2D-layout map and the system automatically generates an animated guided tour in 3D.

The system was implemented using Java and a common VRML browser interface. Also employed were auto-navigation techniques in which several efficient path-planning algorithms were used. The paper benefits the research topic by illustrating an effective technique to improve the usability of browser-based virtual reality.

Matsuba and Roehl (1999) described the development of the VRML Dream Project (a live performance of Shakespeare’s A Midsummer Nights Dream). VRML Dream was the first live, streaming, VRML entertainment project with a running time of greater than two minutes. The production is relevant to the project because it proved that streaming both motion and voice data over standard Internet connections was possible.

Chapter 3

Methodology

Research Type

The project paper was a research-based descriptive study. The key outcome of the investigation was the identification of which VR technologies and applications are currently in use and to determine in which areas VR applications are best applied. In addition, the results of the study were formatted into a ten-minute interactive multimedia presentation titled Virtual Reality on the Internet.

Research Methods Employed

The primary research method employed throughout the course of this project was browser-based Internet searches. The literature reviewed included textbooks, white papers, Web site reviews, trade journals, and magazine articles referenced by a selected set of online resources. Relevant texts were located, ordered, and delivered using the Amazon.com Internet site. The full text articles from trade journals, magazines, and white papers were located and subsequently downloaded from a collection of online resources.

Online Tools and Resources

A variety of online agent technology resources were used to locate and download literature relevant to the goal of the project. These resources included ACM Search (www.acm.org/dl/search.html), Electric Library (www.elibrary.com), Gartner Group (www.gartner.com), and ProQuest Direct (proquest.umi.com). Perhaps the most powerful search tool to be employed during the course of the project was the intelligent search agent, Copernic 2000.

Copernic 2000 is a well-documented freeware search agent. It uses predefined channel sets, which allows researchers to target inquiries to all major Web search engines, search for relevant text in newsgroups, and access popular e-mail directories to find people (Copernic, 2000). Copernic conducts fast, multithreaded, full Boolean searches with progress displays and customizable search depth. Once results are compiled, Copernic displays returns (including name, location, and introductory text) in a right-click-enhanced list box sorted by relevance.

Product Criteria

The multimedia product that was the final deliverable of the project needed to accomplish a number of tasks in order to solve the problem. The product needed to be interactive, use multiple media types, interest the target audience composed of novice Web developers, last about ten minutes, run on an Internet Explorer-connected Windows multimedia PC purchased within the last 18 months, and provide a brief overview of VR on the Internet.

The above criteria were developed using the minimum requirements set forth in the DISS 880 course syllabus along with the realization that PCs older than 18 months would not be representative of computers typically used by Web developers. The criteria was validated by a survey of minimum requirements for other multimedia teaching aids along with consideration for Moore’s law (i.e. PCs become twice as powerful every 18 months) (Silverman, 1998).

Marketing Plan

Three ways to market the multimedia product are through corporate sponsorship, a product Web site, and targeted e-mails (Bayne, 1997). Corporate sponsorship by companies whose products are featured in the presentation would be the first step in funding a more comprehensive marketing strategy. An attractive product Web site would be among those strategies and could be positioned as one of the top ten search hits through the use of Web promotion tools such as Siteadd.com.

A third approach would be the use of direct e-mail marketing. E-mail marketing has proven to offer fast responses along with a high rate of return due to the low cost of the medium (Regan, 2000). E-mail marketing is predicted to increase from $164 million in 1999 to $7.3 billion in 2005. The first step in this marketing approach would be to join VR and Web development newsgroups and to participate in the discussions (Bayne, 1997). Participating in these discussions in a useful and helpful manner would provide a targeted list of potential customers along with their e-mail addresses. Follow-up e-mails introducing the product would be the next step.

Chapter 4

Results

The following chapter begins with a detailed discussion of the development process used to create the presentation, Virtual Reality on the Internet. Topics covered are information gathering, strategy, prototyping, and implementation. These are followed by a section that focuses on usability testing and product evaluation. The usability heuristics and strategies employed in evaluating the presentation along with user selection criteria and survey results are discussed.

Development Process

Developing a multimedia presentation is often a lengthy process that has far‑reaching effects upon both the individuals and organizations involved (Lynch & Horton, 1999). In-spite of this, many multimedia presentations are only ad hoc efforts. They are the result of poor planning and hasty development. In consideration of this fact and recommendations by Fleming, the process used during the development of Virtual Reality on the Internet followed five phases (Fleming, 1998):

1. Information gathering

2. Strategy

3. Prototyping

4. Implementation

5. Evaluation

Phase One: Information Gathering

Phase one focused on the collection of background information. Three important questions needed to be answered during this critical phase. Those questions were:

· What are the purpose and goal of the presentation?

· Who is the target audience for the presentation?

· What resources are available?

Purpose and Goal

The purpose and goal of the presentation was to fulfill a course requirement of DISS 880 – Multimedia Systems. In addition, the goal of the presentation was to provide interactive training for novice Web developers seeking to understand the use and proper application of virtual reality (VR) on the Internet.

Target Audience

Virtual Reality on the Internet was designed with novice Web developers as its target audience. As a result, presentation content was limited to topics that were representative of the current state of VR technology on the Internet. One of the usability problems found in many multimedia presentations is a design that took place without a clear target-user population in mind (Head, 1999).

Resources

Resources available to complete the presentation included high-speed Internet access, a Pentium III, 750 MHz computer with 256 MB of memory, a wide selection of downloadable multimedia viewers, editors, and content, Microsoft’s PowerPoint 2000, numerous VR and usability texts and journal articles, and DISS 780 and 880 class notes.

Phase Two: Strategy

The strategy phase focused on brainstorming and problem solving. Problems solved included choosing a presentation theme and selecting software (i.e. an authoring system and audio, video, and image viewing and editing software).

Theme

During the strategy phase, many VR related Web sites were explored. These sites included: AlertBox.com, TechTV.com, Pint.com, Corbis.com, SGI.com, IEEE.org, VRuniverse.com, ActiveWorlds.com, Web3d.org, and Worlds.net. Each provided a sample of VR use on the Internet along with themes and content that were well received by their target audiences.

For example, the AlertBox site was formatted as a list of links (Nielsen, 1999). The links were brief text descriptions of Nielsen’s latest HCI columns, which provided valuable information about the usability testing process. The other sites visited provided both ideas and content to integrate into the finished presentation. After performing the above site exploration, it was decided that the theme of the presentation would be VR related multimedia images and audio mixed with links to active VR examples on the Internet.

Software

The next step in the strategy phase focused around software selection. Perhaps the most important software application to be selected was the authoring system. PowerPoint 2000 was selected for this role. Early on in the strategy phase, it was decided that it would be technically impossible to distribute a networked version of the presentation. The presentation’s audio and image files when combined with the textual content would require tens of megabytes of data storage, and a networked version would complicate the distribution process with issues such as the need for large aggregate I/O bandwidth and the guarantee of real-time performance to ensure reliable and synchronized playback at the client (Vernick, Venkatramani, & Chiueh, 1996). Currently the bandwidth and QoS requirements to deploy the presentation "as is" are not available over the Internet.

Therefore, a standalone PowerPoint presentation seemed the most appropriate. PowerPoint is a simple yet powerful authoring program whose widespread use minimizes distribution and execution problems. In addition, the author was already familiar with the application, and time constraints did not allow him to become proficient with another.

Once PowerPoint 2000 was chosen as the authoring system, compatible multimedia viewers and browser plug-ins were selected and tested. These included Internet Explorer 5.5, Windows Media Player 7, RealPlayer 8, IPIX plug-in for Internet Explorer, and the IPIX plug-in for Real Player. Finally, audio and image editing applications were selected. These included Cool Edit 2000, Acoustica, MP3 Decoder, and Internet Audio Mix.

Phase Three: Prototyping

In phase three, a rough plan and flowchart were created to predict how visitors would view and navigate through the presentation. First a list of topics was developed. These topics were the introduction, panoramic images, proprietary VR, and VRML. Each topic was covered by one or more slides and was hierarchically at the same level as the others.

Next, the flowchart shown below in Figure 1 was constructed to determine the most logical sequence for the presentation. Flowcharts and storyboards are tools used to assist with content and navigation design. However, storyboards show something that flowcharts are unable to: what is happening in the presentation as the user moves through it (Fleming, 1998). Once the flowchart was drafted, PowerPoint was used to create a rough storyboard. This storyboard was then iteratively improved in phase four until it became the final presentation.

Text Box: Load Presentation
(25 sec)

Multimedia Application Installation and Test

Introduction Credits

(2 Slides)

Figure 1. Virtual Reality on the Internet PowerPoint Presentation Flowchart.

Phase Four: Implementation

Phase four involved the actual construction of the presentation. This included developing the content, designing the slides, selecting the multimedia images and sounds, and working through any problems encountered.

Content

As described above, the presentation’s content focused on the use of virtual reality on the Internet. Before the start of the main topics, two slides were inserted to ensure that the presentation executed without technical difficulties. The first slide, timed to advance in 25 seconds, was included to give slower computers enough time to fully load all files before advancing further. The slide also gives viewers the opportunity to adjust the sound level of their speakers. Background music was provided throughout the presentation to relax the user and to tie the whole presentation together.

The presentation then advances through two slides of introductory credits before stopping at the main menu. This slide contains navigation links to the presentation’s key topic areas: introduction, panoramic images, proprietary VR, and VRML. The introduction contains a voice recording that describes the presentation’s focus along with a text transcript. The slides covering other topic areas contain a mixture of related images, music, and links to Internet examples of the specific VR technology. For example, panoramic image links take the user to 360-degree images and video created using IPIX stitching technology. When users are finished viewing the presentation, an exit link allows them to end the presentation while viewing exit credits.

Slide Design

The slide design process began with the selection of an overall theme. In general, this consisted of slide backgrounds composed of visually pleasing and interesting photographic images. The text used was Arial Unicode MS font in either black or white depending on the background. Navigation links were configured to change color to indicate use. The slide design process continued with the thorough testing of all links and navigation aids. Before moving to phase five, the presentation was edited for spelling and grammar errors.

Multimedia Selection

The multimedia images and audio tracks used throughout the presentation came from a variety of sources. Photographic and GIF images were obtained with permission from Corbis.com and the Microsoft Clip Gallery. Audio tracks were copied from CDs in the author’s music library. The pictures and audio incorporated into the presentation were selected to convey a simple, consistent, and impressive theme to the user.

Problems Encountered

The image and audio files used during the initial stages of development were JPEG and WAV types of considerable size. When combined with the textual content, the presentation required 28 megabytes of data storage. Since the presentation was designed to run as a stand-alone multimedia production, it required considerable hardware, software, and storage capabilities at the client. Computers without extensive processor and memory resources were unable to run the presentation without difficulty (e.g. out of sync, distorted images and background, and generally slow response). The author became aware of this problem from feedback received during the design phase evaluation process described in the next section.

These problems were addressed by reediting all the sound and image files. Audio WAV files were remixed, shortened, looped, and compressed. Image JPEG files were resized, cropped, and saved at lower resolution settings. The result of the reediting was the reduction of presentation size from 28 megabytes to 14 megabytes. This reduction enabled to presentation to be successfully viewed on a larger number of computers (i.e. computers with at least a Pentium III 300 MHz processor and 128 MB of RAM memory).

The most important lessoned learned during this exhaustive reediting process was the importance of minimizing files size during all stages of the authoring process. Other lessons learned dealt with the many limitations inherent in PowerPoint and its viewer. For example, the PowerPoint 97 viewer packaged with PowerPoint 2000 is unable to handle animated GIF files or Internet hyperlinks. This limitation requires users to have the PowerPoint 2000 application loaded on their computers to completely view all aspects of the presentation.

Phase Five: Evaluation

The fifth and final phase of the process was to evaluate the presentation’s usability and its effectiveness in accomplishing the goal of the project (i.e. to provide interactive training for novice Web developers seeking to understand the use and proper application of virtual reality (VR) on the Internet). The evaluation criteria for the presentation was matched to the product criteria as follows:

· Two test subjects from the intended audience (novice Web developers)

· Two evaluations – design phase and finished product

· Was the presentation interactive?

· Did the presentation use multiple media types?

· Was the presentation interesting and informative to its intended audience?

· Did the presentation last about ten minutes?

· Did the presentation successfully run on a PC purchased in the past 18 months?

· Did the presentation provide a brief overview of VR on the Internet?

Using the above evaluation criteria, the evaluation questionnaires provided in Appendices A and B were developed. These contain fixed response questions with the option to provide additional comments. Questionnaires have long been used to evaluate user interfaces (Branaghan, 1999). In fact, a number of reliable and valid questionnaires have been created to assess aspects of usability (Perlman, 1999). Examples are the Heuristic Evaluation Questionnaire, the Computer System Usability Questionnaire, and the Practical Heuristics for Usability Evaluation.

The Design Phase Product Evaluation Questionnaire (Appendix A) was administered about halfway through the implementation phase. The questionnaire’s purpose was to obtain feedback from test users to ensure that the finished presentation would meet all product criteria. The Final Product Evaluation Questionnaire (Appendix B) was administered at the end of the implementation phase.

User Profiles

Both the design and final evaluation questionnaires were sent by e-mail to the same two test users. These users were volunteers selected by the author because they were representative members of the target audience (i.e. novice Web developers). Table 1 below outlines their user profiles.

Table 1. User Profiles

Characteristic	User 1	User 2
Age	34	28
Gender	Female	Female
Education	Post Graduate	Post Graduate
Computer Experience	20 Years	15 Years
Web Developer	Novice	Novice
PC Age	3 Months	12 Months

The questionnaires were e-mailed to the test users along with a link to a Web site from which the presentation could be downloaded. The finished questionnaires were returned by e-mail to the author. Both users easily adapted to the online format of the evaluation process and reported no problems.

Evaluation Results

The results of the Design Phase Product Evaluation Questionnaire (Table 2 below) demonstrated that halfway through the implementation phase the presentation was fulfilling the product criteria. Both test users found the presentation to be informative and interesting. In addition, they found the length to be appropriate and the multimedia elements effectively integrated. However, both users reported technical difficulties (i.e. slow execution and locking up). These problems were diagnosed to be caused by the large multimedia files sizes incorporated in the presentation. As detailed in the phase four discussion above, the problem was eliminated by reducing file sizes.

One of the users commented that the installation and testing procedures given in the second slide were tedious. This procedure was simplified by eliminating the slide’s multiple links to the Internet. These links were replaced by a single link to a Web page that contained detailed installation and testing instructions and links. Performing these setup functions from an Internet page eliminated the need for the user to exit and reenter the presentation multiple times.

Table 2. Results from the Design Phase Product Evaluation Questionnaire

Question	User 1	User 1 Comments	User 2	User 2 Comments
1. Please describe your computer’s basic hardware & software specifications: - Processor - Processor Speed - RAM - Age (Months) - Operating System - Connection Speed	Pentium III 1 GHz 256 MB 3 Months Win ME 56Kb	NA	Pentium III 750 MHz 512 MB 12 Months Win 2000 1.5 Mb	NA
2. How would you rate your ability as a PowerPoint user?	Expert	NA	Expert	NA
3. How would you rate your experience as a Web developer?	Novice	NA	Novice	NA
4. Did you experience any technical difficulties while running the presentation?		Yes, the presentation executed slowly and the sound & images were out of sync.		Yes, the presentation locked up once and seem to be sluggish.
5. I found the presentation to be of interest and helpful as I develop future Web pages.	Strongly Agree	I have often though of including 3D in my Web pages.	Agree	Interesting topic- not sure until I see the finished presentation.
6. The navigation links were effective, clear, and helped to determine where I wanted to go next.	Strongly Agree	The link structure is easy to follow.	Strongly Agree	Simple and clearly marked links.
7. The presentation had the appropriate amount of information for an introduction to virtual reality on the Internet.	Agree		Agree
8. The length of the presentation was:	Just Right		A Little Short	You may need to provide a few more examples.
9. The music was appropriate and added to the experience.	Strongly Agree	The samples provided were excellent.	Agree	I liked the music but the volume varied between selections.
10. I found the audio introduction to be effective.	Agree	However the pace needs to be picked up	Strongly Agree
11. The presentation’s integration of various multimedia types added to its effectiveness.	Strongly Agree		Strongly Agree
12. The background images and text colors were pleasing.	Strongly Agree		Strongly Agree
13. The presentation provided a cohesive overview of virtual reality on the Internet.	Neither Agree or Disagree	Not enough to decide yet	Neither Agree or Disagree	I have to wait till the finished presentation to decide
14. How much extraneous information was given in the presentation?	None so far		None
15. How would you rate the presentation’s flexibility and efficiency of use?	Excellent		Above Average
16. The presentation’s use of examples from the Internet was effective.	Agree	Linking to the Internet is an excellent idea.	Strongly Agree	Should be effective – however a few more links may be needed.
17. Overall, how would you rate the presentation?	Excellent – so far		Excellent
18. Please enter any other suggestions to improve the presentation.	NA	I found that leaving the presentation multiple times to install the various software applications was tedious.	NA	None

After the implementation phase was nearly completed, the Final Product Evaluation Questionnaire (Table 3 below) was distributed to the same two test users. Most significant of their feedback were comments that the finished product was an excellent follow through of the initial design concept. In addition, they found the presentation to be informative and interesting and felt it stimulated interest in using VR technologies in their Web sites. Recommendations for improvement included the rearrangement of the titles on the main menu slide along with the addition of text descriptions whenever graphic links were used (i.e. the proprietary VR slide links). One user also suggested that a text transcript be added to the voice introduction. All recommendations were obvious improvements and were incorporated into the finished product.

Table 3. Results from Final Product Evaluation Questionnaire

Question	User 1	User 1 Comments	User 2	User 2 Comments
1. Did you experience any technical difficulties while running the presentation?	Echo in voice in Introduction	NA	None	NA
2. I found the presentation to be of interest and helpful as I develop future Web pages.	Strongly Agree	Provided a good introductory view of concepts as well as a wealth of useful links.	Agree
3. The navigation links were effective, clear, and helped to determine where I wanted to go next.	Strongly Agree	The links were extremely helpful and provided good sources of information. It would have been helpful to know when I was about to leave the presentation to a Web site, but this understanding was gained quickly.	Neither Agree or Disagree	The links were clear and worked. However, I prefer to have back arrows to reread material without having to go back to the Main Menu. In addition the Web link images in Proprietary VR were not labeled.
4. The presentation had the appropriate amount of information for an introduction to virtual reality on the Internet.	Strongly Agree	I learned a great deal without being inundated. It sparked my interest in using the technology.	Strongly Agree	Very good selection and neat examples
5. The length of the presentation was:	Just Right	I had the ability to move through the presentation at my own pace and find any depth of information that I needed. It was an overview but yet very thorough in establishing the base knowledge necessary for future research.	Just Right
6. The music was appropriate and added to the experience.	Strongly Agree	I was soothing, which was necessary for such a difficult topic.	Strongly Agree	Excellent music
7. I found the audio introduction to be effective.	Strongly Agree	The introductory page established a good baseline of sound and also advanced after a period of time. The moving dots provided a nice count-down for sound adjustment.	Neither Agree or Disagree	It was okay. The titles appeared and disappeared too fast.
8. The presentation’s integration of various multimedia types added to its effectiveness.	Agree	The voice and text were well used to make points and to underscore those points. It kept the user focused. One suggestion may be to have text available (perhaps as a link) to voice sound tracks.	Strongly Agree
9. The background images and text colors were pleasing.	Strongly Agree	This was the most fantastic part of the presentation. It presented strong graphics and technology but was carefully balanced so as not to overpower the presentation. It kept me thoroughly captivated but was easy to read and hear. While I had no previous interest in this topic, through out the whole presentation I was thinking “how did he do that, I want to do that.”	Strongly Agree	Loved the pictures
10. The presentation provided a cohesive overview of virtual reality on the Internet.	Strongly Agree		Strongly Agree
11. How much extraneous information was given in the presentation?	None	None of the information was extraneous; instead, it provided different levels of depth. For example, the first provided definitions and an overview then the hyperlinks provided greater detail and examples.	None
12. How would you rate the presentation’s flexibility and efficiency of use?	Excellent	I was able to control the speed and direction of the presentation.	Above Average
13. The presentation’s use of examples from the Internet was effective.	Strongly Agree	Great resource for further investigation.	Agree	Even with a fast PC, it took quite a few minutes for the Web pages to load.
14. The Final Presentation was an excellent follow through of the initial design concept.	Strongly Agree	I liked the way you moved the application install and testing to the Internet		Strongly Agree
15. Overall, how would you rate the presentation?	Excellent			Excellent
16. Please enter any other suggestions to improve the presentation.	NA	Move the introduction choice to the top so that the flow of the main menu is top to bottom rather than bottom to top. Still, extremely well done presentation.	NA

Chapter 5

Conclusion

After the evaluation process was completed, the presentation was distributed in two formats. The first format was CD-R media, which contained the packngo files created by PowerPoint to simplify distribution. The second format was a download link located at the bottom of the following Web page:

http://www.scisstudyguides.addr.com/multimedia_player_installation.htm

In conclusion, this project report was formatted in five chapters. The first chapter covered the project’s problem statement and goal, final deliverable, relevance, barriers and issues, plan and approach, justification, and milestones. The second chapter provided a review of the literature relevant to the problem. The third chapter described the research methods and online tools and resources that would be employed in completing the project. In addition, the criteria used to develop the multimedia product and its marketing plan were discussed. The fourth chapter of the report included a discussion of the development process, product evaluation criteria, and evaluation results.

Finally, in the future, updated versions of this presentation will be made available at the above Internet link. VR technology on the Internet is constantly improving, and the dynamic nature of online publishing and distribution will allow subsequent versions to quickly reach interested Web developers.

Appendix A

Design Phase Product Evaluation Questionnaire

After viewing the presentation Virtual Reality on the Internet, please complete the following questionnaire. Comments are encouraged and will be incorporated into the final presentation.

1. Please describe your computer’s basic hardware & software specifications:

Pentium I, II, or III _____________

Processor Speed (MHz) _____________

RAM Memory (Mb) _____________

Age (Months) _____________

Operating System _____________

Internet Connection Speed _____________

2. Did you experience any technical difficulties while running the presentation (e.g. audio/video not in sync or Windows crashed)?

Comments:______________________________________________________________

3. How would you rate your ability as a PowerPoint user? (Check one)

____ None

____ Very Little

____ Average

____ Above Average

____ Expert

Comments:______________________________________________________________

4. How would you rate your experience as a Web developer? (Check one)

____ None

____ Very Little

____ Average

____ Above Average

____ Expert

Comments:______________________________________________________________

5. I found the presentation to be of interest and helpful as I develop future Web pages. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

6. The navigation links were effective, clear, and helped to determine where I wanted to go next. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

7. The presentation had the appropriate amount of information for an introduction to virtual reality on the Internet. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

8. The length of the presentation was: (Check one)

____ Too Short

____ A Little Short

____ Just Right

____ A Little Long

____ Too Long

Comments:______________________________________________________________

9. The music was appropriate and added to the experience. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

10. I found the audio introduction to be effective. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

11. The presentation’s integration of various multimedia types added to its effectiveness. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

12. The background images and text colors were pleasing. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

13. The presentation provided a cohesive overview of virtual reality on the Internet. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

14. How much extraneous information was given in the presentation? (Check one)

____ None

____ Very Little

____ More Than a Little

____ A Lot

____ Way Too Much!

Comments:______________________________________________________________

15. How would you rate the presentation’s flexibility and efficiency of use? (Check one)

____ Poor

____ Fair

____ Good

____ Above Average

____ Excellent

Comments:______________________________________________________________

16. The presentation’s use of examples from the Internet was effective. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

17. Overall, how would you rate the presentation? (Check one)

____ Poor

____ Fair

____ Good

____ Above Average

____ Excellent

Comments:______________________________________________________________

18. Please enter any other suggestions to improve the presentation.

_______________________________________________________________________

Appendix B

Final Product Evaluation Questionnaire

After viewing the presentation Virtual Reality on the Internet, please complete the following questionnaire. Comments are encouraged and will be incorporated into the final presentation.

1. Did you experience any technical difficulties while running the presentation (e.g. audio/video not in sync or Windows crashed)?

Comments:______________________________________________________________

2. I found the presentation to be of interest and helpful as I develop future Web pages. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

3. The navigation links were effective, clear, and helped to determine where I wanted to go next. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

4. The presentation had the appropriate amount of information for an introduction to virtual reality on the Internet. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

5. The length of the presentation was: (Check one)

____ Too Short

____ A Little Short

____ Just Right

____ A Little Long

____ Too Long

Comments:______________________________________________________________

6. The music was appropriate and added to the experience. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

7. I found the audio introduction to be effective. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

8. The presentation’s integration of various multimedia types added to its effectiveness. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

9. The background images and text colors were pleasing. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

10. The presentation provided a cohesive overview of virtual reality on the Internet. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

11. How much extraneous information was given in the presentation? (Check one)

____ None

____ Very Little

____ More Than a Little

____ A Lot

____ Way Too Much!

Comments:______________________________________________________________

12. How would you rate the presentation’s flexibility and efficiency of use? (Check one)

____ Poor

____ Fair

____ Good

____ Above Average

____ Excellent

Comments:______________________________________________________________

13. The presentation’s use of examples from the Internet was effective. (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

14. The Final Presentation was an excellent follow through of the initial design concept (Check one)

____ Strongly

Disagree

____ Disagree

____ Neither Agree/Disagree

____ Agree

____ Strongly

Agree

Comments:______________________________________________________________

15. Overall, how would you rate the presentation? (Check one)

____ Poor

____ Fair

____ Good

____ Above Average

____ Excellent

Comments:______________________________________________________________

16. Please enter any other suggestions to improve the presentation.

_______________________________________________________________________

Reference List

Bayne, K. (1997). Internet Marketing Plan. New York: John Wiley & Sons, Inc.

Bourdakis, V. (1997). The future of VRML on large urban models. Paper presented at the UK VRSIG 97, Somerset, UK.

Branaghan, R. (1999). Testing, one -- two -- three: Fundamentals of usability testing. Fitch. http://www.branaghan.com/fun_utesting.htm. Updated October 5, 1999. Accessed September 30, 2000.

Copernic (2000). Copernic 2000. http://www.copernic.com. Accessed October 8, 2000.

England, E., & Finney, A. (1999). Managing Multimedia: Project Management for Interactive Media (2nd ed.). Harlow, England: Addison-Wesley.

Fleming, J. (1998). Web Navigation. Sebastopol: O'Reilly & Associates, Inc.

Head, A. (1999). Design Wise: A Guide for Evaluating the Interface Design of Information Resources. Medford, New Jersey: CyberAge Books.

Jordan, P. (1998). An Introduction to Usability. Bristol, Pennsylvania: Taylor & Francis.

Lawton, M. (1999, March-April). Advancing 3D through VRML on the Web. IEEE Computer Graphics and Applications, 19(2), 4-5.

Li, T., Lien, J., Chiu, S., & Yu, T. (1999, May 26-29). Automatically generating virtual guided tours. Paper presented at the IEEE Conference on Computer Animation, 1999.

Lynch, P., & Horton, S. (1999). Web style guide. New Haven, Connecticut: Yale University Press.

Matsuba, S., & Roehl, B. (1999, March-April). "Bottom, thou art translated": The making of VRML Dream. IEEE Computer Graphics and Applications, 19(2), 45-51.

McKeown, K., Feiner, S., Mukesh, D., & Chang, S. (1998). Generating multimedia briefings: coordinating language and illustration. Artificial Intelligence, 103, 95-116.

Nielsen, J. (1999). AlertBox: Current issues in Web usability [Online]. Available: http://www.useit.com/alertbox [1999, November 20].

Perlman, G. (1999). Web-based user interface evaluation with questionnaires. ACM [Online]. Available: http://www.acm.org/~perlman/question.html [1999, October 30].

Powell, T. (1999). The Reality of VR on the Internet. PINT Publishing. http://www.pint.com/Workshop/mech1-8.htm. Accessed October 8, 2000.

Regan, K. (2000). E-Mail Marketing To Reach $7.3B by 2005. E-Commerce Times. http://www.ecommercetimes.com/news/articles2000/000509-4.shtml. Updated May 9, 2000. Accessed January 28, 2001.

Regan, T. (1998, February 16-20). Taking living worlds into people's living rooms. Paper presented at the Third Symposium on Virtual Reality Modeling Language, Monterey, California.

Rubin, J. (1994). Handbook of usability testing. New York: John Wiley & Sons, Inc.

Sellers, D. (2000). QTVR Used in Panoramic Image Offerings. MacCentral Online. http://www.maccentral.com/news/0002/29.qtvr.shtml. Updated February 29, 2000. Accessed October 19, 2000.

Silverman, D. (1998, May 27, 1998). Balance PC needs with cost, life span. Houston Chronicle.

Tannenbaum, R. (1998). Theoretical Foundations of Multimedia. New York: Computer Science Press.

Vernick, M., Venkatramani, C., & Chiueh, T. (1996). Adventures in building the Stony Brook video server. Proceedings of the Fourth ACM International Multimedia Conference on Proceedings ACM Multimedia 96, ACM, Boston, pp. 287.