Usability study – Google Earth vs. Virtual Earth

On the scope of my thesis I did an usability study of Google Earth and Virtual Earth and in the end compared the results obtained from both.

I did a heuristic evaluation and usability tests with 5 users. Users age is between 20 and 30 years old, they use the Internet very regularly and had used some kind of GIS before.

Concerning the heuristic evaluation results show it is not possible to say that one system it is better than the other. There are points where Google Earth is better than Virtual Earth and vice versa. Observing the chart below it is possible to see for each heuristic (see [1]) which was the system that behaved better.

Google Earth vs. Virtual Earth - Problems weight by heuristic

Google Earth vs. Virtual Earth - Problems weight by heuristic

For example on the “Presentation – Aesthetic and minimalist design” and “Match between the system and the real world” heuristics Virtual Earth is clearly better. In fact, Virtual Earth had a simple and familiar interface while on Google Earth there was the toolbar above the map whose icons were not very familiar and thereby their function not understood. Virtual Earth also had better results on “Errors – help diagnose, recognize and recover from them” heuristic where Google Earth looses because of the lack of usefulness of the information present on error messages. Except for the “Errors prevention” heuristic where both systems have the same problems, on the rest of the heuristics Google Earth had an overall better performance. On the “visibility of system status” heuristic Google Earth had a much better performance than Virtual Earth. Virtual Earth looses because it does not provide any visual feedback concerning what is already selected and what is not, and because information at the current location does not match the expected. (see [2]).

Concerning usability tests (see [3]), results show that overall users liked more Google Earth. On the chart below it is possible to see the results from Google Earth and Virtual Earth concerning the detected users’ frustration level divided by tasks.

Google Earth vs. Virtual Earth - User Frustration Level by Task

Google Earth vs. Virtual Earth - User Frustration Level by Task

Overall Google Earth caused less frustration on users than Virtual Earth, especially on tasks 1.1 and 1.2 from scenario 2, where users had to find hotels and interesting places to visit on Barcelona. On Virtual Earth this task was done using the “Collections” menu. Like it was previously said this concept was not well understood by users and the menu suffers from lack of information and organization.
On tasks 1.1 and 1.2 from scenario 1 the difference between the two systems it is not very high, however Google Earth had better results. This can be explained by the fact that Virtual Earth had some inconsistencies, being one of them the loss of pushpins which users used to mark their start and end points. But on task 1.3 from scenario 1, users found Virtual Earth less frustrating than Google Earth. On this task users had to found the route information. One of the problems identified on Google Earth was that this information was not visible, being placed at the end of the route description. On Virtual Earth this information is placed at the beginning of the route description, thereby being discovered faster by users. The task 2 of scenario 1 has similar results on both systems. In fact many users got very frustrated because systems did not allow inserting a middle point on the route, having some of them given up of performing the task claiming it was impossible.

Regarding the System Usability Scale (see [4]) score comparison shows that Google Earth got an average score of 75,5 and Virtual Earth of 61,5.

This study shows that Google Earth is a more consistent application than Virtual Earth, with less crashes and a more robust task workflow. Virtual Earth is a more simple application, with a simple and clean user interface, however regarding user experience it’s not very robust presenting several inconsistencies (like placing pushpins), the search engine needs some enhancements and the contents are still very poor. Google Earth it’s not perfect, presenting on the user interface too many elements that only confuse users on their tasks, these elements aren’t also well described making their function not understood by users. Error messages are also an improving point as they don’t provide usefull information.

Study made on the scope of a Masters Degree on Computer Engineering from February 2008 to July 2008.
[1] Jakob Nielsen, Ten Usability Heuristics, 2005
[2] Susana Vilaça, Heuristic Evaluation Checklist, in Master Thesis on the Role of Human Computer Interaction on Geographical Information Systems, Faculdade de Engenharia da Universidade do Porto, July 2008
[3] Susana Vilaça, Usability Tests Definition, in Master Thesis on the Role of Human Computer Interaction on Geographical Information Systems, Faculdade de Engenharia da Universidade do Porto, July 2008
[4] Brooke, J, SUS: a “quick and dirty” usability scale, In P. W. Jordan, B. Thomas, B. A. Weerdmeester & A. L. McClelland (eds.) Usability Evaluation in Industry.

Posted in Studies | Tagged , , , , ,

Share this post

2 Responses

  1. Hi, you can get a copy of my thesis here: http://susanavilaca.files.wordpress.com/2008/08/thesis_susanavilaca_v201.pdf

    Thank you for your interest!

  2. zulfiqar

    Hi, i am interested to read this thesis. So can i get a copy of thesis

Leave a comment