Eye-tracking data: What does it mean??

I had an interesting discussion with a colleague today. We had both read a recent eye-tracking study published by a prominent usability expert… and we both had essentially the same take on it: “So what?”

Much of the data are open to widely different interpretations. What does it mean when someone spends a lot of time looking at something? Does it mean he finds it appealing? Or is he just having a hard time understanding it?

If someone scans down the page quickly, does that mean the page has failed to capture his interest? Or does it mean the page was effective in communicating its message quickly?

Last month, I was in a usability seminar in Seattle. The seminar leader — a very accomplished usability professional — cautioned against reading too much into eye-tracking data. For one thing, test results depend very heavily on the instructions and tasks given to test participants. You can't necessarily generalize the results to the public.

He recommended that eye-tracking data should only be used as supplemental information. The more eye-tracking studies I read, the more I'm inclined to agree.