Usability testing for multitouch devices - expert know-how on touch visualization needed
Posted: 06 June 2011 06:38 AM   [ Ignore ]
Joined  2011-05-26
Total Posts:  2
New Member

Hello everybody! :-)

I´m currently working on a bachelor thesis about usability testing for multitouch devices at the University of Applied Sciences in Mannheim, Germany.
I´m trying to find a way to performing usability tests with devices such as:
- Evoluce One Tabletop (with Win7 running)
- Android Smartphone (Nexus One)
- Apple iPhone
- Apple iPad

My approach is to use screen capture software to record the devices´ screens instead of using cameras as it is done normally.
I thought it would be nice to see what´s going on with the “application to test” without having the user´s hands/fingers covering the screen.
I also thought it would be nice to see where the user is really touching the screen. If you perform a usability test with a camera you often can´t see where the touches really went.

For the testing of mobile devices this method has also the big advantage that the user is able to use the device naturally. You don´t have to install a cam at the device or force the user to hold the device under a document cam.

Now the problems to solve are:

1. How do I get the screen into my testing application/to the recording PC? (in my case “Morae” testing-software)
2. How do I visualize the user´s touches in my recording?

For the iPhone, iPad I already solved this problems! I use a jailbroken iPhone with an App called “DisplayRecorder”. It allows you to get the screen out of the phone and additionally it has the abillity to visualize the touches.
Here is a small video to show what it looks like:
As you can see there is a white dot at the position where the finger touches the screen. This works with up to 5 touches simultaneously.

My big problem is the tabletop + Win7:
During my research I haven´t found a software/tool that allows me to visualize the finger touches in Win7! I would need a tool that provides visual feedback on top of any application so that I can record the interactions and the screen together (just like the iPhone App does). Does anyone have any idea how to do this? Or are you aware of a tool that could do this?

I´ve already written to Daniel Widgor (Ripples Project) - he suggested to write a transparent WPF application that runs on top of the application that I want to test. But that´s not working out because of the window management --> only the application on the top has the focus and gets the input when it is touched. So I would have to manipulate the native window management + inject the touches that the application on top gets into windows7…

It would be really nice if you could help me with this or at least provide some feedback - thanks in advance :-)

With the android phone I have the problem that the screen capture tools are so laggy that I can´t use any of them in a usability test. It also seems that there is no app that provides touch visuals except the one in “Dev Tools” called “Pointer Location”.
But the visuals in this app are way to big and disturbing to use them in a test - the look and feel of the application to test would be falsified!

So please let me know if you have experience with that :-)

Looking forward to your feedback,

Posted: 06 June 2011 01:21 PM   [ Ignore ]   [ # 1 ]
Joined  2007-04-08
Total Posts:  2539

If the app is opensource, you can add in that functionality by coding yourself. Otherwise, it would probably require a custom transparent layer. I’m not really sure.

However, I think you should really do two recordings. One of the hands/everything and one of the screen. If you’re only recording the screen and where they finally touch, you’re probably leaving a lot of usability information out - like if they however over objects while thinking or other expressions that could represent usability issues.


MTmini, MTbiggie, & Audiotouch creator & Community Core Vision Co-founder

Follow on:
My Blog | Facebook | Twitter | Youtube

Posted: 06 June 2011 02:59 PM   [ Ignore ]   [ # 2 ]
Joined  2011-05-26
Total Posts:  2
New Member

Thanks for the reply Seth :-)

That´s the point - I´d like to develop a generic test setup for our usability lab. I want to be able to test any multitouch application that runs on the Evoluce system (Win7) without having to customize the code of the application itself.
I thought there would already be a standalone tool that could provide the visual feedback regardless of the application I want to test.

Regarding the recording, I totally agree with you. Just recording the screen + touch visuals would mean a loss of information. My idea was to record the screen+touches (screencapture) AND the user performing his tasks (camera in the room) picture in picture.
So it would be possible to see the user using the system and otherwise have a clear view on what´s going on within the “application to test” itself in a good quaility (like where is the user really touching the screen).

For the iPhone this worked out as far as I tested it :-)

Any ideas on how to provide generic touch visuals in Win 7? I think it´s not as easy as I thought it would be ;-)