Hi,

great idea with the new discussion area Christian! I’m glad to be the first poster here I have put a short article regarding a MT-Menu system in our blog. Would be glad to discuss that here. You can find a PDF file with some rough/preliminary files at http:www.xtuio.com

Cheers,

Hey Sandor,

I took a look at your menu idea. I agree about introducing a hold as a more common input. There needs to be some sort of icon that signifies a hold similar to the x used to close a window. The simple desktop app uses the hold technique.

I think the most effective menu system would have a mouse driver built into it so that you can navigate your non MT apps. Also there should be some vision system controls that are MT enabled such as back ground subtraction. If you could simply touch a button to do this it would allow less interaction with keyboard and mice and you would not have to go back to your blob tracking app to do so.

I just realized right now that quartz composer is a standard part of x-code. I will have to check it out. I have been messing around with other programming methods to figure out which language I want to learn.

I’m weary about the idea of a hold. The main reason is because of standards. What counts as a hold and how long do you have to hold for it to be a ‘hold.’ I used a hold in my audiotouch OS to bring up a radial menu for the audioshape sequencer app. I found most users didn’t really like the hold. They didn’t like the idea of waiting and if a hold didn’t come up in the time they expected they were confused. Also, sometimes they would hold for a bit, see nothing happen and retirgger the hold for longer. I think if a hold is used for something that’s common (like launching a menu) then there has to be some visual cue to show we’re waiting for an amount of time (the hold) until we trigger the event (menu). From my experiences, I think a gesture (with movement) is a much better way to launch common tasks.

Where a hold could be more efficient is if it was combined with a pressure asignment. This is device dependent though and would require hardware that can measure pressure. So instead of a hold, it’s more of a ‘press’ that requires a pressure threshold to trigger it.

I would prefer moving the finger in zig zag (similar do drawing a Z) fashion to call up a menu or bring a system out of hibernation/standby (I tend to use zig zag finger gestures on my EEE 701 track pad until I realize I have to depress the power button to bring the system out of stand by.) An OK/Cancel dialog can appear to avoid accidental menu calls.

I like the idea of a outer ring split to contain a primary menu although I think the second menu would be more scannable if they appeared as a vertical list.

Edit: Mock up based on images on sandor’s site:

In addition to the icons in the radial menu (which are faster to access for those who know which icon is which) a list with icon and short cut titles can appear on the left (I have not synced the order of the icons on the radial menu with the icons on the left.)

Edit:
New image with Icons in sync

Changes:
-Added scroll arrow
-Added focus box to highlighted menu item on left menu list.
-Switched focus icon ring in radial menu
-Added a Back button to appear on top (in relation to user’s view point) of radial menu. If someone wants to move back, it would be the area on the ring they most likely would push (need user testing to be sure.)

Edit:

Added Rotator Handles to the Home/Jumper button.

Hey Guys,

glad that the discussion has started now

@ Jessey: yeah, there should be an option for a virtual mouse driver while you are in menu mode. Well, actually there IS a virtual mousedriver - i don’t know how familiar you are with the BBTouch development, but there’s a little App called BBTuiotest by Ben where a mousedriver is allready implemented as an option. The Xcode files for the mousedriver are far as i know in also in the xPrexxo/xNoder projects…

@ Seth: i definitly agree with the idea of a visual feedback. As for “how long” a hold is, well i would handle that just like a mouse-based traditional “double-cklick”: give the user the possibility to define the amount of time. Another idea would be to use a simple “two-finger-gesture” or the like - ie. please put two fingers in this circle for the menu. Well, that is what my QC-mockup “xBUTTON” is doing right now:

- a left klick is achieved by putting one finger inside the circle
- a double klick - well that is a double klick
- putting two fingers (blobs) inside the circle will let the menu appear (i have named that a “righ-click” whyever...)

But i think i will take some of the ideas here and will extend that mockup a bit (hopefully during the week), so that we can simply play around with some options…

@ Steve: great to see your efforts. I think there are some good points in that you are saying - also i have noticed that there is a bottleneck in my proposal - i am not sure now, how intuitive a “natural-way” with scrolling/spinning the different parts of the menu would be… May original idea was to have the three parts of the menu-system independently scrollable/spinnable. So if you would put your finger inside the upper side of the circle (ie. where the headline “SYSTEM” is) and would spin that part, you would be switching between the “fiorst-level” menu options. Putting your fingers inside the left side, would spin around the “second-level” and inside the right-side you would spin around the “third-level” options… I am not sure how clearly that is what i mean (i’m affraid that my english is not really sufficient for explaining clearly what i mean), but i think that i will try to mockup that as well in the “xBUTTON” thingie…

The idea with the scrollable list is not bad - but that is a bit too “apple-ish” for me I don’t will say that it’s bad or boring, but that’s the way apple is handling their media-applications (like apple TV or the first-gen iPods...).

Anyhow: let’s try to figure out all those scenarios In the end i think that only a user testing will give us the right answers

Cheers,

There are a few pre made gestures illustrations on:
http://www.kickerstudio.com/blog/2008/12/touchscreen-stencils/

@Seth and @Sandor attached are visual cues (from the above stencils) for differentiating when taps are just taps and when the taps visually turn into holds.

One option to use is:
2 rings appear when a finger tap gesture is about to be registered as a hold.  The ‘body’ in between the 2 rings become visually ‘dense’ with time and become completely ‘dense’ when the finger gesture is recognized as a hold.

Perhaps 3 (time measured?) states of a stationary gesture can be used:
Tap (single ring), holding (2 hollow rings) and held (2 full rings.)

@Sandor I see what you mean now by moving/spinning the system menu to reveal the 2 different levels (sets) of menu for the selected.

Perhaps instead of rotator handles, the quit and help button could replace them since they are consistent for all menus?
eg. Call out 1 could be help and call out 2 could be quit.

Yeah, I was thinking about the same visual SteveJB. One problem is having to show the circle cursor on a tap which would possibly mean having to show a cursor for all finger down events which I think is unnecessary. I still think a hold should be used minimally. While yes users can define the length, new users coming up to a system won’t know the length and holds are prone to be triggered on accident a lot if made too sort, while making them too long is somewhat annoying to have to wait. Moreso, I just think there’s two many other options to make a hold a primary one.

What if you have a given area to hold 1 finger at. Like a circle in a corner, and make a zig zag motion with the other hand to activate the menu?
That way, you dont have to be unsure if the menu will be activated, nor to triger it if you didnt intend to.
Or make a given area with an animation that have to be in an given position, like this:

First image is the corner menu activated. Never mind what it says, just an ex.

The secound image is the corner menu inactive.

And the last image is the hold and zig zag to enter the menu.

That would work well overload.

Perhaps a dialog can appear telling the user what gesture to use to activate the menu (or an additional gesture for another option)

I haven’t visually enriched the image below but it should be enough to provide an idea.

I’ve added the idea of using most of the corner to activate the system to cater to individuals who don’t have very high hand-eye co-ordination. Win 95/98 users had issues hitting the start button so XP had the start button use more of the corner it was on to be click-able.

How about instead of corners to add the activation buttons to the center of the edges as below?

That way it would be easier to know in which direction the user is facing (for displaying dialogs to the user without appearing with the text upside down or rotated the wrong way) based on which button is tapped:

P.S the U or Z gestures can be drawn anywhere on the screen, I added them to the illustrations to guide new users.

I did do something similar Steve on my first project/OS and Microsoft now puts hotspots in the corners also to bring up menu:

I essentially used 4 radial dials (mostly because my OS was meant for multiple users). When the dial was turned (you have to wait a bit to see it in the video) it would bring up a menu. You can think of the dial almost as a gesture. The dial made you draw a ‘C’ like gesture in order to launch the menu. While not the best/ideal technique it did make it so false menu launches never occured. You could essentially get rid of the dial, put a circle and make people draw a circle within the circle (make a gesture in a particular designated area). Similar idea and actually what I was aiming for but ran out of time.

I think the ideal is still to be able to bring up a menu whereever and whenever we want, similar to what Han proposes/implements.

@ 00:43 you can also see my hold/circular menu come up.

older picture before completion

@Seth

I was thinking just that, what you showed in the video. With an animation that have to be in a given position to access the menu. It makes it easy to access and hard to trigger by accident.

The circle holds with 8 way switches to activate shortcuts seems popular.

The menu at 1:38 in the above video can be combined with flick gestures referred to on http://msdn.microsoft.com/en-us/library/cc872774.aspx (or the latest shortcuts used for flicks in Win7 or Surface.)

Gesturing C’s and full circles inside the circle could actually be used to scroll through options.

Below is a mock up and a description of how the OS/App can discover the orientation/viewpoint of the table in relation to the user using corner circles from Seth’s original app and overload’s suggestion of a fixed area to fill in gestures:

In the above mock up the user has held the bottom right circle long enough to call up the menu. For cue-ing the user about holding, the red circle appears on tap, orange colors filled in between the red circle and the outer white ring and are filled in when a full hold is registered. Once the hold is registered, the above mock up gesture dialogs appear.

The user can hold the corner circles and gesture at the same time or the menu can appear for 20 seconds (or a more appropriate time interval.) Inclusion of the time interval would be best since it also to cater to individuals who prefer a stylus for most of their actions and for users who may be handicapped (increase accessibility to single armed users so they could at least use basic table functions.)

In the above mock up the user activates the circle at the bottom right and what follows below is a way for the system to assume the orientation of the the table in relation to the user and possibly if the user is right handed, left handed or a user who prefers stylus/single hand gestures.

-If they fill in gestures from the right of the table while holding the circle, the system can assume they used their left hand to hold the corner circle and the user used their right hand to perform the gesture. A UI/profile can be launched for
--A portrait OS Shell since the user will be facing the screen from the right
--Right handed user who has both hands functioning

-If they fill in gestures on the bottom while holding the circle, the system can assume they used their right hand to hold the corner circle and the user used their left hand to perform the gesture. A UI/profile can be launched for
--A landscape OS Shell can be launched since the user will be facing the screen from the bottom
--Left handed user who has both hands functioning

-If a time interval was triggered by releasing the corner circle and the gestures were drawn from the right of the table
--The system starts in a portrait mode for the OS shell
--A UI or profile catering to stylus operation, single armed users or beginners can be launched

-If a time interval was triggered by releasing the corner circle and the gestures were drawn from the bottom of the table
--The system starts in landscape mode for the OS shell
--A UI or profile catering to stylus operation, single armed users or beginners can be launched

Totally new to this forum, this thread, this site - but eagerly desiring more of this stuff!  Been reading about these system menu ideas, and had an idea for a menu call-up:  What if you placed one finger down (presumably your index finger) and then double-tapped with a second finger next to it.  IE - I touch with my index, double-tap with my middle finger.  Any sort of menu could then come up - cicular, text-based, whatever.  This is simply an idea to call up the menu, rather than the hold (which I’m not a fan of) and the zig-zag gestures (also not a fan).  It seems to me that would be hard to accomplish by mistake, would allow you to bring up a menu anywhere on screen, and is relatively simple.  Is this practical?  What are the weaknesses?  I’m not a programmer, and haven’t yet built a functional touch-table, but have spent lots of time thinking about and tinkering with this stuff.  What do you think?

I love the bumptop.com desktop in the vid (shared by SteveJB). Strangely enough I was looking at this today while I was planning some software I could write for the table that I’m building.
Its just wicked with all of those features and all with a single touch point

@SKNelson, the one finger to touch and another to double tap could work. the idea’s even better when a finger on the left hand taps and another finger on the right hand double taps. It’s explaining to new users double tapping with another finger is what they have to do to call up the menu. Come to think of it, double tapping the hot spot should suffice for calling up a menu.

@Nick, the video is actually sourced from links in an earlier thread about using a ZUI system for a multi-touch OS: http://nuigroup.com/forums/viewthread/2538/

hey guys - sorry for not posting any reply to this great thread, but i am preparing a few very important presentations - so no time to go deep enough to continue the discussion. Hope to have some time next week…

Cheers,