just thought I’d add a couple of my own ideas
The gestures themselves are not that great, but the idea I wanted to get across was to have two similar sets of gestures: one to deal with the data (objects) directly (move, scale, rotate), and the other to deal with the environment (such as the desktop) around the data (scroll, zoom, turn). The sets would be almost identical, in order to make it more intuitive, and require fewer gestures to be learned. The only difference between the sets (in my example) is the number of fingers used.

Oh, and I also added a menu gesture, kind of like the one found in the in the perspective pixel video.

Ahh great to see some discussion on this topic, I do like the gesture for the menu methinks added. Here is a video of hans, upclose..

If anything guys, I am just trying to make sense of gestures so we all can be on same page, I will update mine according to the replies so far.. and repost. Thanks for the feedback

Hey
While trying to figure out a gesture set, I came up with a few ideas on how to design them. These are not hard fast rules, but rather some rough hints to help along the way.
I hope they help, let me know what you think.

There are three standard steps to designing anything, which go for the gestures as well.
-Plan (figure out what exactly what you need to do)
-Design (come up with ideas)
-Test (try out your ideas. If they don’t work, go back to Design)

Plan

The first thing to is to define what you are actually designing for. While this may seem like a stupid question, it’s not as easy as just saying “I’m designing for a multitouch”. For example, a large wall panel will use different gestures than a smaller, monitor sized screen. Also important is to consider what you are trying to do with the application that the gesture is for.

It also helps to figure a list of all the commands that we need gestures for. You don’t actually need this, you could make up the gestures one at a time, but it helps to have an overview.

Next we need to set our priorities. Here are my own priorities in designing gestures:
-Efficient
-Streamlined
-Extendable
-Compatible with existing/standard gestures
-Simple (Both to learn and to use)

So what do we mean by all of this?

Efficient - We want the gesture to be as efficient as possible (Duh!), so making the user tap three times, while drawing a circle with two fingers, just to select an object is not a good idea for a gesture.

Streamlined - This is not quite the same as efficient (though it does play a part of it). Rather, it is how well the gesture works together with other gestures. The best way to do this is to figure out what gestures will be used together, and then design them with similar features. For example, chances are that you’re going to be doing movement, scale and rotation together, so it would make sense to use the same number of fingers for each operation. That way you don’t need to change the way you touch the screen to move from gesture to the next, making the whole interface more fluid.

Extendable - Design the gestures so that you can add more gestures later. I don’t know exactly how to do this yet, but it’s a good idea to keep this in mind.

Simple - Obviously, the simpler the better.

Design

Heres the fun part: coming up with the gestures themselves. Not too much I can help with here, the only hint I have is to look at how other people are doing it. Look at some of the videos that have been posted, or gestures that they may have come up with. Also, it might be worth looking at movies for inspiration (Minority report, anyone?)

Testing

Great! Now all that’s left is to plug your gestures in to your system, and to see how they work. Unless, of course, you don’t have working screen yet (like me), or you’re not a programmer, meaning you need someone else to code the gestures. So then what?
Well, let’s play a little game of pretend. That’s right, find a blank wall, or just use the air in front if you, and imagine your application. Then use your hands to simulate the gestures, imagining how the application is being manipulated. I know it sounds corny, but visualization can be a useful tool. This way, you can see how well the gestures actually work together.

As you test out your gestures, you’ll find what works, and what doesn’t. As you find things that don’t work, go back to the design stage and try something that’ll work better.

The last step is to document your results. The easiest way to do this is to draw diagrams like the ones that have already been posted.

Has anyone thought about using the “omit” editor’s mark as a gesture for deletion? it’s a single gesture so it’s more economical.. but having not tested any of these out yet I can’t say whether it will conflict with any other gestures.

methinks - 26 February 2007 05:22 PM

Don’t forget paint programs! (pressure = brush size)

i’d actually say pressure = brush opacity

I have a caveot against any one blob gestures. I’m thinking os integration will be faster if single blobs are reserved for legacy mouse emulation. I.E. the single blob should be a special case where the creation of a blob is a mouseclick at that location. Sort of a very primative touchscreen mouse implementation. Then multiblobs can do their own thing on top of that. I do like the fingerworks example somebody provided. All gestures are multiblob, including simple clicking.

Long story short: Gestures should always be multiblob.

ifakemyadd,

I am kinda inclined mot to try and emulate a mouse (much) at all. The thing I like most about the multitouch environment is that I dont need the classic WIMP type of environment. But then currently I am working with a small-ish screen at low res (800x600) so perhaps its just that it suits my setup.

A single blob that doesn’t “move” is a mouse click but thats about it - everything else is fair game

rich,

I’m just trying to look out for things for fast turnaround. The quicker there’s a reasonable and usable system (i.e. solution built into the os) I think the quicker interest can be generated. Keeping the single blob available for backwards compatibility, just for the time being, helps. I know for sure I’d much rather first try and get my Linux Multi Touch workling like a regular touchscreen, because at that point I have a usable system, which I know will easily be able to take the next step to the kind of implementation you are thinking of. Eventually the single blob usage will be faded out, but I just think if one is developing gesture standards that those standards should not interfere with backwards compatibility/implementation.

In that, I’ll also go along with you that the standards should not imply any legacy mouse usage either. It’s more like, I think an agnostic stance of the single blob for legacy incorporation is best at the moment.

My opinion, we need to create more than one standard. You can not design something that’s efficient for everything. That’s why I would propose that we create 3 target standards:
Large (wall panels)
standard (monitor sized)
and small (pda, etc.)

These three will use radically different gestures: large panels could include movement that uses your whole arm, the other two standards don’t have room to do that. Small interfaces on the other hand, would rely much more on single point gestures.

As for the discussion on multi-blobs, I look at it this way:
There are two types of gestures: those that manipulate something (such as moving or rotating) which I will continue to call gestures, and those that send a signal (such as drawing a symbol to call up a menu) which I refer to as glyphs.

Gestures are equivalent of the way that the mouse is used. I would agree that these should always be multi-blob.
Glyphs are the equivalent of hot-keys. These could well continue to be single-blob, as it would make sense to draw them with a single finger.

Just some thoughts.

Interesting, but I would do a rotation with two fingers.

thumb fixed and the index finger can rotate around the thumb (clock/counterclockwise).

two kinds of rotating:

2D
with the first finger you define the axis, second rotates

3D
with the first two fingers you define the axis, and the third one rotates around it.

ifakemyadd,

agreed - getting something working quickly and keeping the interest level high is very good point. The way I am heading with the input / blob detection stuff should make it fairly easy to have mouse emulation as well so I guess I get that into the bargain.

methinks, your right that the different screens will have different gestures but it would seem to me that they are all generated the same way. Putting it another way a gesture that works on a small screen could (should) work on a bigger screen as well. The reverse is not the case but that it OK. I think it would be a bad precedent to have the same type of gesture doing different things on different screens (although I doubt that whas what you were suggesting).

I agree too that the glyphs as you call them are inherently single touch type actions. They should be quick and easy (to perform and to process)

Another go at the gestures. This time there are two sets, one for navigating the environment (3D environment, or ZUI style infinite 2D surface), the other to manipulate data objects directly. These are only motions, no glyphs. I’ve tried to design them so that you can do multiple gestures at once (like rotating and scaling while moving). They are meant for a large format screen (large enough to use your whole hands).

Wow, a lot of great ideas going round.  Looks like things have simmered down as of late though.  Just to jump in I’m going to make a list of a few points that have some to mind:

1. Create standards, use universal gestures as much as possible
This means that the user shouldn’t have to relearn common gestures on different OSes and on different device types.  So I’d have to disagree about there being three different kinds of sets.  Ideally there would be one set that fits across all devices.  Not to say people can’t customize it, but their should be a standard that is cross-platform and cross-device.

2. Think “Omni-Directional”
We’re entering into an era where people will be accessing their computer on their coffee table or on their kitchen counter.  That is, people can access the interface from any side so plan on gestures needing to keep this in mind.  Because if this is the case then + = x , – = l , and V = L which has its pros and cons.  Pro - the user doesn’t have to care so much about the angle of the gestures as much as whether it has a cross, a joint, etc.  Gestures, by their nature, are not precise movements, they are approximate therefore the gesture success rate goes up.  Con - basically we lose a huge chunk of our gestural alphabet.  I think this will eventaully change as we’re able to detect the angle the user is at with respect to the screen so an X will be an X and a + will be a + no matter what angle you’re at.

3. Have more than one way to skin a cat
One thing I love about gestures is that it gives us a whole new way of expressing ourselves while we work.  And not only that, I think we can design them so that we can create multiple routes to the same end, maybe an intuitive way and an efficient way.  I can imagine myself wanting to shrink a window but using my whole hand to do it, contracting my fingers as if grabbing and squeeezing it, maybe I’m pissed after I just read an article or something.  But Just the same, I could have used two fingers.  But when we’re in any state of intense emotion (laughing, anger, lethargic) our ability to do precise movements decreases so we can make gestures that pre-empt that.

4. The almighty pointer finger
It was suggested that perhaps gestures are done with more than one finger but I think intuitively we always think to use one.  I can imagine using a second as a sort of “right-click” in some cases.  I don’t know though, I just think the most intuitive thing is to prod with our index finger at something

5. Sticky Menus (More interface than gestures)
In a Han video posted you could see a women struggling to be able to use the revealed menu well because it would dissappear for whatever reason.  I’d at least have some sort of timer where it will at least stick for a few seconds so that the user can assess their options.  I’m sure for while we’ll have to deal with the fact that with a traditional mouse you can hover, pan, and go through lists all without clicking.  But because the computer can’t “see” the mouse until it’s already touched the screen we have some interesting usability issues.

6. Context Specific Gestures
Do the same gestures work in all cases?  If I grab two icons with two fingers and pinch them together will the screen zoom out?  I think a large part of defining gestures is defining their context and the rules that govern their activation.

7. Borders
I think a huge part of finding out how to do this right is thinking of the most logical ways in which people will be handling files, sorting, and so on.  I personally don’t think the idea of a window, folder, or album appearing on the screen will go away just because we have multitouch.  With that said, I think a lot of gesture solutions could relate to how one touches the window border.  Something like you first touch the border, as if you’re pinning it down, and then take your other finger and slide it anywhere on the screen which would cause the window to resize in tandem.  Maybe you have to grab the virtual border of an object first before you manipulate it, as if it gets highlighted in some fashion.

7. User Test
This kind goes without saying but I think anything we develop should be considered “delta” without a real usability test done with users of a variety of ages and backgrounds.

End notes: I think the one thing I learned from writing this is that gestures when applied to an active and functional desktop are actually quite complex to design.  It’s easy to say WHAT they can do but the real tricky part is the WHEN.

Has anybody got a range of gestures working with touchlib via flash?
I’d love to see a demonstration if anybody has.
I notice on the SVN there is a gesture dictionary and associated classes…