Elo guys, I’m just curious if anyone here knows of any method that can detect a rotated gesture? I’m thinking that it should be possible with vector based gestures but I’m not sure. I have some ideas that I want to try but before diving right in I’d just like to know if there’s already something out there. So if you know of any methods or have any papers to point me to, please tell me about them!

Cheers

i actually have a few ideas about gesture recognition. read http://nuigroup.com/forums/viewthread/710/ and my pdf is at http://blog.whitespaced.co.za/wp-content/uploads/2007/09/gesturerecognition_dotnet.pdf

the pdf is more aimed at my .net framework, but in theory it should be applicable to any language and platform.

Thanks for the input, but I’ve got a pretty good grasp on regular gestures (pretty much what you describe in your pdf). The problem I have is that I want to use gestures for a multitouch table, so a user should be able to stand in any direction and draw gestures. So I somehow need to figure out the rotation of the user so that I can rotate my gesture. Get what I mean?

If I draw a ‘p’ shape normally, it will look like a ‘d’ when written from the opposite direction (upside down). So instead of thinking that the gesture is a ‘d’ gesture I need to figure out that it’s actually a ‘p’ gesture draw upside-down.

oh crap. i didn’t even think about that! thanks for reminding me!

a quick fix would be to have a gesture box where a person can draw the gesture in for each side of the table. but that is bad...really bad…

you could have 4 rotations(4 sides to a table) for every gesture saved on file. then you can match the gesture made on screen to each of the 4 for every gesture made. not a great solution, but it should work.

otherwise you’ll have to somehow rotate the gesture made on screen through code, maybe for every 90 degrees you check. pretty much the same as having 4 different rotations on file.

for the p-d problem this might be a solution: http://www.bytearray.org/?p=91

I am in progress of building a table (actually, I am building the projector right now), but I have thought of a way to do this.

Gestures are stored as a set of directions relative to the direction of the last gesture segment in 45 degree increments (use -3 to 4, -3 being -135, 0 being 0, and 4 being 180). All Gestures start with a “straight” entry (0 in my case) and all subsequent entries are relative to the last segment. For example, a ‘h’ gesture would be 0,0,4,1,1,1,1 and an ‘o’ would be 0,1,1,1,1,1,1,1. All gestures are loaded into a tree with the trunk being the first straight segment with a branch for each direction in which a gesture is available. eg: for ‘h’ and ‘o’ (With another gesture (0,1,1,1,4) for demonstration purpose)

After the first stroke (Which is always considered straight), a circle appears around the blob. When the blob moves to the edge of the circle, it registers it as a new stroke, classifies it against 45 degree increments and checks it against the possible strokes in the tree. If it doesn’t match, cancel the gesture, otherwise move the tree so the new trunk is the new stroke. Repeat until a no more strokes are available in the tree, and the result is the gesture. In the case of a smaller gesture being the same as the beginning of a larger gesture (like the gestures for ‘l’ and ‘L’) the user will be given feedback that they are currently on a valid gesture, in which case they may release their finger and choose the current gesture, or continue with the larger gesture. A hinting system can also be implemented by drawing lines from the center of the circle to the outside in the directions that valid gestures exist.

This solves the ‘p-d’ problem:
p: 0,0,4,0,1,1,1,1,1
d: 0,1,1,1,1,1,0,0,4,0

Feel free to use it if you want, it is my contribution to the community.

[Edit]
fixed ‘p-d’ example.

I used a quite easy gesture system on normal touchscreens with flash before.
http://drawk.wordpress.com/2007/05/20/as3-mouse-gesture-recognition/
Is quite simple, much like jorgon described ( I think, no expert), and you can easily
tweak the code to make new gestures… maybe just for inspiration

Yes that is very similar to what I described, but it does not account for rotated gestures.

It would be a fairly easy fix to adapt it to rotated gestures though, all that would have to be done is reorient the 8 directions based on the last stroke made.

Wow, cool guys! I’ll give it a try sometime soon and I’ll post here if I’m lucky. Thanks for the help.

Hmm… if multiple people are going to be using this, should we standardize the format of the gestures so we can exchange them easily?

I have already thought of certain extensions to the system I outlined above, including gesture chains (where you could make a ‘H’ the same way you draw it with a pencil), but there is still designing to be done on it. I also thought of replaying the stroke codes into images and displaying them in menus (or possibly using them as hints by overlaying the possible ones on top of the gesture being performed)

my first thought for the format is:
[Format Version]|[Gesture Name]|[Gesture Strokes]|[Parameters]

New lines separate gestures. Each line has a version so all gestures can be stored in the same file.
Comments are signified with // anywhere on the line and continue to the end of the line.

Format Version would be ‘0.1’
Gesture Name is what the gesture is referred to : ‘gesture_o’
Gesture Strokes is the numbered code : ‘0111111’
and Parameters are any meta information about the gesture, optionally implemented (such as NoHint), separated by ‘;’.  Values for the parameters are set with an ‘=’.

Standard Parameters:
NoHint : If present, do not display the hint for this gesture.
Keyboard=[value] : If set, tells the software to treat this as keyboard input with the character [value]. Also uses keywords for special keys. Keywords are case insensitive.

Keyboard Keywords:
Enter
Space
LAlt
RAlt
LCtrl
RCtrl
LMeta : The windows key
RMeta
LShift
RShift
CapsLock
Tab
Backspace
Escape
Menu : The right click key to the right of the RMeta ( I am not sure what it is technically called)
PrintScreen
ScrollLock
Break
Insert
Home
PageUp
Delete
End
PageDown
Up
Left
Down
Right
NumLock
F1 - F24
(Yes, I just used the labels on my own keyboard)

The final | must always be there, even if [Parameters] is empty.

Example:
0.1|gesture_o|01111111|NoHint;Keyboard=o // Keyboard lowercase o

Did I forget anything? This format should be extremely easy to implement (Especially since [Parameters] are optional), and flexible enough to handle everything I can think of.