I have an idea for creating an educational game using fiducial markers.  Here’s the basic idea:

The kids would have blocks with numbers/math symbols on one side and markers on the other.  When they put the markers on the table, they could put blocks down and see the answer.  So if they put a 5 block and a 3 block, with a + block in the middle and an = sign on the end - the answer is 8.  But if the child put a 7 block down after the equal sign, the program would tell them they’re off or whatever.  This would give a child a tactile method of experimenting with mathematics.  I could see the same idea applied to spelling games and the like.

What do you think?

Thanks!
-Philip

Interresting !

But you don’t need a multitouch table to do this, no IR led or laser…
A simple camera seeing the table and fiducials by down should work.

Cheer’s.
(my 2 cents)

I think that the idea is nice, but may be not enough for a whole Gsoc ?

I think this has potential and has great intent, but I agree with txprog; is this really something you can spend 3 months working on and feel is deserving of $5000? You should keep the idea, but expand on it.

Though zebulon is right, using an MT surface for this may be overkill, you don’t HAVE to use an MT setup. Maybe just try a table-top interactive table (put webcam and project above it looking down).

I understand what you guys are saying.  In my opinion, the project would need to be multitouch in order to be a “natural user interface.” I think the disjoint between moving and placing the blocks on a table, then looking up one a screen would be too much for some children, and isn’t really the point of the exercise.  I think it would be much more natural to interface with the table itself rather than another display.  I think this would be especially true with children with disabilities (I work with an a couple autistic children periodically).  So that’s my reasoning for the multitouch.  No, it wouldn’t have to be multi-touch, but i’m not sure many projects need to be multi-touch - but many work better and are more natural that way.

On the scale of the project, it could be scaled up to include a number of different math related games, quizzes, etc.  I could also do some with letters as well - maybe an arrange the alphabet game or something of the sort.  I think the idea of the project is to give children a way to learn that’s fun, and doesn’t have to be adult assisted - in other words, the kids will get feedback from the games that help them figure out how to get the right answer, etc.

What do you think the project would need to include to warrant acception?

BTW - thanks for your feedback.

No, I was talking about displays that augment reality but don’t require the hardware complexity of a full MT table (which would mean that teachers of special needs kids don’t need to invest in a big MT table, they could do it with a single projector and a camera). Just point a projector and a off the shelf webcam with no modifications onto a surface and use software to do fiduciary marker input.

See this image for a basic reference.
Another example image

I’m just thinking that in order to be actually beneficial to special needs kids this system should be available to special needs kids and right now MT table are just not feasible for schools or consumers. But a lot of schools have projectors and webcams can be picked up very cheap. Even more awesome is the idea that you can use the floor as a surface, then the kids could play on the floor which is about as natural as you can possibly get!

Whichever way you choose to go there needs to be quite a bit more info for your GSoC proposal, take a look at some of the other proposals and try to at least write up a PDF with as much information as you can; the mentors and members around here are the kinds of people that want as much technical information as possible.

Ah, I understand what you’re saying.  That makes more sense.  Thanks for the feedback!  Doing it that way would make it feasible to distribute over the web… to any teachers interested (for free), which is a cool idea.

I’ll def. include more information in the proposal and format it nice and neat, etc.

Thanks for your help!  Any other feedback is greatly appreciated!

After thinking about this more, if you’re taking the data in from the top, and the user is looking on from the top, you can’t use (traditional) fiducials - you’ll have to use some sort of software that can recognize letters and numbers… I’ve played some with that sort of stuff in flash, but I don’t know of a program that would take in that kind of data and output tuio events.  Any ideas?  That’s partly why I thought I table would work better… markers on one side, numbers, letters, etc on the other…

Hello,

This is a fun idea ! I think projector and camera on top with children playing on the floor is a good idea too.
May be a solution would be to build a new module for CCV doing form detection (letters, numbers, signs) and associates an Id to each form. Problem to solve is that a letter (or a number) can appear several times (different objects but same letter or number). You should be able to manage these two things : Id of physical object and semantic associated. Another thing to do will be to construct sentences (or formulaes) from position (proximity) of letters or numbers and signs.
Using a table with tags seems easier since tools are already built. On the other hand, it will be much more expansive to use it.

But, if you can do that, possibilities are great ! I imagine an app for small children learning to write. For example, when you put letters “S”, “H”, “E”, “E”, “P” together, a digital sheep appears, walking on the floor. Then if you add letters “W”, “O”, “L”, “F” a digital wolf appears and starts to run after the sheep. Very fun way to learn ! :D
In fact your idea is very close to the example showed at the beginning of Microsoft Vision 2019 video (http://vimeo.com/4501525).

Laurent

Actually I don’t understand why a letter on the floor would be any different than a traditional fiduciary marker; as long as your software is set up to recognize the shape then it doesn’t care if it looks like what we think a marker should look like. It just sees collections of pixels and does pattern matching on those pixels; a letter is a pattern no different than a fiduciary barcode style marker except that we have assigned special lexical meaning to them. It should be very simple to do.

h4t,

You’re surely right.
My remark was about the fact that Philip is a designer and not a developer (Am I right ?). I’m not sure his goal for GSoC is developing a new vision module or customizing an existing vision system.
May be he could partner with another student (developper) who would work on technical part of system.

Laurent