Working Blobtracking with JMyron. testers wanted, source code available. 
Posted: 16 July 2008 07:35 AM   [ Ignore ]
New Member
Avatar
Rank
Total Posts:  54
Joined  2008-07-15

Hello everyone

I have made this processing sketch with my own fixed version of JMyron (blob-center tracking binding was borked in JMyron)
generative.PNG

it is only a very simple example, but it shows how easy this can be done in processing with my fix of JMyron wink
It looks for blobs and finds their centers, then uses the combined data of the blobs to generate a sound
(up/down = pitch, left/right = detune)

i would be happy if some of you would like to test it out and tell me what you think.

in the attached.rar file, you will find executables for Windows, Linux and Mac (although i am not sure wether or not it will work in mac and linux, use at your own risk!)
along with source code in .java and processing .pde sketch, accompanied by the libraries i have used.

quick manual :
When the application starts, hit the webcam settings button, and tune the saturation all the way down, so your picture is black and white.
the two sliders on the left controll the colour-blob tracking. (yes, these are mouse operated, not touch operated!)
The top slider controlls the grayscale colour that the application will try to track, and the bottom slider controlls the tracking tolerance.
You might have to play with these a little to find a setting that will work for you.

if you use FTIR or comparable, track for light colours, if you use a MTMini, track for dark colours.

Want to know more about the JMyron errors?
http://nuigroup.com/forums/viewthread/1818/

Please let me know how well it works. smile

File Attachments
Generative_tracking.rar  (File Size: 1117KB - Downloads: 81)
Profile
 
 
Posted: 16 July 2008 08:02 AM   [ Ignore ]   [ # 1 ]
New Member
Rank
Total Posts:  59
Joined  2008-05-28

Cool, as soon as I have time I will give it a try!!!

 Signature 

Our setup: Our big ass table

Profile
 
 
Posted: 17 July 2008 02:38 AM   [ Ignore ]   [ # 2 ]
New Member
Rank
Total Posts:  59
Joined  2008-05-28
Tunabreath - 16 July 2008 08:02 AM

Cool, as soon as I have time I will give it a try!!!

Played with JMyron last night, works very good. I do not understand everything yet, but I was able to implement a simple the background remove filter and detect touches. I have a day of tomorrow so I will play with it some more.

 Signature 

Our setup: Our big ass table

Profile
 
 
Posted: 17 July 2008 03:41 AM   [ Ignore ]   [ # 3 ]
New Member
Avatar
Rank
Total Posts:  54
Joined  2008-07-15

Awesome, Tunabreath! :D
i’m very excited about this, i’m hoping it will bring rapid development and simple experminting a lot closer for everyone.
i have attached a very simple version of the processing sketch, which is more well-commented, and contains less non-necesary code.
(just the blob detection and drawing the location and outlines of the blobs)

You might want to use this for a starting point for your own sketches, rather than the one i posted earlyer.
Make sure you have the JMyron library with my modified JMyron.jar (get it from http://nuigroup.com/forums/viewthread/1818/ )
And that you have the controlP5 library
Both are available from http://www.processing.org

happy experimenting smile

import JMyron.*;
import controlP5.*;

//define the JMyron object
JMyron m;

//define a controlp5 opbject
ControlP5 controlP5;
 
 
 
//JMyron's blobtracking example
 //Modified and extended for use with an FTIR or similar
 
 //***************IMPORTANT, READ THIS!************************************


 //if you haven't downloaded JMyron/webcamextra yet
 // you can find it at www.proce55ing.org

 //be sure to use my modified version of JMyron.jar
 //download it from http://nuigroup.com/forums/viewthread/1818/
 //and copy it over the original JMyron.jar file in your processing/libraries folder
 
 
 //notes:
 //ideally, one would blur the incoming image slightly to eliminate noise.
 //this should still be implemented.
 
 
   //preset values
  
int tracecolor 255;
  
int tracetolerance 40;
 
 
void setup(){
  
//define width and height separately
  //we need them for the jmyron object and
  //we might do something with these later on.
  
int w 320;
  
int h 240;
  
  
frameRate(24);
  
  
size(w,h);
  
  
//initialise the JMyron object
  
= new JMyron();
  
//start it with the width and height of the sketch
  
m.start(w,h);
  
//turn glob detection on
  
m.findGlobs(1);
  
  
//initialise controlP5
  
controlP5 = new ControlP5(this);
  
  
//make a slider for which grayscale colour to track
  
controlP5.addSlider("Track_colour",0,255,255,10,10,10,100).setId(1);
  
 
//make a slider for which tolerance to use for tracking
  
controlP5.addSlider("Track_tolerance",0,255,40,10,126,10,100).setId(2);
  
  
//make a button to open the webcam settings dialog with
  
controlP5.addButton("Webcam settings",10,100,215,80,20).setId(3);
  
}


/**
*the draw event is called at the speed defined by frameRate(int fps);
**/
void draw(){
  
//clear the sketch by drawing a white background
  //(is only needed if you turn the camera view display off)
  
background (255,255,255,255);
  
  
//set the color we want to track
  //(dark gray)
  //           r g b tolerance
  
m.trackColor(tracecolor,tracecolor,tracecolor,tracetolerance);

  
//have JMyron fetch new data from the camera
  
m.update();
  
  
//get the current image
  //and store it into an array of integers
  
int[] img m.image();
  
  
//define floats for red green and blue channels
  
float ir,ig,ib;
  
  
//first draw the camera view onto the screen
  //comment this piece of code if you don't need to display the camera output
  //ebcam image-drawing code
  //----------------------------------------------------------
  
  
loadPixels();
  
        
//web cam image drawing starts here
  

  //itterate through every pixel on the screen
    
for(int i=0;i<width*height;i++){
      
/*
        //this code inverts all pixels by subtracting their color value from the maximum
        //this is not necessary, but allows you to see better where your globs should occur
        //on your FTIR screen.
        //for some reason, human eyes distinguish white on black better than black on white.
        ir = 255-red(img[i]);
        ig = 255-green(img[i]);
        ib = 255-blue(img[i]);
        
        //add the inverted pixel to an array
        pixels[i] = color(ir,ig,ib);
        */
        
        
        //you can invert the displayed webcam colour by uncommenting the above code and commenting this line)
        
pixels[i] img[i];
   
        
    
}
    
      updatePixels
();
          

  
//webcam image-drawing code ends here
  //----------------------------------------------------------
  
 
  
  
  //**********Enmerate and draw BLOB-CENTERS***********//
  //initiate a two dimensional array that will hold the blob centers x and y positions
  
int[][] a;

  
//fetch them
  
m.globCenters();
  
  
//draw them
  
stroke(255,128,128);
    
fill(128,0,128,128);
  
  
//for every blob center  
  
print("i see                    " a.length " blobs\n");

  
//itterate through all blobs
  
for(int i=0;i<a.length;i++){
    
    
//output to the console which blob this is
    
print("blob " " : ");
    
    
//if the blob is not exactly at 0 x 0
    //(jmyron may have this quirk, but it only rarely occurs)
    
if(a[i][1] != && a[i][0] != 0){

      
//output the x and y coords of the blob to the console
      
print(a[i][0] " X " a[i][1] "\n");
    
      
//draw the exact point of the blob center
      
point(a[i][0],a[i][1]);
    
      
//draw an ellipse around the point to make it easyer to see
      //you might also use it as bounds for tracking objects.
      
      
ellipse(a[i][0],a[i][1],20,20);   
      
    
else {
      
//(if the blob center was at 0x0)
      //output to the console that this blob was skipped
      
print ("skipped. (0x0) \n");
    
}
  }
  
  
// you might want to do more with the blob-centers...
  
  //**********************************//
 


  //draw bounding boxes of blobs
  //(commented because it is merely illustrative)
  
  // DO NOT USE THIS, JMYRON STILL HAS BUGS
  // IN THE BOUNDING BOX ENUMERATION!
  
  /*
  a = m.globBoxes();
  stroke(255,0,0);
  for(int i=0;i<a.length;i++){
    int[] b = a[i];
    rect(b[0], b[1], b[2], b[3]);
  }
  */


  //draw edge pixels of blobs (this and the next chunks of code are processor-hogs)
  //fetch all glob pixels from JMyron
  
int list[][][] m.globPixels();
  
  
//set the stroke colour and weight
  
stroke(10,255,10);
  
strokeWeight(2);
  
  
//itterate through all glob pixels
  
for(int i=0;i<list.length;i++){
    
//fetch a "list" of pixels for each glob
    
int[][] pixellist list[i];
    
//if the list is not empty
    
if(pixellist!=null){
      
//draw a vertex that runs through every single pixel in the "pixel list"
      
beginShape(POINTS);
      for(
int j=0;j<pixellist.length;j++){    
        vertex
pixellist[j][0]  ,  pixellist[j][1] );
       
// print( pixellist[j][0]  +" " +  pixellist[j][1] );
      
}
      endShape
();
     
}
  }
  
  
//return the stroke weight back to 1
  
strokeWeight(1);  
  
  
  
  
//the following lines of codes draw quads
  //i have commented them, because they are processor intensive
  //and i can't think of any use for them yet.
  
/*
  //draw quads - like bounding box, but a 4-pointed polygon.
  (commented for being a processor-hog and i'm not finding anything interested to do with them)
  a = m.globQuads(50,51);
  stroke(0,0,100);
  for(int i=0;i<a.length;i++){
    int[] b = a[i];
    quad(b[0], b[1],
         b[2], b[3],
         b[4], b[5],
         b[6], b[7]);
  }

*/

}


/**
*control events for the P5 interface elements.
**/
void controlEvent(ControlEvent theEvent{
  
//if a control event is received from the P5 controller
  
println("got a control event from controller with id "+theEvent.controller().id());
  
  
//check from what controller it came
  //by looking for the ID element
  
switch(theEvent.controller().id()) {
    
    
//if the ID element is number 1 (the tracing color slider)
    
case(1):
    
//set the tracecolor to the value from the slider
    
tracecolor = (int)(theEvent.controller().value());
    print(
"tracecolor changed to " tracecolor "\n");
    
//end the switch-case
    
break;
    
    case(
2):
    
tracetolerance = (int)(theEvent.controller().value());
    print(
"tracetolerance changed to " tracetolerance "\n");
    break;  
    
    case(
3):
    
//show the JMyron settings dialog
    
m.settings();
    break;  
  
}
}



/**
*if the sketch stops
**/
public void stop(){
  
//stop the jmyron object to release the camera
  
m.stop();
  
//then stop the runtime.
  
super.stop();
}

File Attachments
Coffee_tracking.pde  (File Size: 8KB - Downloads: 56)
Profile
 
 
Posted: 17 July 2008 04:18 PM   [ Ignore ]   [ # 4 ]
New Member
Rank
Total Posts:  59
Joined  2008-05-28

Your code works fine. I have to do a background remove before bobtraking works because the IR leds create 2 big blobs on each side of my FTIR.

Somehow I had a previous version of your example with an error.

recklessness - 17 July 2008 03:41 AM

//draw an ellipse around the point to make it easyer to see
      //you might also use it as bounds for tracking objects.
      
      ellipse(a[i][0],a[i][1],20,20);  //was ellipse(p[0], p[1], 20, 20);

Some ideas for improvement:

- Background remove filter
- Colorpicker for selecting tracking color
- blur filter

 Signature 

Our setup: Our big ass table

Profile
 
 
Posted: 17 July 2008 04:22 PM   [ Ignore ]   [ # 5 ]
New Member
Avatar
Rank
Total Posts:  54
Joined  2008-07-15
Tunabreath - 17 July 2008 04:18 PM

Your code works fine. I have to do a background remove before bobtraking works because the IR leds create 2 big blobs on each side of my FTIR.

Somehow I had a previous version of your example with an error.

recklessness - 17 July 2008 03:41 AM

//draw an ellipse around the point to make it easyer to see
      //you might also use it as bounds for tracking objects.
      
      ellipse(a[i][0],a[i][1],20,20);  //was ellipse(p[0], p[1], 20, 20);

Some ideas for improvement:

- Background remove filter
- Colorpicker for selecting tracking color
- blur filter

thanks for the heads up, i noticed my little error and fixed it allready wink

i’m currently working on a colorpicker.
background removal filter and blur filter are going to take a little bit longer, since i’m still a bit of a noob when it comes to those things rasberry

Profile
 
 
Posted: 17 July 2008 04:37 PM   [ Ignore ]   [ # 6 ]
New Member
Avatar
Rank
Total Posts:  54
Joined  2008-07-15

running fullscreen was also problematic with processing’s native drawing functions
so i made an example that uses openGL for rendering.
it opens fullscreen.

it also has two squares that you should be able to move with your fingers
(they snap to blob centers and move along with them)

please keep in mind that i whipped this code up very quickly, and that it’s probably messy and inefficient wink

import processing.opengl.*;
import JMyron.*;
import controlP5.*;

//define the JMyron object
JMyron m;

//define a controlp5 opbject
ControlP5 controlP5;
 
 
 
//JMyron's blobtracking example
 //full screen with openGL
 //Modified and extended for use with an FTIR or similar
 
 //***************IMPORTANT, READ THIS!************************************


 //if you haven't downloaded JMyron/webcamextra yet
 // you can find it at www.proce55ing.org

 //be sure to use my modified version of JMyron.jar
 //download it from http://nuigroup.com/forums/viewthread/1818/
 //and copy it over the original JMyron.jar file in your processing/libraries folder
 
 
 //notes:
 //ideally, one would blur the incoming image slightly to eliminate noise.
 //this should still be implemented.
 
 
   //preset values
  
int tracecolor 255;
  
int tracetolerance 40;
  
  
 
  
//these are movable objects, see the MUIObject class further down
  
MIUObject test = new MIUObject(250,250,150,150);
  
MIUObject testa = new MIUObject(500,500,250,250);

 
 
void setup(){
  
//define width and height separately
  //we need them for the jmyron object and
  //we might do something with these later on.
  
int w screen.width;
  
int h screen.height;
  
  
frameRate(24);
  
  
size(w,hOPENGL);
  
  
//initialise the JMyron object
  
= new JMyron();
  
//start it with the camera's native w and h
  
m.start(320,240);
  
//turn glob detection on
  
m.findGlobs(1);
  
  
//initialise controlP5
  
controlP5 = new ControlP5(this);
  
  
//make a slider for which grayscale colour to track
  
controlP5.addSlider("Track_colour",0,255,255,10,10,10,200).setId(1);
  
 
//make a slider for which tolerance to use for tracking
  
controlP5.addSlider("Track_tolerance",0,255,40,10,226,10,200).setId(2);
  
  
//make a button to open the webcam settings dialog with
  
controlP5.addButton("Webcam settings",10,100,215,80,20).setId(3);
  
}


/**
*the draw event is called at the speed defined by frameRate(int fps);
**/
void draw(){
  

  
  
//clear the sketch by drawing a white background
  //(is only needed if you turn the camera view display off)
  
background (0,0,0,0);


  
  
//set the color we want to track
  //(dark gray)
  //           r g b tolerance
  
m.trackColor(tracecolor,tracecolor,tracecolor,tracetolerance);

  
//have JMyron fetch new data from the camera
  
m.update();
  
  
//get the current image
  //and store it into an array of integers
  
int[] img m.image();
  
  
//define floats for red green and blue channels
  
float ir,ig,ib;
 
  
  
  
//**********Enmerate and draw BLOB-CENTERS***********//
  //initiate a two dimensional array that will hold the blob centers x and y positions
  
int[][] a;

  
//fetch them
  
m.globCenters();
  
  
test.update(a);
  
testa.update(a);
  
  
//draw them
  
stroke(255,128,128);
    
fill(128,0,128,128);
  
  
//for every blob center  
  
print("i see                    " a.length " blobs\n");

  
//itterate through all blobs
  
for(int i=0;i<a.length;i++){
    
    
//output to the console which blob this is
    
print("blob " " : ");
    
    
//if the blob is not exactly at 0 x 0
    //(jmyron may have this quirk, but it only rarely occurs)
    
if(a[i][1] != && a[i][0] != 0){

      
//output camera's x and y coords of the blob to the console
      
print(a[i][0] " X " a[i][1] "\n");
      
      
//now we need to cast these to the size of the entire screen....
      
float blobx a[i][0];
      
float bloby a[i][1];
      
      
blobx = (blobx/320)*screen.width;
      
bloby = (bloby/240)*screen.height;
    
      
//draw the exact point of the blob center
      
point(blobx,bloby);
    
      
//draw an ellipse around the point to make it easyer to see
      //you might also use it as bounds for tracking objects.
      
      
ellipse(blobx,bloby,20,20);   
      
    
else {
      
//(if the blob center was at 0x0)
      //output to the console that this blob was skipped
      
print ("skipped. (0x0) \n");
    
}
  }
  
  
// you might want to do more with the blob-centers...
  
  //**********************************//
 


  //draw edge pixels of blobs (this and the next chunks of code are processor-hogs)
  //fetch all glob pixels from JMyron
  
int list[][][] m.globPixels();
  
  
  
//itterate through all glob pixels
  
for(int i=0;i<list.length;i++){
    
//fetch a "list" of pixels for each glob
    
int[][] pixellist list[i];
    
//if the list is not empty
    
if(pixellist!=null){
      
//draw a vertex that runs through every other pixel in the "pixel list"
      
beginShape();
      for(
int j=0;j<pixellist.length;j++){   
        
        float vertx 
pixellist[j][0];
        
float verty pixellist[j][1];
        
        
vertx = (vertx 320) * screen.width;
        
verty = (verty 240) * screen.height
        
        
vertexvertx  ,  verty );
       
// print( pixellist[j][0]  +" " +  pixellist[j][1] );
      
}
      endShape
();
     
}
  }

  
  
  
  
//the following lines of codes draw quads
  //i have commented them, because they are processor intensive
  //and i can't think of any use for them yet.
  
/*
  //draw quads - like bounding box, but a 4-pointed polygon.
  (commented for being a processor-hog and i'm not finding anything interested to do with them)
  a = m.globQuads(50,51);
  stroke(0,0,100);
  for(int i=0;i<a.length;i++){
    int[] b = a[i];
    quad(b[0], b[1],
         b[2], b[3],
         b[4], b[5],
         b[6], b[7]);
  }

*/


}


/**
*control events for the P5 interface elements.
**/
void controlEvent(ControlEvent theEvent{
  
//if a control event is received from the P5 controller
  
println("got a control event from controller with id "+theEvent.controller().id());
  
  
//check from what controller it came
  //by looking for the ID element
  
switch(theEvent.controller().id()) {
    
    
//if the ID element is number 1 (the tracing color slider)
    
case(1):
    
//set the tracecolor to the value from the slider
    
tracecolor = (int)(theEvent.controller().value());
    print(
"tracecolor changed to " tracecolor "\n");
    
//end the switch-case
    
break;
    
    case(
2):
    
tracetolerance = (int)(theEvent.controller().value());
    print(
"tracetolerance changed to " tracetolerance "\n");
    break;  
    
    case(
3):
    
//show the JMyron settings dialog
    
m.settings();
    break;  
  
}
}






/**
* this is an example object
* that will draw a square at its location
* and that will 'snap to' and move along with any blob center that hits it.
**/
class MIUObject{
  float x
,  y,  w,  h;
  
  
MIUObject(float xposfloat yposfloat wdfloat ht){
    x 
xpos;
    
ypos;
    
wd;