jump to navigation

Fixed a PSI Parser Issue on Chrome June 27, 2011

Posted by Andor Saga in Open Source, webgl, Arius3D, XB PointStream, point cloud.
add a comment

I spent most of my weekend hammering out a major issue with XB PointStream which the folks at Arius3D pointed out.

The problem was WebGL would not render point clouds on Chrome/Chromium or Webkit when I tried to stream a PSI file over a network. However, after the page loaded and I hit refresh, everything worked.

Additionally, there were no issues on Firefox and when I streamed PSI files locally, all browsers rendered the point cloud without a problem.

I started debugging by checking drawArrays() and getError(), both of which had no issues. I kept trying to look for reasons why the point cloud would not render, but I couldn’t find any evidence of rendering problems.

The Color and the Shape

I made my first breakthrough when I changed the background color. That’s when I saw a single point in the middle of the browser window. I called pointSize() to better see the what it was. It turned out all the points appeared to be clustered together at (0,0,0) with a single black point occluding all the others. This is why it appeared no rendering was taking place. All the points were small, covered by a black point which blended in with the background color.

XHR Browser Differences

This helped me better understand the real issue. I continued debugging, trying to figure out why the vertices were all at the origin. After hours of digging through the PSI parser, I found the crux of the bug. I was reminded there are differences between how Firefox and how Chrome/Chromium/Webkit XHR files. Firefox makes fewer requests but reads larger portions of data. The other browsers make more requests, but read smaller portions of data.

I learned the PSI parser assumed it would read tags such as <Min> and <Max> (which contain important bounding box data) in the first XHR request. These tags do appear near the beginning of PSI files, but assuming we’ll have read that far in the first request just won’t work. In that first request, firstLoad() is called which sets up necessary variables used when parsing the vertices. If <Min> and <Max> aren’t encountered at that time, certain variables aren’t set and the vertices eventually get multiplied by zero placing them at the origin.

F5

So why did refreshing work? Since the file was sitting in cache, it appeared to the PSI parser to download in one XHR request. This allowed it to read those crucial Min and Max tags and load properly.

Fixing and Filing

I added extra logic which forces the PSI parser to wait until necessary tags are read before attempting to parse any vertices. While I added the fix I noticed the parser is in some desperate need of cleaning and restructuring, so I’ll need to file a bug on Lighthouse for that.

Lessons Learned

The core of this entire ordeal boils down to lack of testing. Before doing releases, I run all demos and ref tests—locally. This is why I didn’t catch the bug and why it slipped into the 0.7 release. Since then I uploaded Sundae.js (the ref tester) to a server so I can run all tests over a network as well as locally.

The Demo

If you want to see the running demo, it’s here.

Advertisements

Defenestrating WebGL Shader Concatenation June 23, 2011

Posted by Andor Saga in Open Source, JavaScript, webgl, XB PointStream, point cloud.
6 comments

When I began writing the WebGL shaders for XB PointStream, I placed the vertex and fragment shaders together in their own .js file, separate from the library. In each file I declared the two necessary variables. So, for the cartoon shader it looked something like this:

// Fragment shader ...
var cartoonFrag = 
"#ifdef GL_ES\n" +
"  precision highp float;\n" +
"#endif\n" +

"varying vec4 frontColor;" +
"void main(void){" +
"  gl_FragColor = frontColor;" +
"}";

// Vertex shader ...
var cartoonVert = "..."

If users wanted to render their point clouds with a cartoon effect, they would include the .js resource in their HTML page and tell XB PointStream to create a program object:

// ps is an instance of XB PointStream
progObj = ps.createProgram(cartoonVert, cartoonFrag);

The problem with this approach is the GLSL code is very difficult to read and maintain. Yesterday I finally changed this. Instead of users including the shader resources, I decided users would call a function and pass in the path of the shader file. The function would then XHR the file and return a string with the file’s contents. I started with defining an interface:

var vertSrc = ps.getShaderStr("shaders/cartoon.vs");
var fragSrc = ps.getShaderStr("shaders/cartoon.fs");
cartoonProg = ps.createProgram(vertSrc, fragSrc);

I decided that would be simple enough. I then implemented the getShaderStr function:

this.getShaderStr = function(path){      
  var XHR = new XMLHttpRequest();
  XHR.open("GET", path, false);
  
  if(XHR.overrideMimeType){
    XHR.overrideMimeType("text/plain");
  }
  
  try{
    XHR.send(null);
  }catch(e){
    this.println('Error reading file "' + path + '"');
  }
  
  return XHR.responseText;
};

You’ll notice I made the request synchronous. I did this for a couple of reasons. First, it keeps things simple on the user’s end. They aren’t forced to create callbacks and figure out when they can begin setting shader uniforms. Second, the data sets I’m dealing with (3MB – 40 MB) significantly outweigh a tiny request of 118 bytes. If any performance improvements are to be made in the library, they won’t be made by asynchronously XHR’ing shader code. However, I’m still open to suggestions. Leave a comment if you think this can be done more elegantly.

With those changes:

// cartoon.fs file
#ifdef GL_ES
  precision highp float;
#endif

varying vec4 frontColor;
void main(void){
  gl_FragColor = frontColor;
}

Whew! Much, much cleaner! Notice I have not only removed the obtrusive quotations marks and plus signs, but I’ve also rid the fragment shader of newline characters for the necessary preprocessor definitions. Although this was a relatively small change in terms of the interface, it’s a huge win for the library.

A PLY Point Cloud Parser for XB PointStream June 13, 2011

Posted by Andor Saga in Open Source, webgl, XB PointStream, point cloud.
3 comments

Check me out in 3D!

Last week I was trying to research some PSI file format issues for XB PointStream when I hit a small snag. I decided to switch gears and spend the day implementing a PLY parser for the library. I’ve had a few requests for this and figured it shouldn’t take more than a day.

While investigating the file format, I remembered I had acquired a PLY file at OCE 2011 from the guys at XYZ RGB. They were doing 3D scans, so I had them take a 3D scan of me. They provided me with two files, one PLY and one OBJ.

The PLY file was a polygonal mesh, but I was able to convert it to a point cloud easily by simply changing the header information.

ply
format ascii 1.0
comment VCGLIB generated
element vertex 91020
property float x
property float y
property float z
property float nx
property float ny
property float nz
property uchar red
property uchar green
property uchar blue
property uchar alpha
element face 180810
property list uchar int vertex_indices
end_header 

I changed the element face definition to 0 and I removed all the polygonal face lines which left me with only vertex data. I confirmed this was a valid PLY point cloud by successfully opening the file in MeslLab.

At that point I spent the day writing up the parser and debugging.

So given the library and parser, how would you actually use it? Here are some simple steps:

  1. Get the latest version of XB PointStream
  2. Get the PLY parser
  3. Include the parser resource in your HTML file
  4. Register the parser after setup: ps.registerParser(“ply”, PLYParser);

Here are the results. E-mail me if run into any issues. I hope this is useful to the point cloud community! (:

XB PointStream 0.7 Released June 7, 2011

Posted by Andor Saga in Open Source, webgl, XB PointStream, point cloud.
1 comment so far

It’s been a while since our last release, so I decided to package up all the issues resolved up until now and finally ship a new version. Because of this, the release is smaller and doesn’t have everything I wanted to include in 0.7. On the other hand, shipping smaller more frequent releases is the way to go in open source.

Start All the Downloading!

To get our latest release:

Change Log

Some of the major changes included in this release:

Questions and Comments

I’m getting more and more e-mails as the library matures. It’s exciting to see there’s a need for this kind of work, so please don’t hesitate to send me questions and comments. My e-mail address can be found on my Github profile page. Hope to hear from you! (:

Real-time debugging in Processing June 4, 2011

Posted by Andor Saga in Game Development, JavaScript, Open Source, Processing, Processing.js.
4 comments

Yesterday I saw one of my colleagues working on a real-time interactive graphical sketch in Processing. I noticed he was using Processing’s println() function to debug. println() is great, but not when the state of sprites are changing per frame.

I think a better solution is to develop a simple class which can handle frequent state changes and overlay those states directly on top of the visualization or game, sort of like a HUD. This is what I usually create when working on a larger Processing or Processing.js game. It only takes a few minutes to write up a simple but extremely useful debugger.

Let’s start with an interface:

cDebugger gDebugger;
boolean gIsDebugOn = true;
int gFontSize = 16;

void setup(){
  size(400, 400);

  // We'll obviously need to allow changing the text color.
  gDebugger = new cDebugger();
  gDebugger.setColor(color(100, 255, 100));
}

void draw(){
  background(0);

  // Update the sprite states.
  // ...

  // In every frame, we'll tell the debugger the current state
  // of some variables we think are important.
  gDebugger.add("FPS: " + frameRate);
  gDebugger.add("mouse: [" + mouseX + "," + mouseY + "]");
  gDebugger.add("last key: " + key);

  // Draw world, sprites, etc.
  // ...

  // Now render the states on top of everything.
  gDebugger.draw();
}

void keyPressed(){

  // We should be able to toggle the debugger so
  // it doesn't consume resources.
  if(key == 'd'){
    gIsDebugOn = !gIsDebugOn;
    gDebugger.setOn(gIsDebugOn);
  }

  // The debugging lines can add up quickly.
  // One way to keep everything on screen is to allow
  // the user to adjust the font size.
  if(key == '+'){
    gFontSize++;
  }

  if(key == '-'){
    gFontSize--;
    if(gFontSize == 0){
      gFontSize = 1;
    }
  }

  gDebugger.setFont("verdana", gFontSize);
}

I named the debugger instance gDebugger. I did this only because using ‘debugger‘ is a JavaScript keyword and will break Proecessing.js sketches. On that note, Processing developers should shy away from all JavaScript keywords to keep their sketches Processing.js compatible. If you’re interested how we’re planning on solving this tricky issue, take a look at our Lighthouse ticket.

Okay, now that we have a basic interface, we can focus on the implementation:

public class cDebugger{
  private ArrayList strings;
  private PFont font;
  private int fontSize;
  private color textColor;
  private boolean isOn;

  public cDebugger(){
    strings = new ArrayList();
    setFont("verdana", 16);
    setColor(color(255));
    isOn = true;
  }

  public void add(String s){
    if(isOn){
      strings.add(s);
    }
  }

  /*
    If off, the debugger will ignore calls to add() and draw().
  */
  public void setOn(boolean on){
    isOn = on;
  }

  public void setFont(String name, int size){
    fontSize = size <= 0 ? 1 : size;
    font = createFont(name, fontSize);
  }

  public void setColor(color c){
    textColor = c;
  }

  public void clear(){
    while(strings.size() > 0){
      strings.remove(0);
    }
  }

  public void draw(){
    if(isOn){
      pushStyle();
      fill(textColor);
      textFont(font);
      int y = fontSize;

      for(int i = 0; i < strings.size(); i++, y += fontSize){
        text((String)strings.get(i), 5, y);
      }
      popStyle();

      // Remove the strings since they have been rendered.
      clear();
    }
  }
}

That’s it!

It’s just a simple bare bones real-time debugger. You can easily extend it to add more useful features such as ‘pages’ users can flip through or sprite categories. I’m sure you can think of many more ideas : )