jump to navigation

XB PointStream 0.6 Released March 24, 2011

Posted by Andor Saga in Open Source, webgl, Arius3D, XB PointStream, FSOSS, point cloud.

I’m happy to announce we just released XB PointStream 0.6! In this release we fixed a number of problems and added many features. We managed to tie together a number of tools which use point clouds including Arius3D, 3D Studio Max and Blender. Go ahead and upgrade to Firefox 4 and check out the demos page!

Change Log

The following are some of the major additions we added to the library:

  • Created a PSI parser for native Arius3D point clouds – read the blog
  • Created a PTS parser for 3D Studio Max point clouds – read the blog
  • Created an ASC exporter for Blender – read the blog
  • Added a reference testing framework using Sundae.jsread the blog
  • Added a documentation generator
  • Replaced setInterval with requestAnimationFrame
  • Fixed coloring issues on Chrome on OSX 10.6

Rendering Point Clouds from 3D Studio Max with XB PointStream March 14, 2011

Posted by Andor Saga in Open Source, webgl, Arius3D, XB PointStream, point cloud.
add a comment

Skip the blog, watch the video.

I recently found out about Project Helix by reading a blog by Chris Mounts. Project Helix is a 3D Studio Max add-on that is being developed at Autodesk Labs. This add-on enables users to work with a large amount of point cloud data and export that data to a file. The technology is still under development, but will eventually become native.

There are two types of files used when working with Helix: PTS (ASCII) and PTSB (binary). The add-on can only work with binary file types (for performance reasons) so converting from PTS to PTSB is necessary before editing the cloud. Once you have modified the point cloud, you can export to PTS.

I was excited to see that PTS was plain text, since it meant I could hack together a parser for XB PointStream and import a point cloud straight from 3D Studio Max.

I opened the sample PTS file and saw it wasn’t much different from ASC:

PTS file:

5.74190  7.95260  4.15158  0.1  255 255 255
5.74190  7.95260  4.15614  0.1  240 240 240
5.74190  7.95260  4.15870  0.1  245 245 245

ASC file:

5.74190  7.95260  4.15158  255 255 255
5.74190  7.95260  4.15614  240 240 240
5.74190  7.95260  4.15870  245 245 245

The are only two differences. First, PTS declares a count at the start of each point cloud. Second, PTS contains a per-point intensity value. I was able to quickly adapt the ASC parser to read PTS files because the formats were so similar. I filed a ticket in Lighthouse and committed the patch for 0.6.

Pointilizing Suzanne: Converting Polygonal Meshes to Point Clouds March 6, 2011

Posted by Andor Saga in Open Source, webgl, XB PointStream, point cloud.

I’m often creating demos for XB PointStream, but I don’t have an appropriate point cloud to demonstrate the feature I implemented. There’s a plethora of free polygonal meshes available on the Web, but point clouds are scarce. This lack of available point clouds motivated me to solve this problem.

Over the weekend I had some success in developing a simple process which converts a polygonal mesh to a point cloud. I started with realizing that I only needed two things: a dense mesh and a way to export to .ASC.

If a mesh was dense enough, it could be rendered using point primitives which would neighbour one another, resulting in a point cloud without any ‘gaps’.

I remembered I could use a subdivision surface algorithm to solve my first problem. I started Blender and added Suzanne to the scene. Next, I used the Catmull-Clark SubSurf modifier to create a high-fidelity mesh.

0 levels 3 levels 5 levels

I then wrote a Python script for exporting only the vertices and normals which are needed for the .ASC file. I ran my script which gave me ~126,000 vertices. I figured that would be enough for a ‘closed’ point cloud.

I then tested my script by writing a demo which loads the point cloud with XB PointStream.

Next Steps

The script only exports vertices and normals. I’m interested in expanding on the code sometime in the future to make it export colors if the mesh is textured.

Messin’ With SMS: Transforming Point Clouds With an Accelerometer March 3, 2011

Posted by Andor Saga in Open Source, webgl, XB PointStream, point cloud.

To run the following demo, you need a WebGL-enabled version of Minefield running on a device which supports SMS.
Don’t have those? Watch the video!

MacBook laptops have a Sudden Motion Sensor which helps protect the internal hard drive in case the laptop falls. It turns out Mozilla has added an event listener to get the state of such motion sensors. This allows for some neat applications. For example, I’ve created a demo with XB PointStream which reads these sensor values and transforms a point cloud depending on the orientation of a laptop.

Play with the demo!
Watch the video!

Getting this to work was a lot simpler than it seems. All I did was add an orientation event listener…

window.addEventListener("MozOrientation", handleOrientation, true);

I kept track of the values in my callback…

function handleOrientation(data){
  x = data.y;
  z = data.x;

Then I transformed my point cloud.

function render() {
  ps.translate(0, 0, -50);
  ps.rotateX(x * 2.5);
  ps.rotateZ(z * 2.5);

I think this has some good implications in terms of the way users can interact with point clouds. Something else I’m interested in researching is doing something similar using augmented reality.

Special thanks to Charles J. Cliffe who originally informed me of this Firefox feature.

Hacking Sundae.js March 2, 2011

Posted by Andor Saga in Open Source, point cloud, webgl, XB PointStream.

Carl Desautels from Seneca College has recently released 0.5 of Sundae.js, a ref testing framework for JavaScript libraries which use the canvas element. I’m very excited about this project because it will save me many hours of manually testing XB PointStream.

I downloaded Sundae.js 0.5 from Github yesterday and started poking around. I saw the zip file came with test.json an example file which lists all the tests to be run. This was valuable since it gave me a jump-start on creating my own tests. It didn’t take too long before I had three simple tests ready to go.

"testSuite": [{
  "name": "only verts",
  "dependancyURL": ["../../mjs.js",
  "referenceImageURL": "resources/only_verts.png",
  "run": {"src": "resources/only_verts.js", 
            "func": "start" }
  "name": "acorn unlit",
  "dependancyURL": ["../../mjs.js",
  "referenceImageURL": "resources/acorn_unlit.png",
  "run": {"src": "resources/acorn_unlit.js", 
            "func": "start" }
  "name": "user shader",
  "dependancyURL": ["../../mjs.js",
  "referenceImageURL": "resources/user_shader/mickey.png",
  "run": {"src": "resources/shader/test.js", 
            "func": "start" }

I fed that into Sundae.js, but everything failed.

The problem was immediately obvious. As soon as testing began, XB PointStream started streaming the point clouds and by that time Sundae.js was already done. I needed a way to stall testing until my point clouds were finished being downloaded and parsed. I figured I could just crack open the source and add a fix.

After going over the Sundae JS file, I eventually found these two lines:

injectOrig(a, test.referenceImageURL, whenDone);
injectCurr(b, test.run, whenDone);

I soon realized I could stall the entire testing process by using a timeout. I figured there was some other code waiting for injectOrig to run. So I updated the code to wait three seconds which was long enough for the point clouds to be parsed.

  injectOrig(a, test.referenceImageURL, whenDone);
}, 3000);
injectCurr(b, test.run, whenDone);

This change fixed the timing problem, but the ref test results I saw were much different.

It looked as if either the canvas or the .PNG was being flipped vertically. But I knew at this point I was in good shape. Sundae.js was running and properly comparing the pixels, I just had to tweak the logic to flip either the canvas or the .PNG.

I re-opened the file and started searching for the comparison logic. I found a comment that gave it away : )

//Compare pixel arrays
var cCtx = c.getContext('2d');
var cPix = cCtx.createImageData(c.width, c.height);
var len = bPix.length;
for (var j=0; j < len; j+=4){
  if (Math.abs(bPix[j] - aPix[j]) < valueEpsilon  &&
    Math.abs(bPix[j + 1] - aPix[j + 1]) < valueEpsilon &&
    Math.abs(bPix[j + 2] - aPix[j + 2]) < valueEpsilon &&
    Math.abs(bPix[j + 3] - aPix[j + 3]) < valueEpsilon){
    cPix.data[j] = cPix.data[j+1] = cPix.data[j+2] = cPix.data[j+3] = 0;
  } //Pixel difference in c
    cPix.data[j] = 255;
    cPix.data[j+1] = cPix.data[j+2] = 0;
    cPix.data[j+3] = 255;
    failed = true;                 

All I needed to do was to play with the indices to flip the image. I created row and column variables to easily iterate over the array of pixels. I started at the bottom of a pixel array and read from left to right, bottom to top.

var col = 0;
var row = (4 * c.width * c.height) - (4 * c.width);
for (var j = 0; j < len; j += 4, col+=4){
  if(col === 4 * c.width){
    col = 0;
    row -= 4 * c.width;
  if (Math.abs(bPix[row+col] - aPix[j]) < valueEpsilon  &&
      Math.abs(bPix[row+col + 1] - aPix[j+ 1]) < valueEpsilon &&
      Math.abs(bPix[row+col + 2] - aPix[j + 2]) < valueEpsilon &&
      Math.abs(bPix[row+col + 3] - aPix[j + 3]) < valueEpsilon){
      cPix.data[j] = cPix.data[j+1] = cPix.data[j+2] = cPix.data[j+3] = 0;
  } //Pixel difference in c
    cPix.data[j] = 255;
    cPix.data[j+1] = cPix.data[j+2] = 0;
    cPix.data[j+3] = 255;
    failed = true;

My results started to look much better, save for Mickey.

I found it strange that it was only the pixels on Mickey’s pants and shoes which were incorrect. After looking closely I saw the red pixels in the .PNG were faded in Minefield. I suspect this is why it was failing. At that point I wondered how the test would run on Chrome.

Interestingly, on Chrome the colors are fine, but since it performs anti-aliasing, the test still ‘fails’. I’m going to look into filing a bug with Mozilla since the .PNG colors are fine in Preview. I’m also going to try to disable anti-aliasing in Chrome, which should pass all the tests.

Next Steps

I’m going to file a few tickets in the Sundae.js Lighthouse account. There should be a way for the library being tested to tell Sundae.js to wait until it’s ready to be tested. Also, I’m wondering if the flipY fix I added is only an issue because of the timeout or if the Sundae.js ref testing is indeed upside down.