Querying WebGL Commands and Symbols

The Khronos specification is detailed, organized and official. The WebGL Cheat Sheet by Nihilogic is a great quick reference tool. But sometimes I just want to query the WebGL context and get all the raw commands and symbols. This can be very useful and it’s easy:

var gl = document.createElement('canvas').getContext("experimental-webgl");
var symbols= "";
for(currSymbol in gl) {
  symbols += currSymbol + "\n";
}

symbols will then be filled with the commands and symbols available for your browser:
canvas
viewport
clearColor
clear
enable
blendFunc
...
This of course works for either Minefield, Webkit, but also Chrome. Yeah, CHROME, isn’t that sweetness?

WebGL Browser Stress Tests Using Processing.js

I’m working on XB PointStream, a tool which will emulate Arius3D‘s PointStream. XB PointStream will use WebGL to render thousands of points efficiently in the browsers.

I was curious to see how many points Processing.js could churn out using WebGL. If fast enough, it could potentially be used as underlying rendering engine for XB PointStream. So I wrote a Processing sketch which rendered thousands of points, and I took some measurements. I didn’t have any point clouds at hand, so I just generated a few thousand points myself:

Once I recorded how long it took Minefield and Webkit, I started hacking in some optimizations in the Procesing.js library. I knew I could start with reducing the number of calls to uniformMatrix*. For example, in Processing.js, the perspective and view matrices are set every time a point is rendered, which is unnecessary. I moved these function calls out and ran the browsers through the tests again. I was pleased to see rendering time was cut roughly in half.

I then filed a bug on our lighthouse account to reduce the number of uniformMatrix*. We still have a way to go in performance, especially since these tests are static renders, don’t account for download time and don’t use real-time lighting.

No Chromium Tests Yet

It’s unfortunate that I couldn’t benchmark Chromium. I even took extra steps to get the tests to run by placing the Processing sketch within the HTML file so Chromium wouldn’t throw XHR errors. I also merged one of my Processing.js patches which does not transpose matrices in calls to uniformMatrix*, which the nightly has issues with. But even after doing this, Chromium surprisingly rendered extremely slowly. This was the exact opposite of what I expected since it always outperforms the other browsers. I could try to find the nightly which renders properly, but I don’t have time to sift through all those nightlies. If someone knows when WebGL was broken, let me know.

XB PointStream on Twitter

I created a Twitter account for XB PointStream. This is where we’ll be posting short updates, links to blogs and other XB PointStream related news.

The Twitter avatar was taken from pterjan‘s Flickr photostream. It’s called “Selfridges Birmingham”. I’m still unsure how to probably attribute the work, so feel free to comment if you know anything about CC attribution.

Chromium Fails When Trying to Transpose in uniformMatrix

A few weeks ago I updated Chromium, but the update broke my 3D Processing.js sketches. I didn’t have the time to look into it, so I continued using Webkit and Minefield to run my tests. This week I had a chance to check out the problem, so I loaded the C3DL splash page to see if it would render. Interestingly it worked. I then tried Processing.js, but that was still broken. How was it that Chromium rendered C3DL and not Processing.js? I knew there must be some WebGL command somewhere in Processing.js which Chromium didn’t like. I poked around our library for a while but still couldn’t get anything to render. The only thing that seemed to work was changing the background color.

I then thought about checking for any errors WebGL was reporting. I stuck a few alerts in the library:
alert(ctx.getError());
I started seeing non-zero values. WebGL was reporting error 1280. I converted this to hex (0x500) and ran a search. It turns out the error was because of an INVALID_ENUM. I kept throwing in alerts until I could isolate which command was causing this error.

I eventually found it to be this line:
ctx.uniformMatrix4fv(varLocation, transpose, matrix);
I was passing in true as the second argument and figured that must be causing the problem. I changed it to false and WebGL stopped complaining. Great! Of course nothing rendered. So to work around this problem, I just had to transpose the matrices myself before passing them to WebGL. This didn’t change Minefield or Webkit, they still run fine and now so does Chromium.

XB PointStream

In my last post I wrote about an application by Arius3D called Pointstream, which at its core renders millions of surface splats very efficiently.

I’m working with a few other researchers in the Seneca College CDOT lab. We are starting some work on creating a version of their viewer which works on Minefield, Webkit and Chromium using WebGL. I tentatively named the project XB PointStream in hopes that it will one day be totally cross-browser. If IE implements a subset of DirectX, it would fulfill this hope. That way all the major browsers would be able to render Arius3D scans without a plug-in.

We just started the project, so there isn’t much to show unfortunately. But here are some links we’ll be using:
Lighthouse account
Github repository
A temporary website

If anyone thinks of a better name, let us know soon!

The next thing we’ll need to do is flesh out some specifications.

• What formats to support (PSI, DAE, ..)
• Maximum wait time to render whole object
• Shader lighting (I believe Arius3D uses BRDF)
• User controls
• etc..

Some of these we can derive from the viewer, but we shouldn’t emulate it entirely. Arius3D wants us to create something generic so other developers facing the same issue would be able to use this tool as well. We will probably have another meeting to figure out some of these details. From there we can begin filing bugs in lighthouse and start hacking.

Installing Arius3D PointStream 3DImageSuite Viewer on Windows 7

In my last blog post I wrote I would be starting work on an online version of Arius3D’s point cloud data viewer. To start, I needed to install their viewer. I downloaded the viewer on Windows 7. Then I began installing, but near the end of the installation I received this error message:

3DImageSuite Viewer Error
Error 1925. You do not have sufficient privileges to
complete this installation for all users of the machine.
Log on as an administrator and then retry this installation.

and then another error:
Feature transfer error
Error: -1603 Fatal error during installation.
Consult Windows Installer Help (Msi.chm) or MSDN
for more information.
I logged on as admin, but had the same error. I did some searching and found some hacks involving the registry, but didn’t want to go there. Eventually I just decided to try to run in compatibility mode and see if that would work, and it did.

So to install the viewer:

1. Right-click on the icon
2. Click troubleshoot compatibility
3. Click Try recommended settings
4. Click Start the program
5. Click Next on the Program Compatibility window once the InstallShield Wizard appears
6. Click Next on the InstallSheild window and follow the instructions.

The installation should finish without reporting any errors. Once you start the viewer, you’ll see a pretty butterfly. You can then close the program compatibility.

Initial thoughts on High Quality Surface Splats with WebGL

Last week Mickael and I met up with developers from Arius3D. They specialize in 3D imaging, which includes scanning 3D objects, displaying them on screens and printing them out in 3D. They have a free ActiveX plug-in called 3DImageSuite for IE which allows users to view the 3D objects.

The problem is users are limited to using one browser to view this content. 3DImageSuite is also only available for Windows. So even though IE “dominates” the browser world, Arius3D are limiting their non-IE users.

Because of our experience developing Javascript libraries which specialize in 3D rendering, we have partnered up with them to solve this problem. We’ll be using WebGL to do the rendering. Not just because it’s what we specialize in, but also because it’s becoming a standard. Once WebGL is fully implemented in the other browsers and we complete this technology, their content will be rendered cleanly without any add-ons. It will be IE users who will be at a disadvantage when trying to view this content.

3DImageSuite displays point clouds which consists of XYZ and RGB data. So the problem sounds trivial, especially since we already have the capacity to render 3D points in both C3DL and Processing.js. The real problem is the 3D scans are very dense, they are composed of millions of points. Trying to render this huge amount of data cannot be done in either of our libraries practically.

To get started on this project, I began reviewing my meeting notes with the Arius3D developers and started brainstorming for ideas and thoughts on the solution.

• One of the first things I thought about was allowing the user to control the level of detail. This can be extremely valuable user input. If the user doesn’t require 100% detail, there is not point in wasting resources downloading and rendering it. This can be the case when they are on a limited mobile platform.

• We are in the process of trying to speed up Processing.js by making the browser do a JIT on the Javascript. This will allow the JS to run native C++ speed. We’ll be trying to place as much processing as we can on the GPU and CPU.

• Use compression on server side and decompress data on the client side to reduce download times.

• Implement some form of spatial partitioning, such as an octree to cull splats.

• Stream the data so lower quality ‘versions’ of a model can be displayed quickly and progressively gain quality.

• Use static rendering. That is, render the model only once and not in ‘real-time’. If the user rotates the object, render a low-quality version while the user is rotating it. Once the user stops, re-render 100% of the object. I’ve seen this done before, but I forgot about it. Luckily someone reminded me at OCE. Thanks

• Because self-shadows is something which Arius3D is considering, we may need to use raytracing instead of simple Blinn-Phong shaders.

These are just some of the initial thoughts I’ve had along with ideas from the developers. I’ve begun some research on surface splatting and spatial partitioning and the problem does seem very interesting and exciting. I think more research, development and talks with the developers will lead us in the right direction.

Smoothing and changing point sizes in WebGL

If you have a WebGL-enabled browser you can check out my demo.

If you want to download a WebGL browser, you can get either Minefield, Webkit or Chromium.

So, while working on the Processing.JS project, I ran into an interesting bug. The Processing language provides a way to change the size of 3D points. Processing is Java-based and uses OpenGL to do the rendering. Processing.JS is a JavaScript port of the language and since WebGL is used to render points, all 3D functions eventually call some form of WebGL command.

So if you’re also writing a WebGL script which changes point sizes, keep reading.

To change the size of points you would typically do two things. First you’ll need to enable VERTEX_PROGRAM_POINT_SIZE.

ctx.enable(ctx.VERTEX_PROGRAM_POINT_SIZE);

Next, you need to set the point size. This is done in the vertex shader. You’ll probably want to do this with a uniform or attribute variable to allow different sizes.

attribute vec3 Vertex;
uniform mat4 MVP;
uniform float pointSize;

void main(void){
  gl_PointSize = pointSize;
  gl_Position = MVP * vec4(Vertex, 1.0);
}

This works in WebKit, but if you try it in Minefield, it will fail. Your points will be stuck to the size of 1 pixel. The way around this is to use the literal value of VERTEX_PROGRAM_POINT_SIZE, which is defined in OpenGL as 0x8642. Using the literal is probably a better way to ensure cross-compatibility. You may even want to define your own JavaScript variable for this.

Your cross-browser compatible code that enables point size changes becomes:

ctx.enable(0x8642); 

Done! Unless you want your points smoothed (rendered as circles rather than squares). Again, Minefield has issues with this. Instead of calling:

ctx.enable(ctx.POINT_SMOOTH);

You’ll need to call:

ctx.enable(0x0B10);

Interestingly Webkit renders smoothed points by default whereas Minefield renders non-smoothed points by default. Also, Minefield renders shimmering outlines on smoothed points while Webkit doesn’t. Try the demo and you’ll see.