jump to navigation

Building a Generic WebGL Camera Library May 10, 2011

Posted by Andor Saga in Open Source, webgl, C3DL, XB PointStream, point cloud.
2 comments

One problem with XB PointStream is the lack of a camera system. This exists because the WebGL library was built primarily for parsing and rendering point clouds files.

We’ve left it up to users to ‘create’ their own cameras by calling our low-level matrix transformations functions. However, as more projects begin to use the library (WebGL PhotoSynth Viewer, 3DTubeMe, 3DStream) it’s becoming apparent some form of easy-to-use camera for XB PointStream is necessary.

I began tackling this issue on my spare time since it bothered me so much. First, I investigated which types of cameras are supported by Arius3D‘s 3DImageSuiteViewer. They have three which are fairly well known: Orbit, Free and ArcBall. I started by implementing something simple, an orbit camera I already developed for C3DL. Orbit cameras are fairly simple, but harder to explain, so just play with the working demo:

When I started developing this, my plan was to strip out C3DL-specific code and get it working with XB PointStream. After a bit of tinkering, I realized the fixes I would be making should be fed back upstream to C3DL. Taking this one step further, any WebGL library should be able to make use of this code. So I closured the entire camera.

Here’s how you can use it:

// Create an orbit camera halfway between the closest and farthest point about 
// the default orbit point [0,0,0].
var cam = new OrbitCam({closest:0, farthest:100, distance: 50});

// override the values.
cam.setFarthestDistance(80);
cam.setClosestDistance(10);
cam.setDistance(10);

// go back as far as possible (uses 80)
cam.goFarther(100);

// figure out how much the cursor moved...
var deltaX = ....
var deltaY = ....

// Yaw about global Y.
cam.yaw(deltaX);

// Pitch about orbit point.
cam.pitch(deltaY);

// matrix magic!
pointStream.multMatrix(M4x4.makeLookAt(cam.position, cam.direction, cam.up));
pointStream.translate(-cam.position);

// done!
pointStream.render(pointCloud);

Okay, the demonstration isn’t all that impressive. What is really impressive is the encapsulated and sharable code. Any WebGL developer can take this and instantly have a working camera. The code takes care of making sure the camera doesn’t zoom out too far and always has the correct orientation. Neat eh? Once I (hopefully) finish the other main cameras (ArcBall and Free), I’ll place them on a github repository for other WebGL developers.

Improving c3DL Performance October 3, 2010

Posted by Andor Saga in C3DL, Open Source.
3 comments

Last week Cathy asked me to come up with some ideas on how to speed up C3DL. Now we already have a few techniques which give us pretty good performance improvements like using typed arrays when possible or culling objects using bounding sphere/frustum tests. But there’s still a way to go in terms of speed.

I spent some time researching topics and thinking about different solutions to this problem and then I filed the following tickets which I thought would decrease rendering time.

Tracing

One thing we can do make sure our code is on trace. There’s a few places in our library which Minefield can’t compile. This means there are parts of the library where every rendered frame interprets ‘slow’ JavaScript. We need to review these bits in our code and fix them. This would be a definite win since users don’t need to do anything to gain the benefits.

Billboarding

We can help user scripts stay fast by adding billboarding support. If a user wants to render a high-quality model far from the camera, they may be better off drawing a textured quad which orients itself towards the camera. This shouldn’t be difficult, we already have the this code in the particle system.

Discrete Level of Detail

Next, we could add support for discrete level of detail. Why render a high-quality detailed object far from the camera when it will only occupy a few dozen pixels on the canvas? If the user could provide versions of their model which range in detail, the library could switch between them as the camera approaches the object. The trade-off to this would be the noticeable ‘pop’ as the objects are switched. For this problem, we could implement a form of continuous level of detail, but this isn’t something we should consider soon.

Spatial Partitioning

Spatial partitioning can provide the library with useful high-level information about the scene to render. By splitting up the scene we could perform more hidden surface removal. Also, Since the scene would be sorted, we could reduce use of depth buffer reading and writing, improving performance….theoretically. I ran some tests for this, but I didn’t see any improvement.

Benchmarks

Finally, we need to update our performance test benchmark script and integrate it into our release procedure. Keeping a log of performance results (and visualizing it with Processing.js?) will help us make sure future releases stay fast.

Of course, we shouldn’t limit ourselves to only these tickets. These are just some things I thought we could implement. We can (and should) look into other techniques such as portals, antiportals, pre-lit geometry, instancing, displacement mapping, occlusion culling, etc, etc, etc.

Modularizing C3DL September 18, 2010

Posted by Andor Saga in C3DL, JavaScript, Open Source, webgl.
add a comment

Over the Summer Matthew Postill has done some substantial work on C3DL. He’s fixed bugs, added collision detection and sped up rendering with frustum culling.

We’re expecting the library to continue to grow in size and features. The problem is not all these features will be desired by developers using the library. If a user only wants to render a teapot with C3DL, why force them to load the library in its entirety, with a particle system, a bunch of shaders and collision detection? Cathy has suggested we could tackle this problem by modularizing the library. This would impact the internals of the library quite a bit, but it would also provide much more flexibility.

Firstly, it would allow developers to build the library—a bit like jQuery. The user would select what components they need and the library would be built containing only those selected options.

Secondly, it would allow other developers to create their own components and hook them into C3DL. This means developers could write their own model parsing code instead of being forced to use ours.

Another related change would entail offering release and debug versions. The debug version could include parameter checking at the start of each function, which would be omitted from the release build.

I’m not going to kid myself. There’s a significant amount of work required to make this happen—there’s also an immense payoff. So I’m excited to start working on this, but I know I’ll need some help. I’d love to hear from any developer who has had experience implementing something along these lines.

Summer Reflections September 14, 2010

Posted by Andor Saga in Arius3D, C3DL, JavaScript, Open Source, Processing.js, XB PointStream.
2 comments

During the Summer I had the opportunity to work with some highly motivated and intelligent developers at Seneca’s Centre for Development of Open Technology. For four months we cranked out code for several exciting technologies such as for C3DL, NextJ, The Fedora ARM project, XB PointStream, Popcorn and Processing.js.

This was the first time I have worked in CDOT where there were so many developers working on so many projects. Almost all the projects dealt exclusively with JavaScript, but we also had to work with other libraries and standards like WebGL, JSON, and Video. As the technologies we worked with varied, so did our challenges: documentation was scarce, standards and APIs changed, or our code simply didn’t work and we needed help.

What made working in the CDOT environment actually work is communication. Our days began with a morning Scrum meeting where we exchanged problems we stumbled into the day before and also shared our success stories. The meetings were brief (only 10 minutes) but on a few occasions they were invaluable. As we stated our problems our colleagues gave their ideas and opinions, “Have you tried looking into something like this…?” or “You should read this blog…”. We didn’t always have the answers, but we were good at pointing someone the right direction.

And sometimes we did have the answers. Our cubicles were close so it made sense to ask the regular expression expert a question which would save us half an hour and only steal a few minutes of their time. Other times we posed questions to other developers on IRC or we received extremely useful suggestions on our blog posts.

We also took the opportunity to meet face-to-face with our industry partners. Developers at Arius3D gave us guidance, tips and valuable resources. Down at the Toronto Mozilla office we were given a WebGL walkthrough and help with relevant WebGL tools. We also worked closely with Brett Gaylor–a filmmaker working with us on the video tag. Others of us met with developers from NexJ and Fedora.

All these forms of communication were important for the development of the technologies we worked on. It only reminds me how crucial these are for open source development.

Now that the Fall semester has started I’m back in Seneca taking classes but I’m still excited to be working at CDOT on XB PointStream, C3DL and Processing.js.

Initial thoughts on High Quality Surface Splats with WebGL May 20, 2010

Posted by Andor Saga in C3DL, JavaScript, Open Source, Processing.js, webgl.
2 comments

Last week Mickael and I met up with developers from Arius3D. They specialize in 3D imaging, which includes scanning 3D objects, displaying them on screens and printing them out in 3D. They have a free ActiveX plug-in called 3DImageSuite for IE which allows users to view the 3D objects.

The problem is users are limited to using one browser to view this content. 3DImageSuite is also only available for Windows. So even though IE “dominates” the browser world, Arius3D are limiting their non-IE users.

Because of our experience developing Javascript libraries which specialize in 3D rendering, we have partnered up with them to solve this problem. We’ll be using WebGL to do the rendering. Not just because it’s what we specialize in, but also because it’s becoming a standard. Once WebGL is fully implemented in the other browsers and we complete this technology, their content will be rendered cleanly without any add-ons. It will be IE users who will be at a disadvantage when trying to view this content.

3DImageSuite displays point clouds which consists of XYZ and RGB data. So the problem sounds trivial, especially since we already have the capacity to render 3D points in both C3DL and Processing.js. The real problem is the 3D scans are very dense, they are composed of millions of points. Trying to render this huge amount of data cannot be done in either of our libraries practically.

To get started on this project, I began reviewing my meeting notes with the Arius3D developers and started brainstorming for ideas and thoughts on the solution.

• One of the first things I thought about was allowing the user to control the level of detail. This can be extremely valuable user input. If the user doesn’t require 100% detail, there is not point in wasting resources downloading and rendering it. This can be the case when they are on a limited mobile platform.

• We are in the process of trying to speed up Processing.js by making the browser do a JIT on the Javascript. This will allow the JS to run native C++ speed. We’ll be trying to place as much processing as we can on the GPU and CPU.

• Use compression on server side and decompress data on the client side to reduce download times.

• Implement some form of spatial partitioning, such as an octree to cull splats.

• Stream the data so lower quality ‘versions’ of a model can be displayed quickly and progressively gain quality.

• Use static rendering. That is, render the model only once and not in ‘real-time’. If the user rotates the object, render a low-quality version while the user is rotating it. Once the user stops, re-render 100% of the object. I’ve seen this done before, but I forgot about it. Luckily someone reminded me at OCE. Thanks

• Because self-shadows is something which Arius3D is considering, we may need to use raytracing instead of simple Blinn-Phong shaders.

These are just some of the initial thoughts I’ve had along with ideas from the developers. I’ve begun some research on surface splatting and spatial partitioning and the problem does seem very interesting and exciting. I think more research, development and talks with the developers will lead us in the right direction.

Study Week Update March 10, 2010

Posted by Andor Saga in C3DL, Games, Open Source, webgl.
add a comment


Reading week is over and just like last semester, I didn’t do much reading, or studying for that matter. Last semester I indulged in a solitary week-long hack-a-thon. I attempted to port Crayon Physics to the Web using Processing.js, you can find the final (unfinished) demo here.

This reading week I worked on C3DL. I fixed a few bugs, added support for up_axis for COLLADA models and continued some work on the beginnings of a RTS game using the library. You’ll need a WebGL-compatible browser to view the demo. If you don’t have one, you can take a look at a video I made.

Rendering Blending Objects Last February 16, 2010

Posted by Andor Saga in C3DL, Open Source, webgl.
add a comment

This morning I added a static outline to a building in the C3DL RTS demo. I soon realized there was something wrong with the way the final scene looked. Here is a screen shot, can you see what’s wrong?

The problem is the lines (which make up the circle) are being rendered after the particle system. Since the particle system uses blending, it needs to be rendered last. It takes whatever is in the color buffer and blends it with its own colors, it doesn’t just overwrite the color buffer fragments like lines. Here is how it should look:

Obviously we didn’t have a demo before which used both particle systems and lines, which is why this little bug managed to slip its way into the library. The fix was simple, but I’ll need to add a ticket to our lighthouse account before I can get the changes committed.

Preliminary WebGL RTS Game With C3DL February 15, 2010

Posted by Andor Saga in C3DL, Games, Open Source, webgl.
3 comments

Cathy asked me to make a cool demo using C3DL. C3DL is a JavaScript library which wraps WebGL, but also adds other functionality. I had some ideas, but starting a preliminary real-time strategy game made the most sense. It not only demonstrates a lot of C3DL features such as model loading, transformations, lighting, shaders, picking, cameras, textures, etc, but since animation is kept to a minimum, the frame rate on slower machines shouldn’t be a big deal.

You’ll need a WebGL-compatible browser to run it. You can either download Minefield, WebKit or Chromium. The demo seems to work on all three. Keep in mind I hacked this thing together in a day so the code isn’t pretty, but here it is anyway!

Simplifying the Interface January 23, 2010

Posted by Andor Saga in C3DL, JavaScript, Open Source.
add a comment

Something I need to do is simplify the C3DL interface. When approaching such tasks, one method I found extremely useful was to analyze the code in terms of the interface, or that is from the user’s perspective. To do this I first created a basic demo of a spinning cube by writing the necessary HTML and JavaScript files. As for the HTML, it will need to be something like this:
(I emphasized the code which potentially could be changed)

<html>
  <head>
    <title>Cavas3D Demo</title>
    <script>var SCRIPT_PATH = '../../canvas3dapi/'</script>
    <script src="../../canvas3dapi/c3dapi.js">
    <script src="basic_demo.js"></script>
  </head>

  <body>
    <canvas id="demo" width="500" height="500"></canvas>
  </body>

</html>

It would be great if this could be simplified to something like this:

<html>

  <head>
    <title>Cavas3D Demo</title>
    <script src="../../canvas3dapi/c3dapi.js">
  </head>

  <body>
    <canvas datasrc="basic_demo.js" width="500" height="500">
    </canvas>
  </body>

</html>

I removed the somewhat unintuitive SCRIPT_PATH variable and moved the basic_demo.js resource into the canvas tag making it more obvious which .js file is associated to which canvas. These changes haven’t actually been made, I’m just playing with possible changes, brainstorming which parts might be able to be modified.

Now, very few changes were made here and it will probably require a significant amount of effort to make it work. But this work is justified by the fact that the user’s experience and first impression will be more positive. If the user has to spend more than a few minutes trying to get a simple example to render, they’ll probably look for alternatives.

Let’s see how the JavaScript could be changed. Right now, you need to write something like this:
(Again, I emphasized the code which potentially could be changed)

c3dl.addModel('cube.dae');
c3dl.addMainCallBack(mainDemo, 'demo');

function mainDemo(canvasName)
{
  var scn = new c3dl.Scene();
  scn.setCanvasTag(canvasName);
  var renderer = new c3dl.OpenGLES20();
  scn.setRenderer(renderer);
  scn.init();
  
  var cam = new c3dl.FreeCamera();
  cam.setPosition([0,0,50]);
  cam.setLookAtPoint([0,0,-1]);

  var cube = new c3dl.Collada();
  cube.init('cube.dae');
  cube.setAngularVel([0.001,0.001,0.0]);

  scn.addObjectToScene(cube);
  scn.setCamera(cam);
  scn.startScene();
}

Some changes I had in mind involved getting rid of the global c3dl.addModel() and c3dl.addMainCallback() calls and changing OpenGLES20 to just Renderer. The Renderer is an interesting problem. When WebGL started out as Canvas3D, we called the renderer OpenGLES11, then later it became OpenGLES20 and now it could probably be WebGL. Considering how much it has changed and will change, we’ll have to invest some time to make this stop. One simple solution is to abstract the renderer.

For example, if we change OpenGLES20 class to Renderer, the user will no longer need to be concerned what underlying rendering method is used. If a visitor loads the user’s demo, but visitor’s browser doesn’t support WebGL (i.e. it’s I.E.), the demo still loads. This of course only works if we accommodate for this case. So if the browser is IE, we use DirectX. If the browser supports WebGL, we use that. The end result is the user’s page will create the appropriate renderer without them needing to worry about the user’s browser. So let’s look at what lines changed:

function mainDemo(canvasName)
{
  var scn = new c3dl.Scene();
  var renderer = new c3dl.Renderer();
  scn.init(renderer);
  
  var cam = new c3dl.FreeCamera();
  cam.setPosition([0,0,50]);
  cam.setLookAtPoint([0,0,-1]);

  var cube = new c3dl.Collada();
  cube.init('cube.dae');
  cube.setAngularVel([0.001,0.001,0.001]);

  scn.addObjectToScene(cube);  
  scn.setCamera(cam);
  scn.startScene();
}

In terms of the lines trimmed, I’m not certain how much of it is viable. The reason some of the seemingly redundant code is required was because I couldn’t devise any alternative at the time. Looking at it from a different perspective, I should be able to come up with something. In the end we’ll have an interface which is more flexible, easier to understand and of course simple.

Merging C3DL into PJS November 18, 2009

Posted by Andor Saga in C3DL, Open Source, Processing.js.
6 comments

A few days ago I took my first steps in placing 3D functionality in the Processing.js library. It was confusing and awkward and it led to a demo which is anything but pretty. Not only is the output a joke, the code is quite crude. Regardless, I decided to write a blog post about my small success. As my professor Dave stated I “was brave” in posting it. Looking at it now, I’m very surprise I did, but I guess I get points in releasing early and often. As the blog’s title suggests, it was a bit like force-feeding the Processing.js library C3DL. It was terribly messy, albeit I did succeed. Of course my success lured me to my next challenge: The cube couldn’t move and couldn’t be lit.

After spending some quality debugging time, I did manage to fix those bugs. Here, I can prove it with a JPEG! Without getting into boring technicalities, I’ll just say I needed to make sure the cube model was preloaded before it was instantiated. You’ll have to trust me. The JavaScript evolved to something less messy and a bit more pleasant. It was more like Injecting one library into another. I was getting closer to what I wanted.

So I had both libraries playing nice, but my next problem was packaging this properly so it could be useful to others. I needed to create some sort of architecture which would allow anyone else trying to throw 3D into Processing.js a head start. They could focus on higher level problems rather than diving into WebGL intricacies. I cut out some code, added some code, threw in some comments and made a page with minimal (but existent!) instructions. Chris Tyler helped me minimize the C3DL library so running the demo was slightly faster. The code is getting better. Good enough for me to refer to it as merging one library into another. Harmony is yet to be achieved, don’t worry I’m getting there. : )

Anyway, the files are now up. The screenshot I posted in the beginning of this blog was taken from the running demo. Because C3DL uses WebGL, you’ll need to use Minefield to run the demo. Firefox <3.5 won’t cut it. You can also download the zip with everything packaged up.

Why did I decide to release the code now? It was mostly because I didn’t see the code as Web worthy. Frankly, I see it still as barely Web worthy, but it will have to do for now. The demo has some issues listed on the page in which it renders. I suppose you can decide if it is useful to you or not. If anything I’ll blame the mantra Dave an Chris keep trying to instill: “Release early, release often”.

So, the demo hijacks the Processing.js library since it implicitly places a cube in the scene. How can this useful? Well, my fellow students in the open source course at Seneca College (Yeah, this is a credited course!) can play around with it and see how things are implemented in C3DL then adapt those methods for PJS. They can take a peek into the vertex shaders, the matrix stack, cameras and all sorts of other 3D stuff.

I’m going to keep working on this ‘architecture’, but I’ll shift my focus onto my 0.2 release for my course since it’s due in a couple of days. What I’ll need to do is start removing all the C3DL which will leave me with only WebGL calls for size(), background(), frameRate and similar functions. They aren’t really 3D functions, but still this involves quite a bit of work, I may lose some sleep. But that’s okay, I have a nice proverb posted above my monitor: “Sleep when you die”.

I’d like to thank Cathy and Dave for convincing a silly student to get 3D working in Processing.js the easy way rather than starting from scratch, which I was tempted to do. I believe they saved me a lot of time. Don’t forget: “Good programmers know what to write. Great ones know what to rewrite (and reuse).”