Dangers of glsl testing

Fglrx + while(true)-shader or some other long loop = hanged computer.

Made the tests work on ATI drivers, but then the aforementioned calamity struck and now I probably get to curse myself for using XFS on /home.

On my Geforce 7600 the long shaders didn't cause an unrecoverable hang but it's not a fully programmable part. Maybe the drivers help too...

Programmable GPUs sure need a multitasking system.

GLSL parser work-in-progress git repo

To give meself a swift kick in the arse, here the github page.

The grammar is very unfinished and there are probably bad bits involved. The grammar structure comes pretty much directly from the GLSL 1.20 spec, which is nice, as it's LR.

You might notice that it doesn't deal with preprocessor directives, I hope there's a cpp lib out there somewhere. And it's in OCaml so no one can use it for anything.

I'm rather not appreciating the looming mountain of imagined work associated with validating and normalizing GLSL. I mean, syntactical validation isn't going to cut it (though it does help turning "float foo[5] = float[5](0.0, 1.0, 2.0, 3.0, 4.0);" into something that doesn't hit driver bugs.)

GLSL doesn't have numeric literal type coercion, so 20 + 2.2 is a type error. But the Nvidia driver (when you omit the #version pragma) mixes Cg stuff into GLSL, so 20 + 2.2 works there. On ATI it doesn't. On Nvidia with a #version pragma it doesn't work either. So I guess I'll make the pretty-printer add #version 120 at the top of the shader?

And then you have stuff like typecasting vectors, mix() with boolean alpha (IIRC needs #version 130 on ATI, works on Nvidia #version 120), operator overloading and swizzle assignment.

Oh well, it's just work. Do a little bit every day until the day you stop.

Revising initial schedule of "by July 1st" to "by 2012". O, the morning clouds are pretty, 4 am best time of day.

If you ever wanted to know how a graphics card works

This Radeon R6xx-R7xx programming manual [PDF] is really nifty. (See the rest of the Radeon docs as well.)

ATI installed for testing

Got an ATI graphics card installed. Took maybe an hour to get it physically installed and dualhead Xinerama going (if Xinerama is off, no extended desktop.) The biggest pain was the ATI download site requiring Javascript, so links didn't cut it.

Canvas 3D worked after applying the ATI compatibility patch. I need to rewrite some of my test shaders to add missing precision qualifiers.

O3D shaders fail to compile as the ATI driver (or something) says that MaxInstructions = -1. And O3D would rather prefer it to be a non-negative number, one that's larger than 23. Or whatever the SM 2.0 minimum is.

It's pretty noisy compared to my passively cooled card (big surprise, eh?) Now the question is what does one do with a teraflop of graphics computing power on Linux...

Started on the GLSL lexer

Started writing the GLSL parser by sketching the lexer for an hour today. It has all the identifiers and might even lex numbers correctly, but is not finished, does not compile and has no tests.

I'm working from the GLSL 1.40 spec on the assumption that it's easier to remove features than add them (though 1.40 removed the fixed-function bits, so it's not GLSL ES -compatible.)

The next step is to get the lexer to compile, then start writing the grammar, parser and tests.

Priorities for May

Choices, choices

The Canvas 3D API is getting pretty solid. The next step there would be writing a GLSL parser to validate shaders. I guess it'll run to around 2000 lines of code, so would take two months or so.

Regarding canvas performance issues (API call overhead, GC pauses, slow compositing), what I could do is write a draw list system, which'd be perhaps 500 lines of code (or 20 days) and help in getting rid of API call overhead. The other JS performance fixes would be rewriting the GC (which I would rather not do) and minimizing the JS->C call overhead (which I don't really know anything about, I hear quickstubs would help for the overhead), so I think I won't touch them for now.

Rewriting the compositing, while I believe I could do it, is a whole another can of worms. Basically, it's either going to be hellishly difficult or pretty easy. It would be pretty easy if the surfaces can be uploaded to OpenGL textures and drawn in the window clip area (plus upload timestamp to determine whether to re-upload texture.) But if that's too slow (i.e. if there's a zillion tiny surfaces changing all the time) or needs something special, then it's going to be hard.

The other related projects are writing a proof-of-concept OpenAL binding (guessing 5000 lines) and trying to port the GL canvas to some other browser (maybe a thousand lines?) Or maybe a GL canvas plugin (O3D-style)? The OpenAL experiment would teach some important lessons on the whole advanced audio thing.

On the tests side, a GLSL fuzzer would be nice to have, along with unit tests for the rest of the API. My devious plan to recruit other people to write them for me failed, so I guess I have to write them myself.

Priority lists

The entries in parentheses I'm not going to touch unless struck by divine inspiration.

Compatibility

1. GLSL parser - by July 1st
   1. Spike a quick ocamllex-ocamlyacc implementation to figure out the grammar, GLSL -> AST
   2. Write AST printer to output compatible GLSL
   3. Port over to some C library (first figure out which..)
2. GLSL subset spec - by July 1st
   1. Write in tandem with the parser
3. (GLSL compiler)

Tests

1. GLSL fuzzer - by August 1st
   1. Generate random ASTs and print them out
2. Unit tests for the whole API - by August 1st
   1. Start with one large test page, then split it as it progresses

Performance

1. (GC pauses)
2. (Slow compositing)
3. (General API overhead)
4. Draw list test - by June 1st
   1. Capture GL calls from JS, turn into an array of [function_id, args]
   2. Write a C++ DrawList-method that loops through the list and calls the wrapped GL functions
   3. Benchmark with a complex scene to see if it's worth the trouble

Other

1. OpenAL test, see if it makes sense - by September 1st
   1. Read through the API docs to figure out what it needs (and what it should provide)
   2. Write an OpenAL test app
   3. Strip out the GL-specific stuff from the Canvas 3D extension
   4. Change the API to bind OpenAL instead
   5. audioElement.getContext('moz-openal')?
2. (Canvas 3D plugin)

The whole non-parenthesized part of the above I estimate to take 4-5 months.. see you in September.

Updated 2009-05-06: more detailed plan

Some O3D vs. GL canvas performance analysis

To continue, my opinion on O3D is that it probably has the best approach for flexible rendering of complex scenes in the browser that I've seen yet. To explain, a little bit of background:

The bottlenecks on the JavaScript side are GC, JS -> C++ API call overhead (timed it at around 1.3 ms / 1000 calls here), and 4x4 matrix multiplication (~20-50x slower than C, and you do it a lot.)

O3D works a bit like editable display lists: you have a transform graph of objects that are separated into draw lists which are turned to native API calls in the renderer. And all that happens on the C++ side, so you don't have any JS -> C++ overhead for the drawing calls.

Suppose you want to animate a thousand meshes and don't want to push the matrix math to your vertex shader. To draw a single mesh, you need to bind the mesh's VBOs and textures, then setup the shader uniforms (transform, samplers, lights) and call the draw command. That means a dozen API calls per mesh, so a thousand meshes would need 12000 API calls, which'd take 15 ms just in call overhead.

And if you multiply the matrices for each mesh in JavaScript, you end up creating at least a thousand matrices worth of garbage every frame (~= 7.7 MB/s), triggering a Firefox GC run every 3 seconds (IIRC the max alloc until GC is 24 megs.) And as a thousand matrix multiplications takes 4 ms here, you end up with a total 19 ms JS overhead per frame and a framerate glitch every three seconds courtesy of GC.

The matrix math overhead is livable, but the JS -> C++ overhead and the GC pauses (on Firefox and Webkit) sink it. On O3D's embedded V8 JS engine, the GC pauses are less of an issue, as it uses a generational collector and the temporary matrices should be taken care of by the fast young generation collections (take this with a dose of salt, I haven't timed it.) They still get some hurt from having to do the matrix math in JS, but it's not too bad compared to the API call overhead and GC pauses.

What to optimize?

The best solution for the API call overhead would be to minimize JS -> C++ call overhead in the JS engine, maybe by generating direct native calls from the JIT.

Making an immediate-mode API that works on the concept of editable draw lists that are executed in C++ would get rid of the API call overhead as well. Even a system to draw a mesh with a single API call would drop the amount of API calls to a tenth (I imagine it'd be something like drawObject({buffers: [], textures: [], program: shader, uniforms: {foo4f: [1,2,3,4], bar1i: 2} }), but the draw list approach is probably easier to implement and more flexible. Record calls into a draw list and run through that in C++.)

GC pauses really need to be fixed in browser JS engines.

The matrix math slowdown isn't too bad but it's still nasty. I've heard some talk of adding native Vector and Matrix types to JS and maybe something like Mono.SIMD.