fhtr

2014-08-31

Spinners on the membrane

I wonder if you could do image loading spinners purely in CSS. The spinner part you can do by using an animated SVG spinner as the background-image and inlining that as a data URL. Detecting image load state, now that's the problem. I don't think they have any attribute that flicks to true when the image is loaded. There's img.complete on the DOM side, but I don't think that's exposed to CSS. And it's probably tough to style a parent element based on child states. And <img> elements are fiddly beasts to begin with as they're sort of inlined inline-blocks with weird custom alignment rules, don't take child elements and generally behave like eldritch abominations from beyond time.

If you had a :loading (or some sort of extended :unresolved) pseudo-class that bubbled up the tree, that might do it. You could do something like .spinner-overlay { background: white, url(spinner.svg)50% 50%; width: 100%; height: 100%; opacity: 0; } *:loading > .spinner-overlay { opacity: 1; transition: 0.5s; }. Now when your image is loading, it'd have a spinner on top of it and it'd go away when it stopped loading. And since :loading doesn't care about what is loading, it'd also work for background-images, video, audio, iframes, web components, whatever.

2014-08-30

Sony A7, two months in

I bought a Sony A7 and a couple of lenses about two months ago to replace the Canon 5D Mk II that I had before. Here are my experiences with the A7 after two months of shooting. I've taken around 1500 shots on the A7, so these are still beginner stage thoughts.

If you're not familiar with it, the Sony A7 is a full-frame mirrorless camera at a reasonable (for full-frame) price point. It's small, it's light, and it's full-frame. The image quality is amazing. You can also find Sony E-mount adapters to fit pretty much any lens out there. If you have a cupboard full of vintage glass or want to fiddle with old Leica lenses or suchlike, the A7 lets you shoot those at full-frame with all the digital goodness of a modern camera. You probably end up adjusting the aperture and focus manually on adapted lenses though. There are AF-capable adapters for Sony A and Canon EF lenses, but the rest are manual focus only.

It's not too much of a problem though, since manual focusing on the A7 is nice. You can easily zoom in the viewfinder to find the right focus point and the focus peaking feature makes zone focusing workable. It's not perfect ergonomically, as your MF workflow tends to go "press zoom button twice to zoom in, focus, press again for even closer zoom, focus if need be, press zoom button again to zoom out, frame." What I'd like to have is "press zoom button, focus, press zoom button, frame." Or zoom in just one area of the viewfinder, leave frame border zoomed out so that you can frame the picture while zoomed in.

Autofocus is fiddly. Taking portraits with a shallow depth-of-field, the one-point focus mode consistently focuses on the high-contrast outline of the face, which tends to leave the near eye blurry. Switching to the small movable focus point, you can nail focus on the near eye but now the AF starts hunting more. And I couldn't find a setting to disable hunting when the camera can't find a focus point (like on Canon DSLRs), so AF workflow is a bit slower than I'd like. Moving the focus point around is also kinda slow: you press a button to put the camera into focus selection mode, then slowly d-pad the focus point to where you want it.

The camera has a pre-focus feature that autofocuses the lens when you point the camera at something. I guess the idea is to have slightly faster shooting experience. I turned the pre-focus off to save battery.

The camera's sleep mode is defective. Waking up from sleep takes several seconds. Turning the camera off and on again seems to get me into ready-to-shoot state faster than waking the camera by pressing the shutter button. Because of that and the short battery life, I turn the camera off when carrying it around.

The Sony A7 has a built-in electronic viewfinder. The EVF is nice & sharp (it's a tiny 1920x1200 OLED display!) When you lift the camera to your eye, it switches over to the EVF. Or if you get your body close to the EVF. Or if you place the camera close to a wall. This can be a bit annoying, but you can make the camera use only the rear screen or the viewfinder. Note that the viewfinder uses a bit more battery than the rear screen. Probably not enough to show up in your shot count though.

If you have image preview on, it appears on the EVF after you take a shot. This can be very disorienting and slows you down, so I recommend turning image preview off.

The rear screen can be tilted up and down. That is a huge plus, especially with wide-angle lenses. You can put the camera to the ground or lift it above your head and still see what you're shooting. You can also use the tiltable screen to shoot medium-format style with a waist-level viewfinder. The rear screen has great color and resolution as well, it's just lovely to use.

The glass on the rear screen seems to be rather fragile. After two months of use, I've got two hairline scratches on mine. Buy a screen protector and install it in a dust-free place. Otherwise you'll have to buy an another one, ha! The camera is not weather-sealed either, be careful with it. Other than the rear screen glass and the battery cover, the build quality of the body is good. It feels very solid.

The A7 has some weird flaring, probably due to the sensor design. Bright lights create green and purple flare outwards from the center of the frame. This might improve center sharpness for backlit shots, but for night time shots with a streetlight in the corner of the frame it's rather ugly.

One nice feature of the A7 is the ability to turn on APS-C size capture. This lets you use crop-factor lenses on the A7. And it also lets you use your full-frame lenses as crop-factor lenses. For instance, I can use my 55mm f/1.8 as a 83mm f/2.5 (in terms of DoF, in T-stops it's still T/1.8, i.e. has the same amount of light hitting each photosite). I lose a stop of shallow DoF and a bunch of megapixels but get a portrait lens that uses the sharpest portion of the frame.

Speaking of lenses, my lineup consists of the kit lens (a 28-70mm f/3.5-5.6), the Zeiss FE 55mm f/1.8 and an M-mount 21mm f/1.8 Voigtländer, mounted using the Voigtländer VM-E close focusing adapter. If I use the primes with the APS-C trick, I can shoot at 21mm f/1.8, 32mm f/2.5, 55mm f/1.8 and 83mm f/2.5. Wide-angle to portrait on two lenses!

The Zeiss FE 55mm f/1.8 is an expensive, well-built lens that looks cool and makes some super sharp images. Sharp enough that for soft portraits I have to dial clarity to -10 in Lightroom. Otherwise the texture of the skin comes out too strong and people go "Oh no! I look like a tree!" Which may not be what you want. If you shoot it at f/5.6 or so, the sharpness adds a good deal of interest in the image. It's a sort of hyperreal lens with the extreme sharpness and microcontrast.

The Voigtländer Ultron 21mm f/1.8 is an expensive, well-built lens that looks cool and makes sharp and interesting images, thanks to the 21mm focal length. Think portraits with environment, making people look like giants or dramatic architectural shots. It's manual focus and manual aperture though, so you'll get a lot of exercise in zone focusing and estimating exposure. The Ultron is a Leica M-mount lens, so you need an adapter to mount it onto the A7. One minus on the lens and the adapter is that they're heavy. Size-wise the combo is very compact, but weighs the same as a DSLR lens at 530g.


For a wide-angle lens, the Ultron's main weakness is its 50 cm close focus distance. But on the A7, you can use the VM-E close focus adapter and bring that down to 20 cm or so. Which nets you very nice bokeh close-ups. But blurs out infinity, even when stopped down to f/22.


The Voigtländer on the A7 has some minor purple cast on skies at the edges of the frame. It can be fixed in Lightroom by tweaking purple and magenta hues to -100. The lens has a good deal of vignetting, which is fixable with +50 lens vignetting in Lightroom. When fixing vignetting at high ISOs, take care not to blow the corner shadows into the land of way too much noise.

The kit lens is quite dark at f/5.6 on the long end, and it isn't as sharp as the two primes, but it's quite light and compact. And, it has image stabilization, so you can get decently sharp pictures even at slow shutter speeds. Coupled with the good high-ISO performance of the A7 and the auto-focus light, the kit lens is usable even in dark conditions. I haven't used it much though, I like shooting the two primes more.

Battery life is around 400 shots per charge. Not good, but I manage a day of casual snapping. The battery door seems to be a bit loose, I've had it open a couple of times when I took the camera out of the bag.

The camera has a built-in WiFi that the PlayMemories software uses for transferring images to your computer or smartphone. You can even also tethered shooting over WiFi but I haven't tried that. Transferring RAW photos from the camera to a laptop over WiFi is very convenient but quite slow. Transferring JPEGs to a smartphone is fast, though you want to batch the transfers as setting up and tearing down the WiFi connection takes about 20 seconds. When you're shooting, you probably want to switch the camera to airplane mode and save batteries.

I ended up shooting in RAW. I couldn't make the JPEGs look like I wanted out of the camera, so hey, make Lightroom presets to screw with the colors. The RAWs are nice! Lots of room for adjustment, good shadow detail, compressed size is 25 megs. You have to be careful about clipping whites (zebras help there), as very bright whites seem to start losing color before they start losing value. The RAW compression supposedly screws with high-contrast edges, but I haven't noticed the effect.

Due to shooting primarily in RAW, I find that I'm using the A7 differently from my previous cameras. I used the Canon 5D Mk II and my compact Olympus ZX-1 as self-contained picture making machines. I shot JPEGs and did only minor tweaks in post, partially because 8-bit JPEGs don't allow for all that much tweaking. The A7 acts more as the image capturing component in a picture creation workflow. Often the RAWs look severely underexposed and require a whole lot of pushing and toning before the final image starts taking shape. The colors, contrast, noise reduction and sharpening all go through a similar process to arrive at something more like what I want to see. I quite like this new way of working, it requires more of me and further disassociates the images from reality.

The camera has 16:9 cropped capture mode, which is nice for composing video frames. The 16:9 aspect also lends the photos a more movie-like look. The resulting RAW files retain the cropped part of the 3:2 frame, so you can de-crop the 16:9 photos in Lightroom. Note that this doesn't apply to the APS-crop frames, the APS-size photos don't retain the cropped image data.

I found that I can shoot usably clear (read: noise doesn't completely destroy skin tones) photos at ISO 4000, which is really nice. ISO 8000 and up start losing color range, getting magenta cast in shadows and light leaks when shooting in dark conditions. Still usable for non-color-critical work and very usable in B/W.

Overall, I don't really know why I got this camera and the lenses. I shoot too few images and video to justify the expense. The quality is amazing, yes. And it's not as expensive as a Canon 5D Mk III or a Nikon D800 either. But it's still expensive, especially with the Zeiss FE lenses. I feel that it's a luxury for my use.

The Sony A7 is too large and heavy for a walk-around camera strapped across your back and too non-tool-like for a total pro camera. I was trying to replace my old Canon 5D Mk II with its 50 f/1.4 and 17-35L f/2.8 with something lighter. But this is not it. The A7 has great image quality but the workflow is slower. And while it's light enough to throw in the backpack and go, it's still large and heavy enough to make me not want to dig it out. For me it sits in the uncomfortable middle area. Not quite travel, not quite pro. For occasional shooting it's great! Especially if you get a small camera bag.

2014-02-17

Saving out video frames from a WebGL app

Recently, I wanted to create a small video clip of a WebGL demo of mine. The route I ended up going down was to send the frames by XHR to a small web server that writes the frames to disk. Here's a quick article detailing how you can do the same.


Here's the video I made.

Set up your web server

I used plain Node.js for my server. It's got CORS headers set, so you can send requests to it from any port or domain you want to use. Heck, you could even do distributed rendering with all the renderers sending finished frames to the server.

Here's the code for the server. It reads POST requests into a buffer and writes the buffer to a file. Simple stuff.

// Adapted from this _WikiBooks OpenGL Programming Video Capture article_.
var port = 3999;
var http = require('http');
var fs = require('fs');
http.createServer(function (req, res) {
    res.writeHead(200, {
        'Access-Control-Allow-Origin': '*',
        'Access-Control-Allow-Headers': 'Content-Type, X-Requested-With'
    });
    if (req.method === 'OPTIONS') {
        // Handle OPTIONS requests to work with JQuery and other libs that cause preflighted CORS requests.
        res.end();
        return;
    }
    var idx = req.url.split('/').pop();
    var filename = ("0000" + idx).slice(-5)+".png";
    var img = new Buffer('');
    req.on('data', function(chunk) {
        img = Buffer.concat([img, chunk]);
    });
    req.on('end', function() {
        var f = fs.writeFileSync(filename, img);
        console.log('Wrote ' + filename);
        res.end();
    });
}).listen(port, '127.0.0.1');
console.log('Server running at http://127.0.0.1:'+port+'/');

To run the server, save it to server.js and run it with node server.js.

Send frames to the server

There are a couple things you need to consider on the WebGL side. First, use a fixed resolution canvas, instead of one that resizes with the browser window. Second, make your timing values fixed as well, instead of using wall clock time. To do this, decide the frame rate for the video (probably 30 FPS) and the duration of the video in seconds. Now you know how much to advance the time with every frame (1/FPS) and how many frames to render (FPS*duration). Third, turn your frame loop into a render & send loop.

To send frames to the server, read in PNG images from the WebGL canvas using toDataURL and send them to the server using XMLHttpRequest. To successfully send the frames to the server, you need to convert them to binary Blobs and send the Blobs instead of the dataURLs. It's pretty simple but can cause an hour of banging your head to the wall (as experience attests). Worry not, I've got it all done in the below snippet, ready to use.

Here's the core of my render & send loop:

var fps = 30; // Frames per second.
var duration = 1; // Video duration in seconds.

// Set the size of your canvas to a fixed value so that all the frames are the same size.
// resize(1024, 768);

var t = 0; // Time in ms.

for (var captureFrame = 0; captureFrame < fps*duration; captureFrame++) {
 // Advance time.
 t += 1000 / fps;

 // Set up your WebGL frame and draw it.
 uniform1f(gl, program, 'time', t/1000);
 gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
 gl.drawArrays(gl.TRIANGLES, 0, 6);

 // Send a synchronous request to the server (sync to make this simpler.)
 var r = new XMLHttpRequest();
 r.open('POST', 'http://localhost:3999/' + captureFrame, false);
 var blob = dataURItoBlob(glc.toDataURL());
 r.send(blob);
}

// Utility function to convert dataURIs to Blobs.
// Thanks go to SO: http://stackoverflow.com/a/15754051
function dataURItoBlob(dataURI) {
 var mimetype = dataURI.split(",")[0].split(':')[1].split(';')[0];
 var byteString = atob(dataURI.split(',')[1]);
 var u8a = new Uint8Array(byteString.length);
 for (var i = 0; i < byteString.length; i++) {
  u8a[i] = byteString.charCodeAt(i);
 }
 return new Blob([u8a.buffer], { type: mimetype });
};

Conclusion

And there we go! All is dandy and rendering frames out to a server is a cinch. Now you're ready to start producing your very own WebGL-powered videos! To turn the frame sequences into video files, either use the sequence as a source in Adobe Media Encoder or use a command-line tool like ffmpeg or avconv: avconv -r 30 -i %05d.png -y output.webm.

To sum up, you now have a simple solution for capturing video from a WebGL app. In our solution, the browser renders out frames and sends them over to a server that writes the frames to disk. Finally, you use a media encoder application to turn the frame sequence into a video file. Easy as pie! Thanks for reading, hope you have a great time making your very own WebGL videos!

Addendum

After posting this on Twitter, I got a bunch of links to other great libs / approaches to do WebGL / Canvas frame capture. Here's a quick recap of them: RenderFlies by @BlurSpline - it's a similar approach to the one above but also calls ffmpeg from inside the server to do video encoding. Then there's this snippet by Steven Wittens - instead of requiring a server for writing out the frames, it uses the FileSystem API and writes them to disk straight from the browser.

And finally, CCapture.js by Jaume Sánchez Elias. CCapture.js hooks up to requestAnimationFrame, Date.now and setTimeout to make fixed timing signals for a WebGL / Canvas animation then captures the frames from the canvas and gives you a nice WebM video file when you're done. And it all happens in the browser, which is awesome! No need to fiddle around with a server.

Thanks for the links and keep 'em coming!

2013-12-28

Techniques for faster loading

Speed is nice, right? Previously fhtr.net was taking half a second to get to the start of the first WebGL frame. With a hot cache, that is. Now, that's just TERRIBLE! Imagine the horror of having to wait for half a second for a website to reload. What would you do, who could you tell, where could you go! Nothing, no one and nowhere! You'd be stuck there for 500 milliseconds, blinking once, moving your eyes a few times, blinking twice, maybe even thrice before the site managed to get its things in order and show you pretty pictures. Clearly this can not stand.

Fear not, I am here to tell you that this situation can be amended. Just follow these 10 weird tricks and you too can make your static HTML page load in no time at all!

OK, let's get started! First off, I trimmed down the page size some. By swapping the 400 kB three.js library for 1 kB of WebGL helper functions, I brought the JS size down to 8 kB. This helped, but I still had to wait, like, 350 ms to see anything. Jeez.

My next step in getting the page load faster was to make my small SoundCloud player widget load the SoundCloud sdk.js asynchronously and do a timeout polling loop to initialize the widget once the SDK had loaded. Now, that didn't help all that much, but hey, at least now I was in control of my destiny and didn't have to wait for external servers to get around to serving the content before being able to execute crazy shaders. I also inlined the little logo image as a data URL in the HTML to avoid that dreadful extra HTTP request.

To further investigate the reason for the slowness in the page load, I turned my eye to the devtools network pane. 'Lo behold, what a travesty! I was using synchronous XHR gets to load in the two fragment shaders. For one to start loading, the other had to finish. And they were both loaded by my main script, which was in a file separate from the HTML.

I didn't want to inline the JS and the shaders into the HTML because I don't have a build script ready for that. But I could still fix a few things. I made the XHRs asynchronous so that the shaders load in parallel. Then I moved the shader loader out of the script file into a small script inlined in the HTML. Now the shaders start loading as the HTML loads, similar to the main script file.

Timing the code segments a bit, I noticed that my code for generating a random 256x256 texture was taking ~16 ms. Not too long, but hey, way too long, right? So I moved that out of the main script file and inlined it into the HTML, after the shader-loading snippet. Now the texture is generated while the browser is downloading the shaders and the script file. This squeezed out a few extra milliseconds in the hot cache scenario. Later on, I stopped being stupid and used an Uint8Array for the texture instead of a canvas, bringing the texture creation time to 2ms. Yay! Now it's practically free as it takes the same amount to generate the texture as it takes to load in the scripts.

My other major delays in getting the first frame to the screen were creating a WebGL context (15 ms or so), compiling the shaders (25 ms a pop) and setting up the initial shader uniforms (7 ms per shader). To optimize those, I made the page compile and set up only the first visible shader, and pared down the initial uniforms so as not to overlap with the ones I set before every draw. That brought down my shader setup cost from 64 ms to 26 ms. As for the WebGL context setup, I moved it into the inline script, after texture generation, so that it overlaps with the I/O. Maybe it helps by a millisecond or so. Maybe not.

As for caching. I'm using AppCache. It downloads the whole site on your first visit and keeps it cached. On subsequent visits (even if you're offline!), you get the page served from cache. Which is nice. And a bit of a hassle, as AppCache is going to require some extra logic to update the page after you have downloaded a new version of it.

Well, then. What is the result of all this effort? Let me tell you. On a hot cache and a particularly auspicious alignment of the stars in the eternal firmanent of space, when I sit down in front of my iMac and hit reload, the first WebGL frame starts executing in 56 milliseconds. That's less than the time it takes you to move your eyes from the address bar to the page itself. It's still TOO SLOW because, as everyone knows, websites should load in less than a single frame (at 60Hz).

Furthermore, alas, in a new tab, setting up the page takes ~350 ms. And with a cold cache - who can tell - 650 ms or more. Therefore, the next step in this journey is to spend a few minutes switching the page from GitHub Pages to Amazon CloudFront and hopefully claw back a few hundred ms of I/O time.

Carry on, wayward soldier.

2013-12-26

Opus 2, GLSL ray tracing tutorial

A bit older shader on fhtr.net this time, from the time I was getting started with ray tracing. This one's path tracing spheres and triangles with motion blur. Might even work on Windows this time, but YMMV. I should get a test machine for that.

"So", I hear you ask, "how does that work?" I'm glad you asked! It's a common Shadertoy-style WebGL shader where two triangles and a trivial vertex shader are coupled with a delightfully insane fragment shader. In this case, the fragment shader takes the current time as its input and uses it to compute the positions of a few objects and the color of the background. The shader then shoots a couple of rays through the scene, bouncing them off the objects to figure out the color of the current pixel. And yes, it does all this on every single pixel.

Path tracing

The path tracing part is pretty simple. The shader shoots out a ray and tests it against the scene. On detecting a hit, the ray is reflected and tested again. If the ray doesn't hit anything, I set its color to the background color. If the ray does hit something and gets reflected into the background, I multiply the object color with the background color. If the reflected ray hits another object, I give up and leave the color black. To make this one-bounce lighting look less harsh, I mix the ray color with the fog color based on the distance traveled by the ray. For the fog color, I'm using the background color so that the objects blend in well with the bg.

Digression: The nice thing about ray tracing -based rendering is that a lot of the things that are difficult and hacky to do with rasterization suddenly become simple. Reflections, refractions, transparent meshes, shadows, focus blur, motion blur, adaptive anti-aliasing schemes, all are pretty much "shoot an extra ray, add up the light from the two rays, done!" It just gets slowwwer the more rays you trace.

And if you're doing path tracing, you're going to need a whole lot of rays. The idea behind path tracing is to shoot a ray into the scene and randomly bounce it around until it becomes opaque enough that further hits with light sources wouldn't contribute to the pixel color. By summing up enough of these random ray paths, you arrive at a decent approximation of the light arriving to the pixel through all the different paths in the scene.

For specular materials, you can get away with a relatively small amount of rays, as the variance between ray paths is very small. A mirror-like surface is going to reflect the ray along its normal, so any two rays are going to behave pretty much the same. Throw in diffuse materials, and you're in a whole new world of hurt. A fully diffuse surface can reflect a ray into any direction visible from it, so you're going to need to trace a whole lot of paths to approximate the light hitting a diffuse surface.

Motion blur

The motion blur in the shader is very simple. Grab a random number from a texture, multiply the frame exposure time with that, add this time delta to the current time and trace! Now every ray is jittered in time between the current frame and the next frame. That alone gets you motion blur, albeit in a very noisy fashion.

I'm using two time-jittered rays per pixel in the shader, first one at a random time in the first half of the exposure time, the second in the second half. Then I add them together and divide by two to get the final motion blurred color. It looks quite a bit better without totally crashing the frame rate. For high-quality motion blur, you can bump the ray count to a hundred or so and revel in your 0.4 fps.

Background color

I made the background by adding up a bunch of gradients based on the sun position and the plane of the horizon. The gist of the technique is to take the dot product between two directions and multiply the gradient color with that. Quick explanation: the dot of two directions (or unit vectors) is the cosine of the angle between them, its value varies from 1 at zero angle to 0 at a 90 degree angle and -1 at 180 degrees. To make a nice diffuse sun, take the dot between the sun direction and the ray direction, clamp it to 0..1-range, raise it to some power to tighten up the highlight and multiply with the sun color. Done! In code: bgCol += pow(max(0.0, dot(sunDir, rayDir)), 256.0)*sunColor;

You can also use this technique to make gradients that go from the horizon to the zenith. Instead of using the sun direction, you use the up vector. Again, super simple: bgCol += pow(max(0.0, dot(vec3(0.0, 1.0, 0.0), rayDir)), 2.0)*skyColor;

By mixing a couple of these gradients you can get very nice results. Say, use low-pow sun gradient to make haze, high-pow for the sun disk, a horizon-up gradient for the skydome, horizon-down gradient for the ground and a reversed high-pow horizon gradient to add horizon glow (like in the shader in the previous post).

Let's write a path tracer!

Here's a small walk-through of a path tracer like this: First, normalize the coordinates of the current pixel to -1..1 range and scale them by the aspect ratio of the canvas so that we get square pixels. Second, set up the current ray position and direction. Third, shoot out the ray and test for intersections. If we have a hit, multiply the ray color with the hit color, reflect the ray off the surface normal and shoot it out again.

vec2 uv = (-1.0 + 2.0*gl_FragCoord.xy / iResolution.xy) * vec2(iResolution.x/iResolution.y, 1.0);
vec3 ro = vec3(0.0, 0.0, -6.0);     // Ray origin.
vec3 rd = normalize(vec3(uv, 1.0)); // Ray direction.
vec3 transmit = vec3(1.0);          // How much light the ray lets through.
vec3 light = vec3(0.0);             // How much light hits the eye through the ray.

float epsilon = 0.001;

float bounce_count = 2.0; // How many rays we trace.

for (int i=0; i<bounce_count; i++) {
  float dist = intersect(ro, rd);
  if (dist > 0.0) { // Object hit.
    transmit *= material(ro, rd); // Make the ray more opaque.
    vec3 nml = normal(ro, rd);    // Get surface normal for reflecting the ray.
    ro += rd*dist;                // Move the ray to the hit point.
    rd = reflect(rd, nml);        // Reflect the ray.
    ro += rd*epsilon;             // Move the ray off the surface to avoid hitting the same point twice.
  } else { // Background hit.
    light += transmit * background(rd); // Put the background light through the ray and add it to the light seen by the eye.
    break;                              // Don't bounce off the background.
  }
}

gl_FragColor = vec4(light, 1.0); // Set pixel color to the amount of light seen.

Spheres!

Ok, let's get real! Time to trace two spheres! Spheres are pretty easy to trace. A point is on the surface of a sphere if (point-center)^2 = r^2. A point is on a ray (o, d) if point = o+d*t | t > 0. By combining these two equations, we get ((o+d*t)-center)^2 = r^2 | t > 0, which we then have to solve for t. First, let's write it out and shuffle the terms a bit.

(o + d*t - c) • (o + d*t - c) = r^2

o•o + o•dt - o•c + o•dt + dt•dt - c•dt - c•o - c•dt + c•c = r^2

(d•d)t^2 + (2*o•dt - 2*c•dt) + o•o - 2o•c + c•c - r^2 = 0

(d•d)t^2 + 2*(o-c)•dt + (o-c)•(o-c) - r^2 = 0

Ok, that looks better. Now we can use the formula for solving quadratic equation and go on our merry way: for Ax^2 + Bx + C = 0, x = (-B +- sqrt(B^2-4AC)) / 2A. From the above equation, we get

A = (d•d)t^2   // We can optimize this to t^2 if the direction d is a unit vector.
               // To explain, first remember that a•b = length(a)*length(b)*cos(angleBetween(a,b))
               // Now if we set both vectors to be the same:
               // a•a = length(a)^2 * cos(0) = length(a)^2, as cos(0) = 1
               // And for unit vectors, that simplifies to u•u = 1^2 = 1
B = 2*(o-c)•d
C = (o-c)•(o-c) - r^2

And solve

D = B*B - 4*A*C
if (D < 0) {
  return No solution;
}
t = -B - sqrt(D);
if (t < 0) { // Closest intersection behind the ray
  t += 2*sqrt(D); // t = -B + sqrt(D)
}
if (t < 0) {
  return Sphere is behind the ray.
}
return Distance to sphere is t.

In GLSL, it's five lines of math and a comparison in the end.

float rayIntersectsSphere(vec3 ray, vec3 dir, vec3 center, float radius, float closestHit)
{
  vec3 rc = ray-center;
  float c = dot(rc, rc) - (radius*radius);
  float b = dot(dir, rc);
  float d = b*b - c;
  float t = -b - sqrt(abs(d));
  if (d < 0.0 || t < 0.0 || t > closestHit) {
    return closestHit; // Didn't hit, or wasn't the closest hit.
  } else {
    return t;
  }
}

Demos

Right, enough theory for now, I think. Here's a minimal ray tracer using the ideas above.

float sphere(vec3 ray, vec3 dir, vec3 center, float radius)
{
 vec3 rc = ray-center;
 float c = dot(rc, rc) - (radius*radius);
 float b = dot(dir, rc);
 float d = b*b - c;
 float t = -b - sqrt(abs(d));
 float st = step(0.0, min(t,d));
 return mix(-1.0, t, st);
}

vec3 background(float t, vec3 rd)
{
 vec3 light = normalize(vec3(sin(t), 0.6, cos(t)));
 float sun = max(0.0, dot(rd, light));
 float sky = max(0.0, dot(rd, vec3(0.0, 1.0, 0.0)));
 float ground = max(0.0, -dot(rd, vec3(0.0, 1.0, 0.0)));
 return 
  (pow(sun, 256.0)+0.2*pow(sun, 2.0))*vec3(2.0, 1.6, 1.0) +
  pow(ground, 0.5)*vec3(0.4, 0.3, 0.2) +
  pow(sky, 1.0)*vec3(0.5, 0.6, 0.7);
}

void main(void)
{
 vec2 uv = (-1.0 + 2.0*gl_FragCoord.xy / iResolution.xy) * 
  vec2(iResolution.x/iResolution.y, 1.0);
 vec3 ro = vec3(0.0, 0.0, -3.0);
 vec3 rd = normalize(vec3(uv, 1.0));
 vec3 p = vec3(0.0, 0.0, 0.0);
 float t = sphere(ro, rd, p, 1.0);
 vec3 nml = normalize(p - (ro+rd*t));
 vec3 bgCol = background(iGlobalTime, rd);
 rd = reflect(rd, nml);
 vec3 col = background(iGlobalTime, rd) * vec3(0.9, 0.8, 1.0);
 gl_FragColor = vec4( mix(bgCol, col, step(0.0, t)), 1.0 );
}


Demo of the minimal ray tracer.

I also made a version that traces three spheres with motion blur. You can check out the source on Shadertoy.


Demo of a motion blur tracer.

Conclusion

If you got all the way down here, well done! I hope I managed to shed some light on the mysterious negaworld of writing crazy fragment shaders. Do try and write your own, it's good fun.

Simple ray tracers are fun and easy to write and you can get 60 FPS on simple scenes, even on integrated graphics. Of course, if you want to render something more complex than a couple of spheres, you're probably in a world of pain. I should give it a try...

P.S. I put together a bunch of screenshots from my ray tracing shaders to make a project pitch deck. You can check out the PDF here. The first couple screenies have decals and clouds painted in PS, the rest are straight from real-time demos.

2013-12-22

Opus

Cooking a bit with shaders on fhtr.net. You won't see the cube on Windows as ANGLE doesn't like complex loops very much. The clouds are made by raymarching fractional brownian motion (= hierarchical noise, p=currentPoint; d=0.5; f=0; octaves.times{ f += d*noise(p); p *= 2; d /= 2; }), with the noise function from iq's LUT-based noise (and a dynamically generated noise LUT texture using the power of canvas).

I don't know where that's going. A cursed artifact, long dead. Flying neon beams lighting up the night, the last shards of the sun seeping into the iridescent tarsands. Powercrystals booming to life.

2013-10-03

PDF export tweaks

I did a small tweak to Message's Print to PDF -feature. It now uses the CSS @page { size: 1920px 1080px; } attribute on Chrome to make you nice full HD PDF slides, and falls back to normal printing on Firefox.

What I'd really like to see is a way to save the page to a PDF from JavaScript with something like window.print('pdf'). It'd open a "Save as PDF"-dialog to save a PDF of the current page. In a nutshell, it'd be a shortcut to press the "Save as PDF"-button in the print dialog, with print annotations turned off and defaulting to zero margins.

I created a Chromium issue on that: DOM API for Save as PDF.