art with code


Acceleration, 3

Oh right, right, I was working on this. Or something like this. Technology research post again :(
First I was working on this JavaScript ray tracer with BVH acceleration and voxel grid acceleration. And managed to get a 870k dragon loaded into it and rendered in less than 30 seconds... In quite unoptimized single-threaded JavaScript.

Then I had an idea and spent two weeks doing noise-free single-sample bidirectional path tracing. But it turns the high-frequency noise into low-frequency noise and runs too slow for realtime. I'll have to experiment more with the idea. Shadertoy here. Start with the writeup and screenshots after that.
 Single-sample bidirectional path tracing + hemisphere approximation

 Smooth indirect illumination by creating the incoming light hemisphere on the fly
    based on the hemisphere samples of the surrounding pixels.

 See also: Virtual point lights, bidirectional instant radiosity

 First pass: Create path suffixes = point with incoming ray of light.
  1. Shoot out primary ray and bounce it off the hit surface.
     2. Trace the first bounce and store its hit point, normal and material.
     3. Trace the rest of the path from the first bounce point and store the direction of the path.
  4. Store the amount of light at the first bounce point.

 Now you have a path suffix at each pixel that has enough info to connect any other path to it.

 Second pass: Connect the primary ray to path suffixes in surrounding pixels.
  1. Shoot out primary ray and calculate direct light for it
  2. Sample NxN samples around the pixel from the path suffix buffer
  3. Accumulate the light from the hemisphere. For each sample:
   3.1. Calculate the direction to it: hd = normalize(sample.p - primary.p)
   3.2. Accumulate light according to the BRDFs of the primary point and the hemisphere point.
   [3.3. Scale the light contribution with a pixel distance filter for smoother transitions]

 Extra passes: Create new path suffixes by connecting points in path suffix buffer to other path suffixes
  1. Primary ray hits can act as path suffixes for hemisphere points (and have nice geometric connection at same px coord).
  2. Hemisphere points can act as path suffixes for hemisphere points (but may lie on the same plane near px coord).
  3. Add light from new path suffixes to primary hits.

 Why this might be nice?
  - Get more mileage from paths: in scenes with difficult lighting conditions, the chances of 
   finding light are low for any given path. By summing up the contributions of 10000 paths, 
   you've got a much higher chance of finding light for a pixel.
  - Less noise: noise is variance in hemisphere sampling. If neighbouring pixels have similar 
   hemispheres, there's smooth variance and little noise.
  - If you have the budget, you can cast shadow rays to each hemisphere point and get correct lighting.
  - If you don't have the budget, you can use the hemi points as soft light and get blurry lighting
  - You can use the found hemisphere light to approximate a point's light distribution and guide your sampler.
  - You can use light samples from previous frames.

 What sucks?
  - If you don't cast shadow rays, you get light bleeding and everything's blurry.
   - Casting 1000 shadow rays per pixel is going to have an impact on performance
  - Direct lighting remains noisy 
   (you can use direct light as a hemisphere sample but it causes even more light bleeding w/o shadow rays)
  - This scene smooths out at 64x64 hemisphere samples per pixel, which is expensive to sample
  - Increasing the size of your sampling kernel increases inaccuracies w/o shadow rays
  - It seems like you trade high-frequency noise for low-frequency noise
  - Glossy stuff is hard

 The ugly:
  - Using three buffers to trace because no MRTs, otherwise could store hemi hit, normal, dir, primary hit, 
  hemi color, primary color on a single trace, and recombine on the second pass.
  - Slow SDF ray marcher as the rendering engine
  - Not storing incoming light dir atm, all the lighting equations are hacks in this demo


Acceleration, 2

Oh, yeah, right. I was working on this.

But got distracted by adding features to my path tracing Shadertoy. So. I've got pictures if nothing else. Pictures of the same scene to test bidirectional path tracing, bokeh, diffraction, etc. There you go.

Tune in next week for more of .. something?


Acceleration, 1

Working on Acceleration.

It's not fast going, but it's going bit by bit. I currently have some color pickers, auto-keyframing, save, load, hi(gher)-quality still render creation, on top of the very visual-oriented animation editor. There used to be a 4-view for moving things about but that felt clunky and the shader implementation wasn't great, so it's dormant for now.

Now I've been working on two workstreams: 1) event handling dataflow graph and 2) rendering research. Rendering research is going towards, uh, realtime bi-directional path tracing. Which might kill the whole thing due to "I don't know how to make an acceleration structure for triangle models", but at least I'll get cool screenshots out of it.

Event handling dataflow graph. It's one of those things. You know. You think that it'll just be some "on click, set variable Y to 20"-thing. And then you think about it and end up with some sort of loosely bound lazily evaluated array language execution graph with a query language to select objects. And then you start thinking "How would I build shaders with this?", "Could you run this in parallel?", "Should I compile this down into WebAssembly?"

In a word: utmost care must be taken to avoid rabbit holes that lead to endless destruction in the fiery magma caves under the Earth's crust.

Anyway. The event graph nodes. To execute a node, you first evaluate all its inputs. To evaluate an input, you need to find the object referred by the input object and resolve its value. Why? Passing objects by reference feels brittle. Like. If I've got a node with an input and I want to pass that input to another node (say, I want to modify the scale of the clicked object: OnClick(obj) -> ModifyScale(obj)). If I pass it by reference, the two nodes need to point to the same object. When OnClick's input's value changes, ModifyScale's input's value needs to change as well. And how do you draw it? How do you draw a line from OnClick's input to ModifyScale's input? You need to know that they are the same object, referred to from two different places, and figure out the coordinates for those two places. So a value needs to carry a reference to its render model, so that you can figure out where it's located. Or the value can be defined as a loosely bound address that's resolved at runtime "OnClick.inputs.Object" -> obj = graph.objects["OnClick"]; input = obj.inputs["Object"]; point = obj.renderModel.inputs["Object"].connectorPoint;.

Node {
  renderModel: Model,
  func: Function,
  inputs: {string: Value, ...},
  outputs: {string: Value, ...},
  futures: [Node] // array because if-then-else/switch-statements
  futureIndex: int // which future to follow

Maybe this is .. workable?

On the rendering research side of things, considering a few options. SDFs? Raytraced geometry? Simple primitives and/or triangle soup? Path tracing with a procedural environment map as the main light source? In real-time? Progressive renderer for high-quality stills. HTML elements overlaid on top of the 3D scene. Fancy SDF text that scales to millions of letters in realtime? 3D text meshes? Images, video, particles, what? What's the goal here? Build animations in 15 minutes. Make animation timelines that compose. Renderer to make cool-looking interactives with a unique look.

Right. Anyhow, rendering goals: nice motion blur, shiny CG look, high-quality stills, depth-of-field, glowy blooms, volumetrics. All of which point towards: "just path trace it". It'll impose definite limitations on the scenes that work alright on it. Maybe that's fine? The underlying timeline + event graph stuff should be generic enough to plug in a Three.js renderer. I wonder about the transformation widgets, animation 3D paths, and other "way easier to rasterize"-stuff though. So, rasterize those on top of the scene. The path tracer can write to depth buffer with the primary rays too. Hybrid renderer!

It's complex. Do it piece by piece. Make it simpler until it's possible.

Part 2 on 10th of July. Goals: event graph prototype working.


Electrostatic fluid accelerator

Imagine a fan with no moving parts. Instead of using spinning blades to push air molecules around, you use ions accelerated by an electric field. This, in a nutshell, is an electrostatic fluid accelerator.

(Thinking of the Dyson fans that create a thin high-velocity flow to entrain a larger air volume and pull it along. Now they have a small fan at the bottom to push air through the exhaust. Which makes a whirring noise. What if you replaced the fan with an electrostatic fluid accelerator set around the exhaust.)


Engineering antibiotic-suspectible strains of bacteria

Make antibiotic resistance into a detriment. Petri dish with antibiotic gradient. Radiation steriliser at the high antibiotic section. Microbes that survive in the antibiotic get killed by the radiation. Microbes without resistance survive and supplant the resistant strain.


Latest 8 rumors

8 is going to have three cameras on the back. One with a 16-35 mm equivalent lens, one with a 24-70 mm lens, and a third one with a 70-200 mm lens.

8 uses the input from each of the lenses to stabilize images. The images you see on the 8 screen are going to be extremely stable. By integrating the optical flow from the cameras with the accelerometer readings and the pupil tracking from the forward-facing selfie cam, 8 can make the text you're reading compensate for the motion of the phone, so that the text appears stable and readable even when you're out running. The 8 also compensates for the distance of the phone from your eyes by increasing text size when the phone is further away. With computational reconvolution algorithms and advanced glasses-detecting computer vision techniques, the 8 can re-focus the phone screen so that no matter you're wearing your glasses or not, the 8 looks 20/20.

8 comes with an extendable screen. By using the side-mounted magnetic connectors, you can clip several 8 devices together to expand the size of your screen as far and wide as you want. The UI automatically expands to fill all available devices and switches from phone mode to tablet mode to PC mode to TV mode as the screen grows.

8 has three pico-projectors: one on the back and two in the front. The projector on the back of the phone creates an interactive tablet surface. The image is projected by the projector, and the back cameras can track your hands to make a virtual touch screen. The camera sees the color and the shape of the surface you're projecting on, automatically correcting the colors and the shape of the projection so that the image appears in the right colors and the right shape. You can use the back projector to play Kinect-style games on the wall, watch movies off the seat in front of you, display photos and documents on the meeting room desk. The front projectors track your pupils and plunge you into glassless virtual reality. The front projectors are also used to project a selfie-boosting projection field on your face that smooths out your skin, eliminates wrinkles, enlarges your eyes and slims your cheeks.

8 has 4 GB of HBM2 RAM stacked on top of a heterogenous processing die, composed of a single 4 GHz core, three 1.5 GHz cores, and 128 tiny graphics cores running at 1 GHz. The total bandwidth available for the computing cores is 256GB/s, and they can crank out 1.1 TFLOPS (single precision, ARM NEON 128-bit fused mul-add), in an address space shared between all the cores. Backing the compute cores is a two-level storage subsystem with 256 GB Flash providing 2.5 GB/s bandwidth, and a high-speed 4GB DDR pre-populated cache in front of it, serving frequently accessed data at 20 GB/s. The 8 has a lot of numbers. It comes with a worksheet-style calculator to help you manage them.

8 comes with a deep learning AI stack, used by the 8 to help manage your life. From simple things like running a spam filter on your notifications, to advanced life management techniques for building high-productive schedules, writing emails faster, researching projects and keeping you fit, the 8 puts a whole team of AI assistants at your service. With the Fake News Blocker built into the 8 browser, you won't be hoodwinked by rogue AIs and shady operators. With the 8 Scalable Helper assistant, you can provide a fully automated AI service to others and get paid for each Bit of Assistance provided.

The AI assistant team makes you a full member in today's society. With its Opinion Optimizer feature – it summons a hard-working team of virtual commenters who take your side – you can participate in online discussions as an equal opponent to other fully-armed members of the sociosphere. In case you need that extra bit of help persuading someone, the News Dreamer feature comes fully equipped with all the latest news-generating systems. Late from work? No problem! The News Dreamer can create A/B tested news stories about traffic jams and bridge closures, ready to be presented as alternative facts.

Another exciting feature of the 8 is the built-in Government app. The Government app polls you daily for your input on matters local and global. By participating in the Government, you help discover issues, find solutions for them and take part in implementing the solutions.

By buying the 8, you become a shareholder in the company that runs the Government and the Scalable Helper. As a shareholder, you'll receive a quarterly dividend from the profits generated by the company.

8 has a scan button on its side. By pressing the scan button, the camera view pops up on the screen. With the scan app, you can read QR codes, pay bills, scan business cards, save notes, translate text, see navigation directions overlaid on the view, and pull up information about the things you see. If a QR code is a link to a 3D object, the scan app loads in the object and shows it in augmented reality. You can also use the scan app to contribute information about the things that you see.

The 8 has wireless charging. Using beamforming, the charger aims a supercharged WiFi signal at the 8 with enough power to charge it. While not as fast as cable charging, the wireless charger can top the 8 to full charge in around 4 hours, and works anywhere in a 5 meter radius to the charger.


Healthcare spending

Healthcare is like food: both have a known average lifetime usage. For food, that's the nutrition required by a person over their lifetime. For healthcare, it's the preventive care and restorative care a person needs over their lifetime. Both have pretty clear limits: you can't really eat a whole lot more than you need, and you can't really use a lot more healthcare than is required.

The big difference is the spending pattern. You need to eat every day and pay for your food every day. But healthcare needs tend to come in bursts. You may have a week where you need more healthcare than in the last ten years added up.

For an individual, dealing with healthcare-like burst spending would require either big savings, an insurance system, or a credit system. A buffer to absorb the costs.

At society level, healthcare spending is smoothed out, and looks a whole lot like food spending. You have a stable level of healthcare spending every day, and can budget for it just like for food. Every day, everyone pays to match the required spending for the day. If you don't like being healthy and paying for other people's healthcare, you're free to break your arm and enjoy the care too. Healthy people don't get sick for fun. If you think the payment is too much, make the society more healthy and lower the costs.

Healthcare is also more of a society-level problem. A person in poor health is going to bring down the health of the people around him. A person in poor health is less able to repay the care given to him. If you make amount of healthcare provided depend on the ability to repay, you'll end up with more people in poor health, which will make the people around them less well as well.

About Me

My photo

Built art installations, web sites, graphics libraries, web browsers, mobile apps, desktop apps, media player themes, many nutty prototypes, much bad code, much bad art.

Have freelanced for Verizon, Google, Mozilla, Warner Bros, Sony Pictures, Yahoo!, Microsoft, Valve Software, TDK Electronics.

Ex-Chrome Developer Relations.