Ink Part 2

Canny Edge Detection

The first stage for me is going to edge detection. There are many edge detectors, but I’m going to use the Canny Edge Detector, because I’m vaguely familiar with it and it’s quite well regarded.

Here’s the image I’m going to use for the initial development:

This handsome face *cough* is about to be melted down and turned into a bunch of squiggly lines. It was taken with my webcam in a partially darkened hotel room. On a Sunday. I also look a little shocked for some reason.

Step 1: Noise Reduction

To start with I’m going to blur my image. This might seem a bit counter intuitive, but it’s actually very helpful in cutting down the amount of noise in the picture. I mean, look at it. There’s randomly coloured pixels all over the place, due to poor lighting and a poor webcam. This kind of thing is going to cause interference in the various algorithms I’ll be using. I’ll show off why in a bit.

I’m going to use a very basic “box blur” kernel, not a fancy Gaussian one. Mainly because it’s easier and I’m lazy.

I’ll be using a 3×3 convolution kernel, like this:

Box Blur Kernel

Box Blur Kernel

This gets applied to an image by moving the centre of the kernel along each pixel in the image, and multiplying the kernel with the window in the image. The result is then summed.

Obviously, you have to deal with edge cases. I’ve taken the easy way out and _not_ dealt with them. I’m only blurring the pixels that are not at the very edge, which leaves a strip around the image that is unprocessed, one pixel thick. Visually, this doesn’t matter- I can leave it there or reduce the image size by 2 pixels in the x and y direction.

Original image has been desaturated and blurred, to try to cut down on the noise.

 

Step 2: Get Gradient Intensities:

I’m going to use a basic Sobel filter to perform edge detection. Sobel detects rapid intensity changes in a specific direction. In fact, you need a Sobel filter per direction:

Gx

This filter is applied in the same way as the box-blur filter described earlier. Here’s the output:

Gradient intensities in the horizontal direction

The test image with intensities in the Y direction calculated

These two results images are actually from before adding the box-blur filter. I’m doing this whole writeup in the wrong order.

The two gradient intensity images are summed together to get the final gradient result, using this very simple formula

|G| = |Gx| + |Gy|

That is a shortcut from doing the full equation of:

G = SquareRoot(Gx2 + Gy2)

This equation, applied to every pixel in both images, gives this result:

A basic gradient intensity image. By itself, this doesn’t do much, but I like to get tangible output from these algorithms. It’s a good visual lesson for what I just did.

That’s it for now. In the next post I do on this, I’ll be doing:

  • Edge direction calculations
  • Thin edge detection
  • Edge following

 

Overambitious Project: Ink

I’ve got that itch again. The one that sits at the back of my head going “you haven’t done any cool programming in ages, don’t you think you should?”

It’s an urge to go and work on a personal coding project, where I can experiment with stuff I find interesting, and can set my own pace and goals. This has become especially important recently, since I don’t even do any coding for my job anymore. I’m moving over to a managerial/everyman/”guy who knows stuff” role, so It’s not just a creative urge, it’s an urge to keep my “skills” sharp.

*Disclaimer: I’m not an amazing programmer, but I enjoy problem solving and I like to think I’ve built some cool stuff*

This project is going to be partly a rehash of my Computer Vision project at University – at least, it will use some of the same algorithms and technologies, and a large part of it will be about extracting a “useful” feature set from a set of images. What I do with that feature set is going to be very different though, and things I considered to be “useful” in my old project will probably be very different.

Here’s the abstract from my dissertation:

Reconstruction of 3D Models From 2D Images.
This project is about trying to recognise “interesting” features or points in two dimensional images, and then attempting to find corresponding features in a different image of the same scene. The coordinates of these points in different images can be used to generate 3D coordinates using Ullman’s theorem. This paper explorers a variety of options for detecting features, and several different methods for matching these features over different images. The variables used to generate and extract features are thoroughly tested, in order to find the best settings for the system. The results are then compared to the initial project specification to see if the system can operate as needed.

Now, this is not what I intend to do here at all. I just posted it anyway.

Right-

Ink, Project goal:

Write a program that will take video, and turn it into what looks like a series of hand-drawn ink sketches.

A still might look like this:

An ink drawing I found on the internet somewhere.

The animation should have enough flaws in it to make it seem hand-drawn, and it will probably have a much lower frame rate than the original video.

Part of my inspiration for this is Raymond Brigg’s “The Snowman”

 

I love the way the crosshatching works on this. The animation is low frame rate, the background is mostly static, but the movements are complex and the shading is wonderful.

 

So I want to achieve that programatticaly. You heard me. Any artist who reads this will probably tell me I’m removing all the soul from the animation, and they’d probably be right, but this is a project that I want to do.

Another inspiration is “A Scanner Darkly”, a film that I love for many, many reasons, not least because it’s adapted from a Phillip K Dick novel, and all adaptions of his work to film have been fantastic in their own ways. Even Total Recall is brilliant, but not for the same reasons Blade Runner is.

Well, maybe not.

But aside from the brilliant plot, (which, to be honest, was mostly about following stoners around. And paranoia. And government surveillance  And pharmacological conspiracies. And insanity. And psychotic breaks), it was produced in a unique way.

It was filmed normally, then every frame was redrawn partly by hand and then animated the rest of the way. Visually, it’s stunning, but the animation is so realistic that it creates a bizarre disconnect in your head while you watch it, and in places you can forget it’s animation at all. It very much suits the subject matter of the film.

 

And then there’s this clip, which I’m including for no reason other than the fact that it’s funny. Sort of:

 

Your sins will be read to you ceaselessly, in shifts, throughout eternity. The list will never end. 

So I want to try and build something similar to the system they used here. (They called it “rotoshop”, and they never released the program). It won’t be so fully-featured (damn, there’s an enterprise-y word. And another one! Auugh, what’s happened to my vocabulary?), because I will probably run into problems and get bored or frustrated. Besides, I’m not trying to accomplish the same thing, but I suspect some of the methodology will be similar.

So, here it goes.

We’re going to build a UAV

My friend and I want to build an unmanned aerial vehicle. Unlike the predator drones, it will not be armed with missiles.

This is an educational project for us; I’ve never done any proper “bare metal” coding before, or even very much hardware based stuff. Matt, however, is great at the hands on, practical side, but his coding is lacking.

Between us, we should have enough knowledge to fuck up in new and interesting ways.

I’m going to let Matt explain what kit we’ve got, what each bit of it is for, and how he’s fitting it together. I’m going to cover the coding side, and the maths, and how we’re going to get it to fly itself. Or Matt will probably cover that last part; he is doing an engineering degree. I’ll be taking the theory he gives me, and implementing it :).

To start with, we’ll be doing some “basic” stabilisation stuff. We’ll get the plane to try and stay as level as possible, automatically correcting itself except when recieving input from the remote.

Oh – and we don’t have a plane yet.

Perlin Noise

Shamus Young was messing around with Perlin Noise, which turned out to be normal noise. I dug around in an old VM to find this image:

Blue and white marble effect. Sort of.

The massive sphere in the background is procedurally textured with the Perlin Noise function, that is described in this paper: http://mrl.nyu.edu/~perlin/paper445.pdf

It’s not the best example I ever produced, but it’s the only build of my old raytracer that I could find. Shame on me for poor version control.

The source code for it, in Java, is on Ken Perlin’s website here: http://mrl.nyu.edu/~perlin/noise/

To be honest, it’s a very long time since I’ve looked at that stage of my ray tracer. I ditched the procedural generation of textures because they took way too long, but the basic idea is that you use the “noise” function, which takes X Y Z coords as arguements, and it returns the “next” noise point from the initial permutation list. Or something. These days I comment my own code a lot better, precisely because I can’t remember how this stuff works anymore.

When you’ve got the returned noise amount, you can do cool shit with it. I found a couple of functions around that could turn it into a marble style texture, but you can do a lot more with it.