Ink Part 2

Canny Edge Detection

The first stage for me is going to edge detection. There are many edge detectors, but I’m going to use the Canny Edge Detector, because I’m vaguely familiar with it and it’s quite well regarded.

Here’s the image I’m going to use for the initial development:

This handsome face *cough* is about to be melted down and turned into a bunch of squiggly lines. It was taken with my webcam in a partially darkened hotel room. On a Sunday. I also look a little shocked for some reason.

Step 1: Noise Reduction

To start with I’m going to blur my image. This might seem a bit counter intuitive, but it’s actually very helpful in cutting down the amount of noise in the picture. I mean, look at it. There’s randomly coloured pixels all over the place, due to poor lighting and a poor webcam. This kind of thing is going to cause interference in the various algorithms I’ll be using. I’ll show off why in a bit.

I’m going to use a very basic “box blur” kernel, not a fancy Gaussian one. Mainly because it’s easier and I’m lazy.

I’ll be using a 3×3 convolution kernel, like this:

Box Blur Kernel

Box Blur Kernel

This gets applied to an image by moving the centre of the kernel along each pixel in the image, and multiplying the kernel with the window in the image. The result is then summed.

Obviously, you have to deal with edge cases. I’ve taken the easy way out and _not_ dealt with them. I’m only blurring the pixels that are not at the very edge, which leaves a strip around the image that is unprocessed, one pixel thick. Visually, this doesn’t matter- I can leave it there or reduce the image size by 2 pixels in the x and y direction.

Original image has been desaturated and blurred, to try to cut down on the noise.

 

Step 2: Get Gradient Intensities:

I’m going to use a basic Sobel filter to perform edge detection. Sobel detects rapid intensity changes in a specific direction. In fact, you need a Sobel filter per direction:

Gx

This filter is applied in the same way as the box-blur filter described earlier. Here’s the output:

Gradient intensities in the horizontal direction

The test image with intensities in the Y direction calculated

These two results images are actually from before adding the box-blur filter. I’m doing this whole writeup in the wrong order.

The two gradient intensity images are summed together to get the final gradient result, using this very simple formula

|G| = |Gx| + |Gy|

That is a shortcut from doing the full equation of:

G = SquareRoot(Gx2 + Gy2)

This equation, applied to every pixel in both images, gives this result:

A basic gradient intensity image. By itself, this doesn’t do much, but I like to get tangible output from these algorithms. It’s a good visual lesson for what I just did.

That’s it for now. In the next post I do on this, I’ll be doing:

  • Edge direction calculations
  • Thin edge detection
  • Edge following

 

Overambitious Project: Ink

I’ve got that itch again. The one that sits at the back of my head going “you haven’t done any cool programming in ages, don’t you think you should?”

It’s an urge to go and work on a personal coding project, where I can experiment with stuff I find interesting, and can set my own pace and goals. This has become especially important recently, since I don’t even do any coding for my job anymore. I’m moving over to a managerial/everyman/”guy who knows stuff” role, so It’s not just a creative urge, it’s an urge to keep my “skills” sharp.

*Disclaimer: I’m not an amazing programmer, but I enjoy problem solving and I like to think I’ve built some cool stuff*

This project is going to be partly a rehash of my Computer Vision project at University – at least, it will use some of the same algorithms and technologies, and a large part of it will be about extracting a “useful” feature set from a set of images. What I do with that feature set is going to be very different though, and things I considered to be “useful” in my old project will probably be very different.

Here’s the abstract from my dissertation:

Reconstruction of 3D Models From 2D Images.
This project is about trying to recognise “interesting” features or points in two dimensional images, and then attempting to find corresponding features in a different image of the same scene. The coordinates of these points in different images can be used to generate 3D coordinates using Ullman’s theorem. This paper explorers a variety of options for detecting features, and several different methods for matching these features over different images. The variables used to generate and extract features are thoroughly tested, in order to find the best settings for the system. The results are then compared to the initial project specification to see if the system can operate as needed.

Now, this is not what I intend to do here at all. I just posted it anyway.

Right-

Ink, Project goal:

Write a program that will take video, and turn it into what looks like a series of hand-drawn ink sketches.

A still might look like this:

An ink drawing I found on the internet somewhere.

The animation should have enough flaws in it to make it seem hand-drawn, and it will probably have a much lower frame rate than the original video.

Part of my inspiration for this is Raymond Brigg’s “The Snowman”

 

I love the way the crosshatching works on this. The animation is low frame rate, the background is mostly static, but the movements are complex and the shading is wonderful.

 

So I want to achieve that programatticaly. You heard me. Any artist who reads this will probably tell me I’m removing all the soul from the animation, and they’d probably be right, but this is a project that I want to do.

Another inspiration is “A Scanner Darkly”, a film that I love for many, many reasons, not least because it’s adapted from a Phillip K Dick novel, and all adaptions of his work to film have been fantastic in their own ways. Even Total Recall is brilliant, but not for the same reasons Blade Runner is.

Well, maybe not.

But aside from the brilliant plot, (which, to be honest, was mostly about following stoners around. And paranoia. And government surveillance  And pharmacological conspiracies. And insanity. And psychotic breaks), it was produced in a unique way.

It was filmed normally, then every frame was redrawn partly by hand and then animated the rest of the way. Visually, it’s stunning, but the animation is so realistic that it creates a bizarre disconnect in your head while you watch it, and in places you can forget it’s animation at all. It very much suits the subject matter of the film.

 

And then there’s this clip, which I’m including for no reason other than the fact that it’s funny. Sort of:

 

Your sins will be read to you ceaselessly, in shifts, throughout eternity. The list will never end. 

So I want to try and build something similar to the system they used here. (They called it “rotoshop”, and they never released the program). It won’t be so fully-featured (damn, there’s an enterprise-y word. And another one! Auugh, what’s happened to my vocabulary?), because I will probably run into problems and get bored or frustrated. Besides, I’m not trying to accomplish the same thing, but I suspect some of the methodology will be similar.

So, here it goes.