Tuesday, 13 September 2016

Coding Geometric Images and Videos

There is something compelling about simple geometric images. Something that encourages me to study them by writing short computer programs to generate them.

Adding simple geometric motion to these images  adds another level of interest.

Here I'll talk about my methods for making images and videos and give links to tools for anyone who might want to explore these ideas themselves.

This image is a simple black quadrilateral. It is easily constructed with most drawing packages, where one creates the outline of the shape on the screen by selecting (i.e. clicking on) the four corners and then filling the shape with colour (here, black).

Shapes like that can be composed easily on the screen using direct manipulation, so making something like a collage of geometric shapes is easy. The only learning curve is getting used to the shapes and brushes provided by the drawing package and gaining some experience with picking colours on the screen that will look good when printed or projected.

So why, if you can do it so easily in the drawing application, would one want to write code to do it instead? Especially since learning to code is going to be a lot harder than learning to drive direct on-screen manipulation tools.

There are probably two answers to this question.

The first answer is that coding gives you a great deal more control over your images and is particularly important if you need to generate lots of similar images, which is what you will want to do to create a video.

The second answer, in my opinion, is that coding adds considerably to the variety that you can achieve, doing things that are not supported by the drawing package. One obvious example of this is when we want to blend two images to produce one. Your drawing package will supply many blend modes. But coding will enable you to provide blend modes that no-one else has and this is one way to distinguish your artwork from the usual.

And coding isn't too difficult if you choose the right tools.

For artists, the right programming tool is probably "Processing" (see Processing.org). This is because it's free, it's well documented, widely-used and very well supported. It is also a beautiful tool and an easy language to learn. Right out of the box it is set up to assume you are going to draw a picture. Drawing our black quadrilateral, while not the first shape one might try, is still something that requires only a tiny effort.

Where there might be some difficulty for artists, over the direct manipulation drawing package is the need to make use of some elementary mathematics (co-ordinates, for example).

But this should not be a deterrent. Rather it is an opportunity. Maths was hard at school because it was boring. It was unmotivated. That isn't the case when you are trying your hardest to create an image that is clear in your mind, just not yet down on paper or the screen. That motivation will be all you need to recall the maths you think you have forgotten or never felt you understood in the first place (even though you passed the exam).

Coding comes into its own when you want to experiment. When you know the kind of image you are after but when you come to look at it, it's not quite as it was in your imagination.

Since the parameters of your shapes appear in your code as numbers, it is a simple matter to try lots of alternatives in a very short period of time, until one arrives at an image which does satisfy.

Moreover, it is a straightforward matter to create a whole series of related images which come from the same code but with different parameters. For example, a collage of shapes of different colours. The parameters might be the location of the shapes and the choice of colours. You could fill a wall with a series of this sort, where the "system" of production may well be applauded as itself an artistic statement.

Coding comes even more into its own when we begin more advanced experimentation by, for example, combining existing images by blending them to produce visual effects not readily achieved using drawing packages, or even established printing techniques.

For example, sticking with our monochrome example, the image below is a blend of two separate quadrilaterals.

The innermost (black) quadrilateral is generated by the same code as generated the outer (white) quadrilateral, then these two images are superimposed on a black background.

Or that is how it seems. The truth is rather simpler. The larger (white) quadrilateral is, in its original image, black on white, just like the image at the head of this article. So too is the smaller (black) quadrilateral. They are composed (superimposed, blended) using a mathematical operation that makes the final image black only where the original images are the same shade. So, where the white background of the smaller quadrilateral overlapped the outer black portion of the larger quadrilateral, this is now white because the two shades in the original were not the same.

At some stage, whether using a drawing package or by coding an image yourself, you will need to understand the difference between pixels and vectors as the means of representing the image that you have created.

Basically, vectors are a little more restricted but infinately scale-able without error, while pixels are more versatile (we use them for digital photography, for example) but will not scale up very well because they will not have the detail that is required to scale to arbitrary size.

For me the choice of vectors or pixels comes down to whether I might want to reproduce my images in a wide range of sizes (such as one does with type faces) or whether I am worried about file sizes (vector files can be tiny). Usually I am not concerned with that. Processing, by the way, supports both vectors and pixels.

Let us look, finally, at geometric videos, which has been after all my motivation for writing this piece. Look at this example

What we have here is the same collage as the still images but this time the quadrilaterals are deforming over time in a systematic way. Each of the eight corners actually traces a simple closed curve over a period of 12 seconds. The slowest moving corner completes one such perambulation in exactly 12 seconds. The other corners complete three or four perambulations in the same time. The whole sequence repeats after 12 seconds.

The frame rate of the video is 30fps, so that if this were composed from individual images the whole loop would require 360 distinct images.

Now one could generate those 360 images by hand using a direct manipulation drawing package, editing each image to move the eight corners and saving each as a separate file. Many applications will readily combine those files to make a video. And doing it by hand like that would create a degree of humanisation difficult to reproduce in coding.

But doing it with coding is a lot easier. A lot, lot easier. Assuming you can do the maths.

And you can do it all in Processing. You write the code that produces a single frame, where the eight corner locations are determined by a formula that is parametrised with the time or frame number. The program then saves each of the 360 images to a separate file (a jpg, for example) and when the final one has been saved you combine them into the video. Processing itself will do this for you, but so will many other applications, which you may prefer.

I use Processing a lot for experimentation, because of its ease of use. Now that machines are bigger and faster, Processing is powerful enough to do everything you could want to make production quality video suitable for gallery projection or 4K presentation on a TV screen.

That is not to say that one should do for oneself (through coding) those things that other applications are good at. I use iMovie (or FCP) or their Windows equivalents for combining clips and for adding transitions and titles. You don't usually need to be original in those aspects, so using a standard tool is sensible.


I also us ffmpeg (ffmpeg.org) for more detailed technical aspects of video, such as changing the frame rate, changing the container, separating video and audio streams etc. and for some more technical types of blending that it is well suited to (one might say, designed for).

Also, because it is something that I know well, I use Python (python.org) for much of what I have suggested you use Processing. Python is a much larger language than Processing and correspondingly takes longer to master.

Python is a general purpose language, rather than being specialised to image processing so much of the documentation relates to stuff scientists need rather than stuff artists need. That said, it is useful stuff. In particular, Python supports a style of programming that I use (functional programming) much more fully than Processing and some of the scripts that I have developed make use of that fuller implementation.