Category Archives: Video

"Eclipse" music video

What does the future hold for storytelling — can a machine create cinema?

The release of the trailer for Morgan provides further insight that Artificial Intelligence (A.I.) is creeping slowly onto the creative stage. After initial progress was made in 1996 when IBM’s Deep Blue beat World Chess Champion Gary Kasparov, IBM’s Watson then beat human champions in Jeopardy, and more recently Google’s DeepMind conquered the ancient Chinese game of Go. Google Photos generates videos (with music) automatically from images on my phone. It’s becoming obvious that deep strategic thinking is at least possible using machines.

So, can a machine create a video narrative? Could we tell the difference?

The unfortunate fact is, of course, that the Morgan trailer is hollow and poorly-paced (even with the help of an “IBM filmmaker”), and the musicians behind the AI-directed “Eclipse” music video have distanced themselves from the end product.

Looking deeper into each of these projects, they still required a human hand to direct/collate/guide the machine — it’s a ground-up approach to AI, so there’s no “Watson Video Editing Software” on the market.

However, the building blocks have already been created — the Google search engine uses natural-language processing, and Wolfgram Alpha accepts commands in basic English. We now have (pretty good) automatic web summaries and headline analysers. There’s a reason why Google’s Principal Filmmaker Jessica Brillhart thinks Zork’s language processing will heavily influence the future of VR.

It seems that although we probably have a while to go before creativity is realistically threatened in any way, most people won’t care if something has been created by computer. For example, much of the in-house promotions we currently see on TV channels are packaged in a way that wouldn’t require human intervention — so perhaps it might not be that long after all (for specific situations).

So a machine can assemble a video (I even hesitate to use the words ‘edit’ or ‘direct’). But not very well — at least not yet. However, as Linguistics expert Noam Chomsky said, perhaps we are even asking the wrong question:

“Thinking is a human feature. Will AI someday really think? That’s like asking if submarines swim. If you call it swimming then robots will think, yes.” 

via ibm.com and bbc.com

The Rise of the Diegetic Intertitle

Prior to the integration of sound, movies often displayed text-based information using title cards or intertitles. This form of communication is known as diegetic content, as the actors cannot see it.

However, even after the invention of the talkie, other types of information needed to be displayed — such as translations for a foreign audience. This was usually done in a very perfunctory way, with non-descriptive text (typically achieved using a font such as Times New Roman, with a black outline to contrast with any background) on the lower third of the screen. This text is external to the story, so it seemed natural that it should be stylistically different.

In more recent years, there are increased demands on visual storytelling — small screen devices (e.g. mobile) and computers have started to become part of the language of cinema (and, by extension — the video and digital screens on which they are ‘projected’). Additionally, in today’s more multicultural world, the requirement to show multiple languages within the same film means that different typographic techniques can be used to enhance this aspect of the story. In fact, this is an extension of traditional subtitles — where often sound effects and untranslated languages are still included.

As mobile and internet technology started to appear on screen, an editor would typically cut to a shot of the device — allowing the viewer to read the display. As post-production technology improved, TV’s requirement for an increased speed of plot exposition, and product placement costs (and legal clearances) required a more generic approach, this eventually evolved to show the interface directly incorporated onto the visual frame.

Subtitles, captions and interface design typically sits independently on top of the content as a layer added in post-production — i.e. as a semi-transparent wall between the story and the viewer. Integrating these titles to make them appear part of the content can be quite a technical challenge — especially when they need to be tracked to a moving camera.

This overlaying technique was demonstrated in movies such as Man On Fire (2004), Stranger Than Fiction (2006), Disconnect (2012), and 2014’s The Fault In Our Stars, John Wick, and Non-Stop, as well as TV shows such as (perhaps most influentially) Sherlock (2010) and House of Cards in 2013.

There are two main types of elements in modern cinema: diegetic — anything that the characters would recognise happening within their world (of the narrative story) and non-diegetic — anything that happens outside the story (for example, this would usually be opening credit sequences).

However, (much like modern media itself) on-screen typography has surpassed merely being integrated visually into the background plate. It is now becoming increasingly self-reflexive, and blurs these diegetic lines. This is often referred to as “breaking the fourth wall”, and is perhaps best demonstrated in the opening titles to the 2016 film Deadpool, where even the actual names of producers are subverted into narrative elements.

For more exegesis of the diegesis (sorry, I couldn’t help it), see Tim Carmody’s excellent 2011 SVA Interaction Design presentation “The Dictatorial Perpendicular: Walter Benjamin’s Reading Revolution”.

“A fourth wall break inside a fourth wall break? That’s like, sixteen walls.” — Deadpool

Masters of Videomontage

Some of the most fascinating video animators I’ve ever seen — Cyriak (Brighton, UK), Fernando Livschitz (Buenos Aires, Argentina) and Till Nowak (Hamburg, Germany). Using found footage and masks, they create a surreal and often disturbing view of reality.

As mentioned in the ‘Heroes of Animation film’, Cyriak sees this style as a natural evolution of the Terry Gilliam school — taking photographic elements and moving them in unexpected ways. I would go further and in that it takes Russian Constructivist fine-art photomontage to a natural conclusion.

We are so used to amateur camcorder and mobile films these days that this approach seems to transcend animation — and we are drawn into their world.

We are so used to amateur camcorder and mobile video these days that this approach seems to even transcend animation — and we are drawn into their world. So much so that The Institute for Centrifugal Research seems to be (remotely) plausible.

And here’s how it’s done.

This profile of Cyriak includes a history of his work, and a demonstration of his process. This behind-the-scenes video from The Centrifuge Brain Project shows the CGI overlaid over the source footage, and this After Effects tutorial explains the basics, using a locked-off camera (then you can add natural camera movement afterwards).