‘’The texture of the digital image through the control of sharpness. A real artistic issue’’ by Philippe Ros afc V7

Special thanks to Kommer Kleijn, SBC and Rolf Coulanges, BVK for their precious help. This long version uses Ron Prince’s transcription for the British Cinematographer issue (#68). Thanks to him, to Alan Lowne and to the BC magazine!

The texture of the digital image is a relatively virgin territory, and sharpness plays an important part in it. Choosing the texture and the sharpness are definitively artistic decisions, but in most of the image delivered by the camera, these decisions are essentially created and managed by engineers with talent.

>But these choices don’t necessarily match with the filmic drama nor the filmmaker’s and cinematographer’s expectations.

Historically, in film, the texture of the image has mostly been tackled during the shoot by lighting, make-up, glass filtering, glued stockings on the back of the lenses and by the level of grain linked to film stocks, exposure and lab processes. We can easily feel that the film texture is linked to the chaotic pattern of the film stock as opposed to the still, clinical pattern of the digital image. But is it that simple? The digital era offers many opportunities, but presents also paradoxes and limitations I will try to outline.

Main concerns - 2K and 4K processes.

The issue surrounding texture ans sharpness follows several shoots, and a lot of tests, that I did in 4K with Sony F65, F55, Canon C500 and 1Dc cameras, as well as several workshops I conducted, including one with cinematographers on 4K and Ultra HD in South Korea. During last year’s Camerimage Film Festival, I was also involved in a seminar organised by Tiffen with Steven Poster, ASC. The debate focused on how to soften the 4K digital image.

4K is not often used in Europe but I came to the conclusion that the qualities and the defects we see in 4K are very often the exacerbation of what you start to feel in 2K.

And even if we are not working for a final 4K release we are increasingly using 4K cameras for a 2K final release. Hence the value to fully understand the peculiarities of 4K. In Micro salon 2015 I made a presentation with screening of filter tests and extracts of films using these methodology, shooting in 4K for a 2K exhibition.

Cinematographers frequently now use Sony F55, Red Epic, Canon C500, Black Magic cameras at 4K, with the Panansonic Varicam 35 camera next on the block. And during postproduction we downscale the footage from these cameras to 2K.

Whilst all of these cameras deliver 4K resolution, the question is do they offer the same sharpness ?

As well as 4K give the opportunity to appreciate the peculiarities of the lenses, 4K reveal the specificities of the texture and particularly of the sharpness of camera and lenses. Many parameters are at stake and we are entering in a very subjective area.

>The semantical confusion [1] between sharpness, and resolution don’t help to find his way in the maze.

Specifications of 4K cameras.

It is difficult to find our bearings in the world of 4K cameras, particularly because of the confusion between photosites and pixels. Pixels appear only during sub-sampling and recording. There are no pixels on the sensor, only photosites. The different sizes of the sensors and the size of photosites adds complexity and the table below gives a general idea (without making difference between number of effective photosites and total amount of photosites). class="s5">(Scheme 1)

Regarding the sharpness of a 4K image, what is common between one image shot with an F65 or with an Red Epic Dragon, and one shot with a F55 or with a C500? The first two cameras have almost double, or more, of photosites than the others. The F65 has 19 millions of photosites on the sensor whereas F55 has 9,5 millions, but they both deliver 4K resolution. Even with the same resolution we realize than we can match large shots made with an F65 with a close shot made with an F55 class="s6">– we cannot readily match close shots made with an F65, to large shots made with the F55.

It’s always possible to up-scale the resolution of a camera in post-production, by example :

style="display: inline;">The Sony 900 camera owned 3 sensors with 1920 photosites x 1080, the HDCam recording talking account only of 1440 pixels x 1080. To deliver 1920 x 1080 pixels, up-scaling was used in the HDSDI output signal.
style="display: inline;">Some movies have been using both formats of capture 35 mm and HD. Through the 2/4K digital intermediate the upscaling of the digital format resolution helped the combination.

style="padding-top: 6pt;padding-left: 5pt;text-indent: 0pt;line-height: 152%;text-align: justify;">That’s why, in the table, I retained the Alexa in the 4K cameras, because even before the arrival of the XT and the Open Gate, movies shot with Alexa at a 2.8K resolution have been ‘’blown-up’’ for a 4K digital process, for example Skyfallclass="s7">. I won’t speak about Alexa 65, I have no experience on itclass="s6">..

>In every camera and in any deBayer process, whether it is in the camera for a Codec onboard recorder, or in postproduction, there are sharpness enhancement settings.

style="padding-left: 5pt;text-indent: 0pt;text-align: justify;">But are they always accessible? That is the point.

For Bayer Pattern cameras, strictly-speaking the setting of this function doesn’t belong to the deBayer process, which mathematically generates an RGB file, rather it follows it, and proceeds through various methods including the enhancement of contrast. The control of this setting is becoming more and more within our reach – ARRI was the first to launch this control in the ARRIRAW convertor (ARC).

A lot of cameras are using up-scaling processes to reach 4K resolution and the sharpness control to enhance, often artifically, the details. From these two interventions we can feel too much sharpness in some 4K cameras.

Strategies to lower the sharpness

style="padding-top: 11pt;padding-left: 5pt;text-indent: 0pt;line-height: 150%;text-align: left;">A lot of cinematographers appreciate the sharpness of a Sony F55, of a Canon C500 or a Varicam 35 when it comes to wildlife films or advertising for cars. But others, when it comes to feature film with actors (for TV or cinema release), use glass or digital filtering and/or older lenses to lower the sharpness.

Regarding his work on Mr Turner, Dick Pope BSC said ’’The film looks neither film nor digital," thanks inter alia to a very clever choice of old lenses.

The underlying debate is in fact on the perception of the resolution and of the sharpness, a lot of cinematographers prefer to focus on colour-depth than on the number of pixels.

>Examples include Rodrigo Prieto (ASC, AMC) on “La Voce Umana” short film, where he chose to set the Canon C500 in 2K 12-bit rather than 4K 10-bit.

Colour vs resolution, but also color vs sharpness. It’s a frequent dilemna that we meet in all countries. But it’s not only the number of pixels which determine the impression of sharpness. A scene shot with a F65 and Leica Summilux-C lenses at T1.4 may appear smooth but sharp, whereas another sequence shot at T2.8 may look too sharp requiring glass or digital filtering.

>Choosing to use a 2K downscaling setting in a 4K camera during a shoot is not so simple. In the Red Dragon this setting lead to zoom-in on the sensor, with the consequence of extending the focal lengths and losing the short ones – a 25mm in 4K will become a 50mm in 2K. Many cinematographers prefer to keep the 4K resolution of a F55 by with using glass filtering or digital filtering, because the 2K resolution gives too smooth an imagesclass="s7">.

Changing the OLPF in front of the sensor is an under-developed way for practicable reason. Red is now offering this option on the Dragon and the Scarlet. The Sony CBK- 55F2K can be also used on the F55/F5, I quote ‘’to yields softer, organic images” when shooting 4K high resolution images’’. An interesting idea which in my opinion alters too much the sharpness due to the number of photosites of the F55.

It is during camera tests that we get a good opportunity to assess the sharpness. Have a look at the diagram (Fig 2) to remind yourself of the parameters on-set which can govern the feeling of the sharpness.

style="padding-left: 36pt;text-indent: 0pt;line-height: 237pt;text-align: left;">

style="padding-top: 3pt;padding-left: 5pt;text-indent: 0pt;line-height: 150%;text-align: justify;">We will start to choose the lenses, the type of lamps and diffusion as well as the type of make-up before using filters. Choosing glass filtering with digital cameras becomes more complex on set.

style="padding-left: 5pt;text-indent: 0pt;text-align: justify;">We must be sure that the onboard deBayer process in the camera is reliable.

The WYSIWYG (what you see is what you get) specification of the camera is of paramount importance. But with a 4K camera – even for a 2K release – choosing a filter becomes very difficult with a HD monitor, not to mention the electronic viewfinder. style=" color: #00F;">(Photo 1)

style="padding-left: 10pt;text-indent: 0pt;line-height: 243pt;text-align: left;">

class="s9" style="padding-top: 7pt;text-indent: 0pt;text-align: center;">Example of Tiffen Black Satin 1 filter

A 4K monitor can be a solution – an expensive one – for a first selection but the only way to judge the quality remains the 4K or the 2K projection (depending on the type of release) on a large screen. The subtleties of the gradations cannot be distinguished without a simulation of the 4K or 2K workflow.

The following scheme becomes a reminder to indicate the place of the sharpness settings in the workflows using Codecs. class="s5">(fig 3)

style="padding-left: 37pt;text-indent: 0pt;line-height: 236pt;text-align: left;">

style="padding-left: 5pt;text-indent: 0pt;line-height: 152%;text-align: justify;">When shooting Raw we have one more option to set up the sharpness. In several grading software we have a deBayer process including a sharpness adjustment but they don’t all respect the structure of the image that the camera manufacturer has designedclass="s7">. class="s5">(fig 4)

Arri’s policy which certifies deBayer processes [2] from third parties stems (with the ‘’Certified for ARRIRAW Processing" label) seems clever and efficient. This empowers a manufacturer, who is assumed to know the camera specifications, to delegate responsibility for other companies without endorsing medium-quality processes.

Through peculiarities of the sensor and the processing of the signal we can distinguish differences of texture between the various camera manufacturers. A sharpest image with the F55, a certain roundness with the C500. It is difficult and very subjective, to quantify the texture, because the whole process depends on the choice of lenses, on the glass and/or the digital filtering, especially on the artistic project.

In the diverse strategies to control sharpness, post production enables the addition of film grain to put back some of the ‘’chaos’’, and specifically allows the use of digital filtering during or after grading. Many actresses know perfectly the virtues of the defocus parameters and other subtles shading in the grading suite. Days of digital restauration are sometimes notified in their contracts.

Some colorists are specialized in this discipline. And we can recognize the quality of all these algorithms to defocus and to refocus, all moving fast forward.

Photo 2

Image without glass and digital filtering

Photo 3

Image without glass filtering but with digital filtering. Digital filtering has only been applyed on image left cheek.

Digital filtering provides the advantage an/or the limitation of not having bias during shooting (which can be hazardous as the working time in the grading suite can grow exponentially). Time and costs of the digital filtering remain an obstacle for numerous indie and medium budget movies.

The experiences I had with Laurent Desbrueres, senior coloris tat Digimage facility France on the texture of several films showed me clairly the interest but also the limitations of this digital filtering

Other strategies?

Make up, digital and glass flltering can’t solve all the issues created by over sharpness. On this subject I would recommend reading ‘’Ready for FullHD’’ [3] an exciting compendium written by Isabelle Voinier make-up artist/instructor and Dirk Meier, colorist, both based in Germany. This book explain clearly the essential changes for make-up artists, their tools and products to face the increasing number of pixels.

Many cinematographers and colorists regret having ‘’to fight against the machine’’. This brings us quite naturally to this conclusion: before embarking on the mission to mitigate sharpness through glass or digital filtering, it would be wise to emulate the settings of sharpness upstream, similar to the ones provided by the ARC from ARRI, Canon RAW viewer or Red Rocket during the deBayer process in post, or have the ability to adjust the detail level parameters when codec-recording after the onboard deBayer in the camera. Unfortunately many companies provide only one single setting currently.

Only ARRI with the new ArriRaw Converter software version 5, offer to change the sharpnes uniformly across or separate for each of the 3 color channels- RGB. This is an interesting feature.

All these improvments should be done under the supervision of the manufacturers with the same spirit as Arri is certifying the ‘’third parties’’ who are processing deBayer.

>The following scheme shows the places that we have on the camera and in a workflow to control the sharpness.

(Fig 5)

The Sony F55 currently allows us to access these parameters in-camera when recording internal XAVC even in custom mode with Log curves. For the moment it does not have a large range for adjustments, but it’s an interesting step.

The last meeting of the Technical Committe of Imago has decided to develop the request to all camera manufacturers to open and/or to broaden the sharpness controls through different means:

Allow access to ''Detail Level'' parameters when we are shooting with codecs provided by the onboard deBayer of the camera (e.g. XAVC or ProRes), even when using Log curves.
Allow access to several parameters of sharpness after deBayer processes in post when shooting RAW.

These parameters already exist on some cameras and software in post, but there is no a broad consensus to provide all the parameters to deal with sharpness. As

cinematographer Kommer Kleijn SBC said, ‘’We have been playing for years with, at minimum, three settings of sharpness after deBayer on Photoshop. But why there is often only a single one to control the sharpness for the movie world?’’.

A BVK seminar which took place recently in Hambourg delivered a lot of informations to cinematographers about the deBayer processes. Arri, Colorfront & Codex put emphasis on the differences of textures.

Conclusion

It seems that we are just starting to tackle the real areas of interest of the digital workflow. Managing texture relies on controlling the sharpness. But there’s not only sharpness in the texture, and Jean-Pierre Beauviala led the way with the moving sensor on the Aaton Delta camera.

We can add grain in the image, we can tolerate a certain level of noise-to-signal in the image but we can also regret the chaotic moving structure of the filmstock.

But have we really dug into all the possibilities of changing digital texture?

I’ve been working for years with colorists, digital artists and camera engineers around the settings of the Detail Levels of HD cameras. We found the way to soften the image and we were encouraged by all the possibilities to change the structure of the image in all parts of the contrast curve, highlights, medium and lowlight areas. Playing with the ‘’coring’’ (which reduces noise by smoothing image pixels) and several other parameters, that I will not bore you with here, helped me to understand that the clinical look we often face is also an ideological point-of-view and most of the time it’s just a setting to be changed. The sharpness settings after the deBayer processes are reappearing slowly in several cameras and post softwares.

The texture, the structure of the pixel are part of an artistic approach to be done obviously upstream of the shoot. We no longer need to attempt to imitate filmstocks

The challenge now is how best to handle the digital image with the real point-of-view being on our perceptions. It’s under-explored territory, and more research and development is to be welcomed.

Philippe Ros, cinematographer AFC www.philipperos.com

Special thanks to Laurent Desbruères (Senior colorist Digimage Cinema, Paris), Fred Lombardo (Optical manager RVZ rental house Paris), Eric Martin (Digital Cinema Manager

Technicolor Paris), Aude Humblet (colorist, Eclair group, Paris) Jacques Pigeon (Digital Cinema technologies teaching, ENS Louis Lumière

style="padding-left: 40pt;text-indent: -35pt;line-height: 150%;text-align: left;">You can find on my website the full Tiffen tests I did for a feature film. http://www.philipperos.com/content.php?id=68&page=1

Notes :

[1] Semantical issues
The sharpness is characterized by a parameter called the Modulation Transfer Function (MTF) also known as spatial frequency response
The definition/resolution is characterized by the total amount of pixels in the image. Ex 4K : 4096 x 2160 pixels

[2] ARRIRAW SDK and 3rd Party implementations http://www.arri.com/camera/alexa/workflow/working_with_arriraw/arriraw/de_bayering/

Ready for FullHD’’ Munich 2013Self published revised edition http://www.ready-for-fullhd.com/

https://www.voinier.com/

Informations

Bayer pattern: class="s13">Most of digital camera use one single sensor with Bayer pattern with twice more green filter, than blue and red ones over the photosites. The result is a mosaic image with the consequences of having an image with a lack of information in the blue and red channels. The deBayer process will recalculate through sophisticated algorythms the missing informations. But the use of OLPF filter like in any camera lead to refocus the image just after the deBayer process in the camera for the view-finder and the onboard codecs recordings or in post for the raw recording. And that is during this operation that the over sharpness appear
class="s13" style="display: inline;">DeBayer mathematic approaches: With a camera using a Bayer pattern (most of the digital cinema cameras) it is quite complicate to give the number of photosites on the sensor which allow to have a real 4K resolution. There is an infinite number of mathematic approach to deBayer. Dave Stump, ASC, member of the SMPTE wrote: ‘’The mathematical ratio of photosites to actual resolution usually range between 50% and 66% of the sensor photosite count, although some de- Bayer algorythms claim to produce as much as 80% efficiency’. (in : Digital Cinematography - Edition Focal Press). It’s always possible to push the limits but it’s not without consequences and this the topic of this article. It will be interesting to know more about the 1/1 ratio that Arri will propose soon.
class="s13" style="display: inline;">Delta sensor : The Delta's sensor owns an option which allows the sensor to actually move in place. This capability oscillates the imager by a half-pixel offset each frame, randomizing the noise structure of the image.