Apple’s new ProRAW Photo Format Is Neither Pro nor Raw

Apple recently released iOS 14.3 which includes the new ProRAW photo format feature for iPhone 12 Pro models. As Apple explains:

Apple ProRAW combines the information of a standard RAW format along with iPhone image processing, which gives you more flexibility when editing the exposure, color and white balance in your photo.

This format is intended to offer higher quality photos while still benefiting from the computational photography features of the iPhone. However, Apple is exaggerating a bit, because this is not a raw file, and it’s not really a "pro" file either. Here’s why.

What are raw files?

Raw files contain the most basic data collected by a camera sensor. While all cameras contain processors that can convert this data to JPEG image files – using the camera manufacturer’s decisions on how to handle different aspects of photos, such as adjustments to shadows and highlights, contrast settings, colors, and more – many cameras also allow you to save raw files. You do all the editing with these files after you get them on your computer. (Some cameras offer raw conversion options in the camera as well.)

One of the key elements of raw files is that they are not demosaiced. Demoisaicing is when an algorithm interprets the colors of the various pixels according to how light is recorded after traversing color filters, applies white balance settings, and more. This is complex, and this article from the developers of the Halide camera app explains the process.

When you open a raw file in an editing app, your software processes the file, performing the demosaicing, along with some other processing, and then allows you to then proceed with other edits. Because of this, there are a number of photo editing apps that perform this demosaicing in slightly different ways; photographers choose the app they prefer according to the results (and for other editing capabilities as well).

But Apple’s ProRAW has already done this demosaicing, which means that, well, it’s not a raw file. At best, the resulting linear DNG file (DNG is a format developed by Adobe that can contain raw files, but also other formats), an uncompressed image file, offering much more data for subsequent edits.

It’s worth looking at Apple’s developer documentation discussing this new format (my italics):

By default, the iOS capture pipeline uses advanced computational photography techniques to achieve the highest quality images, and delivers the processed result to your app in a format compressed for efficient storage and display. Processed formats are a great choice in many cases, but because they’re lossy, they typically aren’t a good fit for professional photography workflows. In these cases, the preferred format is RAW.

RAW formats capture image data largely as the camera sensor interprets it. Capturing the image data this way results in much larger files than processed formats, but greatly expands your editing capabilities in post-production. One drawback to shooting in standard Bayer RAW format is you no longer benefit from the advanced image processing you get with compressed formats.

Starting in iOS 14, and available on iPhone 12 Pro and Pro Max, you can use the new Apple ProRAW format. Apple ProRAW is a demosaiced RAW format that gives you the benefits of Bayer RAW capture, but applies many of the multi-image fusion techniques previously unavailable to RAW workflows.

That final sentence explains why this format is not a raw format: the data is demosaiced, which means that the sensor data has been converted into data with color information, which is the first step in processing raw files. And once that is done, you can’t go back to the original data.

In some cases, photographers may want to return to older raw files to reprocess them with newer, more efficient software. It’s not the norm, but not having a true raw file means that this isn’t possible.

Why photographers don’t understand ProRAW

When this format was released, a number of photographers wrote about it, but many got this wrong. For example, Austin Mann, in his article ProRAW Is Here!, made the assumption that these were indeed raw files that needed processing and editing. Here is a screenshot of three photos in his article; the middle photo is labeled ProRAW (Unedited).

Unfortunately, Mr. Mann’s misunderstanding stems from two things. First, the fact that Apple uses the term "raw" suggests that these are, indeed, raw files; they are not. Apple’s marketing is deceptive at best, and this example shows that if the type of photographer who uses raw files confuses this, then they’ve managed to confuse the very demographic they’re targeting.

The second mistake Mr. Mann made was importing the ProRAW file into Lightroom, where he was planning to edit it. Alas, Lightroom does not interpret these files correctly. An Adobe employee explained this on Reddit:

When opening a ProRaw […] photo, Lightroom is still opening the file as if it were a raw, aka sans processing. When you see the file in Apple Photos, it’s applying tone mapping to the photo automatically.

You can use the highlights and shadows sliders in Lightroom to apply our version of tone mapping to the photo (recovering details in the highlights and shadows).

We are now working with Apple to create a profile that will match the look of Apple’s tone mapping (which is materially different from the default one that we apply and use in our HDR approaches) so that you can achieve similar look and feel (but also have control over it).

That new profile will come out in a future (most likely the next) release.

In the comments to Mr. Mann’s article, this was pointed out to him, and he acknowledged that he made a mistake, but has not corrected his article and his examples, so many people will be reading his much-quoted article and getting the wrong information. The article by the Halide developer makes the same mistake; it, too, discusses editing files in Lightroom. ProRaw files are meant to be finished photo files, at least as far as the processing of the camera sensor data is concerned, and this is visible when looking at the files in Apple’s Photos app, or in some other apps.

What’s so special about local tone mapping?

Local tone mapping is the important element here. This is data that records the differences in brightness in "local areas," which could be an entire sky, or just a small element of a photo. Local tone mapping information works sort of like HDR (high dynamic range) photos, enhancing exposure on dark areas and reducing it on light areas. This data is stored in ProRAW files, and is used by Apple’s Photos app, and a few others, but not by Lightroom.

Local tone mapping is an excellent tool for working with these photos, and one app in particular, Raw Power, takes full advantage of this, offering a slider allowing you to adjust this local tone mapping. This video by the Nik Bhatt, the developer of Raw Power, explains how this works, and discusses other elements of Apple’s ProRAW format. (Mr. Bhatt was the lead on Aperture at Apple for many years, and also led the teams responsible for Core Image and Apple’s RAW Camera library.)

If you want to understand ProRAW, and how you can leverage it, take 15 minutes and watch this video.

Not so pro

If ProRAW was really meant for pros, it would include the raw data, along with the information necessary to display photos using Apple’s computational photography features. But this would take up too much space. As it is, ProRAW files are as much as 10 times the size of JPEGs, so they’ll fill up your iPhone – and your iCloud storage – quickly. But real pros would be willing to accept that to have the actual raw files, plus Apple’s special sauce.

You can, of course, shoot raw on the iPhone, and have been able to do so for years, using third-party apps such as Halide. And you can shoot ProRAW with Halide and other apps, including the default Camera app. But there’s no way to combine both feature sets to retain the raw data and all of the adjustments that Apple makes. This would be very difficult, because each photo taken on an iPhone is a combination of several photos. As Apple explained for the iPhone 11:

Deep Fusion uses advanced machine learning to do pixel-by-pixel processing of photos, optimizing for texture, details and noise in every part of the photo.

With ProRAW, as Vice President, Camera Software Engineering Jon McCormack told PetaPixel:

Apple created a new imaging pipeline that combines computational photography techniques but then saves the result of those computations out to a digital negative.

There’s no way to have both a raw image and the powerful computational techniques that Apple uses, and ProRAW is a very good way to split the difference. If you have access to a local tone mapping adjustment, as in Raw Power (and I hope in other editing apps, notably Apple’s own Photos, at some point), then you have a lot of flexibility working with these files. But even if not, by default, the extended dynamic range, and the fact that you have more data to perform local edits, makes this a very interesting photo format for those who want more than just snapshots. But it’s not pro, and it’s not raw.

Listen to a discussion of Apple ProRAW on the PhotoActive podcast.

Thanks to Jeff Carlson and Nik Bhatt for their input to this article.