Transitioning to Filmmaking Day 14: Denoising a Short Film Using AI Part 2
As I wrote in part 1 of this 2 part series, a filmmaker I know, Chris Browne, who was also on my podcast, asked me if I could help him denoise his short film, Pleroma, using AI. I gladly jumped at the chance to help him out and took it as a fantastic opportunity to learn more about denoising and working with real world footage using various AI models.
In the previous journal entry, I covered Artemis, the AI model in Topaz AI. Per their app description, "Artemis is a general enhancement model that offers a good balance of improved detail and reduced noise + artifacts. Includes variants trained for low, medium and high-quality source footage with different problems like halos or aliasing." I first used that model with a low quality setting.
It worked well overall for removing the noise from the footage provided, but I found some places where there wasn't a huge improvement, specifically, in the darker spots of the footage.
Luckily for me (and him), Topaz just released a new AI model called Nyx, which focuses on fixing noisy footage shot with a high end camera, but with bad lighting conditions. Perfect for the darker, lower lit spots of the footage!
I ran Nyx against the footage and the low light portions looked amazing, but guess what, the other portions didn't look as good as it did with Artemis.
Both where exported out of Topaz Video AI as DPX 10-bit image sequences.
What I decided to do to improve the overall quality of the final footage was to combine all 3 together in Davinci Resolve by pancaking the three tracks in the following manner.
Track 3 -> NYX AI Model Track -> Opacity set to 80%
Track 2 -> Artemis AI Model Track -> Opacity set to 90%
Track 1 -> Original -> Opacity set to 100%
I then exported the final file as a ProRes 422 HQ 1920x1080 at 23.976 FPS. I then took this new master file and uprez it to 4K. I'll cover that in the next journal entry tomorrow!
Here are some sample screen shots with the different AI models applied to each track in DaVinci Resolve.
Until the next entry!