I started this little project in order to add sound effects to all the game’s cutscene. What I didn’t know was that two missing cutscenes would appear out of nowhere. And I didn‘t know either, that those cutscene would be in need of some restoration work. Something I’ve never done before…until now.
A few days ago Reidor posted two missing cutscenes (see my blogpost Lost and found). Apparently they were filmed from a PC monitor and then put on a dvd. This resulted in a low quality video, with lots of flickering and even a visible mouse cursor in some scenes. Nevertheless watching it was awesome. IllidanS4 took over the job of impoving the quality. He used an upscaling method named waifu2x, which had been developed a few years ago in Japan for upscaling anime movies and images. It was a big improvement. Still, a few things needed further work. Being filmed from a PC monitor, the video had heavy flickering and many frames were corrupted. He asked for help but nobody answered. So I wandered if I could do it.
My first approach was VIVA from Algosoft. It is a professfional film restoration software, used by the recent Metropolis release. It’s a pay per use software, so a short 2.5 minutes sequence wouldn’t have been that expensive. Sadly the software crashed quite often on my machine, up to a point where it became REALLY annoying. So I searched for alternatives. I tried a plugin for Adobe Premiere, which achieved next to nothing. Then I remembered VirtualDub, a freeware video editing programm with lots of free filters. The gui is self-explanatory so things like cropping and resizing worked like charm.
But what about the flickering? Well, as it turned out there is an excellent free filter for that, called MSU Deflicker. With it I was able to remove almost all flickering. I was amazed about how great the result of this freeware filter was.
The upscaling was done with waifu2x caffe, a waifu2x program, which uses cuDNN from NVIDIA and therefore is much faster compared to just using the CPU. This program doesn’t upscale video files but images. So I had to convert the cutscene into over 5000 images/frames. Once again this could be achieved with VirtualDub.
Then the real work started. I analysed every single frame and searched for corrupted ones. Looking at over 5000 frames is not the funniest way of spending someones time, but it was worth the time.
Even one single corrupted frame can interrupt an otherwise fluid animation. And I found a lot. So I replaced them or repaired them in Photoshop. After that I loaded most of them into Photoshop and did another cleaning pass (thank god you can automate tasks in Photoshop). Waifu2x had already done the major task, but especially lines around the charakter’s eyes were still too fuzzy. I had to leave out some scenes because they didn’t have a lot of detail in the first place and another filtering would have decreased their quality even more.
Finally I reassembled the images, once again via VirtualDub, and converted the newly generated video to smk via Rad Tools. It was quite an undertaking but, considering the difference in quality between the source material and the end result, totally worth it. The filtering process made the image look a little softer but after being converted to an 640 x 480 smk file, it was much better than the original.
Below you can see a comparison between the original DVD footage and the final digital remastered version.
An now it’s time for the complete cutscene. Enjoy! [EDIT: Updated link to BETA 3]