A new job

I started this little project in order to add sound effects to all the game’s cutscene. What I didn’t know was that two missing cutscenes would appear out of nowhere. And I didn‘t know either, that those cutscene would be in need of some restoration work. Something I’ve never done before…until now.

A few days ago Reidor posted two missing cutscenes (see my blogpost Lost and found). Apparently they were filmed from a PC monitor and then put on a dvd. This resulted in a low quality video, with lots of flickering and even a visible mouse cursor in some scenes. Nevertheless watching it was awesome. IllidanS4 took over the job of impoving the quality. He used an upscaling method named waifu2x, which had been developed a few years ago in Japan for upscaling anime movies and images. It was a big improvement. Still, a few things needed further work. Being filmed from a PC monitor, the video had heavy flickering and many frames were corrupted. He asked for help but nobody answered. So I wandered if I could do it.

My first approach was VIVA from Algosoft. It is a professfional film restoration software, used by the recent Metropolis release. It’s a pay per use software, so a short 2.5 minutes sequence wouldn’t have been that expensive. Sadly the software crashed quite often on my machine, up to a point where it became REALLY annoying. So I searched for alternatives. I tried a plugin for Adobe Premiere, which achieved next to nothing. Then I remembered VirtualDub, a freeware video editing programm with lots of free filters. The gui is self-explanatory so things like cropping and resizing worked like charm.

virtualdub
VirtualDub is a free and easy to use video editing software.

But what about the flickering? Well, as it turned out there is an excellent free filter for that, called MSU Deflicker. With it I was able to remove almost all flickering. I was amazed about how great the result of this freeware filter was.

deflicker
The MSU Deflicker filter for VirtualDub.

The upscaling was done with waifu2x caffe, a waifu2x program, which uses cuDNN from NVIDIA and therefore is much faster compared to just using the CPU. This program doesn’t upscale video files but images. So I had to convert the cutscene into over 5000 images/frames. Once again this could be achieved with VirtualDub.

waifu2x
Like VirtualDub, waifu2x-caffe is free and easy to use.

Then the real work started. I analysed every single frame and searched for corrupted ones. Looking at over 5000 frames is not the funniest way of spending someones time, but it was worth the time.

frames
The complete cutscene consists of 5143 frames.

Even one single corrupted frame can interrupt an otherwise fluid animation. And I found a lot. So I replaced them or repaired them in Photoshop. After that I loaded most of them into Photoshop and did another cleaning pass (thank god you can automate tasks in Photoshop). Waifu2x had already done the major task, but especially lines around the charakter’s eyes were still too fuzzy. I had to leave out some scenes because they didn’t have a lot of detail in the first place and another filtering would have decreased their quality even more.

Comparison.png

Finally I reassembled the images, once again via VirtualDub, and converted the newly generated video to smk via Rad Tools. It was quite an undertaking but, considering the difference in quality between the source material and the end result, totally worth it. The filtering process made the image look a little softer but after being converted to an 640 x 480 smk file, it was much better than the original.

Below you can see a comparison between the original DVD footage and the final digital remastered version.

An now it’s time for the complete cutscene. Enjoy! [EDIT: Updated link to BETA 3]

Advertisements

2 thoughts on “A new job

  1. You are amazing! I didn’t realize how much time this kind of restoration would take. Though I have myself also used waifu2x-caffe to upscale and denoise the frames, and a semi-automatic tool of mine to remove the cursor pointer, you have done really hard work to fix them all even more. The result is really marvelous.

    Like

  2. Thank you for your feedback! The next version is already in the works. I’ve corrected two animation errors (one of them thanks to you) and I’ve manually removed some ghosting the deflicker filter introduced.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s