AI Video Upscaling: Super-Resolution Group Test
This time were going to test out some applications that use artificial intelligence to produce high or super resolution videos from low resolution files. Normally, if we scale up low resolution footage, all that happens at the pixels are enlarged, and the final result therefore lacks clarity and sharpness. For example, if this shot were produced at 240p, then to appear sharp in this 1080p video, it would have to be reproduced at this size and if we scaled it up to the full size of a 1080p frame, things dont look good, as we can see in This comparison to improve the quality of scaled footage, super resolution, machine learning, attempts to identify the defining characteristics of a low resolution, image or video file. This that allows it to be scaled to a high resolution, while still exhibiting clarity and sharpness in the rest of this video were going to test the free trials of two commercial ai video scaling packages, as well as a completely free application. But if you want to know more about the computer science behind super resolution and not just see the results of some packages, then there are lots of useful resources out there on the internet and ill provide links to these in the video description right were now going To be testing out, avc labs, video enhancer, ai topaz labs, video enhance ai and finally wafer 2x extension gui. Here we can see a bit more detail with the avc labs and topaz labs programs being commercial windows and mac applications that offer watermarks trials whilst wayfu 2x extension gui is a windows program, thats free for personal use or its available for commercial application via a patreon Subscription before we begin its important to note that ai video upscaling is a pretty processing intensive activity with all of these programs requiring at least eight gigabytes of ram, as well as a modern processor thats as powerful as possible.
Here well be running our tests on a windows 11 system that has a hexacore, 10400, i5 cpu, 16 gigabytes of ram and a 2 gigabyte graphics card, and this means that our system here meets or exceeds the minimum hardware requirements for all of the applications. We also need a test file and ive got that down here. This is a 640 by 360 resolution, clip of a red admiral butterfly that i shot many years ago in which were going to scale up to hd. Right lets run up our first test program, which is avc lab video, enhancer, ai downloading and installing the trial. For this, software is very quick and easy. Theres no registration required. However, once installation is complete and you click on finish. Larger downloads and installations are triggered, which do take a bit of time, but when everythings complete, we can just browse for our test clip there. It is theres a red, admiral butterfly and then were presented with a very straightforward interface, and indeed this program has got the most straightforward interface of the three were looking at here. So all we have to do is to first of all select. The feature were going to use which here is ai upscaler, but you can also use this software to track and blur a face, although were not using that here and then, after that, we select the model were going to use for our upscaling and there are four Models available, standard, ultra and then standard ultra with multi frame and ive already tried using the standard model, and this was the results i got, which are pretty good, although theres some shimmering top left, and so what im going to try here is setting to ultra and Multi frame – and this means the program isnt going to process each frame individually, itll, look ahead to try and avoid things like shimmering im not going to select the noise.
I dont think we need that on this clip im not going to select face refinement because i dont think well see the face on the butterfly here: im not going to change any of the standard video settings, but we can, if we wish, i dont need to Crop the frame or de interlace and already have set here previously the video size to a 1080p 1920 1080, although you can go all the way up here to 8k. Finally, we just need to set the video format here, im using an mp4 container with h.264 encoding, but we can use h265. We can use prores or uncompressed avi if we wish im particularly pleased to see prores, but well stick with an mp4 container h264 for now and finally, the save directory is set to be the same place that the source clip is located. The only other settings here are found up in the menu under guess. What settings where we can select if we want to use our graphics card or cpu as the ai processor, and we can select maximum memory, consumption which ive left on the default, which is high. So lets kick off the processing and, as this is a trial, it comes up with this screen here, which indicates ive already used one by three encodes in my previous test. But well now continue to do test number two and there we are. We now have to wait for things to complete my previous test using the standard model took about seven minutes to process, but this im sure will take a lot longer than that, although we can see here that the results look like theyre going to be very good, We compare whats going on here with that bit of the wing to that.
It really is a remarkable improvement, and so well just have to be a bit patient, and here i am back again. The process finished in about 58 minutes, so lets view the video output, its lets open up the folder. Now we can see a new file its created and if we open that up its looking pretty good, there are still some problems top left. But i think theyre better than before lets look at a comparison of eclipse scaled in a conventional fashion compared to the output of avc labs, video, enhancer, ai and clearly our package here is doing a good job. Weve got a much much sharper result, so thats a good first test and well now move on to test out our second package. Next up, we have topaz labs video enhance ai, where once again, its very easy to install a trial. Although when we click on try for free, we do have to provide an email address to get a download link, but once weve got the download, the install is very straightforward, so lets just run up the program. Weve got it over here. There we are gives us a nice picture of the sailing ship and then just as previously, we can browse to select the clip we want to upscale there. We are theres our butterfly and we can select the clip to work on down there and then the interface here is slightly more complicated. We can select a video quality, low, medium or high.
Various previews come up. As you can see, we can set whether our video was shot progressively or interlaced, or whether its derived from cg models weve got the setting for how it deals with video artifacts. If i select none in terms of processing, because its initial footage is of good quality and then under that we can set the ai model, which we use for the ai upscaling were here. Im going to stay on, whats recommended artem is high quality and its worth. Noting this is already set to adjust to zesting models, they click up there. We can take off a suggested, setting and theres all sorts of models. We could apply so theres lots of potential here to try things out to get the best result. After that, we need to set our output size as previously ive set this to 1080p, but we could go all the way up to a 8k and then under that theres various settings for grain for video format, as previously, where once again, ive gone for an mp4 Container h.264 encoding, although we could use prores or uncompressed png or tiff frames before we start processing ill, also point out, theres, quite a lot of preferences. Here more than we had in the previous package. We can again set whether were going to use our graphics card or cpu. As the ii processor weve got a memory setting as previously, and we can manage all of the different available ai processing models so lets start processing lets make this thing.
Do some work there. We are, and, as you can see, it tells us this is a trial. We know that so well. Add a watermark, thats, okay, so well click on process with watermark. It brings in the appropriate model and gets on with the task and there we are its finished. So lets now take a look at the final file, which i think is a down here. Oh there we are its created us, a new file, which is that one and whats it look like not bad using that particular setting, i think, not quite as sharp as we had in the previous test, but we dont have a problem with the top left, which Is good and, as previously lets bring up a comparison of eclipse scaled in a conventional fashion compared to the output from topaz video enhance a i and again its pretty good. I still see not quite as good as we saw in the last test, although of course not having the shimmer in the top corner might make the clip a bit more usable. It all depends what you want to do with the final output and im sure it depends greatly on the source material, but well now move on to test out our final package greetings. Here we are with wayfu 2x extension gui from aaron feng, and this is free for public use and makes use of the popular wayfu, 2x and other open source super resolution, machine learning technologies and, as an aside, if you want to scale up still images its well Worth checking out this waifu 2x site, but backward wafer, 2x extension gui.
The download is fairly chunky at 1.8 gigabytes and is a 7z compressed file that needs extracting with 7zip or similar. When this is complete. No installation is necessary, as we launch the program by executing its start batch file, for which you could create a desktop shortcut if you wish, and as we can see on first run, the program performs a compatibility test here in the package. The interface is tab based with the first tab, requesting a donation to support the development of the software, but for now well, click on maybe later and go to the home tab and well bring in a red admiral video clip there. It is lets bring it in. Like that, and as you probably gathered the interface here is the most complex of the three packages were looking at in this video theres lots of opportunities to experiment and tweak. But there are also some settings presets so here, for example, were defaulting to a 3d real life enlarge only we could use medium or highest quality, but well stick with that one there, the fast one, although it is worth noting that this software and wayfu 2x, on Which it is based were initially developed to scale up anime art and so its important when youre setting things up to make sure youve selected 3d real life rather than 2d anime. If youre upscaling traditional video content over here, we can set a resolution. Its picked up the custom resolution ive been using so far, so if i just select the clip and apply, we can see the resolutions applied up here.
Ive already set my output folder and over in the video settings tab we can set. The container format were going to use, as this is a package for personal use. Only we only have the choice of mp4 or mkv and well stick with mp4 and id point out. You cannot select the settings for your video output unless you buy the premium version of the software. We also here have a tab for engine settings for the particular neural network model thats going to perform the actual scaling you can see all of them here. You can set all their different characteristics. Lots of things to twiddle with, as ive said, weve got a similar thing for image settings because you can use this software to scale still images. Not just video and weve got some additional settings for the package and the compatibility test that ran on first boot and which indicates everything is working on this computer. So lets go back to home and start the process there. We are its pretty fast on the fast settings, but nevertheless we will speed on through and there we are its finished. So lets take a look at the final file, which is hopefully in the folder over here. Yes, there we are theres the way through output, and here we are, and this isnt bad im, pretty impressed with the results of this package. I dont think its quite as sharp as what we had with the avc labs product, but its not bad and its its.
Not watermarked so lets go across to our comparison of the clip scaled conventionally and the wafer 2x extension gui, and here i think, im going to change my mind. If i may, i dont think the result here is as impressive as i just thought. It was its not particularly better than the traditional scale, so lets use the magic of filmmaking to do this and now were looking at a comparison of our traditional scale compared to output from wafer, 2x extension gui, but output at maximum quality. This took two and a half hours to output compared to nine minutes for what we were just looking at, and here i think the quality is significantly improved, its worth spending that extra time in processing right here i am back with a four way. Comparison of a traditional scale and the output from the three different packages and ive used here, the maximum possible quality i can get from each of the packages and for topaz. What i did was to re output the file not using different model settings, but output into a prores file, because the h.264 file produced by this package leaves at its maximum settings at a rather low data rate. So what are my final conclusions? Well, the first one is that ai enhancement is worth doing. All of these three images here are clearly better than what we see in a traditional scale, particularly if we look, for example, at this edge of the wing.
It is so much blurrier here in a traditional scale than it is, for example, in well any of these, but i think its probably best in abc labs, probably next in wayfu, probably next in tophat. And of course, i need to point out that youtubes encoding of this video will affect the quality you can see and if youre not watching this video in 1080p on at least a 1080p screen, you wont necessarily see the subtleties im talking about here. But for me i think the best package is by and large avc labs. The quality of the inside of the wing. The edge of the wing is better than the other packages, although potentially maybe wayfu and topaz theres, maybe slightly more natural in what theyre outputting. I also find it fascinating the way that the original clip has been processed differently by the different models that have been applied. So here, for example, the antenna is blurring into two parts, its separated into two parts by avc labs, but its definitely one part by wayfu and topaz. So weve got to be aware that when we start using software, like this, its going to change the nature of the images lets, let this play as video there we are, we can see things moving, which is what video needs to do, and i think my final Conclusion here is that wayfu 2x extension gui is a great package to experiment with you can get very good results if youre prepared to wait quite a long time for the processing.
But if you had a large batch of material to enhance, i would go with avc labs purchase a month of their subscription. Youd get things rendered out quicker with slightly higher quality results. Many people, myself very much included, have a large archive of standard definition and other low resolution, video files, and so the possibilities opened up by ai video scaling by super resolution. Machine learning technology are fantastic and, of course, this technology is just in its early stages. Its just in its beginning, its going to get better and better and its not impossible to imagine that in not that many years time itll become a real time technology that can be built into graphics cards or monitors or televisions. So we can see in better detail standard definition and other low resolution footage, but now thats it for another video if youve enjoyed what youve seen here. Please press that, like button, if you havent, subscribed, please subscribe and i hope to talk to you again very soon.