Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Using Neural Networks to Resolve Lossy Artefacts (Read 2731 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Using Neural Networks to Resolve Lossy Artefacts

Hello everyone!

Just spitballing here, I have no background in neural networks or AI, however I wondered if any research was being done to use these technologies to compensate or resolve issues introduced by lossy encoding? ie, reduced bandwidth, pre echo artefacts etc.

From what I understand, we can know what lossy encoder was used, we can profile that with huge data sets (lossy and lossless examples of music). Can we not use this data to reconstruct what was removed with a reasonable level of confidence? I recently got Izotope RX 9 and it can almost perform miracles using neural networks, so I was wondering what the next step could be.

Is there any work being done on this? Is it completely impossible?

Re: Using Neural Networks to Resolve Lossy Artefacts

Reply #1
Assume it is possible, for optimal results, such a product may require a large and constantly updated online database which means users may need to subscribe for the service, could be expensive.

But for now, there are something like this:
https://www.stereotool.com/documentation/9.23/delossifier/

Probably not powered by AI I suppose. Not very convincing result either, IMO.

Re: Using Neural Networks to Resolve Lossy Artefacts

Reply #2
I wouldn't expect any miracles from such tech. Said iZotope tools, also produce unnatural-sounding results, with artifacts similar to those of lossy codecs when pushed to the limits.

 Besides, the usefulness is also debatable, as often it's possible to just find a better source to work with.
a fan of AutoEq + Meier Crossfeed

Re: Using Neural Networks to Resolve Lossy Artefacts

Reply #3
I wouldn't expect any miracles from such tech. Said iZotope tools, also produce unnatural-sounding results, with artifacts similar to those of lossy codecs when pushed to the limits.

 Besides, the usefulness is also debatable, as often it's possible to just find a better source to work with.

If you use them too aggressively, yes. However, 10 years ago people would have said some of this processing would have been impossible. They’re pretty damn good for a first generation attempt.

Personally, I feel like we’re not too far of some very impressive reconstruction techniques.

Re: Using Neural Networks to Resolve Lossy Artefacts

Reply #4
Even if one hires a real human to do the audio restoration, your employee may, and mostly likely have a different opinion of what is a "good" restoration. You still need to somehow train your employee to do what you wanted to do.

Speaking of my personal experience when being hired as a restoration / sound editing guy who needed to fulfill different needs of different clients. They can have very different tastes and requirements.

Re: Using Neural Networks to Resolve Lossy Artefacts

Reply #5
Even if one hires a real human to do the audio restoration, your employee may, and mostly likely have a different opinion of what is a "good" restoration. You still need to somehow train your employee to do what you wanted to do.

Speaking of my personal experience when being hired as a restoration / sound editing guy who needed to fulfill different needs of different clients. They can have very different tastes and requirements.

I’m not sure a human can restore lossy artefacts. The tools don’t exist

Re: Using Neural Networks to Resolve Lossy Artefacts

Reply #6
I’m not sure a human can restore lossy artefacts. The tools don’t exist
Then you can't put too much trust on Neural Networks either, which simulates human behavior.
Don't know if you have interest in video / graphic upscaling or not, including frame rate upscaling or pixel resolution upscaling. There are indeed Neural Networks based video upscaler to reduce compression artifacts or interpolating missing frames to create a smoother looking video. There are different upscaling algorithm to choose from, like NNEDI3, NGU, Waifu2x and such. The result can look better than traditional, non neural network based algorithms like bicubic, lanczos and such, but of course, these algorithms have their own limitations as well. If one has patient or being hired, it is indeed possible to do a better job by using something like PhotoShop and AfterEffect to do frame by frame restoration.

Don't know what is your intention to open this thread. Confirming your own belief? Asking if such an audio restoration tool is ready to use that you can buy right now? Asking for a technical demo? If it is the first one, then just wait, there is no need to ask.

Re: Using Neural Networks to Resolve Lossy Artefacts

Reply #7
I’m not sure a human can restore lossy artefacts. The tools don’t exist
Then you can't put too much trust on Neural Networks either, which simulates human behavior.
Don't know if you have interest in video / graphic upscaling or not, including frame rate upscaling or pixel resolution upscaling. There are indeed Neural Networks based video upscaler to reduce compression artifacts or interpolating missing frames to create a smoother looking video. There are different upscaling algorithm to choose from, like NNEDI3, NGU, Waifu2x and such. The result can look better than traditional, non neural network based algorithms like bicubic, lanczos and such, but of course, these algorithms have their own limitations as well. If one has patient or being hired, it is indeed possible to do a better job by using something like PhotoShop and AfterEffect to do frame by frame restoration.

Don't know what is your intention to open this thread. Confirming your own belief? Asking if such an audio restoration tool is ready to use that you can buy right now? Asking for a technical demo? If it is the first one, then just wait, there is no need to ask.

Audio artefacts are not compatible to visual/video artefacts, I don’t believe.

There are no manual tools for the removal of pre echo or the reconstruction of high frequencies (reconstruction and not distortion, generation).

I posted this to see what the state of affairs is and if there is anything on the horizon yet. I believe this will be a valid route for technology in the near future.

This is a discussion forum, surely we’re free to discuss topics without having an agenda.

Edit: I don’t agree with your example at all. If one is hired they can do a better job manually? How? And without using the traditional scale techniques? You mean redraw the whole frame?

Re: Using Neural Networks to Resolve Lossy Artefacts

Reply #8
Obviously you don't need to agree with me. However someone else agree with me. One example is enough:

https://hydrogenaud.io/index.php/topic,112548.0.html

See it? The OP said the file I attached in Reply #15 has improved, and the source is a lossy file.

Sure you can say it sounds bad, and you have to say so to support your claim. But the fact is the original lossless source is unavailable, so there is no way to know if my restoration is closer to the original or not. The same applies to what you wanted to achieve as well.

Re: Using Neural Networks to Resolve Lossy Artefacts

Reply #9
I mean, there is no reason it shouldn't be "statistically possible". Just like pop/crackle removal software can make "better than wild" guesses that this is an unwanted artifact and not part of the lossless source, the knowledge about the particular encoder used, could be exploitable. And "reconstruction" of what was lost by a low-pass filter could be guesstimated from real music.

But for one thing, the trade-off between type I and type II errors is not at all god-given.

Re: Using Neural Networks to Resolve Lossy Artefacts

Reply #10
https://www.deutsche-kinemathek.de/en/kinemathek/press/frame-by-frame-restoring-films
Quote
“Workshop” takes visitors into film restoration workshops to see how a complete film is recovered from fragments and decaying prints – in a process that is partly manual skill and partly digital procedures.
Which means, even with technology advancement, real human still has a role in these aspects. Even if the tools being used are AI-assisted, the results can still be further improved by trained human workers.

Of course, with average untrained human the results can be very poor as well.
https://www.bbc.com/news/av/world-europe-23693176

Another important point is that the outcome of the restoration is judged by human, which means "perceptually better" and "mathematically better" can be quite different.

Speaking of vinyl restoration with using specialized software or AI...
Original flac:
https://hydrogenaud.io/index.php/topic,115012.msg949366.html#msg949366
Restored to 135kbps opus (without intentional loudness adjustment):
https://hydrogenaud.io/index.php/topic,115012.msg949463.html#msg949463

Re: Using Neural Networks to Resolve Lossy Artefacts

Reply #11
Another important point is that the outcome of the restoration is judged by human
That of course, can be used to train an AI. Objective is not a pre-determined formula, objective is "humans prefer it" - and then machine learn a model for that. Who knows if there will ever be enough data for the model to impress anyone.


Re: Using Neural Networks to Resolve Lossy Artefacts

Reply #12
Speaking of vinyl restoration without using specialized software or AI...
Typo. Thinking about AI's role in spelling correction, when the user is not a native English speaker.