FFmpeg Patches Allow For "Fully Functional" Multi-Threaded CLI

[phoronix]

Phoronix: FFmpeg Patches Allow For "Fully Functional" Multi-Threaded CLI

After more than two years of work, a patch series was posted this weekend for a "fully functional" ffmpeg multi-threaded command-line application with multi-threaded transcoding pipelines being wired up...

https://www.phoronix.com/news/FFmpeg-CLI-Multi-Threaded

[Danny3]

Wonderful!
Let's hope this really bring better performance and efficiency.

[aviallon]

Nice! I will finally be able to drop my old Python wrappers.

[Anux]

Uhh, looks like it's time for a small benchmark on one of those 64 core CPUs.

[Chugworth]

It will be interesting to see some benchmarks. Hopefully the most common tasks aren't "dominated by a a single component". It always bugs me when I'm waiting for a video to transcode and I see that the total CPU usage is nowhere near its full capacity.

[schmidtbag]

I'm confused by what exactly this is. For the better part of a decade, ffmpeg had the -threads flag and some codecs I think dynamically expand with thread count.

[Quackdoc]

it's worth noting, this likely won't speed up the actual encoders themselves which is where 70-80% of the bottlenecks are. To do that you will still need some kind of chunking. which I can't tell if from the video is part of this but it doesn't seem like it. It seems like this will speed up nearly everything else however? this could be particularly useful for decoding slow formats, (IE. HEVC) using complex cpu filter processing chains, etc.

This probably still wont be as good as using dedicated encoding tooling like av1an which is designed to extract the most out of the hardware (and interface with complex gpu filtering software like vapoursynth) but this does seem like it will be quite nice

[caligula]

it's worth noting, this likely won't speed up the actual encoders themselves which is where 70-80% of the bottlenecks are. To do that you will still need some kind of chunking. which I can't tell if from the video is part of this but it doesn't seem like it. It seems like this will speed up nearly everything else however? this could be particularly useful for decoding slow formats, (IE. HEVC) using complex cpu filter processing chains, etc.

Care to explain how? ffmpeg has already supported multiple threads previously. I can see from top/atop/htop that it's using multiple cores. Do you mean it can now do multiple streams simultaneously. Like gnu parallel + ffmpeg?

[stormcrow]

Care to explain how? ffmpeg has already supported multiple threads previously. I can see from top/atop/htop that it's using multiple cores. Do you mean it can now do multiple streams simultaneously. Like gnu parallel + ffmpeg?

My hot take is that this only speeds up the ffmpeg side of the things. If your coder is still essentially serially bound because it can't break up a file into chunks for recoding, this isn't going to help you any. It's going to remain the bottleneck when you're transcoding single files as the mailing list post states.

[dwagner]

This patch certainly won't do magic things... if you need to use some single-threaded filter somewhere, any additional multi-threading elsewhere may not make too much of a difference, and if the only expensive part of your run is a certain encoder, chances are that encoder already uses multiple CPU cores.
I can envision larger benefits only in the (relatively rare) cases where complex filter chains or multiple sources are involved.