[Arch Linux] Hardware accelerated video decoding on Intel Graphics

[Radium]

Hello!

I have a laptop that has a Broadwell-U processor which has Intel HD Graphics but I'm unable to get software to use the hardware accelerated video decoding features that Intel have provided for quite some times now. This makes watching videos unbearable since since software decoding makes the processor warm really fast and it doesn't take long until the fan spins at full speed. I've installed Arch Linux and i use Gnome for now and I've tried both X and Wayland.

I've reached so far as noticing VDPAU for Intel isn't working. I've installed the necessary packages (and their dependancies):

• xf86-video-intel
• mesa
• libva (includes vainfo)
• libvdpau-va-gl
• mesa-vdpau
• libva-vdpau-driver
• libva-intel-driver
• vdpauinfo

I followed guides and forum threads regarding this, including VDPAU, VA-API and Xorg but the command "Xorg :0 -configure" always gives me error and the information that I've found out there is that some setups don't need configurations such as Intel HD Graphics. I also defined two environment variables:

• VDPAU_DRIVER=va_gl
• LIBVA_DRIVER_NAME=vdpau

I execute vdpauinfo to see if my configuration is correct but it doesn't go all the way through, it just stops right here:

display: :0   screen: 0
[VS] Software VDPAU backend library initialized
libva info: VA-API version 0.38.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'vdpau'
libva info: Trying to open /usr/lib/dri/vdpau_drv_video.so
libva info: Found init function __vaDriverInit_0_35

 

If I change LIBVA_DRIVER_NAME to i965 (which only the oldest guides and threads mention) then it works but all I get is this:

display: :0   screen: 0
[VS] Software VDPAU backend library initialized
libva info: VA-API version 0.38.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'i965'
libva info: Trying to open /usr/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_38
libva info: va_openDriver() returns 0
API version: 1
Information string: OpenGL/VAAPI/libswscale backend for VDPAU

Video surface:

name   width height types
-------------------------------------------
420     1920  1080  NV12 YV12 UYVY YUYV Y8U8V8A8 V8U8Y8A8 
422     1920  1080  NV12 YV12 UYVY YUYV Y8U8V8A8 V8U8Y8A8 
444     1920  1080  NV12 YV12 UYVY YUYV Y8U8V8A8 V8U8Y8A8 

Decoder capabilities:

name                        level macbs width height
----------------------------------------------------
MPEG1                          --- not supported ---
MPEG2_SIMPLE                   --- not supported ---
MPEG2_MAIN                     --- not supported ---
H264_BASELINE                  51 16384  2048  2048
H264_MAIN                      51 16384  2048  2048
H264_HIGH                      51 16384  2048  2048
VC1_SIMPLE                     --- not supported ---
VC1_MAIN                       --- not supported ---
VC1_ADVANCED                   --- not supported ---
MPEG4_PART2_SP                 --- not supported ---
MPEG4_PART2_ASP                --- not supported ---
DIVX4_QMOBILE                  --- not supported ---
DIVX4_MOBILE                   --- not supported ---
DIVX4_HOME_THEATER             --- not supported ---
DIVX4_HD_1080P                 --- not supported ---
DIVX5_QMOBILE                  --- not supported ---
DIVX5_MOBILE                   --- not supported ---
DIVX5_HOME_THEATER             --- not supported ---
DIVX5_HD_1080P                 --- not supported ---
H264_CONSTRAINED_BASELINE      --- not supported ---
H264_EXTENDED                  --- not supported ---
H264_PROGRESSIVE_HIGH          --- not supported ---
H264_CONSTRAINED_HIGH          --- not supported ---
H264_HIGH_444_PREDICTIVE       --- not supported ---
HEVC_MAIN                      --- not supported ---
HEVC_MAIN_10                   --- not supported ---
HEVC_MAIN_STILL                --- not supported ---
HEVC_MAIN_12                   --- not supported ---
HEVC_MAIN_444                  --- not supported ---

Output surface:

name              width height nat types
----------------------------------------------------
B8G8R8A8            53    53    y  
R8G8B8A8         -906199177 -906199177    y  
R10G10B10A2      -906199177 -906199177    y  
B10G10R10A2      -906199177 -906199177    y  
A8               -906199177 -906199177    y  

Bitmap surface:

name              width height
------------------------------
B8G8R8A8          8192  8192
R8G8B8A8          8192  8192
R10G10B10A2       8192  8192
B10G10R10A2       8192  8192
A8                8192  8192

Video mixer:

feature name                    sup
------------------------------------
DEINTERLACE_TEMPORAL             -
DEINTERLACE_TEMPORAL_SPATIAL     -
INVERSE_TELECINE                 -
NOISE_REDUCTION                  -
SHARPNESS                        -
LUMA_KEY                         -
HIGH QUALITY SCALING - L1        -
HIGH QUALITY SCALING - L2        -
HIGH QUALITY SCALING - L3        -
HIGH QUALITY SCALING - L4        -
HIGH QUALITY SCALING - L5        -
HIGH QUALITY SCALING - L6        -
HIGH QUALITY SCALING - L7        -
HIGH QUALITY SCALING - L8        -
HIGH QUALITY SCALING - L9        -

parameter name                  sup      min      max
-----------------------------------------------------
VIDEO_SURFACE_WIDTH              -  
VIDEO_SURFACE_HEIGHT             -  
CHROMA_TYPE                      -  
LAYERS                           -  

attribute name                  sup      min      max
-----------------------------------------------------
BACKGROUND_COLOR                 -  
CSC_MATRIX                       -  
NOISE_REDUCTION_LEVEL            -  
SHARPNESS_LEVEL                  -  
LUMA_KEY_MIN_LUMA                -  
LUMA_KEY_MAX_LUMA                -  

 

I've tried to get hardware accelerated video decoding (H.264) to work properly but the features that should be there just never show up. Programs can run in GL mode just fine, Wayland works out of the box so the Intel driver and Mesa seems to work.

[The_Decryptor Veteran]

What player are you using? Configuring X to use VDPAU won't help if you're running Wayland, or if the player you're using doesn't talk to X for video decoding (ffmpeg and gstreamer both handle it themselves)

[Radium]

I try to get Firefox to support H.264 hardware acceleration, that support still won't show up. Will I have to run a desktop that doesn't have Wayland support at all for this to work at all? Gnome allows me to pick between X and Wayland and it doesn't matter what I pick. I follow the guides, run Gnome under X and still can't get it to work.

[Radium]

What player are you using? Configuring X to use VDPAU won't help if you're running Wayland, or if the player you're using doesn't talk to X for video decoding (ffmpeg and gstreamer both handle it themselves)

I just installed xfce and I tried that. I can't play any video at all unless I unset those environment variables. But how do I go about making Firefox and similar programs support H.264 hardware decoding? Chromium is just as problematic, it refuses to use H.264 hardware acceleration.

[The_Decryptor Veteran]

For Firefox it'll need to use gstreamer, and you'll need the right plugins installed for gstreamer to play video (I think it wants to play via VA-API instead of VDPAU), so look into libva and vainfo. That being said, Firefox doesn't currently use hardware accelerated rendering under Linux, so even if you're using HW video decoding, you're then doing readback to the CPU, which can make it even slower than unaccelerated decoding.

Chrome uses ffmpeg, but it's own copy of ffmpeg. I'm pretty sure it's up to them to enable it, and I've got no idea if they have or not. Other clients (like VLC) would have their own options, but generally anything using gtreamer should work fine if the hardware supports it and the plugins are right.

[Radium]

For Firefox it'll need to use gstreamer, and you'll need the right plugins installed for gstreamer to play video (I think it wants to play via VA-API instead of VDPAU), so look into libva and vainfo. That being said, Firefox doesn't currently use hardware accelerated rendering under Linux, so even if you're using HW video decoding, you're then doing readback to the CPU, which can make it even slower than unaccelerated decoding.

Chrome uses ffmpeg, but it's own copy of ffmpeg. I'm pretty sure it's up to them to enable it, and I've got no idea if they have or not. Other clients (like VLC) would have their own options, but generally anything using gtreamer should work fine if the hardware supports it and the plugins are right.

So how do I make gstreamer use the right plugins? The plugin packages are named in a very strange way and how on earth am I supposed to know which plugin to install?

[simplezz]

HTML 5 video's in Firefox are dog slow and chew the cpu. It's even worse than flash (freshplayer/pepper) on my system. You can get around that by installing mozplugger and setting a different default player. Personally, I use mpv. HW acceleration works great with it. For example, with a H.264 video file:

 (+) Video --vid=1 (*) (h264)
 (+) Audio --aid=1 --alang=eng (*) (aac)
libva info: VA-API version 0.38.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/dri/fglrx_drv_video.so
libva info: Found init function __vaDriverInit_0_33
libva info: va_openDriver() returns 0
Using hardware decoding.
AO: [alsa] 48000Hz stereo 2ch float
VO: [opengl] 1920x800 vaapi

My mpv config file:

vo=opengl
hwdec=vaapi
#af=drc
softvol-max=400

bashrc:

export VDPAU_DRIVER=va_gl

[Radium]

HTML 5 video's in Firefox are dog slow and chew the cpu. It's even worse than flash (freshplayer/pepper) on my system. You can get around that by installing mozplugger and setting a different default player. Personally, I use mpv. HW acceleration works great with it. For example, with a H.264 video file:

 (+) Video --vid=1 (*) (h264)
 (+) Audio --aid=1 --alang=eng (*) (aac)
libva info: VA-API version 0.38.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/dri/fglrx_drv_video.so
libva info: Found init function __vaDriverInit_0_33
libva info: va_openDriver() returns 0
Using hardware decoding.
AO: [alsa] 48000Hz stereo 2ch float
VO: [opengl] 1920x800 vaapi

My mpv config file:

vo=opengl
hwdec=vaapi
#af=drc
softvol-max=400

bashrc:

export VDPAU_DRIVER=va_gl

Looks good. But does mozplugger+mpv play videos inside the HTML element or does it start a new window?

How do I configure mozplugger to use mpv for H.264 or any video at all? I look in /etc/mozpluggerrc and nothing makes sense. The default conf is packed with pointers to players that I don't have installed nor do I understand how to elegantly call mpv. I tried to enter my custom entry but all the other entries flood my terminal. Do I clean out all the junk and create a tailored made rc for the things I want?

[simplezz]

Looks good. But does mozplugger+mpv play videos inside the HTML element or does it start a new window?

How do I configure mozplugger to use mpv for H.264 or any video at all? I look in /etc/mozpluggerrc and nothing makes sense. The default conf is packed with pointers to players that I don't have installed nor do I understand how to elegantly call mpv. I tried to enter my custom entry but all the other entries flood my terminal. Do I clean out all the junk and create a tailored made rc for the things I want?

You don't need to edit the whole config. Only the default video player. The Arch wiki tells you how to do it:

To change the default of MPlayer so that mpv is used instead, change the appropriate lines such that:

/etc/mozpluggerrc

...
### MPlayer

#define(MP_CMD,[mplayer -really-quiet -nojoystick -nofs -zoom -vo xv,x11 -ao esd,alsa,oss,arts,null -osdlevel 0 $1 </dev/null])
define(MP_CMD,[mpv -really-quiet $1 </dev/null])

#define(MP_EMBED,[embed noisy ignore_errors: MP_CMD(-xy $width -wid $window $1)])
define(MP_EMBED,[embed noisy ignore_errors: MP_CMD(--autofit=$width -wid $window $1)])

#define(MP_NOEMBED,[noembed noisy ignore_errors maxaspect swallow(MPlayer): MP_CMD($1)])
define(MP_NOEMBED,[noembed noisy ignore_errors maxaspect swallow(mpv): MP_CMD($1)])

...

#define(MP_AUDIO,[mplayer -quiet -nojoystick $1 </dev/null])
define(MP_AUDIO,[mpv -really-quiet $1 </dev/null])

#define(MP_AUDIO_STREAM,[controls stream noisy ignore_errors: mplayer -quiet -nojoystick $1 "$file" </dev/null])
define(MP_AUDIO_STREAM,[controls stream noisy ignore_errors: mpv -really-quiet $1 "$file" </dev/null])
...

https://wiki.archlinux.org/index.php/Browser_plugins#MozPlugger

You're basically replacing  the default video player (mplayer) with mpv.

I've only really tested it on Youtube where I use the ViewTube script in conjunction with GreaseMonkey. It works well on my system:

http://imgur.com/o5OxZNw

[The_Decryptor Veteran]

So how do I make gstreamer use the right plugins? The plugin packages are named in a very strange way and how on earth am I supposed to know which plugin to install?

The Linux media stack is kinda a mess, there's 2 main versions of gstreamer in use (0.10 and 1.0), I think you need 1.0 for proper hardware acceleration, but even though Firefox supports both, most older distros still ship with 0.10, so they build their Firefox to target that. If your distro has 1.0, and builds their Firefox to target 1.0, and you've got the right plugins installed (VA-API support via libva), everything should just work.

But as I said, since Firefox doesn't use hardware accelerated rendering (The graphics stack is also a mess), it can end up being slower than unaccelerated decoding. Using a plugin to play video could help since it'd control its own rendering output to the screen, but also "breaks" stuff like YouTube that will only then offer 2 video qualities because they require native browser playback for normal operation.

[Radium]

The Linux media stack is kinda a mess, there's 2 main versions of gstreamer in use (0.10 and 1.0), I think you need 1.0 for proper hardware acceleration, but even though Firefox supports both, most older distros still ship with 0.10, so they build their Firefox to target that. If your distro has 1.0, and builds their Firefox to target 1.0, and you've got the right plugins installed (VA-API support via libva), everything should just work.

But as I said, since Firefox doesn't use hardware accelerated rendering (The graphics stack is also a mess), it can end up being slower than unaccelerated decoding. Using a plugin to play video could help since it'd control its own rendering output to the screen, but also "breaks" stuff like YouTube that will only then offer 2 video qualities because they require native browser playback for normal operation.

It's a complete mess.
Arch Linux has stopped packaing gstreamer 0.10 and only 1.x is availible now so I would assume that the one who package Firefox has used that which is availible in the repository. And everything doesn't work at all out of the box. I've installed all that libva stuff and Firefox has always insisted on software decoding despite X loading the driver. VA-API (drivers and gstreamer-vaapi) is the first thing I install before I even try anything else. I haven't found any information on how to fiddle with gstreamer to see where things go wrong.

 

You don't need to edit the whole config. Only the default video player. The Arch wiki tells you how to do it:

https://wiki.archlinux.org/index.php/Browser_plugins#MozPlugger

You're basically replacing  the default video player (mplayer) with mpv.

I've only really tested it on Youtube where I use the ViewTube script in conjunction with GreaseMonkey. It works well on my system:

http://imgur.com/o5OxZNw

I finally got it to work after installing Greasemoney and ViewTube, however, I wish I had the same simple controls and direct options for fullscreen mode that YouTube has out of the box. All ViewTube (with MPV) allows is fullsize. No fullscreen button.

Thanks a lot! At least it plays videos without the fan starting.

[simplezz]

I finally got it to work after installing Greasemoney and ViewTube, however, I wish I had the same simple controls and direct options for fullscreen mode that YouTube has out of the box. All ViewTube (with MPV) allows is fullsize. No fullscreen button.

You can always press f11 and go fullscreen in Firefox. The video window will automatically resize. Or alternatively just download the video and play it locally.

If you're serious about avoiding the bloat of Firefox though, I'd suggest going for something like youtube-viewer (excellent tool) or Minitube.

[Radium]

You can always press f11 and go fullscreen in Firefox. The video window will automatically resize. Or alternatively just download the video and play it locally.

If you're serious about avoiding the bloat of Firefox though, I'd suggest going for something like youtube-viewer (excellent tool) or Minitube.

Youtube-viewer is sweeeeet!
I downloaded gtk-youtube-viewer from Arch Linux AUR and it's really nice. It just lacks that random suggested video startpage and the suggestions to the right but I can live without that until Firefox gets a grip around VA-API.

Apparently, Firefox has had problems with VA-API hardware acceleration off and on. Some have even had crashes. There are open issues about VA-API hardware acceleration not being enabled despite working damn fine in all media players.

[simplezz]

Youtube-viewer is sweeeeet!
I downloaded gtk-youtube-viewer from Arch Linux AUR and it's really nice. It just lacks that random suggested video startpage and the suggestions to the right but I can live without that until Firefox gets a grip around VA-API.

Apparently, Firefox has had problems with VA-API hardware acceleration off and on. Some have even had crashes. There are open issues about VA-API hardware acceleration not being enabled despite working damn fine in all media players.

I like the cli version myself.

Ye it's a shame Firefox is so damn slow on Linux (HTML5 media). Even FreshPlayer/Pepper is smoother. I wish Mozilla would stop wasting their time on side projects (Windows 10/metro crap), and make the main browser solid and performant (HW accelerated) on all platforms.

[Radium]

I like the cli version myself.

Ye it's a shame Firefox is so damn slow on Linux (HTML5 media). Even FreshPlayer/Pepper is smoother. I wish Mozilla would stop wasting their time on side projects (Windows 10/metro crap), and make the main browser solid and performant (HW accelerated) on all platforms.

Yeah, they need to trim Firefox down, make it more modular so that you can disable non-browser parts (such as Hello) and not just hide a button (turn the stuff into dynamic libraries). Port it to GTK+ 3 and cut all connections to X11 and stick to GTK+ so that it will run on both Xorg and Wayland natively and follow the advancements of GTK and device drivers tightly without having to spend too much on their own UI coding and spend more time on making Gecko better, improve support for content of all flavors and keep it on the forefront of the web. The CPU usage for just moving around your cursor on a site that has lots of links, images and scripts such as YouTube is crazy high even if you enable hardware accelerated rendering of the content. RedHat is spending time porting Firefox to GTK+ 3 and solve all X11 dependencies. Shouldn't this be in the best interest of Mozilla to trim Firefox down to make it run natively on Wayland which is widely used on all kinds of embedded systems that use hardware accelerated interface such as mobile phones and TVs. This will make Firefox enter more markets and they can make it run on anything and they will become much more relevant again.

gtk-youtube-viewer is buggy, it doesn't clear the search results when you search for something new. Minitube didn't work at all, the version in the Arch repo is too old and I didn't figure out how to compile the latest version due to Google's silly API key. So I'm sticking with smtube which interacts with a HTTP server running on developer's site which returns content for you (formats the search results, includes thumbnails and links to the media on YouTube's CDN). It's slim and easy to use but has no support for logging in, commenting, reading comments etc. It's better to let something light like smtube launch mpv for you than fill Firefox with extensions and plugins and still having to browse YouTube's website, have a player placeholder and still have it launch something external or replace it with a player that lacks the native player's ease of use.
I'll stick with smtube until Firefox and gstreamer plays well together with VA-API hardware acceleration.

Once I figured out that VDPAU_DRIVER should be set to "va_gl" and LIBVA_DRIVER_NAME should be "i965" then all media players play any H.264 I throw at them. I pick "vaapi" as the decoder where it's availible.

I avoid Flash as much as possible since it's no longer developed for Linux, is an enormous resource hog most of the time and is easy to exploit.

Chromium has the same issues with hardware accelerated decoding for certain hardware and the developers don't seem to care much about it either.

Thanks for your help.

[The_Decryptor Veteran]

I like the cli version myself.

Ye it's a shame Firefox is so damn slow on Linux (HTML5 media). Even FreshPlayer/Pepper is smoother. I wish Mozilla would stop wasting their time on side projects (Windows 10/metro crap), and make the main browser solid and performant (HW accelerated) on all platforms.

Yeah, Mozilla is going to drop Windows support to focus on Linux.

They've been trying to do hardware acceleration for years, issue is so many drivers crash that they couldn't enable it by default. They're looking at it again as part of the Wayland port, removing X also reduces their dependence on drivers a fair bit (Since before they'd be shipping X11 surfaces to the GPU, now they'll be doing image surfaces)

[Mindovermaster Moderator]

Yeah, Mozilla is going to drop Windows support to focus on Linux.

Where did you hear that? I heard nothing of that...

[Radium]

 

Where did you hear that? I heard nothing of that...

I'm wondering the same. They did say that they would drop the "Metro" port for Firefox but that was back when "metro" and desktop was separate. Maybe they've changed their mind.

I've done some further testing and Firefox does indeed use hardware accelerated video decoding but it's terrible at handling the decoded video as it seems to take a detour back to RAM before it's integrated into the rest of the page and then being sent back to video RAM before displayed. Changing from Cairo to Skia did lower CPU usage but still not satisfied, CPU usage is still way too high, moving your mouse around seems to be very intense for the CPU. Horrible coding, just horrible.

[Mindovermaster Moderator]

I'm wondering the same. They did say that they would drop the "Metro" port for Firefox but that was back when "metro" and desktop was separate. Maybe they've changed their mind.

I've done some further testing and Firefox does indeed use hardware accelerated video decoding but it's terrible at handling the decoded video as it seems to take a detour back to RAM before it's integrated into the rest of the page and then being sent back to video RAM before displayed. Changing from Cairo to Skia did lower CPU usage but still not satisfied, CPU usage is still way too high, moving your mouse around seems to be very intense for the CPU. Horrible coding, just horrible.

You code at all?  j/k

[simplezz]

Yeah, Mozilla is going to drop Windows support to focus on Linux.

Huh? I didn't even say that. I said Mozilla should concentrate on making the desktop (aka not metro) browser solid and performant rather than wasting development time and resources implementing a browser exclusively for Windows 10. That includes the Windows desktop client. It needs a lot more attention and I fear Mozilla's propensity to get side tracked on useless features will be their downfall.

[simplezz]

I've done some further testing and Firefox does indeed use hardware accelerated video decoding but it's terrible at handling the decoded video as it seems to take a detour back to RAM before it's integrated into the rest of the page and then being sent back to video RAM before displayed. Changing from Cairo to Skia did lower CPU usage but still not satisfied, CPU usage is still way too high, moving your mouse around seems to be very intense for the CPU. Horrible coding, just horrible.

That's unfortunate. Mozilla really needs to get their act together. We shouldn't have these kinds of problems in 2015.

From my limited testing, Firefox's HTML5 video playback performs a lot worse than flash (fresh player/pepper). Clearly it's not working correctly.

[Radium]

 

You code at all?  j/k

Not graphics, no. But I've seen several comments on why CPU use is sometimes higher when hardware accelerated video decoding is on. The decoded video frames are brough back into software (RAM) as the content is structured for being sent to the graphics card again in a different part of the memory used by the hardware renderer if that's even used. If not, each frame is rendered in software and each frame is then just sent to the display server. Firefox defaults to software rendering. OpenGL by default is long overdue and other browsers have the same problems. There's no standardized solution yet and it's only now when hardware accelerated graphics in X11 has become the norm and the arrival of Wayland that the push towards a fully hardware accelerated interface is become a topic. Hardware accelerated graphics in X11 has been availible for many years and web browsers still can't handle it.

Check CPU usage when you scroll or move your mouse in Firefox. It shouldn't be spamming multitple threads with workload that put the CPU cores under 20-30% load. Let's not talk about 1080p60+ playback. Completely outrageous even when only one tab open and no other software running in the foreground, CPU use 40%+. My Broadwell CPU heats up really fast and that never happend when I used Windows. Even watching several videos after another and doing some scrolling and multitabbing at the same time as a video is playing didn't cause the fan to even start. It doesn't take much for Firefox to cause the fan to start spinning. I have turned on OpenGL and off-main-thread-compositing which lowers the CPU use slightly and it's still using crazy amounts of CPU power. Firefox is a known resource hog on all platforms.

[Mindovermaster Moderator]

I only use Chrome these days. Because of the Flash games don't work, I need the latest version.