Announcement

Collapse
No announcement yet.

VAAPI Segmentation fault right after va_getDriverName()

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • VAAPI Segmentation fault right after va_getDriverName()

    Hi,

    It's that time again that i try out VAAPI with XvBA and yet again it manages to take nearly a full day of screwing around trying every possible combination to get the vainfo command running. Here is the output i get:

    libva: libva version 0.31.1
    Xlib: extension "XFree86-DRI" missing on display ":0.0".
    libva: va_getDriverName() returns 0
    Segmentation fault
    I tried multiple libva versions. The one provided in the archlinux repo, the git version and the one provided on splitted desktop systems. None seem to work.

    I also have xvba-video 0.7.8 installed and the latest catalyst (11.2) is installed. I'm completely out of ideas to try and get this darn thing running.

    Anyone any ideas that i can try? I'm running archlinux fully updated.

    Regards,
    Mark

    ps. Why isn't this seg fault fixed..? I mean, it's not like it's a "new" issue since if you search for it on google it's seems to happen a LOT!

  • #2
    AWESOME! I found a total of 3!!! freaking bugs!

    1. if libva passes a null pointer to va_openDriver for the "driver_name" then the malloc function below will segfault! However, that is the effect. The cause of the effect is that the function is entered when it should not do that. For that i made a patch.
    PHP Code:
    VAStatus vaInitialize (
        
    VADisplay dpy,
        
    int *major_version,     /* out */
        
    int *minor_version      /* out */
    )
    {
        const 
    char *driver_name_env NULL;
        
    char *driver_name NULL;
        
    VAStatus vaStatus;

        
    CHECK_DISPLAY(dpy);

        
    va_TraceInit(dpy);

        
    va_FoolInit(dpy);

        
    va_infoMessage("libva version %s\n"VA_VERSION_S);

        
    driver_name_env getenv("LIBVA_DRIVER_NAME");
        if (
    driver_name_env && geteuid() == getuid())
        {
            
    /* Don't allow setuid apps to use LIBVA_DRIVER_NAME */
            
    driver_name strdup(driver_name_env);
            
    vaStatus VA_STATUS_SUCCESS;
            
    va_infoMessage("User requested driver '%s'\n"driver_name);
        }
        else
        {
            
    vaStatus va_getDriverName(dpy, &driver_name);
            
    va_infoMessage("va_getDriverName() returns %d\n"vaStatus);
        }

        if (
    VA_STATUS_SUCCESS == vaStatus && driver_name != 0)
        {
            
    vaStatus va_openDriver(dpydriver_name);
            
    va_infoMessage("va_openDriver() returns %d\n"vaStatus);

            if (
    vaStatus != VA_STATUS_SUCCESS && !driver_name_env)
            {
                if (
    va_hasCrystalHD())
                {
                    
    vaStatus va_openDriver(dpy"crystalhd");
                    
    va_infoMessage("va_openDriver() returns %d\n"vaStatus);
                }
            }

            *
    major_version VA_MAJOR_VERSION;
            *
    minor_version VA_MINOR_VERSION;
        }
        else
        {
            
    vaStatus VA_STATUS_ERROR_UNKNOWN;
        }

        if (
    driver_name)
            
    free(driver_name);
        return 
    vaStatus;

    Really, shame on the creators for leaving this in so long!
    Oh and btw. I don't know the inner workings of libva but the function above (vaInitialize) really doesn't need to fetch the driver_name_env to driver_name.. Just fetch it, chec if it's not null and proceed if it has some value. My "patch" in that regards is not fine since i don't fix that but i'm not the libva dev so it's up to those people to fix this. Note: my patch is just a quick fix to prevent it from segfaulting!

    That was bug 1 and the most severe bug.
    Bug 2 was a unset LIBVA_DRIVERS_PATH
    Bug 3 was a unset LIBVA_DRIVER_NAME

    To be complete. Just setting those 2 defines like so (in my case):
    export LIBVA_DRIVER_NAME=xvba
    export LIBVA_DRIVERS_PATH=/usr/lib/dri/

    solved my issue and vainfo works again.

    Comment

    Working...
    X