How to program with 2D acceleration

[tball]

Hello

I wonder how you could use the capability of 2d (or 3d acceleration) from you gfx card, for lets say, stream a video from you webcam.

If for an example I wan't to draw a frame from opencv (webcam capture lib) to a QImage (qt image object), I have to convert the CV image frame to a QImage format. But the webcam fetching / conversion / drawing to the screen is taking about 30 % cpu. Thats a lot for a C2D 2,5 GHz.

Is there any way I could do this conversion / showing via the GPU instead of the CPU? Or is there any other approach I could try?

[monraaf]

Hello

I wonder how you could use the capability of 2d (or 3d acceleration) from you gfx card, for lets say, stream a video from you webcam.

If for an example I wan't to draw a frame from opencv (webcam capture lib) to a QImage (qt image object), I have to convert the CV image frame to a QImage format. But the webcam fetching / conversion / drawing to the screen is taking about 30 % cpu. Thats a lot for a C2D 2,5 GHz.

What's the capture size and frame rate? And how much cpu is it taking when you use the opencv libary to display the video?

Is there any way I could do this conversion / showing via the GPU instead of the CPU? Or is there any other approach I could try?

Maybe you can try the video 4 linux interface for capturing, some webcams give you back the image in YUV420P, color space conversion and scaling can then be done with Xv.

[tball]

What's the capture size and frame rate? And how much cpu is it taking when you use the opencv libary to display the video?

It 20 fps, at 640x480. It actually I think opencv uses the v4l interface behind the all, but I could be wrong.

Maybe you can try the video 4 linux interface for capturing, some webcams give you back the image in YUV420P, color space conversion and scaling can then be done with Xv.

Actually before I saw your reply, I have already tried implementing a fast opencv -> sdl -> show to screen. It works quite nice actually. God the cpu usage down to ~9%.

Would xv be faster? If so, which lib should I use. Any docs?
Thank for your reply.

[monraaf]

I have already tried implementing a fast opencv -> sdl -> show to screen. It works quite nice actually. God the cpu usage down to ~9%.

Would xv be faster? If so, which lib should I use. Any docs?
Thank for your reply.

If you want to display the webcam video at it's native size I doubt it would be much if any faster. If however you want to display the image at a larger size, then yes.

SDL can make use of Xv, but the data you feed it must be in planar YUV format. Basically you first create a YUV overlay of type SDL_YV12_OVERLAY:

SDL_CreateYUVOverlay(...)

then in a loop you read the data from your webcam, copy it into the overlay and then display the overlay:

SDL_LockYUVOverlay(...)
memcpy(..) // y
memcpy(..) // u
memcpy(..) // v
SDL_UnlockYUVOverlay(...)
SDL_DisplayYUVOverlay(...)

[tball]

Okay, I actually found out that using top instead of the kde monitor, my cpu is using ~21 % cpu, so I decided to try the XVoverlay out.

When using a standard surface, my test scheme is this:

    SDL_Surface *screen;
    SDL_Surface *SDLimage; // I have filled this surface with a static picture before running the while loop.

screen = SDL_SetVideoMode(640,480,24,SDL_HWSURFACE|SDL_NOFRAME)))


    while(1)
    {
    //Draw pixels to screen here
    SDL_BlitSurface(SDLimage, NULL, screen, NULL);
    SDL_FreeSurface(SDLimage);
    SDL_Flip(screen);

    SDL_Delay(50);   // Delay, so you will get ~ 20 fps
    }

And my test scheme for a xv overlay is:

#define SDL_YV12_OVERLAY  0x32315659  /* Planar mode: Y + V + U */

SDL_Rect where;
    where.h = 480;
    where.w = 640;
    where.x = 0;
    where.y = 0;
SDL_Surface *screen;
SDL_Overlay *xvoverlay;


screen = SDL_SetVideoMode(640,480,24,SDL_HWSURFACE|SDL_NOFRAME)));
xvoverlay = SDL_CreateYUVOverlay(640, 480, SDL_YV12_OVERLAY, screen);


while{1}
    SDL_LockYUVOverlay(xvoverlay);
    SDL_UnlockYUVOverlay(xvoverlay);
    SDL_DisplayYUVOverlay(xvoverlay,&hvor);

    SDL_Delay(50);   // Delay, so you will get ~ 20 fps

}

But the last one with a xvoverlay, is quite more cpu hungry? It eats approx 30 % cpu. Am I doing something wrong? In the xvoverlay code I doesn't even copy any YUV data into the overlay.

Or is it just faster using a sdl surface, and blit it to the screen?

[monraaf]

screen = SDL_SetVideoMode(640,480,24,SDL_HWSURFACE|SDL_NOFRAME)));

Changing the 24 to 0 should fix it. You can check if Xv is being used by playing a video with mplayer. It will output something like this:

[VO_XV] Could not grab port 57

But the last one with a xvoverlay, is quite more cpu hungry? It eats approx 30 % cpu. Am I doing something wrong? In the xvoverlay code I doesn't even copy any YUV data into the overlay.

Or is it just faster using a sdl surface, and blit it to the screen?

It all depends on the source image format and if you're planning to scale the image to a higher resolution.

[tball]

Changing the 24 to 0 should fix it. You can check if Xv is being used by playing a video with mplayer. It will display something like this:

[VO_XV] Could not grab port 57

Wauu now it uses ~ 0 % cpu? Can that be real? Is it using the gpu now?
When you change it to 0, wouldn't it mean the depth is 0 and the image useless? Or isn't the xv overlay using it.

Thanks for your time and help.

EDIT:

Actually my goal is to put all the work onto the GPU, because the cpu has to make a lot of calculations on the image in parallel.

[monraaf]

When you change it to 0, wouldn't it mean the depth is 0 and the image useless?

"If bpp is 0, it is treated as the current display bits per pixel."

http://www.libsdl.org/docs/html/sdlsetvideomode.html

[tball]

"If bpp is 0, it is treated as the current display bits per pixel."

http://www.libsdl.org/docs/html/sdlsetvideomode.html

Ahh I see. Thanks.