Per seat USB is actually fairly easy since Pulse Audio. I use the setup described here

https://help.ubuntu.com/community/MultiseatX...