I have tried Puppy Linux off a USB drive. It works well enough on a lot of the newer PC's, but the boot can be problematic.
I think the boot time is always a bit slow because initial loading of boot sectors are using usb 1.1 for maximum compatibility.
The BIOS is often at fault. Some pc will boot from a fat32, some only fat16. Some will boot only if the drive has a MBR, some will only boot when the drive is formatted like a floppy. Some will boot only on cold start, etc.
I also don't like to have an USB sticking out the side of the machine.
It would be nice to be able to boot from Compact flash or SD, but only a few laptops (mostly tablets) have that capability.
I would not use the flash for swap, it may burn out the card.