Here is how you can do about as close to a double-blind test on your own:
Directions:
* make sure you have mplayer installed.
1. copy this script to /tmp and name it "comparison.sh" and make it executable
Code:
#!/usr/bin/env bash
CDIR=$(pwd)
mkdir links
find ./ -maxdepth 1 -type f | while read i
do
TF="$(tempfile --directory "${CDIR}"/links/)"
rm "$TF"
ln -s "${CDIR}/$i" "$TF"
done
find ./links|sort|while read i
do
mplayer -FS "$i" > /dev/null
done
Be careful with that script.
2. Make a new directory called /tmp/test
3. Copy all the media files you want to compare to that directory.
A older Theora comparison can be found here if you can't find a better comparison:
http://people.xiph.org/~greg/video/y...omparison.html
4. Change to that directory and run the script like such:
cd /tmp/test
../comparison.sh
------------------------
How that works is that it creates a /tmp/test/links directory and creates randomly named symbolic links back to your files in /tmp/test directory. Then it has mplayer play them in alphabetical order according to the randomly generated names.
If you want to you can replay them over and over again using:
find links |sort|while read i; do mplayer -FS "${i}" > /dev/null ;done
This is nice because since mplayer is command line and your supressing output then you should have no way to know what codec or rate is encoded any file. (Other then you viewed the files prior the test and recognized visual artifacts that indicative of one codec or another)
Then you record your perception of quality according to the order you played them.
Then after your satisfied you know how the files rank in your comparison you can reveal the codecs and rates by using "ls -l links".
That way, as long as your honest, you can get about as close as a real comparison as your going to get. And since it matches up with how you percieve stuff then it will show you the best choices for yourself.
-------------
Oh. And the above works really well for audio files, too.
Although for audio files you will want to throw in the original uncompressed (or lossy flac comparessed) into the comparison.
People have shown that after listening to a particular type of audio compresson they begin to favor the "sound" that that codec tends to produce. For example mp3 tends to make the "highs" stand out, and people begin to favor that. So you can use this technique to "reset" your ear so you know what is actually more accurate sound reproduction, if you care about that sort of thing.
After all its all reproduction and will never accurately show the original performance... so the best sound quality is always going to be "live".
Unless your a fan of Britney Spears or other pop acts since they actually just play back recordings at concerts!