frod Posted January 15, 2004 Share Posted January 15, 2004 i figured i would post this here in the mac forum as it sort of relates to the itunes music store. i have witnessed a few arguments about whether or not aac is better than wma or vice versa. using their "points" at whether or not aac was a viable choice for apple and the itms. well i came across this site: http://audio.ciara.us/test/128extension/results.html it tested 128kbps encoded files from aac, mp3, mpc, ogg, and wma in the end, my favorite codec, mpc (musepack) won, heh. unfortunately i can't use that on my mac :( no codecs for quicktime. aac was a close second. here are the scores, averages of the scores received for various songs: mpc: 4,51 aac: 4,42 wma: 4,30 ogg: 4,28 lame: 3,66 i guess mp3 is truly being left in the dust as lame scores were pretty abysmal in these tests. Link to comment Share on other sites More sharing options...
hornett Posted January 15, 2004 Share Posted January 15, 2004 I think they should of adjusted all the codecs to be using roughly the same average bitrate. MPC's average is 20kbps more than Lame's. Link to comment Share on other sites More sharing options...
jaffa Posted January 15, 2004 Share Posted January 15, 2004 i figured i would post this here in the mac forum as it sort of relates to the itunes music store.i have witnessed a few arguments about whether or not aac is better than wma or vice versa. using their "points" at whether or not aac was a viable choice for apple and the itms. well i came across this site: http://audio.ciara.us/test/128extension/results.html it tested 128kbps encoded files from aac, mp3, mpc, ogg, and wma in the end, my favorite codec, mpc (musepack) won, heh. unfortunately i can't use that on my mac :( no codecs for quicktime. aac was a close second. here are the scores, averages of the scores received for various songs: mpc: 4,51 aac: 4,42 wma: 4,30 ogg: 4,28 lame: 3,66 i guess mp3 is truly being left in the dust as lame scores were pretty abysmal in these tests. dude you forgot Sony's Atrac3 codec :happy: Link to comment Share on other sites More sharing options...
lexor Posted January 15, 2004 Share Posted January 15, 2004 (edited) for both of the above complaints please visit the hydrogenaudio.org forum (where the test was conducted) all those concerns were answered and then some. to give you a sneak peek, in vbr mode you tell encoder that you want average of 128 and it allocates accordingly, it doesn't waste when it doesn't need to. so Lame encoder decided that it needs 20 less. Also 128 is a LONG TERM average, each song, even 3 consecutive songs shouldn't neccesserely average to 128 ;) that is ABR mode that does that not VBR which was tested. though I find 64kbps test more interesting since it shows every1's claims that they can offer MP3 128 quality at 64kbps false. Edited January 15, 2004 by lexor Link to comment Share on other sites More sharing options...
DreamweaverN Posted January 15, 2004 Share Posted January 15, 2004 So these are hydrogenaudio tests? I was just wondering because it kind of looks like it was made up (not saying it was just want to know abit about the background of the tester/s). Link to comment Share on other sites More sharing options...
lexor Posted January 15, 2004 Share Posted January 15, 2004 yes the test was conducted by rjamorim at HA.org that's just a host he was using for the page/test files finished test thread: http://www.hydrogenaudio.org/index.php?showtopic=11936& all tests: http://audio.ciara.us/test/ Link to comment Share on other sites More sharing options...
DreamweaverN Posted January 15, 2004 Share Posted January 15, 2004 Thanks! Interesting though, AAC was right up there. I think I might rerip and use AAC now :). Just got to figure out what software :D. Link to comment Share on other sites More sharing options...
SimplyPotatoes Posted January 15, 2004 Share Posted January 15, 2004 people just listened to these to figure out which is best? Link to comment Share on other sites More sharing options...
lexor Posted January 15, 2004 Share Posted January 15, 2004 (edited) that depends on what you mean by "people just listened to these" - the codecs? well du'h... they did - the files? well the files were specifically chose to represent most if not all spectrum of music (especially difficul to handle instruments like harpsichord for ogg) - if you mean the test was "just conducted" then no, it's from last summer. Oh yeah I just remembered, there was a thread on HA made after some1 called the developer of MPC. Basically the developer considered MPC's performance "a slap in a face" for other codecs, which he thought had a technologically superior design. Basically he blamed that AAC implementation is poor at the moment and has quite a bit to improve. Edited January 15, 2004 by lexor Link to comment Share on other sites More sharing options...
kairon Posted January 15, 2004 Share Posted January 15, 2004 Glad to see AAC beats OGG. That might shut up those who say otherwise. Kinda saddening to see the codec that won isnt even useable on a mac. Link to comment Share on other sites More sharing options...
SimplyPotatoes Posted January 15, 2004 Share Posted January 15, 2004 that depends on what you mean by "people just listened to these"- the codecs? well du'h... they did - the files? well the files were specifically chose to represent well what a half ass test then, why trust peoples opinions when you could create a test involving the math! i mean i made a little prog to check the output of my aud2 just 2 see if all the different drivers changed anything....i mean it wouldn't be very hard to create a test which would compare the musical quality of the encoded files to the original maybe include the sensitivity of human hearing freqs or w/e, if they spend that much time listening to get an opinion...thats just retarded. Link to comment Share on other sites More sharing options...
RauL Posted January 15, 2004 Share Posted January 15, 2004 HA to those people saying WMA was crap just beacuse it was from MS. the 9series were a great step forward (a lot better than encoder 7 /8) Link to comment Share on other sites More sharing options...
lexor Posted January 15, 2004 Share Posted January 15, 2004 well what a half ass test then, why trust peoples opinions when you could create a test involving the math! i mean i made a little prog to check the output of my aud2 just 2 see if all the different drivers changed anything....i mean it wouldn't be very hard to create a test which would compare the musical quality of the encoded files to the original maybe include the sensitivity of human hearing freqs or w/e, if they spend that much time listening to get an opinion...thats just retarded. :laugh: no seriously go to HA and propose it. they'll be better equiped to unswer, but you might wanna search their forum first, couse that has been brough up before. in short there is no mathematical way to test quality if there was do you think some1 wouldn't have come up with it yet? The only scientific test and statistically significant for music quality is a double blind test, which this was. Nothing better exists atm. Howerver HA holds a statistical/pragmatical view of audio quality, and some members there do occasioanlly get flamed at Audiophile boards (aka Audiophiles believe they can hear a difference between 12 and 18 guage speaker cable) If you come up with a stricked math test that can measure quality and shut up all the Audiophilic mambo jambo, guys at HA (me included) would be VERY grateful :yes: Link to comment Share on other sites More sharing options...
threetonesun Posted January 15, 2004 Share Posted January 15, 2004 I've been switching up between ogg, aac, and mp3 lately. Can't say I can tell the difference between ogg and aac personally, but maybe I'll have to try some of those other options now. Link to comment Share on other sites More sharing options...
the evn show Posted January 15, 2004 Share Posted January 15, 2004 If I were going to scientificly detrimine which codec sounded better I would: 1) get a few dozen songs from varous genres with different sounds, lengths, and features. 2) Encode all using MP3 (the various encoders), and figure out which was best overall 3) Repeat step 2 using WMA, AAC, MPC, MP3Pro, RA, and whatever else is used 4) Take the "best overall" encodings and output them to WAV 5) Take the wave form for each encoding and pull them into photoshop. 6) Overlay the above wave with the original, set the top layer to "difference" 7) There you have it: the one with the most black is farthest from the original. The problem with the above method is that only tells us which one tosses out the most data. Equally important is what frequency losing that data has on the final sound that you hear (ie: if you loose everything from 18-22khz that's bad. if you loose everything from 500-1500hz that's much worse). Losing a little here and a little there may be indistinguishable from the original to unaided ears. That's why you'd need double blind listening tests that look for how close a given encoding sounds to the original reference. Looking for "what sounds best" isn't good because - for example - most people like bass, and a slight increase in the thumpiness of song might sound 'better' to a listener than the original. The problem is that if you have an encoder coloring the sound - it may make some tracks sound "better" (but wrong) but it's equally likely to take something that doesn't need any more bass and turning it into crap. EDIT: spelling sucks, i just woke up. I've got to start leaving my computer somewhere other than my bedroom. Link to comment Share on other sites More sharing options...
xRKx Posted January 15, 2004 Share Posted January 15, 2004 Thanks! Interesting though, AAC was right up there. I think I might rerip and use AAC now . Just got to figure out what software On my XP box, I just use FB2K to encode all of my stuff in mp4 format (though I keep the mp4 extension rather than aac.) and it does a beautiful job. MP4's simply a great format anyway - you can often get smaller file sizes (due to better compression,) than MP3, mpc or ogg. That, and of course, mp4/AAC is a standard, like MP3. Link to comment Share on other sites More sharing options...
lexor Posted January 15, 2004 Share Posted January 15, 2004 (edited) I don't get you the evn show, what are you saying? you seemed to describe the test as they've done it in your 7 points, then you say they should compare to unaltered/uncompressed source (which they did, but it's not shown on the graph), and when you test you don't know which one was the source and if you rated the source less than ecoded your test score was discarded. (which makes sense) the source represented the top of the scale, Blade the bottom (because it's known to be one of the worst if not the worst implementations of mp3). everything was ranked in between. using double blind test because you don't know which sample is which when you listening, you are only offered rating sliders to drag. and computer can't relate hints (i.e. can't point at one slider and say that's mp3) thus the test was double blind. now what are you saying? I don't understand if you support or disaprove of it? Edited January 15, 2004 by lexor Link to comment Share on other sites More sharing options...
frod Posted January 15, 2004 Author Share Posted January 15, 2004 i had a friend that did a similar test using an audio signal analyzer piece of software. he compared mp3, ogg, mpc, vqf and wma to cd audio. musepack came closest to cd audio at the lowest bitrate of the 5 codecs tested. mp3 theoretically became "cd quality" at around 256kbps. wma was a little better, and mpc was somewhere around 160-192 for "cd quality". at the time ogg was farely new and did horrible in the tests. vqf also did horrible although no one really cares about vqf. Link to comment Share on other sites More sharing options...
Recommended Posts