Microsoft's WDP vs. Standard JPG, in Numbers

I heard you. Many users commented on my previous news post saying that they weren’t happy with the tests I performed. I understand where you were coming from, and I plan to address what I believe is probably the most important of the tests that were suggested. Now that an overview of the plug-in for WDP is out there, let’s verify Microsoft’s claim: “It compresses data twice as efficiently as JPEG, with either twice the quality at a given file size or half the file size at a given quality.”

My final test was performed by taking the two file formats in question and comparing which one can offer the smallest compression, without a major loss in quality. I saved the same file in both formats with lower and lower quality settings, until I hit visible artifacts. I then saved the file again a step up in quality (without artifacts) and compared to see if there were any obvious quality differences, while noting the file sizes. As I concluded in my previous article, WDP offers a much broader range of quality settings, and I wanted to see if this is why Microsoft claims that it can compress data twice as efficiently. Here are my findings for five images, two in black and white and three in colour:

  • A 1.0MB black and white TIFF image of a crouching man, at a resolution of 1024 x 1024
    • JPG: 75.6KB (Quality: 0/12)
    • WDP: 42.3KB (Quality: 0.3/1)
    At the lowest quality, JPG did not have artifacts, but it did get very fuzzy. With WDP just a bit over half of JPG’s file size, JPG was the clear winner in quality. The man’s face and hands were much more clear and crisp.
  • A 1.85MB colour TIF aerial image of the world, at a resolution of 2048 x 1024
    • JPG: 86.5KB (Quality: 0/12)
    • WDP: 83.1KB (Quality: 0.4/1)
    Even with 3KB of difference, I could distinctly notice that JPG retained the quality much better than WDP did. There was no point of comparing at half the file size.
  • A 12.9MB black and white TIF image of a boxer, at a resolution of 4096 x 3038
    • JPG: 209KB (Quality: 0/12)
    • WDP: 92.0KB (Quality: 0.4/1)
    Well below half of the JPG’s file size, the WDP file was very hard to tell a part from the JPG. There were differences, but they were so minor that I spent a good while making sure which one had a better quality. JPG did have better quality, but from what I could see, it was an immaterial difference.
  • A 10.3MB colour CR2 image of Dave Legg’s laptop taken by the man himself, at a resolution of 3888 x 2592, with a Canon EOS 400d 10.1MP camera.
    • JPG: 206KB (Quality: 0/12)
    • WDP: 115KB (Quality: 0.4/1)
    Almost half of the JPG’s file size, the WDP file was very hard to tell a part from the JPG. There were differences, but they were so minor that I could not decide which had the better quality.
  • A 13.4MB colour Olympus Raw File of a metal art plate, at a resolution of 3264 x 2448
    • JPG: 252KB (Quality: 0/12)
    • WDP: 169KB (Quality: 0.4/1)
    Not quite half of the JPG’s file size, the WDP file was very hard to tell a part from the JPG. There were differences, but they were so minor that I could not decide which had the better quality.
As we can see from the above, the bigger the image, the better WDP performs. When you want those really small images in order to save bandwidth, it looks as if you’ll need a pretty large starting image. This isn’t a problem with newer and newer digital cameras hitting the market, many of them supporting a RAW file format. Microsoft hopes to push this file format and the obvious place to start is with standardizing it for digital photography. Nevertheless, it needs to be able to perform better with other images, including those found on the Internet.

So, what happens when the two images are compared at the same compression percentage? Firstly, you will notice very little or no difference in quality. Will you notice a difference in size? Yes. Will it be half the size? Not always.

  • A 1.0MB black and white TIFF image of a crouching man, at a resolution of 1024 x 1024
    • JPG: 309KB at 75% (9/12)
    • WDP: 253KB at 75% (0.75/1)
    • JPG: 211KB at 50% (6/12)
    • WDP: 83.2KB at 50% (0.5/1)
    At 75%, there is no visible difference in quality but WDP is not half the size of JPG. At 50%, WDP is less than half the size, but a loss in quality can be perceived.
  • A 1.85MB colour TIF aerial image of the world, at a resolution of 2048 x 1024
    • JPG: 340KB at 75% (9/12)
    • WDP: 335KB at 75% (0.75/1)
    • JPG: 204KB at 50% (6/12)
    • WDP: 121KB at 50% (0.5/1)
    At 75%, there is no visible difference in quality but WDP is nowhere near half the size of JPG. At 50%, WDP is nearing half the size and a loss in quality can be perceived.
  • A 12.9MB black and white TIF image of a boxer, at a resolution of 4096 x 3038
    • JPG: 979KB at 75% (9/12)
    • WDP: 642KB at 75% (0.75/1)
    • JPG: 603KB at 50% (6/12)
    • WDP: 124KB at 50% (0.5/1)
    At 75%, there is no visible difference in quality but WDP is not half the size of JPG (at least a third less though). At 50%, WDP is one sixth of the size and no loss in quality can be perceived.
  • A 10.3MB colour CR2 image of Dave Legg’s laptop taken by the man himself, at a resolution of 3888 x 2592, with a Canon EOS 400d 10.1MP camera.
    • JPG: 1.06MB at 75% (9/12)
    • WDP: 766KB at 75% (0.75/1)
    • JPG: 558KB at 50% (6/12)
    • WDP: 160KB at 50% (0.5/1)
    At 75%, there is no visible difference in quality but WDP is not half the size of JPG (a quarter less though). At 50%, WDP is over a third of the size but no loss in quality can be perceived.
  • A 13.4MB colour Olympus Raw File of a metal art plate, at a resolution of 3264 x 2448
    • JPG: 959KB at 75% (9/12)
    • WDP: 528KB at 75% (0.75/1)
    • JPG: 581KB at 50% (6/12)
    • WDP: 221KB at 50% (0.5/1)
    At 75%, there is no visible difference in quality and WDP is almost half the size of JPG. At 50%, WDP is less than half the size and no loss in quality can be perceived.
In conclusion, WDP isn't always at 50% but often enough it is smaller than JPG with little loss to no loss in quality. Nonetheless, one must take into consideration that the higher the resolution, the harder it is for the human eye to detect discrepancies between images in different file formats. Therefore, since WDP offers a wider choice in settings, it is better at compressing larger images without major difference in quality, simply because the loss is inconspicuous at bigger sizes. That is what matters when it comes to digital photography, especially as cameras improve. Microsoft's new format will, however, have to perform better than JPG with smaller images, if it's going to make it. What really peaks my curiosity is the fact that this is a beta release. How much of an improvement (if any) will the final be from the beta?

Note: If you are wondering why there are no images that you can compare by yourself, it is because to get a true comparison, I would have to display both WDP and JPG. Browsers currently do not display WDP images. The links are there for you to do the tests yourself, or you can just take my word for it!

Report a problem with article
Previous Story

YamiPod 1.0.3 Beta

Next Story

Mp3tag 2.37h

43 Comments

Commenting is disabled on this article.

Ok, I am so tired of everyone trying to compare HD Photo (Windows Media Photo) and JPG.

Either you don't get it, or you are trying to create controversy that doesn't exist.

HD Photo DOES several things you cannot do univerally in other formats, including JPG. The fact that the image quality is on par with JPeg is just one of the side effects.

Here is what IS important about HD Photo...

· HD Photo does Mipmaps (for example, a website could provide a HIGH RESOLUTION IMAGE in HD Photo format, but only the displayed thumbnail size selected is sent to the client. Examples of this can be found in the British Library applications that are using HD Photo and Microsoft's 3D Photo project that combines photos in high resolution into a semi virtual 3D space.)

· HD Photo provides higher bit storage than JPEG, this gives professional photographers the ability to use the full spectrum of what the new digital imaging chips are producing, even though most people cannot see the difference with the naked eye. And yes the extra bits are VERY important when dealing with image manipulation and output.)

· HD Photo allows for BOTH Lossy and Non-Lossy compression. JPEG is Lossy only, and other formats are usually all non-Lossy. HD Photo gives the best of both world options in one format. (i.e. Non-Lossy great for Text and Graphic Arts, Lossy Great for Photos)

· HD Photo is designed to chunk data specifically in the way digital imaging devices capture the Raw image, this speeds up the process of getting the data from the imager to the storage device on the card, and by also being faster than JPEG extra speed is added even further.

There currently is NO image format standard that meets just these features that HD Photo does in one format. Right now we are having to choose between PNG, TIFF, JPG, etc. And each has advantages and limitations making any form of universal image currently not possible. For example, you could decide to go with PNG for your media, but PNG doesn't support meta tags, or you could go with TIFF, but it is more of an enclosure layer and the compression used inside can be many different types and cause problems with software that don't understand the internal compression, or you could choose JPG, but lose resolution, color fidelity and lossless compression.

Instead, you could use HD Photo which doesn't have the same limitations, and we could eventually move to a universal image format that not only does more than what we need today, but is extensible enough to support even future imaging needs.

All this comparing JPG to HD Photo is crap, as you are using beta software and then trying to determine the quality based on your eye. And if your monitor is more contrasted than someone else’s, one format is going to look better, and if your monitor is less contrasted another format is going to look better. Let alone the fact you are viewing these images on LCDs and Monitors that CANNOT even product the entire spectrum of the format in the first place. If you think you can see the difference between 24bit and 36bit on an LCD that displays at 18bit, you are insane.

So please stop the comparisons..

And I apologize to the person who wrote this article in advanced, but if this is your knowledge and understanding of digital imaging, you need to find someone else to do these types of reviews.

Why do people keep comparing it to jpeg2000? That format is practically dead thanks to all sorts of legal reasons. I mean ffs, look at the name, it's been around since 2000 and 7 years on, nobody uses it for anything.

Instead of relying on the 'art' where lines and colours can be lost, why don't you do a test with alphanumeric content?

If a high resolution picture of a letterboard, maybe submitted in raw format from someone who has a camera capable enough is posted ... will that not be a better judge than a few pictures with fine details that getlost and unnoticed?

Yeah, we care because we want to know whether it looks gd and doensnt eat my space. We are discussing whether it does or not. How else would you know if it was better or not?

Well, if you wanted to show us what the difference was, why not just save the original file as a WDP and JPG, then re-save them as PNG images?

thenewbf said,
Well, if you wanted to show us what the difference was, why not just save the original file as a WDP and JPG, then re-save them as PNG images?

Which is what I did a few posts up. I'm shocked that the HDP doesn't look as good as JPEG in this case at the same file size. I've posted on Bill Crow's blog about this and I'm interested in seeing his response.

PNG is a lossless format that uses sliding window compression and predictive filters to compact the image.

JPEG is a lossy format that removes high frequency information in order to compact the image

You can't compare apples to oranges. They serve two different purposes.

PNG are good for web buttons and such while JPEG are good for photos.

dduardo said,
You can't compare apples to oranges. They serve two different purposes.

Aren't both PNG and JPEG files considered to be image formats?

kizzaaa said,

Aren't both PNG and JPEG files considered to be image formats?

Spaceships and automobiles are considered vehicles, but they serve two different purposes.

I can't say I'm too impressed by HD Photo yet. JPEG is certainly due for a replacement, but a quick test shows the quality to be worse at the same file size. Perhaps the Photoshop plugin isn't up to the task yet, but the results aren't even as good as JPEG.

Here's an example showing exactly what I mean. To me, the HDP version doesn't look nearly as good:

http://farm1.static.flickr.com/128/4176383...745780e3c_o.png

Ahhhaaa, the goggles, they do NOTHING! x)

You should have compressed that picture in a completely lossy format. It would have looked better.

You're completely right there, the Jpg looks better!
For further illustration, here's a comparison between Jp2 (Jpeg2000) and Jpg.
The pic used for this comparison is the uncompressed Tiff showing the world map (pic 2 from the test):
http://veimages.gsfc.nasa.gov/2433/land_sh...w_topo_2048.tif

It has been saved to Jpg and Jp2 with the same file size of 87kB! While the Jp2 on the left still looks nice, the Jpg on the right rather looks like something you would find in the Museum for Modern Art :laugh:
The two files, along with the other comparison pics, are included in this archive, so you can see for yourself:
http://rapidshare.com/files/20515297/Neowi...rison2.zip.html


I've done a comparison between Jpg and Jpeg2000 using the same Tiff files given above, again using the freeware FastStone Image Viewer (btw, XnView is another free program that fully supports Jpeg2000).
I used the same size for the Jp2 files as for the Wdp files in the test above to have a good base for comparison.

I've uploaded Jpg and Jp2 in a zip here so you can compare for yourself:
http://rapidshare.com/files/20515297/Neowi...rison2.zip.html

First part (lowest Jpg quality)
1) Crouching man
With nearly half the size of the Jpg file, Jp2 doesn't have any artifacts, but loses a little bit of sharpness.

2) World map
With hardly any difference in file size, the quality of Jp2 is clearly superior to Jpg (which by itself is superior to Wdp... now there's food for thought).

3) Boxer
Even though the Jp2 file has less than half the size of the Jpg file, it still has clearly better quality.

4) Dave Legg’s laptop
Again, the Jp2 file has a clearly better quality than Jpg, even though the Jp2 is only about half the size.

5) Metal art plate
Even with the lowest quality possible (1%), I couldn't get the Jpg smaller than 377kB. With Jp2, I easily hit 173kB at a quality of 20% (so it could've got quite a bit smaller still). Again, the Jp2 quality beats that of the Jpg.

Second Part (Jpg 50% quality)
1) Crouching man
At 50% quality, the Jpg is 100kB, while the Jp2 is 51kB. Pretty much half the file size for the same quality.

2) World map
Here, the Jpg is 331kB with 50%, while Jp2 is 308kB, but retains a better quality. With a quality of 20%, Jp2 has a comparable quality with a file size of 123kB (well below half).

3) Boxer
Jpg with 50% quality reaches 410kB, while Jp2 is reaching 200kB there (less than half).

4) Dave Legg’s laptop
Using 50% quality, Jpg takes still 2.5MB, while Jp2 takes only 1.4MB while still having better quality. Jp2 reaches a comparable quality at 25% with a file size of 737kB (less than a third).

5) Metal art plate
A full 2.6MB is the space a Jpg with 50% quality still requires here. Jp2 manages quite an amazing feat here as its file at 50% quality is only 411kB, which is only about 16% of the size!

Conclusion:
With greyscale pics, Jp2 generally has half the file size (or even less) of Jpg at the same quality setting.
With colour pics, the quality setting though is not comparable. While Jp2 pics with the same setting (e.g. 50%) are still bigger than half the size, they still have better quality, so you generally can reduce the setting to get a file with half the file size and still have a quality that is comparable to the Jpg file.

In essence, one could say that Jp2 (unlike Wpd) fully fulfills the claim: “It compresses data twice as efficiently as JPEG, with either twice the quality at a given file size or half the file size at a given quality.”

Hm.....
you can see difference only if you compress picture to the maximum, and who will make that ? If you converting picture you like to be good not to be distorted. When converting from RAW or PSD there is not difference. I take shot with my Canon 400D with RAW quality and compare pictures. There is not BIG differences. It is hard to compare all formats because WDP format can be opened an seen only in Windows Photo Gallery in Vista and Photoshop. PSD format can't be seen in Windows Photo Gallery so you can't compare this formats here. Programs like ACDSee or XnView have support for PSD, JPG, JPEG2000 but not for WDP, so I compare all formats but screenshot is only from WDP 16 bit at Low quality and JPG 8 bit at Low quality saved with Photoshop (0 quality). It's a part of a picture. Test archive with all formats is 35,0 MB I can't post it here.
This is RapidShare link: Compare fileformats

Normal JPEG's can be optimized further (I.E. it is not well optimized by default). This application lets you set different compression levels for color and luminance values, and can increase PERCEIVED quality by quite a bit.

Also, there is a difference between Black and White (1bit), Grayscale (8/16bit) and Color (24/48bit). Grayscale != Black and White.

See now this kind of discussion is what makes Neowin awesome.
It's still largely a "this is what i notice" test, but it's still a good indicator.

A side question. Let's take A 1.0MB black and white TIFF image of a crouching man. When I zoom it in to max using default Windows Picture Viewer, I do not see any pixelation, pretty cool. Why is that??? It's not like TIFF is a vector format.

nigor said,
A side question. Let's take A 1.0MB black and white TIFF image of a crouching man. When I zoom it in to max using default Windows Picture Viewer, I do not see any pixelation, pretty cool. Why is that??? It's not like TIFF is a vector format.

because picture viewer uses advanced interpolation, this doesn't mean the detail is there, picture viewer just makes it up

XerXis said,

because picture viewer uses advanced interpolation, this doesn't mean the detail is there, picture viewer just makes it up :)

Well, essentially it just blurs it. Can be quite annoying if you want to read a detail of a small image.

For those who are familiar with signal processing:

HD Photo Versus JPEG

HD Photo is based on lapped biorthogonal transform (LBT), which in itself is based of discrete cosine transform (DCT), which is based on discrete fourier transform (DFT).

LBT was developed to handle the blocking issue that occurs between borders with a normal DCT.

So if you're comparing JPEG to HD Photo, of course HD Photo is going to look better, but at the cost of performance because of the additional "smoothing" operations.

Also, since LBT is based on DFT, they are both going to distort at approximately the same rate. This means that as you drop the file size the image quality for both HD Photo and JPEG should drop by the same amount.

HD Photo Versus JPEG 2000

JPEG 2000 is based of this new concept called discrete wavelet tranform (DWT).

Fourier Transform basically takes a function and tries to represent it with sines and cosines. The coefficients are the values that are stored, which makes the images smaller.

Wavelets on the other hand try to generalize what the Fourier Transform does by allowing basis functions other then sine and cosine. The basis functions must still average to zero, just like the sine and cosine functions.

Theoretically speaking, wavelets allow for better compactification versus fourier because you can choose the optimal basis function that allows you to represent the signal you're working with.

Here is an paper that shows that JPEG 2000 can be significantly better then HD Photo as you compress the images more. Remember, PSNR is logarithmic, therefore any small change in PSNR dramatically alters the quality of the image.

http://www.compression.ru/video/codec_comp...mparison_en.pdf

From what I have seen, Jpeg2000 sufffers even more from oversmoothign images even at higher quality than HDPhoto does.

ERvery Jpeg2000 image I have seen at any quality level smooths out details at a very agressive level.

HawkMan said,
From what I have seen, Jpeg2000 sufffers even more from oversmoothign images even at higher quality than HDPhoto does.

ERvery Jpeg2000 image I have seen at any quality level smooths out details at a very agressive level.

It's possible that you've seen a crummy implementation or just bad compression settings of JPEG 2000.

I haven't actually looked at any pictures or read anything about Microsoft's new format but I have some questions about your "test".

Quote - Emil Protalinski
I then saved the file again a step up in quality (without artifacts)

JPG: 75.6KB (Quality: 0/12)
JPG: 86.5KB (Quality: 0/12)
JPG: 209KB (Quality: 0/12)
JPG: 206KB (Quality: 0/12)
JPG: 252KB (Quality: 0/12)

Is this a typo or are you trying to make me believe the a jpeg saved at maximum compression doesn't have artifacts?

Quote - Emil Protalinski
A 1.85MB colour TIF aerial image of the world, at a resolution of 2048 x 1024

* JPG: 86.5KB (Quality: 0/12)
* WDP: 83.1KB (Quality: 0.5/1)

Quote - Emil Protalinski
A 1.85MB colour TIF aerial image of the world, at a resolution of 2048 x 1024

* JPG: 340KB at 75% (9/12)
* WDP: 335KB at 75% (0.75/1)
* JPG: 204KB at 50% (6/12)
* WDP: 121KB at 50% (0.5/1)

At 75%, there is no visible difference in quality but WDP is nowhere near half the size of JPG. At 50%, WDP is nearing half the size and a loss in quality can be perceived.

Here you have the same image with different filesize at WDP@0.5/1. How did that happen?

What about the difference in filesize on land_shallow_topo_2048.tif? Is there an explanation for that?

EDIT: Sorry, didn't see that that was the typo. But what about jpeg @ 0/12?

Much better Emil, this is a much better test using much higher quality photos (with much higher bit per pixel original images). This new format has lots of potential, especially considering its going to be free. It looks like you're saving something like 30-40% size at what you think is a not noticeable compression level. If this gets implemented into digital cameras thats a lot more photos per CF or SD card (with the potential to have those "more" photos being of a higher quality and more colourfull).

I'm not sure jpeg2000 is even relevant, regardless of its abilities, it's been out for years now and I have yet to see one image posted anywhere that uses that format... ok except for today when someone posted one in Emil's earlier thread....before that, never. It sounds like this is due to licensing costs, which will no doubt, never change so I'm not sure why it keeps getting brought up.

Thanx for the more indepth and useful comparison Emil.

Frazer

Justin- said,
You can make a TIFF of the actual two images side by side after they are compressed so we could see them. :)
QFT (Though I'd rather see him use PNG since that's also lossless ;))

One major problem with your tests is that you're basing them on your own perception. How about measuring the SNR between the original and compressed versions. That way you'll have a numerical comparison instead of saying "because I said so." I have no doubt that HD Photo is better, but you don't have any concrete numbers.

Another thing is that HD Photo's rival should really be JPEG2000, not JPEG. Comparing HD Photo to JPEG is like comparing VC1 or H.264 to MPEG1.

MS's format has to displace JPEG, not JPEG2000, so the comparisons do make sense. JPEG2000 certainly doesn't look like it's going anywhere. My browser can't even display it, and it's been out for years.

And how can we sure that this beta software is producing the right results.
Is this beta software selecting the right parameters when saving the image.
May be the default parameters are teaked for performance rather than quality or completely wrong who knows for sure.

The reason JPEG2000 has never gone anywhere is that so many different companies hold patents on the technology in JPEG2000 that if you wanted to use it in any commercial product you would have to give both kidneys in order to pay the licensing fees.

Conversely although Microsoft hold patents on the technology in HD Photo they have agreed to license it to anyone for free.

Mog

The_Decryptor said,
they don't like the OSS guys though, they can't use HD Photo, currently.

And Safari can display JPEG2000 ;)

Even if they could, would they? I think the answer is no, HDP in it's final form (because this is still a beta plug-in), could be the best thing ever and OSS guys still wouldn't like it anyways, it's an MS made format thus it's evil!.

The licensing could be totally open to them also and they still wouldn't like it, They'll insted try to make their own version just for the hell of it.

Fact is though, it's doing more or less what MS said it would, bigger HD photos are being shrunk to smaller sizes with the same or better quality of JPEG. I don't know about the rest of you, but If I can get it to look as good but shave off even 100kb, then i'll switch over to HDP. The size difference might not be exactly hafl, but 30%-40% adds up with every new picture you take and store on your memory card or PC.

GP007 said,
Even if they could, would they? I think the answer is no, HDP in it's final form (because this is still a beta plug-in), could be the best thing ever and OSS guys still wouldn't like it anyways, it's an MS made format thus it's evil!.

The licensing could be totally open to them also and they still wouldn't like it, They'll insted try to make their own version just for the hell of it.

Who are those OSS guys you're talking about, and who do you think you are to speak in their name?

ichi said,

Who are those OSS guys you're talking about, and who do you think you are to speak in their name?

And who are you? I like everyone else who posts am stating my opinion of the subject at hand.

GP007 said,

And who are you? I like everyone else who posts am stating my opinion of the subject at hand.

I'm and OSS user and developer, so maybe I should take it personally when you go talking BS about what those "OSS guys" would or wouldn't do.