Google Releases New Compression Algorithm


Recommended Posts

  Quote
The Zopfli Compression Algorithm is a new open sourced general purpose data compression library that got its name from a Swiss bread recipe. It is an implementation of the Deflate compression algorithm that creates a smaller output size compared to previous techniques. The smaller compressed size allows for better space utilization, faster data transmission, and lower web page load latencies. Furthermore, the smaller compressed size has additional benefits in mobile use, such as lower data transfer fees and reduced battery use. The higher data density is achieved by using more exhaustive compression techniques, which make the compression a lot slower, but do not affect the decompression speed. The exhaustive method is based on iterating entropy modeling and a shortest path search algorithm to find a low bit cost path through the graph of all possible deflate representations.

The output generated by Zopfli is typically 3?8% smaller (PDF warning) compared to zlib at maximum compression, and we believe that Zopfli represents the state of the art in Deflate-compatible compression. Zopfli is written in C for portability. It is a compression-only library; existing software can decompress the data. Zopfli is bit-stream compatible with compression used in gzip, Zip, PNG, HTTP requests, and others.

Due to the amount of CPU time required --- 2 to 3 orders of magnitude more than zlib at maximum quality ? Zopfli is best suited for applications where data is compressed once and sent over a network many times, for example, static content for the web. By open sourcing Zopfli, thus allowing webmasters to better optimize the size of frequently accessed static content, we hope to make the Internet a bit faster for all of us.

Source: http://googledevelopers.blogspot.com/2013/02/compress-data-more-densely-with-zopfli.html
  UXGaurav said:
Wonder how it does against LZMA and LZMA2 considering both reasonable speed and compression ratio.
This is about DEFLATE-compatible compressions, so LZMA/LZMA2 don't enter into the picture.

But yes, LZMA/LZMA2 traditionally has superior compression ratio. Too bad that still not used in HTTP transfer-encoding.

I read the PDF. It's decent, but has a very specific use case. That is: One-time compression and multiple decompressions for static data (they do acknowledge this). Compressing files using this algo is epic slow compared to gzip for the same file (talking 5 seconds for gzip compared to 7 minutes for the same file using Zopfli). It'll be very good for conserving bandwidth on, say, file distribution sites though where CPU time is inconsequential compared to bandwidth requirements.

This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • I'm old I guess, first thing I think of is just regular input/output.
    • Now, kids, Dan O'Dowd is what we call a professional hater.
    • Billionaire slams 'Tesla Cultists' for praising Robotaxi, says it's 5+ years behind Waymo by David Uzondu Image via Depositphotos.com The Tesla Robotaxi program has kicked off in Austin, Texas, and reactions are pouring in from all corners of the internet. A select group of investors and influencers have been invited to try the service, which operates within a limited area of South Austin for a price of $4.20. While the vehicles are operating without anyone in the driver's seat, the program has specific rules for this pilot phase, including a human "safety monitor" who rides along in the passenger seat just in case things go sideways. Of course, the launch did not go unnoticed by Tesla's most vocal and well-funded critic, Dan O'Dowd. O'Dowd is the billionaire founder of a group called The Dawn Project, which has dedicated itself to highlighting what it calls critical safety failures in Tesla's Full Self-Driving software. He refers to himself as an expert in creating "unhackable" software for military and aerospace clients, and ran for U.S. Senate back in 2022 on a single-issue platform: to "make computers safe for humanity" by banning Tesla's FSD. In 2023, He was banned from advertising on X after He made promoted posts that show Tesla FSD among other things, failing to stop at Stop signs. Last year, his group, The Dawn Project, paid for a Super Bowl ad, where a Tesla equipped with FSD did not act on a child-sized mannequin in the road. That commercial ends with a message, urging parents to "boycott Tesla to keep your kids safe." Today, O'Dowd took to X to slam the launch of the Robotaxi service, saying the "Tesla Cultists are celebrating victory" over a system he believes is years behind the competition (especially Waymo). He pointed out that with only fourteen cars operating for half the day, the system was already making significant errors, a rate he claims is consistent with community-tracked FSD data. The videos shared by the creators (Rob Maurer and Ed Niedermeyer), O'Dowd mentioned in his post, appear questionable, depending on your perspective. In Maurer's video, a trip that was otherwise smooth had a few unnerving seconds of the vehicle slightly swerving into the wrong lane, correcting itself, swerving again, correcting itself, and then finally settling. The other video from Ed Niedermeyer shows something entirely different. Niedermeyer captured a Tesla Robotaxi approaching an "extensive crime scene" with multiple police vehicles parked on the side of the road. On his personal Bluesky account (Ed stopped posting on X late last year, in protest of Musk), He claims the Tesla braked hard twice for no clear reason. In his commentary, Niedermeyer argued the car "shouldn't react to any of these police vehicles," and that it was concerning how it reacted to some but not others, before stopping in the "middle of the road instead of defaulting to a minimal risk condition."
    • Arch is now also using Wayland as the default session for Plasma 6.4, with X11 session becoming optional (so upgrading to Plasma 6.4 on X11 Arch might need manual intervention). It's been well over a decade in making, but I guess the time for Wayland to be the default is finally upon us.
  • Recent Achievements

    • Week One Done
      ravenmanNE earned a badge
      Week One Done
    • Conversation Starter
      Brett76 earned a badge
      Conversation Starter
    • One Month Later
      Miguel Batista earned a badge
      One Month Later
    • Dedicated
      moojay67 earned a badge
      Dedicated
    • One Month Later
      Jim Dugan earned a badge
      One Month Later
  • Popular Contributors

    1. 1
      +primortal
      663
    2. 2
      Michael Scrip
      229
    3. 3
      ATLien_0
      219
    4. 4
      Steven P.
      148
    5. 5
      Xenon
      141
  • Tell a friend

    Love Neowin? Tell a friend!