data compression link collection

Data Compression

This is the catch all category for the library. Wherever possible, I try to categorize an entry with as refined a label as possible. But some web pages, papers, or books cross through many different categories, including lossy, lossless, image, audio, etc. Those things have to be categorized here. A good example would be the comp.compression newsgroup, which covers everything related to data compression.


A program by Andrew Frolov on a site dedicated to complete programs that are less than or equal to 256 bytes in length.


Posted in May 25th, 2004


This is an open-source package of DjVu programs and libraries, including encoders, viewers, browser plugins, and various utlities. The DjVu standard for document encoding was once an ATT research project, but now has been commercialized by LizardTech. This project is an attempt to popularize and evangelize the DjVu technology, with at least the benign awareness of LizardTech.

Release 3.5.13 shipped in April of 2004.


Posted in April 25th, 2004


DACT is a brute force compressor written by Roy Keene that tries out a whole library of compression routines on a given file, and then simply picks the best performer.

Release 0.8.36 was shipping in early March, 2004.


Posted in March 7th, 2004

ANSI-C Bit Manipulation Libraries

Michael Dipperstein has written a few compression programs, which naturally requires that you be able to read and write bits one at a time, and possibly in chunks of other sizes. He has packaged up this capability as a separate library, and makes it available to the world here.

* * * * *

Posted in February 14th, 2004


Compression is concerned with detecting patterns in order to reduce redundancy. Complearn is an attempt to take that pattern-recognition ability and use it for different ends.


Posted in January 18th, 2004

BitJazz Compression Software Links

Published in Links, Data Compression

The folks at BitJazz maintain a nice list of links to data compression directories. That makes this page sort of a meta-directory.

* * * * *

Posted in December 3rd, 2003

The Open Compression Toolkit for C++

The Open Compression Toolkit is a set of modular C++ classes and utilities for implementing and testing compression algorithms.

  • Simple interface and skeleton code for creating new compression algorithms.
  • Complete testing framework for validating and comparing new algorithms.
  • Support for algorithms that use external dictionaries/headers.
  • Utility classes and sample code for bitio, frequency counting, etc.


Posted in August 25th, 2003

Introduction to MMX Programming

An article that gets into the details you need to know about in order to use the MMX instruction set found on Intel processors. The author uses image processing as a demo app.

* * * *  

Posted in July 15th, 2003

Xceed Streaming Compression Library

The Xceed Streaming Compression Library is a high-performance “raw” compression library. It offers the ability to compress and decompress streaming data, buffers, strings or single files and supports multiple compressed data formats. Unlike the Xceed Zip Compression Library, this ultralight library doesn’t offer Zip file handling capabilities.

* * * * *

Posted in June 21st, 2003

Data Compression Researchers

Published in People, Data Compression

The page from the Google directory.


Posted in May 8th, 2003

Data Compression Bibliography

The University of Washington has a nice bibliography here, with pointers to books on Data Compression, VQ, Wavelets, and Information Theory.


Posted in May 4th, 2003

Source Coding Matlab Toolbox

John Kieffer from the University of Minnesota has posted a nice library of Matlab code to be used for data compression.


Posted in May 1st, 2003

EE 5585: Data Compression

A collection of class notes and links for a University of Minnesota course on Data Compression.


Posted in May 1st, 2003


LuraDocument represents an innovative compression software custom-tailored to the compression of scanned color documents composed of both images and text. The documents are partitioned into pure picture and pure text segments through the use of its state-of-the-art segmentation algorithm. Each of these segments is then compressed using an algorithm optimized for this type of data. As a result, scanned color documents can be stored at extremely high quality despite small file sizes. LuraDocument is capable of reaching compression rates of 1:150 to 1:500 while still preserving full readability of the text and excellent quality of the pictures. This makes LuraDocument far superior to even the wide spread PDF format.


Posted in April 8th, 2003

How File Compression Works

The How Stuff Works web site tackles the question of just how Data Compression works, pitched to the general audience. All in all not a bad stab at it, with links to may other How Stuff Works pages for the curious.


Posted in April 7th, 2003

File Compression: New Tool for Life Detection?

Researchers looking for signs of life have decided that complexity might be a good indicator. What better to measure complexity than gzip?


Posted in March 31st, 2003

Michael Walden’s Compression Pointers

A comprehensive set of compression pointers. Unfortunately, Michael is using some sort of software that makes bookmarking into his index impossible. So instead, you must link to the main page, shown here, and locate the links to “Data Compression”. Under that he has links to General Resources, Software, and Theory.


Posted in March 23rd, 2003

What’s New In Math - October, 2002 - Compression Codes and Technologies

The American Mathematical Society runs a monthly column called ‘What’s New In Math,” featuring topics of general interest to their readers. In October 2002, the topic was data compression, which led to a fairly lengthy column with many, many references. Good reading!


Posted in March 23rd, 2003

Archiving Encrypting Tiny File Splitter

Oleg Z. has created this program to to split, encrypt with your own unique key, compress and restore to original state large files of any type. The author says the program has an intuitive GUI, and works on most Win32 platforms.

Version shipped on March 6, 2003.


Posted in March 8th, 2003

Miliki Super Compressor Professional

This product is advertised as a revolutionary compression package from QuickCAT Technologies. It comes in two versions: a Pro package that compresses documents and images, and a Basic version that compresses only images.

* * * * *

Posted in February 28th, 2003

Data Compression - Systematisation

T. Strutz offers up an overview and taxonomy of Data Compression. If you’re looking for the big picture, this might be a good place to start. A PowerPoint presentation is available for download as well - might be just the thing if you only have an hour left to prepare that lecture!


Posted in February 18th, 2003

Data Compression Corpora

Published in Files, Data Compression

Jürgen Abel has a great Data Compression site, and keeps a set of pointers to standard sets of files used for compression. He recently added a reference to the Protein Corpus, a set of difficult to compress files that were first published at the 1999 Data Compression Conference.

* * * * *

Posted in February 17th, 2003


The XTREAM Data Management System (XMS) is a robust, scalable data compression system that offers increased compression ratios and/or reduced compression processing time for efficient data management.

Note: I have to confess that after looking at this web page, I have absolutely no idea what this product does, how one would use it, or what the potential market is.


Posted in January 19th, 2003

Data Compression Conferences

Jürgen Abel is now tracking conferences which will be covering data compression topics. This listing includes the conference, dates, web page, and submission deadline. You are encouraged to keep Jürgen updated with any new information.

* * * * *

Posted in January 19th, 2003

Hacking Data Compression

Andy McFadden was nice enough to post a series of a dozen lessons on Data Compression. It looks as though these were posts to the now-forgotten GEnie information service, once a pretender to the CompuServe popularity crown.


Posted in October 30th, 2002

High School Kids Win Prizes for Compression Algorithm

A couple of high school kids from Saratoga, CA, were regional winners in the Siemens Westinghouse Science and Technology competition.


Posted in October 12th, 2002

Multimedia Paper Summaries

A set of links to paper summaries, including a batch on Coding, Compression, and Media Formats.


Posted in September 29th, 2002

Free Specs from the ITU

According to Pete Fraser, you can download three free specifications from the ITU per year, after free registration. This is a great deal - these specs can run into the hundreds of dollars per copy, and if you aren’t near a university library, you may have no other way to get them.


Posted in September 15th, 2002

Data Compression Site

This site is in Russian, so I am not able to tell you much about it. It has links to some papers, and seems to have a link to PDF files that comprise much of a book on compression.

Reader Dan L. says The best site about comression!

* * * * *

Posted in September 15th, 2002

DjVu - Next generation image compression technology

DjVu is an image compression technique that is meant to be used on scanned documents. AT&T has created a browser plugin that supports DjVu files. AT&T claims that DjVu files are 5 to 8 times smaller than GIF or JPEG images of similar quality. Some public source code is provided here, but the exciting parts of DjVu are not available.

Update: I believe this project has morphed away from AT&T and into the Open Source World. The AT&T site has been gone since 6/2002, I hope that this represents its new incarnation. I’m also moving the project status from Commercial to Non-Commercial

* * * * *

Posted in September 3rd, 2002