Data-Compression.org

data compression link collection

Lossless Compression

This category covers lossless compression methods that can’t be pinned down to one of the more refined topics, such as LZW or Zip. In some cases, the items here are esoteric algorithms that don’t’ merit their own topics. In other cases, they span one or more existing topics. The unifying theme is of course that they are lossless. This means that after compressing and then decompressing, the data set in quesition will be bit-for-bit identical.


TTA Lossless Audio Compressor

A lossless codec developed in Russia originally for radio telescope data. Apparently that specialized codec turned out to be good on audio data as well. Distributed under a free license.

http://tta.iszf.irk.ru/index.php?module=pagemaster&PAGE_user_op=view_page&PAGE_id=6&MMN_po
sition=21:21

* * * * *

Posted in September 12th, 2004

SynCE - Dynamite

Dynamite is a tool and library for decompressing data compressed with PKWARE Data Compression Library and it was created from the specification provided by a post in the comp.compression newsgroup.

http://synce.sourceforge.net/synce/dynamite.php

         

Posted in July 29th, 2004

Stuns

This program tries to unpack the given file by application of several algorithms byte-by-byte. Result of work of the program is the set of files with the unpacked data. Many of the produced files are not correct. However, among them there can be correctly unpacked data. Correctly unpacked files have mainly significant sizes that distinguishes them from dust.

http://falinc.narod.ru/Stuns.zip

         

Posted in July 28th, 2004

axPAQ

A compressor built with the world-beating PAQAR 3.0 compressor. axPAQ wraps a GUI around the engine, and includes complete source.

http://www.alphaxsoftware.com/downloads/

         

Posted in July 16th, 2004

Estimating entropy rates with Bayesian confidence intervals

I’m pleased to announce with my co-authors availability of a preprint on our new algorithm to estimate the Shannon entropy rate (bits/symbol) or (bits/sec) of an observed sequence of low-alphabet symbols. It uses the Context-Tree-Weighting universal compression method, but doesnot use the compression ratio directly as an entropy estimator but as a scaffold for a Bayesian estimate. The result is significantly lower bias.

ftp://lyapunov.ucsd.edu/pub/nonlinear/bayesian_information/Kennel_Shlens_et_al_2004.pdf

         

Posted in July 10th, 2004

Java FLAC Codec

FLAC is a port of the Free Lossless Audio Codec (FLAC) library to Java. This library allows java developers to experiment and write programs that use the FLAC algorithms.

Version 0.5 is shipping as of June, 2004.

http://jflac.sourceforge.net/

         

Posted in June 20th, 2004

PJL Compressing Filter

A J2EE servlet filter which compresses data written to the response. It supports several algorithms (gzip, deflate, etc.) and emphasizes minimal memory usage and high throughput. Also provides detailed performance stats.

http://sourceforge.net/projects/pjl-comp-filter/

         

Posted in June 20th, 2004

zisofs-tools

The zisofs filesystem is an extension to the ISO9660 filesystem that allows files, on a file-by-file basis, to be stored compressed and decompressed in real time. The zisofs filesystem is supported by recent versions of Linux (2.4.14 or later). Legacy systems can still read uncompressed files. zisofs-tools contains the tools necessary to create such a compressed ISO9660 filesystem and to read compressed files on a legacy system.

http://freshmeat.net/projects/zisofs-tools/

         

Posted in June 20th, 2004

MoleBox

Pack all your files into a single executable with MoleBox or MoleBox Pro.

http://www.molebox.com

         

Posted in May 24th, 2004

Hybrid Lossless Audio Compression

WavPack allows you to losslessly compress (and restore) both 16 and 24-bit audio files in the .WAV format. Unlike “lossy” compression schemes (like MP3) that discard information, WavPack converts the audio data into a more compact form so that the restored files are digitally identical to the original source. It’s somewhat like the file compression portion of WinZIP except that it’s optimized for audio data. Like other lossless compression schemes the data reduction varies with the source, but it is generally between 25% and 50% for typical popular music and somewhat better than that for classical music and other sources with greater dynamic range.

http://www.wavpack.com/

* * * * ½

Posted in May 15th, 2004

Parallel Implementation of Data Compression Technologies for Multi-Gbit/s Networks

This group at Loughborough University in the UK would like to use sophisticated compression techniques in high speed networks. To make it all happen, they need to do it in hardware, and do it in parallel. This page has information about their efforts, along with links to papers and other information.

hhttp://www.lboro.ac.uk/departments/el/research/esd/projects/parallel_comp.htm

         

Posted in May 15th, 2004

X-Match Pro

A fast ASIC core designed for lossless compression.

http://www.lboro.ac.uk/departments/el/research/esd/projects/web_xmatch/XMatchPRO_home.htm

         

Posted in May 15th, 2004

Monkey’s Audio

This is a lossless audio compression format that has support for WinAmp and Windows Media Player. Retain perfect fidelity for your music recordings, at the cost of additional disk space.

Version 3.99 is shipping as of May, 2004.

http://www.monkeysaudio.com

* * * * *

Posted in May 2nd, 2004

Compression and Encryption Sources

Links to a variety of lossless coders, includes source for Huffman, arithmetic, LZSS, and other compressors.

http://www.larkin.lanit.ru/compress/compr.htm

* * * * *

Posted in May 1st, 2004

Linux File System Compression

This open source package, e2compr, provides transparent compression and decompression of Linux ext2 file systems.

Version 0.4.44 is shipping as of April, 2004.

http://sourceforge.net/projects/e2compr/

         

Posted in April 25th, 2004

Compression Algorithms

Arkadi Kagan has created a C++ project that implements a batch of our favorite lossless algorithms, including LZ77, LZ78, LZW, RLE, along with arithmetic and Huffman coding.

Version 1.1 shipped in April, 2004.

http://sourceforge.net/projects/compressions/

         

Posted in April 19th, 2004

A Simple Yet Highly Efficient Compressing Technique

A nice CodeGuru article describing compression of a dictionary.

http://codeguru.com/Cpp/Cpp/algorithms/compression/article.php/c6403/

         

Posted in March 27th, 2004

The Benefits and Drawbacks of HTTP Compression

A paper by Timothy McLaughlin that gives an overview of HTTP Compression and tries to decide whether it’s a good thing for not.

http://www3.lehigh.edu/images/userImages/cdh3/Page_3456/LU-CSE-02-002.pdf

* * * * *

Posted in March 7th, 2004

OptimFROG SDK

Florin Ghido has packaged his lossless compressor into a nice SDK - take advantage of his great audio compression in products of your own.

http://losslessaudiocompression.com/SDK.php

         

Posted in March 7th, 2004

Maximum Compression

Werner Bergmans has created a new benchmark site that aims to show the best compression ratios possible for multiple file types, including English text, executables, graphics, and so on. Werner says he is running these tests with 80-100 programs for each file type!

Reader Werner B. says Useful site to compare results of different compression programs. Regularly updated.

http://www.maximumcompression.com/

* * * * *

Posted in February 28th, 2004

waRmZip

This is a Win32 script that can be used to compress and/or delete files after a certain amount of time. Nice and simple.

Version 1.2 shipped in January, 2004.

http://winadmin.sourceforge.net/warmzip.html

         

Posted in February 1st, 2004

(X)MedCon - Medical Image Conversion Utility

A free program to deal with image formats in popular medical formats, including DICOM, SPM, PNG, and GIF.

Version 0.9.1 of (X)MedCon shipped in January, 2004.

http://xmedcon.sourceforge.net/

* * * * *

Posted in January 11th, 2004

The MLP Lossless Compression System

A paper that details the lossless compression system used in DVD-Audio.

http://www.meridian-audio.com/w_paper/2_1.PDF

         

Posted in January 11th, 2004

FlacPack - Free lossless instrument file compression

This project aims to define a standard for the lossless compression of MIDI patch files. From the title, you might guess that the lossless audio compression is performed using FLAC - other binary data is compressed w/zlib.

http://swami.sourceforge.net/flacpak.php

         

Posted in December 14th, 2003

MG4J: Managing Gigabytes for Java™

A Java implementation of the inverted-index compression systems described in the book Managing Gigaybtes. This GPLed effort doesn’t appear to have any connection with Witten, Moffat, or Bell.

Version 0.8.2 is shipping in October, 2003.

http://mg4j.dsi.unimi.it/

         

Posted in October 29th, 2003

DualStream - Near Lossless Audio Compression

DualStream is aimed at filling the big gap between perceptual coding and lossless coding. The goal is to offer real transparent audio coding (not only perceptually transparent) at half or less the bitrate generally used by lossless coding, and also to permit progressive consistent increase of the quality level, until lossless coding is reached. To eliminate the problem of choosing between lossless and near lossless, DualStream has an option to create a correction file, which may be eventually stored separately and used at a later time to restore the original. The advantage is that the two files (near lossless and correction) have together approximately the size of the lossless coded original file.

http://losslessaudiocompression.com/DualStream.php

* * * * *

Posted in October 12th, 2003

OptimFROG - IEEE Float

A lossless compressor for audio data stored as IEEE floating point data.

http://losslessaudiocompression.com/IEEE_Float.php

* * * * *

Posted in October 12th, 2003

OptimFROG - squeezing the bits out of audio

OptimFROG is a lossless audio compressor. Lots of claims for great performance!

OptimFROG 4.507 is shipping as of October, 2003. Features added since my last report include numerous bug fixes and performance tweaks. The author claims to have superior lossless compression ratios, includes numerous player plugins, and is free for non-commercial use.

Version 4.507 is shipping as of October, 2003.

http://ghido.shelter.ro/

* * * * *

Posted in October 9th, 2003

Algorithms for Triangulated Terrains

by Marc van Kreveld. This paper looks at a method for compressing geographical elevation data.

http://archive.cs.uu.nl/pub/RUU/CS/techreps/CS-1997/1997-32.pdf

         

Posted in October 7th, 2003

PDF Compress

Free PDF Compressor that removes duplicate PDF objects, optionally takes advantage of new compression features of latest PDF specification (1.5), and optionally takes advantage of a new proposed format called “Compact PDF” that for many classes of documents compresses 30 - 60% better than what is possible in PDF 1.5.

Note: Navigate up two levels to get to Tom’s download page.

Version 2.2 is shipping as of February, 2004.

http://http.cs.berkeley.edu/~phelps/Multivalent/Tools/pdf/Compress.html

         

Posted in September 12th, 2003