All about flooble | fun stuff | Get a free chatterbox | Free JavaScript | Avatars    
perplexus dot info

Home > Algorithms
Shrink needed (Posted on 2006-10-18) Difficulty: 2 of 5
A file compressor is great for shrinking stored files, but it depresses me whenever I see a file grow instead of shrink. So what I am looking for is a file compression algorithm that never inflates any files, although it is allowed that some files (not all of course!) have the same length after "compression". Ideally it should work on files of all sizes, but I would be satisfied with a compressor that operates only on files larger than 1MB. Can you provide such an algorithm? No programming knowledge is required for this problem.

  Submitted by JLo    
Rating: 3.6667 (6 votes)
Solution: (Hide)
Zipper for ALL files
There is no zipper that compresses ALL files without enlarging some. First observe that the zipped version of two different files must not be identical, otherwise the unzipper would not know into which of the two original files to unzip. Now consider the very small files first and assume we have a zipper that does not enlarge any files. We will see that this zipper will not truly compress any file:
- A 0-byte file cannot become smaller, therefore when zipping it you end up with the same 0-byte file.
- Assume you could truly compress a 1-byte file, then it would become a 0-byte file. But the 0-byte file is already the zipped version of the 0-byte file. Therefore, if our zipper should not enlarge any files, it can only make 1-byte files out of 1-byte files. In other words, our zipper would merely "permutate" the set of all 1-byte files, e.g. by flipping all bits or something similar.
- Same consideration for 2-byte files: All 0- or 1-byte files have already been proven to be zipped versions of some other files, therefore zipping 2-byte files results in 2-byte files and our zipper merely permutates the set of 2-byte files.
- The above logic holds for 3-byte files, 4-byte files, 5-byte- files,... etc. just as well and we conclude by induction that our zipper maps n-byte files to n-byte files, i.e. no single file is compressed!

Zipper for large enough files
When we only require that our zipper works for sufficiently large files, a simple algorithm exists. A "zipping" algorithm that operates on all but the 0-byte file works as follows:
- If the file contains n bytes, all of which are zero, then we zip the file to a (n-1)-byte file with all bytes being zero.
- If the file contains non-zero bytes, we leave the file unchanged. As an alternative (if you don't like the fact that the zipper does not really do anything), you write down all bits of the file, interpret it as a binary number and add 1 to it to obtain the bit sequence of the "zipped" file. Except of course if all bits are equal to 1, in which case your "zipped" file would be 00...001. This algorithm permutates all n-byte files having non-zero bytes.
Obviously this zipper does not enlarge any file. It also compresses some files, even an infinite number of them (the ones with zero-bytes only). And no two files get zipped into the same file, which makes it possible to provide an unzipper. The unzipper would simply add a zero-byte to every zero-byte-only-file and leave all other files unchanged (Or, for the alternative, subtract the number 1 in the bit sequence, with the exception that the file with bits 00..001 gets "unzipped" to 11..111.) The "empty" file with length 0 would also be considered to have zero-bytes only (or can you find any 1-bytes in it?) and would be unzipped to a 1-byte file containing a zero byte. Not sure though if the zipper would be very useful for practical applications...

Tricky ways out
A lot of great post were made regarding impossibility proofs and possible solutions. There are indeed some tricky ways to write a zipper that compresses ALL files, some of which were mentioned, that I'd like to discuss briefly. There are probably other tricky solutions.

What if our zipper simply calls any commercial zipper and if the file truly compresses, fine, otherwise we rather leave the file unchanged. Wouldn't that work?
That would be a way out if our unzipping algorithm could somehow determine whether the compressed file was really compressed or if it was simply left unchanged. That would actually require one more bit of information to store somewhere... One could encode that extra bit into the name or extension of the file, e.g. by adding the extension ".ZIPPED" if the file was "really" compressed. The main problem with this approach is, that real world operating systems have file length limits, because what happens if our original file has already maximum length? (If the operating system does not have this restriction, there is an even better tricky way out, see below) Another option is, that the zipper gives a message to the user a la "not really zipped" if the file could not be shrinked, and the user just needs to remember which files were "really" compressed. I.e. we store our "compressed"-bit inside the user, which is a bit of cheating. Similar tricks involve storing that information in the zipping executable itself or in some other system file.

If our operating system allows for arbitrarily long file names and if we allow our zipper to rename the original file, we can even compress all files to length zero! It goes as follows:
- With some suitable algorithm, encode the whole file content into a huge string consisting only of the letters a-z. The only requirement is that the algorithm can be reversed; this is easy to do.
- With some suitable algorithm, encode the whole file name (including extension) into a huge string consisting only of the letters a-z. Again the algorithm must be reversable.
Generate a zero-length file. For its file name, concatenate the encoded file content with the character "0" and the encoded file name. The "0" is to tell the unzipper which part of the file name contains content and which part the name. By reversing the encoding steps for content and name, our unzipper can restore the original file from the name of the compressed file.

Comments: ( You must be logged in to post comments.)
  Subject Author Date
Provably impossibleJort Bloem2006-10-23 22:24:16
Some Thoughtsre(3): No impossibility -- doubt itFederico Kereki2006-10-23 08:17:13
Some Thoughtsre(3): Any compressor will do -- doubt itFederico Kereki2006-10-23 08:15:47
loopholesTristan2006-10-22 17:22:30
re(2): No impossibility -- doubt itvswitchs2006-10-22 11:53:14
re(2): Any compressor will do -- doubt itArt M2006-10-22 03:53:01
SolutionA tricky way outFederico Kereki2006-10-21 14:34:11
Some Thoughtsre: Any compressor will do -- doubt itFederico Kereki2006-10-21 07:27:27
Any compressor will doArt M2006-10-21 03:00:50
Some Thoughtsre: No impossibility -- doubt itFederico Kereki2006-10-20 22:23:09
Thought on text compressionbrianjn2006-10-20 21:32:33
SolutionNo impossibilityvswitchs2006-10-20 16:50:57
Invent an impossibility?brianjn2006-10-20 11:30:45
SolutionNo zipper existsOld Original Oskar!2006-10-20 09:41:12
Themesbrianjn2006-10-20 09:31:24
SolutionImpossibility #3e.g.2006-10-20 08:23:45
Reviewbrianjn2006-10-20 04:17:38
Some Thoughtsre: IdeasFederico Kereki2006-10-19 14:03:29
re(2): IdeasJer2006-10-19 11:26:18
re: IdeasLarry2006-10-18 20:45:07
SolutionImpossibility #2e.g.2006-10-18 13:29:35
IdeasJer2006-10-18 12:04:55
Some Thoughtsre: Impossibility?Federico Kereki2006-10-18 10:11:31
re: Impossibility?brianjn2006-10-18 09:41:19
Some ThoughtsLightly Squeeze Mebrianjn2006-10-18 09:30:57
SolutionImpossibility?e.g.2006-10-18 09:03:16
Please log in:
Login:
Password:
Remember me:
Sign up! | Forgot password


Search:
Search body:
Forums (0)
Newest Problems
Random Problem
FAQ | About This Site
Site Statistics
New Comments (14)
Unsolved Problems
Top Rated Problems
This month's top
Most Commented On

Chatterbox:
Copyright © 2002 - 2024 by Animus Pactum Consulting. All rights reserved. Privacy Information