History of Compression
In 1952, David Huffman invented the Huffman coding to compress text. He based his analysis on the redundancy of bytes.
The more redundant bytes the better the compression.
He did an excellent job. Thanks to him we have WinZip, WinRar, Power Archieve, 7-Zip and alot of compression tools.
we use them daily.
Are the compression Algorithms mature enough?
When we try to compress exectuables and binaries that are naturally doesn't contain so much redundant bytes, we didn't get so much out of it.
We Suffer. We suffer when we want to send a rar achieve of 10 MB the email keep bouncing back. We suffer when we want to download large files such as music, movies, maps, ISOs. We suffer when we want to open a webpage filled with flash, images, rich texts or videos. We suffer when we want to want to review an online album.
Why only those with 4Mb T connections can enjoy the fast speed just because files are getting bigger and bigger and WinZip and WinRar failed to make them smaller?
I was working on this for the last 4 years.
I failed 3 times, I realize I am in a dip and that at the other side of the dip there is a promising IT world .
This is what I want to do:
I want to extract a 20 KB file from a 1GB file, that file is the essence and should be used to "regenerate" the 1GB file.There must be a new approach very different then the classical one, this will involve new technologies and science such as Quantum Physics .
I have setup some approaches that will make some of you laugh. I'll continue to add more approaches from you guys
The idea is to extract a unique descriptive identifier of the subject file. (Think of this as taking a seed from a given tree) The (seed) is a very small file that can regenerate the original file (tree).
Technically, the seed will be a mathematical formula. By using complex numerical analysis I want to get a unique formula for each file, I started to read some numerical analysis books. Few people told me that its not possible because NA gives you an approximation of the formula. But I want to see for myself.
We know that, 1 Byte = 255 decimal. A typical file looks something like this.
offset 0 -> 125
offset 1 -> 56
offset 2 -> 12
offset 3 -> 55
Until the end of the file. This will generate a graph, we only need to get the formula.
Second approachThis is the backup plan. I need to study some biology first to achieve that. Please, open the window now, and look, really look outside.
What can you see? I bet you can see a lot, you can see trees, houses, there is a girl playing with her doll, you can see the colors, you can see the movements, there are birds flying, you can see the color of the birds how fast they are moving you can measure the distance you can see which object is near, which one is far.
You can see a lot amount of details. All these data, are handled by a very tiny part of our body. The eye. If your eye can capture and shrink this amount of details into a place smaller then an egg.
Can't we make a 1 GB file into a 20 KB ?
Third approachDigital Seeds with Fourier Series, I wrote about it here.
My readers suggestions
In this section I will list the approaches suggested by my lovely reader.
This guy is really interested in Ceed idea, in fact he wrote a lot of suggestions more than I've even did, you can read his comments. I love this guy.
If you can't find such a formula, then maybe you can divide the file in two arbitrary parts and then find the formula for each. That way, you're increasing the probability of a hit. You can shift this divider both ways for a more greater possibility of determining a formula.
Dr. Christian Lapp
What about considering every byte as one dimension in a multi(n-)dimensional space. so the entire file would be the surface of a n-dimensional figure, itself having the dimension n-1.
1 byte --> a point (0D)
2 bytes -> a line (1D)
3 bytes -> a plane (2D)
etc. Check out the idea of parallel coordinates
Now that I think of it, wasn't there another compression method, called Rainbow Paper? I can't find it anywhere, but I do recall that it should be able to print a 1GB file on a paper, using a regular inkjet-printer and read it back in using a regular flatbed-scanner.
Why Ceed?Imagine the life with Ceed.
You will download the Ceed version of the video to your machine very fast (20 kb) then regenerate it in your machine very fast and play it as if it was locally there. No more stopping the video to let youtube download it full.
Google earth will send Ceeds Satalite images quickly to your computer then regereate the original files in your machine. Your internet speed doesn't matter so much. Because The work will be done on the machine as if you are browsing your own computer data.
I don't need to wait for days to download a file.
1 GB game can be shrinked into its orginial ceed 20 KB and downloaded easily, or sent via MSN messenger.
No more .html, or .asp or .php
its .ceed, www.yahoo.com/index.ceed. This will contain the whole page with the images and the text and videos and flash into about 50 kb size ceed file. Your ceed browser will download the ceed to your machine and the browser regenerates the original page easily, as if you are opening a page on your machine. No flickering, no loading. Nice.
You may add to the list.
Why I'm sharing thisI don't want the idea to die with me by being selfish, I have to let people read it and understand and share it and add to it.
That's why I am sharing it today.
Your comments are most welcomed.