This is why I love my blog – smart people taking time out of their day to debunk my crazy ideas! One important thing to note, and I should have clarified this in my post, is that by no means was this intended as a *compression* algorithm- this is an *encoding* algorithm.

The metaphor I should have used is more like this. Think about encoding a secret message in a book by using the first letter of each paragraph. To encode the message “This idea is crazy”, I would need to find a page in a book where the first paragraph starts with “T”, the second starts with “H”, and so on…

Now, if I want to send this message to someone, I just need to tell them where in the book to start. (i.e. War and Peace, page 217, paragraph 3). While the full book of ‘War and Peace’ is much, much larger than my message, I can send my message just by giving these simple instructions.

In my mind, where this idea breaks down is this question:

What is the mathmatical probability that for longer messages you could find a book that has the exact letters laid out in sequential paragraphs?

For an arbitrarily small input (a single letter, for example) the probability is 1. For an arbitrarily large input (20 billion letters), the probability is very close to 0. The real question is, does that probability drop fast enough that encoding anything useful is not practical?

My guess? If you actually did the math, the probability of doing anything useful would likely be very, very small. Small enough to make this whole debate moot.

Your legal argument also probably holds water with most courts – which is why this becomes more of a hypothetical theory rather than something any sane person would set out to do.

And if you would, you would find out that, on average, you’d have download 3 MB in tweets to uniquely represent a 3MB mp3 file.

Secondly, you’d have to search an infinite time to find a tweet sequence representing your 3MB file.

If you want to look into it beyond this comment, I suggest you lookup the “Pigeon Hole Principle”, as well as find any nutcase compression algorithm based on finding any piece of data’s numerical representation in the processing of the digits of pi (which is mathematically proven to contain every number provided you expand its digits long enough). You run into the exact same issue you are going to run into.

Simply put:

You can’t use X bits to represent more than at most 2^X different cases.

Let’s set X to 8 for a moment. If I have a compression algorithm that can take *any* data that is 9 bits long and reduce it to 8 bits, then we can clearly say this compression algorithm is as impossible as 2+2 being equal to 3. After all, the compressed data has 2^8 = 256 different ‘pigeon holes’, but you’ve got 2^9 = 512 different data files. You can’t fit 2 pigeons in one hole, or your compression scheme would be arbitrary and a compressed file can be decompressed into 2 different files, which would make it useless.

The key, then, is to prove that a compression scheme cannot compress random data just as well as non-random data, for if you can prove that, you know the pigeon hole principle will apply and you haven’t got a compression scheme at all, just one to reshuffle bits. And, usually, this mapping function actually serves to make the ‘compressed’ data *BIGGER* than the original, as most of these mapping schemes, be it twitter or the digits of pi, contain redundancy.

Something like the scheme you started this post out with can be proven to spectacularly fail on random data. It specifically exploits repetition. Data with no repetition at all should not compress (in fact, to accomodate the table, it would become LARGER. This is a key factor in avoiding the pigeon hole principle – this HAS to be true for any compression algorithm, just like 2 + 2 has to be 4).

In regards to the RIAA trick: That won’t fly either, and there are far easier ways to accomplish this.

Example scheme: Take a movie. XOR it with the compressed complete repository of debian linux. Without the debian linux kernel this data is utter random jibberjabber – in fact, you can take the same seemingly random stream of data and, provided you XOR it with the right stream of other data, get anything you want with that length. And yet, with one particular secondary input source (debian linux repo, which is free and open source), it ‘magically’ turns into some copyrighted movie.

It’s been tried. It’s sufficiently nuanced that a judge will either not understand it and go back to the simple lemma that by downloading this thing you can trivially get the copyrighted movie out of it, and even if a judge does understand the math involved, he’ll see it as a violation of the spirit of the law, which in cases like this is more than enough.

Hence, we arrive back at where I started: This entire post was *COMPLETELY CRAZY*.