Brain & Information

This is about the brain now not the human.

A brain can be conceived as a compression tool. It takes vast amounts of data and compresses this into discrete entities.

Compression and Information theory have always been of interest to me. I completed a model on Friday which illustrated to me how it works. The basics are on Wikiedia.

Information at its most basic representation comes in "on" and "off". Anything less than this i.e. "on" and "on" doesn't tell us anything. IF information is about diferences all differences can be broken down into sequences of "on" and "off". That is the definition of how much information is in something - the smallest number of "on" "off" required to code for it.

Brains are not exactly "on","off" but Daniel Dennett uses a Von Neuman argument to show that they could be modelled on a computer so I'll take that unquestioned for now.

Now real world data comes in layers (I suspect like fractals). In this text on this page there are thewords, the letters and these are coded in ASC format to use 8 bits each. A compression algorithm could compress any layer.

There is a nice proof that there is no optimum compession algorithm because if such a thing existed how would be compress the algorithm itself when it was coded to compress itself? Classic loop disproof like so many (Godel etc) and I suspect the ultimate undoing of brain research one day.

One reason why there is no optimum compression came to me more clearly doing my own model. And this explains something else that has puzzled me for years about the seeming essential distinction between data and code.

When we compress a data we need to know what we are looking for in advance. Even a tool that built itself around the data has got to start somewhere. There is information already present in the coding which enables the compresson to occur because that information does not need to enter the compressed file... its standard data in the program file.

Here is another way to look at this. One way of compression is to extract commonly occuring patterns and store them in an index. The index location then becomes the data in the main file. Of course this only works if the index location in bits is smaller than the data being stored. There are other complications also. I found that this does not work very well when sed at the bit level ad the model was based on random data also (which has the highest entropy and highest eneregy). Real world data has much lower information and entropy. For example this page uses only 27 or so characters (max 5 bits each). Examine it at the bit level and it is a mess.

Knowing to group bits in bytes and that there are only 27 characters would enable a very good compression algorithm but only because this information is coded into the algorithm.

All this is leading to the obvious conclusion that the brain has evolved to process certain types of environments and search and encode certain things. We know for sure that congenital parts of the visual cortex identify edges and lines at different orientations. Can't that be expanded to other skills like language... after Noam Chompsky.

This answers a long held question of mine: is the world simply a product of culture or are their built to see it a certain way. And does the world make our brain a certain way: or does our brain makes the world a certain way. For example we see things in space. Thus we visualise the brain in space. So is the brain actually in real space and that spacial thing gives us the power to see (as Dennett brain thinks) or does the mind come first and its properties determine how we see the brain. Personally I've never been able to escape the latter let alone believe the former, and world thought with the minority exception of Western science and Anglo-American philosophy sees the latter also.

However in this blog entry the suggestion that the brain must have a certain form in order for it to process and compress data into discrete units leans toward the brain view. Of course that is assuming that the original arguments were not constructed within the Mind!