Stock Gradient Stochastic

Sure this is covered by existing technicals but anyway a spin off of trying to characterise the nature of stock charts was looking at the dy/dx or speed of a chart.

Take a 20 day moving average (in blue) of the log daily closing stock price (in heavy black - in reverse with most recent to the left) then calculate the difference between successive days (magenta) - this is an approximation of the instantaneous gradient of the M.A. (with x=1 day). Take a 10 day moving average of the gradient for clarity (green).

Where the green passes 0 the gradient is at a turning point (from simple calculus). If the gradient is heading up then the curve is moving away from a low and if it is heading down through 0 then it is moving away from a high. Thus we buy when the green line passes up through 0 and sell when it passes down through 0. The best signal seems to be to buy as the green line passes down through 0 and sell as the gradient approaches its lowest level.

A simple enough signal which so far seems to work. Only tested on two graphs (this is the second: a 1 year passage of the FTSE last year). Will test this fully in the completed genetic algorithm once a model of stock behaviour has been isolated.

20 days was used simply because it is commonly used in technicals and it provides a suitable smoothing at the scale of a year. No smoothing and the speed chart just scratches up and down pointlessly. 10 days again was arbitrary, just enough to smooth the speed. This means however that the green line is some 30days behind the black stock price line - and maybe it is just luck that this syncs with the chart. Really the green line ought to b

e displaced to the left 30days - to be done...

Another approach might be to get the gradient by regression. Here the magenta line is a regression of the 20 days previous (speed) and the blue line is a 10 day regression of that line (acceleration). It is much smoother so gives more accuracy. Green lines from left to right are buy signals (when blue line peaks above magenta); red lines from left to right are sell signals (when magenta passes below blue).

Genes and SRH

Occurred to me a few weeks ago that maybe genes contradict the SRH. The point about genes, and the reason for Dawkin's selfish label, is that one can be certain that a gene will be affected by its own phenotypic expression. This is the only certainty. A gene that helps an organism find a mate, for example, does benefit all the genes in that organism, but with random mixing of genes during sex there is no assurance that any of these genes will necessarily benefit from this gene in the next generation except of course that gene itself. It is the only gene that generation after generation benefits from its contribution to the organisms shape and behaviour. Thus the selection pressure on itself over many generations gradually comes to represents the actual fitness of the gene and gradually successful genes will be whittled from the unsuccessful genes. This random mixing during Prophase I is essential to ensure that no cheat genes take a free ride on the back of good genes - the pack being shuffled every deal so to speak. Good genes get freed from parasitic genes and optimised by this fundamental tweak of the cell division process. If ever randomness needed to be illustrated in cell division this is a place where by definition it is essential.

Any gene that could hijack the Prophase I process and ensure its transmission to the next generation would have a very high fitness, and suddenly things aren't so neat. If genes and evolution really do control "everything" in the cell then genes would always evolve to hijack the previous structural level. These genes, like well adapted parasites, would necessarily have no noticeable phenotype so as to not upset the organisms fitness, they would simply pass from generation to generation inside the very machinery of cell division. And any other genes that evolved, that could weed them out, would simply become prey to their own hijackers. The battle for power would reign beneath the scenes of the phenotype with hijacking genes try to avoid being isolated by Prophase I or similar innovations. I believe this messy political world of cell division and cell instruction is actually what researchers are finding. Last time I did any of this was at college in 1992 so I'm completely out of date; should read up some time.

I digress though from my point. It was to say that Dawkins is focusing on selection from the genes point of view. Genes constitute the environment also. In a karmic way a gene that succeeded by being selfish, would run into problems when it met itself. Such genes that "cheat" (i.e. seek to maximise their outcomes at the cost of others) spread fast through populations but come to a sticking point when they start to encounter their progeny and end up fighting endlessly in lose-lose contests. Axelrod's tournaments revealed that Tit-For-Tat is a much better strategy where cheats are punished but friends are trusted (sound familiar ;-). So for optimality a gene's fitness is not governed only by its contribution to the organism that it is in, but also by its frequency in the organisms in the rest of the "local population". A high degree of mixing will favour the cheat (the opposite of Prophase I), but organisms that keep to a limited "friend group" cannot afford to cheat. This is without doubt the cause of modern crime rates and what older people call social disintegration; the fact that today's level of social mixing and the vast market place means that cheats are only encountered once and so are harder to track. Under these conditions genes are selfish. But where mixing is less, where mobility is less, genes are made accountable for what they do more and small populations of cheats will fail while small populations of friends will prosper. Family units make an idea group not just because of selfish genetic relatedness (as Dawkins would argue) but because cheating is harder to sustain in small groups. The suspicion of strangers and the clique instinct in humans are all strategies to limit mixing and show up cheats. This would be another reason for the hierarchical nature of human society previous un-analysed! A company will promote on the basis of loyalty more than ability to provide a profit because capital interests are more concerned with not losing through being cheated than with making more money!

A lot of musing here and not entirely thought through so need to check.

On the SRH: a gene thus has a predicate (fitness) which is determined by its impact on the set of genes in the organism. So here there is direct feedback on the gene. However most importantly the expression of the gene is not affected by the fitness! That is to say that the gene creates an expression independent of its fitness, and its fitness is a clear result of that expression. Therefore the self is not actually referring to itself. It is rather the situation of the self throwing a rock in the air and then after the rock has been thrown the self then being hit by the rock - the thrower and the victim being essentially separate entities. To illustrate suppose a gene created a phenotype which had a very high fitness by being able to digest fish. It might lead to selection of other genes to make the organism aquatic. Then were that same gene to mutate again to enable digestion of mountain growing lichen sadly it will be housed in an organism that was aquatic and ate fish! Hardly self-reference!

On Women

An important idea in this blog is the idea that two people working together are better than 1. However on second look how is it that a man and a women create more work together than each by themselves? Most obviously the creation of children is the exemplar of this observation. But it seems that women, in there very instinct, create work for men. It goes back to sex maybe.

A man is very different from a woman in the bed room. It seems to me that the work done in the bedroom is essentially the man's. Indeed women try to please men by doing very many things but what I mean is that for the woman to be satisfied requires work from the man. It is very much easier for a man to please himself! This leads to men being accused of being selfish in bed - but isn't this because women can't? This fundemental difference it seems extends to everything. Jane Austin it seems twisted the truth to get the premise of her famous book which opens: "It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife." Only a woman could have written this because the truth, starkly prevalent in the actual substance of the book, is that all women want to be wife to a man in possession of a good fortune! That I believe is the actually acknowledged truth, but an interesting literary strategy never-the-less from Austin to declare the opposite of the truth upon which the book then ironically then proceeds to operate. I can't find the quote from hollywood actress that i found while researching the best man speech but it says roughly that marriage can't ever work because a man wants peace and a woman only ever wants to cause trouble. This it seems (in my limited experience) just seems to be true.

In the new BBC series Ancient Civilisations the narrator refers to a letter written by a wife Lamasi (need to check spellings) to her merchant husband who has moved away to trade her "textiles". She is complaining of him having taken all the household items with him and left her in poverty, which is a quite understandable thing to complain about. Her case isn't helped by her last point however that the neighbour Salima Huum has built a house twice the size since he left. This sounds very familar to me. The letter isn't really about her poverty it is about the age old preoccupation of women of setting up homes and showing off to other people their success. A house buying discussion board during the financial crisis was full of men admitting that it was their wives/girlfriends who drove them to buy a house against their better instincts.

Now I accept many men do the same in a different sphere of life namely building business empires and showing off to other men their success by what they wear, what they own and the women they have in tow. But I don't accept that it is the women who are the oppressed in this arrangement. Rather I suspect the Jane Austins are carefully plying their trade to capture wealthy men and exploit their territory and capital. The films Alien/Aliens is about femaleness, child-birth, sexuality and species identity and I think the predatory way in which aliens prey on human victims is a very realistic analogy to the way that babies prey upon their female hosts. But in turn women prey on male hosts to realise the demands of the child also. It is that parasitic seed of the unborn foetus planted in the mind of woman which seems more than anything to command the behaviour of women and therefore the lives of men who have anything to do with women.

OK that is a very one sided point of view. I have at least one male friend who wanted children while his wife didn't. So it seems that men have the seed also. But the balance does seem to be in the favour of women. My own mother it turns out was the force behind my existence, my father had little interest. It is women certainly in my experience who dominate the cogs of the world and drive it forward seeking especially a stable domestic realm and a "better" life. They are famously the ones capitivated by the bright lights of shop windows and easily suckered in by sales strategies. Men it seems are famously dominated by one weakness, that of women, which is enough to ensure that they are ensnared enough by genes to ensure genome transmission to a new generation.

I have dreamed this week of this quandary and in my dreams been left with the above analysis that while women are attractive and have many positive qualities that make life with them worthwhile they also represent an increase in the amount of work and trouble in ones life that as far as I can ascertain in my dreams comes out equal. If we marry we gain a certain type of peace and satisfaction but we have to let go of ever being allowed to be peaceful as our wife will always be driving us into action; like the dust she will toil her whole life to remove from mantle pieces and tables. On the other hand if we remain single we lose the happiness of domesticity but gain the solid repose of not being goaded by insatisable appetites and endless toil.

What is a self?

Time to get serious with this most basic question... these arguments lead to an investigation of a seeming contradiction in the idea of creating a self.

Do un-self-conscious selves exist?

Let us assume that selves exist, and let us begin with the assumption that everything has a self. Let us call it a soul to begin with.

Firstly is a chair with a soul any different from a chair without a soul? They are to all appearances the same. Indeed we cannot ever see if they have a soul. It may be that some future scientific investigation discovers evidence of the soul; that is irrelevant here, we are only talking about what is apparent to us today.

Take a chair with a soul. Does it know that it has a soul? No. Only I know that it has a soul. So we say that it has a soul-for-me.

We can now question the meaning of a soul-for-me since there is no evidence for it. It is unknowable and is purely an imaginary thing for now.

While souls are not known from the outside as souls-for-me, the are more commonly accepted as souls known from the inside.

A human with a soul is different from the table: not because we can see their soul but because we believe that they know their own soul; they know they have a soul. They are, in contrast to the table, a soul-for-itself.

Souls-for-me are completely doubtable; souls-for-themselves are the essence of the soul idea. It is these souls that we call selves. The relationship between a self and itself is central to the self idea. You cannot really have a self/soul existing by itself (like the table) without also including the idea that it knows of itself. Essentially if it doesn't know of itself then who does! Self-consciousness is thus an important feature of selves.

A Self must know that it is a self-knowing self (a loop)
If a self knows of itself then what does it know? Does it know itself as a material object like a table? If so we have the problem of an unconscious self above. The self must know itself as a thing-which-knows-itself, i.e. must see itself as different from a table whose self is not knowable. In other words a self-for-itself must know that it is a self-for-itself.

The creation of a self
At some point in the past let us say that a self doesn't exist. This means that at some point it must have been created. At the point of creation it is not enough to create a soul that doesn't know that it is a soul. This is the table problem again. The condition is that it must be a soul that knows that it is a soul. Yet it can't be a soul that knows itself until it is a soul-that-knows-itself that it can know! That is to say the condition for being a soul/self is not simply that one exists as a table exists because then it makes no difference if we are a soul or not (there is no external difference between having, and not having, a soul). Having a soul is all about the difference in the internal (private) state. Having a soul means that we know we have a soul. That is the human experience that all these ideas are born from and seek to describe. Yet if having a soul means at the same time that we know we have a soul then to create a soul is not just creating an ordinary thing with external features (like a table). It is to create something without external features and with the central internal feature being that it knows itself. It must know itself in other words to be itself. At the point of creation, before it is itself, how can it know itself and thus become itself?

The idea of a soul/self implies Eternalism and existence without creation. It is interesting that while the argument above seems quite involved and complicated these features of souls (self aware and eternal) are central to the common uninvestigated concept of them.

No mention of the SRH! The SRH would make the above discussion pointless since nothing can relate to itself anyway - but that has been avoided.

A being that exists by virtue that it knows itself (the definition of a soul/self) is hard to picture. The original version of the above argument went differently: from notes last night...

A self must be created - that is the rub because a self (or a belief that something has a self) is by definition the relationship with itself because a relationship with anything else is just plain old thinghood. Something being created, just a plain old anything being created doesn't make a self. It is only a self when we believe that it relates to itself. Yet how can we know if it relates to itself? Only it can know - that is the belief. And if a self must relate to itself then what was it before it related to itself? Does a self relate to anything which relates to itself or must a self relate to anything which is already a self? Surely it is the latter otherwise it is just ordinary thinghood relating. (Start argument by staying that there exists things and selves and relating to a thing is ordinary, while relating to a self is the key to self). So if a self must relate to itself as a self already, then at the creation of a self the relation to itself and itself must be created at the same time i.e. if not created then not exist!

Can a self exist without knowing itself? or knowing that it is a self? i.e. the other exists for-me but not for-itself but is self-existing without me?

"self" is really a symbol for the relationship-between where it relates between that relation and itself... yet isn't this a fudge since it is non-constructive and an infinite regress.

===

Realise that the essence of the point here is the difference between having a self and being a self. Other people we speculate may or may not have a self/soul while I AM a soul. It is not a question of analysing data or thoughts as if they belonged to another person but the very nature of these things at all - which I guess leads to the Tat Tvam Asi where one realises that the theatre or sphere of the World (Brahman) is actually our own Soul (Atman) and vice-versa in that the phenomena and experiences that we have are those of our Soul so the existence of these things which we normally call the World is also evidence of our Soul. BUT one must be careful because this path (as expressed here) doesn't rid us of ego. Seeing ourselves and the world as one we may still claim the world as Ours - e.g. still thinking of my perceptions or my thoughts or my wisdom or my clear thinking and understanding - but if we get the Tat Tvam Asi right who do those perceptions, thoughts, wisdoms belong to other than themselves and the world? "Other" people don't "have" souls any more then that I "am" a soul. They are not (at root) flesh and bone, perception or thought any more than I am. There is only One.

Variance model

Rearranging the equation given before and then calculating the coefficients for the BP (NYSE) data over the same period gave the following two equations:

pFTSE(x) = 0.9716 Exp[-122.54 x]

pBP(x) = 0.9622 Exp[-80.956 x]

x is the deviation in the log stock price and p(x) gives the probability that the deviation is at most this.

A deviation of 0 (zero) or more is a certainty so the multiplier is 1. Comparing the x scale factors with the standard deviation of the two data sets gives the simple equality:

p(x) = Exp(-x/S.D.)

Plotted here in red against the actual data in blue for the FTSE.

To reiterate: this gives the probability that a stock will deviate by at most x at close of the next trading day.

Unfortunately this doesn't escape the fat tail problem and still

underestimates the probability of extreme events! Visible here above about 0.03. This corresponds to events of only 3% which occur about 1% of the time (in the actual data sets used).

Here is the fat tail problem again: FTSE data in green, BP data in blue. Taking the stacked up ordered differences in closing prices, this is the -ve LOG of the percentage distance along the line (i.e. the probability, y) against the number of population standard deviations the move was (x).

As the standard deviation increases the events become rarer. Despite being almost 700 data points the individual black swan outliers can be seen. But as events become bigger the plot bends to the right meaning that they aren't becoming as rare as they

should! Moves of more than 4 S.D. are on a completely different trajectory to those below. I wonder what feature of the world causes that. Is it that moves of more than 4 S.D. trigger a hysterical selling/buying response which leads to greater changes than previous expected.

4 S.D. = 1.032 for the FTSE and 1.049 for BP.

After a 3.1% drop or 3.2% rise in the FTSE or a 4.7% drop or 4.9% rise in BP shares there is a change in investor behaviour (only hypothesis) which makes further decreases/increases more likely than random! I wonder if this correlates to the mean stop-loss placement.

Behaviour of the stock also becomes more erratic, unpredictable (chaotic) during extreme events. How interesting they look!!!! I think these boys hold a few secrets!

For movements less than 4 S.D. the following equation holds

y = 1.0593 x R2 = 0.9922

p = Exp[-1.0593 x] (probability of deviation of x S.D.s)

Essentially the equation above. This applies to 98.2 % of all the stock market closes since 1984

For greater than 4 S.D.

y = 1.9106 x^0.569 R2 = 0.9858

p = Exp[-1.9106 x^0.569]

This equation safely over estimates the probability of events after 4 S.D. and so solves the fat tail problem! It applies to just 1.8% of all closes since 1984.

x is deviation in log prices in S.D.s, y = -Ln[p], p = probability of event.

===

A better model is this. Numerically estimating the gradient successively and then integrating suggested that a polynomial function ought to be attached to the exponential. Subtracting the exponent part and then fitting a power curve to the higher probabilities and a polynomial to the lower probabilities an summing gave this equation which works at the extremes. The graph above is rotated so that given a probability x, the function gives the deviation above which that many events fall. e.g. a probability of 1 means that all deviations lie above it and so y=0.

y = 0.0234*EXP(-162.49 * x)-0.0978*x^0.0826-0.006*x^2+0.0079*x+0.0959

Unfortunately it can't be expressed in terms of the S.D. to find the probability of events falling above that S.D.

===

NOTE: Another thing I am interested to investigate (apart from variance and risk) is the phenomenon of buying before a drop and selling before a rise. It happens far more commonly than random for the "rational" investor while happening at random for the random investor! This suggests that human nature actual leads us to make bad decisions ... or more likely the market has so evolved so as to exploit these aspects of human nature: greed and fear. So Contrarian analysis exists to exploit this; buying when others are selling and vice versa. However interestingly negating our greed/fear instincts leads us into the same trap! A trading decision engine TDE (which is what I hope to create eventually) must take into account these instincts and indeed the very motivation for Humans to play stock markets.

Partial Theories

Probably not the best time to update the blog I'm tired and off to bed but just rereading some of the stuff from before the wedding upset the train of thought, brings a lot of recent stuff together...

The world is a series of Partial Theories - my name for the fact that we can form a good theory of something without needing to know anything else. This links with the observation that we can form a theory of the brain without really understanding it "all".

Now my critique of materialism is the realisation when we talk about the brain that we are not just talking about some lump of nerve tissue that other people have but also something that we ourselves have and the more we attribute to that lump of matter in other people the more we have to attribute to our own piece of nerve matter up until the final material jump which is realising that our own piece of nerve tissue is what is producing my thoughts. So these thoughts I am having about my brain being the source and cause of my mental activity (from seeing to thinking to believing) are actually just the product of a lump of nerve tissue. Now that is the leap the SRH says can't happen.

However this assumes that there is a some fundamental root matter to the brain: a total theory so to speak, waiting to be uncovered; a final truth of the brain that one day we will know. Yet the world is a set of Partial Theories. The existence of Partial Theories is not what happens until you have a complete theory - it is fundamentally different from Total Theories. In a Total Theory world you know nothing until to know everything. You can't say what the weather today in UK is until you know the weather everywhere else in the world at all times in history. In a way this is true. But as I read last week in a stochastics book (linked to the stock market model being worked on at the mo), statistics is the product of this ability to isolate a system from its surroundings and see those surroundings distributed across a random variables. If we knew the positions and momentums of every particle (and sub atomic particle.... etc) in the universe then with a sufficiently large computer we could model the entire universe- but with statistics we don't need to. We just need to say that an even dice has a 1/6 chance of falling on any face and the rest of the universe randomly distributes across those 6 measures. Statistics proves that Partial Theories are possible without needing to know everything - patterns exist even within the incompletely known! In a Total Theory universe presumably we wouldn't be able to predict the outcome of any dice until we knew everything else in the universe! This would mean, like Rosencrantz and Gildenstern in the play/movie, that we could test whether we were in a Total or Partial Theory universe by repeating tosses of a coin and it not only being unpredictable but the distribution of Heads and Tails never producing any distribution - one moment seeming even and the next being all heads and then all tails and even after 1,000 tosses or 1E6 tosses neither side winning or being even or any distribution emerging. Its a bit frightening to think about that type of randomness - does it exist?

Anyway that is not how it is. The more you throw it the more likely you will have a comparable ratio of heads and tails. In other words you average out the rest of the universe and are left with just your system. A Partial Theory!

So we can form a Partial Theory of the brain. So if we are to identify ourselves with that brain which bit of the theory do we identify ourselves with? Straight from Hofstadter now: do we identify with the nerves, the networks, the electrical stimulation, the psychology, the thoughts, the dreams, the meanings etc.

So when Hofstadter refers to himself what does he mean to say that he is a brain? What does any materialist mean. What can they mean? Well actually what do they mean when they even refer to brain?

My argument against materialism is against the belief which I think materialists hold: that there is some kind of material discoverable stuff in the universe. A kind of hard "ether" out of what everything is built. A solid, definite, unquestionable stuff that acts as a root or foundation for what is true; while what is false has no such solid stuff backing it. A bit like the search for a lost jewel. When we find the jewel and hold it in our hands we know that it is real. Until that day it may just be a ghostly piece of imagination. One has material backing; the other is a myth and phantom and is not real. I think this is how materialists think. But you put a materialist in a detective role searching out the perpetrator of a political killing and they are suddenly out of their depth. They find the person who pulled the trigger and find that they were only under orders from another person who might even be the person who gives orders to the detective. Suddenly the command structure which employs them as a detective is also the one who commanded the murder and now the solidity of what they are even searching for has evaporated - and yet a murder still happened! (Old story line used many times: Sherlock Holmes in search of Jack the Ripper, JFK, Angel Heart)

What materialism can't do (obviously, surely) is give material backing to their ideas. Where is there room for ideas in Materialism?

The point is that Materialism is just a Partial Theory itself. The idea of Partial Theories must be a Partial Theory itself! The idea of Total Theories on the other hand would have to be only known once we knew everything else in the universe... (there is a simple Q.E.D. argument here: have a think when you are rested). This is getting very close to the SRH again :-) Need sleep, party 2morrow... signing off

More on Variance

To get a measure of the probability of up-step versus down-step to measure a trend strength... p = 0.5 + mean / (2 n I)

===

Actual Data

A plot of I calculated from the FTSE since 1984 gives a normal like distribution but with fatter tails! This is not good news...

http://en.wikipedia.org/wiki/Fat_tail

It

is possible that the random walk will fundamentally fail as a model because stock prices simply don't fit! To be investigated.

Yet Black Swan events are as many commentators point out exactly that "black swan" events and White Swans are the norm. So predictability does reign in the stock markets. There is only 1 event in the 26year daily data sequence I am looking at which stands out as a black swan. The rest of the events obey the distribution even though it is not known at this stage what it is. The probability of variance can be calculated for the FTSE simply by looking at the back data ignoring any modelling.

Analysis of FTSE (1984 to present)

VAR0002 is the difference between daily price closes for the FTSE rounded to 3 decimal places. Values are in 1/10%s.

Statistics
VAR00002
N	Valid	6725
	Missing	0
Std. Deviation		11.25086
Variance		126.582
Skewness		-.386
Std. Error of Skewness		.030
Kurtosis		8.795
Std. Error of Kurtosis		.060

The distribution is pretty central

which is good but the Kurtosis is massive meaning that the stock market data is leptokurtic and spikey with fat tails clearly visible here against the normal distribution. Also evident in the Q-Q test below. The Normality test proves that it is not normal.

Tests of Normality
Kolmogorov-Smirnova
Statistic	df	Sig.
.072	6725	.000
a. Lilliefors Significance Correction

So some searching later I give up on probability distributions (none of which fit) and go back to the ranked data. The graph below shows the absolute difference in log daily price closes for the FTSE since 1984 simply ordered and stacked up on the x axis. It crosses the x axis at x=6617. It is a very crude probability density function!

The probability of getting a deviation of more than 0.02 in the FTSE corresponds to x = 500 which is 500/6617 of the days' closes and a probability of 7.6%. That is 13/100 days in the FTSE history the price has closed up or down by at least Log[0.02]!

A deviation of 0.02 means a price change by a factor of Exp(0.02) = 102%.

This is real data not a model with assumptions. so this is meaningful and is an actual historically holding measure of risk!

Does this relationship hold for other charts?

Can it be used to derive a proper probability function that can be used to determine actual market variance and am actual measure of risk?

Also up 2do is the cellular automata model of stock markets. Will use the model presented in a paper firstly: buy/sell orders placed randomly (not decided the variance function yet) around the last exchange price in a number of stocks. An automatic exchange will match the orders as it can. A number of bots will play this game and keep account of their portfolios. Once this is working a graph will be made of the stock price movements so see if it mirrors actual stock markets.

Next up more advanced decision algorithms using moving averages.

Next up a genetic basis to the algorithms so they can be evolved.

Multi-fold interest here. (1) Test my own strategies to see which provide maximum return. (2) Discover new strategies by evolution (3) Watch how the strategies interact and spread through trading populations (e.g. if everyone is contrarian then it pays to act normally and vice-versa). (4) Discover if a stock market chart can be created in a model so its variance etc can be understood better.

LOT OF WORK!

Haley v Newton methods

Comparing Haley's method and Newton's method of finding roots: nice webpage,

http://www.math.hu-berlin.de/~llehmann/topics/newton-fractal/

Information is relative

I once asked a friend at college (who occasionally used to read this blog) what his course Information Theory was about. He gave a most excellent answer that has been suck in my head ever since:

Which sentence contains more information: 'it snowed in January' or 'it snowed in July'.

Clearly big deal for the first sentence (technically it has low surprisal) but the second sentence is a very rare sentence and so has high surprisal and carries a lot of information. I might point out further that "you are reading some words" to show how low surprisal is as good as no information at all and we ignore it.

Of course to someone from abroad who has never seen snow both sentences are equally important because they are learning about snow. It is only when someone says we never get snow in the summer that the difference in information described above becomes apparent to the student.

So the information content depends upon the state of the receiver, and the state of the receiver is an historical feature. There is no absolute information content then independent of the environment or other information sources.

The Kolmogorov complexity which is measured by the smallest possible algorithm that produces a given set of data seems to be a search for an absolute information content. For example the Mandelbrot set can be produced by a very simple algorithm Z^2+C > 4 but most humans aren't very happy with that and like the picture. The picture is instructive to experts also because it shows the limit of the set - the famous apple man - which can't be deduced from the algorithm above. What is needed is a program to render that equation. The usual loops and graphics functions. What if there was hard coded graphics function:

M(complex c_position, double zoom)

For a given complex number and zoom it rendered the Mandelbrot set on the computer screen. Can't get much lower Kolmogorov complexity than that. But it is a cheat and just hides the real complexity. We need to put all the sub-calculations in to get a more realistic idea of the complexity.

But what if the algorithm produced the image in red and green and the receiver was colour blind. It would still fail to produce the apple man. And what if the receiver was blind all together and the algorithm had to drive a Braille printer which took up more coding space. The algorithm has to assume things about the receiver. I assume that Kolmogorov complexity refers to the ability of algorithms to produce sets of numbers rather than pictures and for any fixed set of numerically literate users it will be a relative measure where indeed there may be a smallest program that generates using a fixed minimum instruction set the required data. But peppered throughout the preceding description are the relative conditions of that measure. It is not absolute. No measure is absolute by definition because measuring involves providing a ratio against something else. That is pure SRH.

In passing SRH goes further to say that the ratio of something and itself is not 1 but undefined since no such comparison can be made meaningfully. One may argue that ruler A is 30cm long and a stick of wood the same length is 30cm long. One can then use the stick of wood to say that ruler A is 30cm long. But can you? What if ruler A was really 25cm long? We have correctly identified that the stick and the ruler are the same length. That is the ratio that we know. We can say that ruler A and the stick are the same length. We cannot deduce from this that ruler A has an "actual" length however, and then say that this imaginary actual length is the same as itself. We can only ever say that Ruler A and the stick are the same. If Ruler A becomes our standard then we have a system of measurement but that is a cultural and political point not a logical one - hence the pun on the word "Ruler"! This is why the ancient Greeks (being more truthful) were stuck with ratios and it took much later developments to develop the idea of actual Number. That is the crux of the problem... and ah ha as I get closer to Buddha's teachings I get closer to the SRH! Maybe SRH is just Anatta (non-self). The same argument works with humans. We find ourself in relations to other people, in relationships. We deduce from these relations that it is a relationship between "people" just as we incorrectly think that things being the same length means they have some intrinsic length outside the measurement. It follows as I was contemplating during writing the "Book of 7 Stories" for "my muse" that two rulers that have measured one another and been seen to be the same if they are then parted and sent into different universes can no longer be said to be the same! "My muse" did indeed depart for a different universe and I have wasted a long time trying to examine that measurement which I now see no longer exists. Indeed Death does win against those who believe they are together but maybe Riswey is learning the lesson of Elrus (that I confess I only vaguely grasped at the time of thinking out and sending to "my muse") that "we" really were never the same, and she never was "the One" because there were no people there to be measured and commensurated and so certainly no people to be linked by consumation. The relationship was what it was, the measurements, the similitude, existed while were together but once one of the rulers was broken the other ruler ceased to measure also. They say size matters, but how large am I? I need relationships to find that out and i will be a different size in each relationship... but all of human life is relationship because there is no ruler to exist out side of measurement. In days of old that density of meanings would fashion a poem but I'm off for a cup of tea now instead leave the blank page that follows to be the poem that measures what is said here.

===

The absolute crux of this argument is that it is a mistake to say that something is the same length as itself!

A=B

B=C

C=A

Is correct, although I spent considerable time as a child looking for a "reason" why this was true. What is false is to then argue that:

A=A

The difference between this statement and the 3 above is that they are describing relationships "between" things. There is no relationship between A and itself, and if there was it is intrinsic and inaccessible to the outside world. We are inventing some internal property which we are saying is internally equal to itself as though this was meaningful. I will invent some internal quality called "sinustance" which all things have. Now I can say that A = A because they both share the same sinustance and so the sinustance is in direct relationship with itself. There we are I've proven the ontology of sinustance. But sinustance is completely made up as indeed is length when it is used to compare something to itself. The idea that the length of A=B means that A has some persistent quality "length" which exists after the measurement and which can then be used to put A in direct relationship with itself is exactly the same myth as sinustance. It is the basis of the materialist fallacy. A and B have length that comes from being measured, it wasn't there before and it isn't there afterwards. Shades of collapsing waveforms here, that become discrete when measured... I wonder if all that hokum is just an aberrations of models that are based upon old materialist thinking and maths rather than anything intrinsic to the data sets or "reality".

These are not words!

Was discussing with a friend an old philosophy question of his: "Can you be sure that you are looking at a sheet of paper". We all recognise that the moment he starts to answer that question he has belied any attempt to throw doubt on it being a sheet of paper, and it being written upon in English and this being a philosophy exam &c. The answer, if it believes it is an answer, must recognise the normative assumption of question and answer papers &c.

Another take emerged. What if the question is answered Magritte style. "You can be certain that this is not a sheet of paper." This is in fact true because a sheet of paper is not in the English language, only "a sheet of paper" is in the English language. So when you are reading you are on a different level from "realit

y". Words become what they mean when you read, they are not the black marks on the page. Using Google translate I can create a sentence in Chinese "這些都是不言". To me these are just marks on a page. To Chinese readers they are just marks on a page, but they can also undergo the Necker Cube shift to those minds skilled in Chinese reading and become meanings in the mind.

Now show me a "word"? Let me take the words "a tree". What does this mean? It is not a trick question we all know who speak English. It means any, not being specific, tree which is usually a tall plant with woody stem and in summer has lots of leaves. So a tree is not a word: it is a tree! A word is the marks on the page, referred to as "a tree" with quotation marks. Ths btw is how do you refer to the quotation mark itself... with the name "quotation mark".

So if words are the marks on the page then exactly as with the sheet of paper, the sentence "these are not words" is actually true when it is read and has meaning. It is only false when it is just marks on a page, but then it has no semantics and so can't be true or false. They are not words these are in fact meanings. You are reading words is true. But "you are reading words" has no semantics, as we officially aren't bothered with the meaning of the words in quotes. In reality a reader will try the semantics of the quote to get to know what is said. But ")D£Z" is a valid part of an English sentence because quotes don't have to make sense; they simply draw attention to the marks on the page.

If "These are not words" is a true sentence when read for meaning then how can "this has four words" be true when read for meaning? If words make up a sentence then "this is a sentence" is also false when read for meaning.

So how can a sentence refer to itself? We allow the marks on the page to be a symbol or name for the meanings. My name "Alva" doesn't mean anything when used as a name with a capital letter. But in Spanish, where the name is from, "alba" means dawn. Obviously my name is not to say I am the light that comes before the sun rises (the exact nuance I was told by a Spanish speaker) it is just a blank symbol for me. So we can ask, for instance, what the meaning of "The swan swam down the Swale." is using the marks on the page to refer to an, as yet not got, meaning. And this sentence is perfectly meaningful: "The sentence you are reading" because, it correctly identifies that you are reading a sentence. But a sentence can't directly refer to itself because the marks on the page are different from the meaning. If the marks on the page were the meaning then we wouldn't need to read and sentences would just be true by themselves (this is the basis of SRH)! In which case what about the old problem "This sentence is false". Thankfully this is just marks on a page. It is only a problem when some unfortunate soul reads it and tries to make sense of it. Actually it could be revised for better accuracy in light of this blog to, "The meaning of the sentence you are reading is false." since the "sentence" is just syntactic marks on a page and it is the meaning that we are referring to and saying is false. But what is the negation of that sentence?

Let a function x:M(x) = {1,2,3} separate sentences into three sets those for which M(x) =1,2 or 3. We have a semantic function then on our set of possible sentences X whose members are named small x.

Let another function y:T(y) ->{true, false} separate sentences into true or false.

The standard liar paradox above applies the truth function directly to the sentences in X so that we can say that "this sentence is false" - as though the sentence was actually materially false independent of any external judge and jury to evaluate the sentence - that is the problem with the traditional interpretation and where the SRH has its challenge. But a bit of thought in this blog has shown that this is actually over simplified. What is actually happening is that the meaning of the sentence named by the marks on the page is false - rather than those marks on the page. So it is actually the M(x) function that is false. So the sentence is saying that M(x) is separating sentences into the wrong sets. Now we always take truth T(x) to be part of the meaning function M(x) so when we understand the meaning of the sentence "It is false to say 'dogs can fly'" we also get the truth of the quote. But what is being rejected in "The meaning of the sentence you are reading is false." is not the sentence "itself" (which SRH says is impossible) but the meaning function M(x). So the paradox doesn't create a loop any more, because once the meaning has been rejected we just take the sentence to be marks on a page and read on which is what we actually do with the liar paradox (who ever gets stuck for very long). "There is no meaning to this sentence" looks like our new version of the paradox but actually this sentence is just false, because there is meaning to the sentence which is how we start to get into paradox but then we just reject the sentence as making a false claim. Simple.

It seems this deeper look at words and meaning has dusted out the old paradoxes and while touching on the SRH it doesn't use it. It turns out that meaning can refer to itself but without the creation of paradoxes. I need to let this sink in and think about what is happening here in more depth.

Random market variance

Investment Maths and Random Walk looked at mean returns in a random market and stock movements in a random market. Next up is variance.

Let p=probability of stock going up, q=(1-p)=probability of stock going down, and 'i' is the percentage change so that I = Ln(i). After a single period of time the stock may rise with probability p or fall with probability q.

Ln[s(1)] = Ln[s(0)] + pI - qI

After n periods the possible outcomes will produce the usual binary tree with probabilities:

(p+q)^n = Sum(w=0 to n) [ E(n,w) ] = 1

with each term of the expansion being:

E(n,w) = C(n,w) * p^w * q^(n-w)

Where w is the number of wins (up movements) and C(n, w) the binomial coefficient. Each outcome will produce a stock movement of (wins-losses), equivalent to (2*wins - n), which when multiplied by its

probability from above gives its average contribution to the average stock movement. Each term is thus:

Return due to path w, R(n,w) = wins - losses = I(2*w-n)

Average return R = Sum(w=0 to n) [ E(n,w) * R(n,w) ] = n I (2p-1)

Variance, S.D.^2 = Sum (w=0 to n) [ E(n,w) * (R(n,w) - R)^2 ] = 4 * n * I^2 * p * q

S.D. = 2 * I * Sqrt(n * p * q)

For p=0.5 the distribution of outcomes follows the Normal Distribution (continuous version of the binomial expansion) so in general we can approximately calculate (better for large n) the probability of a particular return using N(R, SD). For an example the probability of getting a deviation of 2I with n=4:

z = 2I / 2I = 1 standard deviation

Which occurs with probability = 34% (using z table).

A deviation of 2I corresponds to 2*Log(i) which if i=1.05 means a deviation of 0.04 in Log space. This corresponds to an actual stock market deviation of:

Upper deviation = 10^(Log(s) + 0.04) = s*10^0.04 = s*1.103

Lower deviation = 10^(Log(s) - 0.04) = s*10^-0.04 = s*0.907

This is calculated for an investment in a single stock. Now what happens if we invest in two stocks.

Variance in portfolio of 2 Stocks

The combined portfolio (S) results from the addition of stock 1 (s1) and stock 2 (s2):

S(n+1) = s1(n+1) + s2(n+1)

Each stock follows the model above:

Log[s1(n+1)] = Log[s1(n)] + I*(

2 * wins1 - n

)

Log[s2(n+1)] = Log[s2(n)] + I*(

2 * wins2 - n

)

So these cannot be expressed in Log space. Instead they need to be expresse

d in real stock price space.

s1(n+1) = s1(n) * i^(

2 * wins1 - n

)

s2(n+1) = s2(n) * i^(

2 * wins2 - n

)

So that,

S(n+1) =

s1(n) * i^(

2 * wins1 - n

) +

s2(n) * i^(

2 * wins2 - n

)

Assume the portfolio is evenly split then s1(n) = S(n)/2

S(n+1) = S(n) * [

i^(

2 * wins1 - n

) +

i^(

2 * wins2 - n

)]

The distribution of outcomes from each stock is as above and obeys the Binomial expansion above. For each outcome in stock 1, stock 2 obeys the Binomial expansion also. If n is the number of trading periods then there are (n+1) terms in the binomial expansion of both stocks so there are a total of (n+1)^2 combined terms. In general that is total paths:

TP = (n+1)^sigma, where sigma is the number of stocks in the portfolio.

So for any particular outcome of wins and losses we multiply the coefficients:

Number path with this combination of wins NP(wins1, wins2) = C(n,wins in stock 1) * C(n, wins in stock 2)

In general for sigma stocks:

NP(wins1, wins2,...,wins_sigma) =

C(n,wins in stock 1) * C(n, wins in stock 2) * ... *

C(n, wins in stock sigma)

Thus, assuming that wins and losses are equally probable, the probability of a particul

ar outcome is NP/TP

The value of that outcome is calculated from the equation above for S(n+1).

Mean and Variance for 2 Stocks

These values can't be meaningfully calculated in price space since the distribution is skewed. The mean and variance need to be calculated from the Log(price) values. There is no simple way (that I know) to simplify these calculations so I

switch to a program to do the job of calculating.

Interesting to note what a massive calculation this is. A portfolio of 20 stocks taken over just 10 trading periods will take almost the entire age of the universe for my dual-core computer running an interpreted basic program to calculate!

This graph shows the standard deviations in terms of Log(i) in the y, for various portfolio sizes (x axis) and over increasing trading sessions (z axis).

It seems to settle down very fast as portfolio size increases which suggests at first look that portfolios don't need to be very diversified to reduce risk in the random market.

By inspection of values it can be seen that this follows an exact pattern:

standard deviation = Sqrt(number trading sessions/no of stock) * Ln(i)

This crude measure can be used as the basis of measuring risk in a portfolio. The larger the S.D. the greater the chance of big gains, but also the greater the chance of big losses. However if we reduce risk we don't make any gains or losses.

Below is the distribution of all outcomes in a 6 stock portfolio over only 4 trading periods, in a random market with equal chances of a step up as a step down. X axis is Log(return)%, Y axis the number of outcomes. While it is a very complex irregular pattern it "seems" to fit within a broad normal distribution. The gaps are due to certain paths being impossible given that the system is very restrained only allowing up or down moves which excludes the possibility of staying the same. One possibility is to examine the multinomial version of this model with (win + lose + same)^n. The graph following that is the same but with the normal distribution roughly fitted and unadjusted log(return) values in the x direction.

The Model

So the exact distribution in natural log space of a stock chart following a random walk will approximate to the binomial distribution with:

mean = 0

S.D. = Sqrt(n/sigma) * Ln(i)

Where n is the number of trading sessions, sigma the number of stocks in the portfolio, i the average percentage increase/decrease of the stock in trading sessions.

Let us take an example:

Closing prices for the FTSE from 1/7/2010 until 12/11/10 downloaded from Yahoo. An up trend that looks like a rising wedge to me and so a reversal pattern. At the same time the DJI is breaking resistance and possibly pushing higher.

Calculated from the whole FTSE data starting 1984 the average daily factor of increase/decrease

is i=1.0079829. That is if

stock price s1 > s2 then i = s1/s2

otherwise i=s2/s1.

The number of trading periods (days) in this graph is n=89. The gap up is r=992 so LN(r) = 6.9

S.D. = Sqrt(89/1) * 6.9 = 0.65

z = 10.6 and the probability of a random walk going this far is p=3E-25!!!!

In other words it is impossible that this is random! Rejecting H0 we must accept that this is an actual up trend! Now the great question how long will it last? 2Do

I need to check all this working and logic but this is the first outline of a crude method. 14/11/2010