esoteriic.com

As part of my interest in trading and investing I decided to look at price patterns for the UK stock market. This school of thought might be labelled as technical analysis. Some believe that all the necessary information is already in the price. While I do not believe this to be the case, I do believe that a lot of effort is put into understanding charts and trying to extract patterns from the data. The movement of the price is not wholly random, there are clear trends in the data. The key is not to simply see the trend which has happened but to anticipate where the price will go based upon the past evidence. What I did was to take the numerical derivative of the price with respect to time. I tried higher order derivatives and found a potential pattern emerging. This seems to corroborate with the VIX. The derivatives display a structure that looks like beat phenomena.

Getting the data: Went to Yahoo finance, copied the monthly prices for the FTSE 100, and then pasted into a Google docs spreadsheet.

All the data from Yahoo is free and since I don't have access to the same numerical tools that I used to then I had to put up with Google docs (which is overly obnoxious). The data has 5 different prices, I discarded all of them except for the 'close'. The data is spaced in time by 1 month. This is a coarse sampling of the daily price data but it is dense enough to look continuous. I think it faithfully represents the underlying trend well enough to be of use. There is an obvious wave-like structure to the data. You can see where the FTSE rose then crashed in 1987, as well as rising in 2000 as part of the Dot-com boom. The direction is never straight up, nor is it particularly smooth but there is a clear wave-like nature. This is obviously the inspiration behind Elliott wave analysis and to some extent 'charting' (as part of technical analysis).

Here's what I did: Nothing too fancy. I simply took the numerical derivative of the price with respect to time, and then tried the higher order derivatives and found a potential pattern emerging. The derivative was calculated as f'(x) =  (f(x+h) - f(x))/ h and set h = 1 (month).

First here is a look at the FTSE chart from 1984 to 2012 (in blue), plus the first derivative (in red). The wave-like structure of the price is clear but the structure of the derivative is not clear from this pictures. I only included the derivative here to show how it matches up to the price: a big rise or fall in price produces a spike in the derivative. The average (mean) of the derivative data is about 13, the positive sign is indicative of the main price trend being upward. That is, from 1984 to 2012 the price has gone upwards. You could also look at this as the average speed, both are first derivatives with respect to time.

If we examined just the first derivative without the price then we could see a potential pattern emerge. We need to be careful that we are not just seeing what we want but rather looking at something that is actually there. Then we wan to use this past data to construct a potential future scenario. I am claiming that there is a long term (circa 10 years) oscillatory structure to the first derivative, by eye you can see a pattern where the magnitude is diverging then converging over a period of about 10 years. The real emergence of the pattern comes from identifying the periods of 'calm', where the price moves least. There are some distinct 'islands of calm' against a sea of 'noise'.

The price is fairly calm through most of 2004, and also from late 1994 to early 1996. Between those years was the tech boom / bust, so there is some wild price action. After 2004 we see a divergence in the price derivatives until the late 2008 when it has the largest nominal change, then  the derivative begins to converge. The periods of calm have been during bull moves. So if the price derivative converges (gives us a period of relative calm) then we will be in a bull market. I'm anticipating that it will converge before 2015 and hence by 2015/2016 we will be in the up leg of the next bull. Part of my bias in claiming that is a potential scenario generated by (1) charting (a bullish pennant) and (2) from Elliott wave analysis.

Higher order derivatives: Naturally, I expanded the analysis to higher order derivatives to see if there was another pattern. I was hoping there would be something obvious. I plotted the first four derivatives on the same graph and you can see that with each successive order in the derivative there is a greater amount of 'signal to noise'. The pattern becomes more pronounced. Plotting D^4 (fourth order derivative) along with the price shows the pattern well. What is really needed is to see if any of the derivatives give an even stronger statement about the direction of price. Perhaps the price lags a high order derivative and hence we would have a leading indicator of where price is going. I have identified a potential pattern in the first and second derivative which states that if the derivatives is above some value X then the next month will display a significant drop in price. I don't want to say much more there until I can be sure the pattern is reliable.

(blue is first derivative, red = 2, orange = 3, green =4)

The average of D^2 is positive but not far from zero, this suggests that the change in price is accelerating in a nominal sense. Not a huge surprise since the indices are climbing higher than the monthly change in price will be larger (although the percentage change can be the same). Given the wealth of data it will take some time to make sense of this and see if there is something more solid that can be pulled out from it.

Dow Jones: Unsurprisingly, the same pattern emerges in the Dow Jones 30. The two indices are correlated so the price action is similar, the emergence of beats in the higher order derivatives is also present. In a blind test without the numbers on the axis it would be difficult to tell the two apart, obviously if you observe the ratio of the first and second major peaks in each case then you would see which is which.

DJIA / FTSE:

So let's plot the Dow with its D^4 (blue and red) plus the FTSE and its D^4 (orange and green). The top two lines are the price, while the bottoms ones are the fourth order derivatives. The highly correlated nature is very apparent.

Recap:

If the price derivative converges then we will be in a bull market.

I'm anticipating that it will converge before 2015 and hence by 2015/2016 we will be in the up leg of the next bull.

Next steps:

To identify solid trends and hence build a leading indicator for price, if possible.

Potentially smooth the data and see if helps to identify key trends (that aren't obvious from the above graphs).

To take the daily data and see how similar it is to the above monthly data.