Cyclic analysis in finance - science and art in action
Do cycles work in finance? I always feel myself uncomfortable answering this question, as it depends on what we are looking for... The answer to this question is NO, if we are looking for the same cyclic models that are used in physics, geology, cosmology, meteorology etc. What these sciences apply is a classical cyclical analysis; any mathematical package (like Mathematica or MathLab or any other) is able to perform this task perfectly. However, this approach does not work for finance data. Physics, geology, cosmology, meteorology - all of them deal with persistent cycles, the cycles that have worked in the past, are working now and will work in the future. The standard mathematical criteria that are used in cyclical analysis (like Bartels significance test) are developed specially for these cycle, and one of their goals is to reveal hidden cycles in any set of data. It is possible to do just because these sciences deal with the same subject/subjects (generally speaking). For example, when we discuss geology, we understand that - no matter what time span is considered, one or ten million years ago, - we still consider that same Earth as the one that is our home planet now. Sure, we take into account that the Earth has its own evolutional path, though the forces that drive these changes are still the same.
It is another matter when we speak about phenomena that involve human activity - like sociology, economics, stock market. Here we are entering a totally different field. Here we deal with fast evolution of open systems. The stock market now and the stock market in early 90s are totally different entities - electronic trading systems have changed the look of this land a lot. The stock market in early 70s and in 90s are different entities as well - the money of big financial institutions have changed the landscape of the stock market land significantly. And the stock market in the beginning of 20th century compared to the stock market in 70s is also a different entity - economical forces have been changed a lot within this period.
Another peculiarity of financial cycles (and any phenomena where the humankind plays the main role) is that they work inside an OPEN system. What forces move or affect the stock market? It seems to me - everything: FED news, economical/corporate reports, the weather, political perturbations and many other factors plus something that we do not know yet. This is what we call fundamental factors. If in geology we can specify the most influential factors and create models based on these factors, in regards to the stock market we cannot do that as there are too many factors. As a result. a geologist can be a very specialized professional used to deal with a very specific set of factors, while a financial analyst cannot do so. A financial analyst has to keep an open mind taking into consideration too many factors.
Add to that the fact that the influence of fundamental factors on the stock market is indirect. There is an agent between the two, and it is the humankind. FED news does not affect the stock market directly, it is a reaction of traders/investors on these news that affects the stock market. Therefore, creating forecast models (for example, a Neural Network model) that take into account fundamental factors (FED actions, oils price, etc.) and cook from these factors a probable price movement, we forgot about about humankind who estimates all these fundamental factors.
Flexible/Adaptive/Non rigid technology
First 20 years of my scientific career I worked in the Institute of Nuclear Research in Russia (http://www.inr.ac.ru/), and the last 15 years I spent developing software for modeling and forecasting the stock market. Analysis of huge data sets, modeling of physical processes and a scientific forecast based on these models was always of a special interest to me. So, when I have started developing financial software 15 years ago, due to my experience and education, my first approach was to apply classical math methods. At that time I believed that the problem is in finding correct cycles... or combination of cycles... or trying some exotic cycles, exotic metrics... then create a model - and it is done... It took me quite some time to recognize that modeling of financial markets is not the same thing as modeling of physical processes (though for nuclear physics), to recognize that something is missing in this picture. As an example, take the idea of some fundamental factor X affecting the stock market. While thinking about it, we have to ask ourselves how this X factor will be accepted by traders/investors as this reaction can be different in different moments of time. I tend to consider the stock market as a non-rigid system; if we want to model it, some non formal methods are needed in any case. The use of formal methods only (i.e. classical math methods) is not enough. There must be some union of formal models (like spectral analysis) and non formal ones (like Fibonacci, pitchfork or other charting tools).
Step #1: Wavelet - cyclical alert
I have tried to combine formal and non-formal methods in the the new module, Wavelet Cycle Hunter. It is an attempt to maximize the advantage of formal math approach, while providing flexible solutions for users that help to deal with always changing stock market reality.
This is how it works.
As the first step, we should apply a formal math procedure. Look at this diagram:
The diagram shows the life of cycles in time. We can see here how cycles are born, live and die. These bright yellow colored regions correspond to the periods when cycles manifest themselves. It means that within this period a cycle is strong, and we can apply methods of cyclical analysis for this time interval. The earlier we recognize the moment of birth of these cycles, the better, as we would be able to use these cycles to our benefit. This is why the wavelet module is developed: we run this module in real time and watch when these bright zones appear. I would say this is a kind of cyclical alert.
The diagram above shows that during two weeks of November four cycles have appeared - 246, 215, 342 and 141-minute cycles. The life time of these cycles is 1-2 days; and we can classify these time intervals as periods when the stock market is in cyclical mode. It means that within these 1-2 days the cyclical phenomena, the cycles, play an important role in the changes of the stock market behavior. Outside these periods, the stock market follows some other laws, the cycles do not work there.
We can compare these cyclical periods with the waves produced by the stone that is dropped into the water. Look at this:
And now look at another picture; this is a wavelet for 171-minutes wave (Dow Jones 1 min chart is used for this example):
The analogy between a stone dropped into the water and the appearance of cycles in the stock market behavior is the best illustration of wavelet analysis's application. The beginning of bright yellow wavelet zones (cyclical periods) corresponds to the moment when cycles are born. Here they are:
Any cycle is alive for some time; then it loses its energy and dies - exactly like the circles on the water surface created by a stone. I guess that in the most of cases the moments when cycles are born correspond to some significant fundamental events (it is just an idea, this issue needs more research). A fundamental event like a stone dropped into the water creates some perturbation on the stock market (water surface). This fact affects the stock market for some time, and then the stock market forgets about it.
We may say that these waves, limited in time, are a language that the stock market uses to communicate with the World. It looks like each fundamental event causes a special vibration of the stock market's voice, like some event that has occurred on November 16, 2011 (maybe at the end of Nov. 15 ) made the stock market to answer by 171-minute wave.
At this point I would like to emphasize a huge difference between the classical cyclical analysis and the cyclical analysis applied to financial data. We may apply classical analysis mostly when we analyze the phenomenon itself (like geology mostly studies the Earth itself) - this is the science. In finance we study how one entity, the Stock Market, communicates with another entity, the World; this is the science and the art. We cannot explain the variety of this World using a formal logic only. However, using the formal logic( i.e. wavelet analysis), we can hear the voice of the Stock Market early, learn the tune of this voice (which is a period of the cycle) and enjoy it; this is what a trader needs.
When we know the tunes that the stock market uses to communicate with the world, we would definitely like to know what the stock market has to say. It would be a projection line.
It is very simple to do in the new module. Simply drag the mouse cursor over this bright zone, - and you will immediately get the projection line based on this cycle:
You can drag the mouse cursor in a different way, covering more price history data (for example, include November, 16); you will get a bit different projection line, like this one:
At this point the wavelet module works like any charting software: you can drag this wave over the screen modifying a period and a waveform and watch how this cycle works. It is possible to create more complicated models that include several cycles, like on the diagram below:
It takes several seconds to create the three cycles model above. You need just to drag a mouse cursor three times covering three bright/cyclic zones, plus you can easily modify these cycles. While you drag the mouse cursor, the program performs complicated calculations fantastically fast and displays the resulting projection line. This system works in real time, i.e. while the new price history arrives in real time, the program automatically recalculates the wavelet diagram. Thus you can see the birth of new cycles in real time.
Here is the screenshot of this module working in real time regime:
During one trading session here, two short term cycles have appeared. In total, these cycles worked for 3 hours.
How can you apply this wavelet system to your trading? Let me show it in just one example. In this example we will consider very fast models, and the forecast horizon here is about one hour only. I have made more examples, using the models with bigger forecast horizon, 2 hours. Here is the description of these models: preview.htm
Before applying any wavelet model for your financial instrument, you need to test this instrument's nature, just to see that it has some tendency to be cyclic. There should be present long living cycles with the life time of 2-3 full periods. For example, you see on the wavelet diagram that there is one hour cycle; this cycle should be shown there as active at least for 2 hours, better for 3 hours. If it is so, we can consider it as a relatively stable cycle. Stable cycles appear on the wavelet diagram as horizontal bright zones (see the diagram for Rule #1 below).
Here are the rules you should remember while creating a wavelet cyclic model:
Rule #1 `Stable cycles: This is the most important rule. It states that we use horizontal oriented wavelet patterns only. Look at this example:
On November 21, 2011, at the beginning of trading session we have a very bright wavelet Pattern #1 with vertical structure. It covers a variety of cycles, with periods in the range from 20 minutes to one hour. It means that here we still have a chaotic movement caused by a huge overnight drop. Its energy is distributed among many cycles with different periods.
Now look at Pattern #2. It has a horizontal structure, it is not so bright as the first one. However, the energy is concentrated on one cycle, with the period about 28 minutes. This cycle can be considered as useful for the forecast.
Also, you can see that the life time
of this pattern is two full cycles: 
,
i.e. this is a relatively stable cycle, and we can use it for forecasting.
Some financial instruments have no horizontal patterns, only vertical ones. To us, it means that we cannot apply the cyclical analysis for these instruments. Though some cycle can be very strong here, its life time is very short. The example of a non-cyclical financial instrument is S&P daily chart (not intraday); this is a wavelet for it:
You see there are only vertical patterns there. (For such financial instruments, we need to apply some other methods of analysis, not wavelets and not cycles.) A financial instrument, ideal for forecasting, should have more horizontal stripes (and the narrower they are, the better) than vertical ones.
Rule #2: cycles do not live forever. Sooner or later, the cycles stop working. Usually they live for 2-7 periods. It is not possible to use one and the same cycle forever. So you should always keep in mind that cyclic models have a limited forecast horizon.
Rule #3: do not play with Chaos. If a wavelet diagram does not show any regular horizontal patterns, it means that the stock market is not in a cyclic mode. Do not try to apply cycles based models for these periods. According to John Ehlers' study, the stock market is in the cyclic mode approximately 30% of its time. Cyclic models sometimes work and sometimes do not work. The wavelet diagram can serve as an alert system that reveals the cyclic activity inside financial data.
Now, when the rules are established, let us consider just one trading day, November 15, 2011. The trading session opens at 9:30. The trading instrument is S&P, and I have downloaded 10-second price chart for it (I would not take this financial instrument as an example for daily data due to the reason discussed above; we are OK with intraday data).
Below you will see what actually was going in real time. I use here the past tense, "was", just because you are reading this article now, and November 15th is in the past for you. However, I took the actual real time data (and screenshots), as my purpose is to show you how the wavelet diagram was changing when the new price information arrived. After reading and understanding this, you should be able to do wavelet analysis for other days and other financial instruments.
10:00 AM Nov 15 2011
Usually the cyclic picture for the first half an hour after the opening is too uncertain. We may say that the stock market accommodates the new information. This is the wavelet diagram calculated at 10:00 AM, half an hour after the opening:
There is a bright zone in the area of a very high frequency (1-10 minutes cycles); it is an illustration of what I have said in the previous sentence. These chaotic movements signify that His Majesty the Stock Market searches for an equilibrium point, for a correct tone to incorporate the incoming information. What else do we see here? Keep an eye on another bright zone that is around 20 min cycle; it is possible that a new cycle starts here. (Just watch it, do not do anything else yet.)
11:00 AM Nov 15 2011
This is the wavelet diagram one hour later:
Do you remember the very bright yellow zone in the high frequency area? It is gone now. And there is definitely some development for the cycle we were watching. On this new diagram, the very bright zone that corresponds to the cycle with the period of 29 min informs us that the cyclical phenomenon is very strong now. We may say that His Majesty the Stock Market has found the correct tone regarding the incoming information. The length of that zone is already 2 cycle periods (a little bit more). We can take it as a cyclic alert.
Now we have a cycle to work with. We would like to know how this cycle works for our financial data. It is very simply to do with Timing Solution's new module: just drag the mouse cursor over this bright zone; immediately you get the projection line for this cycle:
You see here the projection line (a red curve) together with the actual price data. Red zone is testing /out of sample interval, i.e. it is used to verify our model only and it was not use for any other calculations (it is a non-future leaks technology). As you see, the projection line forecast is not bad at all. In reality you will be able to recognize this cyclic bright zone earlier, because the program updates the wavelet diagram in real time. It is a spectacular view - to see how the cycles appear and then go away.
Noon AM Nov 15 2011
Here is the wavelet diagram for that moment:
It seems to me this cycle is still alive though it is losing its energy. This is an updated projection line (updated due to the new information received):
Though we still may use this cycle for modeling and forecasting, it is time to become cautious with it, it loses its energy and may disappear soon giving a stage to new players.
By the way, it is a great feature of wavelet analysis compared to other techniques: with the wavelet diagram, you are able to see when cycles lose their energy. It is a sign for you to look for another model.
1 PM Nov 15 2011
One more hour passed, and one more bright zone appeared on our wavelet diagram. It corresponds to 23 minutes cycle:
We do not consider the old 28 minutes cycle any more, it is not working now. So, when we drag the mouse cursor over the newly appeared bright zone, we get this forecast:
At that moment, the life time of this cycle is too short (one period only). We can watch it for a while (as we did with the previous cycle), or we can take some risk in the hope that it will live more.
I have chosen to risk and received the projection line that you see above.
2 PM Nov 15 2011
Guess what? My assumption was correct. This cycle is still alive, though its period is defined more correctly now, it is 21 min 30 sec.
So, drag the mouse cursor over this bright zone to cover the new coming price history. We get this projection line:
3 PM Nov 15 2011
One hour later it looks like there is no any cyclic activity now:
Our financial instrument is in non-cyclic mode now. And this is a good news for me as the trading day is going to its end...
Working with this module, I am more and more convinced that classical wavelet formula does not fit well for financial data. The most important disadvantage of classical algorithms is that they contain future leaks, and this becomes very dangerous when we apply them for real trading. That is why new algorithms have been developed (you can always switch to classical wavelet algorithms as well and compare the results).