"The
correlation as so far is a necessary palliative. Actually this is a big-big
problem for me how to estimate the effectiveness of the projection
line"
I'd like to present how I deal with this
issue, and my purpose is to present an alternative for evaluating forecasts.
In no way am I asking for something to be "fixed". I
also want to respond to your comment:
"I
believe that it makes sense to create a back test system that will simulate
the trader's behavior."
Here I'd like to show how I use
Forecasts in trading - that is, prior to "turning them over" to a
tech-based trading-system.
- This doesn't mean I'm saying this is
the ideal way, or that I'm recommending it. This works for me.
- I doubt if the process I use can
be automated
- If your goal is to create one
forecast from which you trade directly I'd recommend skipping this post
I don't use forecast correlations at
all. The forecasts I use on a regular basis I developed two years ago by
manually reviewing ("eyeballing") the forecast - I zoom in no
more then two months on the X axis so I can clearly see the relationship of
the forecast line to the out-of-sample prices. (Summer 2005 I added an ADX
(trend strength) forecast to my arsenal, that won't be coverd here)
I am looking for two things:
- is the forecast calling the
turning-points within a couple of days
- by relative comparison to other
forecasted swings, is the forecast identifying when larger swings will occur
- I am not looking for anything else!
If it looks good, then I review the
exact same forecast on different time periods of the same market
(cross-validation testing). What that means is I use TMIN and TMAX to select
different years to load, then rerun the forecast using the same length
training interval, target, and criteria. Then of course I manually review
again. Then I select another financial market.
While this is a slow process, and
somewhat subjective, when you find a forecast that passes all your tests you
can re-use it for years.
Hint: if you're using an RPO and you'd
like to try this approach out, DO NOT use a '1' as the first parm of the RPO.
Using '1' is best if your aim is to find a day-to-day high correlation.
The next two images below show both why
I ignore correlation and how I manually evaluate. This first image had a low
correlation - yet its picking the turning-points just fine. My guess is the
low correlation reading is for two reasons, the first being the sideways price
action in the last week of May and the second, and main reason being the
lack of upwards amplitude during the correction in early June. Yet to my eye
this is fine, to yours it may be unusable, and to correlation it was a pretty
bad forecast.
The second reason I ignore correlations is
that Inversions happen in all astro-sourced forecasts. Obviously an inversion
will wreak havoc on a correlation reading. Its an out-of-scope topic that
happens. While Inversions obviously aren't ideal, here's whats good about them
for me:
- since I use a tech-based system to
trade them, I rarely get filled. For example, the below forecast has an
upswing starting around June 18. As there was no upswing and I use a
"price-confirmation" entry technique (ie price has to be rising thru
a SwingLine) I don't get filled, hence no loss
- At the close of trading on June 21 I
know I'm most likely in an inversion. I have a choice now - I can remove the
"Long Only" filter on my tech-based system and go short and/or I can
wait for the next turning point in the last week of March
- its noteworhty that the forecast did
accurately call the start and end of the swing; to some an inversion is a bad
thing; to others it still gives the start and end times of swings, a good
thing; it may also provide the relative amplitude of the anticipated swing,
another good thing
Here's a forecast with a negative
correlation, it would be dismissed immediately as unusable.
But if I zoom in on the first month
it looks different. How much data past the LBC you use is relevant in the
correlation calc.
In the same forecast, zooming in on
June, the two circled areas show why the correlation reading may be poor. The
forecast is just not tracking day-by-day. Also look at June 21. The High
of the day correlated with an upswing, the Close and Low didn't. For
correlation purposes, which is right? If my objective in using a
forecast:
- has nothing to with tracking
price day-by-day
- but rather with figuring out the
approximate start and end days of swings and their relative amplitudes
Then I'm happy (possibly ecstatic) with
this forecast.
The forecast I use most is usally off a few
days in the exact start and end times of swings.
So I always run other models whose only
purpose is to tell me about non-directional turning-points. Non-directional
means I don't care if the market moves up or down before or after the
trend-change date. What I care about is:
- if the market was moving up I expect it
move down after the trend change date
- if the market was moving down I expect
it move up after the trend change date
- beforehand I have no directional bias
Some people have a very hard time
understanding "non-directional".
Below is an image of one of my turning-point
models for financial markets. Its run in ULE - it does not
do any training, it just plots a vertical grey line when the event is due.
The EfficiencyTest does a fine job of evalating turning-point events - llok
for a "V" shape around culmination. If you have MT's Statistic
Expert you can evaluate a few thousand events to see if they are useful in
predicting trend-change times (in Step 3 check only "Mixed")
As you can see most each vertical line
coincides with a trend-change in the market (I call the below model
"Lunar Occultation Index"). What I do is wait for when a
forecasted swing is predicted and then "fine-tune" the anticipated
start of that swing with turning-point models like below (then i turn it
over to the tech-based trading-system).
Below is another trend-change model for
the EurUsd (shown below is the dollar index).
Note that the only thing it forecasts is
the time of anticipated trend-changes. Some are up, others are down
(no-directional turning-points). To some "non-directional" makes it
unusable, to others its incredibly valuable information.
If what you're looking for is one
forecast model that you can trade directly from, and you're attempting to
find that model based on high correlation readings, it may take you a
long time to find it! In the mean time, you may already have at your disposal
techniques that allow you trade profitably, but may not recognize these models
because of the way you're evaluating them.
It may be worthwhile to:
- manually review forecasts to see
if you're catching the start and end of swings; if you're missing short
counter-moves the correlation of that model will be low, but you have to
seriously consider asking yourself this question: "If
I know the start and end days of large market swings within a couple of
days - is there any way, any way at all, that I could put that
information to use?".
- create separate models whose only
purpose is to provide the dates of anticipated trend-changes
- find a simple technical-analysis based
swing-trading system that confirms your forecasted swing has occured, sets an
initial stop-loss, and uses a trailing-stop to capure the bulk of the swing if
it occurs
