Professional Documents
Culture Documents
@reds i think the current script would not be of much use for what you are asking for. I believe you
probably need to rank the correlations ( StaticVarGenarateRanks), and then for that output that you
are looking for perhaps using AddMultiTextColumn would produce the result. I have not attempted that
as it is a big job and I am too busy. But if you are able then please come back and share with us here
on the forum. Good luck.
Exploration with sequential watchlists, or exploration with ratios that use one security as nu…
Thank you Larry for the suggestions. I will post back if I am able to find a solution but it may be out of
my limited abilities…Thanks…Mike
@reds , this can be done by storing the corr. table to matrix at the end of exploration (using matrix
functions). The sorted top N get picked from that matrix and output via Addrow.
It is a bit tricky (but not difficult). I’m saying tricky because you have to think a little bit.
Here is proof that it is possible to do that (output of sorted table (from largest to lowest together with
standard correlation table below of it). As aside Addrow does not offer cell coloring (yet). I have output
all but of course you can pick just a top N via MxGetBlock also.
Here is another example just outputting overall top five pairs + corr.
https://forum.amibroker.com/t/correlation-matrix/1184/32 1/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
BTW, instead of using Foreign for default correlation table it is much much faster using StaticVarAdd to
store second array of Correlation function. In fact that method is lightning fast compared to using
Foreign. Milliseconds per each exploration run.
function _TRACE_(msg) {
https://forum.amibroker.com/t/correlation-matrix/1184/32 2/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
if (debugTrace)
_TRACE(msg);
}
actionStatus = Status("action");
if (( actionStatus == actionExplore ) OR (actionStatus == actionBacktest)) {
symlist = GetCategorySymbols( categoryWatchlist, wlNumber);
Filter = 0;
stockNum = Status("stocknum");
if (stockNum == 0)
{
// Do this only ONCE
if (symList != "") {
EnableTextOutput( 0 );
size = 1 + StrCount( symlist, ",");
if (maxColRes >= size) {
maxColRes = size-1;
}
if (maxColRes <= 0) {
maxColRes = size-1;
}
if (maxColRes >= MAX_PAIRS_IN_TABLE) {
maxColRes = MAX_PAIRS_IN_TABLE; //
}
_TRACE_("Coupling elements");
// Print result in the Analisys window using AddRow (to output too
for( row = 0; row < size; row++ ) {
} else {
_TRACE_("Too many tickers. Skipping");
}
} else {
_TRACE_("Empty watchlist. Skipping");
}
}
_TRACE_("Script completed for stock " + WriteVal(stockNum, 4.0));
} else {
_TRACE_("Script ignored. Status " + WriteVal(actionStatus, 1.0));
}
(In this sample I applied the correlation logic used by @portfoliobuilder - using only the last value of
the correlation arrays; you may want to change it as needed).
I use a matrix to store the correlations and then sort it multiple times; one for each ticker row - using the
first row of my original unsorted matrix as an index for the tickers to display in the correlation table.
Seems to work well enough for medium/sized watchlists (I used a max of 500 tickers)
To display the resulting data in the Analysis window I used the AddRow() function with the filter set to 0
(search here in the forum other examples about its usage). Unfortunately as said also in the previous
message no colors…
For the exploration, you have to choose a watchlist (selecting its corresponding number as a
parameter).
In order to get the table a little faster, I suggest setting the analysis filter to “current” (it is actually
ignored but the process will be done only once).
This sample raw code (where I left on purpose many commented lines to show how some code was
developed/moved/replaced) is provided “as it is” (it is not fully tested); take it mainly as a starting point
for your own further development.
P.S. Note that all the relevant code happens when the Status(“stocknum”) == 0 so it is run in a SINGLE
THREAD (I did it on purpose to be able to easily follow the code logic tracing/debugging).
Portfoliobuilder, fxshrat, and beppe…Thank you for all you recommendations and direction! I will
consider all and move forward…Happy New Year! Best…Mike
https://forum.amibroker.com/t/correlation-matrix/1184/32 5/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
osinv Jan 19
(1) Why are the correlation values on both sides of the diagonal sometimes different for the same pair of
symbols? Which correlation is then the good one?
(2) If we want to do a correlation analysis using the weekly or monthly returns instead of the daily
returns, would changing the "periodicity" in the settings window be the right way to do it? (Assuming that
we also change the AFL number of bars lookback also obviously)
Thanks,
Oscar
aknagdev Jul 3
ArnaudP 5d
I would like to seek your valuable help in resolving my challenge. I'm trying to sum up the correlation of
each and every symbol for later use in a ranking scheme.
Having browsed through the entire forum it seems there is a discussion about this topic on the old
yahoo group but I can't access it... I'm able to sum up the correlation in exploration mode but can't seem
to be able to store them into a static variable which is pretty frustrating
I attempted several ways but nothing I've tried seems to work.. the values I'm after are highlighted in
yellow on the screenshot below.
ArnaudP.
https://forum.amibroker.com/t/correlation-matrix/1184/32 6/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
beppe Regular 5d
@ArnaudP do you want to store each total in a single staticVar with a name based on the column
header (for instance "CorrelationMatrixTotal_APPL", "CorrelationMatrixTotal_AXP", ...,
"CorrelationMatrixTotal_TRV") or in some other ways?
Moreover, do you want to include in the totals also the "autocorrelation" value (1.0) as displayed in the
screenshot, probably done using a AddSummaryRows(1, 1.2) line?
mradtke 5d
If you've calculated all the correlations and summed them up, then the hard word is already done. What
issue are you having with Static Variables? Could you post some code to help us help you find your
error?
ArnaudP 5d
@beppe & @mradtke . I appreciate that two legends of the amibroker community are looking into my
issue
@beppe , yes this is exactly what I intend to do, I would like to have the ability to retrieve the ColSum
correlation of each asset to add this component to a bigger weighting scheme. Basically what I'm trying
to achieve is the FAA (flexible asset allocation) scheme as detailed in the following SSRN paper:
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2193735
While I have no problem coding it with R (as per this example:
http://systematicinvestor.github.io/strategy/Strategy-FAA )
I encounter an issue when it comes to getting the sum of the correlation to include to my three
components weighting scheme.
And yes you are right I used AddSummaryRows to obtain the top line on my screenshot.
@mradtke the code I generated is essentially the same as the one posted by portfolio builder in the
same post, my only trouble is to generate & store the cummulated correlation values of each asset.
I know it may sound trivial but I'm only beginning my journey with AFL and facing my first roadblock -
I intend to share the full FAA implementation to this forum (which is quite close to GTAA by
trendXplorer) once successfully succesffuly coded.
Thanks !
ArnaudP
mradtke 5d
We don't need the full FAA code at this point. Just a simplified version that shows how you're calculating
the sum of correlations and attempting to store them in static variables. Without seeing your code, we
really have no way to point you in the right direction other than to implement our own solution from
scratch. That's not a good use of our time nor the best way for you to improve your AFL skills.
fxshrat 5d
https://forum.amibroker.com/t/correlation-matrix/1184/32 7/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
AddSummaryRows( 1, 1.5 );
_TRACE( symlist );
https://forum.amibroker.com/t/correlation-matrix/1184/32 8/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
ArnaudP 4d
@fxshrat Thanks a lot this is definitely very helpul. Hopefully it will be of some use for other users too.
Cheers
fxshrat 4d
Here is modification to move the whole matrix part outside of exploration since (I suppose) you want to
use it elsewhere...
Copy
https://forum.amibroker.com/t/correlation-matrix/1184/32 9/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
Tickers = Name();
AddSummaryRows( 1, 1.5 );
/// added by fxshrat to store corr sum to "summary row" matrix vector
/// called outside after exploration being finished, for further use cases...
//if ( stocknum == 0 )
{
_TRACE( symlist );
TrendXplorer 22h
After contacting @ArnaudP in private, he asked me to share the FAA code I have for monthly data.
https://forum.amibroker.com/t/correlation-matrix/1184/32 10/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
Please do comment on possible improvements. I am especially interested how to modify the code to
work with daily data while (still) observing the exact monthly endpoints (not being i.e. 4 x 21).
Importantly: this code is not to correct the matrix approach shared by @fxshrat . It's just another way,
originating from the era when AmiBroker did not have build-in matrix functions.
In the below code the correlation matrix is calculated in two different ways. For Explorations, the code
uses a loop in similar fashion to the earlier shared approached. For the rotational calculations to
determine the monthly position sizes, the code uses a custom function CalcPfC( symbol, tickerlist,
length ).
Acknowledgement goes to the old Yahoo board member SanzProphet, who kindly helped me along with
this code back in 2014.
Running the code on the N10 investment universe and the separate out of market fund as shown on the
screenshot with TopSize=10, results in the following long term performance chart:
Point to note: next to the filterlist with both all investment assets as well as the out of market fund, the
code needs a dedicated watchlist with only the investment assets. Otherwise the correlation
calculations are off.
Copy
///////////////////////////////////////////////////////////////////////////////////
//
// --- FAA_Dual_Monthly_v1.1.afl ---
//
// --- introduction ---
//
// based on:
// "Generalized Momentum and Flexible Asset Allocation (FAA), An Heuristic Approach
// Keller and Van Putten, 2012
// FAA Electronic copy available at: http://ssrn.com/abstract=2193735
//
// FAA strategy rules:
https://forum.amibroker.com/t/correlation-matrix/1184/32 11/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
CorSum = 0;
if ( tickerlist != "" )
{
for ( j = 0; ( ticker = StrExtract( tickerlist, j ) ) != ""; j++ )
{
data1 = Foreign( ticker, "C" );
data2 = Foreign( symbol, "C" );
log1 = log( data1 / Ref( data1, -1 ) );
log2 = log( data2 / Ref( data2, -1 ) );
CorSum = CorSum + Correlation( log1, log2, Length );
}
}
// take out the diagonal of the correlation matrix and average
PfC = ( CorSum - 1 ) / ( j - 1 );
return ( PfC );
}
StaticVarRemove( "Cor*" );
StaticVarRemove( "MVC*" );
StaticVarRemove( "Pos*" );
StaticVarRemove( "Sum*" );
StaticVarRemove( "num*" );
StaticVarRemove( "cas*" );
RestorePriceArrays();
if ( symbol == _Cashticker )
{
CashMom = Mom;
Mom = Vol = Cor = Null;
PosSize = IIf( RankMVC <= TopSize AND Mom > 0, 100 / TopSize, 0 );
SetForeign ( symbol );
RestorePriceArrays();
if ( symbol == _CashTicker )
{
AddColumn( CashMom, "Momentum", 3.3, 1, ColorMom );
}
else
{
AddColumn( Mom , "Momentum", 3.3, 1, ColorMom );
}
AddColumn( Vol , "Volatility" , 1.3 );
AddColumn( Cor , "Portf.Cor" , 1.3 );
///////////////////////////////////////////////////////////////////////////////////
//
// --- end of code ---
//
///////////////////////////////////////////////////////////////////////////////////
portfoliobuilder 21h
@TrendXplorer thanks for sharing your code JW. I was uncertain what you were looking for with this
line,
TrendXplorer:
I am especially interested how to modify the code to work with daily data while (still) observing the
exact monthly endpoints (not being i.e. 4 x 21)
I thought perhaps you were looking for some examples of using different time frame data (in this
example daily/weekly/monthly) together. So here is a small example with a couple of different methods
of using the longer timeframe for display on the shorter time frame.
Copy
TimeFrameSet(inWeekly);
WeeklyROC26=ROC(C, 26);
TimeFrameRestore(); // restore time frame to original
TimeFrameSet( inMonthly);
https://forum.amibroker.com/t/correlation-matrix/1184/32 17/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
MonthlyROC6= ROC(C,6);
TimeFrameRestore(); // restore time frame to original
// Explore to debug our variables and illustrate what the calculations are doing
Filter=1;
dynamic_fg_Color = IIf(eow, colorWhite, colorDefault);
dynamic_Bk_Color = IIf(eow, colorLightBlue, colorDefault);
dynamic_fg_ColorM = IIf(eom, colorWhite, colorDefault);
dynamic_Bk_ColorM = IIf(eom, colorBlue, colorDefault);
AddColumn(eow,"End Of Week", 1.0, dynamic_fg_Color, dynamic_Bk_Color);
AddColumn(eom,"End Of Month", 1.0, dynamic_fg_ColorM, dynamic_Bk_ColorM);
https://forum.amibroker.com/t/correlation-matrix/1184/32 18/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
I am not sure that is what you were interested in but I hope it helps.
TrendXplorer 6h
While thinking about how to explain to @portfoliobuilder what I actually was looking for, I managed to
find a partial solution (ROC solved below, but Volatility and Correlation not: array to number issue).
For running strategies like FAA with monthly lookback settings on the daily time frame I wanted to get
same end-of-the-month readings when comparing daily ROC's against monthly ROC's. Since the
number of trading days typically changes each month, using i.e. 6-months times 21 days for a
ROC(126), the reading won't always match with a 6-month ROC. This is what I meant with "preserving
monthly endpoints".
Copy
https://forum.amibroker.com/t/correlation-matrix/1184/32 19/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
CountDays = Cum( 1 );
LastDayCount = ValueWhen( MonthEnd, CountDays, 1 );
FirstDayCount = ValueWhen( MonthEnd, CountDays, length + 1 );
DaysInPeriod = LastDayCount - FirstDayCount;
Filter = MonthEnd;
AddColumn( ChangeOverDays , "Daily ROC% over " + NumToStr( length, 1.0 ) + " mon
AddColumn( DaysInPeriod , "Days in " + NumToStr( length, 1.0 ) + "-month period"
AddColumn( ChangeOverMonths, "Monthly ROC% over " + NumToStr( length, 1.0 ) + " mon
AddColumn( match , "Match", 1.0, 1, matchColor
However, for calculating i.e. the (exact) 6-month volatilities and correlations on daily periodicity, the non-
stationary number of days in that period are required. Since both functions require a number instead of
an array as period input, the above used DaysInPeriod array can't be used as such and returns an
error:
Perhaps someone could show how to extract the appropriate (changing) number of days from the array
into a scalar?
portfoliobuilder 4h
@TrendXplorer for your volatility calculation I believe you are using Standard Deviation and that
indicator can already accept an Array as well as a Scalar.
http://www.amibroker.com/guide/a_varperiods.html
https://forum.amibroker.com/t/correlation-matrix/1184/32 20/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
If you need some function that is not on that list, you can usually easily implement variable-period
version using existing functions, for example:
return 100 * pa / ( pa + na );
}
https://forum.amibroker.com/t/correlation-matrix/1184/32 21/22
8/19/2018 Correlation Matrix - AFL Programming - AmiBroker Community Forum
x = 100*(Close - ll)/(hh-ll);
Doing so typically requires knowledge of how indicator works internally. Although it is good idea to
know how indicators work to build the system that is based on their inner workings, and it is really
better to use above strict implementations, sometimes users urge to have “quick” solution…
So here it is brute force solution to “any indicator with variable period” problem. It allows to
transform constant period functions into variable period functions. It does that by calculating
constant period indicator N times where N is a number of different “period” values used in the
calling code. It is not very efficient but not bad as a quick hack.
Copy
result = Null;
return result;
}
Otherwise, as you have probably read over many different posts in the forum, if you want to process
things bar by bar, you need to write explicit loop and then loop counter gives your current bar inside the
loop. A variable would return a number instead of array if outputting an element of the array.
I tried using the built-in functions returning element of an array -- SelectedValue(), LastValue(),
BeginValue(), EndValue() , but unable to find a solution so far.
https://forum.amibroker.com/t/correlation-matrix/1184/32 22/22