Thursday, February 6, 2025
HomeForexMission Inconceivable: MetaTrader 5 doesn't help testing and optimization of buying and...

Mission Inconceivable: MetaTrader 5 doesn’t help testing and optimization of buying and selling robots primarily based on renko indicators – Different – 26 February 2024


This can be a continuation of a collection of blogposts about buying and selling by alerts of renko charts. Earlier than this level we have now mentioned many points of utilizing customized symbols for renko implementation. The newest findings are described within the article Use of customized tick modeling to stop illusory grail-like backtests of Renko-driven buying and selling robots.

Probably the most apparent questions on this collection is why can we use customized symbols for Renko implementation slightly than one thing else. In reality, the “one thing else” could be very restricted. Drawing Renko on a canvas or utilizing graphical objects are impractical, as a result of it doesn’t enable for backtesting. It appears rather more pure to make use of Renko indicators as an alternative.

Sadly, indicators in MetaTrader 5 are designed in such a method that it is unattainable to work with Renko indicators within the tester. And right here is why.

Renko indicator

To start out our analysis we want an indicator that calculates graphical collection which appear to be Renko containers. Let’s don’t invent the wheel and take certainly one of present indicators, for instance, Blue Renko Bars.

Because it turned out, this program required to make some bug fixes and enhancements, most necessary of which we’ll clarify one after the other. The ultimate modification is hooked up to the submit.

Because the drawing kind of the plot is DRAW_CANDLES, the unique directive indicator_buffers is inaccurate, as a result of this sort doesn’t help extra buffer for colours (versus DRAW_COLOR_CANDLES, which makes use of 4+1 buffers).

#property indicator_separate_window
#property indicator_buffers 4         
#property indicator_plots   1

DRAW_CANDLES requires 4 buffers. The buffer array brickColors is ineffective and has been eliminated in every single place.

double brickColors[];

The opposite buffer arrays are initialized at start-up:

int OnCalculate(const int rates_total,      
                const int prev_calculated,  
                ...)
{
   ...
   if(prev_calculated == 0)
   {
      ArrayInitialize(OpenBuffer, 0);
      ArrayInitialize(HighBuffer, 0);
      ArrayInitialize(LowBuffer, 0);
      ArrayInitialize(CloseBuffer, 0);
   }
   ...
}

We have now launched new variable lastBar to detect formation of recent bar on the host chart. At these moments we have to initialize simply added buffer components (underneath sure situations).

datetime lastBar;
   
int OnCalculate(const int rates_total,
                const int prev_calculated,
                const datetime &time[],...)
{
   ...
   if(lastBar != time[0])
   {
      OpenBuffer[0] = 0;
      HighBuffer[0] = 0;
      LowBuffer[0] = 0;
      CloseBuffer[0] = 0;
   }
   
   lastBar = time[0];
   ...
}

The counter of obtainable renko containers in renkoBuffer array of MqlRates was not dealt with appropriately for all conditions and will produce “out of sure” exception (the indicator can be stopped and unloaded).

int OnCalculate(const int rates_total,      
                const int prev_calculated,  
                ...)
{
   
   int measurement = ArraySize(renkoBuffer);
   
   ... 
   
   int first;
   if(prev_calculated == 0) 
   {
      ...
      first = (rates_total > measurement) ? measurement : rates_total; 
   }
   else
   {
      
      
      
      first = measurement;
   }
   
   for(int i = first - 2; i >= 0; i--)
   {
      ... 
      HighBuffer[shift + i + 1] = renkoBuffer[i].excessive;
      LowBuffer[shift + i + 1] = renkoBuffer[i].low;
      ...
   }
}

Within the operate RenkoAdd which provides new field to the renkoBuffer we modified the precept of the operation: as an alternative of heavy ArrayCopy, we wrap the decision to ArrayResize with two calls to ArraySetAsSeries.

void RenkoAdd()
{
   int measurement = ArraySize(renkoBuffer);
   ArraySetAsSeries(renkoBuffer, false);       
   ArrayResize(renkoBuffer, measurement + 1, 10000);
   ArraySetAsSeries(renkoBuffer, true);        
                                               
   ...
}

Additionally the brand new ingredient is initilized by empty struct.

void RenkoAdd()
{
   ...
   const static MqlRates zero = {};
   renkoBuffer[0] = zero;
}

Try N1 (ChartSetSymbolPeriod)

Now allow us to recall a bit how indicators work and what this implies for the renko containers.

When a brand new bar is added to the chart, the charges and regular indicators (if utilized) are shifted to the left by 1 bar, however the renko containers ought to keep nonetheless (as a result of new containers seem by their very own “schedule”). However, when a brand new field is generated, we have to shift all earlier containers to the left, however charges and different indicators stay on the similar place.

It is necessary to notice that any renko indicator should handle these issues.

To unravel this desynchronization this indicator reserves a variable RedrawChart (which isn’t even an enter parameter) holding various bars to redraw. By default it is 1 and is substituted by CHART_VISIBLE_BARS. In consequence, renko is right solely on the final CHART_VISIBLE_BARS bars. Furthermore, ChartGetInteger(0, CHART_VISIBLE_BARS, 0) returns at all times 0 bars whereas testing/optimizing with out visible mode! This answer is partial and never common, doubtlessly resulting in miscalculations, if utilized in automated buying and selling.

Particularly, it is flawed within the following facet. Many buying and selling methods use a mix of indicators to generate buying and selling alerts. For instance, throughout the collection of the blogposts we have now been utilizing a easy take a look at technique on 2 MAs crossing on high of renko. To implement it with renko indicator we have to apply MAs to the renko indicator.

And right here is the issue: indicators in MetaTrader, calculated as soon as on all bars, are then re-calculated in “quick circuit” method – solely on the most recent bar. Even when you replace CHART_VISIBLE_BARS bars within the renko indicator, the MAs (or different indicators) utilized on high of the renko, will replace solely on the most recent bar. In consequence, it is unattainable to get right crossing of MAs.

To beat the issue we have added a brand new function to the renko indicator. After new bar creation or after new renko field formation we request the chart to replace fully, together with all moreover utilized indicators. For that puspose the calls to ChartSetSymbolPeriod are added into 2 locations: OnCalculate and RenkoAdd.

int OnCalculate(const int rates_total,
                const int prev_calculated,
                const datetime &time[],...)
{
   ...
   if(lastBar != time[0])
   {
      ...
      ChartSetSymbolPeriod(0, _Symbol, _Period);
   }
   
   lastBar = time[0];
   ...
}
   
void RenkoAdd()
{
   ...
   if(lastBar)
   {
     ChartSetSymbolPeriod(0, _Symbol, _Period);
   }
}

Now the MAs are correctly up to date in sync with renko re-positioning.

Renko indicator with applied MA with period 1 (to make sure it works by close prices)

Renko indicator with utilized MA with interval 1 (to ensure it really works by shut costs, CHLO see under)

OHLC -> CHLO

But there’s one other small drawback. The renko is represented as 4 buffers with Open, Excessive, Low, and Shut costs of the candles. When a further indicator is utilized to a different indicator, it makes use of the very first buffer. Therefore our 2 MAs are utilized on Open costs, which isn’t usually a desired impact within the case of renko-based technique. As a substitute, the MAs needs to be utilized to the Shut costs of the renko containers. To take action we have to change Open and Shut buffers within the renko indicator.

The brand new mode is switched on or off by new parameter SwapOpenClose.

enter bool SwapOpenClose = false; 
   
int OnInit()
{
   ...
   if(SwapOpenClose) PlotIndexSetString(0, PLOT_LABEL, "Shut;Excessive;Low;Open");
   ...
}
   
int OnCalculate(...)
{
   ...
   for(int i = first - 2; i >= 0; i--)
   {
      OpenBuffer[i + 1] = SwapOpenClose ? renkoBuffer[i].shut : renkoBuffer[i].open;
      HighBuffer[i + 1] = renkoBuffer[i].excessive;
      LowBuffer[i + 1] = renkoBuffer[i].low;
      CloseBuffer[i + 1] = SwapOpenClose ? renkoBuffer[i].open : renkoBuffer[i].shut;
      ...
   }
}

This appears to be like like a completed renko indicator appropriate for buying and selling automation. This can be a deception, however we’ll uncover this a bit later, and can attempt to add different options to get it to work as anticipated.

Professional adviser primarily based on 2 MAs crossing on the renko indicator

Presently let’s attempt to adapt our take a look at EA – MA2Cross, initially utilizing renko customized symbols – for working with the renko indicator. The modified model has the identify MA2CrossInd.mq5.

Enter paramaters are added for underlying indicator:

enter int  BrickSize    = 100;    
enter bool ShowWicks    = true;   
enter bool TotalBars    = false;  
enter bool SwapOpenClose = true;  

The indicator with the given parameters is created in OnInit, and its deal with is handed to the sign filter as an alternative of former Signal_2MACross_MAPrice parameter (really it was Shut value on a regular basis).

int OnInit()
{
  ...
  const int deal with = iCustom(_Symbol, _Period, "Blue Renko Bars", BrickSize, ShowWicks, TotalBars, SwapOpenClose);
  if(deal with == INVALID_HANDLE)
  {
    Print("Cannot create indicator, ", _LastError);
    return(INIT_FAILED);
  }
  ChartIndicatorAdd(0, 1, deal with);
  ...
  filter0.MAPrice((ENUM_APPLIED_PRICE)deal with); 
  ...
}

This program might really commerce on a web based chart! Nevertheless it cannot be backtested and optimized!

The explanation for it’s because the operate ChartSetSymbolPeriod shouldn’t be working within the tester. In consequence, 2 MAs will not be recalculated correctly and provides incoherent alerts.

What can we do?

Try N2 (PLOT_SHIFT)

Considered one of concepts was to implement the renko indicator with one other precept of re-positioning renko containers in opposition to common bars. It is primarily based on the property PLOT_SHIFT.

As you already know we will shift visible illustration of indicator’s buffer to the fitting or to the left, relying from the property: constructive values transfer curves to the fitting for the required variety of bars, whereas adverse values transfer them to the left. So, when a brand new common bar is created, we will apply the shift +1 to our renko containers to maintain them visually on the identical place. And when a brand new renko field is added, we will apply the shift -1 to maneuver previous containers to the left, preserving alignment with the final common bar.

As a result of we do not know beforehand the path wherein future value will shift out graph extra, we have to make a reserve of empty invisible containers on the proper facet. The reserve is offered in corresponding enter parameter and used to initialize a variable with present shift.

enter int Reserve = 0;
   
int shift;
   
int OnInit()
{
   ...
   shift = Reserve;
   PlotIndexSetInteger(0, PLOT_SHIFT, shift);
   ...
}

Then in OnCalculate improve the shift on new bars. Nonzero Reserve can also be used as a flag to allow the brand new mode.

int OnCalculate(...)
{
   ...
   if(lastBar != time[0]) 
   {
     if(!Reserve)
     {
       ChartSetSymbolPeriod(0, _Symbol, _Period);
     }
     else
     {
       PlotIndexSetInteger(0, PLOT_SHIFT, ++shift);
       Remark("++", shift);
       OpenBuffer[0] = 0;
       HighBuffer[0] = 0;
       LowBuffer[0] = 0;
       CloseBuffer[0] = 0;
     }
   }
   ...
}

Additionally lower the shift on new containers in RenkoAdd.

void RenkoAdd()
{
   ...
   if(!Reserve)
   {
      ChartSetSymbolPeriod(0, _Symbol, _Period);
   }
   else
   {
      PlotIndexSetInteger(0, PLOT_SHIFT, --shift);
      Remark("--", shift);
   }
}

In fact, the shift have to be used to regulate indices throughout writing information into the buffers.

int OnCalculate(...)
{
   ...
   for(int i = first - 2; i >= 0; i--)
   {
      OpenBuffer[shift + i + 1] = SwapOpenClose ? renkoBuffer[i].shut : renkoBuffer[i].open;
      HighBuffer[shift + i + 1] = renkoBuffer[i].excessive;
      LowBuffer[shift + i + 1] = renkoBuffer[i].low;
      CloseBuffer[shift + i + 1] = SwapOpenClose ? renkoBuffer[i].open : renkoBuffer[i].shut;
      
      if(i == 0) 
      {
         OpenBuffer[shift + i] = SwapOpenClose ? shut[i] : renkoBuffer[i].shut;
         HighBuffer[shift + i] = upWick ? upWick : MathMax(renkoBuffer[i].shut, renkoBuffer[i].open);
         LowBuffer[shift + i] = downWick ? downWick : MathMin(renkoBuffer[i].shut, renkoBuffer[i].open);
         CloseBuffer[shift + i] = SwapOpenClose ? renkoBuffer[i].shut : shut[i];
      }
   }
}

Sadly, even though the brand new strategy is working excellent visually, the information shift shouldn’t be detected correctly by indicators utilized on high of the renko.

This can be a identified common limitation of MetaTrader platform. The property PLOT_SHIFT cannot be detected outdoors an indicator, and when OnCalculate is known as in dependent indicators, they obtain and course of information unshifted. Let’s remind you ways the quick type of OnCalculate appears to be like like:

int OnCalculate(const int rates_total,
                const int prev_calculated,
                const int start,
                const double &information[]);

You’ll be able to see right here that indicator receives related and considerably associated property PLOT_DRAW_BEGIN, which is handed through start parameter. However there’s nothing telling that information array makes use of shifted indexing. The one strategy to overcome that is to enter the shift to all indicators as enter. However in our case we alter the shift dynamically in renko, and therefore it is unattainable to regulate the shifts in MAs on the fly (except you re-implement all required indicators your self and ship the shift through chart occasions, or world variables, or one thing else – that is too costly).

Try N3 (say “no” to rates_total)

Yet another strategy to attempt which involves thoughts supposes to not replace renko containers in any respect. Simply pile them at first of the tester time and return precise quantity from OnCalculate, as an alternative of rates_total.

We are able to use adverse Reserve worth as a flag to allow this mode. When it is on, buffer indices needs to be mapped into the vary [0..size], which is completed by adjusting shift variable.

int OnCalculate(...)
{
   ...
   if(Reserve < 0)
   {
     shift = rates_total - measurement;
     if(shift < 0) Print("Renko buffer overflow, will terminate...");
   }
   
   for(int i = first - 2; i >= 0; i--)
   {
      OpenBuffer[shift + i + 1] = SwapOpenClose ? renkoBuffer[i].shut : renkoBuffer[i].open;
      HighBuffer[shift + i + 1] = renkoBuffer[i].excessive;
      LowBuffer[shift + i + 1] = renkoBuffer[i].low;
      CloseBuffer[shift + i + 1] = SwapOpenClose ? renkoBuffer[i].open : renkoBuffer[i].shut;
      ...
   }
   ...
   return(Reserve < 0 ? measurement + 1 : rates_total);
}

As well as, all calls to vary PLOT_SHIFT property needs to be wrapped into acceptable guard situations:

int OnInit()
{
   ...
   if(Reserve > 0)
   {
     shift = Reserve;
     PlotIndexSetInteger(0, PLOT_SHIFT, shift);
   }
   ...
}
   
int OnCalculate(...)
{
   ...
   if(Reserve > 0)
   {
      PlotIndexSetInteger(0, PLOT_SHIFT, ++shift);
      Remark("++", shift);
   }
   ...
}
   
void RenkoAdd()
{
   ...
   else if(Reserve > 0)
   {
      PlotIndexSetInteger(0, PLOT_SHIFT, --shift);
      Remark("--", shift);
   }
}

The indicator is calculated and displayed correctly on this mode whereas working within the visible tester (bear in mind, it’s worthwhile to scroll to the start of the chart to see the containers). However that is one other deception.

If it will work as anticipated, we would want to vary the sign era module Signal2MACross.mqh barely by including the next methodology (see hooked up Signal2MACrossDEMA.mqh):

class Signal2MACross : public CExpertSignal
{
   ...
   int StartIndex(void) override
   {
      const int base = iBars(_Symbol, PERIOD_CURRENT) - ::BarsCalculated(m_type);
      return((m_every_tick ? base : base + 1));    
   }
}

Right here m_type is a kind of value on which MA indicators are utilized, and we will assign a deal with of the renko indicator to it, because it was talked about earlier than:

  ...
  const int deal with = iCustom(_Symbol, _Period, "Blue Renko Bars", BrickSize, ShowWicks, TotalBars, SwapOpenClose);
  ...
  filter0.MAPrice((ENUM_APPLIED_PRICE)deal with);
  ...

This fashion the renko shut value from our EA can be used for MA calculations.

Sadly, all of this isn’t working as a result of the worth returned from BarsCalculated is at all times rates_total, not precise worth returned from OnCalculate.

We’re attaching a barely modified instance of DEMA indicator (DEMAtest.mq5, to be positioned in MQL5/Indicators/Examples/), which lets you hint and output precise OnCalculate’s parameters obtained by the indicator, when it is utilized on the renko indicator with diminished variety of calculated bars (containers). This indicator can also be used within the Signal2MACrossDEMA.

int OnCalculate(const int rates_total,
                const int prev_calculated,
                const int start,
                const double &value[])
{
   const int restrict = (_AppliedTo >= 10) ? BarsCalculated(_AppliedTo) : rates_total; 
   ...
   #ifdef _DEBUG
   Print(rates_total, " ", prev_calculated, " ", restrict, " ", start);
   #endif
   ...
}

You’ll be able to guarantee that the variety of out there bars is at all times reported as rates_total, and because of this – the sign module above will learn information at incorrect indices.

I take into account this a bug of the platform, as a result of it is passing right values through start parameter, however not through rates_total.

Backside line

On the time of writing, it is unattainable in MetaTrader 5 to run a backtest or optimization of EA which is predicated on alerts of a renko indicator. There are some workarounds out there.

You should utilize customized symbols with renko, because it’s described in my earlier blogposts.

You’ll be able to calculate Renko just about inside EA. This will turn out to be a tough routine job if it’s worthwhile to apply completely different technical indicators to Renko, as a result of it’s worthwhile to re-implement them from scratch in your digital constructions.

Or you should utilize solely a restricted subset of alerts relying to renko containers (with out extra indicators), for instance, checking Shut[i] in opposition to one another. Blue Renko Bars indicator is prepared for this state of affairs.

RELATED ARTICLES

Most Popular

Recent Comments