Many merchants like renko and different “timeless” charts for his or her readability and value noise discount. The charts are used not just for technical evaluation however for constructing buying and selling methods on prime of them. They usually require a correct backtesting. Nevertheless, not many merchants notice that Renko backtesting could also be (and often is) extremely misleading on account of specificities of the chart and platforms that present it.

On this put up we’ll deal with the pitfalls of renko realization in MetaTrader 5 and how one can overcome a few of them. However earlier than we start and for the sake of brevity allow us to remind you of earlier publications on this subject.

The story begins with the introduction of the Renko chart generator – RenkoCharts – it is technical preview (out of date) first appeared on this weblog as effectively. It builds renko from quotes/charges and don’t think about precise value actions inside each bar. If the field measurement is smaller than a spread of particular bar, this may increasingly result in a simplified, unrealistic formation of renko bins on historical past. Consequently, backtests can run smoother than in actuality. Please be aware that, when on-line, the generator works by incoming ticks, so the standard of renko is unaffected, however not everybody can afford to gather ticks for months on-line to construct a trustworthy renko (although it will be unfeasible on account of possible interruptions on account of bugs, connection losses).

The apparent resolution of the issue is to construct renko on the historical past from ticks as a substitute of quotes. In fact, that is doable provided that the tick historical past is obtainable.

Let’s be aware of the issue and its resolution:

A. Keep away from constructing renko from common timeframe bars, even M1, and use ticks if doable

This strategy is applied in one other product – RenkoFromRealTicks.

It gives higher high quality than the previous generator, however does so on the expense of upper sources utilization (disk area, bandwidth, CPU load) and execution instances.

And guess what? It doesn’t clear up all the issues with renko accuracy.

One in every of them arises from the truth that implementation of renko charts in MetaTrader 5 relies on {custom} symbols. So far as they’re generated regionally in your PC and have distinctive names totally different to the names of authentic symbols supplied by dealer, it is not possible to commerce renko symbols on-line. Paradoxically, we will commerce {custom} symbols within the tester simply. However this is able to be theoretical backtests, which cannot be mapped to actual buying and selling with out extra coding and don’t assure precisely the identical (good) outcomes.

To satisfy renko charts and authentic symbols your robotic ought to subject orders for authentic symbols by indicators detected on corresponding renkos. There’s a devoted blogpost describing some strategies of Adapting EA robots to buying and selling on {custom} renko charts in MetaTrader 5 and high quality metrics of their backtests.

In truth, such robots are multi-symbol ones and may handle synchronization between at the least 2 tickers: {custom} and authentic. Mainly, it’s best to detect formation of renko bins, however this isn’t so trivial as you might suppose. Nuances exist relying from buying and selling library you employ (for instance, commonplace library) and concerned instruments (akin to indicators or charges evaluation immediately, any of which depends on bars completion). We’ll talk about this matter a bit later, in part D.

Now simply be aware of the duty:

B. Guarantee cross-symbol synchronization

Some consideration on constructing extra dependable renko-driven checks have been made in one other blogpost – Buying and selling on renko charts constructed from actual ticks.

At these stage there have been nonetheless 3 foremost issues – their frequent impact will be described as peeking into the long run, which produces unrealistically good buying and selling outcomes:

1. a niche in entrance of reversal field;
2. each field is M1 bar aligned by 1 minute boundary, whereas its precise formation (both of open time and finish time) occur at some moments inside close by minutes;
3. quick and distinguished value actions might produce bins forward of actual time, as a result of they’re M1 bars (bars cannot be positioned extra tightly to one another than as soon as each 1 minute);

Allow us to think about the clauses extra carefully, ranging from the 1-st.

Keep in mind that considered one of doable approaches to adapt EA to work on a {custom} image is to position EA on the chart (renko in our case, which is used for synchronization and technical evaluation) however redirect buying and selling orders to authentic image. Usually, all renko indicators are taken from closed bins (as a result of the right-most forming field can find yourself in any route, and even its “open” value). And each reversal field is began with the hole, as a result of the tester is able to simulating common bars solely. In actuality, there is no such thing as a such hole between the shut value of the earlier field and the open value of the following field. It is only a conference of drawing reversal renko field with the hole.

Execution of trades on 100pt renko chart of EURUSD with synthetic ticks or no ticks (left) vs actual ticks (proper). Be aware how faux gaps on the left chart decrease losses compared to the appropriate chart the place there are not any gaps.

As you might even see, if a buying and selling sign is occured for the time being when reversal field is began, the deal value is shifted from the true value for that second by the field measurement (in common). Normally this unfair correction goes in favour of the dealer – a shedding commerce turns into much less unprofitable, or worthwhile commerce will increase.

In case your EA is positioned on a chart with authentic image and trades it immediately, acquiring indicators from bins and/or indicators on renko image, you then get rid of the 1-st downside. But when it is extra appropriate so that you can place EA on renko (for instance, EA is constructed so by a developer and cannot be tailored), we’ll present an alternate resolution beneath.

Now allow us to think about the 2-nd case.

The second when a value crosses a boundary for subsequent field completion is random. It lies someplace between two consecutive 1 minute marks. When it occurs we create a brand new field with a timestamp rounded as much as nearest entire minute. The true value at this actual minute (with out seconds and its fractions) was not the identical as the worth for the time being when the earlier field was closed and the brand new field was began. The imply of this distinction is a half of M1 bar vary in common. Once more, it implies that trades are opened and closed by extra useful value than in actuality, upfront, so to talk. You could give it some thought as if renko chart is all the time nearly shifted half-minute forward of time.

Execution of trades on common EURUSD chart pushed by sign from 100pt renko with synthetic ticks or no ticks (left) vs actual ticks (proper). The sign is shaped by upward field accomplished at 1.08100 on 16:05 M1 bar. The left purchase is executed too early (at bar opening, therefore with too low value and surprising bonus), whereas the appropriate purchase is executed at actual time 16:05:46 and actual value.

The three-rd downside will be eradicated by selecting sufficiently massive field measurement that point overflows don’t occur (not all the time doable for devices with giant adjustments in volatility), however overwise it’s sadly unsolvable by canonical charts, and requires to invent some tips (mendacity outdoors the scope). It would not essentially imply that all of “run-through” bins results in a displaced sign, however a few of them will be mishandled.

Execution of the identical indicators on overflown (wanting forward) bins when EA is positioned on common M1 chart (left) vs renko (proper). Large draw back transfer round 13:30 M1 was mapped into a number of renko bins with “future” time, the place 2 MAs crossing sign produced at synthetic 13:35. When that sign is handed to EA working on common M1 chart, 13:35 bar stands at earlier second, consequently the promote is executed upfront with increased (higher) value.

One approach to probe right here is to run EA solely on renko chart (when within the tester). So far as EA takes indicators from newest accomplished field, it is unimportant that the time marks of the bins might go forward of actual time.

Fortunately the 1-st and 2-nd issues talked about above will be mounted by the next strategy:

C. Generate actual ticks for renko bins

Keep in mind that bars (bins) are simply containers for ticks, so an M1 bar would not have to start out with a tick at 0-th second or find yourself with a tick at 59-th second. As we create our {custom} image, we will generate ticks for renko field with actual timestamp when the field was open. The primary tick will include actual precise value as effectively. All different ticks may even include actual costs, however we have to regulate their time to the identical timestamp to be able to preserve them contained in the field. If we protect actual time to the renko ticks, the tester and the terminal would kind extreme bars (in keeping with they “understanding”), which break correct renko formations.

With such “actual” ticks inside renko bins we will run checks primarily based on actual ticks for renko symbols and execute trades with actual costs. No synthetic gaps on reversals, no parasitic half-M1 deviations.

So okay, we’d like the true ticks. The truth that RenkoFromRealTicks builds bins from actual ticks doesn’t imply robotically that there are ticks contained in the bins. Till model 1.4 RenkoFromRealTicks didn’t generate ticks for the bins on historical past.

Now it could.

New parameter TickHistoryMode permits you to select one of many tick technology modes (so as of accelerating accuracy on the expence of useful resource consumption):

• Single tick per field
• OHLC ticks per field
• Bulk ticks per field

The primary mode is sort of the identical because it was earlier than. It generates a single tick per field in historical past. The distinction is that new variations preserve timing with seconds and milliseconds (the outdated ones used M1 time).

The second mode generates a number of ticks per field, often 2 or 3 (relying from whether or not a field has a wick), which mimics OHLC attribute costs of a bar (therefore the identify), with 1 extra heat up tick.

The third mode generates all ticks from authentic image.

You could select an optimum mode in keeping with your buying and selling algorithm and sources.

When OHLC or bulk ticks modes are enabled, the ensuing renko image helps testing with the settings “each tick primarily based on actual ticks”. It is advisable to examine tester log to guarantee that the true ticks are in impact. For instance, if the principle image, chosen within the tester, has tick historical past, the start of the log incorporates some strains about ticks synchronization (downloading them from the terminal, if it is 1-st run for that image, or about synchronized already ticks, in any other case).

A number of strains beneath there ought to be particular line:

EURUSD_r100,M1 (Your Dealer): producing primarily based on actual ticks

For those who use a number of renko symbols, all of them must also be talked about as having actual ticks:

...
EURUSD_T_r100: ticks synchronized already [57 bytes]
EURUSD_T_r200: ticks synchronized already [57 bytes]
EURUSD_T_r300: ticks synchronized already [57 bytes]
...
EURUSD_T_r100 : actual ticks start from 2023.01.01 00:00:00
EURUSD_T_r200 : actual ticks start from 2023.01.02 00:00:00
EURUSD_T_r300 : actual ticks start from 2023.01.02 00:00:00
...

Be sure that to generate renko historical past with at the least 6-12 months reserve forward of deliberate beginning date of testing. The dates within the log excerpt above is chosen by the tester in keeping with the settings to check from starting of 2023 12 months. The renko symbols have been generated because the starting of 2022.

The tester logs this data just for the principle tester image, for instance:

EURUSD_r100: historical past information begins from 2022.01.02 00:00
EURUSD_r100: ticks information begins from 2022.01.02 00:00

For all extra symbols, loaded out of your EA ad-hoc (through CopyRates and so on) and added to the tester’s market watch, this data isn’t supplied within the log (I do not know why).

For those who use different {custom} image turbines for comparability or cross-validation, please, keep watch over additional messages in logs. Probably, even when actual ticks are enabled, there will be issues with tick historical past on some bars. RenkoFromRealTicks doesn’t permit this, however different instruments might do.

Right here is an instance of a {custom} image with 17 problematic bars:

EURUSD_T_r100 : 2023.01.02 23:59 - actual ticks absent for 1 minutes out of 16 whole minute bars inside a day
EURUSD_T_r100 : 2023.01.12 23:59 - actual ticks absent for 4 minutes out of 51 whole minute bars inside a day
EURUSD_T_r100 : 2023.02.03 23:59 - actual ticks absent for 3 minutes out of 37 whole minute bars inside a day
EURUSD_T_r100 : 2023.03.12 23:59 - actual ticks absent for 2 minutes out of 8 whole minute bars inside a day
EURUSD_T_r100 : 2023.03.22 23:59 - actual ticks absent for 1 minutes out of 22 whole minute bars inside a day
EURUSD_T_r100 : 2023.05.05 23:59 - actual ticks absent for 1 minutes out of 14 whole minute bars inside a day
EURUSD_T_r100 : 2023.07.07 23:59 - actual ticks absent for 2 minutes out of 18 whole minute bars inside a day
EURUSD_T_r100 : 2023.09.20 23:59 - actual ticks absent for 1 minutes out of 15 whole minute bars inside a day
EURUSD_T_r100 : 2023.10.08 23:59 - actual ticks absent for 1 minutes out of 3 whole minute bars inside a day
EURUSD_T_r100 : 2023.11.14 23:59 - actual ticks absent for 1 minutes out of 18 whole minute bars inside a day
EURUSD_T_r100 : actual ticks start from 2023.01.01 00:00:00
EURUSD_T_r100 : 2023.01.01 00:00 - 2024.02.01 00:00  actual ticks absent for 17 minutes of 3230 whole minute bars, each tick technology used

For these bars the tester will generate synthetic ticks that are incompatible with renko and should have an effect on accuracy.

One essential factor to recollect: the tester retains its personal native cache of all symbols utilized in checks. For those who recreate {custom} image with altered properties (together with full removing and constructing from scratch) the terminal ought to usually present new historical past to the tester, or in different phrases, the tester ought to detect the adjustments and obtain the historical past anew. Sadly, it would not work so on the time of writing, so you have to clear up the tester cache manually if you happen to plan to alter {custom} image. The cache is positioned within the following 2 folders for charges and ticks, correspondingly:

/Tester/bases/<dealer>/historical past/<custom-symbol>/
/Tester/bases/<dealer>/ticks/<custom-symbol>/

One might imagine that while you delete a {custom} image from the terminal, the image is robotically deleted from the tester cache as effectively, nevertheless it’s not.

D. Implementation

Because it’s talked about earlier, a multi-symbol robotic wants synchronization between symbols. The indicators of renko is sensible just for accomplished bars, therefore we have to examine bar completion (or initiation of a brand new bar, if you want). As you understand, MetaTrader 5 is an asynchronous factor. The incoming ticks are processed by the platform and accrued as bars, however it could occur with slightly delay. In different phrases, your EA can obtain new tick marked by subsequent minute, however corresponding M1 bar isn’t but obtainable (remains to be being constructed). Consequently, indices utilized in expressions like iClose(…,0), iClose(…,1), CopyBuffer(…,1) nonetheless returns values from the previous (nearly outdated, about to be shifted) bars.

Normally this unsychronized state lasts not more than 1 tick, if it occurs in any respect. However it’s adequate to get incorrect indicators from outdated bars and run into losses.

Sadly, this facet isn’t taken into consideration in the usual library (if you happen to use MQL5/Embody/Knowledgeable) and likewise might not correctly dealt with in different supply libraries.

For instance, allow us to look into the strategy CExpert::Refresh which is named upon new ticks.

bool CExpert::Refresh(void)
{
   MqlDateTime time;
   if(!m_symbol.RefreshRates())
      return(false);
   
   TimeToStruct(m_symbol.Time(), time);
   if(m_period_flags != WRONG_VALUE && m_period_flags != 0)
      if((m_period_flags & TimeframesFlags(time)) == 0)
         return(false);
   m_last_tick_time = time;
   m_indicators.Refresh();
   return(true);
}

RefreshRates of m_symbol calls SymbolInfoTick internally, and the time from the tick is used as a latch for not processing the bars twice. For that goal the tick is theoretically mapped in all timeframes (calculated in TimeframesFlags). If the tick advances on one of many timeframes, the code shops m_last_tick_time and refresh indicators. In different circumstances, the refresh is skipped. However as we already know, the brand new tick will be not but propagated to bars, so the refresh (which is definitely equal of required CopyBuffer’s) shops the outdated information, which is about to alter in a fraction of a second. But when the charges are up to date, and indexing is shifted, the latch doesn’t permit to refresh anymore (as a result of subsequent ticks have instances inside the theoretical bar, detected on the 1-st tick).

Because of this we have to apply some patches to the usual library or do related stuff in our supply codes.

Right here is an instance, the way it could possibly be carried out in an skilled adviser, primarily based on the usual library.

class SyncSymbol
{
  protected:
    const string image;
    const ENUM_TIMEFRAMES timeframe;
    const string interval;
    datetime lastUnsync;
    lengthy countUnsync;
    
  public:
    SyncSymbol(const string s = NULL, const ENUM_TIMEFRAMES tf = PERIOD_CURRENT):
      image(!StringLen(s) ? _Symbol : s), timeframe(tf == PERIOD_CURRENT ? _Period : tf),
      interval(StringSubstr(EnumToString(timeframe), StringLen("PERIOD_")))
      {
      }
    
    bool isReady()
    {
      MqlTick t;
      SymbolInfoTick(image, t);
      const string now = StringFormat("%s'%03d", TimeToString(t.time, TIME_SECONDS), t.time_msc % 1000);
      if(!SeriesInfoInteger(image, timeframe, SERIES_SYNCHRONIZED))
      {
        PrintFormat("%s wait unsync %s %s", now, image, interval);
        return false;
      }
      
      if(t.time / 60 * 60 != iTime(image, timeframe, 0))
      {
        PrintFormat("%s wait unsync with %s on %s %s", now, (string)iTime(image, timeframe, 0), image, interval);
        lastUnsync = t.time / 60 * 60;
        countUnsync++;
        return false;
      }
      
      if(lastUnsync == t.time / 60 * 60 && countUnsync)
      {
        PrintFormat("Synced after %ld ticks on %s %s", countUnsync, image, interval);
        countUnsync = 0;
      }
      return true;
    }
};
  
void OnTick()
{
  static SyncSymbol image;
  
  if(!image.isReady()) return;
  
  ExtExpert.OnTick();
}

This demonstrates check-up of present _Symbol solely. For multi-symbol EAs it is a good suggestion to examine all symbols in the identical method.

In case of unsynchronization the thing will output one thing like this:

2023.01.03 10:25:53   10:25:53'645 wait unsync with 2023.01.03 09:42:00 on EURUSD_r100 M1
2023.01.03 10:25:53   Synced after 1 ticks on EURUSD_r100 M1

You could discover a new revision of the check EA MA2Cross2.mq5 and its sign module connected to the put up. Additionally a few essential patches for ExpertSignal.mqh and ExpertTrade.mqh are included.

And listed here are some outcomes achieved with this EA backtested in numerous modes.

Tester report for buying and selling EURUSD by artificially generated ticks in EURUSD 100pt renko, equal of greatest accuracy supplied by former variations of RenkoFromRealTicks (pre-1.4)

Tester report for buying and selling EURUSD by actual ticks in EURUSD 100pt renko, with improved accuracy

Tester report for buying and selling immediately on EURUSD 100pt renko by actual ticks, which demonstrates related outcomes to earlier one and is appropriate for validation at smaller useful resource footprint

You may check EA in numerous modes and anticipate on-line buying and selling efficiency to be close to the decrease sure amongst all outcomes.

F. Closing notes

Visualization of the renko within the tester can look unusual – bins could also be smaller or bigger than predefined measurement. It could’t be mounted – that is how the tester works.

Allow us to make clear. Renko bins/bars are correctly formed by the generator in static historical past and correctly re-shaped on-the-fly on-line. When the ticks are “performed” within the visible tester, the generator is uncontrolled, and the bars are shaped by ticks solely. Because of this they give the impression of being unusual.

On this regard, it is essential to notice that the attribute value of the renko field is its shut value. So, if you happen to use situations for checking indicators, as a substitute of shut(i) vs open(i) think about using shut(i) vs shut(i+1). Open(i) will be someplace in a spread of 0-2 field sizes from shut(i) and even equal to shut(i), if i-th bar is a single tick bar drawn throughout massive bounce of value.

If you are going to use indicators (like MAs) utilized to renko, additionally setup them for shut costs completely. Although highs and lows must also be dependable.

The same issues of exact backtesting pertain to different particular charts which get rid of a time in a way that bars are usually not alined by time scale however shaped upon information pushed situations, for instance, by quantity restrict. Additionally lots of such charts have gaps in entrance of reversal bars, for instance Level And Figures.