Dealing with Backtest Execution Failures

Hello traders,

This is Tim, and we're continuing our video tutorial series on trading strategy optimization, using Pine Optimizer.

In the last few sessions, we’ve explored how to create, execute and manage your Pine Optimizer campaigns.

In this session, we'll look at backtest execution failures, and how you should deal with them, when they occasionally occur.

To give this some context, we're going to return to the strategies view, where you'll note that I have created a new strategy for this demo.

But, before we get to that, let's take another look at the MACD strategy that we recently optimized.

The history view shows that, to date, the MACD strategy version has executed 9 backtests, and we can see from their status icons that they were all successful.

By clicking on any of the items in the history list, we can see an execution log for the associated backtest. In this case, there's not much to see, because the backtest runs were all successful, so their logs will all show the execution sequence of a successful backtest.

But, from time to time, you may see a backtest that fails, and its execution log can give you valuable clues about what went wrong, and what you can do to resolve the issue.

For example, if TradingView experiences a momentary internet outage, while trying to execute a backtest, it will appear in the history list as a failure.

Similarly, if the Pine Script code for your strategy has a syntax error, TradingView will reject it, and every backtest in the history list will appear as a failure. Their execution logs will explain the cause of these failures and you'll need to fix the errors in your script, before re-launching the campaign.

Of course, you're only likely to see the syntax error issue if you're working with a new Pine Script that hasn't yet been fully debugged. If you've already run your script in TradingView and seen that it executes correctly, it'll work just the same in Pine Optimizer.

When a campaign sequence completes with one or more backtests that failed, those backtests will be reverted to the pending state and you can re-run them - just the failed ones - simply by re-launching the campaign.

And remember that you can always tell how many pending backtest a campaign has, by hovering over its launch button.

Occasionally, you might see a backtest that repeatedly fails, even if you re-launch the campaign multiple times. When this happens, it usually indicates that you're trying to execute a backtest configuration that is not compliant with the rules or limits imposed by the TradingView Strategy Tester.

As an example, let's take a look at this Volty strategy, which has just a single backtest that failed. If we open its log, we can see that TradingView rejected it, and the reason given was that the maximum number of orders was reached.

What's actually happening here is that this strategy is one that executes very frequent trades, with the result that it will generate many trade orders over the duration of each backtest run.

But, TradingView limits each backtest run to just 3,000 trade orders, which means that any run that exceeds this limit will fail. It won't necessarily fail with every parameter configuration - you may find some that work just fine - but if you use one that generates too many orders, like this one does, it will fail consistently.

There's no solution to this problem, as you're trying to execute a backtest that is just not suited to TradingView, so it's best to simply ignore the issue and work with the successful backtests in the campaign. There will usually be many more successful backtests than failures, so this is not typically something that will interfere with your optimization goals.

So, to summarize this session, there are 3 types of backtest failures, and the way you should deal with each of them depends on the type:

- the first type is intermittent failures, such as internet outages, which can be resolved simply by re-launching the campaign

- the second type is Pine Script errors, which will cause a failure for every backtest run in a campaign, and the solution is to fix the error in your script and then try again

- and the last type is TradingView compliance errors, which are very rare, but since they are not resolvable, the only solution is to ignore the affected backtests and work with the data yielded by the successful ones

And that means we’ve reached the end of this session on backtest execution failures.

In the next session, we'll be looking at how to customize Pine Optimizer to suit your personal trading style. In the meantime, thanks for watching.