#Quora: How does one address the issue of regime shift in algorithmic trading?

How does one address the issue of regime shift in algorithmic trading? by Laurent Bernut

Answer by Laurent Bernut:change-of-season

Fascinating topic that has kept me awake for years. It is a thorny issue. As the patent clerk Einstein used to say: answer are not at the same level as questions. Answer is not at the signal (entry/exit) level. These are position sizing and order management issues.

Definition of regime change

Everybody has a savant definition, so i might as well come up with a simple practical metaphor. imagine You drive at 150 km/h on the highway and then all of sudden, this 4 lanes turns into a country road. If You do not react quick, you will go tree-hugging.

There are three market types: bull, bear and more importantly sideways. Each type can be subdivided into quiet or choppy. So, we have six sextants. Regime change is when market moves from one box to another.

It can be either a new volatility regime, or a move from bear to sideways, sideways to bull or vice versa. Markets rarely move from bull to bear. There is some battle between good and evil.

Why it matters?

Many strategies are designed to do well in a particular environment. They make a lot of money that they end up giving back when regime changes. Examples:

  1. Short Gamma OTM: sell OTM options and collect premium. Pick pennies in front of a steam roller. 2008–2009, August 2015, CHF depeg, Brexit etc. 1 day those options will go in the money and game over
  2. Dual momentum: trade the shorter time frame but determine regime on longer time frame. When market hits a sideways period, market participants get clobbered on false breakouts like poor cute little baby seals
  3. Mean reversion: works wonders in sideways markets until it does not. An extreme version of that is Short Gamma
  4. Value to growth and vice versa: that is probably the most lagging one. By the time, my value colleagues had thrown the towel and loaded up on growthy stocks, market usually gave signs of fatigue…
  5. Fundamentals pairs trading: relationships are stable until market rotation. For instance, speculative stocks rally hard in the early stage of a bull market, while quality trails. Example: Oct-Dec 2012, Mazda went up +400% while Toyota rallied +30%

The vast majority of market participants are trend followers, whether it is news flow, earning momentum, technical analysis. Trending bulls they do great, trending bears, they can survive. Sideways is where they lose their shirts on false positives.

The issue often boils down to how to enter a sideways regime without losing your shirt

The value of backtests

I agree that backtests will help you identify when your strategy does not work. This is actually the real value of backtest. This is when you modify the strategy and adapt the position sizing to weather unsavoury regimes. Then trade this version, not the ideal fair weather strategy.

Reason is simple: everybody’s got a plan until they get punched in the teeth. So don’t lower your guard, ever.

Asset allocation across multi-strats

Some market participants like to develop specific strategies for specific markets: sideways volatile, awesome for pairs trading, great for options strangle/straddle short. Breakouts are good for trending markets etc.

How do You switch from one to the next ? Fixed asset allocation is as clunky, as primitive and as MPT as it gets. There is something far more elegant and simple:

  1. Calculate trading edge for each sub-strategy: long Trend Following, pairs etc
  2. pro-rate trading edge for each strategy
  3. Allocate by pro-rata
  4. Allocate a residual minimum even to the negative trading edge strategies

This is a simple way to put money where it works best

Regime change for single strategy

When not to trade

Van Tharp believes that there is no strategy that can do well or at least to weather all market conditions. I believe there is more nuance. Is the objective to make money across all market regimes? For example, sideways quiet are preludes to explosive movements. In order to make any money, you need to trade big and if you are on the wrong side when things kick off, game over

We trade a single unified (mean reversion within trend following) strategy. What follows is our journey through solving the regime change issue. We found that the three best ways to manage regime changes are

  1. Stop Loss:
  2. position sizing:
  3. order management:

Stop Loss

Regime changes are usually accompanied with rise in volatility. Volatility is not risk, volatility is uncertainty #de-friendSharpeRatio

We used to have a complex stop loss rule. Now we have a uniform elegant stop loss. It just lags current position. It is fashionably late, hence its name: French Stop Loss.

The idea there is to give enough breathing room to the markets to absorb changes without giving back too much profit.

Our solution is to consider stop loss as a fail safe and allow the market to switch direction bull to bear and vice versa within the confine of a stop loss.

The result is we end up switching from one regime to the next more fluidly. We have reduced the number of stop losses by 2/3, which in turn has materially increased our expectancy.

Stop losses are costly. They are the maximum you can lose out of any position.So, unless you have a kick ass win rate or extremely long right tail, you want to reduce their frequency and allow the market to transition.

Position sizing

The first thing You need to understand is that You cannot predict/anticipate a regime forecast. You will find out after the facts. This has some immediate consequences on position sizing: always cap your portfolio risk

Many position sizing algorithms use equity in layers or staircase. We base our calculation of peak equity. So, drawdowns however small have an immediate impact on position sizing. This slams the break much faster than any other method i know.

The other side of the equation is risk per trade wich oscillates between min and max risk. When regime becomes favourable again, it accelerates rapidly.

The whole premise is early response to change in regime through position sizing

Another feature is trend maturity. Betting the same -1% at the beginning and at the end of a trend is asking for a serious kick in the money maker. Trends are born, grow, mature, get old and eventually die one day, just like believers in Efficient Market Theory. So, risk less as you pyramid.

Order management and hibernation

Along with the position sizing comes a vastly underrated feature in most order management: trade rejection.

Secondly, we use a position size threshold. When markets get too volatile and we experience some drawdown, stop losses dilate. As a result, position sizes get smaller. When sizes are too small, trades get rejected.

Order management is vastly under-utilised in most systems i have seen. It is often binary, all-in/all-out, or one way scale in or scale out.

In our system, taking profit off the table is a not a function of chart, technical analysis but driven by risk management. Until price goes a certain distance, exits are not triggered. If exits are not triggered, entries cannot be triggered either. Since volatility goes up and position sizes get smaller, the system drops into hibernation.

We have factual evidence that hibernation during an unfavourable regime is a powerful mechanism to weather regime change. It involves a lot more sophistication than muting indicators, slope flattening etc. It encompasses position sizing, order management, stop loss and to a lesser degree signals.

Conclusion

Multi-strat asset allocation based on pro-rated trading edge is the easy way to go.

Single strat across multiple market regimes means acceleration/deceleration but also hibernation. This is not an easy problem. There is one thought that kept me going through the frustration of figuring this out: “Building a system is like watch making. Time will always be off until the final cog fits in”

How does one address the issue of regime shift in algorithmic trading?

2 replies
  1. Chenri says:

    Hi Laurent, thanks gor your sharing, I am formulazing my position sizing algorithm based on your posts and podcast
    1) reducing risk: cut half position when reaching halfwaf to stop loss
    2) handling free loader: cut half position when position already take one period of average trading cycle and not providing good profit or showing loss, liquidate all on 2 period
    3) position size: based on volatility of 3 atr and risk per trade and cut loss price
    4) cut loss: based on market price rather than percentage of equity or balance

    All of those are for limiting risk and free loaders, but the part of maximizing, I remember you mention about it on closing half position to realized the profit and add more position if the trend continue, but I cannot find your post about it. I read someone in the comment said about the pyramid, reverse martingale, adding smaller and smaller position as profitable trend continue. What do you think about it?

    1) adding smaller position to profitable position, wont it just salting the sea, i mean it might add profit but become neglible, ex: having 10 lot and then add 5 lot when reaching +5%, then adding 7 lot after +10%. Hmm after I write this, I see the point, but still what do you think about this?
    2) how does the reverse martingale match with what you said on realizing profit as profitable trend continue? I am thingking something like: have 10 lot, then liquidate 5 lot when +2% (realized 1%) and then when +4% add another 5 lot, when 6% then liquidate 5 lot (realized +2%) and repeat. The new position is half the the original position so its not a reverse martingale with smaller and smaller position which made the average price went a bit high

    3) how do you work on your final profit exit? My take: using trailing stop after position reach positive, doing the scale out and scale in all the way till the position get stopped out by trailing stop or free loader mechanism

    Best regards, Ps:I really enjoy your posts and podcasts.

    Reply
    • lbernut says:

      Hello Chenri,

      Thank You. That is too much honour.

      ” formulazing my position sizing algorithm based on your posts and podcast
      1) reducing risk: cut half position when reaching halfwaf to stop loss”

      Here are simple algorithms:
      if loss% < = -0.5 * Avg Profit% Then positionSize[n] = 0.5*positionSize[n1]

      2) handling free loader: cut half position when position already take one period of average trading cycle and not providing good profit or showing loss, liquidate all on 2 period
      For every position

      if holding period >= Trading Cycle / 3 && Contribution < = percentile(Contributions,0.33) Then positionSize[n] = 0.5*positionSize[n1]

      3) position size: based on volatility of 3 atr and risk per trade and cut loss price
      4) cut loss: based on market price rather than percentage of equity or balance

      Not sure i understand. I probably meant add 2-3 ATR to a logical stop loss (swing high or swing low) so as to recapture volatility in the position sizing. This in turn will help You size and rank positions

      1) adding smaller position to profitable position, wont it just salting the sea, i mean it might add profit but become neglible, ex: having 10 lot and then add 5 lot when reaching +5%, then adding 7 lot after +10%. Hmm after I write this, I see the point, but still what do you think about this?

      Trading edge = Win% * AvgWin% -Loss% * AvgLoss%
      The above techniques make sure losses are kept small. Now profits look big only to the extent losses are kept small. When You stack positions on a winning stock, your win% goes up and your Avg Win% goes up. Conversely, when you reduce size on your losers, avg loss% goes down. So, for one winning stock with multiple entries, You have n*Wins for 1*loss plus a bigger win/loss P&L
      This is a mechanical technique to boost your trading edge, taht is also consistent with “ride your winners, cut your losers”

      2) We use a depreciation method called declining balance for the maximum risk per trade for each new position. I am originally a CPA and i thought this would be consistent with a form of amortisation. It works well.

      3) That is a deep question. Beginners do not want to give profit back. Veterans do not want to close too early. Your question is a stop loss problem. How do you set the stop loss tight enough so as not to give back too much but loose enough to allow positions to wiggle ?
      Our answer, (in this case, it is both a math and a trader own comfort) is: we allow enough wiggle rooms for trends to change direction within the boundaries of their stop loss. So, bearish trends turn bullish and vice versa. We just close all open positions and enter on the other side, provided risk management allows us to. This is a continuous system. I wish i could attach charts and you would see how it smoothly moves from one dying trend to a burgeoning one on the opposite side.
      Every now and then, positions cross their stop loss before maturing into an opposite side trend. This happens when big news collide into prices, like Brexit or Fed. Positions are of course closed, but then the algo waits for a reversal signal (same as above) to enter.

      Thank You for your questions. Please do not hesitate to ask and please share this website to your friends and colleagues

      Reply

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *