Posts

Algorithmic trading: How to get started building an algorithmic trading system?

/0 Comments/in , , /by

This is an answer to a question on Quora: Algorithmic trading: How to get started building an algorithmic trading system?

Answer by Laurent Bernut:

If investment is a process, then the logical conclusion is automation.
Algorithms are nothing else than the extreme formalisation of an underlying philosophy.

This is the visual expression of a trading edge
Trading edge = Win% *Avg Win% – Loss% *Avg Loss%
It changed my life and the way I approach the markets. Visualise your distribution, always. It will help You clarify your concepts, shed light on your logical flaws, but first let’s start with philosophy and belief elicitation

1. Why is it important to clarify your beliefs ?
We trade our beliefs. More importantly, we trade our subconscious beliefs. “If You don’t know who you are, markets are an expensive place to find out”, Adam Smith
Many people do not take the time to elicit their beliefs and operate on borrowed beliefs. Unanswered questions and faulty logic is the reason why some systematic traders tweak their system around each drawdown. i used to be like that for many years.
Belief elicitation exercises:

  1. The Work by Byron Katie. After i completed a 2 beliefs a day challenge for 100 days, i could explain my style to any grandmother
  2. 5 why ? Ask yourself a question with “why” and dive deeper
  3. “I am”: beliefs about self are best elicited when we start with this sentence stem
  4. Sentence completion exercises: Nathaniel Branden in his work on self-esteem has pioneered this method. For example, “to me a robust trading is…”

Mindsets: expansive and subtractive or masala smoothie Vs band-aid
There are two types of mindset, and we need both at different times:

  1. Expansive to explore concepts, ideas, tricks etc. Expansive mindset is useful in the early stage of strategy design
  2. Subtractive: to simplify and clarify concepts. Complexity is a form of laziness. Instead of adding yet another redundant factor, try subtracting one or two. Work hard until You find an elegant and simple solution

Systematic traders who fail at being subtractive have a smoothie approach. They throw all kinds of stuff into the optimization blender and waterboard data until it confesses. Bad move: complexity is a form of laziness.
Overly subtractive systematic traders have a band aid mentality. They hard-code everything and then good luck patching
“Essentialist traders” understand that it is a dance between periods of exploration and times of hard core simplification. Simple is not easy
It has taken me 3,873 hours, and i accept it may take a lifetime

2. Exit: start with the end in mind Counter-intuitive truth
The only time when you know if a trade was profitable is after exit, right ?
So, focus on the exit logic first.
In my opinion, the main reason why people fail to automate their strategy is that they focus too  much on entry and not enough on exit.
The quality of your exits shapes your P&L distribution, see chart above
Spend enormous time on stop loss as it affects 4 components of your trading system: Win%, Loss%, Avg Loss%, trading frequency
The quality of your system will be determined by the quality of your stop loss,

3. Money is made in the money management module
Equal weight is a form of laziness. The size of your bets will determine   the shape of your returns. Understand when your strategy does not work and reduce size. Conversely, increase size when it works.
I will write more about position sizing on my website, but there are many resources across the internet

3. Last and very least, Entry
After you have watched a full season of “desperate housewives” or “breaking bad”, had some chocolate, walked the dog, fed the fish, called your mom, then it’s time to think about entry.
Read the above formula, stock picking is not a primary component. One may argue that proper stock picking may increase win%. Maybe, but it is worthless if there is neither proper exit policy, nor money management.
In probabilistic terms, after you have fixed exit, entry becomes a sliding scale probability

4. What to focus on when testing
There is no magical moving average, indicator value. When testing your system, focus on three things:

  1. False positives: they erode performance. Find simple (elegant) ways to reduce them, work on the logic
  2. periods when the strategy does not work: no strategy works all the time. Be prepared for that and prepare contingency plans in advance. Tweaking the system during a drawdown is like learning to swim in a storm
  3. Buying power and money management: this is another counter-intuitive fact. Your system may generate ideas but you do not have the buying power to execute. Please, have a look at the chart above

I build all my strategies from the short side first. The best test of robustness for a strategy is the short side:

  1. Thin volume
  2. brutally volatile: this
  3. shorter cycle: stocks go up the elevator and go out the window
  4. Borrow availability:

Platforms
I started out on WealthLab developer. It has a spectacular position sizing library. This is the only platform that allows portfolio wide backtetsing and optimisation. I test all my concepts on WLD. Highly recommend. It has one drawback, it does not connect position sizer with real live trading.

Amibroker is good too. It has an API that connects to Interactive brokers and a decent poisition sizer.

We program on Metatrader for Forex. Unfortunately, Metatrader has gone down the complexity rabbit hole. there is a vibrant community out there.

MatLab, the expensive weapon of choice for math graduates. No comment…

R: much better cost-performance than Matlab, simply because it is free

Tradestation
Perry Kaufman wrote some good books about TS. There is a vibrant community out there. It is easier than most other platforms

Final advice
If You want to learn to swim, You have to jump in the water. Many novices want to send their billion dollar ideas to some cheap programmers somewhere. It does not work like that. You need to learn the language, the logic.
Brace for a long journey

Algorithmic trading: How to get started building an algorithmic trading system?

What programs are usually used to back-test trading strategies?

/2 Comments/in , , /by

Answer by Laurent Bernut:

Testing a strategy is a journey. For any journey, You need the right equipment. I test everything on WealthLab then program on other platforms. Here is why

The objective of back-testing is to simulate real trading. When You trade real money, You face the following problems:

  1. Position size: You do not trade 1 share per trade. You may have sophisticated position sizing algo. Platforms offering position sizing algos in back-testing phase are few and far between: WealthLab & amiBroker. QuantConnect and Quantopian may but I have not tried
  2. Cash constraint: You strategy may generate signals but if You do not have cash to execute either a Buy Long or a Buy to Cover, it is a moot point. WLD and amiBroker
  3. Portfolio rotation: this is similar to the above point. Test may look great/poor on one security but in real life, you trade a portfolio.
  4. Slippage & commission: always set slippage at max. My settings is -0.5% one-way, that is 1% slippage + commission for a round trip
  5. Volume: only academics and efficient market believers trade on thin markets and banking holidays
  6. Borrow and uptick: structural shorts are a dime a dozen. Profitable structural shorts are the unicorns of short-selling. If You find one, capture it safely and let’s study it

The journey:
The first part of the journey is to get the signal right. This is where most people stop. They identify a signal that they test across a population of stocks one after the other. This may seem fine in theory but in practice You will rapidly come across a concept called Buying Power. If You don’t have the cash, You can’t execute. If You can’t execute, then it’s all academic. This excludes primitve versions of Matlab, MT4 and older versions of TradeStation
Math formula for that level is: Signal Validity = Win%

Second stage is when You start testing at portfolio level
You have realised that signals must be executable to be worth anything.
At this stage, You will test signals. These are where most people stop. This is the province of most sell-side quants using R or Matlab. Tradestation does not go any further. Not sure about Ninja, never tested that one
It assumes all positions are equal weight. This is primitive
math is  Win rate = Win% – Loss %
Bloomberg BTST is a rudimentary version of that

Third stage is when You start working on the real trading edge formula
Trading Edge = Win% *Avg Win% -Loss%* Avg Loss%
This is when You start to realise that size does matter in the markets, particularly when strategy fails to deliver.
At this stage, there are only two platforms which deliver: WealthLab Developer and Amibroker. Both have a comprehensive position sizing library.
Matlab can deliver but You will need to buy an additional expensive module. Matlab is an expensive guillotine, hardly suitable for neurosurgery. I do not recommend it.
WealthLab and amibroker are both C#. They both have canned strategies You can learn from and a vibrant community.

Personally WealthLab changed my life. It permanently changed the way I perceive the markets.

A word about optimisation: the masala smoothie approach
When I started working on the strategy, I, as everyone else, ventured on the side of complexity. I resorted to optimistion to find the best indicator value.

Most people, most quants, love to optimise and re-reoptimise and re-re-optimise their optimisations.
Bad news: it gives the right mathematical answer to the wrong question. Optimisation is the monosodium glutamate MSG of strategies: it gives depth and substance to poor quality food.
Work on the logic, not on the value of some hypothetical Billion Dollar idea.
Do not throw everything in hoping it will stick. Optimisers will waterboard the data until it confesses everything

PS: 100% Free Kick -A@#resource:
Andrew Swanscott interviewed me on Better System Trader.
Interview with Laurent Bernut – Better System Trader
Off line, he suggested I should offer something to his audience. Here is a tool I use daily.

The Trading Edge Visualiser User’s Manual
The way to use it to load your backtest data and look at distribution. I use this tool many times a day during strategy development.

What programs are usually used to back-test trading strategies?

The four horsemen of apocalyptic position sizing used by professional investors

/7 Comments/in , /by

4 horsemenDespite picking a fair share of good stocks, it is still tough to generate some consistent serious alpha. Picking the right stocks and exiting them well tells You how often You win. How much You win, however, is a function of how much You bet. Some professional investors pay surprisingly little attention to their bet sizes. Below are four algorithm often practiced by professionals that can

  • Four popular bet sizing algorithms used by professionals that have negative gain expectancy
  • Size does matter in the markets: 1$ or 100 will have a different outcome

 

When it comes to bet sizing, there are only two sizes: either too much, either too little. As a professional short-seller, position sizing is mission critical. Successful positions shrink. Not only do they contribute less and less, but they also tilt exposures (net & net Beta). To add insult to injury, they become less noticeable. On the other hand, unsuccessful positions balloon. They immediately hurt. So, I have spent years studying the science of bet sizing. I sought to learn from other investment professionals. It eventually dawned upon me that Long biased people rarely ask themselves the same questions. For them, bet sizing does not have the same degree of urgency. Worse even, it became apparent that some position sizing algorithms had outright negative expectancy, or nasty side effects that they were never even aware of.

Aral Sea Ships

Insufficient liquidity

Horseman 1: Liquidity. If You can’t get out, You don’t own stuff. Stuff owns You
Getting into a position is like buying a boat, or a second house. You can do that any day of the week. Now, selling a boat is tough (been there, done that). It may take time to build a position in a stock. Time is an expensive luxury few market participants can afford when they want to liquidate.So, no matter how attractive a story may be, if you can’t exit easily, just don’t enter.

Rule 1: don’t size your positions so that they may go Hotel California on You:
“You can check-out anytime You like, but You can never leave!”, Don Henley, Hotel California

 

 

Bernie-Madoff-mugshot

Example of long conviction

Horseman 2: High conviction: feel-good position sizing
Disclaimer: this position sizing is used by the greatest and the worst investors. The classic rationale is: “if You believe in something, then you should go big or go home”. What else is it but a feel good position sizing algorithm ? Risk is not quantified but subjectively assessed. The problem is mental accounting, or the constant emotional revisionism of the situation. Jack Welch said: “what can be measured can be improved”. If You can’t quantify your risk, then don’t expect improvement in consistent alpha generation capability.

The greatest investors also use conviction as a position sizing algorithm. The only difference is that they express conviction in units of risk. They quantify risk first and then put chips on the table according to their perception of the reward. If an idea does not pan out, risk can be parred down.

 

Horseman 3: Equal size: one-size-fits-all and the volatility roller coaster
This position sizing algorithm will not bring ruin, but it has negative side-effects that may prevent You from achieving your obejctives in terms of performance, attractiveness to investors and quality of life…

Equal weight is a form of laziness:
First, let’s look at the math behind equal weight. All trading systems boil down to their trading edge (Avg Win% * Win% – Avg Loss% *|Loss%|). Since all bets are equal, equal weight implicitly puts emphasis on the signal, and excludes the value of money management. In other words, stock picking has to be consistently above 50% to absorb losses and generate a profit. Unfortunately, no system works all the time. So, equal weight carries cyclicality in performance.

Ignoring volatility at the position sizing level invites volatility in the portfolio
Not all stocks have the same personality. Some are more volatile than others. For example, internet stocks tend to be much more turbulent than utilities. If all positions are sized equally, then the most volatile stocks will drive the volatility of the overall portfolio. Morality, ignoring volatility at the position sizing level will in turn invite volatility in the portfolio.

Horseman 4: Average down, martingale and the certainty of ruin
Rookie gamblers always come up with some elaborate scheme to break the casino. It is usually a variation on the theme of doubling down after each loss. They believe that the losing streak will end and they will recoup their losses. This position sizing algorithm is known as martingale. Let’s look at the math behind this algorithm

1. Adding to a losing position reduces the hit ratio
2. even if You had infinite capital, the most favourable outcome would be break-even. First, do You have infinite capital ? Second, any other outcome before the most favourable one carries an interesting probabilistic property called certainty of ruin
3. Doubling down means adding to losers. Resources have to come from somewhere, probably winning trades. Books written by successful market participants always emphasize “cut losers, ride winners”. Do You know any successful market guru who says “cut your winners, ride your losers ?”

In conclusion, there is a reason casinos have gold, marble, paintings from masters and gamblers declare bankruptcy. Double down on losers and You will go broke. One more thing about probabilities, it’s not about if, it’s about when.

Conclusion
Size does matter in the markets. Not paying enough attention to position sizing has consequences that range from unpleasant volatility to certainty of ruin. Position sizing is not a glamorous topic, but in highly competitive sport, every little bit of edge counts

The view from the short-side: how we process emotions and the market signature of the 5 stages of grief

/4 Comments/in , /by

Market participants are constantly asked to defend their conviction. The moment they have to justify their positions is the moment they lose impartiality. They become attached to whatever they have to defend. Being right is no longer about the process (taking calculated risks), but about the outcome (making money). A losing position is an attack on the ego. For this reason, market participants process emotions through a 5 stage cycle defined by Elizabeth Kubler Ross as the psychology of grief. Each phase has a distinctive market signature and even a specific language.

Summary:
  • Short sellers have a unique perspective on how market participants process emotions
  • The 5 phases described: market regime, market signature, language and profitable course of action

 I have been a professional short seller for almost a decade. For 8 years, my mandate was to under-perform the longest bear market in modern history: Japan equities. I have always searched for a way to identify the signature of human emotions across markets. Many respected market gurus have come up with charts plotted with emotions ranging from euphoria to despondency. Yet, they never really resonated with the short seller in me. Those were written by market participants with a natural Long bias. The short side offers a unique perspective on how investors process emotions. We, short sellers, never sell against buyers. We ride the tails of those who were once holders and now have to “accept” losses and “let go” of attachment.
There are three market regimes: bull, bear and sideways. Each regime can be subdivided into two categories: quiet or choppy. Bear markets usually start in sideways choppy markets: an epic battle between bears and bulls. They usually end in indifference: sideways quiet or bull quiet. Everyone has thrown the towel and no-one cares anymore.
The psychology of grief has five phases: denial, anger, bargaining, depression and acceptance. We will examine each phase looking at the market regime, the market signature, the language and a profitable course of action.
Phase 1: Denial
  • Market regime is usually sideways choppy. Stocks stop making new highs. They are trapped in a volatile range. Short interest is low. Bulls fight bears.
  • Market signature is the compression of estimates. Optimistic and pessimistic analysts have fairly close estimates. All available information has been “baked in” the estimates. The decisive factor is a sudden penetration through support level.
At this stage, analysts jump in and say two things:
  1. This is a “one-off”, “inventory adjustment”, “seasonal adjustment” etc
  2. This is a “Buy on Weakness opportunity”: Analysts are usually quite vocal as they appeal to market participants who were waiting for a pullback to enter a position
If a stop loss was not triggered, it is best to wait until the ensuing rebound is over to make a decision. If the new peak is below the previous high, then start trimming. When in doubt, reduce position size.
Phase 2: Anger
  • Market regime has morphed into a choppy bear. Stocks make lower highs and lower lows. Volatility remains elevated. Short interest start to tick up. Professional short sellers, such as myself, put a chip on the table just to see. Fast money, those who bought the dips and lost money, turn around and engage in some revenge short selling.
  • Market signature is characterized by institutional reducing their weight. Initial sellers are Long Onlys trimming their weights. Mutual funds may well keep their bets over the index, but they still trim their weights so as to reflect under-performance.
At this stage, analysts express their frustration:
  1. “…But the market does not understand …”: that is always an interesting argument, particularly after years of out-performance, institutional participation. Market probably knows something analysts refuse to accept yet
  2. “Short-sellers and speculators are taking the stocks down…”: ignorant analysts and market commentators blame us for stocks tanking, yet facts are stubborn: short interest is low. Secondly, in order to sell short, we need to locate borrow. Borrow availability represents a tiny fraction of the free float. Simply said, we just do not have the might to take anything down.
At this stage, it is prudent to aggressively reduce bet size for two reasons: 1) Volatility remains high and 2) performance does not justify a big position anyway. For market participants with a Long/Short mandate, this is a good time to anchor a small short bet. Position sizing is crucial as volatility remains elevated. Those anchors become invaluable when stocks move into the next phase as they embed substantial profits.
Phase 3: Bargaining
Me: “Doctor, if I eat my vegetables, stop drinking, smoking, eating poorly and exercise more, will I live longer ?”
Doctor: “I don’t know, but it will feel much longer anyway”
  • Market regime shifts from bear choppy to bear quiet. Bears have won the battle.
  • Market signature is a softening of a leading indicator that triggers a downgrade of estimates. Negative earnings momentum attracts short-sellers. Short interest start to rise.
At this stage analysts bargain with their conviction:
  1. “We take our estimates down, we revise our target price, we extend our investment horizon, but…”: Since analysts were ardent supporters, they believe they cannot change their mind at once
  2. “… We keep our Buy rating, because the long-term story is still intact”: the softening is not perceived as the symptom of a disease but a temporary setback
Analysts devote their existence to a few stocks. They know something is wrong but they cannot publicly admit that it is time to let go, but market participants can read through the lines and sell. Price action has already shown some weakness, but quantitative short sellers see negative earnings momentum as the sign to build positions. Short interest rises and so does the cost of borrow. This is when anchoring a small position in the previous phase becomes invaluable: borrow was secured at a cheap cost.
 
Phase 4: Depression
  • Market regime is bear quiet to bear volatile because of short squeezes
  • Market signature is 1) deterioration of newsflow , 2) radio-silence from the analyst community and 3) rapid increase in short interest
At this stage analysts:
  • crawl under their desks: they hardly contact companies or market participants
  • “this is a stock for long-term investors”: to which there is only one retort: “then it should be matched by long-term commissions”. If they frown, sell short…
Short interest rises quickly. The quality of borrow deteriorates (callable stocks, usury borrowing rate etc). It becomes costly and difficult to sell short. As a rule of thumb, do not sell short when short utilization (shares borrowed/shares available) rises above 51%. Volume is thin so any tiny event can trigger a short squeeze. Amateur short sellers are forced to cover, which trigger a cascade of short cover.
Phase 5: Acceptance
When the inexorability of reality sets in, there is a sort of euphoric relief.
  • Market regime is either quiet bull (small higher highs, higher lows) or sideways quiet
  • Market signature is terrible news-flow, massive downgrade from the analyst community, elevated short interest (crowded short). It is also muted price action: stocks do not react to a torrent of bad news anymore
At this stage, analysts are frustrated and no longer afraid to tarnish their standing with companies. They downgrade ratings, estimates and publish some vitriolic content such as:
  1. “Structural short”, “flawed business model…”, “…mismanaged”: it sometimes becomes personal, because analysts had a rough inner journey being champions of a lost cause
When all the negativity, particularly the words “structural words”, does not move share price anymore, it is a sign that the worse is over. There is one logical thing left to do: cover the short and go long.
Discussion
Markets are the ultimate mental sports. As much as we would like to think we are rational, the moment we are asked to defend our opinions is the moment we lose impartiality. The irony is that we intuitively know when something is not quite right. Still, we feel obligated to defend our stance. We refuse to admit reality, so we go through a painful process that eventually leads to acceptance. Pain is inevitable, suffering is optional. Making money in the markets is not about trying to be right, it is about accepting to be wrong and move on. Would You like to learn simple powerful techniques designed to reconcile the need for conviction and the reality of losses ?
Please leave a comment and share if You care: