Predicting Volatility & Liquidity Regimes using Machine Learning

There are many potential applications for trying to understand what particular state or regime a market is currently in, and more importantly, what regime is predicted.

For example, attempting to predict price momentum from an alpha or execution timing perspective, or predicting volatility and liquidity regimes to assist in execution decision making. At BestX, our regime research has initially focused on the latter and in order to provide a predictive component to the regime analysis we have employed the use of machine learning, a particularly hot topic in its own right with many different methods and approaches now available.

Rather than simply choosing the most complex sounding method for quantitative and intellectual satisfaction, we have conducted a rigorous study of 6 different methods to determine which is the most appropriate to help solve our particular problem of predicting regimes in volatility and liquidity. Interestingly, we found that the more complex deep learning/neural net methodologies were not as successful for regime prediction as a simpler classification method. This has reiterated the importance to us of ensuring you pick the right tools for the job.

If you are a BestX client and would like a copy of the research paper, please email us at contact@bestx.co.uk.

Finding and balancing the “Sweet Spot” on an RFQ Panel

Our latest research attempts to tackle the long standing question of ‘how many participants should I have on my RFQ panel’. Not an easy question to answer, and for fear of sounding like a politician, the answer is ‘it depends’. On what? Well, it generally depends on how you trade and what your objectives are for example, if you trade full size and don’t tend to build into a position over a period of time, then market impact may not be a key priority and you may simply want to focus on minimising spread costs. If, however, you execute in slices, building into positions, then impact can have a significant impact on the goal of achieving best execution.

We explore a causal approach to answering the question, trying to balance the ‘sweet spot’ of minimising the spread paid, whilst at the same time, minimising the potential information leakage and subsequent market impact.

Please email contact@bestx.co.uk if you are a BestX client and would like to receive a copy of the research.

High Frequency Volatility Measures: An Exploration

In our latest research we explore the arcane subject of comparing different methodologies for measuring high frequency volatility. This can be more art than science, but a robust intraday volatility measure is extremely useful for many aspects of modelling.

Within the BestX framework, we use such a metric to help estimate opportunity cost within our Pre-Trade module. Going forward we also plan to introduce a machine learning based model to help predict intraday volatility regimes, and this work serves as a building block for the more complex models to come.

Please email contact@bestx.co.uk if you are a BestX client and would like to receive a copy of the article.

Measuring the Unmeasurable?

“Somebody said it couldn't be done”
Edgar A. Guest

 At a recent fixed income conference, the title of the obligatory TCA session was ‘measuring the unmeasurable’. There are many in the industry that still hold this view, and that traditional TCA is either not relevant or impossible to measure within Fixed Income. However, at BestX, we would prefer to think along the lines of ‘some of it isn’t measurable, but for a large proportion of the Fixed Income market, it is possible to generate some meaningful analysis that can add value’. Admittedly, not as snappy a title for a conference session, but a fair reflection of the reality of the current status. In this article we draw on our own experiences of expanding our BestX product across Fixed Income to highlight some of the issues and how we’ve tried to address them.

There are many hurdles to leap when attempting to build viable analytics to measure TCA for Fixed Income markets. To begin with, it is worth clarifying that in our view, TCA is a somewhat misleading term. We see measuring transaction costs, in the traditional TCA sense, as an essential component of an overall suite of best execution analytics, that seek to add value across the lifecycle of the trade and best execution process. But only a component.

Let’s return to first principles and recap on what best execution actually is. Not in terms of the formal regulatory text definition but in practical terms, what does it really mean? We distil the essence of best execution into the following 6 bullet points:

1.      Best execution is a process
2.      Covers lifecycle of a trade, from order inception, through execution to settlement
3.      Requires a documented best execution policy, covering both qualitative and quantitative elements
4.      Process needs to measure and monitor the execution factors relevant to a firm’s business
5.      Any outlier trades to the policy need to be identified, understood and approved
6.      Requires continual performance assessment, learning and process enhancement, i.e. a best ex feedback loop

Ok, so if we agree this is what we are trying to achieve in order to deliver upon our fiduciary responsibility to asset owners and our desire to optimise performance, it is clear that a simple pdf or Excel report with a single cost number, measured versus a single closing price of the day, is not going to be sufficient. Clearly a technology solution is required, that can be configured to an institution’s specific business, trading protocols and best execution policy. This solution needs to measure the relevant performance factors, automate the outlier monitoring process and provide flexible, interactive reporting analytics to dive into performance results to seek areas where additional value can be extracted.

Data Issues

Within such a solution, as accurate a measurement of cost as possible is obviously extremely important, and this was the first challenge we sought to tackle in our expansion to Fixed Income. For non-Fixed Income people, it may not be obvious why this is such a challenge so for fear of stating the obvious, it is all about market data availability. The issues here are numerous, for example:

-         Coverage
-         Timeliness
-         Cost
-         Quality

Now, we were hoping, as were many others, that the sunlit uplands post-MiFID II were going to be filled with overflowing volumes of cheap, if not free, market data that all market participants could readily delve into. This has not transpired. However, there are sources of quality, timely data available, at a price, and this is where we turned. We didn’t want to build a Fixed Income product that wasn’t commensurate with the quality of our award-winning FX product, so high quality market data was essential. However, given the sheer breadth and complexity of the Fixed Income market, where there are millions of securities traded around the world, there are always going to be gaps. Such gaps may be short term due to, for example, new issues, or more structural, for example, complex structured sections of the market just don’t trade in the conventional sense. This required thought when building the trade parsers and analytics engine to mitigate gaps in market data coverage and quality, a challenge made easier given the modern cloud-based technology stack we are working within at BestX.

With the best market data available, and applying innovation to the analytics, there is still the need for a healthy dose of pragmatism when measuring transaction costs in Fixed Income. Indeed, a client recently told us that in Fixed Income “a price is not a price, it is simply an opinion”! There are always going to be highly illiquid sections of the market that do fall within the unmeasurable category, but we have found that it is possible to construct accurate measures of mid for the vast majority of bonds. This then, obviously, allows decent spread cost numbers to be measured for a given time stamp(s).

Time stamps. Another data issue altogether, although one we are familiar with from FX land. Fixed Income execution is becoming increasingly electronic and automated, a required development as buy-side execution desks are increasingly asked to do ‘more with less’, with traders having to execute more tickets, become responsible for more products and develop experience in more execution protocols. From the analysis we have done so far, time stamping for trades executed over the various MTFs all look pretty robust, as you would expect. Issues tend to arise in voice executed business, although here the quality of time stamps is improving post MiFID II. It goes without saying but we will say it again anyway, it is impossible to measure anything accurately without a decent time stamp.

Issues around data accuracy: trade data, time stamps, market data and benchmarks appear to be the number one priority for clients when research surveys are conducted. Getting all this as right as possible is deemed much more important than, for example, automated connectivity with EMS/OMS platforms, at least for now. We obviously expect this demand to rise going forward once the basics are in place.

Making it actionable

Another of the common complaints around applying TCA in markets such as Fixed Income is ‘what do we do with the output?’. Measuring a simple cost number on an infrequent basis doesn’t lend itself to making any such analysis actionable. This is why it is key to implement any TCA metrics as part of a best execution workflow and lifecycle.

Back in October 2016[1] we talked about feedback loops in the best execution process and applying the concept of marginal gains to improve performance. There are many decision points in the execution process where additional data-driven evidence can help the trader make a more informed decision, for example:

-         What time of day should I trade?
-         What size of trade should I execute?
-         Who should I execute with?
-         Should I trade principal or request the counterparty to act as agent for my order?
-         If I trade principal, should I trade via the phone or electronically?
-         If I trade electronically, which platform should I trade on?
-         Should I hit a streaming firm electronic price, or should I trade RFQ?
-         If I RFQ how many quotes should I request?
-         How quickly should I trade?

To ensure any output from your Fixed Income analytics can be actioned it is essential to have the following components to your TCA/best ex platform:

1.      Intuitive, graphical interface which allows large data sets to be queried and filtered quickly
2.      Enough data to make the performance results significant
3.      Timely information and analysis
4.      Ability to get into detailed diagnostics on a trade by trade level if required

We were already aware how important the last point was whilst building out our FX product. However, Fixed Income, with its more complex analytics, has increased this requirement even further. For example, it is imperative to be able to understand where a specific cost number is coming from, down to an understanding of which benchmark bonds were used to construct the yield curve if there weren’t sufficiently non-stale prices available in the specific security.

Conclusion

So, is Fixed Income TCA measurable? For a large part of this diverse and complex market, we think it is. Is it perfect? No, but our philosophy has always been to be pragmatic, rigorous and thoughtful when building what is possible under the given constraints. Getting such a first iteration out and used by clients allows us to evolve and improve over time, whilst at the same time hopefully benefitting from improved availability, quality and coverage of market data. For example, who knows, but a Fixed Income Tape, as mandated by MiFID II, may even appear one day.

To quote Edgar A. Guest’s tremendous poem again,

Somebody scoffed, "Oh, you'll never do that
At least no one ever has done it."
But he took off his coat, and he took off his hat,
And the first thing we know, he'd begun it.

With a lift of his chin and a bit of a grin,
Without any doubting or "quit-it".
He started to sing as he tackled the thing
That couldn't done. And he did it. 

Ok, in true BestX style, there hasn’t been a lot of grinning, and certainly no singing, but we have done it.

BestX Case Study: Making RTS28 Useful

This brief article continues our series of case studies for the practical deployment of the BestX execution analytics software. Please note these case studies should be read in conjunction with the BestX User Guide, and/or the online support/FAQs. This latest article explores the Regulatory Reporting module of BestX, and how we feel this can be utilised to actually make the RTS28 reporting obligations under MiFID II to be of some value. We can hear the cries of derision upon reading this statement, but trust us, we do think that it is possible to make some of the reporting under RTS28 add real value to your best execution process and the overall goal of improving performance.

Generating the classic Top 5 report appears to add little value on its own – however, if this is produced in conjunction with a report that summarises performance according to your best execution policy (or in MiFID II speak, Order Execution Policy, as defined under Article 27(5)), then we enter the realms of added value. Assessing your top Venues, Counterparties and/or Channels (e.g. MTFs) on an overall performance basis and then marrying this up with where your actual volumes are executed can produce some interesting insights, and potentially actionable items to help improve performance going forward. This is one of the intended, as opposed to one of the many unintended, consequences of MiFID II and requires a systematic, and configurable, framework to measure and compare performance.

If you are a contracted BestX client and would like to receive the full case study please contact BestX at contact@bestx.co.uk.

BestX Case Study: Generating YTD Reports

This brief article continues our series of case studies for the practical deployment of the BestX execution analytics software. Please note these case studies should be read in conjunction with the BestX User Guide, and/or the online support/FAQs. Given that 2018 is drawing to a close, the focus of this third case study is the generation of year-to-date reports.

BestX provides a multitude of different reporting types, that have many different purposes and use cases. In this case study we cover 3 of the most common: 

1. Generating annual post-trade performance reports – used for internal best execution review meetings etc

2. Generating annual exception report summaries – useful for showing documentary evidence of the implementation, monitoring and adherence to a firm’s best execution policy

3. Generating annual total consideration reports – useful for high-level cost summaries for client accounts etc

The flexibility of BestX allows users to run such reports on the entire portfolio, or configured to specific elements of the year’s trading activity to analyse key components.

If you are a contracted BestX client and would like to receive the full case study please contact BestX at contact@bestx.co.uk.

Exploring data driven methods for measuring expected transaction costs in FX

At BestX we are continuing to research methods for enhancing measurement of expected costs within the OTC markets.

A parametric based model provides a good approximation, but runs the risk of becoming increasingly complex to try to model all edge cases, especially for less liquid currency pairs, or times of day or larger sizes. We have been conducting research in parallel to develop a new framework, which is purely data driven using machine learning methods. More work needs to be done but this paper provides results for this framework when applied to the measurement of the forward component of FX transaction costs, a notoriously difficult part of the market to model given the voice driven, OTC nature of this part of the market.

For a copy of the paper please email us at contact@bestx.co.uk.

BestX Case Study: Monitoring Trends in Performance

This brief article continues our series of case studies for the practical deployment of the BestX execution analytics software. Please note these case studies should be read in conjunction with the BestX User Guide, and/or the online support/FAQs.

The focus of this second case study is the monitoring of trends in execution performance, a key component of the feedback loop required within a best execution policy. The BestX Trend module allows the construction of tailor-made analyses of trends in any of the different metrics that BestX computes, thereby providing flexibility to add value to any institution’s specific policy and process.

Monitoring execution performance over time, and learning lessons to help refine the policy and process further, is not simply something to satisfy any regulatory or fiduciary responsibilities. In our view it is a fundamental part of improving performance to improve returns, and can be applied to many constituents of the execution process, for example:

• Monitoring liquidity provider performance over time, by product, by ccy pair, by trade size, by time zone etc

• Monitoring algo product performance

• Assessing how different venues perform over time

• Are they any market structure changes occurring that are resulting in different execution methods performing differently e.g. RFQ vs streaming prices?

• Is market liquidity changing over time, e.g. is my business creating more impact than it used to?

We explored the concepts of feedback loops and marginal gains in a previous article, published in Oct 2016. This case study helps put concepts into practice using the BestX software.

To receive the full case study, please email us at contact@bestx.co.uk

FX Algos – a proposed taxonomy

Style is a simple way of saying complicated things
Jean Cocteau

A common request from many of our clients over recent months has been to categorise algo products into a number of different styles. This is far from straightforward given the plethora of algo products now available (in BestX alone we have now seen 113 different algos across many providers), and also due to the fact that the market, and product innovation, continues to evolve rapidly. Given this breadth and complexity, it is probably pragmatic to start with a reasonably simplistic taxonomy of styles, which can always be refined over time. In this short article we introduce our initial family of algo styles based on a number of discussions with our clients.

To kick off, we are proposing 4 key algo styles, summarised in the diagram below, which stem from the initial key objective: is the algo attempting to achieve a specific benchmark or not?

For non-benchmark algos, we have suggested 2 style groups:

1.       Get Done
2.       Opportunistic

Whereas for algos that are designed to minimise specific benchmark slippage, we have suggested an additional 2 style groups:

3.       Interval Based
4.       Volume Based

In addition, for each algo there are additional attributes that can be used to describe behaviour:

1)      Limit – whether the algo had a limit price applied to it or not, and,

2)      Urgency – a data field describing the urgency, or aggressiveness, of the algo. There are many different forms of Urgency used in the market, so to simplify, we are condensing into 3 values: Low, Medium or High

The additional attributes are important to allow an apples vs apples comparison, for example comparing a sample of algos within a category where some have hit limits and others have not could pollute the performance results. Equally, if you were analysing a group of Opportunistic algos, it would be preferable to stratify the sample into groups with similar Urgency settings.

Going into these 4 categories in a little more depth:

1.       Get Done – this family of algos is expected to include more aggressive algo types where the priority is less on minimising market impact or earning spread, but more focused on getting a specific amount of risk executed as quickly and efficiently as possible. Many providers offer products that are named ‘Get Done’.

2.       Opportunistic – this group is anticipated to include an array of products, that don’t have a specific benchmark they are looking to achieve. Algos falling within this group are expected to include the array of products in the market that are seeking to maximise spread capture, unencumbered by a strict schedule dictated by a benchmark. Urgency is often a key parameter within this style, determining how passive the algo is prepared to be in order to earn spread.

3.       Interval Based – this group includes all algos that are attempting to minimise slippage to a benchmark where the algo slices the parent notional according to an interval, or time, based schedule. So, for example, this group would include all TWAP algos, the most commonly used algo within the FX market currently.

4.       Volume Based – we anticipate this group to be the most sparsely populated given the largely OTC market structure of FX. Products such ‘Percentage of Volume’ or ‘POV’ would fall within this style, which are algos attempting to execute in line with a specified % of volume traded within the market. This algo style has been adopted from the listed markets, where it is obviously easier to target a % volume target given the availability of volume data. In FX, any target will be approximate given the inexact nature of total volumes traded at any given point. VWAP, or Volume Weighted Average Price, algos would also fall within this style, but again, they are less common in FX given the difficulties in measuring actual volumes.

The proposed taxonomy is, by construct, a simplification of a complex ecosystem of algos, which will result in compromises to be made when categorising products. Hopefully, the additional Limit and Urgency fields, will help with the grouping to some extent. The objective of this exercise was to propose something simple, pragmatic and understandable, whilst trying to provide a decent representation of the current algo product array. The Algo Style field will be going live within the next BestX release. We recognise this is likely to be an iterative process, but we felt it was important to respond to client demand and take the initiative. We do not seek to impose our view on the market, but as always hope for feedback, and expect that this concept will continue to evolve over time as we seek to represent the majority view in the market

BestX Case Study: Identification & Management of Outlier Transactions

This brief article introduces the first in a series of case studies for the practical deployment of the BestX execution analytics software. Please note these case studies should be read in conjunction with the BestX User Guide, and/or the online support/FAQs.

The focus of this first case study is the identification of outlier trades, and the management of the workflow around identification, explanation and approval of such exceptions. We have explored this topic conceptually in an early article, ‘Red Flags and Outlier Detection’, whereas this case study further explores how to implement the concept in a practical context using the BestX software.

There is a clear consensus across the industry that the key element of any best execution policy is the process, and not necessarily individual outcomes of specific trades. The objective is to implement and monitor a best execution policy, and then refine it over time based on the iterative process of reviewing performance on an ongoing basis. A core component of this process is the identification of trades which are exceptions to the policy to help provide insights into where the policy may need refining. The BestX product allows an institution to ‘codify’ its specific best execution policy, allowing user defined thresholds to specify exactly which trades should be identified as exceptions.

At BestX we have now observed many different use cases for the exception reporting functionality, and not all are for compliance purposes. For example, systematic identification of particular algos that create significant market impact in a chosen group of currency pairs, may be a useful input into refining the best execution policy around algo selection. Common examples of exception reports include:

1.           Notification of any trade where the actual spread cost incurred is greater than the estimated BestX expected cost

2.           Identification of any trades that breach agreed commission rates to, for example, the WMR mid benchmark rate

3.           Identification of trades generating significant pre or post trade market impact

4.           Identification of any trades falling outside the trading day’s range

5.           For a matched swap portfolio, identification of any trades where the cost arising from the forward points exceeds either the BestX expected cost, or a defined threshold

6.           Identification of algo trades creating potential signaling risk, or significant market impact

Clearly, identification of outlier or exception trades is a critical component of best execution. It is also essential that when implementing such a process, flexibility is required, both in terms of which metrics you wish to monitor, and also the thresholds you specify for these thresholds, above which defines an outlier trade to your policy. We have learnt at BestX that there really isn’t market consensus or a ‘one size fits all’ approach to defining outliers across the industry.

To receive the full case study, please contact us at contact@bestx.co.uk

Volume Prediction in the FX Market – learning from Earthquakes and Tremors

Prediction is very difficult, especially if it's about the future
Niels Bohr

In the context of achieving best execution, there is a growing focus on the pre-trade stage of the process. Accessing the most appropriate information at the point of trade can help execution desks make informed decisions around their various execution choices, including timing and style of execution.

When making an informed decision at the point of trade, one key input is an understanding of the prevailing liquidity conditions, and if not trading via risk transfer, an indication of how those conditions will develop during the period that the order is worked in the market. This is not as straightforward as it sounds, even for a listed market where liquidity data is readily available. For a market such as FX, which has a complex, hybrid market structure, with liquidity fragmented over many venues, it is extremely difficult.

Hence the need to look for help outside of traditional financial theories and practices. At BestX we are always creatively looking to solve problems by thinking laterally and learning from other disciplines and industries.

We are aware that the statistical properties of trading activity in FX exhibit clustering behaviour, similar to volatility, in that periods of high trading activity are more likely to be followed by another period of high trading activity. This ‘memory’ effect is evident, for example, around key points in a trading day such as the release of key economic data, times of key benchmarks such as WMR etc. This behaviour led us to exploring a statistical framework that was first used to model the impact of earthquakes in terms of predicting the frequency and magnitude of subsequent tremors. This is analogous to the observed characteristics of trading activity and volumes, i.e. there is a ‘shock’, for example an unexpected interest rate rise from a central bank, which results in a significant spike in trading volume, which then proceeds to have ripple effects throughout the market over an ensuring period of time. Clearly, there would be significant value if it would be possible to have some form of estimate of how the ensuing liquidity conditions would evolve, both in terms of magnitude and frequency.

Before we explore such a framework further, lets just return to why this should be of any use whatsoever? Well, for example, if you are looking to execute a large block of risk, say 1bn USDJPY, and have discretion on the execution approach and timing, then having some idea of how volumes in USDJPY may evolve over the next hour could be a key input in your decision making. If you’re objective is to minimise spread cost, and you are therefore leaning towards selecting an algo designed to be passive in the market and ‘earn’ spread,  key variable for the performance of such an algo will be the depth of liquidity available to the algo whilst it is working. If, from previous experience, you know that the algo you are planning to use performs particularly well in market states characterised by high volumes, then a prediction of volumes will help decide whether to press ahead with this strategy or perhaps select a different product. Or, you may be seeking to minimise market impact and wish to time the trade to coincide with a high volume period to help ‘hide’ the trade. Insights into when such periods may occur over the next few hours is clearly helpful in this decision making.

Back to earthquakes. A statistical framework called the Hawkes Process has been used for some time for modelling earthquakes. This framework relies on historical information but is ‘self exciting’, which is the technical term for describing the phenomena where when an event occurs, e.g. an earthquake, then the probability of more events occurring, i.e. tremors, increases. From a volume prediction perspective, this can be thought of as a spike in trading activity due to a data release, will generally increase in increased subsequent trading activity. Over time, assuming there aren’t any further events, then the trading activity will revert back to its normal state. The Hawkes Process attempts to capture this behaviour.

As a brief aside, just to further clarify a key point – we are not attempting to predict an actual earthquake, or market shock. We are trying to predict what the likely impact of such an event will be. We were once asked if BestX had a model for predicting ‘flash crashes’, the query was met with an incredulous pause. If we had a model for predicting flash crashes then it is highly likely that the BestX business model would be very different, and probably involve relocating to the Cayman Islands and making lots of money.

Going back to Hawkes. Importantly, the hierarchical nature of the framework allows for the capture of the effect of different volume distributions typically observed on different days of the week. Also, the amount of trading also varies, with the quietest day of the week usually seen on Mondays, and Wednesdays showing, on average, the largest volumes. The charts below display the average intraday volume profiles for USDCAD over the course of 2016, also highlighting how different the pattern is on days such Non-Farm Payroll days, where trading activity is concentrated around the time of the data release.

So, does it work? Well, still early days but the initial results look very encouraging. The charts below show the predicted volume profile compared to the actual profile (black line), and the shaded area represents the 95 percentile confidence interval for this prediction.

We are now going to put the research into a development environment which will allow a more systematic testing of the framework, across many more currency pairs and using a much deeper data set to ‘train’ the framework. In addition, we wish to model other key event days such as ECB, FOMC meetings days, and also month, quarter and year-ends. Assuming the framework continues to perform to our satisfaction, we will then look to incorporate this within the BestX Pre-Trade module at a later date.

Measuring the impact of the Turn in FX markets

“Turn and face the strange”
David Bowie

In our latest paper we continue our research into measuring the impact of the ‘turn’ within the FX markets. This somewhat strange phenomenon, which manifests itself around key dates throughout the year, is generally caused by supply and demand for funding by large financial institutions, which can create dislocations in the forward curves for certain currencies.

In this latest research we empirically measure the impact of the turn around a range of different dates, including year, quarter and month end, but also event days such as NFP, FOMC etc. In summary, we find that the impact is most significant for year and quarter ends, with, for example, an average magnitude at year-end of between 0.6 and 1.5 bps for EURUSD depending on the tenor of the transaction. The work has helped us prioritise where to adjust the forward curve interpolation to better estimate mid for broken dates.

To receive a copy of the paper please email us at contact@bestx.co.uk