Measuring execution performance across asset classes
It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong.
Richard P. Feynman
Transaction cost analysis means many different things to many different people, depending on the role or seniority of the user and whether the user specialises in a particular asset class or has cross-asset class responsibilities. This article explores some of these key differences in the context of the challenges that they raise when measuring costs and designing an execution analytics platform for such a diverse set of users. We’ll discuss post-trade analysis for now, given that is where the demand is currently focused and already raises enough complexities, and leave pre-trade for a later date. This subject matter is particularly pertinent to us at the moment as we plough ahead with the build of BestX Equities.
Cost definition
One key area that causes significant confusion across the industry is simply defining what transaction cost actually is. Depending on a user’s background and asset class interest or experience, ‘cost’ can be computed in a multitude of ways. Some examples are provided below:
1. Equities – costs within this asset class are often defined as slippage to a VWAP over a defined period, or Arrival Price, measured at the time the order arrives in the market.
2. FX – VWAP is impossible to compute accurately, so many individuals define cost as slippage to a mid price observed at either market arrival time or completion time.
3. Fixed Income – Far Touch, defined as slippage to the observed bid or offer at the completion time, is often referred to as the transaction cost
Compounding the complexities of the different definitions that different segments of the market have grown up with is the issue of inconsistencies in available data to measure costs, both time stamps and market data. For example, the early adoption of electronic trading in equities and the rise of an order-driven market structure resulted in widespread availability of accurate and multiple time stamps for a given order. Hence, the focus on Arrival Price as a cost definition within the equity asset class as market arrival time stamps are widely available and have been for decades. In OTC markets such as FX and Fixed Income, such time stamp data is often not available. Of course, transactions executed electronically tend to have some form of market arrival time stamp recorded, although there is not an industry wide definition of exactly what this time stamp represents. For example, for an FX trade executed electronically via an RFQ protocol, different platforms may measure the market arrival time as the time the buy-side trader submits the order whereas some may measure the time that the broker ‘picks up’ the order, or when they actually provide a quote. Fixed Income represents an even murkier picture than FX, with a large proportion of transactions outside of liquid sovereigns, dealt over the phone, resulting in even greater challenges in recording time stamps. Thus, equities and the majority of spot FX aside, the concept of measuring to an accurate and consistent market arrival time stamp is still a pipe dream.
However, there is a need to measure ‘cost’ consistently across a multi-asset class portfolio, and not just for regulatory cost and charges disclosures or obligations. Heads of Trading wish to see the total cost across their desks and traders, CIOs and best execution committees require regular reporting of total execution costs of the business etc.
Solution – BestX Factors
At BestX we have taken the following approach to provide a solution to this problem. The diagram below in Figure 1 summarises the issue, whereby each asset class has its own specific benchmarks and metrics they focus on, but where possible, there are a subset of common metrics, that are measured on a consistent basis and therefore can be aggregated accordingly.
Figure 1: Cost/benchmark metrics across asset classes
In the design phase of BestX we focused on the need to allow users to select the execution factors they wish to measure, which are relevant to their business and the way they execute. This was also a requirement to ensure the product was deemed compliant for MiFID II. These BestX Factors, an example of which is shown in Figure 2, form a core part of the platform, and provide the ability to view multiple, alternative views of ‘cost’ in the broadest sense.
Figure 2: Example BestX Factors
The two BestX Factors relevant to this particular discussion are:
1. Spread
2. Price
We have defined ‘Spread’ as our core cost metric, which is measured to an independent mid-price observed in the market at the completion time of the trade. This metric sits at the centre of the Venn diagram above, and allows a user with cross-asset class responsibilities to aggregate a measure of cost. Why did we choose completion time? For the reasons described above, generally the one common time stamp that we are provided across all asset classes is the completion time (although even this may be subject to gaps and errors, especially from, for example, sub-custodians). We do provide clients with the option of having everything measured at an arrival time stamp, if this is available for all of their trades across all asset classes, although in our experience this is a minority at this stage.
The inconsistency of time stamp availability also creates issues for other metrics, such as Implementation Shortfall. The ‘classic’ view is to measure slippage with reference to the earliest time stamp, which is typically ‘order origination time’, i.e. when the portfolio manager first raises the order. This is typically only available from the Order Management System (OMS) of the client, as it is not passed through to the Execution Management System (EMS) or executing broker. However, it is rare for an OMS to have all the details of individual fills, e.g. which venue an order was filled on, passed back to it and stored. This raises additional complications in that data needs to be stitched together from multiple sources if the user is to get a complete picture.
Another option is to measure with reference to an arrival time stamp, but as already discussed, this is not always available. Indeed, for one institution, they may have such origination or arrival time stamps available for some trades but not others. When wrestling with this issue for FX when building BestX in early 2016, we took the decision to effectively ‘reverse’ the shortfall measure, and go back in time measuring slippage to the completion time, which was the only point we knew clients could provide us. This at least allowed us to calculate measures that could cope with missing or incomplete trade lifecycles. Not ideal but a purely pragmatic decision to cope with gappy data and as these gaps fill in over time we will revisit this definition.
It is also essential that the common cost metric is computed in a way that can be aggregated across asset classes, for example, always convert into basis points which is a common unit (rather than pips or price cents etc). This issue is particularly complicated in Fixed Income where users may wish to see costs computed in a variety of ways e.g.
i) Computed on price, but measured in cents,
ii) Computed on price, but measured in basis points,
iii) Computed on yield, as a number of fixed income sectors trade on yield rather than price.
We then use the ‘Price’ factor as a way of accommodating all of the other cost measures that specific asset classes require e.g. VWAP in equities. Clearly, such metrics cannot necessarily be aggregated across every trade, so these metrics tend to fall in the other segments of the Venn diagram in Figure 1. In some cases, there may be commonality for some benchmark measures e.g. a client may have market arrival time stamps for their equity and FX trades, but not their fixed income trades. It is therefore also key that the factors can be configured by asset class to allow such aggregation of a subset of trades.
In this way the user should be able to ‘have their cake and eat it’ in terms of common aggregation and also asset class specific cost metrics. Why is this so important? The diverse array of users of TCA software at an institution necessitates such flexibility e.g.:
- A bond trader will want to see Fixed Income specific measures, such as cost computed on a yield basis, or slippage to Far Touch on an individual trade level, whereas the Global Head of Trading at such an institution may want to see aggregate summary reports of costs across all asset classes, whilst retaining the ability to drill into the Fixed Income specific view if required.
- Users within oversight functions such as Risk and Compliance will typically wish to see metrics computed in such a way that provide meaningful exception trades, but they also need to ensure that when such exceptions are discussed with the trader responsible the conversation makes sense to both parties.
- Regulatory and client reporting teams require consistent cost metrics that allow portfolio costs to be computed and aggregated accurately. Such teams will have less interest in idiosyncratic asset class specifics.
In conclusion, it is clear that building a cross-asset class TCA product is more complicated than perhaps you’d first think. The devil really is in the detail and it would already be a very large can of worms without all of the issues caused by inconsistent and missing data across the industry. Starting with a purist view to design a theoretically beautiful product is obviously a laudable initial objective, but this quickly starts to unravel when you have to get it working in the real world, dealing with many different users who have very different requirements whilst also trying to consolidate and normalise extremely messy data. This article explored some of the trickier areas where pragmatic solutions have been required to try to solve for some of the problems that have arisen during our development process. The ultimate objective is obviously to deliver a product solution, that caters to a diverse user base whilst providing the ability to measure costs consistently across FX, Fixed Income and Equities. We are on target to deliver on this objective by the end of the year.
