Investment managers operate in one of the most competitive and highly regulated marketplaces. Gaining an advantage over the market has huge pay-offs, but achieving an advantage within the sometimes shifting rules is not always easy. As firms modernise their investment processes to incorporate new analytical techniques and new sources of data, careful planning and oversight is recommended. In recent years, new information sources that potentially contribute to information advantage over the market have emerged. This information is called alternative data and often requires artificial intelligence to unlock its true value.

Figure 1. Alternative data

Understanding alternative data

Alternative data most often originates as subset of “exhaust data”, as identified in Figure 1, which is data generated by processes that have a primary purpose other than generating data. Some alternative data is sensor derived, for investment purposes, but these datasets are in the minority. Most of the exhaust data remains, underutilized and not monetized, until its value is discovered by the originating firm or some enterprising investment firm looking for new ways to estimate key performance indicators in their investment models.

Alternative data comprises of all digital data such as climate, geo-location, employment, internet traffic and activity, mobile app downloads, consumer sentiment, email receipts, cargo movements, retail credit and much more. Research signals from this data can be extracted to tailor investment strategies across geographies, industrial sectors, commodities, asset classes and to design specific investment products. Navigating this infinite universe of data and converging on the most appropriate data sets is an iterative process that is a function of time, budget, and creativity. Discovering alpha requires a balanced mix of research on data sets, due diligence on data providers, a mature alternative data infrastructure, and a dedicated alternative data team comprising of data scouts and data strategists.

Implementing alternative data

The investment process is the primary mission critical process for active investment managers. Changing that process to incorporate alternative data, even as a supplement to the existing process, invokes the resistance-to-changes behaviours that are present in most organisations. Getting started can be difficult, especially without taking the time to build consensus, and bake alternative data into the firm strategy and vision – in the unique way appropriate for each individual firm.

Figure 2. Alternative data adoption

The process of implementing alternative data can be divided into the eight steps in figure 2. In the first three steps, fundamental investment firms getting started with alternative data should follow the principle of “the power of with,” emphasising that alternative data is a tool for them, not a replacement. In steps four through eight, firms are making alternative data work for them and their unique approach to generating alpha with alternative data, while also managing risk. The process can vary from fully integrated input to investment models to stand apart signals based on the alternative data analysis, that can be taken into consideration by human decision makers.

Processing alternative data

Managing unstructured data by itself is a complex task. Many raw alternative data sets are unstructured and require significant processing before they can digested by investment oriented AI algorithms. Once the data sets have passed this normalisation process, quality control and back-testing of the data set can begin in earnest, as displayed in figure 3. Depending on the processing engine, insights into the data, trading signals and portfolio construction rules can be applied. Leading practices have these steps integrated into one application, reaping some quality and efficiency benefits.

Figure 3. Alternative data processing 

Once the raw data set is cleansed, processed and ingested into the target investment model, data scouts would look for patterns and correlations. Alternative data findings could be used to either forecast publicly available but infrequently updated data (e.g. GDP, Census, Employment Data etc.) or to nowcast economic indicators that may not be publicly available, such as revenue figures for unlisted entities, market shares or anticipated price points for future products. Alternative data may serve as a proxy or complementary data for investment strategy depending upon its correlation with traditional data.

The next step is to design the technique to transform the patterns into signals. Differing techniques adopted by two investors accessing the same data set would lead to entirely different outcomes. For example, a linear regression model cannot exploit non-linear relationships in the data that a deep learning model would naturally do. The techniques could employ a diverse mix of investment mandates, time horizons, style and risk appetite for each portfolio manager. Alternative data, and the associated processing,  can be wielded like a tool, supporting the myriad of unique approaches to investment decision-making that portfolio managers use. While larger investment firms may allocate resources to discover alpha through alternative data in-house, others may buy research signals from an aggregator who manipulates alternative data sets on their behalf.

The potential results of using alternative data

Using alternative data is not without its share of challenges. The early entrants faced data risks due to immature risk control processes at data providers such as data provenance risk, data accuracy/validity risk, privacy risk and material non-public information (MNPI) risk. The alternative data ecosystem is maturing, but these initial risks may still arise. Many firms implementing alternative data face model risks, talent risk, and regulatory risk. At the same time, the late adopters run the risk of strategic positioning, strategy execution, reputation, and flight of capital as market expectations shift to requiring alternative data capabilities.

So why go to all the trouble? Because, alternative data, properly managed, can yield information advantage and alpha. There are numerous examples where the use of alternative data improve predictability of financial outcomes (earnings for individual companies and macro-economic metrics) over analysts' consensus expectations. When properly matched with portfolio management strategies, information advantage can generate alpha. Satellite imagery, for example can be analysed in many different ways that generate information advantage. A notable example: images of light intensity at night can inform export activity on a country level, especially for countries with less reliable economic activity reporting. Export activity can be linked to forecasting GDP, or when other data sets are linked, individual company performance, such as shipping firms or large manufacturers.

When coupled with a specialised investment management AI engine, alternative data and traditional data can be back-tested for alpha routinely. Figure 4 shows the results of one such back-test. Multiple portfolio management and trading parameters can be set in the application. This example is a simple risk-on/risk off trading decision on the S&P 500. The alternative data added to supplement traditional market data is consumer sentiment, derived from Zillow activity and consumer transactions. The back-test of the data and AI in this case show the ability to predict major down-turns effectively, based on a weekly holding and decision period.

Figure 4. Risk-on / risk-off back-test results of AI generated trading signal for alternative and traditional data


This example is just one of the many that are currently supporting information advantage, and in production would warrant additional quality reviews and perhaps running in simulated production.

Data available for analysis is rapidly growing, technical processing capability is growing just as fast, and the human creativity required to develop tradable insights is also advancing rapidly. The question for investment managers is transitioning from a state of thinking about whether alternative data be used to support active management strategies, to how to develop this capability fast enough to keep up with the competition.

Deloitte helps firms to reap the full benefits of alternative data by aggregating cost-effective access to alternative data sets, offering skill sets to clean and normalise data, and mine alpha. We also help companies build the right foundation for risk management and regulatory compliance to support the use of alternative data.

With thanks to my US colleague Doug Dannemiller, and my Canadian colleague Avinash Kumar who co-wrote this blog with me.