Model-driven Data Management

Published on August 19, 2013 by Karthik Vijayapalan

Data-driven modeling has impacted both the buy side and the sell side in areas such as valuation, asset pricing, risk management, macroeconometrics, forecasting, investment strategy, investment analytics, and a slew of other data analytics. Without a shadow of doubt, optimal data management is going to be the single biggest differentiator that sets apart good research and analytics programs from the rest. In the current context, data management is much more than creating a golden copy of data for research by aggregating data from multiple sources. Today, data management also encompasses capturing derived data from models and adding the models themselves within the data framework. This is done to address the rapid evolution of models and model multiplicity. Models also point out gaps in data and help augment both data dimensions and quality. Since models feed back into data management, I call this ‘model-driven data management’.

The importance of data management has increased tremendously over the past decade, thanks to the influence of several different factors acting together.In the context of financial econometrics, the popularity of reduced-form models over structural models, owing to advantages in interpolation and forecasting, has led to the data dominance.Indirectly, this means that a richer research database will yield better models and, consequently, result in far more useful research and analytics.

Easy access to computing power and low cost of data storage has further accelerated this process. Unlike the models of physical sciences, there is no unique model that explains an economic phenomenon. Today’s preferred model may not remain so in a couple of years. Over the past decade, we have witnessed both evolutionary and revolutionary changes to models of financial data. In areas that involve decision making and risk, multiple models are combined using model-averaging techniques to eliminate model risk. Today, (model-driven) data management consists of maintaining a robust and flexible data platform, which also tracks all the models along with their regularly recalibrated parameters and implications.

Some leading investment banks and asset managers have realized the importance of model-driven data management and more importantly, the significance of running this in a process-driven manner. I find that this is increasingly becoming a core part of their outsourcing strategy. Some common themes that emerge based on our interactions with senior management of both buy side and sell side are as follows:

“We believe that the data that we have accumulated is our biggest competitive advantage. How do I institutionalize data management and also stay current with changing market conditions? Moreover, can I do so in a cost-effective manner?”

“Data management is taking up more than 80% of the time of our onshore researchers/analysts. How do I effectively combine process automations and offshore processes to increase the productivity of our onshore researchers/analysts?”

“Different sub-groups and functions within our group maintain different versions of data. We need to eliminate inefficiencies of database silos, data duplication, process redundancies, and data flow dead-ends. How do we achieve synergies of a central data management platform in a cost-effective manner? More importantly, how do we ensure data management converges with our group needs on an ongoing basis?”

“We have separate platforms for data management and analytics. We have also invested in a variety of third-party tools and systems for the same. Can we leverage our investments better by integrating these processes?”

There is one caveat that must be taken into consideration when outsourcing data management. A process-driven approach combined with technology is necessary, but not sufficient. To stay relevant and have a cutting edge, you also need to ensure that your outsourcing partner has the right people with capital markets knowledge and a strong grasp of the downstream uses of data and modeling. Expensive data warehousing projects, which are not model-driven, have often resulted in poor returns on investment. An understanding of the business data flow and research framework (model-driven data management) enables you to scale up your data management platform by applying lean principles, leading to a cost-effective solution. In essence, data management for research and analytics programs should be treated as a knowledge process to make your outsourcing strategy efficient and successful. This is important to effectively transform data into information, knowledge, and wisdom.

---

---