Data catalog competition heats up as spending cools

As the world of market data becomes both more voluminous and complex, providers of technology platforms known as data catalogs—which track what datasets a firm already has access to, and some of which also provide listings of comparative datasets available in the market—are seeing greater interest in their offerings.

“The tide has changed dramatically in the last few weeks. We’re having much more senior conversations now,” says Richard Clements, founder and CEO of VendEx Solutions, which provides—among other things—a detailed data catalog of vendors and their services.

Driving this renewed interest are not only factors such as ever-increasing data complexity, but also economic forces making it harder to manage market data.

Keiren Harris, a Hong Kong-based market data consultant and principal of MarketDataGuru, says the need for data directories is a result of financial firms’ spending and cost-cutting decisions. This year’s economic uncertainty, interest rate hikes, bank collapses, and geopolitical concerns such as the wars in Ukraine and Gaza have prompted firms to tighten their purse strings.

Banks now are not only saying they need to control costs but ... to cut costs. And often, the first thing to go is the market data managers

Keiren Harris, MarketDataGuru

“Banks now are not only saying they need to control costs, but—where some parts of their business aren’t making money—to cut costs. And often, the first thing to go is the market data managers who know what they’re doing,” Harris says. “So, people have to cope with far more complicated products than five or six years ago, but don’t have the experience or training. And when you’re dealing with 300 or 400 vendors, that’s too much.”

Another force driving greater interest in data catalogs is the recent news that Boston Consulting Group’s Expand Research division is building a comprehensive data catalog with six tier-one banks. The service will allow users to better manage the data they already license in-house, and to identify comparable datasets to meet specific end-user needs.

Eddie Molloy, director at Expand Research, says the firm is 90% done building the catalog, and is on track to release its API to the founding member banks by the end of October.

Though banks are excited about the transparency it will provide into vendor offerings, making it easier to find the right datasets to fulfill requests from data consumers on the business side, and giving them leverage over competing vendors during licensing negotiations, Molloy says a key driver for developing the catalog was the ability for firms to be more efficient and to leverage the vendor and product taxonomy Expand had already built to support its market data benchmarking service.

“Our catalog leverages the same taxonomy we’ve used for our benchmarks for well over a decade. This taxonomy and naming convention has been internalized by multiple firms and is also the foundation for the catalog,” Molloy says.

This means firms can have multiple services—such as inventory management, plus Expand’s benchmarking and other insights running on the same language as the catalog.

“If a firm uses another catalog, that’s another tool using another language, and that’s fine—however, it’s not efficient and effective for all potential users. It may be OK for experienced market data people who are used to ‘translating’ between systems. But, firms plan to use our tool not only for the market data team; they plan to roll it out organization-wide,” Molloy says.

For example, some banks will provide access to sales professionals and traders for self-serve needs. Molloy says that these users are not interested in minutia, such as how long a management team has been at a particular vendor or when they were established. “They want to know what they can get access to internally, and if they can’t get access internally, how they can get access to that dataset, and what the approval process is,” he says.

The front office wants to find new datasets that can deliver alpha, and they want that information in a language they understand and doesn’t need translating. The thinking here is that the catalog will save market data teams “significant time and effort, as they are no longer channeling every single inquiry,” he says.

In response to the positive reaction, Expand has also extended the deadline for firms to become founding members in the project to the end of this month for firms that are existing clients of Expand’s benchmarking service, which already uses the proprietary terminology that underpins the catalog. And it is using conversations with other interested parties to help shape how organizations beyond its tier-one founding members can access the data held within the catalog.

“Most of these don’t have an internal GUI to plug our API into, so we’re in active discussions with the firms to ascertain what specific functionality and user experience they require from a GUI, ahead of our own release. So, we’re using those conversations to conduct market research around developing our GUI,” Molloy says.

New tricks, old dogs

But while Expand may be the newest entrant to the data catalog space, it isn’t the only game in town.

In fact, London-based 3D Innovations (3Di) began building its Profiler catalog 15 years ago, in 2008. Originally built as a tool to help market data analysts and procurement staff, it allows firms to monitor the data assets they already license, and also to investigate the plethora of other datasets available in the market. Like Expand, Profiler was created to support 3Di’s other businesses—specifically, consulting on cost optimization and data governance projects, and their need to compare different products.

The economic arguments in support of data catalogs are essentially this: someone tasked with buying data within a firm to support a specific end user or business function may not have full visibility into all the data the firm already buys. Armed with that knowledge, that person may be able to find a dataset in house—and already paid for—that meets their needs. Or, they may be able to identify cheaper alternative datasets to what they already use or to what they were considering buying.

3Di’s Data Compliance & Dependency Manager (DCDM) provides a container that can be populated with data to create an internal catalog of what a firm already has in house, and enrich that with information such as policy changes that may affect the price of data or how a dataset can be used.

This industry needs a system like this. And it’s costly and time-consuming for each firm to build that on their own.

Richard Clements, VendEx

But doing that is no easy task, says Stephen Veasey, co-founder, and chief finance and operations officer at 3Di.

“Taking the data directories of all the vendors and their datasets and securities would be impossible. So, maybe you have an internal catalog and you link that to an external one. We’re having conversations around that and how to make them work together.”

The vendor also uses APIs to make it possible for firms to work with Profiler’s content, and use it to enrich their existing directories or inventory platforms, even if they don’t use the product’s interface.

“Profiler may not be everyone’s cup of tea in terms of the way it’s rendered, but they can pull that data in through our APIs so that they can unlock all the naming and definitions,” covering the 3,000 vendors and nearly 6,000 data products and related applications contained in Profiler, Veasey says.

Not-so-wild west

In comparison, San Francisco-based data technology vendor VendEx Solutions launched its VSource digital data catalog a year ago, and now has more than 4,000 vendors listed on it, and has cataloged—which is to say, has compiled a full listing and assigned each product and dataset a distinct proprietary identifier to—680 of them, covering 5,548 products.

Like 3Di, VendEx has a product already up and running. Unlike Expand, VendEx hasn’t signed up a consortium of banks as backers, yet. But VendEx has collected input into the system from 18 of the largest financial institutions and vendors.

VendEx’s aim is to build “an ecosystem to bring vendors and consumers together,” as a “for-profit industry utility to minimize frictions in the industry,” Clements says. “We’re looking for the industry to own the catalog and, more importantly, our patented identifier—think of it as the barcode for data; not just for financial services data, but all data—because the ID has the potential to solve more industry problems.”

For example, he envisages the identifier being used in the future to alleviate contractual burdens, and to fully integrate invoice reconciliation processes with third-party inventory management systems, and can be combined with its Usage Rights Code to determine permitted and prohibited usage rights from contracts.

VendEx has focused on allowing data vendors to access and upload their information into the system. On the end-user side, it is initially targeting data management professionals, though as it builds out the system further, it will focus more on end users on the business side who are the actual consumers of the data, just as Expand plans to do.

Any data entering VendEx’s catalog is assigned a detailed numerical identifier that allows VendEx to define a dataset down to the level of individual data fields. For example, some currency datasets may list the data based on the currency pairs, but not by individual fields, such as whether a price is a bid, offer, or midpoint, or what dealing desk the price was sourced from.

Vendors upload their own information, including product descriptions, via the vendor’s VLink gateway, which are then edited by VendEx’s analysts to the company’s own style and to standardize descriptions for easier comparison. Clements says VendEx has a team of seven analysts building the catalog and its content. Users can also request vendors and products that they want added to the system.

“Every vendor has a vendor page, and every product has a product page. We have a strict style book for every vendor and product, and we remove all marketing language,” Clements says.

No one has the silver bullet for these organizations. And if they want to raise their game, they may need to work with several of us.

Stephen Veasey, 3Di

Vendors can validate the descriptions, and can also map products to users, use cases, and job functions, as well as to specific articles within regulations to which their products apply. This allows users to search for products based on, for example, what would be useful to someone in a specific job function, or who needs data to fulfill a certain regulatory requirement.

A key component of VendEx’s solution is making information about datasets available to those who need it, while shielding it from those who don’t. For example, Clements says vendors can secure sensitive data, such as interdealer brokers who don’t want rival IDBs to know what data fields they offer, behind a protective wall. Similarly, index providers can secure components of their indexes and related documentation, and only make them available to those who are permissioned to see them.

“Almost my entire executive team comes from the vendor side, and we know there is considerable cost involved in creating and normalizing data products, so we’re ensuring that vendors can protect their IP,” he says.

Now, to be sure, only a certain segment of the market may need all those features. While MarketDataGuru’s Harris says everyone should avail themselves of a data catalog, not everyone needs an all-singing, all-dancing service covering company information, procurement, and inventory.

In some cases, data professionals and others just need a simple directory that lists who does what. This would include private-equity firms who want to understand the competitive landscape of vendors and comparable companies in a region that may be open to investment, or corporate treasurers who may need to understand sources of ESG data for their own compliance. MarketDataGuru, for example, provides one such directory, which users can access free of charge and set criteria using a series of drop-down menus.

Harris acknowledges that MDG’s catalog is “very basic,” adding that its main function is to attract potential clients to the company’s research and offerings, and that turning it into an enterprise-grade, premium product would require a big investment of time and resources.

“The products and services are constantly changing, ownership changes, so keeping on top of 4,000 different vendors is a big task,” he says. “Our directory is primarily for people to find names defined by business types, products offered, and geography.”

Common ground

One thing all parties agree on is that vendors and end users alike need to get their data houses in order, and that catalogs—especially those developed and run on a utility model—will play a key role in achieving that.

“This industry needs a system like this. And it’s costly and time-consuming for each firm to build that on their own,” says VendEx’s Clements.

“Everyone has been doing this in isolation,” says Expand’s Molloy. “But by working in isolation, are they built for success? We have a team working on this, but we take our guidance from the industry. We’re soliciting input from stakeholders, and I don’t think anyone has taken that approach before. It’s been designed with the banks and built for the banks.”

3Di’s Veasey agrees that working in isolation isn’t the best approach, but instead suggests that the solution may involve a mix of different providers’ systems working in collaboration to solve specific challenges.

“It’s quite frightening to build a central data catalog, so when we sell DCDM, we’re often also asked by the client to help populate it,” Veasey says. “If people aren’t already using a product like DCDM, they’ll have to build links between different enterprise data management platforms—and we’ve also been asked to do that. But that’s not straightforward: it’s a question of taking Excel spreadsheet inputs, reading Kafka queues, and SQL dumps.”

Moreover, if it’s hard for companies that have been doing this since 2008, it’s even harder—and without the economies of scale afforded by a vendor or utility model, more expensive—for firms to do this in-house.

“No one has the silver bullet for these organizations. And if they want to raise their game, they may need to work with several of us. We have clients doing that, and more looking at it and realizing that it’s expensive to build in-house,” Veasey says.

Whether it’s a result of an industry realizing its needs at the same time, or whether Expand’s announcement has generated a rising tide that raises all boats, it seems that a data catalog—whether an industry utility or an off-the-shelf vendor product—will soon be a critical part of any firm’s market data management processes.