BloombergGPT: Terminal giant enters the LLM race

Gideon Mann knows a thing or two about machine learning and natural language processing. Just to establish his bona fides: He holds a Ph.D. from Johns Hopkins University. He has authored over 30 publications and more than 20 patents in the field of AI. In 2007, soon after getting his doctorate, he joined Google as a research scientist where his team focused on R&D in ML and NLP, which included the development of Colaboratory. And in 2014, he joined Bloomberg, where today he is head of machine learning product and research, guiding Bloomberg’s corporate strategy for ML and NLP development, information retrieval and alternative data.

This is all to say that he’s an expert in the field. But there have been struggles. While at Johns Hopkins, his thesis was on machine reading comprehension. So, for example, there are four paragraphs about the Library of Congress and then three questions—who built it; how many books does it have; where is it located? The model would then try and answer those seemingly simple questions. It actually proved to be quite difficult.

Then, in 2020, OpenAI released GPT-3, which uses deep learning to generate human-like text, answer questions, translate languages, and perform a variety of other NLP tasks. (Editor’s note: That sentence was written by ChatGPT. Us writers might be in trouble.) It proved to be a game-changer—and the tipping point for Bloomberg developing its own large language model to power the Terminal.

“We’ve been doing NLP for a long time. The impetus to look at large language models came after the publication of the GPT-3 model,” Mann tells WatersTechnology. “For me, it was something I had worked on in my [doctorate] thesis, and being able to achieve progress on my thesis work using a language model was something I never thought possible; it changed my mind about what language models would be able to do.”

And with the release of a research paper on March 29, Bloomberg has entered the large language model canon with its own LLM, BloombergGPT.

While it shares the “GPT” moniker with OpenAI’s tool—GPT being short for generative pre-trained transformer—it is not a chatbot. Rather, BloombergGPT will help to assist—and provide more depth to—the Terminal’s sentiment analysis, named-entity recognition (NER), news classification, charting and question-answering capabilities, among other functions. Or, more simply, Bloomberg hopes it will become the engine that will supercharge the Terminal.

Shawn Edwards, Bloomberg’s CTO, tells WatersTechnology that, initially, the enhancements to the Terminal will be more behind the scenes.

“This is all very early days and we’re just scratching the surface—this is really Day One,” he says. “But the dream—and we’ve been working on this for years—has been for a user to walk up to the Terminal and ask any question and get an answer or lots of information; to get you to the starting point of your journey.”

Additionally, BloombergGPT will help make interactions with financial data more natural, Edwards says. For example, an existing way to retrieve data is via the Bloomberg Query Language, which can be used to interact with different classes of securities, each with its own fields, functions, and parameters. But, if you’re not proficient in BQL, it’s also challenging to learn a new programming language. BloombergGPT can be utilized to make BQL more accessible by transforming natural language queries into valid BQL, according to the paper.

So if a user inputs something simple, like, “Get me the last price and market cap for Apple,” BloombergGPT will produce the BQL output of, “get(px_last,cur_mkt_cap) for([’AAPL US Equity’])”.

‘Things we could never do’

The ubiquitous Terminal already has powerful search, analytical, and charting capabilities, but the belief is that this LLM will make users—such as analysts, portfolio managers and quants—more efficient by adding greater depth to their research, Mann says.

“The interesting thing about the LLM technology is that, out of the box, after you’ve trained on all of these datasets, it gives very good performance on tasks like NER. So part of it is the things we were already doing with NLP, we’ll be able to do better, faster, and cheaper using LLMs. And then there are probably things we could never do because the cost of setting up an engineering team and doing an annotation exercise was just too large.”

In the longer term, it really is going to help us sift through this flood of information for our customers so they can get the most pertinent, relevant, and timely information and calculation

Shawn Edwards, Bloomberg

To train BloombergGPT, the company constructed a dataset dubbed FinPile, which consists of financial documents written in English including news, filings, press releases, web-scraped financial documents, and social media drawn from the Bloomberg archives. These documents have been acquired through Bloomberg’s business process over the past two decades, according to the report. FinPile is paired with public datasets widely used to train LLMs, including The Pile, C4 and Wikipedia. The result is a training corpus of more than 700 billion tokens. (Edwards notes that client data has not been used—just datasets that are public, that the company itself has generated, or that it has contractual rights to use for AI development and training.)

While the team that built BloombergGPT started from scratch, Mann says open-source tools such as transformer-based language model BLOOM (no relation to Bloomberg) and non-open models like PaLM, Chinchilla, Galactica and Gopher proved valuable in helping the team to learn how to incorporate LLMs.

“Usually the machine-learning paradigm is, you build a model and you can run it overnight and you get all these little loops,” Mann says. “With this, it’s more like building a rocket ship that you launch, and you can make some point corrections, but you only get to launch it once. Looking both at their code and their experience was really important for us to understand how to build this model.”

This was one of the main reasons for releasing the BloombergGPT paper on arXiv.

“We didn’t use any of their models as starting points, but just seeing their experience was very important,” he says. “Partly, that’s why we want to release the paper, because there’s just not that many examples in the academic community of how you do this—when the rocket ship is in the air, do you need more gas or less gas?”

According to the company, BloombergGPT was validated on existing finance-specific NLP benchmarks, a suite of Bloomberg internal benchmarks, and broad categories of general purpose NLP tasks from popular benchmarks, such as LLM-specific benchmark BIG-bench Hard, Knowledge Assessments, Reading Comprehension, and Linguistic Tasks. Additionally, according to its tests, BloombergGPT outperforms existing open models of a similar size on financial tasks, while still performing on par or better on general NLP benchmarks, according to Bloomberg metrics (see table below).

Talk to me

Back to that whole chatbot thing.

Edwards says that right now the team is focused on building new products and features in the Terminal. Over the next six months or so, a lot of that work will be behind the scenes and not immediately noticeable to users.

Quite frankly, for all the amber coloring on the screen, users don’t tend to have a problem with the Terminal’s interface and the way it charts and displays information, so building a chatbot akin to ChatGPT or Google’s Bard is not a priority.

“I think that, over time, the way people interact with the system is going to change,” Edwards says. “And in the longer term, it really is going to help us sift through this flood of information for our customers so they can get the most pertinent, relevant, and timely information and calculation. And that’s hard—that’s a really hard problem. We work on a lot of systems to solve that problem. And this is a really great tool in the toolbox to help us accelerate that vision.”

Mann concurs: “It’s not a chat experience because so much of the things we produce are analytic and visual outputs. I think we’re going to really increase the ability of getting deep analytics and deep visualization through natural language.”

With that said, as Edwards previously noted, it’s very early days for BloombergGPT, and the team is “still exploring” what the LLM can do. So while it is not a chatbot, that doesn’t mean it can’t become one in the future.

“There are a lot of things we believe we can do, and we’ll be able to build products that we haven’t been able to build before,” Edwards says. “So, is that a chat interface? No, it’s probably more of a question-and-answer interface. Will we incorporate it into our chat system and have a chatbot? Well … we’ll talk to you soon about that.”

Additional reporting by Nyela Graham