Fool’s Gold: Data Mining Digs Up Explosive Errors
Recent studies reveal the prevalence of poor-quality data, exacerbated by increased use of machine learning that allows users to dredge far bigger datasets and identify spurious correlations.
A review of 100 major psychology studies, for instance, found that only 36 percent had statistical significance. Over half the alien planets identified by Nasa’s Kepler telescope turned out to be stars. And in preclinical cancer research, a mere six out of 53 breakthrough studies were found to be reproducible. Quantitative finance does not fare much better.
“It’s a gigantic problem—spurious results are the norm,” says Zak David, co-founder of analytics firm Mile 59, and former engineer of high
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: http://subscriptions.waterstechnology.com/subscribe
You are currently unable to print this content. Please contact info@waterstechnology.com to find out more.
You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.
Copyright Infopro Digital Limited. All rights reserved.
As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (point 2.4), printing is limited to a single copy.
If you would like to purchase additional rights please email info@waterstechnology.com
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5.
If you would like to purchase additional rights please email info@waterstechnology.com
More on Data Management
Tech VC funding: It’s not just about the money
The IMD Wrap: It’s been a busy year for tech and data companies seeking cash to kick-start new efforts. Max details how some are putting the fun into fundraising.
BNY uses proprietary data store to connect disparate applications
Internally built ODS is the “bedrock” upon which BNY plans to become more than just a custodian bank.
Waters Wavelength Ep. 296: Questions about data quality
It’s all about the data, data, data.
The AI boom proves a boon for chief data officers
Voice of the CDO: As trading firms incorporate AI and large language models into their investment workflows, there’s a growing realization among firms that their data governance structures are riddled with holes. Enter the chief data officer.
FactSet launches conversational AI for increased productivity
FactSet is set to release a generative AI search agent across its platform in early 2025.
If M&A picks up, who’s on the auction block?
Waters Wrap: With projections that mergers and acquisitions are geared to pick back up in 2025, Anthony reads the tea leaves of 25 of this year’s deals to predict which vendors might be most valuable.
ICE Connect adds data integration capabilities for proprietary data
Intercontinental Exchange’s desktop platform is collaborating with CloudQuant to allow customers to integrate in-house data and analytics with the datasets found on its ICE Connect platform.
MIAX taps DataBP for exchange data licensing, custom contracts
To support planned growth of its data business, the exchange group has implemented DataBP’s platform to strengthen its licensing process and scale up its distribution capabilities in anticipation of end-user demand.
