Marketing advisory, strategy and analytics company Forethought has released a new white paper addressing the current role and limitations of synthetic data in commercial decision-making.
Led by Professor Ujwal Kayande from Melbourne Business School, The Capability of GPT-4o in Predicting Consumer Choice tested 1,000 synthetic respondents against a human sample size of 204 respondents, across several choice models pertaining to aviation and mobile plans.
The findings of the study demonstrate how synthetic data can support certain types of management analysis, but warns that without sufficient human guidance, ChatGPT-4o currently produces nonsensical synthetic data that is unsuitable for complex decision-making.
Presented during Forethought’s recent Gen AI in marketing webinar, the white paper provides the following key insights:
- Both synthetic and human datasets achieved over 70% predictive accuracy in replicating choice preferences across test models.
- But synthetic data showed only half the variability in attribute impact scores compared to human data, impacting its ability to capture consumer nuances.
- Testing showed that synthetic and human data led to different pricing recommendations in 45% of simulations.
- Human and synthetic data agreed on the top-ranked attribute in three out of four choice models, but deviations appeared in detailed preference patterns.
- Synthetic data is shown to be effective in tasks such as cross-tabulation and augmentation, providing efficient data solutions in scenarios where nuanced consumer variability is less critical.
- For high-stakes tasks like consumer segmentation and behavioural prediction, synthetic data may fall short, often failing to replicate the diversity and variability that real human data provides.
- Synthetic data can inherit and amplify biases from the original datasets used to train models, which can undermine decision-making in applications requiring unbiased insights.
Professor Kayande said: “There’s immense potential in synthetic data, but it’s essential to understand where it aligns – and doesn’t – with human data.
“With advances in AI and technology there is enormous temptation for brands to lean on synthetic data for fast and seemingly practical insights, but without fully understanding its capabilities, brands risk basing critical decisions on incomplete or overly simplified data.
“This study was designed to address that knowledge gap, offering guidance on where synthetic data can drive value and where it may fall short. Our goal is to equip brands with the knowledge to use synthetic data effectively, ensuring it enhances rather than compromises their decision-making,” he said.
Forethought Executive Chairman and Founder, Ken Roberts, said: “Synthetic data holds enormous potential for scaling insights, but we have proven that it must be applied in a highly discerning manner.
“We’re committed to helping brands understand both the advantages and limitations of synthetic data, namely that it must serve as a complement to human insights rather than a substitute. That is, at least in the technology’s current form. As Gen AI continues to evolve, our goal is to guide brands toward making choices that drive genuine value and meaningful outcomes.”
Forethought’s flagship platform, BrandComms.AI, and continued investment in data-driven research underscore its dedication to equipping clients with informed, strategic insights in a rapidly changing market.
