Dreamware / Work / Mapping Ideas Worth Spreading — TEDx Topic Analysis
Mapping Ideas Worth Spreading — TEDx Topic Analysis
Built the scraping and analytics infrastructure that powered a landmark analysis of 24,000 TEDx talks across 147 countries, revealing the hidden mathematical structures connecting ideas.
The Challenge
Sean Gourley, co-founder and CTO of Quid, and ecologist Eric Berlow wanted to map the entire landscape of TEDx ideas — 24,000 talks across 147 countries representing 50 languages. The goal was to discover the hidden connections between seemingly unrelated topics by analysing the mathematical structures underlying the talks. Nobody had attempted to map an ideas ecosystem at this scale before.
John Jansen was working at Quid in San Francisco, building the company's scraping and analytics infrastructure. The TEDx mapping project was a natural extension of that work — but the scale and complexity of the data extraction challenge was a step beyond anything attempted before. Transcripts, metadata, viewer counts, comments, and engagement data needed to be extracted from thousands of sources, normalised across languages and formats, and structured for analysis. Without clean, comprehensive data, the network analysis and NLP work downstream would be meaningless.
Our Approach
John Jansen built all the scraping technology and analytics infrastructure from the ground up. This meant designing and implementing systems to extract transcripts, viewer data, comments, and metadata from YouTube and TED platforms at scale across 147 countries and 50 languages. The infrastructure needed to handle rate limiting, language detection, deduplication, and data normalisation across wildly inconsistent source formats.
Once the data pipeline was robust and delivering clean, structured data, John handed off to a team of data scientists who applied NLP to extract key concepts from each talk and used network graph theory to map the "meme-ome" — the mathematical structures underlying the ideas. The infrastructure JJ built continued to power the ongoing data feeds as the analysis evolved.
The Outcome
The work culminated in a TED Talk by Eric Berlow and Sean Gourley — "Mapping Ideas Worth Spreading" (watch the TED Talk) — presented to the main TED conference audience. The visualisation revealed unexpected clusters and connections: how education talks connected to storytelling, how social media linked to video games, and how the human brain bridged healthcare and technology.
The project demonstrated the power of combining robust data engineering with advanced network science. The infrastructure John built didn't just support a one-off analysis — it proved the model for Quid's approach to mapping ideas at scale, which became central to the company's intelligence platform.
Key result
Presented as a TED Talk, 24,000 talks analysed
Tech Stack
Related Services
Interested in similar work?
Tell us about your project. We'll have an honest conversation about what's possible, what it costs, and how we'd approach it.