The Knowledge Tracker: what happens when the first version doesn't work
The challenge
Building knowledge is a real motivator for FT readers: customer research told us as much, over and over. But readers struggled with context: what should they read next, and why? More importantly, nobody could tell them whether their reading was actually building expertise on a topic, or just adding to a pile of open tabs.
This project brought together the FT, Google’s Digital News Initiative (which part-funded it) and CRUX, an Israeli knowledge-quantification start-up, around a shared hypothesis: give readers a visible “knowledge score” for a topic, and a compelling reason to come back and grow it, and engagement should follow. It also mattered that whatever we built wouldn’t patronise or annoy the FT’s notoriously discerning readership, which is exactly why we were brought in to run the discovery and lead the stakeholder negotiation alongside the design work.
What we did
We opened with a kick-off workshop to pin down the problem, hypothesis and vision with stakeholders from the FT, GDNI and CRUX, including a design-studio exercise where every participant sketched their own version of a “knowledge component” before we converged on shared direction.

From there we ran the design the way we run any evidence-based project: several visual variants, each tested with real readers and marked pass or fail against what we were actually trying to learn, not against personal taste, plus a separate workstream on how heavily the score itself should be weighted, so an active reader’s number moved fast enough to feel earned without becoming meaningless noise.
We tested the concept directly with FT readers over several rounds of interviews, then sat down after every round to debrief honestly on what we’d heard rather than what we’d hoped to hear.

That second round of feedback was the uncomfortable one. Readers didn’t dispute that the score was accurate. They disputed that it meant anything to them, or that they wanted to be told it at all. Notes from that session still make the point better than we could summarise it: “How can ten articles make me an expert”, “Does the score really represent the value to me”, “The phrase ‘knowledge quant’ is meaningless”.
The honest result
The first version, tested directly on ft.com, didn’t land. FT.com readers found a visible knowledge score more “in-your-face” than useful (several said directly that they didn’t like being told how knowledgeable they were) and the quantitative testing agreed: it wasn’t moving engagement for that audience. CRUX, understandably, wanted the component shipped regardless; it was a significant proof point for their start-up. We didn’t dismiss that pressure, but we didn’t let it override the evidence either. We combined the quantitative results with what readers had told us directly to make a case that was genuinely difficult to argue with, and the component didn’t ship on ft.com in that form.
That’s not where the story ends, though. The underlying concept was sound. It just needed the right audience. FT’s Sifted brand (a paid subscription site for founders and investors, where knowing your topic coverage is a much more natural, professional instinct than it is for a general news reader) adopted the same technology as the Knowledge Tracker. There, it worked: according to reporting on niemanlab.org, readers shown the widget increased their engagement by 55% (an average of 15.5 articles read versus 10 for a control group), and CRUX reported paid registrations from engaged users rising 16% over two months.

The partnerships, technology and testing rigour we built during the ft.com round weren’t wasted either: the same underlying recommendation approach now powers “read next” story suggestions on ft.com itself, just without the score readers had told us they didn’t want to see. And the testing standard we set for this project (rounds of variant testing validated against real reader behaviour, not stakeholder preference) became the default evidence-based process for FT product teams generally, well beyond this one component.
This is what “make decisions with evidence” means in practice: building something a partner was genuinely excited about, listening when the data (and the readers) said it wasn’t right for that audience, and finding the form where it actually worked, rather than either shipping it anyway or shelving the idea completely.
← Back to all case studies