Google Says It Doesn’t Care About Tensor’s Performance In Benchmarks

Benchmarks have been the most reliable way to measure a chip’s performance for a long time, but Google says it has enough confidence in Tensor’s performance that it doesn’t matter if it’s not winning the benchmarks.

Speaking on the Made by Google Podcast, Monika Gupta, senior director of product management for Google Silicon Teams, highlights the company’s efforts to make its Tensor chips into a rival for Snapdragon and Exynos. Her team is intended “to focus on what [Google] need[s] five years from now” for its chips.

First, she discussed Google’s in-house approach to designing chips for its Pixel devices like Pixel 7 and 7 Pro . The company’s silicon team constantly talks with AI researchers to “know exactly where machine learning models are trending in five years.”

“I’m not making decisions based on where machine learning is today, and I can say that because I work at Google. Same with the software that our software team is doing. I know where the software team wants to take the user experiences five years from now.” Gupta said. “That’s the benefit of not being a merchant silicon supplier, but an in-house silicon supplier. So those trade-off decisions are very tough, but I think they get a little easier when you’re vertically integrated.”

Google believes end-user experience is more important than Tensor’s performance in benchmarks

The Pixel maker is now heavily focused on adding AI-driven features to its products, and that’s the approach that the company prefers instead of switching focus to benchmarks. Additionally, Gupta says Google is “perfectly comfortable” with not winning the benchmarks as they’ve prioritized the end-user experience.

Gupta argues that classical benchmarks are not meant to serve a purpose these days when AI innovations are heading to smartphones. She claims this approach- focusing on AI- can deliver helpful experiences.

“They may tell some story, but we don’t feel like they tell the complete story. And so for us what we benchmark are the actual software workloads that we are running on our chip and then we strive with every generation of tensor chip to make them better, whether it’s better quality, better performance, lower power.”