AI chip startups can challenge the sector's Goliaths by building substantially better hardware for emerging inference demands, much as Juniper challenged Cisco with a new architecture suited to the internet’s rapid growth. To win customers, writes guest author R K Anand, co-founder and chief product officer of Tensordyne, challengers must prove their advantages, make switching easy and use small, agile teams to anticipate needs that incumbents have yet to address. The pace of change in AI outruns anything we saw in previous tech cycles: Model generations turn over in weeks, not years.
The hardware beneath it has to keep shifting to serve them, and designing from first principles has real merit. Lessons from Davids that came before Hardware suppliers of the dot-com era followed the same peaks and troughs. Cisco, founded in 1984, dominated internet routing for most of the decade, and in March 2000 it was the highest market capitalization company in the world.
Then Juniper Networks, a startup where I was a founding engineer, shipped its first core router in 1998. Juniper went from a fairly anonymous upstart to 17% of the core router market at the end of 1999, then again to 38% in early 2021. Challengers now can lean on history for some guidance.
First, don’t build a cheaper version of the incumbent’s product. Build a far better one. At Juniper, our bet was that the then-adolescent version of the internet was going to demand an exponentially higher throughput than before.
Introducing an entirely new architectural approach to how routers are built, along with an innovative design philosophy focused on performance and reliability, we didn’t set out to sell a better router, so much as redefine what a core router had to be. The boom in AI inference is today’s version of that strain on throughput. The initial hyperfixation on training AI models is fading, and the industry is collectively moving on to using them — known as inference.
Inference is where AI’s promised productivity gains actually materialize, putting trained models to work across every task. As the enterprise applications for AI grow and become more complex, so too does demand for inference compute. Google recently reported that it now processes more than 3.2 quadrillion tokens a month, up from 480 trillion a year earlier, as AI moves from chat to agents that plan and act on our behalf.
Serving tokens at scale is a different beast from when it was just for model training, with different constraints on memory, power and cost. The core principle we use applies to all sectors: Challengers have a better chance at optimizing for the market that will come, not the market that is already well-serviced by the incumbent. Be known by your track record Challengers start without the incumbent’s biggest asset: a reputation.
Buying from the market leader is the safe choice, so a startup has to replace that trust with evidence.
Source: Crunchbase News · Summarized by HeadlinesBriefing