This seems to overlook the massive (seriously, massive) limitation of requiring kernel/NIC/switch cooperation that hardware doesn't support at all yet. That's a huge level of coordination from theoretical hardware that just doesn't exist. And if you're going to map messages to priority schedules/queues, you'll have to map that somehow, with some large buffer/switch ready to prioritize smaller messages over large ones, etc. And then you're trusting that the grant/request round-trip machinery is lite enough to not create a new set of bottlenecks that supersede the existing RoCE implementation... it's probably possible to make Homa more efficient than sender-side protocols, but I'm really skeptical that it's worth the effort to get there instead of just continuing to scale up hardware to handle sender-side bottlenecks.
Source: Hacker News · Summarized by HeadlinesBriefing