HeadlinesBriefing favicon HeadlinesBriefing.com

sllm democratizes expensive AI GPU access

Hacker News •
×

sllm addresses the prohibitive cost of running large language models by allowing developers to share GPU nodes. Running DeepSeek V3 (685B) requires 8×H100 GPUs costing about $14k/month, yet most developers only need 15-25 tokens/second. The service enables multiple developers to access expensive hardware without bearing the full cost individually.

Developers reserve spots with payment cards and aren't charged until their cohort fills completely. Prices start at $5/month for smaller models. The service ensures complete privacy with no traffic logging and provides an OpenAI-compatible API using vLLM, making integration straightforward for existing applications.

Currently offering a few models, sllm represents a practical solution for democratizing access to advanced AI capabilities. The cohort-based approach optimizes resource utilization while maintaining performance isolation between users, making expensive AI infrastructure accessible to individual developers and small teams.