An app called Harbor will help turn multiple Macs into a private AI inference server, keeping data local while making the process of creating an AI server much easier. The Mac has become a very useful tool for AI research, but it could be even more useful. Apple is working on an app that can make it easier to run large language models (LLMs) locally, so long as you have enough processing power and memory available.
Formerly known as Fabric, Harbor is the app that creates a private AI inference server on a Mac, according to Aaronp613 on X. It is capable of running open-weight models on the Mac itself, and apparently is extremely user-friendly. Apple is working on a new app called Harbor that turns your Mac into a private AI inference server.
Run open-weight models locally on Apple silicon -- your data never leaves your hardware, and nothing reaches the cloud. Connect multiple Mac computers over Thunderbolt 5 to pool unified memory, enabling large models and higher concurrency that a single computer can handle alone. Set up in minutes, not hours Harbor automatically detects your devices over Thunderbolt 5 to form a cluster.
Plug in, start, and you're running -- no manual configuration, no command line, no infrastructure expertise required. Download and run models directly Browse, download, and load open-weight models from within the app. Harbor handles model management so you can focus on the work, not the setup.
Cluster for scale Connect two or more Thunderbolt 5-capable Mac computers and Harbor pools their unified memory into a single inference resource. Run models that exceed any single computer's capacity, or serve more concurrent requests across your cluster. Harbor is currently in a private preview under the Claris brand.
A massive help for AI development Like pretty much any new technology trend, it can be difficult to get into working with AI. Especially if you don't want to deal with cloud-based services. Not everyone wants to simply hand their data over to Open AI or Anthropic in exchange for their services.
Making a local LLM server eliminates that, but at a fairly high financial cost of new hardware. But at the same time, there's also the work that needs to be put in to get the server up and running. If you have multiple Macs available, clustering them together to handle tougher w...
Source: AppleInsider · Summarized by HeadlinesBriefing