We work at Alma Media, a Finnish digital services company developing AI solutions for real estate listings. Our system handles hundreds of thousands of annual photos that arrive without content descriptions. To support search, recommendations, and internal use cases, we automatically tag images across 23 classes, including room types like living rooms and kitchens, plus schematic content like floor plans. This is a classic multi-label classification task where spaces often contain multiple tags simultaneously.

Building reliable classifiers requires careful architectural decisions. While modern defaults often rely on third-party vision-language APIs, training open-source foundation models offers distinct advantages. We fine-tuned Google SigLip to solve an under-labeling problem and generate precise classification confidence scores. These scores are critical for determining whether background elements should influence results based on specific user queries.

Before committing to any approach, evaluate three core questions. First, determine if your task can even be solved via simple prompting versus requiring trainable weights. Second, assess whether you need calibrated confidence metrics or just raw predictions. Finally, consider infrastructure costs and data privacy requirements. Fine-tuning LoRA adapters resolved our labeling gaps, but whether this path suits your business depends entirely on these foundational constraints.