You guys need to read this because I'm obsessed with how smart their data decisions are! This is part four of Photoroom's PRX series, and they dive into what actually makes the model work under the hood: the training data pipeline. Their guiding principle at pretraining is breadth over perfection — a diverse dataset teaches the model about the world far better than one small beautiful corpus, because pretraining builds concept understanding while fine-tuning builds taste later on. They use a mix of public and internal datasets that already come curated for safety and quality rather than reinventing their own, which makes sense at this scale. One of my favorite details is their caption philosophy: they found that long captions — ones that describe every object and scene in detail — work far better than short summaries because the model learns each visual element as a controllable attribute you can call up by name later!
They also made some really sharp engineering choices about data formats. They use Lance for building and exploring the dataset because it has fast filtering and vector search, while MDS handles streaming during training since those two tools play different roles perfectly: assemble in Lance, stream from MDS. Instead of precomputing text latents, they compute them on-the-fly with Qwen3-VL — a 4% throughput hit that lets them keep their dataset small enough to fit entirely on an SSD cluster filesystem rather than streaming it over the network and also makes switching encoders trivial later. And here's my favorite technical win: they encode everything as JPEG at quality 92 because most real images are already JPEG-compressed, so storing losslessly as PNG is waste — they actually measured this across ten re-encode cycles on both high-res (1–2MP) and low-res images and the PSNR barely budged while saving huge storage. Their data pipeline approach shows you how to build a massive generative model without overcomplicating it at any single step!
Source: https://huggingface.co/blog/Photoroom/prx-part4-data
They also made some really sharp engineering choices about data formats. They use Lance for building and exploring the dataset because it has fast filtering and vector search, while MDS handles streaming during training since those two tools play different roles perfectly: assemble in Lance, stream from MDS. Instead of precomputing text latents, they compute them on-the-fly with Qwen3-VL — a 4% throughput hit that lets them keep their dataset small enough to fit entirely on an SSD cluster filesystem rather than streaming it over the network and also makes switching encoders trivial later. And here's my favorite technical win: they encode everything as JPEG at quality 92 because most real images are already JPEG-compressed, so storing losslessly as PNG is waste — they actually measured this across ten re-encode cycles on both high-res (1–2MP) and low-res images and the PSNR barely budged while saving huge storage. Their data pipeline approach shows you how to build a massive generative model without overcomplicating it at any single step!
Source: https://huggingface.co/blog/Photoroom/prx-part4-data