Text as an Image, Then a Magnifying Glass

Strange but true: “A growing line of work compresses text by rendering it as images for VLMs.” That's the premise of the LensVLM paper. Its Figure 7, shown here, packs the same passage into three image sizes. By 15× compression, the text is barely legible.

Why it matters

A long document consumes context even when the answer needs only one part of it. Rendering the text as images gives a vision-language model a smaller initial view. LensVLM scans that view to pick a relevant region, then uses an Expand tool to read the retained source text. The model doesn't have to reconstruct unreadable letters from pixels.

Key technical notes

The authors report question-answering accuracy comparable to a full-text baseline at 4.3× effective compression across seven benchmarks. That rate counts both the initial visual tokens and the text returned by Expand. In their batch-size-one test, a typical single expansion took about 17 seconds, versus about 8 seconds for the text baseline. The trade is less context and KV-cache memory for another inference turn.

Image: Figure 7 from Xie et al., LensVLM.