Vision models have an awkward blind spot: they can recognise something in an image, but struggle to tell you exactly where it is. Ask an agent to crop the May panel from a dense calendar, and it might understand perfectly which month you mean while giving you coordinates that clip half the dates.
The term is spatial grounding: connecting what the model sees to a location in the image. In my answer-sheet experiment, I suggested uniquely labelling each cell to make this easier. I’ve now built a tiny tool that applies the same idea to arbitrary images: Foveate Images.
It gives the image a map with stable addresses. On a calendar divided into four rows and three columns, May becomes ROOT/B2. The agent asks for that address, and the tool returns a lossless crop. If the details are still too small, it can subdivide that region and keep going. Each crop comes with a labelled copy for navigation and a raw copy for reading the actual evidence.
There’s a little classical computer vision underneath it. A Hough transform finds horizontal and vertical rules, useful for tables and bordered grids. The gutter detector splits along broad bands of whitespace or background. The panel detector looks for repeated rectangles of similar size that line up, like the months on a calendar. These heuristics propose regions; when the evidence is weak or ambiguous, the tool falls back to a regular grid. The agent can inspect the map and choose an alternative.
The bundled skill adds a few rules aimed at the model’s blind spots:
- Check every region. For exhaustive tasks, inspect every relevant tile and zoom further if it isn’t readable.
- Confirm the context. Read the month, row header, or panel title before recording a detail.
- Count each item once. Overlapping crops provide context, but each item belongs to one tile’s core region.
- Keep colour evidence clean. Read colours from the raw crop, since annotations can bias interpretation. Use the legend’s printed colour names rather than inventing your own.
- Keep a list per tile. Merge these lists and check coverage before answering, so a plausible partial answer doesn’t become the final one.
This is the same instinct as jqi and my other small tools for AI development: give the agent a small tool shaped around a specific weakness. A few deterministic operations and some carefully chosen heuristics can make the rest of its job much easier.