My coding agent invented its own vision

I was working with my coding agent today and noticed some interesting behavior.

I am using deepseek-v4-flash:0731 which is a text-only model. I’m using it with an Oh-My-Pi backend and Paseo frontend. I described a UI rendering bug to it.

The model, knowing it cannot “see” images, came up with its own clever way to determine if it had fixed the bug or not.

It spawned a Chromium browser, and used it to take a screenshot. Then it wrote a python script to brute force it’s way through the pixels in the image to find the rendering problem I had described. It verified the problem existed, fixed the relevant code, and then verified that the problem was fixed.

My initial prompt was only to fix the problem, and it did all that extra work on it’s own. To be clear, this is using a very cheap open source model ($0.08in $0.18out $0.016 cached/ 1M tokens at time of writing via DeepInfra).

Here’s a snippet of the script it generated to “see” the image.

ssh -o BatchMode=yes [email protected] 'python3 - <<'"'"'PY'"'"'
from PIL import Image
im = Image.open("/code/frontend/repro.png").convert("RGB")
W,H = im.size
print("size", W, H)
def runs(rowvals, label, y):
    out=[]
    prev=None; start=0
    for x,c in enumerate(rowvals):
        if c!=prev:
            if prev is not None:
                out.append((start,x-1,prev))
            start=x; prev=c
    out.append((start,len(rowvals)-1,prev))
    print(label, "y=",y)
    for a,b,c in out:
        print(f"  x {a}..{b}  rgb={c}  w={b-a+1}")
# horizontal row at y=300
runs([im.getpixel((x,300)) for x in range(W)], "HORIZ y=300", 300)
# horizontal at y=500
runs([im.getpixel((x,500)) for x in range(W)], "HORIZ y=500", 500)
# vertical column x=100
runs([im.getpixel((100,y)) for y in range(H)], "VERT x=100", None)
PY'

size 390 844
HORIZ y=300 y= 300
  x 0..7  rgb=(11, 18, 32)  w=8
  x 8..9  rgb=(9, 15, 30)  w=2
  x 10..13  rgb=(9, 15, 29)  w=4
  x 14..14  rgb=(41, 50, 65)  w=1
  x 15..15  rgb=(5, 11, 26)  w=1
  x 16..16  rgb=(4, 8, 25)  w=1
  x 17..17  rgb=(5, 11, 27)  w=1
  x 18..18  rgb=(4, 9, 25)  w=1
(etc...)

From that output, it managed to find and fix the problem.

Pretty extraordinary if you ask me.