Using Jev to Answer the Internet's Hardest Questions

Settling some old arguments with TypeSafe's Jev.

David Gérouville-Farrell avatar
  • David Gérouville-Farrell
  • 6 min read

Last Thursday I was at a Figma AI event, How AI Built This with Figma. I’m going to write up the event properly later, but one of the demos was showing off how Figma used Jev - TypeSafe’s model that answers questions with typed outputs and probabilities rather than generated text - and somebody made a joke about how we could finally figure out what colour that dress was.

Inspired by that, I have decided it’s finally time to answer each of the internet’s hardest questions.

The Dress

We have to start with the 2015 dress that split the internet between folk who saw it as blue and black and folk who saw it as white and gold.

The dress
The dress
{
  "The Dress!": {
    "type": "choice",
    "instructions": "What colours is the dress in `photo`?",
    "criteria": {
      "Blue and black": "The dress is blue with black lace stripes",
      "White and gold": "The dress is white with gold lace stripes"
    }
  }
}
Jev’s verdict

Jev’s verdict

Jev itself seems a bit confused here, but leans towards blue and black, which is obviously wrong.

The Checker Shadow

Going way back to 1995, we have Edward Adelson’s checker shadow illusion, in which squares A and B look like different shades but are in fact the same shade.

The checker shadow
The checker shadow
{
  "The Checker Shadow!": {
    "type": "choice",
    "instructions": "Which is darker in `photo`, square A or square B?",
    "criteria": {
      "Square A": "Square A is darker than square B",
      "Neither": "Squares A and B are exactly the same shade of grey"
    }
  }
}
Jev’s verdict

Jev’s verdict

Jev nails this one.

The Strawberries

In 2017 Akiyoshi Kitaoka shared this photo of strawberries that look red but don’t contain a single red pixel.

The strawberries
The strawberries
{
  "The Strawberries!": {
    "type": "choice",
    "instructions": "What colour are the strawberries in `photo`?",
    "criteria": {
      "Red": "The strawberries are red",
      "Grey": "The strawberries are grey"
    }
  }
}
Jev’s verdict

Jev’s verdict

I was surprised by this one - Jev sees them as red. I’d have thought this particular illusion was a variation on the checker shadow, so I expected Jev to get it correct here.

The Müller-Lyer Illusion

The Müller-Lyer illusion is an interesting one - all of the lines are the same length, but the arrowheads make some look longer than others. It usually comes in two versions, so I sent them to Jev one at a time with the same question.

{
  "The Müller-Lyer Illusion!": {
    "type": "choice",
    "instructions": "Which line in `photo` is longer?",
    "criteria": {
      "The left line": "The left line is longer than the right line",
      "Neither": "Both lines are exactly the same length"
    }
  }
}
The lines joined
The lines joined
Jev’s verdict

Jev’s verdict

The lines apart
The lines apart
Jev’s verdict

Jev’s verdict

What’s interesting here is the difference between how Jev performs on the two. With the lines joined it’s a coin flip, leaning slightly towards the illusion, but with the lines apart it’s fairly sure they’re the same length. Also notice how its confidence differs between the two - 10% with the lines joined versus 54% with them apart.

The Ebbinghaus Illusion

In the same vein, we’ve got the Ebbinghaus illusion, in which the two orange circles are exactly the same size but appear different sizes because of the circles around them.

The Ebbinghaus illusion
The Ebbinghaus illusion
{
  "The Ebbinghaus Illusion!": {
    "type": "choice",
    "instructions": "Which orange circle in `photo` is bigger?",
    "criteria": {
      "The right circle": "The orange circle on the right is bigger than the one on the left",
      "Neither": "Both orange circles are exactly the same size"
    }
  }
}
Jev’s verdict

Jev’s verdict

I’m kind of liking the fact that Jev falls for the same illusions that people do. But you can see the difference here between a typical decision and a decision with confidence. Jev picks the right circle, but it’s only 50% confident, and you can add a lot of nuance to your system when you have confidence as well as a decision.

The Café Wall

In the café wall illusion the horizontal lines are perfectly straight and parallel, but the offset tiles make them look like they slope. However, this one tricks people but not Jev.

The café wall
The café wall
{
  "The Café Wall!": {
    "type": "choice",
    "instructions": "Are the horizontal lines in `photo` sloped or straight?",
    "criteria": {
      "Sloped": "The horizontal lines slope and are not parallel",
      "Straight": "The horizontal lines are straight and parallel"
    }
  }
}
Jev’s verdict

Jev’s verdict

Jev nails this one.

The Duck-Rabbit

The duck-rabbit doesn’t really have an answer, I think, but here’s what Jev thinks anyway. (I cut off the original caption, since it gives the game away.)

The duck-rabbit
The duck-rabbit
{
  "The Duck-Rabbit!": {
    "type": "choice",
    "instructions": "What animal is in `photo`?",
    "criteria": {
      "Duck": "The animal is a duck",
      "Rabbit": "The animal is a rabbit"
    }
  }
}
Jev’s verdict

Jev’s verdict

I think I lean slightly towards rabbit too.

Rubin’s Vase

Rubin’s vase feels a bit unfair to me, because I don’t think it’s either - it’s both.

Rubin's vase
Rubin's vase
{
  "Rubin's Vase!": {
    "type": "choice",
    "instructions": "What is in `photo`?",
    "criteria": {
      "A vase": "A vase",
      "Two faces": "Two faces looking at each other"
    }
  }
}
Jev’s verdict

Jev’s verdict

Jev, however, sees faces as more likely (but with low confidence).

My Wife and My Mother-in-Law

My Wife and My Mother-in-Law is the oldest one I’m aware of (the earliest known version is an 1888 German postcard), and it’s one where I’m able to switch which version I see depending on which part of the image I focus on.

My Wife and My Mother-in-Law
My Wife and My Mother-in-Law
{
  "My Wife and My Mother-in-Law!": {
    "type": "choice",
    "instructions": "How old is the woman in `photo`?",
    "criteria": {
      "Young": "A young woman looking away",
      "Old": "An old woman in profile"
    }
  }
}
Jev’s verdict

Jev’s verdict

However, Jev seems to really lean towards the young wife rather than the mother-in-law.

Dog or Mop

The dog in dog or mop is a Komondor. I picked A and B (outlined in blue) and asked about each one.

Dog or mop
Dog or mop
{
  "Dog or Mop!": {
    "type": "choice",
    "instructions": "Is `photo` a dog or a mop?",
    "criteria": {
      "Dog": "A dog",
      "Mop": "A mop"
    }
  }
}
A

A

B

B

Good news for dogs - Jev can tell the difference.

The Impossible Elephant

Roger Shepard’s impossible elephant swaps its legs with the gaps between them, so it’s hard to say how many legs it has.

The impossible elephant
The impossible elephant
{
  "The Impossible Elephant!": {
    "type": "choice",
    "instructions": "How many legs does the elephant in `photo` have?",
    "criteria": {
      "Four": "The elephant has four legs",
      "Five": "The elephant has five legs"
    }
  }
}
Jev’s verdict

Jev’s verdict

I like the fact that Jev isn’t sure here, because there really isn’t a correct answer.

Chihuahua or Muffin

And finally, we’ll finish with two classics. First, the famous chihuahua or muffin meme. I took these four and sent them to Jev one at a time.

Chihuahua or muffin
Chihuahua or muffin
{
  "Chihuahua or Muffin!": {
    "type": "choice",
    "instructions": "Is `photo` a chihuahua or a muffin?",
    "criteria": {
      "Chihuahua": "A chihuahua",
      "Muffin": "A blueberry muffin"
    }
  }
}
Top left

Top left

Top right

Top right

Bottom left

Bottom left

Bottom right

Bottom right

The good news here is that Jev can be trusted for food safety.

Labradoodle or Fried Chicken

A variation on the above, labradoodle or fried chicken. I sent each of these separately.

Labradoodle or fried chicken
Labradoodle or fried chicken
{
  "Labradoodle or Fried Chicken!": {
    "type": "choice",
    "instructions": "Is `photo` labradoodles or fried chicken?",
    "criteria": {
      "Labradoodles": "Labradoodle puppies",
      "Fried chicken": "Fried chicken"
    }
  }
}
Left half

Left half

Right half

Right half

I’m a wee bit worried about the labradoodles here, given that Jev is almost 30% convinced they’re chicken.

Recommended for You

Trust, Data and Governance in Housing Finance

Trust, Data and Governance in Housing Finance

What I said on Ivy Rock's AI in Housing Finance panel, plus some great examples from Paul McGrady and Victor Ochieng.

Claude in the Enterprise

Claude in the Enterprise

Reporting from the field on the different types of impact AI is having in large organisations.