Research

I’m Jobin Ayathil.

I've spent the last five years in developer relations, which mostly means sitting where a complicated system meets the people trying to use it, and fixing whatever makes them give up.

I was studying how large public systems decide who gets help, to understand how they manage it at the scale of millions, and came away seeing that none of them is perfect. In the US the cracks are wide: billions in benefits go unclaimed every year, largely because working out what you qualify for is harder than it should be, and that is how BennyFit was built as a solution.

What the measurements say

Fewer errors, 13× faster and 185× cheaper.

BennyFit runs every eligibility judgement through Jev, TypeSafe’s typed-readout model. We put Claude Sonnet 5 through the identical pipeline in two configurations, on the same held-out households, and measured all three.

13×faster to a full screening: 2.2 s against 29.3 s for Claude Sonnet 5 (35.3 s under a schema)

185×cheaper per household: $0.00057 at list price against $0.105 billed ($0.195 under a schema)

3 of 819verdicts wrong on held-out households, against 7 and 14 for Claude Sonnet 5 in the two configurations

96%of facts read from fresh text with Jev choosing, against 83% for code alone

37%fewer follow-up questions (81 to 51) once they are chosen by expected value

53eligibility checks answered for a household in one request

21programs, with state rules across 51 jurisdictions

34/34hand-computed rule cases that match their federal sources

One pass, every rule

Jev answers all of a household’s checks in a single request, each as a probability over a fixed set of options. A general-purpose model writing the same answers under a schema had to be split into batches to stay within its limits.

Probabilities, not prose

Every answer arrives as a distribution, so BennyFit can tell a settled fact from an open one and ask only where the expected change in benefits is worth a person’s time.

Numbers stay in code

Jev never writes a figure. Code finds every amount, Jev says what each one is, and code does the arithmetic, so an amount cannot be invented or mistyped.

Measured on 49 synthetic held-out households, 21 programs each. The error counts are small, the difference from the schema configuration is not statistically significant, and on households missing a deciding fact Jev reached a wrong verdict more often (4 of 10 households, against 1 and 3). The report below covers all of it, with the method and every limitation. Results

Technical report

Screening households for U.S. benefit programs from a short description

A typed-readout decision model compared with schema-constrained generation. Jobin Ayathil, September 2026. 12 pages.

Opening the report…

Page 1 of 12Download PDF