Evaluate tlc

Run the same representative requests directly and through tlc.

Set up the comparison

Keep the prompt set, reference model, provider settings and quality criteria fixed. Preserve normal caching on both sides. Separate first-time requests from repeated use.

Record the complete result

Measure	Record
Quality	Correctness, usefulness and failures for each request.
Cost	All model calls and tlc overhead, including retries, checks and fallbacks.
Latency	Time to first output and complete answer, including slow responses.
Conditions	Model versions, sample size, cache state and how each outcome was judged.

Calculate dollar savings

Subtract the total cost through tlc from the total cost of the direct model calls. Divide that difference by the direct-call cost to calculate the percentage saved. Compare runs with the same quality criteria and record the model prices used.

Public benchmarks

See the benchmark page for the comparison method.

The Transductor is in early access. Join the waitlist.