BENCHMARKS
Less spend.
Same quality bar.
30%+ lower model costs. The comparison is your application’s requests, with and without tlc.
WITHOUT tlc
Your request.
Your model.
The prompt goes directly to the model you use, with its normal caching and settings.
WITH tlc
Your request.
Through tlc.
The same prompt goes through the Transductor. The answer is evaluated against the same criteria, with every model call included in the cost.
MODEL AGNOSTIC
Measure against
the model you use.
Repeat the comparison for each model and workload. Keep the reference model, prompt set and quality criteria fixed within each comparison, so the result answers what tlc changes for that setup.
WHAT WE MEASURE
The complete result.
- Answer quality
- Correctness and usefulness for the request. For coding and tool use, whether the work succeeds.
- Dollar savings
- Total spend with and without tlc: inference, processing, checks, retries and fallbacks. Include provider caching on both sides.
- Response time
- Time to first output and to the complete answer, including typical and slow responses.
- Conditions
- Prompt set, model versions, sample size, pricing date and evaluation method. Separate first-time requests from repeated use.
RESEARCH ARCHIVE
Earlier memory evaluations.
Source data from the organizational-memory research that preceded the Transductor. These runs cover a different system and workload.