KV Cache Performance Demo
Toggle cache to see speedup/memory diff!
Prompt
Max Tokens
↺
10
100
Use KV-Cache
Clear
Submit
Generation
Performance
Share via Link