For some reason, Gemini API (2.5 Pro and 2.0 Flash) are counting extra tokens.
The same system prompt would count around 10,000 tokens for other LLMs but whenever I choose Gemini 2.5 Pro or 2.0 Flash, it would count as around 40,000 tokens for the same system prompt!
Checked a few hours ago, those 2 APIs are still counting 4x the tokens than others. Is this an issue on Google’s end or MS?
@Kuane please retest. Gemini 2.0 and 2.5 family of models have been fixed to reflect token count (not character count).
Yup seems fixed. Thanks Jerry!
