Hi all,
Having an issue where Generate Text is ending early despite OpenAi 4.1 being set to 100k tokens.
I’m making a document analysis & summary tool, and when I was testing the workflow on the free plan with 4o-mini, I was able to complete 100/125 questions on my document with the 16k token limit.
When I upgraded to the Pro plan to gain access to 4.1’s 100k limit, I can’t get past 25/125 questions
I’ve reviewed the prompt and it is solid. What is the issue here?
nvm i figured it out. there are two settings for token limit which is super confusing, one on the MainFlow Automations tab when you click on the Generate Text block towards the bottom right, and another in the MainFlow Model Settings. you can set max to 100k in the second one, but if you try setting 100k in the first one, you get an error message saying youre capped at 32k. in case any else has this issue, i’d like to also recommend this be cleared up in a patch.
The Model Settings tab controls the default model settings for all blocks in the workflow. You can override those settings (e.g., to do something like use a different model for only one step in a workflow) from the Inspector pane of a block to set specific models or parameters on a per-step basis.
In general, the number of tokens should be thought of as a “maximum limit”. It represents the max number of tokens the model is allowed to generate. However, the models also use their own discretion to decide when to end the conversation turn. For example, setting the Max Tokens to 100k and then prompting the model with “Write a poem” isn’t going to write a 75,000 word poem for you! If you need longer output, sometimes it also helps to literally just tell the model to keep going by including instructions like “Write at least 5,000 words” or something similar in your prompt.