SillyTavern API costs: why longer chats cost more

By Arscelare
On this page

SillyTavern is free to use. If you connect it to a hosted model, your provider may charge for each reply. With a metered API, you pay for the text sent to the model (input tokens) and the text it generates (output tokens).

A reply can get more expensive as the chat grows because SillyTavern sends earlier messages again. Even a one-word message can come with thousands of tokens of chat history. If you run the model locally on your own hardware, there is no hosted API charge for those replies.

Your latest message is only part of the input

SillyTavern builds a prompt for each generation. Depending on your setup, it can include your character card, persona, system instructions, activated lorebook entries, scene notes and the chat history that fits. Your new message joins that material.

Earlier model replies become input when SillyTavern includes them in later requests. A long reply can therefore contribute to both the output charge now and input charges on subsequent turns.

SillyTavern's prompt documentation explains how the request is assembled. The chat window alone does not show everything the provider receives.

Regenerating a reply is another request with another input to process. It can incur another charge even if you discard the result. Any cache discount depends on what the provider actually reports for that request.

Compare two requests at the same rates

We asked Arscelare 1 to write a station clerk's reply to a traveler carrying the wrong suitcase. We sent the same request again with a short backstory in front of it. Both requests went directly to the Arscelare API with reasoning off.

RequestTotal inputCached input (included in total)OutputCost (USDC)
Scene only440 tokens400 tokens43 tokens0.000339
Scene with backstory499 tokens400 tokens45 tokens0.000434

The API reported the token counts. We calculated the costs using Arscelare's listed rates. The extra 59 uncached input tokens explain most of the difference. The second reply also used two more output tokens. The cache count stayed the same in this test; yours may differ.

This test did not run through SillyTavern. Its character cards and chat history can make a request much larger. For scale, at the same rates, a request with 10,000 uncached input tokens and 1,000 output tokens would cost 0.018 USDC. With 40,000 uncached input tokens and the same output, it would cost 0.063 USDC. Those larger counts are illustrative.

Changing models inside an existing long chat does not necessarily mean starting with a small prompt. The new model may receive the same large history at a different rate. Check a request's full input before extrapolating from the advertised price per million tokens.

Costs do not have to rise forever

SillyTavern has a context setting that limits how much it sends. Its Common Settings documentation describes the response allowance and the dotted line marking which earlier messages fall outside the active context.

Once the prompt fills the available space, older history may stop being included as new messages arrive. Input usage can level off rather than keep growing indefinitely. Activated lore, notes and reply lengths can still change it between turns.

A smaller context can reduce the history included, but it can also remove details you wanted the character to remember. Check what disappears before reducing it purely to lower a bill.

The client and provider enforce different limits. Arscelare checks the fully rendered prompt plus your requested output allowance against its context window. It rejects overflow; it does not silently trim your messages or reduce that allowance. See the model limits before choosing client settings.

Cache hits and reasoning change the calculation

Some providers charge a reduced rate for reported cached input. Those tokens are still part of the request, but the service can reuse work from earlier prompt processing. A long prompt with a substantial reported hit can cost less than an equally long prompt without one.

Do not budget on every turn hitting the cache. Eligibility, prompt changes and cache availability depend on the service and model. Some providers also price cache creation separately. See what cache hits mean in SillyTavern for an example of a partial hit and how to read it on your bill.

Output needs the same care. A reasoning model may generate tokens your client does not show alongside the reply. Arscelare bills reasoning at its output rate even when the client hides it. A short visible answer can therefore have a larger output charge than its displayed text suggests.

Use the provider's usage records for billing comparisons. A client-side token estimate or visible word count is not a complete receipt.

Find which part of your bill changed

Compare a few requests from your own session before changing several settings. This worksheet keeps the comparison small:

Record for each requestWhat it helps you distinguish
Selected model and its ratesA price change versus a usage change
Total input tokensMore history or other prompt material
Reported cached inputA different amount of discounted processing
Total output, including reasoningA longer generation despite a similar visible reply
Regenerations or extra model callsMore requests than the kept chat messages suggest
Actual charged amountWhether the token categories explain the change

After a reply, use SillyTavern's Prompt Itemization to inspect the prompt composition. If history grew, that explains one source of extra input. If a large lore entry appeared or a preset added repeated instructions, inspect that instead.

Request logs can contain private conversation text and credentials. Keep inspection local and redact material before sharing it.

Reduce input deliberately

Start with duplicate instructions or irrelevant lore that the assembled prompt shows you are sending. Remove one item, then check the next request and the writing.

If you replace older history with a shorter summary, preserve the original chat. Check the summary's facts and confirm that the old history was actually replaced rather than sent alongside it. An automatically generated summary can add its own model call and charge.

Choose a response style you want to keep. Repeatedly generating long replies and discarding them can cost more than the final chat suggests. Conversely, forcing a tiny output ceiling can leave replies unfinished and lead to more retries.

If you use Arscelare, the SillyTavern connection guide explains setup and the shared reasoning/output allowance. Start an unfamiliar model with a short scene, then inspect actual usage before carrying a long-running chat over.