Next liveBuild Your Own Calendly · zoomwithchipp.comSave your seat →

Published in Analytics and operations

Performance Dashboard

The Performance tab gives you real-time visibility into how fast your AI agent responds. Use it to identify bottlenecks, compare models, and optimize the experience for your users.

Accessing Performance

Open any app in the builder, go to the Insights group, and click Performance. Data starts collecting automatically after the first message is sent to your agent.

Time Range

Use the range selector at the top to filter all metrics by time period:

RangeShows
24hLast 24 hours
7dLast 7 days (default)
30dLast 30 days
90dLast 90 days

All cards, charts, and session lists update when you change the range.

Overview Cards

The top section displays four key metrics:

Median TTFC (Time to First Chunk)

The most important metric. This is how long a user waits before the first word appears in the response. The card is color-coded:

ColorRangeMeaning
GreenUnder 800msExcellent — feels instant
Yellow800ms — 2sAcceptable for most use cases
RedOver 2sUsers may notice a delay

A trend arrow shows whether TTFC is improving or degrading compared to the previous period.

P95 TTFC

The 95th percentile response start time. This tells you the worst-case experience — 95% of requests start faster than this value. If your median is fast but P95 is slow, some users are hitting edge cases (complex prompts, large knowledge bases, or slow tool calls).

Success Rate

The percentage of streams that completed without errors. A healthy agent should be above 98%. If this drops, check for:

  • Model API outages
  • Tool execution failures
  • Rate limiting

Total Streams

The total number of chat responses generated in the selected period. A trend arrow shows volume changes compared to the previous period.

Pipeline Breakdown

This section shows where time is spent before the first token arrives. The bar is split into two phases:

  • Setup: Context gathering, tool resolution, message history loading, and knowledge retrieval. This is work your agent does before calling the LLM.
  • Model: The time from when the LLM request is sent to when the first token arrives. This is the LLM provider’s latency.
💡

If Setup is the majority of your TTFC, consider reducing the number of active tools, trimming conversation history length, or optimizing your knowledge sources. If Model dominates, try switching to a faster model or reducing your system prompt size.

Token Usage

Shows the average input and output tokens per request. High input token counts can increase TTFC because the model has more context to process. Common causes of high input tokens:

  • Long system prompts
  • Large knowledge source context
  • Deep conversation history

Model Comparison

If your agent has used multiple models (e.g., after switching models or using model overrides), this table compares them side-by-side:

ColumnDescription
ModelThe model identifier
StreamsNumber of requests processed
Median TTFCMedian time to first chunk
P95 TTFC95th percentile TTFC
DurationMedian total response time
Input / OutputAverage token counts
ErrorsPercentage of failed streams

Click a model row to filter the entire dashboard to that model. A “Filtered by” bar appears at the top with a Clear button.

ℹ️

Model comparison is especially useful when evaluating whether a model upgrade improves response speed. Switch models, wait for data to accumulate, then compare the before and after.

Recent Sessions

A list of individual chat sessions with performance data (click Load more sessions to see older ones). Each row shows:

FieldDescription
StatusA check for success, or an alert icon for an error
SessionShort session ID and how long ago it occurred
ModelModel used
TTFCTime to first chunk, color-coded
DurationTotal response time
TurnsNumber of turns, when the session has more than one

Click a session to open the Session Detail Drawer, which shows a per-stream breakdown including:

  • Individual phase timings as a waterfall (config, billing, session, context, save message, tools, messages, agent, first chunk)
  • Whether knowledge sources were used
  • How many tools were called
  • Conversation history length at the time of the request
  • Whether deep thinking or deep research was active

Why Chats Are Slow

Below the Recent Sessions list, the Performance tab shows a Why chats are slow section. It appears once your agent has some chats in the selected range. It reads a timing record kept for every chat reply and answers three questions:

  • How long do people wait? Two cards show the wait for the first sign of activity and the wait for the first words. Each shows the median and the p90 (the wait that 9 in 10 chats beat).
  • How many chats are slow? A reply counts as slow when the first sign of activity takes more than 5 seconds, or the first words take more than 15 seconds.
  • What slowed them down? Each slow reply is given one main cause. The section lists the causes with their share of slow chats and the evidence behind them.
CauseWhat it means
Model provider slow to startThe model took a long time to begin answering
Very large promptInstructions, knowledge, tools or history made the request big
Prompt cache missThe model could not reuse the start of an earlier request
Knowledge retrievalSearching your knowledge sources took the time
Tool and integration setupLoading tools, integrations or MCP servers took the time
Tool calls before the first wordsThe agent ran tools before it began to answer
Model reasoning before answeringThe model thought for a long time first
Retries or model fallbackThe first model call failed and was tried again
Chipp platform overhead and QueueingTime spent on our side before the model was called

Causes tagged on Chipp are ours to fix. No setting on your side changes them.

The section also lists How to speed it up (for example a faster model, a shorter system prompt, fewer knowledge sources, fewer tools, or less reasoning, each with the time it should save), a Reasoning effort trade-off table when your chats ran at different effort levels, and the Recent slow turns. It shows timings and sizes only, never the text of a conversation.

Ask the Builder

You do not have to read the section yourself. If a reply feels slow, ask the Builder why. For example:

Why is my chat slow?

The Builder checks the slow turns from the last 7 days (ask for 30 or 90 days if you want a longer view), then tells you the main causes with the numbers behind them. It does not guess. If a cause is on Chipp’s side, it says so instead of suggesting a setting. For causes you can change, it suggests the fix and, if you agree, makes the change for you, such as switching to a faster model or lowering the thinking effort.

In a software factory project, ask Builder chat the same question about the project’s chat agent.

Optimization Tips

Check your Pipeline Breakdown first

If Setup is slow, the bottleneck is on your side (tools, knowledge, history). If Model is slow, the bottleneck is the LLM provider.

Compare models

Smaller, faster models like GPT-4.1 Mini often have significantly lower TTFC than larger models. Use the Model Comparison table to find the right balance of speed and quality.

Watch your P95

A good median with a bad P95 means some users are having a poor experience. Drill into slow sessions to find the pattern — it is often a specific tool or a large knowledge source query.

Monitor trends over time

Use the trend arrows on the overview cards to catch regressions early. A sudden TTFC increase after a configuration change tells you exactly what to roll back.

Troubleshooting

No Performance Data

Performance metrics are collected from streaming chat responses. If you see the empty state:

  1. Send a test message to your agent
  2. Wait a minute for data to appear
  3. Refresh the Performance tab

TTFC Seems High

  1. Check the Pipeline Breakdown — is Setup or Model the bottleneck?
  2. Review your knowledge sources (large document collections increase context time)
  3. Check if deep thinking or deep research is enabled (these add processing time by design)
  4. Try the same prompt with a faster model to isolate the issue

Ready to build it?

Tell Chipp what you want. Start free, or book a session and build it with our team.