When you run a prompt through an AI node, the text response is only part of the story. Beneath the surface of that output lies a set of metricsâlatency, cost, and token countsâthat reveal the true efficiency of your process. Understanding these metrics is not just an administrative task; it is a fundamental skill for anyone looking to build scalable, reliable AI workflows. Tokens, the fragmented word pieces that models use to process information, dictate both your financial cost and the limits of the model's context window.
Equally important is how you judge the results. The evaluation card system is designed to introduce deliberate friction into your workflow, forcing you to pause and critically assess the output. Rating a response and writing detailed notes requires effort, but this honest evaluation is the only way to expose the flaws in your instructions. Optimism has no place in this phase; rigorous, critical analysis is what drives improvement.
Assignment
Run a complex prompt through an AI node in the lab. Review the resulting metrics, specifically noting the latency, cost, and the ratio of input to output tokens. Then, use the evaluation card system to rate the output. Provide a strict, honest rating (1-5) and write at least three sentences of notes detailing exactly where the output succeeded and where it fell short.
Learning Objectives
- Interpret AI node outputs beyond just the response text.
- Understand what tokens are and why they matter for cost and context.
- Apply the evaluation card system to rate outputs honestly.
Tokens
Tokens are the fundamental units of data processed by language models, roughly equivalent to word pieces or syllables. They are the currency of AI interactions, dictating both the cost of a call and the limits of the context window.
Input vs. Output Tokens
Input tokens represent the prompt and context you provide to the model, while output tokens are the text generated in response. Monitoring both is essential for managing costs and understanding the efficiency of your prompts.