Token Usage

Tokens are the units of text that AI models process. Understanding token usage helps you optimize performance and manage costs.

What Are Tokens?

Tokens are pieces of text that AI models read and generate:

• A token is roughly 4 characters or 0.75 words in English
• "Hello, world!" ≈ 4 tokens
• A 1,000-word document ≈ 1,300 tokens
• A meeting transcript might be 5,000-50,000 tokens

Token Examples

Token Components

Each AI request uses tokens in two areas:

Input Tokens (Prompt)

What you send to the AI:

• System message
• User message
• Context data (meeting info, transcript)

Output Tokens (Completion)

What the AI generates:

• The response
• Structured output

Total tokens = Input tokens + Output tokens

Token Limits

Per Request Limits

Practical Limits

For optimal performance:

• Keep prompts under 10,000 tokens
• Target outputs under 1,000 tokens
• Use summaries instead of full transcripts

Reducing Token Usage

1. Use Transcript Summaries

The biggest token savings come from using summaries:

❌ High token usage:

{{ json.callRecording.transcript }}
// Could be 20,000+ tokens

✅ Low token usage:

{{ json.callRecording.transcriptSummary }}
// Typically 500-1,000 tokens

Impact: 10-40x reduction in input tokens

2. Include Only Relevant Data

Only include data the AI needs:

❌ Everything:

Meeting: {{ json.meeting | json }}
// Includes unnecessary metadata

✅ Only relevant fields:

Meeting: {{ json.meeting.title }}
Attendees: {{ json.meeting.attendees | map: "name" | join: ", " }}
Summary: {{ json.callRecording.transcriptSummary }}

3. Keep Prompts Concise

Every word in your prompt uses tokens:

❌ Verbose:

I would like you to carefully analyze the following meeting
transcript and then provide me with a comprehensive summary
that captures all of the key points and discussions...

✅ Concise:

Summarize this meeting in 3 bullet points.
Focus on decisions and next steps.

4. Limit Output Length

Constrain output to necessary length:

Provide a summary in exactly 2-3 sentences.
Maximum 100 words.

5. Use Appropriate Return Types

Simple types use fewer output tokens:

Token Estimation

Quick Estimates

Calculating Total Usage

Example: Meeting Summary Workflow

Input:
- System prompt: 100 tokens
- Meeting metadata: 50 tokens
- Transcript summary: 800 tokens
- Total input: 950 tokens

Output:
- Summary: 150 tokens

Total: ~1,100 tokens per execution

Example: Full Transcript Analysis

Input:
- System prompt: 100 tokens
- Meeting metadata: 50 tokens
- Full transcript: 15,000 tokens
- Total input: 15,150 tokens

Output:
- Analysis: 500 tokens

Total: ~15,650 tokens per execution

Cost Optimization Strategies

Strategy 1: Filter Before AI

Don't send everything to AI:

[Load Meeting] ──▶ [If: has recording?]
                        │
                        ├── Yes ──▶ [AI: Analyze]
                        │
                        └── No ───▶ [Simple notification]

Strategy 2: Use Lower Tiers

Lower model tiers often have lower per-token costs:

Strategy 3: Batch Similar Operations

One comprehensive prompt vs multiple simple prompts:

❌ Multiple calls:

[AI: Extract summary] → tokens
[AI: Extract action items] → tokens
[AI: Extract sentiment] → tokens

✅ Single call:

[AI: Extract summary, action items, and sentiment] → tokens (once)

Strategy 4: Cache Results

If the same meeting might be processed multiple times, consider caching AI results.

Monitoring Token Usage

Execution Logs

Each AI execution shows:

• Input token count
• Output token count
• Total tokens used

Cost Tracking

Track over time:

• Average tokens per workflow
• Tokens by workflow type
• High-token workflows

Optimization Indicators

Watch for:

• Workflows using > 10,000 input tokens
• Full transcripts being processed
• Redundant AI calls in same workflow

Token Usage by Task

Common Tasks

High vs Low Token Patterns

Low Token Pattern:

[Load Meeting] → [AI: Is urgent? (boolean)] → [Route]
~500 tokens

High Token Pattern:

[Load Meeting] → [AI: Full transcript analysis] → [Report]
~15,000+ tokens

Best Practices Summary

Token Limits and Errors

Input Too Long

If input exceeds limits:

• Error: "Input exceeds maximum context length"
• Solution: Use summary instead of full content

Output Truncation

If output is cut off:

• Response may be incomplete
• Solution: Ask for shorter output, split into multiple calls

Handling Token Errors

[AI Prompt]
    ├── Success ──▶ [Use output]
    └── Error ────▶ [Retry with smaller input]

Related

• Model Selection
• Performance Optimization
• Prompt Engineering