Data Format Selection for Multi-Agent LLM Systems: An Empirical Analysis of Token Efficiency

Token costs in multi-agent LLM systems compound fast — every agent-to-agent exchange, every result payload, every context update adds up. We ran controlled experiments to find out whether the serialisation format actually matters. It does, significantly, but not in the way most format advocates claim.

We investigated TOON (Token-Oriented Object Notation) for our Agent-to-Agent (A2A) protocol and measured its actual performance against JSON, YAML, and CSV. TOON achieved 51.6–53.8% token savings on simple and moderately nested structures, but failed completely on deeply nested data. More importantly, CSV sometimes edged out TOON on token counts — but that advantage disappears when queries require reasoning across nested relationships.

The core finding: format selection depends entirely on data structure, not on which format has the best marketing.

Experimental Methodology

We tested four serialisation formats (JSON, YAML, TOON, CSV) across three dataset types using the Claude API for token counting:

Each dataset was serialised in all four formats and token-counted via Claude Haiku 4.5 to ensure measurement consistency.

Token Efficiency by Data Structure

Simple Tabular (AgentResults)

CSV outperforms TOON by 2.8 percentage points on flat data. TOON's structural overhead — array length declarations, field schemas — provides no advantage when data is already uniform.

Moderately Nested (KnowledgeGaps)

CSV again shows a small token advantage. However, CSV's flattening of nested structures creates ambiguity that hurts LLM parsing accuracy on relational queries — which is where the comprehension data matters.

Deeply Nested (BookstoreBorrowingHistory)

TOON and CSV both fail completely on deeply nested structures. For hierarchical data beyond two levels, JSON is the only practical format — with YAML offering a modest ~26% saving.

LLM Comprehension Accuracy

Token efficiency alone is misleading. In multi-agent systems, parsing errors cost far more than the tokens saved — a misinterpreted result can cascade through an entire workflow.

Independent research shows comprehension varies significantly by format and data structure:

Flat tabular data: A GetCrux.ai study tested Claude 3.5 Sonnet on flat question-answering. CSV achieved 95.45% accuracy at $0.23 per 1K queries; JSON achieved 87.5% at $0.41. For pure lookup queries on uniform rows, CSV wins on both dimensions.

Nested data: The official TOON benchmarks tested 209 questions across four models. TOON scored 73.9% overall vs JSON's 69.7%. Critically, CSV was excluded from 100 of those 209 questions (48%) because it cannot represent nested structures without losing information.

Mixed text and tabular data: An independent benchmark across 11 formats found CSV scored 44.3% — among the weakest — when data includes narrative text or irregular structures, against JSON at 57.1%.

The pattern is consistent: CSV excels on pure flat tables; TOON excels on nested structures; JSON remains the baseline for everything complex or irregular.

Why Structure Determines Accuracy

Three concrete examples show why format choice affects LLM comprehension.

Array Length Ambiguity

In CSV, the LLM must count rows to answer "how many orders are there?" — error-prone with streaming or partial context. TOON's [3] declaration states the count explicitly.

orders[3]:
  order_id,customer,total
  1001,Alice,124.98
  1002,Bob,45.50
  1003,Carol,89.99

TOON benchmarks show 73.9% accuracy on count queries; CSV scores 59.8% — a 14-point gap.

Nested Relationship Flattening

To answer "what is Alice's total order value?" in CSV, the LLM must identify rows by customer, multiply qty × price per row, then sum. In TOON, total: 124.98 is directly accessible at the parent level. The CSV multi-step path is where GetCrux's 95.45% figure breaks down — that accuracy holds only for single-cell lookups, not aggregations.

Field Schema Ambiguity

order_id,customer,items
1001,Alice,"Widget,Gadget,Doohickey"

When a field contains comma-separated values, CSV becomes ambiguous — is "Widget,Gadget" one field or two? TOON's items[*] declaration makes the array explicit, eliminating that parsing uncertainty.

Format Selection Reference

Based on our testing and independent benchmarks:

Two notes on edge cases: CSV comprehension drops sharply on aggregation queries and mixed-text data. YAML is rarely the right choice for agent-to-agent communication — it's designed for human readability, not LLM parsing.

How We Applied This at WYZER

For our A2A protocol, format selection now follows data structure rather than a single house standard:

• Task definitions with flat parameters → CSV (56.6% token saving, high lookup accuracy)
• Search results with metadata and nested relevance scores → TOON (51.6% saving, reliable relational queries)
• Knowledge fragments with deep citation hierarchies → JSON (TOON and CSV not applicable)
• Analytics exports for external tools → CSV (interoperability requirement)

The main engineering effort was building per-exchange format conversion layers and updating agent communication schemas to declare expected structure. Instrumenting error rates per format was the most useful early diagnostic.

What the Data Shows

Token efficiency and comprehension accuracy pull in the same direction once you account for data structure:

• Flat tables: CSV wins on both tokens and comprehension
• Nested structures: TOON wins on both tokens and comprehension
• Deep hierarchies: JSON is the only option

Claims of universal format superiority are wrong. No single serialisation format is optimal across all data shapes. Match the format to the structure of the data, and empirical testing with your actual payloads will bear that out.