LASER: What Happens When You Turn Chunking Into an Optimization Problem

pip install lasr

from lasr import chunk

chunks = chunk(document)  # that's it

---

The problem nobody talks about

Every RAG pipeline has a chunking step. You take documents, split them into pieces, embed the pieces, and retrieve the most relevant ones at query time.

Most teams spend weeks choosing their vector database, days evaluating embedding models, and about fifteen minutes on chunking — usually settling for whatever LangChain's RecursiveCharacterTextSplitter does by default.

This is backwards. Chunking determines what your retrieval system can find. If a chunk splits a critical clause in half, no embedding model or reranker will recover that information. If your chunks are too small, you lose context. Too large, you dilute the signal. The boundaries matter more than most people realize.

And yet the most widely-used chunking methods are embarrassingly simple: split every N tokens, or split when a sliding window's cosine similarity drops below a threshold. These approaches are local and greedy — they decide each boundary independently, without considering how that decision affects the rest of the document.

We asked a different question: what if you optimized all the boundaries at once?

---

Chunking as physics

LASER treats chunking as a global optimization problem. Instead of scanning through a document and making local split decisions, it considers every possible way to partition the text and selects the one that minimizes a global objective we call Action.

The intuition comes from physics. The Action has two terms:

Tension measures semantic dispersion inside each chunk — how scattered the content is. A chunk that covers one coherent topic has low tension. A chunk that mixes unrelated paragraphs has high tension. The optimizer wants to minimize this.

Cost is a boundary penalty. Every split you introduce costs you something. The optimizer has to justify each boundary by showing it reduces tension enough to be worth the cost. This prevents over-fragmentation.

The balance between these two forces is controlled by a single parameter α (alpha). Higher α means splits are more expensive, producing fewer, larger chunks. Lower α produces more, smaller chunks.

The key insight: because the optimizer sees the entire document at once, it can make globally coherent decisions. It won't split a contract clause in half just because the local similarity dipped. It won't fragment a paragraph that needs to stay together. It finds boundaries that make sense for the whole document, not just for the local window.

The optimization is solved exactly using dynamic programming. No approximations, no heuristics, no stochastic sampling. Given the same document and parameters, LASER always produces the same output.

---

The results

We benchmarked LASER against seven established chunking methods across five retrieval datasets. Every single result below uses α = 2.5 and all-MiniLM-L6-v2, a 22-million parameter open-source embedding model with 384-dimensional vectors. This is deliberately the smallest, cheapest sentence embedding model in common use.

Main benchmark table

LASER is first on every benchmark. On FinanceBench, the margin is 30 percentage points. On MSMARCO, it misses one passage out of 500 queries.

QuALITY is the exception that proves the rule — all methods score near zero because the questions require literary inference that no sentence embedding model can capture. When the embedding model can't bridge the gap between question and answer, no chunking strategy helps. The spread across all eight chunkers is under 2 percentage points.

Full FinanceBench results (84 SEC filings, k=5)

LASER achieves 93% recall with an average of 1.57 chunks per document. It's looking at 84 SEC filings and deciding that most of them need at most two chunks. And it's right — the relevant passage almost always lands in the top result (MRR 0.899).

Full CUAD results (102 legal contracts, k=5)

LASER beats the next best (LlamaIndex SemanticSplitter) while using fewer chunks and no LLM inference. Notice the chunk counts: semantic-threshold produces 142 chunks per contract. Hierarchical produces 183. LASER produces 11. Fewer, better chunks.

Full MSMARCO results (500 web passages, k=5)

MSMARCO is the benchmark everyone in information retrieval recognizes, and LASER essentially solves it — 0.999 recall means one missed passage in 500 queries. The MRR of 0.900 means the correct chunk is almost always ranked first.

---

LASER adapts its granularity automatically

One of the most interesting emergent behaviors: LASER automatically discovers the right level of granularity for each domain, without being told anything about the document type.

Same algorithm, same parameters. The optimizer looks at each document's internal semantic structure and makes globally optimal boundary decisions. Short, coherent documents get kept whole. Long, multi-topic documents get segmented at natural topic boundaries.

This is not a special case or a fallback — it's the same optimization producing different outputs because the inputs have different structure. When the boundary penalty exceeds the tension reduction from splitting, the optimizer doesn't split. When splitting reduces tension enough to justify the cost, it does.

---

The α parameter: one knob to control everything

The boundary penalty α controls the tradeoff between granularity and coherence. Higher α means fewer, larger chunks. Lower α means more, smaller chunks.

We swept α across our benchmarks to map the cost-quality frontier:

FinanceBench α sweep (84 filings)

FinanceBench is remarkably insensitive to α. Even at α=1.0, LASER crushes every baseline. The optimizer finds the right boundaries regardless of how much you penalize splitting, because the documents have such clear topical structure.

CUAD α sweep (102 contracts)

Legal contracts show clear α sensitivity. At α=1.0, LASER over-fragments — 27.6 chunks per contract means it's splitting mid-clause. As α increases, chunks consolidate into coherent clauses and recall climbs. LASER crosses LlamaIndex SemanticSplitter (0.775) around α≈1.8 and keeps climbing.

Even at LASER's worst α (1.0 on CUAD), it still beats every baseline except LlamaIndex SemanticSplitter. You can't misconfigure it badly enough to lose to fixed-size or recursive splitting.

For most use cases, α=2.5 works well across domains.

---

Context bleed: a small trick that matters

When LASER finds the optimal partition, each chunk gets a 1-sentence "bleed" from its neighbors — one sentence of context before and one sentence after. The core text stays pure (for embedding), but the enriched version is available via chunk.text_with_context for vector database insertion.

This costs nothing computationally and helps retrieval systems that benefit from slightly overlapping context windows. The DP-optimal boundaries are not affected — bleed is applied after optimization, purely for downstream enrichment.

---

What the benchmarks reveal about existing methods

Our results expose a few things the chunking community doesn't talk about:

Semantic-threshold chunkers fail catastrophically on real documents. Greg Kamradt's approach (split when cosine similarity drops) is widely used, but it produced 142 chunks per contract on CUAD, 263 chunks per article on our segmentation benchmark, and consistently scored near the bottom. Local similarity dips are terrible signals for topic boundaries in structured text.

Our implementation of late chunking scored poorly across all benchmarks. Despite significant interest in the technique, the approach — running full documents through a transformer and pooling token-level hidden states within fixed-size spans — produced 0.063 recall on FinanceBench, 0.280 on CUAD, 0.353 on MSMARCO. We implemented the core principle (contextual token embeddings pooled per chunk) but not Jina's specific model or ColBERT-style late interaction, so these results reflect the general approach rather than any specific product.

Embedding-based splitting (LlamaIndex SemanticSplitter) was our strongest competitor. It uses percentile-based dissimilarity thresholds with sentence buffering and consistently placed second. On CUAD and FinanceBench, it significantly outperformed simpler methods. However, LASER beats it on every benchmark while requiring only a single parameter (α) instead of buffer sizes and percentile thresholds.

Fixed-size splitting is more competitive than people think. On MSMARCO, recursive splitting hits 0.985 recall — nearly matching LASER. For short, well-structured web passages, naive splitting actually works. The gap only appears on longer, structured documents where boundary placement matters.

---

All of this uses a free, open-source embedding model

Every result in this post uses all-MiniLM-L6-v2 — a 22-million parameter model with 384-dimensional embeddings. It's the smallest, cheapest, fastest sentence embedding model in common use. People typically use it when they need speed and don't care about quality.

LASER with this tiny embedding model outperforms methods that cost orders of magnitude more per document.

This tells you something important: the optimization framework is doing the heavy lifting, not the embedding model. When you optimize boundaries globally, even noisy, low-dimensional embeddings provide enough signal to find the right partition. Methods that make local, greedy boundary decisions need better signal to compensate for their suboptimal strategy.

---

When LASER doesn't help

We believe in reporting limitations honestly.

Literary comprehension (QuALITY): When questions require inferential reasoning over narrative text, embedding-based retrieval fails regardless of chunking. All methods scored under 6% recall. This is an embedding and retrieval paradigm limitation, not a chunking one.

Topic segmentation on very short text: On a Wikipedia section boundary detection benchmark (3,543 documents averaging 300 characters), LASER's boundary penalty prevents splitting because the documents are too short for tension reduction to justify a boundary. Dedicated segmentation methods that optimize for boundary detection rather than retrieval quality outperform LASER on this specific task. This is by design — LASER optimizes for retrieval, not for matching human-annotated boundaries.

Chunking latency: LASER requires computing sentence embeddings and running the DP optimization. On CUAD (102 contracts), the chunking step takes approximately 510 seconds versus 0.1 seconds for recursive splitting. The vast majority of this time (~99%) is spent on sentence encoding, not the DP solver itself — the dynamic programming runs in milliseconds. For offline indexing pipelines, this is irrelevant: you chunk once and query many times, and LASER's smaller index makes every subsequent query faster. For real-time streaming applications, the encoding latency may matter. GPU acceleration and batched encoding reduce it significantly.

---

Try it

pip install lasr

from lasr import chunk

# Default — works well across domains
chunks = chunk(document)

# Control granularity
chunks = chunk(document, alpha=2.5)

# Each chunk has context for richer retrieval
for c in chunks:
    c.text                # core DP-optimal text (for embedding)
    c.text_with_context   # with 1-sentence neighbor bleed (for storage)

LASR integrates with any pipeline that accepts a list of text chunks. Drop it in, replace your existing chunker, and measure the difference.

---