A broker friend handed me 400 leases and asked what the AI could surface that human reviewers were missing. The answer was uncomfortable. Three patterns nobody had flagged.
|
THE STACK • Claude 3.5 Sonnet • LlamaParse • Streamlit • DuckDB • Polars |
Four Hundred Leases and a Dare
Here's what I just tried: a commercial real estate broker I know manages a portfolio of about 400 tenant leases across a dozen properties. His team does manual lease abstracts — pulling key terms like rent escalations, renewal options, exclusivity clauses, and co-tenancy provisions into a spreadsheet. It takes one paralegal about 20 minutes per lease. For 400 leases, that's roughly 133 hours of work per annual review cycle.
He handed me all 400 leases and said: "Tell me what humans are missing." The answer was not comfortable.
The Stack
• Claude 3.5 Sonnet — extraction and reasoning model; reads each lease and outputs structured JSON for 28 key fields
• LlamaParse — PDF parsing; commercial leases are complex PDFs with tables and cross-references that break naive parsers
• Streamlit — the analysis dashboard where the broker's team can query results and flag leases
• DuckDB — in-process analytics engine; runs aggregate queries across 400 JSON records instantly
• Polars — fast DataFrame library for wrangling between LlamaParse output and Claude input
The Extraction Architecture
Each lease went through a three-step pipeline:
Parse: LlamaParse converts the PDF to clean markdown, preserving table structures. For 400 leases averaging 45 pages each, this took about 3 hours on LlamaParse's API.
Extract: Claude reads each parsed lease and outputs a 28-field JSON object: base rent, escalation schedule (flat/CPI/percentage), lease term and expiration, renewal option count and notice requirements, co-tenancy clause presence and trigger conditions, exclusivity scope, CAM cap structure, termination rights, HVAC responsibility, signage rights, and more. Parallelized across 10 concurrent Claude calls — total extraction time: about 4 hours.
Analyze: all 400 JSON objects load into DuckDB. Cross-portfolio SQL queries in milliseconds.
The Three Things Nobody Had Flagged
Pattern 1 — Co-tenancy exposure clustering: 23 leases (5.75%) had co-tenancy clauses tied to the same anchor tenant in the same shopping center. If that anchor closes, all 23 leases get rent-reduction or termination rights simultaneously. The team knew each lease had a co-tenancy clause — they just hadn't noticed they all pointed to the same anchor. In the current retail environment, that's a material portfolio risk.
Pattern 2 — Escalation cliff: 31 leases had CPI-linked escalations with no cap. Six of those tenants were already flagged as renewal risks in the broker's CRM — a connection between uncapped escalations and churn risk that nobody had drawn explicitly.
Pattern 3 — Notice-period mismatch: 18 leases required renewal notice 18 months before expiration, versus the standard 6-12. Four of those expire within 14 months — meaning the renewal notice deadline had already passed. The broker went very quiet when I showed him this one.
Where It Almost Went Wrong
Claude's extraction reliability varied by lease quality. Older pre-2010 leases were poorly scanned, with cross-references like "as defined in Exhibit C-1" that required following the reference to understand the clause. The first prompt version produced "N/A" for about 12% of fields on these older leases.
Fix: I added a second-pass verification prompt for any "N/A" or low-confidence field, providing surrounding document context and asking Claude to try alternate section references. That dropped the incomplete-field rate from 12% to 3.4%. The remaining 3.4% genuinely weren't in the documents.
The Number
12 hours of compute versus 133 hours of manual paralegal time — an 11x speed improvement. The broker's team corrected about 15 fields across 400 leases (99.6% field-level accuracy on well-formatted leases). The co-tenancy clustering pattern alone could affect a seven-figure rent-roll decision.
Try This
1. Use LlamaParse for commercial documents — its table extraction and cross-reference handling is worth the API cost for legal PDFs.
2. Define your extraction schema as a JSON schema upfront and give it to Claude in the system prompt. Structured output with explicit field names dramatically reduces variance.
3. Load all JSON outputs into DuckDB immediately — cross-portfolio analytics are where the real value is. Single-document analysis is useful; fleet-level pattern detection is where you find what nobody knew.
4. Add a second-pass verification prompt for "N/A" or low-confidence extractions. Many apparent gaps are findable with better context.
5. Build the Streamlit dashboard for your domain expert, not yourself. Have the broker or paralegal sit with it for 30 minutes and watch what they try to filter first — that's your product roadmap.
DIAGRAM_HINT: pipeline diagram showing 400 lease PDFs → LlamaParse parsing → Polars data wrangling → parallel Claude 3.5 Sonnet extraction (28-field JSON per lease) → DuckDB aggregate analytics → Streamlit portfolio risk dashboard with co-tenancy and escalation pattern visualization
Figure 9. pipeline diagram showing 400 lease PDFs → LlamaParse parsing → Polars data wrangling → parallel Claude 3.5 Sonnet extraction (28-field JSON per lease) → DuckDB aggregate analytics → Streamlit portf…
Comments (0)
Join the conversation!