How Climate Drives Crop Insurance Losses: 35 Years of USDA Data

The US federal crop insurance program pays out billions of dollars annually to farmers affected by drought, flood, heat, and other perils. The USDA Risk Management Agency publishes detailed cause-of-loss data for every indemnity payment — and when you pre-join it with NOAA climate data, the relationship between weather anomalies and agricultural losses becomes strikingly clear.

In this tutorial we'll load the ClarityStorm USDA Crop Insurance + Weather dataset, profile loss patterns by crop and cause, visualize the drought-loss connection, and identify the most climate-vulnerable counties in the US. All in about 50 lines of Python.

What's in the Dataset

• 2M+ county x crop x year indemnity records from USDA RMA (1989-2023)
• Pre-joined with NOAA nClimDiv drought (PDSI), precipitation, and temperature anomalies
• 130+ crop types, all causes of loss (drought, excess moisture, heat, freeze, etc.)
• Premium, liability, and policies-in-force alongside indemnity payouts
• 3 tables: indemnities, weather, and a pre-joined ML-ready table

Loading the Data

import pandas as pd

indemnities = pd.read_parquet("usda_crop_insurance_indemnities.parquet")

print(f"Total records: {len(indemnities):,}")
print(f"Year range: {indemnities['year'].min()} to {indemnities['year'].max()}")
print(f"Unique crops: {indemnities['crop_name'].nunique()}")
print(f"Total indemnity: {indemnities['indemnity_amount'].sum() / 1e9:.1f}B USD")

Top Causes of Loss

Drought dominates crop insurance losses across the 35-year history, but the pattern varies by region and crop. The following analysis reveals which causes drive the most payouts — information critical for agricultural lenders, reinsurers, and climate adaptation planners.

import matplotlib.pyplot as plt

cause_totals = (
    indemnities.groupby("cause_of_loss_desc")["indemnity_amount"]
    .sum()
    .sort_values(ascending=False)
    .head(10)
    / 1e9
)

plt.figure(figsize=(10, 5))
cause_totals.plot(kind="barh", color="#16a34a", edgecolor="white")
plt.title("Top 10 Causes of Crop Insurance Loss 1989–2023 ($B)", fontsize=14, fontweight="bold")
plt.xlabel("Total Indemnity (Billions $)")
plt.gca().invert_yaxis()
plt.grid(axis="x", alpha=0.3)
plt.tight_layout()
plt.savefig("crop_loss_causes.png", dpi=150)

The Drought-Loss Connection

The pre-joined weather table includes Palmer Drought Severity Index (PDSI) values by county and year. Negative PDSI indicates drought. Plotting drought severity against indemnity payouts reveals a strong, almost linear relationship — as PDSI drops below -2 (severe drought), crop losses spike dramatically.

joined = pd.read_parquet("usda_crop_insurance_joined.parquet")

# Bin PDSI into drought categories
bins = [-6, -4, -3, -2, -1, 0, 2, 4, 6]
labels = ["Extreme
Drought", "Severe", "Moderate", "Mild
Drought", "Normal", "Wet", "Very Wet", "Extreme
Wet"]
joined["drought_cat"] = pd.cut(joined["pdsi_mean"], bins=bins, labels=labels)

avg_loss = joined.groupby("drought_cat")["indemnity_amount"].mean() / 1e3

plt.figure(figsize=(10, 5))
avg_loss.plot(kind="bar", color="#ea580c")
plt.title("Average Crop Loss by Drought Category ($K per record)", fontweight="bold")
plt.ylabel("Avg Indemnity ($K)")
plt.xticks(rotation=45)
plt.tight_layout()
plt.savefig("drought_loss_correlation.png", dpi=150)

What to Build Next

• Climate risk model: predict county-level crop losses from seasonal weather forecasts using the pre-joined ML table
• Crop vulnerability index: rank crops by drought sensitivity using loss-ratio-to-PDSI elasticity
• Portfolio stress testing: simulate insurance portfolio losses under climate scenario projections (RCP 4.5, 8.5)
• Precision agriculture: identify crop-county combinations resilient to temperature anomalies for farm planning
• Time-series forecasting: model year-over-year indemnity trends to project future federal program costs