USDA Crop Insurance Indemnities + Weather Anomaly 1989–2023

Name: USDA Crop Insurance Indemnities + Weather Anomaly 1989–2023
Creator: ClarityStorm
License: https://www.claritystorm.com/license

USDA Risk Management Agency (RMA) Cause of Loss + NOAA nClimDiv County Climate

The definitive dataset for US crop insurance loss analysis. Covers 2M+ county × crop × year × cause-of-loss indemnity records from the USDA RMA Cause of Loss database (1989–2023) — every federally insured crop loss paid in the US. Pre-joined with NOAA nClimDiv county-level drought (PDSI), precipitation, and temperature anomalies by county-year. Three linked tables, ready for climate risk modeling, actuarial analysis, and agricultural finance.

Sample: Public DomainPaid: Commercial LicenseCSV + ParquetAnnual Updates1989–2023 (35 years)Pre-Joined Weather3 Tables

Why not pull it directly from USDA RMA?

The RMA publishes 35 annual ZIP files in two incompatible formats (pipe-delimited vs. comma-delimited) with inconsistent column names across vintages. Here's what we handled:

✓ 35 annual files unified — RMA publishes separate files per year in different formats; we download, parse, and normalize all years into a single consistent schema
✓ Cause-of-loss categories computed — RMA uses 17+ numeric cause codes; we map all to 9 interpretable categories (drought, flood, hail, frost_freeze, wind, heat, insects_pest, disease, other) with an is_weather_cause flag
✓ FIPS codes normalized — state and county codes are inconsistently padded across years; we standardize all to 5-digit zero-padded FIPS for reliable county joins
✓ NOAA weather pre-joined — NOAA nClimDiv is a separate fixed-width text file from NCEI with its own encoding; we parse, aggregate to annual/growing-season metrics, and join on county FIPS + year so you don't have to
✓ Drought classification computed — we map PDSI values to standard drought classes (extreme → severe → moderate → mild → near_normal → moist) for immediate categorical use
✓ Parquet output — the raw CSV files total 3GB+ across 35 years; Parquet columnar storage enables fast analytics on 2M+ rows without loading everything into memory

⏱ Skip 35 annual ZIP files, two format variants, fixed-width NOAA parsing, and FIPS normalization. Three linked tables, ready to query in minutes.

2M+

Indemnity Records

Years of Data

130+

Crops Covered

Linked Tables

Use Cases

Climate Risk Modeling

Quantify how drought, flood, and temperature anomalies translate to actual crop insurance losses by county. Use PDSI × indemnity correlations to project future loss exposure under climate scenarios. 35 years of actual loss-weather pairs across every US crop county.

AgTech & Precision Agriculture

Build county-level crop loss risk scores by combining indemnity frequency, cause-of-loss categories, and weather anomalies. Power precision agriculture risk tools, advisory platforms, and farm management applications with historical loss distributions.

Actuarial & Insurance Analytics

Analyze loss ratio trends by crop, insurance plan, coverage category, and region. Study how plan type (APH vs. RP vs. RPHPE) affects loss ratios under drought versus excess moisture. Build experience-based actuarial tables for crop insurance pricing.

ESG & Climate Finance

Map agricultural supply chain climate exposure at county level. Assess the financial impact of climate change on food-producing regions. Model agricultural value-at-risk for commodity traders, lenders, and climate-focused investment funds.

Federal Program Analysis

Evaluate USDA RMA program effectiveness — subsidy efficiency, CAT vs. buyup coverage mix, and geographic distribution of federal crop insurance support. Compare premium-to-indemnity ratios across crops and regions over time.

Agricultural Research

Study climate-agriculture vulnerability: how drought severity (PDSI) correlates with corn/soybean loss ratios, whether temperature anomalies predict wheat frost claims, or how flood frequency affects NFIP + crop insurance co-occurrence by county.

Schema

Table 1: Indemnity Records

Primary table — usda_crop_insurance_indemnities. 2M+ rows, one per county × crop × year × cause-of-loss × insurance plan combination. Fields marked computed are derived by ClarityStorm.

Field	Type	Description
record_id	int	Unique row identifier
commodity_year	int	Crop year (1989–2023)
state_fips	string	2-digit state FIPS code
county_fips	string	5-digit county FIPS code (join key)
county_name	string	County name
state_abbr	string	State abbreviation
commodity_code	string	USDA RMA commodity code
commodity_name	string	Crop name (Corn, Soybeans, Wheat, Cotton, etc.)
insurance_plan_code	string	Insurance plan code (APH, RP, RPHPE, etc.)
insurance_plan_name	string	Insurance plan full name
coverage_category	string	Coverage category: CAT (catastrophic) or BUYUP
cause_of_loss_code	string	RMA cause of loss code
cause_of_loss_desc	string	Cause of loss description (e.g. Drought, Excess Moisture, Hail)
cause_category	string	Normalized cause category: drought / flood / hail / frost_freeze / wind / heat / insects_pest / disease / other (computed)
is_weather_cause	bool	True for weather-driven causes — drought, flood, hail, frost/freeze, wind, heat, tornado (computed)
month_of_loss	int	Month the loss was reported (1–12)
policies_indemnified	float	Number of policies that received an indemnity payment
unit_months_indemnified	float	Unit months indemnified
net_planted_quantity	float	Net planted acres/units insured
net_endorsed_quantity	float	Net endorsed acres/units
liability	float	Total insurance liability (coverage amount) in USD
total_premium	float	Total premium collected in USD (farmer + government share)
subsidy	float	Federal premium subsidy in USD
state_private_subsidy	float	State or private subsidy in USD (where applicable)
indemnity	float	Total indemnity payment in USD
loss_ratio	float	Loss ratio = indemnity / total_premium

Table 2: County Weather Anomalies

usda_crop_insurance_weather — NOAA nClimDiv annual and growing-season climate aggregates by county. Join to indemnity table on county_fips + commodity_year.

Field	Type	Description
weather_id	int	Unique row identifier
county_fips	string	5-digit county FIPS code (join key to indemnity table)
year	int	Year (1895–2023)
pdsi_annual_mean	float	Palmer Drought Severity Index — annual mean. <−2 = drought, <−3 = severe, <−4 = extreme
pdsi_growing_season_mean	float	PDSI — growing season mean (April–September)
pcpn_annual_mean	float	Precipitation — annual mean (inches)
pcpn_growing_season_mean	float	Precipitation — growing season mean (April–September, inches)
tmpc_annual_mean	float	Temperature — annual mean (°F)
tmpc_growing_season_mean	float	Temperature — growing season mean (°F)
drought_class	string	Drought classification: extreme_drought / severe_drought / moderate_drought / mild_drought / near_normal / moist / very_moist (computed)

Table 3: Pre-Joined Analysis Table

usda_crop_insurance_joined — indemnity aggregated to county × year × cause_category level, pre-joined with weather anomalies. Ready for regression modeling and exploratory analysis without requiring a manual join.

Quick Start

import pandas as pd

# Load indemnity and weather tables
indemnity = pd.read_parquet("usda_crop_insurance_indemnities.parquet")
weather = pd.read_parquet("usda_crop_insurance_weather.parquet")

# Or use the pre-joined table directly
joined = pd.read_parquet("usda_crop_insurance_joined.parquet")

# Top crops by total indemnity paid (1989-2023)
top_crops = (
    indemnity.groupby("commodity_name")["indemnity"]
    .sum()
    .sort_values(ascending=False)
    .head(10)
)
print(top_crops)

# Drought losses by state (most recent 5 years)
drought = indemnity[
    (indemnity["cause_category"] == "drought") &
    (indemnity["commodity_year"] >= 2019)
]
print(drought.groupby("state_abbr")["indemnity"].sum().sort_values(ascending=False).head(15))

# Loss ratio trend by year — are crops getting riskier?
print(indemnity.groupby("commodity_year")["loss_ratio"].mean().round(3))

# Correlate drought severity with corn indemnity by county-year
corn = indemnity[indemnity["commodity_name"].str.contains("Corn", na=False)]
corn_agg = corn.groupby(["county_fips", "commodity_year"])["indemnity"].sum().reset_index()
corn_weather = corn_agg.merge(
    weather[["county_fips", "year", "pdsi_growing_season_mean", "drought_class"]],
    left_on=["county_fips", "commodity_year"],
    right_on=["county_fips", "year"],
)
print(corn_weather.groupby("drought_class")["indemnity"].mean().sort_values(ascending=False))

# County risk ranking — top 20 highest-indemnity counties
county_risk = (
    indemnity.groupby(["county_fips", "state_abbr", "county_name"])["indemnity"]
    .sum()
    .reset_index()
    .nlargest(20, "indemnity")
)
print(county_risk)

Pairs Well With

ClarityStorm: NOAA Storm Events Database

Join crop insurance drought and flood losses (county_fips, commodity_year) with NOAA storm events (CZ_FIPS, BEGIN_YEARMONTH) to correlate specific named weather events (hurricanes, derecho, flash floods) with crop loss spikes at the county level.

ClarityStorm: FEMA NFIP Flood Insurance Claims

Combine crop flood indemnities (county_fips, commodity_year) with FEMA NFIP flood claims (county_code, year_of_loss) to build a complete picture of flood financial exposure in agricultural counties — both the farm-level and residential/commercial property dimensions.

Pricing

Sample

Free

1,000 rows (CSV) + full schema documentation

Public Domain

Download Sample

Complete

$99

All 3 tables — 2M+ indemnity records, 175K+ weather rows, pre-joined table. 1989–2023, CSV + Parquet.

Commercial License

Buy Complete Dataset

Annual

$199/yr

All tables + annual updates as USDA RMA publishes new cause-of-loss data

Commercial License

Subscribe Annual

Data Provenance

Primary source: USDA Risk Management Agency (RMA) — Summary of Business, Cause of Loss data. Published annually at rma.usda.gov

Weather source: NOAA National Centers for Environmental Information (NCEI) — nClimDiv county climate dataset. Monthly PDSI, precipitation, and temperature by county FIPS (1895–present).

Coverage: 1989–2023. Indemnity records begin with USDA RMA's digitized history. All 50 US states and territories with active federal crop insurance programs. 130+ commodity types — corn, soybeans, wheat, cotton, rice, sorghum, tobacco, fruits, vegetables, and specialty crops.

Computed fields: cause_category, is_weather_cause, state_fips, county_fips, drought_class, pdsi_growing_season_mean, and pcpn_growing_season_mean are derived by ClarityStorm from the raw source data.

Update frequency: USDA RMA publishes updated Cause of Loss files annually (typically Q1 following the crop year). Annual subscribers receive updated files when ClarityStorm re-runs the pipeline.

License: USDA RMA and NOAA NCEI data are US federal government works in the public domain. Paid tiers are licensed under the ClarityStorm Commercial Data License covering our pipeline work (multi-year normalization, cause-of-loss categorization, NOAA weather join, drought classification, and Parquet conversion).

Need custom date ranges, specific crops or states, or bulk licensing for agricultural finance?

Contact Sales