Data methodology

How we turn hundreds of inconsistent government permit feeds into one clean, queryable, honestly-labeled dataset — and exactly what we will and won't claim about it.

Where the data comes from

Every permit in PermitStack originates from a government authority having jurisdiction — a city or county building department — and is published through that government's own permit portal or open-data infrastructure. We do not buy data from a reseller and we do not infer permits from imagery. We read each jurisdiction's official feed directly.

U.S. jurisdictions publish on a handful of common platforms, and we built a dedicated connector for each:

• Socrata (Tyler Data & Insights / SODA API) — the most common open-data platform for large cities.
• ArcGIS (Esri FeatureServer / MapServer layers) — GIS-native permit layers, often county-level.
• CKAN (DataStore API) — used by many mid-size municipal open-data portals.
• CARTO (SQL API) — geospatial open-data feeds.
• Tyler EnerGov (Citizen Self Service) — the public JSON search behind Tyler's permitting product, covering a large swath of cities and counties.
• Accela (Citizen Access portals) — for jurisdictions with no open-data feed, we read their public Citizen Access portal, then normalize the result like any other source.

A small number of very large statewide datasets — for example California's NEM solar interconnection data (~2.2M records) and New York's statewide solar data (~280K) — run through dedicated bulk loaders but land in the exact same schema. You can see every jurisdiction, its platform, and its current freshness on the coverage page.

Nightly ingestion cadence

We run ingestion every night, beginning at 03:00 UTC. Each run is incremental: for every jurisdiction we ask its source only for permits added or changed since our last successful pull, which keeps the load light on the upstream portal and lets a newly issued permit appear in our API within days of issuance. Statewide bulk feeds refresh on their own schedule (the largest, California solar, monthly), and we re-run any feed in full when a source migrates or changes shape.

A schedule is not the same thing as currency, so we do not claim one from the other. Right now 289 of our 902 active sources sit behind a host that is refusing our requests; they keep their label, carry an 'access blocked' note, and their data is older than the rest. Every jurisdiction on the coverage page shows the date its newest permit runs through, measured nightly, so you can judge currency city by city. For comparison, Shovels.ai — the category leader — refreshes roughly twice monthly (per their own docs, verified 9 September 2026).

Normalization into one schema

Raw permit feeds disagree about almost everything: field names, date formats, status vocabularies, coordinate projections, even what counts as a "permit." Our loader maps each source's columns onto a single canonical record, so a permit from a Socrata city in Texas and an ArcGIS county in California come out identical in our API. Normalization includes:

• Field mapping — each jurisdiction config maps source columns to canonical fields (permit_number, address, date_filed, date_issued, status, valuation, contractor_name, latitude/longitude, and more).
• Geocoding & coordinate handling — we standardize coordinates to WGS84, transforming sources published in a local state-plane projection so every permit has a consistent lat/lng.
• Status normalization — every source's local labels collapse into our fixed set: filed, issued, in_progress, final, expired, cancelled, revoked, unknown (see permit status).
• De-duplication — records upsert on a stable per-jurisdiction key so re-ingesting a feed never creates duplicates.

Classification into ~20 categories

Permit "types" are free text and wildly inconsistent — a roof replacement might be filed as REROOF, RE-ROOF, or a sentence in a description field (see reroof vs. re-roof). Our classifier reads each permit's type and description and assigns one of ~20 normalized categories, so you can query "all solar permits" or "all roofing permits" across every jurisdiction with a single filter instead of guessing local spellings.

Current category counts across the dataset:

Each major category also has a glossary entry — for example solar, roofing, HVAC, electrical, and new construction — and several map to a dedicated landing page like solar permit data.

Future-date rejection & data integrity

Upstream feeds are not clean. Several publish placeholder or typo dates — a permit "issued" in the year 2099, or filed next month. We do not pass those through. During ingestion we reject impossible future-dated values on date_filed, date_issued, and date_completed, nulling the bad date and logging it rather than letting a garbage timestamp pollute a roof-age or freshness calculation. A failed record in a batch never takes down the good records around it.

Honest labeling: the data_status flag

This is the part we care most about. Coverage counts are easy to inflate by leaving dead cities in the total. We don't. Every jurisdiction carries a data_status flag with one of three values:

• active — the source publishes current data; we re-read it nightly. (902 jurisdictions, of which 289 are currently access-blocked)
• historical_archive — the upstream feed is frozen or decommissioned; we keep the historical records for research but expect no new permits. (75 sources)
• frozen — the source itself has stopped updating, not proven decommissioned; we're watching it.

This flag is exposed in API responses and rendered on the coverage page next to every jurisdiction, with a freshness indicator showing how recently each source published. The discipline is simple: a "no permit found" should never silently mean "we quietly stopped getting data here." We'd rather tell you a jurisdiction is archived than let you draw a wrong conclusion from a gap.

What we will and won't claim

We report live counts, not round marketing numbers — the figures on this page are read from the production database when it's generated. Permit data is a near-complete but not total record of construction activity: some work is unpermitted, and coverage varies by jurisdiction. We say so, we label it, and we point you to the coverage page so you can verify before you rely on a city. For the vocabulary behind any field referenced here, see the building permit glossary.