Methodology and data sources

Vote Unbiased publishes official government data and checks what politicians say against it. We don't model, forecast or score anyone: every number on the site traces to a primary federal (or official) publisher, and every page links to it.

Comparing administrations

The benchmark compares 10 administrations, from Richard Nixon (1969) to Donald Trump (second term). Each is aligned to its own inauguration: month 1 is the month it took office, and the comparison runs to month 48, the first term. Administrations are compared at the same month in office, so a sitting president is never measured against another's full four years.

Measures that grow with the economy over decades (real GDP, retail sales, manufacturing jobs, gas prices, purchasing power, trade) are ranked on the % change since each president's first month, not on the level, so a recent president isn't "#1" simply because the economy is bigger. Rates (unemployment, inflation, interest rates) are ranked on the level; where lower is better (unemployment, inflation, debt-to-GDP, rates), the lowest ranks first.

The Data tab has two views of the same 29 measures. Same point in office lines up each administration month by month from its inauguration. Whole terms shows one value per calendar year, from the same official series: the year's average, except jobs (added December to December), GDP growth (BEA's annual figure), inflation and wages (change in the annual average), trade (the year's total) and the budget balance (the fiscal year). Each year belongs to the president in office for most of it: the inauguration year goes to the new president, 1974 to Nixon and 1975–76 to Ford. Median income, poverty and inequality are published once a year; in the same-point view each year's figure is shown across its twelve months.

Presidents do not control most of these numbers: the Federal Reserve, Congress, global conditions and the business cycle often matter more. The rankings describe what happened on each watch, not who caused it.

The 26 measures and their sources

Data is retrieved from the Federal Reserve Bank of St. Louis (FRED), which republishes the agencies' releases, and refreshed hourly. Where available we record when each latest observation was first published (ALFRED vintages).

Live fact-checks

During live broadcasts, every checkable claim with a number or a verifiable fact is compared with the official figure on record. Verdicts: True (matches the record), Mostly true (right in substance, off in a detail), Misleading (accurate number, wrong impression or missing context), False (contradicted by the record). Claims that can't yet be settled, such as forecasts, are labelled and left out of the "matched the data" score, which is true ÷ (true + misleading + false).

Repeated claims are grouped across broadcasts, so each fact-check shows how many times a line has been said. Every claim has its own page with sources: see fact-checked broadcasts.

When a line is repeated, its claim page shows a timeline of every time it was said. Each point keeps the verdict it was given at the time and shows the figure that had been published on that day (from the ALFRED archive), before later revisions. The timeline doesn't re-judge anything. Verdicts on the same line can differ between dates: sometimes because new data came out, sometimes because the wording or the evidence a check found was different.

Keeping verdicts consistent

The same claim, checked against the same official data, should get the same verdict. Two rules make sure it does:

• Carry-over. When a speaker repeats a claim we have already checked, in substantially the same words, and no new official figure has been published since (for claims with no data series, within 60 days), the earlier verdict carries over. The claim page says so and links to the earlier check. Claims checked against our own data series are recomputed every time, because that calculation always gives the same answer.
• Review. Where earlier checks of the same claim, made while the official data was unchanged, reached different verdicts, they are reviewed together. The reviewer sees every check's evidence and gives the group one verdict. The original verdict stays on record and is shown on the claim page, and every change is listed publicly at /api/consistency.

Verdicts still change when the data does. If a new release moves the figure, the next time the claim is made it is checked fresh.

Localizing the data

Readers can pick a state and whether they rent or own. The site then adds the official figure for their area next to the national one, wherever one exists. This is context only. Claims are about national figures, so verdicts always use the national number. The setting is stored in a cookie on the reader's device. We never infer location from an IP address.

Official data isn't published at state level for everything, so we use the smallest area that exists and say which one:

• Prices (inflation, rent, owners' equivalent rent): BLS publishes CPI for 4 Census regions, not for states. We show the region the state belongs to: all items since 1967, rent since 1978, owners' equivalent rent since 1983 (12-month changes). Region and US figures both use the not-seasonally-adjusted series, the basis BLS uses for its 12-month change, so the US figure here can differ by about 0.1 point from the seasonally adjusted one used in the benchmarks.
• Gas: EIA weekly and monthly retail prices for regular gasoline in 5 regions (PADDs), since 1992.
• Wages: BLS average hourly earnings of private-sector workers, by state, since 2007 (12-month change). This is a true state figure.
• Home prices (owners): the FHFA all-transactions house price index by state, quarterly, against the FHFA US index.

Renters see the rent component of CPI; owners see owners' equivalent rent (the BLS estimate of what owning a home would cost as rent, not a cash payment) and state home prices. We don't build a blended "personal inflation" number, because that would need weights we would have to choose.

News: coverage and blindspots

The News tab reads 100 U.S. outlets every 20 minutes (national outlets, wires and major regional papers), mostly through their own politics, U.S. and business feeds and groups articles about the same event into stories. Grouping is automatic and deliberately strict (shared distinctive words in headlines and summaries, articles within 36 hours, at least three outlets); a missed link makes two smaller stories rather than wrongly saying an outlet covered something. We show headlines and link to each outlet's own article; we never republish articles.

Left, center, right comes from published research, not from us: Robertson et al. (2018), "Auditing Partisan Audience Bias within Google Search" (Northeastern University), which scores each outlet by which registered Democrats and Republicans shared it on Twitter, from −1 to +1. It measures an outlet's audience, not the content of any article, and it dates from 2017. We call an outlet left below −0.25, right above +0.25, center in between; outlets the study didn't score are shown as not rated and left out of the split. We add outlets in roughly equal numbers on each side so no side dominates the raw counts. Where an outlet blocks automated access to its own feeds (AP, Reuters, USA Today, The Washington Post and several regional papers), we read Google News's public listing of that outlet's own articles instead; CNN and MS NOW (formerly MSNBC, which keeps msnbc.com's score) are read from their Google News sitemaps.

Blindspot. Because we read different numbers of outlets on each side, we compare coverage rates: the share of each side's outlets that covered a story. A story is a blindspot for one side when that side covered it at no more than a quarter of the other side's rate, the other side reached at least 20% of its outlets, and at least six rated outlets covered it.

The numbers in the coverage. When a story is about an official release (inflation, core inflation, unemployment, jobs), we read the figure in each headline and feed summary and compare it with the official number: matches (the latest or the previous month's figure), rounded, the monthly rate given as the yearly one, a different figure, or none. Full articles aren't read.

Corrections

Official data is revised. When a revision changes a figure we show the latest published value and, where it matters, the value as first published.