As Congress returns this fall, we’re likely to see a flood of data about Congressional races and advocacy issues. From Real Clear Politics poll postings, the last eleven public surveys from mid-August to today (early afternoon) show Trump job approval anywhere from +10 to -18. Based on this 28-point range, it is no wonder people want to know why so many polls are wrong.
There can be methodological problems in surveys, especially with party ID in the sample. But we are seeing an unfortunate trend of data being used to validate narratives rather than for honest assessments. This has definitely been the case with polling, as much of the “polling” we see in the public domain is driven by a preferred narrative or motivated by fundraising— not situation awareness. In 2022, the “Red Wave” media and fundraising frenzy was fueled by conservative-leaning polls showing campaign leads that never materialized on Election Day, and certain races being competitive that turned out not to be close. Some organizations in the private sector are also guilty of promoting happy polling numbers for their advocacy issues that don’t reflect reality.
Questions about data manipulation are now being raised about government statistics and not just polls. President Trump recently accused (former) BLS Commissioner Erika McEntarfer of producing “rigged” numbers from the downward revisions of the jobs report. The discussion over what is a legitimate data result versus a manipulated result is not the first we have seen in Washington. A June 2023 White House presentation about Bidenomics trumpeted that “gas prices are down from their summer 2022 peak.” However, the chart only displayed data from January 2022 forward and omitted data points showing price increases from Biden’s first year in office in 2021. This was a case of legitimate data being selected from the most favorable timeframe to support a narrative.
How can you tell if numbers are being manipulated to validate a narrative? Here are three red flags:
1.) All of the data points are favorable to a particular narrative or agenda. The first red flag of manipulated data is when all the numbers are favorable for one side, most likely for the agenda of the data sponsor. If every number being presented is favorable to one side, it becomes obvious that the less favorable data points are being hidden or were not asked in order to generate a one-sided result.
2.) A data report sounds more like a press release than an analysis. Consisting of only favorable data points, the report reads like a press release and doesn’t provide a complete picture. There is no inclusion of data points that might run counter to the agenda being promoted.
3.) The data source is unavailable or limited to prevent verification. In the case of polling, demographics should be available. If government statistics are cited, sources should be cited and available for independent verification. For both polling and government statistics, data should be available for complete timeframes rather than partial timeframes that may omit certain data points.
