Preliminary vs revised job-change estimates in Wisconsin have 75 percent "margin of error," Lt. Gov. Rebecca Kleefisch says

No doubt: the primary target of a massive recall campaign in Wisconsin is Republican Gov. Scott Walker. But Democrats and their allies are also collecting hundreds of thousands of signatures on petitions seeking to recall Lt. Gov. Rebecca Kleefisch.

With the recalls as the backdrop, Kleefisch appeared Dec. 18, 2011 on WISN-TV’s weekly public affairs show in Milwaukee. Host Mike Gousha asked her about job creation, which is not on pace to meet the Walker campaign promise to create 250,000 private-sector jobs during his first term.

"The last five months, we've lost private-sector jobs in the state of Wisconsin," Gousha said. "If the reforms are working, if the strategy is working, why are we losing jobs when other states are adding them?"  

Kleefisch began to smile and replied:

"Well, first of all, I think we have to acknowledge how those figures come out. Our Department of Labor actually surveys only about 4 percent of job creators throughout the state of Wisconsin. And so we're talking about a margin of error of plus-or-minus 75 percent, like what we saw in October."

Kleefisch also claimed the survey "has been way off" in estimating the number of jobs gained or lost in eight of the months since she and Walker took office in January 2011. She was referring to the difference between the preliminary and revised estimates that are done for each month.

To be sure, Kleefisch and Walker did not complain the job estimates were unreliable when they showed job gains during the first six months of 2011.

And "margin of error" is really a different measure -- how off the numbers are from reality, not from one estimate to the next. But in the interview, Kleefisch defined her terms, so viewers had a good sense of what she meant.  

So, is Kleefisch right -- do the state’s monthly estimates on jobs gained or lost have a discrepancy of 75 percent? And are the preliminary estimates often "way off" when compared to the revised estimates?

How numbers are generated

Let’s start with how the government generates its numbers.

Each month, the Wisconsin Department of Workforce Development, which is part of the Walker administration, releases employment statistics. The figures, which garner regular news coverage, include the number of jobs added or lost since the previous month.

The numbers are not actual job counts. They are estimates produced by the U.S. Bureau of Labor Statistics based on surveys it does of employers throughout the country.

The method is statistically valid because the employers surveyed reflect the makeup of each state, said Ken Robertson, the federal bureau’s current employment statistics program manager.

A preliminary estimate of the jobs added or lost each month is released after three weeks of surveying, Robertson said. The surveying continues, however, and a revised estimate is reported following the end of the month. The revised figures almost always vary from the preliminary ones because they are based on a larger number of surveys.

Kleefisch’s claims

Given that Kleefisch’s claims were made in connection with the recall campaign, we asked the the Republican Party of Wisconsin, rather than her staff, for evidence to back her claims. Spokesman Ben Sparks confirmed that Kleefisch was referring in the interview to the preliminary and revised job figures for October 2011.

But she misidentified what she was citing.

Kleefisch had been asked about private-sector job losses, but the 75 percent variation she cited actually was for private- and public-sector jobs -- that is, all jobs, excluding farm jobs -- according to the Department of Workforce Development figures.

The preliminary estimate for October was a loss of 9,700 non-farm jobs in Wisconsin, with the revised estimate showing only 2,400 jobs lost. The revision, as Kleefisch suggested, was a 75 percent reduction.

For private-sector jobs lost, the revision was actually higher -- 85 percent.

But Kleefisch cited the October 2011 job figures only as an example -- claiming most of the 2011 monthly reports were "way off." How do the preliminary numbers compare to the revised job figures for the first 10 months of 2011?  

We’ll look at both private-sector jobs and total non-farm jobs, since the question dealt with the first and her answer the second.

We calculated the percentages based on figures provided in the state Department of Workforce Development’s Dec. 15, 2011 report, the most recent monthly report available.

Here’s an example of one calculation: In January 2011, the revised estimate was that 7,700 total non-farm jobs were added that month; the preliminary estimate was 6,200 jobs added. That’s a difference of 24 percent.

Again, the following percentages represent the difference between the revised and preliminary estimates, not the rate at which jobs were lost or added each month.

Total non-farm jobs

January -- 24 percent
February -- 58 percent
March -- 15 percent
April -- 52 percent
May -- 22 percent
June -- 16 percent
July -- 22 percent
August -- 78 percent
September -- 11 percent
October -- 75 percent

So, the 78 percent variation in August was even higher than the 75 percent revision in October that Kleefisch cited. In the other months, the variances were significant, but considerably lower. The average variance over 10 months was 37 percent.

(One note for perspective: An estimated 2.75 million people were employed in Wisconsin in October 2011. So, while the final estimate for the month -- 2,400 jobs lost -- was 75 percent smaller than the preliminary estimate, it represents a fraction of the total number of jobs in the state.)

Private-sector jobs

Here’s how much the preliminary and revised estimates varied on monthly job changes for private-sector jobs only:

January -- 12 percent
February -- 17 percent
March -- 22 percent
April -- 20 percent
May -- 0 percent
June -- 15 percent
July -- 14 percent
August -- 75 percent
September -- 11 percent
October -- 85 percent

So, once again, the variations were 75 percent or higher in August and October. The average monthly variation was 27 percent, lower than for total non-farm jobs.

Robertson of the federal Bureau of Labor Statistics said the August and October variations are "larger than we like to see." He said that, on the national level, variations of roughly that magnitude occur about once a year, but he didn’t know about individual states.

Our conclusion

Kleefisch said the monthly jobs numbers have a "margin of error" of 75 percent and were "way off" in eight of the first 10 months of 2011.

In her response to the question, Kleefisch did some mixing of apples and oranges and using the phrase "margin of error" was misleading. But she did define her terms in the interview.

The difference between the preliminary and revised numbers was 75 percent or higher in two of the first 10 months of 2011. As for "way off," on average, the variations between preliminary and final estimates in 2011 have been significant: 37 percent for total non-farm jobs and 27 percent for private-sector jobs.

Kleefisch’s statement is partially accurate but left out important details -- our definition of Half True.