Microsoft has come out publicly against Consumer Reports, stating that it disagrees with the publication’s Surface reliability findings. But it doesn’t offer much in the way of evidence.

“The Surface Team’s mission is and has always been to make devices that deliver great experiences to our customers and fans,” Microsoft corporate vice president—and the public face of Surface—Panos Panay writes. “It’s the motivation for everything we do, and we are proud of the Surface devices we have built.”

Yesterday, of course, Consumer Reports—a respected and well-established consumer products testing organization—removed its “recommended” label for all Surface products, citing their industry-worst failure rates. They based this decision on a survey of over 90,000 readers who own PCs and tablets using a methodology that has been tried and tested for automobiles and other products over many decades.

So this is a tough challenge for Microsoft. This isn’t a guileless tech blog or some ranting anti-Microsoft neophyte. It’s Consumer Reports. And what Microsoft offers in return to the organization’s real-world results is a bit of emotion and a hint at facts without any hard numbers.

So let’s examine every bit of evidence that Microsoft provides.

First, we get this interesting comment, which is a response to the Consumer Reports contention that an estimated 25 percent of those with Microsoft Surface devices will experience “problems by the end of the second year of ownership.”

“In the Surface team we track quality constantly, using metrics that include failure and return rates—both our predicted 1-2-year failure and actual return rates for Surface Pro 4 and Surface Book—are significantly lower than 25 percent,” Mr. Panay writes.

How much lower? We don’t know.

Does changing the time frame from “by the end of the second year of ownership” to “1-2 year failure rate” skew the results because more failures happen later in a product’s lifetime?

Also, he introduces the notion of “return rates” here. By definition, the feedback that Consumer Reports receives is from product owners, not those who have returned products. If someone is almost two years into device ownership, they are not returning the product. They’re just using it. And dealing with it.

So consider the issue muddled, in just one carefully-constructed sentence. Which I believe was crafted to confuse the issue.

But there is more.

“Additionally, we track other indicators of quality such as incidents per unit (IPU), which have improved from generation to generation and are now at record lows of well below 1 percent,” Panay offers.

It’s not possible to understand how an “incident” relates to a “failure.” Mostly because he doesn’t explain the term. Likely because doing so would betray that this is an apples to oranges comparison.

“Surface also ranks highly in customer satisfaction,” he writes. “98 percent of Surface Pro 4 users and Surface Book users say they are satisfied with their device.”

I will point the reader to Welcome to Surfacegate, my description of Microsoft’s feeble attempts to ignore and then slowly fix endemic issues with those exact two Surface models. And anecdotally, I’ll point to the fact that the three Surface Book models I’ve used have all had reliability problems. And that, contrary to that, my Surface Pro 4 has never had any issues at all. Because that’s why the Surface Book and Pro 4 reliability issues are so vexing: Some never have issues, but others never stop having problems.

But the biggest issue I have with “customer satisfaction” is that it’s kind of a bullshit measurement when it comes to premium products.

As I’ve often said of Apple customers, when you buy something very expensive and it fails you in some way, you tend to cover it up because admitting to this failure betrays some mistake in your own decision making. For example, Apple customers bring their iPhones, Macs, and other products to the local Apple Store to get them fixed and then they brag about the company’s stellar customer service. Do Surface customers do the same? When one brings a Surface device in for service, and Microsoft asks them if they like the device, I bet a lot of people say they love it. Of course they do. They bought it. They’re probably still paying off the credit card bill.

On a related (and, yes, anecdotal) note, I suffer from this issue myself. I brought two Surface devices with me to Barcelona, because I still prefer Surface for some reason. But I just had a non-responsive keyboard issue, yet again, with Surface Book that required me to undock and then re-dock the Clipboard screen. This is the type of thing that Surface owners just get used to. My Surface Book experiences have been negative in so many ways. But I still really like Surface Book. It’s goofy.

“Our Surface Laptop and new Surface Pro continue to get rave reviews,” Panay adds.

Um.

Reviewers are patently unable to gauge device reliability, short- or long-term. So this has nothing to do with the conversation. In fact, Consumer Reports recommended Surface until they got the reliability data back. So you can see how this really works.

Ultimately, what we’re left with here is Microsoft doing what it must do, which is defending Surface. But this is also Microsoft offering no hard evidence that Consumer Reports was wrong to drop its recommendation of Surface.

So the conclusion here is obvious.