Microsoft has re-released the October 2018 update six weeks after pulling it for a file deletion bug that wiped the Documents directory of a small number of users. The delay was then further extended by the discovery of additional bugs that had also slipped past the company’s detection methods. As a result, Microsoft has decided to share some additional details on its testing practices — and those details show exactly how this flaw managed to ship in the first place, though I’m not sure that was Microsoft’s intention.
According to Michael Fortin, Corporate VP of Windows, Microsoft tracks a great deal of information related to Windows’ overall level of bugginess and customer problems, as one might expect. The shift to Windows-as-a-service came with a number of additional changes. Microsoft fired most of its original QA team and began requiring developers to perform their own function testing. It added additional telemetry detection to find errors and catalog them automatically. And, of course, it drastically overhauled its own development process to deliver two major updates per year rather than a single service pack every few years. The company states that it also began tracking its own internal quality metrics differently and that it’s seen consistent improvement in key metrics in the years since Windows 10 debuted.
Microsoft makes much of the fact that its programmers eat their own dog food and use the same versions of Windows they distribute, as well as what it calls a declining incident rate and a steady improvement in the number of people who would recommend Windows 10 (this is referred as the Net Promoter Score). But while it’s interesting to see the overall Incident rate decline, this clearly has nothing to do with the fact that a killer bug that wiped dozens or hundreds of GB of personal data made it through Microsoft’s testing. Similarly, the fact that more people would recommend Windows 10 in general has nothing to do with whether the OS is susceptible to small-but-devastating bugs that destroy personal data.
Furthermore, the report doesn’t contemplate that more people are likely recommending Windows 10 for the simple reason that the alternatives have an increasingly limited shelf life. The “Year of Linux on the Desktop” has become a joke because, despite Linux’ overwhelming and undeniable success in server and enterprise markets, it’s clearly never going to dent the consumer Windows ecosystem on desktops and laptops. Ditto for Mac, where Apple has enjoyed increased market share following its switch to x86 chips, but never enough momentum to pose a serious threat to Microsoft. That means the two options for Windows users, realistically speaking, are Windows 7 and Windows 8. Microsoft has taken steps to break the deployment of both OS’s on newer systems, and while those steps can themselves be bypassed, most users are not comfortable doing so and typically don’t.
In 2015, when Windows 10 was new, Windows 7 still had a full five years of security updates coming to it. Today, there’s barely a year left on the calendar. Windows 8.1 will hang on until 2023, but is anybody — anybody — pining to run Windows 8.1? We’re dubious. When you don’t have an alternative, you have to account for the fact that people may be recommending your operating system because it’s the only choice they perceive themselves to have.
Microsoft is promising to “increase transparency and communication,” but if this blog post is representative of the company’s efforts to do that, it mostly reads like a self-congratulatory pat on the back for work the company has already done, as opposed to an acknowledgment of a problem that royally screwed a small number of people. Meanwhile, Peter Bright at Ars Technica has made an extremely credible case based on interviews and discussions with MS employees that the root causes of Microsoft’s Windows update issues have nothing to do with the frequency of updates, but the manner in which the company evaluates problems and its practice of allowing unstable code commits to the principle development branch, with full bug evaluation and fixes rolling out only after buggy code has been integrated and deployed. Until that process changes, it’s hard to imagine the overall state of Windows will improve, and the fact that Windows Insider builds are often unstable and unsuited for daily drivers means that these builds are deployed in VMs and secondary systems by the users whose feedback MS depends on. This behavior is supposedly part of why previous bugs, like bluescreens caused by plugging in USB peripherals, made it through to final deployment.
Windows 10 has more features in 2018 than it did in 2015, but I wouldn’t honestly call it more stable. Every version of Windows I’ve used based the Anniversary Edition has had one manner of oddity or another. Most of these have been admittedly small — I’ve been putting up with Notification updates that repeat the same messages since the Spring update — but the first time I applied the October 2018 update it broke SearchUI.exe functionality on my desktop and left me unable to search the Start Menu until I installed Classic Shell. Installing new major Windows Updates has become an adventure in determining which bugs I’ll be swapping for new versions as opposed to a time when I expect any kind of functional improvement in my desktop operating system. I can’t think of a more tone-deaf way for Microsoft to usher in its new era of transparency than by proudly describing all of the test systems that caused a world-class catastrophic data loss for its customers, even if the absolute number of customers impacted is small.
Then again, perhaps we shouldn’t be surprised. Communication has never been Microsoft’s strong suit, and we’ve been collectively griping about it for a very long time.