https://www.pugetsystems.com/blog/2024/08/02/puget-systems-perspective-on-intel-cpu-instability-issues/ Skip to content Main Navigation Puget Systems Logo * + Search [ ] * Solutions + Recommended Systems For: + Content Creation o Photo Editing # Recommended Systems For: # Adobe Lightroom Classic # Adobe Photoshop # Stable Diffusion o Video Editing # Recommended Systems For: # Adobe After Effects # Adobe Premiere Pro # DaVinci Resolve # Foundry Nuke o 3D Design & Animation # Recommended Systems For: # Autodesk 3ds Max # Autodesk Maya # Blender # Cinema 4D # Houdini # ZBrush o Real-Time Engines # Recommended Systems For: # Game Development # Unity # Unreal Engine # Virtual Production o Rendering # Recommended Systems For: # Keyshot # OctaneRender # Redshift # V-Ray o Digital Audio # Recommended Systems For: # Ableton Live # FL Studio # Pro Tools + Engineering o Architecture & CAD # Recommended Systems For: # Autodesk AutoCAD # Autodesk Inventor # Autodesk Revit # SOLIDWORKS o Visualization # Recommended Systems For: # Enscape # Lumion # Twinmotion o Photogrammetry & GIS # Recommended Systems For: # ArcGIS Pro # Agisoft Metashape # Pix4D # RealityCapture + AI & HPC o Recommended Systems For: o Data Science o Generative AI o Large Language Models o Machine Learning / AI Dev o Scientific Computing + More o Recommended Systems For: o Compact Size o Live Streaming o NVIDIA RTX Studio o Quiet Operation o Virtual Reality + Business & Enterprise We can empower your company + Government & Education Services tailored for your organization * Products + Computer System Styles: + Desktop Workstations o AMD Ryzen # Ryzen 7000: # Mini Tower # Mid Tower # Full Tower o AMD Threadripper # Threadripper 7000: # Mid Tower # Full Tower # Threadripper PRO 5000WX: # Full Tower # Threadripper PRO 7000WX: # Full Tower o AMD EPYC # EPYC 9004: # Full Tower o Intel Core # Core 13th Gen: # Small Form Factor # Core 14th Gen: # Mini Tower # Mid Tower # Full Tower o Intel Xeon # Xeon W-2400: # Mid Tower # Xeon W-3400: # Full Tower + Custom Computers + Laptop Workstations o Puget Mobile 17'' + Rackstations o AMD Rackstations # Ryzen 7000: # R121-4U # R550-6U 5-Node # Threadripper 7000: # T121-4U # Threadripper PRO 5000WX: # WRX80 4U # Threadripper PRO 7000WX: # T141-4U # EPYC 9004: # E140-4U o Intel Rackstations # Core 14th Gen: # C131-4U # Xeon W-3400: # X141-4U # X141-5U + Custom Rackmount Workstations + Puget Servers o Puget Servers # AMD EPYC: # E200-1U # E140-2U # E280-4U # Intel Xeon: # X200-1U + Custom Servers + Storage Solutions o Network Attached Storage # QNAP NAS Recommendations o Puget Storage # Puget Storage: # 12-Bay 2U # 24-Bay 2U # 36-Bay 4U + Recommended Third Party Peripherals Curated list of accessories for your workstation + Puget Gear Quality apparel with Puget Systems branding * Publications + Articles + Blog Posts + Case Studies + HPC Blog + Podcasts + Press + PugetBench * Support + Contact Support + Support Articles + Warranty Details + Onsite Services + Unboxing * About Us + About Us + Contact Us + Our Customers + Enterprise + Gov & Edu + Press Kit + Testimonials + Careers * Talk to an Expert * My Account 1. Home 2. / 3. Blog Posts 4. / 5. Puget Systems' Perspective on Intel CPU Instability Issues [14th-Gen-Instability-Update] Puget Systems' Perspective on Intel CPU Instability Issues Posted on August 2, 2024 (August 2, 2024) by Jon Bach You may have heard about instability issues with Intel Core 13th and 14th Gen desktop processors. The issue has attracted more attention as time goes on. I am posting to share what we've experienced here at Puget Systems and what we're doing about it. The issue gained attention over the past three months, led by GamersNexus, Level1Techs, and others. Details were vague at first because the issue tended to surface only over time. Early on it looked like it was related to motherboard power management, making it uncertain whether it was a motherboard issue, an Intel issue, or both. As time went on and more information and speculation developed, the issue became particularly alarming because it seemed to represent a physical degradation of the processor, which is not recoverable. The concern is not only with the nature of the instability, but the incident rate. Some game development studios and cloud gaming providers have come forward with concerning failure rates upwards of 50%. Last week, Intel posted an official statement, in which they shared that elevated voltage requests to the processor was a significant factor, and that a microcode patch would be delivered once validated, with mid-August as the target release date. Intel did not officially address the physical degradation issue, but the general community consensus is that the microcode update is expected to prevent, but not reverse, that degradation. How Puget Systems is Unique At Puget Systems, we HAVE seen the issue, but our experience has been much more muted in terms of timeline and failure rate. In order to answer why, I have to give a little bit of history. Going all the way back to 2017, with the Intel 8700K processor, we published an article titled Why Do Hardware Reviewers Get Different Benchmark Results? which helped call attention to the fact that motherboards were shipping with "Multicore Enhancement" enabled, which set the CPU "All Core Turbo" to be equal to the "Single Core Turbo" frequency. This essentially was overclocking the CPU, by pushing it past official Intel specifications, and had negative effects on stability and temperatures. At Puget Systems, we have always valued stability first and we actively made the choice to follow Intel specifications. Behind the scenes, this meant encouraging Intel to make those specifications public on Intel ARK and pushing motherboard ODMs to follow Intel guidance as their default settings. JayzTwoCents helped drive public awareness of the issue, and for a short time it appeared that things were back on track. Since that time, our stance at Puget Systems has been to mistrust the default settings on any motherboard. Instead, we commit internally to test and apply BIOS settings -- especially power settings -- according to our own best practices, with an emphasis on following Intel and AMD guidelines. With Intel Core CPUs in particular, we pay close attention to voltage levels and time durations at which those levels are sustained. This has been especially challenging when those guidelines are difficult to find and when motherboard makers brand features with their own unique naming. Nevertheless, we kept that approach with confidence due to the high amount of real-world testing we do here. We've even developed our own suite of PugetBench Benchmarks, whose goal is to test real-world scenarios, guided by years of experience and learning through our customers and partners. Our approach has always led us to be conservative with our power settings, especially when we have shown that the real-world performance impact to be a small 1-2% range. Puget Systems Intel Core Failure Rates So, with that understanding of WHY we may be seeing things differently than others in the industry -- what ARE we seeing here at Puget Systems? [Puget-Systems-Intel-Core-CPU-Failures-Per-Month-and-Generation-1024x481] Even though failure rates (as a percentage) are the most consequential, I think showing the absolute number of failures illustrates our experience best. I decided to go back all the way to the launch of Intel Core 10th Gen to give some historical perspective. Starting with 10th Gen, we have only sold the top 2 SKUs (XX700K and XX900K) in volume, which gives us a nice clean set of data. Looking at that chart, you'll notice a few things. First, your attention undoubtedly is drawn to the recent spike of failures with Intel Core 14th Gen. Second, you can see that Intel Core 11th Gen CPUs had a failure rate at nearly the same level, even though it didn't get as much press at that time, that I can recall. Third, I'll draw your attention to a steady and elevated failure rate on 13th Gen processors. I can also plot this same data, but instead of coloring it by CPU generation, I'll color it based on whether we caught the issue on our production floor (shop failure), or if the issue made it out to the customer (field failure). Obviously, a field failure is dramatically more severe of a problem because it now impacts our customer experience. [Puget-Systems-Intel-Core-CPU-Failures-Per-Month-and-Type-1024x481] The most concerning part of all of this to us here at Puget Systems is the rise in the number of failures in the field, which we haven't seen this high since 11th Gen. We're seeing ALL of these failures happen after 6 months, which means we do expect elevated failure rates to continue for the foreseeable future and possibly even after Intel issues the microcode patch. Based on this information, we are definitely experiencing CPU failures higher than our historical average, especially with 14th Gen. We have enough data to know that we don't have an acute problem on the horizon with 13th Gen -- it is more of a slow burn. We do expect an elevated failure rate on 14th Gen while Intel finishes finding a root cause and issuing a microcode update. While the number of failures we are experiencing is definitely higher than our historical average, it is difficult to classify 5-7 failures a month in the field as a huge issue, and it is definitely a lower rate of failure than we are hearing about from others in the industry. The recent spike in 14th Gen failure rates stands out mostly because how incredibly low historical CPU failure rates tend to be. We believe that our commitment to internally developed power settings is why we have been much less impacted than others by these Intel stability issues. This is shaping our approach over the coming months. Failure Rates in Context Everything I've shown you so far is our raw number of failures, but what matters most is failure rate percentages. Let's look at total failure rates in the context of multiple generations and with comparison to AMD Ryzen CPUs. [Puget-Systems-Intel-CPU-Failure-Totals-by-Group-1024x454] You can see that in context, the Intel Core 13th and 14th Gen processors do have an elevated failure rate but not at a show-stopper level. The concern for the future reliability of those CPUs is much more the issue at hand, rather than the failure rates we are seeing today. If it is true that the 14th Gen CPUs will continue to have increasing failures over time, this could end up being a much bigger problem as time goes by and is something we will, of course, be keeping a close eye on. 14th Gen isn't as rock solid as Intel's 10th or 12th Gen processors, but at least for us, it isn't yet at critical levels. Based on the failure rate data we currently have, it is interesting to see that 14th Gen is still nowhere near the failure rates of the Intel Core 11th Gen processors back in 2021 and also substantially lower than AMD Ryzen 5000 (both in terms of shop and field failures) or Ryzen 7000 (in terms of shop failures, if not field). We aren't including AMD here to try to deflect from the issues Intel is currently experiencing but rather to put into context why we have not yet adjusted our Intel vs. AMD strategy in our workstations. Our Plan of Action Even if we are not seeing the same level of failures as others, this is a real issue that we are addressing internally. But what exactly are we doing about it? 1. In the majority of cases, we are staying the course for now. Various BIOS updates have been launched by motherboard manufacturers to provide more conservative power settings, but in our opinion, they don't quite hit the mark. They are either too conservative in some places (leading to unacceptable loss in performance) or they are not conservative enough. We trust our internally developed settings more. We also are concerned with the rise in failure rate, but it is not at a level of severity that changes our CPU recommendations for our customer workflows. 2. We will immediately validate the Intel microcode update when it is released. We will start with internal testing for stability and performance. If it passes that testing, we will begin using it on our shipping configurations as soon as possible. 3. We will contact all our affected customers to provide the Intel microcode update. We will do this after gaining some internal experience and confidence with the update, and have developed detailed guides on how to install it while preserving our recommended BIOS settings. 4. We are extending our warranty to 3 years for all customers affected by this issue, regardless of warranty purchased. With a Puget Systems PC, you should be able to count on it working for you. If we no longer have supply of 13th or 14th Gen processors, we'll upgrade you to a more current generation. We'll all stay tuned together for Intel to release their microcode update in August. In the meantime if you have any questions or concerns, please reach out to me directly or our support team! Tags: Instability, Intel Core, Intel Failure, MicroCode Posted in Business, Hardware, Industry, TechnologyTagged Instability, Intel Core, Intel Failure, MicroCode Who is Puget Systems? Puget Systems builds custom workstations, servers and storage solutions tailored for your work. We provide: Extensive performance testing making you more productive and giving better value for your money Reliable computers with fewer crashes means more time working & less time waiting Support that understands your complex workflows and can get you back up & running ASAP A proven track record as shown by our case studies and customer testimonials Get Started Browse Systems Puget Systems Mobile Laptop Workstation Icon Mobile Puget Systems Tower Workstation Icon Workstations Puget Systems Rackmount Workstation Icon Rackstations Puget Systems Rackmount Server Icon Servers Puget Systems Rackmount Storage Icon Storage Latest Articles * Puget Systems' Perspective on Intel CPU Instability Issues * Puget Mobile Workstation Accessories * Puget Mobile 17'' - Power Profiles * Microsoft Addresses Critical Wi-Fi Driver Vulnerability * Topaz Video AI 5.1 - Consumer GPU Performance Analysis View All Post navigation NAB Show 2024 Recap Puget Systems Logo Build Your Own PC Site Map FAQ facebook instagram linkedin rss twitter youtube Optimized Solutions * Adobe Premiere * Adobe Photoshop * Solidworks * Autodesk AutoCAD * Machine Learning Workstations * Content Creation * Engineering * Scientific PCs * More Support * Online Guides * Request Support * Remote Help Publications * All News * Puget Blog * HPC Blog * Hardware Articles * Case Studies Policies * Warranty & Return * Terms and Conditions * Privacy Policy * Delivery Times * Accessibility About Us * Testimonials * Careers * About Us * Contact Us (c) Copyright 2024 - Puget Systems, All Rights Reserved.