A Timely Discovery: Examining Our AMD 2nd Gen Ryzen Resultsby Ian Cutress & Ryan Smith on April 25, 2018 11:15 AM EST
Last week, we published our AMD 2nd Gen Ryzen Deep Dive, covering our testing and analysis of the latest generation of processors to come out from AMD. Highlights of the new products included better cache latencies, faster memory support, an increase in IPC, an overall performance gain over the first generation products, new power management methods for turbo frequencies, and very competitive pricing.
In our review, we had a change in some of the testing. The big differences in our testing for this review was two-fold: the jump from Windows 10 Pro RS2 to Windows 10 Pro RS3, and the inclusion of the Spectre and Meltdown patches to mitigate the potential security issues. These patches are still being rolled out by motherboard manufacturers, with the latest platforms being first in that queue. For our review, we tested the new processors with the latest OS updates and microcode updates, as well as re-testing the Intel Coffee Lake processors as well. Due to time restrictions, the older Ryzen 1000-series results were used.
Due to the tight deadline of our testing and results, we pushed both our CPU and gaming tests live without as much formal analysis as we typically like to do. All the parts were competitive, however it quickly became clear that some of our results were not aligned with those from other media. Initially we were under the impression that this was as a result of the Spectre and Meltdown (or Smeltdown) updates, as we were one of the few media outlets to go back and perform retesting under the new standard.
Nonetheless, we decided to take an extensive internal audit of our testing to ensure that our results were accurate and completely reproducible. Or, failing that, understanding why our results differed. No stone was left un-turned: hardware, software, firmware, tweaks, and code. As a result of that process we believe we have found the reason for our testing being so different from the results of others, and interestingly it opened a sizable can of worms we were not expecting.
An extract from our Power testing script
What our testing identified is that the source of the issue is actually down to timers. Windows uses timers for many things, such as synchronization or ensuring linearity, and there are sets of software relating to monitoring and overclocking that require the timer with the most granularity - specifically they often require the High Precision Event Timer (HPET). HPET is very important, especially when it comes to determining if 'one second' of PC time is the equivalent to 'one second' of real-world time - the way that Windows 8 and Windows 10 implements their timing strategy, compared to Windows 7, means that in rare circumstances the system time can be liable to clock shift over time. This is often highly dependent on how the motherboard manufacturer implements certain settings. HPET is a motherboard-level timer that, as the name implies, offers a very high level of timer precision beyond what other PC timers can provide, and can mitigate this issue. This timer has been shipping in PCs for over a decade, and under normal circumstances it should not be anything but a boon to Windows.
However, it sadly appears that reality diverges from theory – sometimes extensively so – and that our CPU benchmarks for the Ryzen 2000-series review were caught in the middle. Instead of being a benefit to testing, what our investigation found is that when HPET is forced as the sole system timer, it can sometimes a hindrance to system performance, particularly gaming performance. Worse, because HPET is implemented differently on different platforms, the actual impact of enabling it isn't even consistent across vendors. Meaning that the effects of using HPET can vary from system to system, as well as the implementation.
And that brings us to the state HPET, our Ryzen 2000-series review, and CPU benchmarking in general. As we'll cover in the next few pages, HPET plays a very necessary and often very beneficial role in system timer accuracy; a role important enough that it's not desirable to completely disable HPET – and indeed in many systems this isn't even possible – all the while certain classes of software such as overclocking & monitoring software may even require it. However for a few different reasons it can also be a drain on system performance, and as a result HPET shouldn't always be used. So let's dive into the subject of hardware timers, precision, Smeltdown, and how it all came together to make a perfect storm of volatility for our Ryzen 2000-series review.
Post Your CommentPlease log in or sign up to comment.
View All Comments
jjj - Wednesday, April 25, 2018 - link" however it is clear that there is still margin that benefits Intel at the most popular resolutions, such as 1080p."
That's a false and highly misleading statement, it's not about the resolution it is about an over-dimensioned GPU for a given resolution so , easiest way to put it, high FPS gaming.
90% will game at 1080p with a 1060 not a 1080.
Marketing might have moved rich children from 30-60FPS to 120FPS but people are not made out of money and you know very well how limited high end GPU volumes are.
For now you should test with and without HPET at least for a few results and highlight the HPET impact..
One thing I did not notice being addressed after flying over the article is the accuracy of the results with HPET disabled. How certain are you that the results are not way off to favor Intel now?
Maxiking - Wednesday, April 25, 2018 - linkThe only one misleading and false statement is that 90% will play at 1080p with a 1060.
Remember, in the future, 1160 will be probably more powerful than 1080, 1260 than 1280 and so on. The bottleneck is still here, not gonna disappear, will get only bigger with more powerfull cards.
Regardless, how certain are you that the results are not way off to favour AMD now?
Maxiking - Wednesday, April 25, 2018 - linkDamn it, why there is no option to edit messages. *Powerful* kek.
RafaelHerschel - Wednesday, April 25, 2018 - linkGames get more demanding. I'm convinced that at some point 1080p will become obsolete, but we are not there yet. For me 1080p maxed out (sometimes with DSR enabled) looks good enough and ensures that I get the smoothness that is important to me.
mapesdhs - Sunday, May 6, 2018 - linkWhere's the evidence games are becoming more demanding? If that were true, typical frame rate spreads in reviews would not be going through the roof. It's been a very long time since any GPU review article talked about new visual features to enable more complex and immersive worlds. These days, all the talk is about performance and resolution support, not fidelity.
jjj - Wednesday, April 25, 2018 - linkPeople buy GPUs by targeting the FPS they need inside a budget and sane people do not buy more than they need.
And ofc as someone else pointed out, games evolve too, otherwise we would not need better GPUs.
Remember that GPUs have been around for decades, we know how things go.
eek2121 - Sunday, April 29, 2018 - linkBenchmarks should not be done on a 1060. The purpose of a CPU benchmark is to measure CPU performance. IMO a 1080ti at MINIMUM should be used to elimininate GPU bottlenecks. There are some games out there that still bottleneck at 1080p.
eva02langley - Wednesday, April 25, 2018 - linkYou are damn wrong. Sure you can see CPU bottleneck... however, can you? Now with HPET put into light, you can alter results dramatically for Intel, however is HPET a default function for the OS?
Basically, you are telling me that benchmarks should have HEPT off, a configuration that is supposed to be set as default, just because we can see which architecture is better in a non conventional use?
So what is the value of those precious 1080p benchmarks if they don't represent the configuration the typical end user is going to use the product for in its intended use?
It is coming back to the USE CASE.
If a budget user buy an RX 560, CPU choice at 1080p won't matter.
If a mid range user buy an RX 580/1060 GTX, CPU choice at 1080p won't matter.
If a high user buy a 1080 GTX/Vega 64, CPU choice at 1080p @ 144 Hz will barely matter.
If an enthusiasm user buy a 1080 TI, CPU choice will matter @ 144 Hz.
And now... what happens with HPET in the picture? How can you accurately render results without biasing yourself anymore?
One thing for sure, Intel needs to fix their stuff.
eva02langley - Wednesday, April 25, 2018 - link"If an enthusiasm user buy a 1080 TI, CPU choice at 1080p will matter @ 144 Hz."
malakudi - Wednesday, April 25, 2018 - linkThank you for the analysis. Can you somehow verify that very large variations (RoTR 1-2-3, Civ6) of performance on i7-8700K with HPET not forced are real? Is it possible that the reported FPS are wrongly calculated when using non-HPET timer? Can you also get a comment from the developers of those games about this result? 45,76 and 69% performance difference does not seem normal.