Tweet
By Joel Hruska
Over the past few months, we’ve seen a disturbing trend from first Kingston, and now PNY. Manufacturers are launching SSD's with one hardware specification, and then quietly changing the hardware configuration after reviews have gone out. The impacts have been somewhat different (more on that in a moment) but in both cases, unhappy customers are loudly complaining that they’ve been cheated, tricked into paying for a drive they otherwise wouldn’t have purchased.
A blog post at Tweaktown details a reader’s unhappy experience with a new PNY Optima drive. This individual bought the drive expecting that it would feature a Silicon Motion controller, only to discover that the drive they purchased had a different firmware version and a different, SandForce-based controller.
When Tweaktown inquired as to the situation, PNY sent back the following: “Yes we did ship some Optima SSD’s with SandForce controllers, but only if they meet the minimum advertised performance levels (in most of the benchmark tests, LSI controllers outperform SMI controllers).” (emphasis added)
Practically, it’s not clear what “minimum advertised performance levels” means because PNY hasn’t actually advertised any apart from a spec sheet that says “up to 60,000 IOPS” with a footnote reading “*Performance based on PNY Internal tests using an IOMeter.” It’s not even clear if they mean Intel’s IOMeter program, or something else — and IOMeter can be configured to test a dizzying array of configuration options.
The Tweaktown blog post doesn’t make clear what performance issues were observed, though it implies they exist. We have more concrete evidence with Kingston and its V300.
Kingston’s bait-and-switch
Kingston’s V300 debuted well over a year ago to generally strong reviews — I covered the drive for PC Mag. This was a budget SSD family with a great price/performance ratio and one of the first budget drives that didn’t have any “Gotcha” in the performance department.
At least, it didn’t have any initially — further analysis from websites like Anandtech and Nordichardware demonstrated that at some point, Kingston made the switch from synchronous to asynchronous NAND.
Asynchronous vs. synchronous NAND
Synchronous NAND moves data on both the rising and falling edge of a clock signal, similar to DDR memory. Synchronous NAND use a clock signal, while asynchronous NAND’s performance is tied to a separate strobe (the RE signal above) that times read/write events separately from the data clock. In the future, asynchronous and synchronous NAND may actually operate at equivalent speeds due to other architecture changes, but for now asynchronous NAND is much, much slower than its counterpart.
This can be seen in numerous tests run by websites like Nordichardware, and in Kingston’s own published documentation, which shows the V120S (the fast version) speeding past the asynchronous counterpart.
Data courtesy of Nordichardware
Data from Kingston’s own documentation
Kingston, to be sure, doesn’t see it that way — the company has released a statement previously saying: “If the drive is tested with ATTO Benchmark the speed results should meet or exceed advertised specifications which is officially 450MB/s read and write.”
This is akin to saying: “Both Kingston SSD's should reach approximately the same velocity when fired out of a cannon,” at least as far as real-world drive performance is concerned. The reason many publications have moved away from ATTO is precisely because it’s such a meaningless test. It’s meaningless enough, in fact, that Kingston’s own data charts show enormous performance differences between the two products.
Imagine this in other contexts
Imagine buying a high-end Core i7 or AMD CPU, opening the box, and finding a mid-range part sitting there with an asterisk and the label “Performs Just Like Our High End CPU In Single-Threaded SuperPi!”
Anybody want a 300-500W PSU? How about a GTX 770ish from Nvidia?
Somehow, I think not. In any other context, this kind of substitution would be utterly unacceptable.
The fact that SSD's are far faster than hard drives (Kingston actually references this as an excuse) is not an excuse. It’s not clear yet if the PNY drive controller shift is hitting performance as significantly as Kingston’s NAND swap, but this is a trend that needs to stop, now.
The SSD market is cutthroat these days, but that means the last thing a company wants to do is scare people off its own hardware.
If reviewers can’t trust that the performance they see is the performance end-users will receive, they’ll never recommend your components. Quoting terrible benchmarks that don’t test anything worth testing or providing utterly impenetrable performance claims is not the way to build trust in a brand.
Even when benchmark performance is equal, controller changes can cause different reliability results. This question is difficult enough without behind-the-scenes hardware changes.
Over the past few months, we’ve seen a disturbing trend from first Kingston, and now PNY. Manufacturers are launching SSD's with one hardware specification, and then quietly changing the hardware configuration after reviews have gone out. The impacts have been somewhat different (more on that in a moment) but in both cases, unhappy customers are loudly complaining that they’ve been cheated, tricked into paying for a drive they otherwise wouldn’t have purchased.
A blog post at Tweaktown details a reader’s unhappy experience with a new PNY Optima drive. This individual bought the drive expecting that it would feature a Silicon Motion controller, only to discover that the drive they purchased had a different firmware version and a different, SandForce-based controller.
When Tweaktown inquired as to the situation, PNY sent back the following: “Yes we did ship some Optima SSD’s with SandForce controllers, but only if they meet the minimum advertised performance levels (in most of the benchmark tests, LSI controllers outperform SMI controllers).” (emphasis added)
Practically, it’s not clear what “minimum advertised performance levels” means because PNY hasn’t actually advertised any apart from a spec sheet that says “up to 60,000 IOPS” with a footnote reading “*Performance based on PNY Internal tests using an IOMeter.” It’s not even clear if they mean Intel’s IOMeter program, or something else — and IOMeter can be configured to test a dizzying array of configuration options.
The Tweaktown blog post doesn’t make clear what performance issues were observed, though it implies they exist. We have more concrete evidence with Kingston and its V300.
Kingston’s bait-and-switch
Kingston’s V300 debuted well over a year ago to generally strong reviews — I covered the drive for PC Mag. This was a budget SSD family with a great price/performance ratio and one of the first budget drives that didn’t have any “Gotcha” in the performance department.
At least, it didn’t have any initially — further analysis from websites like Anandtech and Nordichardware demonstrated that at some point, Kingston made the switch from synchronous to asynchronous NAND.
Asynchronous vs. synchronous NAND
Synchronous NAND moves data on both the rising and falling edge of a clock signal, similar to DDR memory. Synchronous NAND use a clock signal, while asynchronous NAND’s performance is tied to a separate strobe (the RE signal above) that times read/write events separately from the data clock. In the future, asynchronous and synchronous NAND may actually operate at equivalent speeds due to other architecture changes, but for now asynchronous NAND is much, much slower than its counterpart.
This can be seen in numerous tests run by websites like Nordichardware, and in Kingston’s own published documentation, which shows the V120S (the fast version) speeding past the asynchronous counterpart.
Data courtesy of Nordichardware
Data from Kingston’s own documentation
Kingston, to be sure, doesn’t see it that way — the company has released a statement previously saying: “If the drive is tested with ATTO Benchmark the speed results should meet or exceed advertised specifications which is officially 450MB/s read and write.”
This is akin to saying: “Both Kingston SSD's should reach approximately the same velocity when fired out of a cannon,” at least as far as real-world drive performance is concerned. The reason many publications have moved away from ATTO is precisely because it’s such a meaningless test. It’s meaningless enough, in fact, that Kingston’s own data charts show enormous performance differences between the two products.
Imagine this in other contexts
Imagine buying a high-end Core i7 or AMD CPU, opening the box, and finding a mid-range part sitting there with an asterisk and the label “Performs Just Like Our High End CPU In Single-Threaded SuperPi!”
Anybody want a 300-500W PSU? How about a GTX 770ish from Nvidia?
Somehow, I think not. In any other context, this kind of substitution would be utterly unacceptable.
The fact that SSD's are far faster than hard drives (Kingston actually references this as an excuse) is not an excuse. It’s not clear yet if the PNY drive controller shift is hitting performance as significantly as Kingston’s NAND swap, but this is a trend that needs to stop, now.
The SSD market is cutthroat these days, but that means the last thing a company wants to do is scare people off its own hardware.
If reviewers can’t trust that the performance they see is the performance end-users will receive, they’ll never recommend your components. Quoting terrible benchmarks that don’t test anything worth testing or providing utterly impenetrable performance claims is not the way to build trust in a brand.
Even when benchmark performance is equal, controller changes can cause different reliability results. This question is difficult enough without behind-the-scenes hardware changes.