Tuesday, August 30, 2022

About M1 Killers and Bad Reviewers

I am in a position where I regularly need to recommend computer and audio hardware to students, artists, and others. I can't test all of it myself, so I rely on people who dedicate themselves to this task. Essentially I act as an aggregator, leveraging my ability to spot BS. My evaluation process is made difficult by the number of poor reviewers on YouTube. In truth, it takes a lot to be a good reviewer. But there are far too many popular channels that are driven by brand loyalty instead of technical knowledge.

I started writing a comment for one such video, cleverly entitled "They said THIS was the M1 Killer.. <crying laughin emoji> AMD 5900HX Mini PC!" Yes, I should have known what I was in for from the title alone. Unfortunately with half a million views and hundreds of positive comments, such content gets far more praise than it deserves.

My comment got out of hand, in terms of length if nothing else, so I relocated it here to my blog.

Hello Mr. Tech,

I should start by saying that I didn't expect much from your review given the title, which wallows in affect, makes an incorrect claim, and doesn't even know what product it's reviewing.

When it comes to performance, no-one has ever said that the AMD 5900HX, a CPU with limited graphics capability, was an "M1 Killer". Only you made this claim, as a clickbait headline. In fact, everyone knows that Apple's M1 chip is a wonderful accomplishment, packing a great deal of processing power into a mobile package. It's sets a new bar for what can be accomplished in the low power regime.

Not content with simply praising where praise is due, you instead set up a false conflict between different systems. When you inevitably arise victorious, your minions can acclaim you Emperor. We all know how this works.

From the beginning, it's confusing as to what you are reviewing. Is it the chip itself, as some of your metrics imply, or is it the computer, the EliteMini HX90? I'll assume the latter, because you are simply not equipped to test the chip in itself. Strange, then, that you don't mention this specific model in your headline.

Your description launches with a falsehood: "AMD's 5900HX is by far the FASTEST mobile APU chip on the market." Actually, it's only marginally faster than the 5600G and underperforms the Ryzen 7 5700G (on Passmark and Cinebench R23). Naturally, truth only gets in the way of your mandate.

Your testing suffers from being one-sided, badly explained, becoming ultimately an exercise in cherry-picking data.

At 4:46 you show a graph of "sustained wattage", whatever you mean by that. The M1 Mac is graphed at 15W, which is a number you got off a spec sheet, not by testing. We know it's wrong, because back at the 1:24 mark, you mention the system maxes out at 39W. Meanwhile the HX90 is graphed to 65W, which again is its maximum, not sustained value. In fact we can even see (at 4:28) that the system lowers to 30W and even 23W. We don't know how low it gets, because you cut away from the test footage, after making a big deal about the maximum. So, it could be that the Mac M1 is 39W against the AMD's 23W. That graph would sure look different, right? How do we know which is correct? Well, first we must define our terms, testing methodology, and procedure. Otherwise anyone can pick any numbers to justify any conclusion.

Admittedly it's desirable to use no more power than required, but do any users care about how much power their computer is drawing? Not unless it is running off a battery, because then this factor limits uptime. No, users worry far more about actually getting work done.

A few moments later you claim "I'm guessing that we are getting some thermal throttling..." without, you know, actually testing this fact. Judging by what other more fastidious reviewers have written, it's very unlikely that there's throttling. Let me quote from Sebastian Bade, Senior Tech Writer at Notebook Check.

"Thus, there is no reason to panic about the temperatures. It is not surprising that the 3D performance does not drop even after the stress test. The result of the 3DMark11 benchmark is almost identical compared to the cold start and only fluctuates by a few points in the range of the measurement tolerances."

I am sure you mention this topic because it's widely known who is the king of throttling: Apple. Prior to the M1 chip being introduced, all of their laptops were advertised at speeds the systems could never attain. This has been tested and verified multiple times.

The gaming comparison is a joke. How long did it take you to find one game where the Mac had better FPS in a synthetic test? Oh, and was this a test of maximum FPS or sustained FPS? How are the drop-outs? Every gamer knows that it's the lowest frame rate numbers that matter the most.

Unless you compare a suite of different games across different use cases (screen resolution, graphics settings), you can't make any conclusion about gaming performance. Choosing just one game is transparent manipulation. I'm surprised your readers don't notice. Does anyone actually buy a Mac for gaming?

There's also something fishy about other test scores. A trip to the Cinebench website will reveal that your numbers are much lower than what everyone else gets for the 5900HX. Perhaps the cheap single-channel memory in the default EliteMini HX90 build is limiting performance. This is why it's important to be clear about what you are evaluating: the chip or the computer. your readers could then learn something valuable. Perhaps they would then choose to buy the barebones HX90 system and stock it with memory of their own choosing. But since this is something you absolutely cannot do with the Apple product, I can understand why you didn't wish to mention it.

You generously mention that the EliteMini HX90 has more ports than the Mac Mini, something of no little consequence to daily convenience. In fact you can run four external monitors on the HX90 and still have six free USB ports! You skip lightly over this and three further advantages of this computer.

First, the RAM, storage, and WiFi can be freely upgraded at any time. As your needs change, so does this computer, extending its useful life. By contrast, you cannot upgrade the Mac Mini at all. Apple believes in vendor and customer lock-in, an unsustainable practice that looks particularly bad in today's climate catastrophe.

The second advantage is that you can add two extra drives to the HX90, using the standard 2.5" laptop form factor. To optimise a production workflow I dedicate one SSD to the OS and applications, a second for cache, a third for project files. The Mac is limited to one drive, which is sad for a desktop computer.

Third advantage? At time of initial purchase, you can double the RAM and SSD on the EliteMini for about $100. That makes it far more capable than the Mac, for the vast majority of tasks. It's widely known that Apple has a policy of fleecing customers for any increase in specifications above base. That's how they make all their money. They steal it from willing victims like you, Max. (May I call you Max?)

The value proposition of these computers is quite lopsided. Here's the Mac Mini M1 in three configurations:
16/256 = $900
16/512 = $1100
32/512 = unavailable at any price

Here's the EliteMini HX90:
16/256 = $650
16/512 = $670
32/512 = $750

So, you can purchase a 32GB 512GB HX90 and stock it with 2Gb worth of additional storage, for less than the cheapest Mac Mini. Wouldn't you rather have double the memory (32 versus 16GB)? Wouldn't you rather have ten times the storage (2.5 versus .25 TB)? Wasn't this value proposition worth mentioning?

(I note that the HX90 has come down in price since your video, as non-Apple computers generally do. This doesn't fundamentally change the conclusions. Add $100 to the HX90 pricing if you wish to time travel back a year.)

When it comes to video performance, there is no doubt the M1 is a champ. That's because Apple have put codec code on the M1 chip itself, optimising it for contemporary video streams. (Of course when new codecs arise this advantage might be lost. Buy a new computer, I guess?)

The bottom line is that your video compares an optimised system from the most profitable company in the world to a no-name system from some lads in China. Seriously. Considering the massive scale tilted in Apple's favour, the EliteMini does very well. Once you learn how to actually use Resolve (hint: proxies) your timeline won't stutter. Yes, you will still need to wait longer on the rare occasions you need to render. You can use these precious minutes to have a coffee. Smell the flowers. Get some perspective.

You dearly need it.

P.S. I've never used an EliteMini and don't endorse them. I do use Apple M1 computers but won't endorse them either.


No comments:

Post a Comment