System Performance: UL and BAPCo Benchmarks

Our 2022 test suite for Windows 11-based systems carries over some of the standard benchmarks we have been using over the last several years, including UL's PCMark and BAPCo's SYSmark. Starting this year, we are also including BAPCo's CrossMark multi-platform benchmarking tool.

PCMark 10

UL's PCMark 10 evaluates computing systems for various usage scenarios (generic / essential tasks such as web browsing and starting up applications, productivity tasks such as editing spreadsheets and documents, gaming, and digital content creation). We benchmarked select PCs with the PCMark 10 Extended profile and recorded the scores for various scenarios. These scores are heavily influenced by the CPU and GPU in the system, though the RAM and storage device also play a part. The power plan was set to Balanced for all the PCs while processing the PCMark 10 benchmark.

Futuremark PCMark 10 - Essentials

Futuremark PCMark 10 - Productivity

Futuremark PCMark 10 - Gaming

Futuremark PCMark 10 - Digital Content Creation

Futuremark PCMark 10 - Extended

The first couple of workloads see the JSLM-MINI come out on top. With higher clock speeds compared to the CI331 nano, this is expected. However, gaming and digital content creation see the CI331 nano emerge ahead. This is despite the comparatively weaker GPU. As we shall see in a later section, this has probably got to do with the PL1 and PL2 limits set by ZOTAC for the product.

BAPCo SYSmark 25

BAPCo's SYSmark 25 is an application-based benchmark that uses real-world applications to replay usage patterns of business users in the areas of productivity, creativity, and responsiveness. The 'Productivity Scenario' covers office-centric activities including word processing, spreadsheet usage, financial analysis, software development, application installation, file compression, and e-mail management. The 'Creativity Scenario' represents media-centric activities such as digital photo processing, AI and ML for face recognition in photos and videos for the purpose of content creation, etc. The 'Responsiveness Scenario' evaluates the ability of the system to react in a quick manner to user inputs in areas such as application and file launches, web browsing, and multi-tasking.

Scores are meant to be compared against a reference desktop (the SYSmark 25 calibration system, a Lenovo Thinkcenter M720q with a Core i5-8500T and 8GB of DDR4 memory to go with a 256GB M.2 NVMe SSD). The calibration system scores 1000 in each of the scenarios. A score of, say, 2000, would imply that the system under test is twice as fast as the reference system.

SYSmark 25 - Productivity

SYSmark 25 - Creativity

SYSmark 25 - Responsiveness

SYSmark 25 - Overall

SYSmark 25 also adds energy measurement to the mix. A high score in the SYSmark benchmarks might be nice to have, but potential customers also need to determine the balance between power consumption and the performance of the system. For example, in the average office scenario, it might not be worth purchasing a noisy and power-hungry PC just because it ends up with a 2000 score in the SYSmark 25 benchmarks. In order to provide a balanced perspective, SYSmark 25 also allows vendors and decision makers to track the energy consumption during each workload. In the graphs below, we find the total energy consumed by the PC under test for a single iteration of each SYSmark 25 workload. For reference, the calibration system consumes 8.88 Wh for productivity, 10.81 Wh for creativity, and 19.69 Wh overall.

SYSmark 25 - Productivity Energy Consumption

SYSmark 25 - Creativity Energy Consumption

SYSmark 25 - Overall Energy Consumption

The observations made for the PCMark 10 workloads hold true here also. The JSLM-MINI's clock speeds help it come out on top for the productivity workloads. However, higher power limits help the CI331 nano wrest the lead overall. It is also not a surprise that the overall efficiency is the least for the CI331 nano - it consumes significantly more energy compared to the JSLM-MINI in order to get the honors in the performance score leaderboard.

BAPCo CrossMark 1.0.1.86

BAPCo's CrossMark aims to simplify benchmark processing while still delivering scores that roughly tally with SYSmark. The main advantage is the cross-platform nature of the tool - allowing it to be run on smartphones and tablets as well.

CrossMark 1.0.1.86 - Productivity

CrossMark 1.0.1.86 - Creativity

CrossMark 1.0.1.86 - Responsiveness

CrossMark 1.0.1.86 - Overall

CrossMark is a relatively short-running benchmark (approx. 20 minutes) compared to SYSmark 25 (minimum of 4 hours). Here, the thermal throttling of the Z3 doesn't come into play, and the higher clock speeds in the JSLM-MINI can be sustained right through. The nett result is that this delivers results as one might expect based on the paper specifications - the JSLM-MINI on top with its high clock speeds and a thermal solution that works effectively. The Z3 is next, as long as it doesn't thermally throttle to such as extent that it falls behind the performance of the Celeron N5100 in the CI331 nano.

Setup Notes and Platform Analysis System Performance: Miscellaneous Workloads
Comments Locked

52 Comments

View All Comments

  • mode_13h - Saturday, July 9, 2022 - link

    > tldr both benches would have been a wash one way of the other.

    Huh? If old Skylake is 50% faster, and Jasper Lake is 3.5x as fast as Pi 4 Model B (which seems rather generous), then it wouldn't be "a wash", which is defined as:

    13. an action or situation in which the gains and losses are
    equal, or closely compensate each other.
    (source: http://dict.org/bin/Dict?Form=Dict2&Database=g... )

    or

    8: any enterprise in which losses and gains cancel out; "at the
    end of the year the accounting department showed that it was
    a wash"
    (source: http://dict.org/bin/Dict?Form=Dict2&Database=w... )

    Since both comparisons are projected to be substantially lopsided, I think what you meant to call it is a "washout"?
  • abufrejoval - Thursday, July 14, 2022 - link

    I have a PI4 with 8GB of RAM in a metal case that supports a 2GHz overclock without active cooling: pretty much the best PI you can have these days.

    I also have an Nvidia Tegra based Jetson Nano with 4GB of RAM.

    At 2GHz the PI reaches 272/648 on Geekbench 4, the Tegra has to make do with 206/718 at 1.4GHz. The N6005 Jasper Lake reaches 781/2540 very similar to a Sandy Bridge i7-2600 at 3.8GHz Turbo.

    The Jetson Nano actually does reasonably well on my 43" 4k desktop for basic 2D work, because it has a GPU with 128 Maxwell cores. Of course its CPU power is at the level of a Snapdragon 800 mobile phone.

    The PI struggles badly at 4k, because the GPU has much less muscle. The slightly faster CPU is hard to notice.

    Actually it was when Tom's hardware did a report on a PI compute cluster, that I wanted to retort just how stupid that project was, because you could get a single Jasper Lake Atoms for much less money, that would run rings around that cluster and could in fact simulate it all in software via VMs.

    And that's when I found that finally a Jasper Lake NUC was available for purchase at €200 (including VAT) and immediately ordered one of the first and last ever sold here.

    And yes, it runs rings around both with roughly 4x the CPU power, 64GB of RAM expandability and quite a reasonable GPU performance on a 4k display.

    My favorite usability test is to use the "3D Globe View" on Google Maps under a Chrome based browser on Windows and to then tilt and turn a city landscape there. It's about the most efficient 3D graphics pipeline I've ever seen (puts Flight Simulator to total shame!) and performs quite reasonable on such a Jasper Lake NUC. With Firefox it's much worse on these low power devices, but with a beefy PC you'd never notice.

    After quite a bit of tweaking I managed to get it to work on both the PI and the Tegra at 1920x1080 and the Tegra even gave a bit of interactivity thanks to its much stronger GPU. But on the PI that was about one frame a minute.

    The PI and Nano are toys and ok for the €100 I spent on each.

    A Jasper Lake NUC is quite a reasonable desktop machine and even an interesting micro server for some real workloads.

    At €200 (without RAM or storage) the price/performance ratio is very hard to beat, but evidently none of the vendors really want you to know or buy that. I think it's the major reason you never could.
  • mode_13h - Thursday, July 14, 2022 - link

    > At 2GHz the PI reaches 272/648 on Geekbench 4, the Tegra ... 206/718 at 1.4GHz.

    Keep in mind that Jetson Nano has ostensibly 2x the memory bandwidth of the Pi v4. That surely helps offset the difference in raw CPU performance, as well as with 4k display performance.

    Oh, and if that test was with the machines driving a 4k display, then merely refreshing your monitor will have been using a non-insignificant amount of the Pi's memory bandwidth (about 1 GB/s).

    > N6005 Jasper Lake reaches 781/2540

    Wow! Dual-channel memory configuration, I presume?

    > on the PI that was about one frame a minute.

    Uh... that sure sounds like you were using a software rendering path. The Pi's GPU is trash, but that's simply atrocious!

    > evidently none of the vendors really want you to know or buy that.
    > I think it's the major reason you never could.

    I'm reasonably confident it's actually just supply chain-related. Intel has been steering its limited fab capacity towards more profitable models and probably steering its limited supply of Jasper Lakes to chromebooks, where they're probably desperate not to lose market share.
  • timecop1818 - Friday, July 8, 2022 - link

    There are Chinese mini PCs with
    Intel Celeron N5100 that are like 250$ with 16G ram and 256gb sata SSD.

    https://www.lazada.com.my/products/walkfish-m6-11t...

    there's like 5 different "brands" selling same thing on AliExpress etc. it runs win 10 just fine and is enough for 1080p Minecraft and basic office computing. great deal. most models have Intel 2.5G Ethernet too.
  • Jorgp2 - Friday, July 8, 2022 - link

    I just want a Jasper lake motherboard with plenty of sata and a PCI-E slot
  • mode_13h - Friday, July 8, 2022 - link

    You could get SATA, but not PCIe. According to this, Jasper Lake and Elkhart Lake have only x8 PCIe 3.0 and x2 SATA ports.

    Most boards are probably going to give you a x4 NVMe slot. Then, they could use a 3rd Party SATA controller to give you 4 more ports. Then, if they compromise on the bandwidth to that SATA controller, you can have a second Ethernet port and then a x1 PCIe slot that just might be open-ended (but probably not), to support a graphics card.

    Sorry, but they really kneecapped this platform relative to what it could've been. You might do better with some equivalent Atom-branded CPUs. Atom C-series (Parker Ridge) has 16 integrated SATA ports, x32 PCIe 3.0 lanes, and up to 8 cores. P-series (Snow Ridge) has the same, but up to 24 cores.

    * https://ark.intel.com/content/www/us/en/ark/produc...
    * https://ark.intel.com/content/www/us/en/ark/produc...
  • mode_13h - Friday, July 8, 2022 - link

    Oops, forgot the link for Jasper Lake. For good measure, here's Elkhart Lake, as well.

    * https://ark.intel.com/content/www/us/en/ark/produc...
    * https://ark.intel.com/content/www/us/en/ark/produc...
  • Thala - Friday, July 8, 2022 - link

    Interestingly my 3 years old Surface Pro X scores higher than any of the tested devices in Cinebench R23 under x64 emulation!
  • mode_13h - Saturday, July 9, 2022 - link

    The Surface Pro X from 2019 has a Microsoft SQ1 SoC, which is basically a Snapdragon 8cx and consists of 4x Kryo 495 Gold @ 3 GHz+ 4x Kryo 495 Silver @ 1.80 GHz (manufactured on TSMC 7 nm). According to wikichip, these are tweaked A76 and A55 cores. So, that seems credible, if not exactly an outcome I'd have presumed.

    Something to keep in mind is that Jasper Lake is meant cheap chromebooks. Like, sub-$200 cheap, whereas Snapdragon 8cx is a premium part.
  • nandnandnand - Saturday, July 9, 2022 - link

    They wanted it to be thought of as premium, it's more of an expensive joke. Like Lakefield but with no excuses.

    https://semiaccurate.com/2021/12/01/qualcomm-8cx-g...

    https://www.gizchina.com/2022/01/04/qualcomm-blame...

    Snapdragon 7c (Gen 1?) should be more comparable in price to Jasper Lake. I think I've seen that as low as $170-200. Also, the Apcsilmic Dot 1 and ECS LIVA Mini Box QC710 mini PCs recently launched with the 7c starting at around $219.

    If the leaks about Alder Lake-N are true, it will shake things up, if the price is right.

Log in

Don't have an account? Sign up now