A Look At Triple-GPU Performance And Multi-GPU Scaling, Part 1
by Ryan Smith on April 3, 2011 7:00 AM ESTIt’s been quite a while since we’ve looked at triple-GPU CrossFire and SLI performance – or for that matter looking at GPU scaling in-depth. While NVIDIA in particular likes to promote multi-GPU configurations as a price-practical upgrade path, such configurations are still almost always the domain of the high-end gamer. At $700 we have the recently launched GeForce GTX 590 and Radeon HD 6990, dual-GPU cards whose existence is hedged on how well games will scale across multiple GPUs. Beyond that we move into the truly exotic: triple-GPU configurations using three single-GPU cards, and quad-GPU configurations using a pair of the aforementioned dual-GPU cards. If you have the money, NVIDIA and AMD will gladly sell you upwards of $1500 in video cards to maximize your gaming performance.
These days multi-GPU scaling is a given – at least to some extent. Below the price of a single high-end card our recommendation is always going to be to get a bigger card before you get more cards, as multi-GPU scaling is rarely perfect and with equally cutting-edge games there’s often a lag between a game’s release and when a driver profile is released to enable multi-GPU scaling. Once we’re looking at the Radeon HD 6900 series or GF110-based GeForce GTX 500 series though, going faster is no longer an option, and thus we have to look at going wider.
Today we’re going to be looking at the state of GPU scaling for dual-GPU and triple-GPU configurations. While we accept that multi-GPU scaling will rarely (if ever) hit 100%, just how much performance are you getting out of that 2nd or 3rd GPU versus how much money you’ve put into it? That’s the question we’re going to try to answer today.
From the perspective of a GPU review, we find ourselves in an interesting situation in the high-end market right now. AMD and NVIDIA just finished their major pushes for this high-end generation, but the CPU market is not in sync. In January Intel launched their next-generation Sandy Bridge architecture, but unlike the past launches of Nehalem and Conroe, the high-end market has been initially passed over. For $330 we can get a Core i7 2600K and crank it up to 4GHz or more, but what we get to pair it with is lacking.
Sandy Bridge only supports a single PCIe x16 link coming from the CPU – an awesome CPU is being held back by a limited amount of off-chip connectivity; DMI and a single PCIe x16 link. For two GPUs we can split that out to x8 and x8 which shouldn’t be too bad. But what about three GPUs? With PCIe bridges we can mitigate the issue some by allowing the GPUs to talk to each other at x16 speeds and dynamically allocate CPU-to-GPU bandwidth based on need, but at the end of the day we’re splitting a single x16 lane across three GPUs.
The alternative is to take a step back and work with Nehalem and the x58 chipset. Here we have 32 PCIe lanes to work with, doubling the amount of CPU-to-GPU bandwidth, but the tradeoff is the CPU. Gulftown and Nehalm are capable chips on its own, but per-clock the Nehalem architecture is normally slower than Sandy Bridge, and neither chip can clock quite as high on average. Gulftown does offer more cores – 6 versus 4 – but very few games are held back by the number of cores. Instead the ideal configuration is to maximize performance of a few cores.
Later this year Sandy Bridge E will correct this by offering a Sandy Bridge platform with more memory channels, more PCIe lanes, and more cores; the best of both worlds. Until then it comes down to choosing from one of two platforms: a faster CPU or more PCIe bandwidth. For dual-GPU configurations this should be an easy choice, but for triple-GPU configurations it’s not quite as clear cut. For now we’re going to be looking at the latter by testing on our trusty Nehalem + x58 testbed, which largely eliminates a bandwidth bottleneck in a tradeoff for a CPU bottleneck.
Moving on, today we’ll be looking at multi-GPU performance under dual-GPU and triple-GPU configurations; quad-GPU will have to wait. Normally we only have two reference-style cards of any product on hand, so we’d like to thank Zotac and PowerColor for providing a reference-style GTX 580 and Radeon HD 6970 respectively.
97 Comments
View All Comments
Ryan Smith - Sunday, April 3, 2011 - link
It took awhile, but we finally have 3 120Hz 1080P monitors on the way. So we'll be able to test Eyefinity, 3D Vision, and 3D Vision Surround; all of which have been neglected around here.Kaboose - Sunday, April 3, 2011 - link
I await these tests with breathless anticipation!veri745 - Sunday, April 3, 2011 - link
While this article was very well written, I think it is hardly worth it without the multi-monitor data. No-one (sane) is going to get 3x SLI/CF with a single monitor, so it's mostly irrelevant.The theoretical scaling comparison is interesting, but I'm a lot more interesting in the scaling at 3240x1920 or 5760x1080.
DanNeely - Sunday, April 3, 2011 - link
This is definitely a step in the right direction; but with other sites having 3x 1920x1200 or even 3x 2560x1600 test setups you'll still be playing catchup.RK7 - Sunday, April 3, 2011 - link
Finally! I created account just to write that comment :) That's what's missing and what definitely needs to be tested! Especially 3D Vision Surround - it's good to know if it's worth to put so much money into such setup, because single card may be on the edge of performance for modern games in stereoscopic mode with single monitor (good example is Metro 2033, that blows mind when in 3D, but I found with single GTX 570@900MHz is playable only at 1600x900 in 3D with maximum settings without DoF and AA, and even in such case it could drop for some action scenes with heavy lighting to ~12 fps...). So if three cards can achieve a good scaling and provide performance per monitor for 3 monitors setup close to single card for one monitor, then we're there and it's worth it definitely, but if numbers will be alike to those for single monitor scaling, then folks should be aware that there's no way for maximum visual quality gaming with current hardware on 3 monitors...Dustin Sklavos - Monday, April 4, 2011 - link
Not completely neglected. I've added triple-monitor surround testing to my boutique desktop reviews whenever able. :)Crazymech - Sunday, April 3, 2011 - link
I'm having my doubts about the capabilities of the 920 OC'd to 3.33 GHz matched up with 3 of the most powerful single GPUs.I understand straying away from SB because of the lanes, but you could at least have upped the OC to 3,8-4, which many people do (and I would think most that considers a tripple setup would use).
To underline it I point to the small differences between the 4.5 GHz 2600K and the lower overclocked one in the boutique builds reviews, with the highest clocked CPU coupled with weaker GPU's nipping at the heels of the more powerful GPU.
I suggest you at least experiment in a single test (say metro for example.. or battlefield) what a higher clocked X58 (or the 980's 6 cores) could do to the setup.
If I'm wrong, it would at least be good to know that.
BrightCandle - Sunday, April 3, 2011 - link
The fact that sandy Bridge has a PCI-E lanes problem is grounds for testing the impact.Still I would rather see the numbers on X58 and triple screen gaming before seeing the impact that SB makes the performance of SLI/CF setups.
Ryan Smith - Sunday, April 3, 2011 - link
For what it's worth, 3.33GHz is actually where this specific 920 tops out. It won't take 3.5GHz or higher, unfortunately.We'll ultimately upgrade our testbed to SNB - this article is the impetus for that - but that's not going to happen right away.
Crazymech - Monday, April 4, 2011 - link
It wont take 3.5? Really? That amazes me.Though very unfortunate for the purpose of this test.
The main focus is (of course) always on how the new GPU in relation to a standard CPU improves the framerate, but once in a while it would be interesting what different CPU's do to the GPU and FPS aswell. Like the old Doom III articles of showing Athlon dominating PenIV.
Thanks for the answer anyhows :).