EPYC has a SOC inside the package and does not require a traditional chipset. From a look at the pictures the visible PCB's are low density - possibly power supply (the bigger PCBs) and I/O (the smaller ones). Remember as part of a supercomputer, very few of the CPUs will have traditional I/O (disks etc).
Shasta is just a model of available systems from Cray. Sort of like how Ford makes several different models of cars. Shasta systems can be configured with different cpu's and other options just like a Ford Focus can be configured with different engines, transmissions, etc. Rome cpu's won't be available until early 2019 that's probably why they're not officially listed as an available configuration yet.
Ok that is what I thought - it just a different model of line - different vendors based on customer requirements and preference. Of course 2019 will have other models also.
The title "Managing 16 Rome CPUs in 1U" , then "eight processors in a 1U system", then "Firstly, the eight CPU design with split memory cooling was obvious, but CoolIT stated that these processor cooling plates were designed to cool CPUs above and below, for a total cooling power of 500W apiece. This means that in this design, the full cooling apparatus could go up to sixteen CPUs and 4kW."
I imagine you're limited to 1 die per channel with a 16 processor configuration, so you'd remove half the ram that's in the current picture, and turn it upside down.
Indeed there does. The edit to the article seems to have cleared this up, but just to be clear, there is already only 1DPC in the picture - EPYC has 8C memory, and there are quite clearly 8 dimms per CPU here :) Hence my question!
Article updated to "Correct" 8 CPU now. Which makes more sense, if it was 16 CPU the density improvement on 1U would have been 4x rather than then 2x as stated.
Which leads to next question, 4KW per 8 CPU? that is roughly ~420W per CPU excluding DIMM. That is a lot for a single CPU. Last time Intel tried a 500W push overclock it requires liquid nitrogen. So is that 4KW number correct as well?
Water cooling loops have to be oversized for heat dissipation or you'll have a chance that you'll get a hot spot in the loop, it's just a safety factor like any other engineering.
For example the configuration Cray has chosen here is going to supply the last CPU's in the loop with less water because the pipe are connected in a serial configuration and the water will choose the shortest path (unless they run this at extreme pressures). The extra cooling could be to cover that scenario. They could run this at lower wattage cooling but they'd need a manifold at the inlet to each server and individual lines to each cooling loop. The expense of that configuration, not to mention the lack of room for 8 lines of half inch tubing, is likely prohibitive when considered against just having excess cooling capacity to cover it.
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
26 Comments
Back to Article
HStewart - Monday, November 19, 2018 - link
I am curious how many Chipsets the systems has - it looks like to me it possibly have 4 or 8 chipsets on the same board.SaturnusDK - Tuesday, November 20, 2018 - link
EPYC CPUs do not require chipsets at all.HStewart - Tuesday, November 20, 2018 - link
So the question is it seed as 8 desktop computers or one.SaturnusDK - Tuesday, November 20, 2018 - link
No matter how many of these blades you have they can be seen as one computer.Duncan Macdonald - Monday, November 19, 2018 - link
EPYC has a SOC inside the package and does not require a traditional chipset. From a look at the pictures the visible PCB's are low density - possibly power supply (the bigger PCBs) and I/O (the smaller ones). Remember as part of a supercomputer, very few of the CPUs will have traditional I/O (disks etc).HurleyBird - Monday, November 19, 2018 - link
Quad channel memory?HurleyBird - Monday, November 19, 2018 - link
Nevermind, optical illusion!HStewart - Monday, November 19, 2018 - link
When I search for Cray Shasta - I see another CPU then these. So I not sure this is real information.Ej24 - Tuesday, November 20, 2018 - link
Shasta is just a model of available systems from Cray. Sort of like how Ford makes several different models of cars. Shasta systems can be configured with different cpu's and other options just like a Ford Focus can be configured with different engines, transmissions, etc. Rome cpu's won't be available until early 2019 that's probably why they're not officially listed as an available configuration yet.HStewart - Tuesday, November 20, 2018 - link
Ok that is what I thought - it just a different model of line - different vendors based on customer requirements and preference. Of course 2019 will have other models also.iwod - Tuesday, November 20, 2018 - link
The title "Managing 16 Rome CPUs in 1U" , then "eight processors in a 1U system", then "Firstly, the eight CPU design with split memory cooling was obvious, but CoolIT stated that these processor cooling plates were designed to cool CPUs above and below, for a total cooling power of 500W apiece. This means that in this design, the full cooling apparatus could go up to sixteen CPUs and 4kW."So is it 8 CPU? or 16 CPU per 1U? I am confused.
SaturnusDK - Tuesday, November 20, 2018 - link
16 CPUs in 1U. 8 on the top side. 8 on the bottom side. But still in the 1U size.Dug - Tuesday, November 20, 2018 - link
Thanks, because that was clear as mud.iwod - Tuesday, November 20, 2018 - link
And yet Servethehome suggest only 8 CPU and 1024 Thread. So who is right who is wrong?rahvin - Tuesday, November 20, 2018 - link
Look at photo #2, they show half the water blocks with the Rome card on the "top" side.Valantar - Tuesday, November 20, 2018 - link
I get that you can (theoretically) fit 8 more CPUs on top of the waterblocks here, but where does the RAM for the additional CPUs go?tomatotree - Tuesday, November 20, 2018 - link
I imagine you're limited to 1 die per channel with a 16 processor configuration, so you'd remove half the ram that's in the current picture, and turn it upside down.tomatotree - Tuesday, November 20, 2018 - link
DIMM, not die. There really needs to be an edit button.Valantar - Wednesday, November 21, 2018 - link
Indeed there does. The edit to the article seems to have cleared this up, but just to be clear, there is already only 1DPC in the picture - EPYC has 8C memory, and there are quite clearly 8 dimms per CPU here :) Hence my question!iwod - Tuesday, November 20, 2018 - link
Article updated to "Correct" 8 CPU now. Which makes more sense, if it was 16 CPU the density improvement on 1U would have been 4x rather than then 2x as stated.Which leads to next question, 4KW per 8 CPU? that is roughly ~420W per CPU excluding DIMM. That is a lot for a single CPU. Last time Intel tried a 500W push overclock it requires liquid nitrogen. So is that 4KW number correct as well?
rahvin - Tuesday, November 20, 2018 - link
Water cooling loops have to be oversized for heat dissipation or you'll have a chance that you'll get a hot spot in the loop, it's just a safety factor like any other engineering.For example the configuration Cray has chosen here is going to supply the last CPU's in the loop with less water because the pipe are connected in a serial configuration and the water will choose the shortest path (unless they run this at extreme pressures). The extra cooling could be to cover that scenario. They could run this at lower wattage cooling but they'd need a manifold at the inlet to each server and individual lines to each cooling loop. The expense of that configuration, not to mention the lack of room for 8 lines of half inch tubing, is likely prohibitive when considered against just having excess cooling capacity to cover it.
iwod - Wednesday, November 21, 2018 - link
Thx for the explanation!. Now just waiting for EPYC 2 to arrive.SetiroN - Tuesday, November 20, 2018 - link
Not to sound like an ass but... are those cellphone pictures? They're pretty bad.PaoDeTech - Tuesday, November 20, 2018 - link
How many blades do you need to run Crysis @60 fps?Godaniabilo - Thursday, November 22, 2018 - link
Hi, my name is Isabel. I like roleplay and storyplay. write me there http://badoos.mlMensentona - Friday, November 23, 2018 - link
Hello everyone) I am a girl who is tired without attention and looking for a young man. Write me there http://badoos.ml