主机并行运算问题

[李东岳]

@青山 我那个标准算例的log

[青山]

@李东岳

/*---------------------------------------------------------------------------*\
 =========                 |
 \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox
  \\    /   O peration     | Website:  https://openfoam.org
   \\  /    A nd           | Version:  7
    \\/     M anipulation  |
\*---------------------------------------------------------------------------*/
Build  : 7-3bcbaf946ae9
Exec   : pisoFoam -parallel
Date   : Jan 18 2022
Time   : 10:22:57
Host   : "ps"
PID    : 4116
I/O    : uncollated
Case   : /home/ps/OpenFoam/benchMark/core_16
nProcs : 16
Slaves : 
15
(
"ps.4117"
"ps.4118"
"ps.4119"
"ps.4120"
"ps.4121"
"ps.4122"
"ps.4123"
"ps.4124"
"ps.4125"
"ps.4126"
"ps.4127"
"ps.4128"
"ps.4129"
"ps.4130"
"ps.4131"
)

Pstream initialized with:
   floatTransfer      : 0
   nProcsSimpleSum    : 0
   commsType          : nonBlocking
   polling iterations : 0
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 10)
allowSystemOperations : Allowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0

Reading field p

Reading field U

Reading/calculating face flux field phi

Selecting incompressible transport model Newtonian
Selecting turbulence model type laminar
Selecting laminar stress model Stokes
No MRF models present

Creating finite volume options from "constant/fvOptions"

Selecting finite volume options model type meanVelocityForce
   Source: momentumSource
   - selecting all cells
   - selected 2097152 cell(s) with volume 10
   Initial pressure gradient = 0


Starting time loop

fieldAverage fieldAverage1:
   Starting averaging at time 0

Time = 0.1

Courant Number mean: 0.731999 max: 1.22083
GAMG:  Solving for p, Initial residual = 0.629082, Final residual = 0.00572723, No Iterations 3
time step continuity errors : sum local = 1.06089e-05, global = -1.10517e-20, cumulative = -1.10517e-20
Pressure gradient source: uncorrected Ubar = 0.133489, pressure gradient = 0.000115089
GAMG:  Solving for p, Initial residual = 0.0561029, Final residual = 0.000467237, No Iterations 4
time step continuity errors : sum local = 1.28615e-06, global = -3.71636e-20, cumulative = -4.82152e-20
Pressure gradient source: uncorrected Ubar = 0.133489, pressure gradient = 0.000117874
GAMG:  Solving for p, Initial residual = 0.0152276, Final residual = 9.92056e-07, No Iterations 30
time step continuity errors : sum local = 2.62628e-09, global = -2.27344e-20, cumulative = -7.09497e-20
Pressure gradient source: uncorrected Ubar = 0.133489, pressure gradient = 0.000117425
ExecutionTime = 5.66 s  ClockTime = 6 s

Time = 0.2

Courant Number mean: 0.731698 max: 1.1641
GAMG:  Solving for p, Initial residual = 0.758295, Final residual = 0.00701504, No Iterations 3
time step continuity errors : sum local = 9.27309e-06, global = -2.6639e-13, cumulative = -2.66391e-13
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 0.000119203
GAMG:  Solving for p, Initial residual = 0.0736589, Final residual = 0.000696871, No Iterations 6
time step continuity errors : sum local = 6.00649e-07, global = -2.6639e-13, cumulative = -5.32781e-13
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 0.000117205
GAMG:  Solving for p, Initial residual = 0.0148478, Final residual = 9.08119e-07, No Iterations 43
time step continuity errors : sum local = 7.91859e-10, global = -2.66391e-13, cumulative = -7.99172e-13
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 0.00011676
ExecutionTime = 11.78 s  ClockTime = 12 s

Time = 0.3

Courant Number mean: 0.731607 max: 1.18248
GAMG:  Solving for p, Initial residual = 0.476883, Final residual = 0.00315427, No Iterations 4
time step continuity errors : sum local = 2.37198e-06, global = -3.08998e-13, cumulative = -1.10817e-12
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 0.000119187
GAMG:  Solving for p, Initial residual = 0.0807666, Final residual = 0.000618685, No Iterations 5
time step continuity errors : sum local = 3.9245e-07, global = -3.08998e-13, cumulative = -1.41717e-12
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 0.000117229
GAMG:  Solving for p, Initial residual = 0.018049, Final residual = 8.80851e-07, No Iterations 45
time step continuity errors : sum local = 5.48708e-10, global = -3.08998e-13, cumulative = -1.72617e-12
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 0.00011678
ExecutionTime = 18.04 s  ClockTime = 18 s

Time = 0.4

Courant Number mean: 0.731599 max: 1.17717
GAMG:  Solving for p, Initial residual = 0.1167, Final residual = 0.000969819, No Iterations 5
time step continuity errors : sum local = 6.01509e-07, global = -3.34128e-13, cumulative = -2.06029e-12
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 0.00011923
GAMG:  Solving for p, Initial residual = 0.0809347, Final residual = 0.000631617, No Iterations 5
time step continuity errors : sum local = 3.90613e-07, global = -3.34128e-13, cumulative = -2.39442e-12
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 0.00011725
GAMG:  Solving for p, Initial residual = 0.0184664, Final residual = 6.81898e-07, No Iterations 47
time step continuity errors : sum local = 4.15027e-10, global = -3.34128e-13, cumulative = -2.72855e-12
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 0.000116801
ExecutionTime = 24.52 s  ClockTime = 25 s

Time = 0.5

Courant Number mean: 0.731615 max: 1.19242
GAMG:  Solving for p, Initial residual = 0.108148, Final residual = 0.00095674, No Iterations 5
time step continuity errors : sum local = 5.80503e-07, global = -3.47575e-13, cumulative = -3.07612e-12
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 0.000119271
GAMG:  Solving for p, Initial residual = 0.0827827, Final residual = 0.000641763, No Iterations 5
time step continuity errors : sum local = 3.89751e-07, global = -3.47575e-13, cumulative = -3.4237e-12
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 0.000117272
GAMG:  Solving for p, Initial residual = 0.0189672, Final residual = 9.67464e-07, No Iterations 42
time step continuity errors : sum local = 5.75885e-10, global = -3.47575e-13, cumulative = -3.77127e-12
Pressure gradient source: uncorrected Ubar = 0.1335, pressure gradient = 0.000116822
ExecutionTime = 30.63 s  ClockTime = 31 s

[李东岳]

从log上看没问题，不清楚为什么这么慢，之前他们别人也遇到过类似这种问题。所以买CFD服务器最好实测一下。

[星星星星晴]

@李东岳 所以我之前问他们内存是怎么搞的，一般情况下，这玩意cpu肯定没问题，硬盘应该也不是瓶颈，内存是不是走的多通道模式是有可能是限制，但是看了一下，好像也是多通道的。原生Ubuntu，不是走虚拟机什么的，Case的log看起来也没什么问题。 也不是多节点，都是本机的，也就没啥能慢的情况了啊。。而且这个应该不是自己弄的组装机吧？所以真心看不出来咋回事，重装一下系统看看好了～ 说不准神奇的解决了，或者试试老ubuntu ub18啊 ub16什么的

[星星星星晴]

@青山 最简单粗暴的说，你们的内存条插满主板了么？或者说内存条有没有隔一个插一个，插了一半？没自己装过cluster，但是这玩意基本原理都是通用的。。支持8通道的话，8根插到对应的通道上就好。。他也是8根。完全看不出来为什么

[cccrrryyy]

@青山 我记得mpi官网还是哪里可以找到非常简单的并行测试程序，类似于输出很多“Hello World”这种。你这个问题感觉不仅仅是openfoam的，可能任何一个并行的程序你调用都会出现无法并行的情况？

[青山]

@cccrrryyy 并行是没问题的，就是速度太慢，跟没并行差的不多

[青山]

@星星星星晴 两个cpu槽插了一个cup，然后在cpu旁边的8个内存插槽都插了，搞不懂啥问题

[李东岳]

@青山 之前有人从别人家买服务器发生过类似问题，但都在外地。如果在北京的话，可以让我学生看看你那个机器，我也好奇怎么回事。

[青山]

@李东岳 感谢东岳老师，机子在杭州。超微的主板，买来都是配好的，一直没测试，最近测了几个案例才发现问题。标准算例里面8核的并行速度比16核还快我是真想不通

[cccrrryyy]

@青山 同在杭州，看来杭州现在做CFD多起来了啊

[李东岳]

@青山 你这个现在只能是重新做系统试一下。虽然我感觉基本不会有改变。感觉不是软件的问题。很奇怪怎么回事。

另外就是你们可以把机器发北京过来，现在学生都放假了，只能我在春节期间在硬件层面给你们debug一下。但是需要你们老师出一定的费用。如果要这么搞的话，可以邮件联系我。

[青山]

@李东岳 感谢东岳老师，打算先找售后过来看看，如果他们说硬件没问题，换系统也没有改善的话，年后可能会麻烦东岳老师帮忙 先提前向东岳老师和CFD中文网的朋友们问声新年好！

[李东岳]

@青山 他们做服务器，你跑CFD算例跟他们说对比结果，估计他们彻底懵掉了，哈哈。

谢谢，新年好，剩下10天过年了