1. Home
  2. OpenFOAM
  3. 单核计算没问题，mpi并行无法计算？

单核计算没问题，mpi并行无法计算？

  • C

    弄了一个算例，单核计算没有问题，拿到服务器上并行计算就不能计算了，显示错误如下：

    /*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  4.0                                   |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 4.0
Exec   : interDyMFoam -parallel
Date   : Nov 28 2016
Time   : 20:33:01
Host   : "shzx-T630"
PID    : 19786
Case   : /home/shzx/ubuntu/wdx/yanzheng
nProcs : 20
Slaves : 
19
(
"shzx-T630.19787"
"shzx-T630.19788"
"shzx-T630.19789"
"shzx-T630.19790"
"shzx-T630.19791"
"shzx-T630.19792"
"shzx-T630.19793"
"shzx-T630.19794"
"shzx-T630.19795"
"shzx-T630.19796"
"shzx-T630.19797"
"shzx-T630.19798"
"shzx-T630.19799"
"shzx-T630.19800"
"shzx-T630.19801"
"shzx-T630.19802"
"shzx-T630.19803"
"shzx-T630.19804"
"shzx-T630.19805"
)

Pstream initialized with:
    floatTransfer      : 0
    nProcsSimpleSum    : 0
    commsType          : nonBlocking
    polling iterations : 0
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Allowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0

Selecting dynamicFvMesh dynamicMotionSolverFvMesh
Selecting motion solver: displacementLaplacian
Selecting motion diffusion: inverseDistance

PIMPLE: Operating solver in PISO mode

Reading field p_rgh

Reading field U

Reading/calculating face flux field phi

Reading transportProperties

Selecting incompressible transport model Newtonian
Selecting incompressible transport model Newtonian
Selecting turbulence model type laminar

Reading g

Reading hRef
Calculating field g.h

No MRF models present

No finite volume options present

DICPCG:  Solving for pcorr, Initial residual = 0, Final residual = 0, No Iterations 0
time step continuity errors : sum local = 0, global = 0, cumulative = 0
Reading/calculating face velocity Uf

Courant Number mean: 0 max: 0

Starting time loop

Courant Number mean: 0 max: 0
Interface Courant Number mean: 0 max: 0
deltaT = 0.00119048
Time = 0.00119048

--------------------------------------------------------------------------
mpirun noticed that process rank 15 with PID 19801 on node shzx-T630 exited on signal 8 (Floating point exception).
--------------------------------------------------------------------------

    这个问题我以前也遇到过，不知道怎么解决，有的时候可以并行，有的时候就不可以。
    我怀疑是网格的问题，但是检查了几次，也重新定义了patch，还是不行。由于同学并行其他例子没有问题，因此目测mpi本身应该没问题。
    以下是我写的脚本：

    #!/bin/bash

mkdir 0

echo blockMesh meshing...
blockMesh > blockMesh.log

echo snappyHexMesh meshing...
snappyHexMesh -overwrite > snappyHexMesh.log

echo Preparing 0 folder...
rm -fr 0
cp -r 0.org 0

echo Setting the fields...
setFields > setFields.log

echo decompose the fields...
decomposePar > decomposePar.log

echo Running...
mpirun -np 20 interDyMFoam -parallel > interDyMFoam.log

echo Simulation complete.

    在Running这一步出问题，其他都正常，running这一步不并行的话没任何问题，但是算的太慢了，所以还是想要并行。。。。

    请教大神们，这个类似于“浮点数溢出”的错误到底是咋回事。。

  • C

    这是控制台里面的一堆错误报告，由于太多，我取了一部分：

    Running...
[5] #0  [7] #0  Foam::error::printStack(Foam::Ostream&)[13] #0  Foam::error::printStack(Foam::Ostream&)[16] #0  Foam::error::printStack(Foam::Ostream&)[17] #0  Foam::error::printStack(Foam::Ostream&)[18] #0  Foam::error::printStack(Foam::Ostream&)[0] #0  Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&)[8] #0  Foam::error::printStack(Foam::Ostream&)[9] #0  [12] #0  Foam::error::printStack(Foam::Ostream&)[15] #0  Foam::error::printStack(Foam::Ostream&)[1] #0  Foam::error::printStack(Foam::Ostream&)[2] #0  Foam::error::printStack(Foam::Ostream&)[3] #0  Foam::error::printStack(Foam::Ostream&)[4] #0  Foam::error::printStack(Foam::Ostream&)[6] #0  Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&)[14] #0  Foam::error::printStack(Foam::Ostream&)[19] #0  Foam::error::printStack(Foam::Ostream&) at ??:?
 at ??:?
 at ??:?
 at ??:?
[5] #1  Foam::sigFpe::sigHandler(int) at ??:?
[8] #1  Foam::sigFpe::sigHandler(int)[12] #1  Foam::sigFpe::sigHandler(int)[2] #1  Foam::sigFpe::sigHandler(int)[13] #1  Foam::sigFpe::sigHandler(int) at ??:?
[0] #1  Foam::sigFpe::sigHandler(int) at ??:?
 at ??:?
 at ??:?
[16] #1  Foam::sigFpe::sigHandler(int)[15] #1  Foam::sigFpe::sigHandler(int) at ??:?
[4] #1  Foam::sigFpe::sigHandler(int)[14] #1  Foam::sigFpe::sigHandler(int) at ??:?
 at ??:?
[19] #1  Foam::sigFpe::sigHandler(int) at ??:?
[3] #1  Foam::sigFpe::sigHandler(int)[17] #1  Foam::sigFpe::sigHandler(int) at ??:?
[7] #1  Foam::sigFpe::sigHandler(int) at ??:?
 at ??:?
 at ??:?
[1] #1  Foam::sigFpe::sigHandler(int)[9] #1  Foam::sigFpe::sigHandler(int)[6] #1  Foam::sigFpe::sigHandler(int) at ??:?
[5] #2  ? at ??:?
[13] #2  ? at ??:?
[19] #2  ? at ??:?
 at ??:?
[15] #2  ? at ??:?
[3] #2  ? at ??:?
[8] #2  ?[18] #1  Foam::sigFpe::sigHandler(int) at ??:?
[1] #2  ? in "/lib/x86_64-linux-gnu/libc.so.6"
[13] #3  Foam::tuibanwavegeneration::updateCoeffs() in "/lib/x86_64-linux-gnu/libc.so.6"
[19] #3  Foam::tuibanwavegeneration::updateCoeffs() in "/lib/x86_64-linux-gnu/libc.so.6"
[5] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
[9] #2  ? in "/lib/x86_64-linux-gnu/libc.so.6"
[15] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
[12] #2  ? at ??:?
[17] #2  ? at ??:?
[7] #2  ? in "/lib/x86_64-linux-gnu/libc.so.6"
[3] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
[4] #2  ? at ??:?
[14] #2  ? at ??:?
[0] #2  ? in "/lib/x86_64-linux-gnu/libc.so.6"
[1] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
 at ??:?
[2] #2  ?[16] #2  ? in "/lib/x86_64-linux-gnu/libc.so.6"
[9] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
 in "/lib/x86_64-linux-gnu/libc.so.6"
 at ??:?
[19] #4  Foam::displacementLaplacianFvMotionSolver::solve() in "/lib[13] #4  /x86_64-linux-gnu/libc.so.6Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
"
[17] #3  Foam::tuibanwavegeneration::updateCoeffs()[5] #4  Foam::displacementLaplacianFvMotionSolver::solve()[7] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
[15] #4  Foam::displacementLaplacianFvMotionSolver::solve() in "/lib/x86_64-linux-gnu/libc.so.6"
[8] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
[6] #2  ? at ??:?
[3] #4  Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
[1] #4  Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
[9] #4  Foam::displacementLaplacianFvMotionSolver::solve() in "/lib/x86_64-linux-gnu/libc.so.6"
[12] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
[5] #5  Foam::motionSolver::newPoints() in "/lib/x86_64-linux-gnu/libc.so.6"
 at ??:?
[0] #3  [7] #4  Foam::displacementLaplacianFvMotionSolver::solve()Foam::tuibanwavegeneration::updateCoeffs() at ??:?
[19] #5  Foam::motionSolver::newPoints() at ??:?
 at ??:?
 in "[17] #4  Foam::displacementLaplacianFvMotionSolver::solve()/lib/x86_64-linux-gnu/libc.so.6"
[15] #5  Foam::motionSolver::newPoints() at ??:?
[4] #3  Foam::tuibanwavegeneration::updateCoeffs()[13] #5  Foam::motionSolver::newPoints() in "/lib/x86_64-linux-gnu/libc.so.6"
 in "/lib/x86_64-linux-gnu/libc.so.6"
[2] #3  Foam::tuibanwavegeneration::updateCoeffs()[16] #3   at ??:?
Foam::tuibanwavegeneration::updateCoeffs()[3] #5  Foam::motionSolver::newPoints() at ??:?
[1] #5  Foam::motionSolver::newPoints() at ??:?
[9] #5  Foam::motionSolver::newPoints() in "/lib/x86_64-linux-gnu/libc.so.6"
[14] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
[7] #5  Foam::motionSolver::newPoints() at ??:?
[5] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
[19] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
[15] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
[17] #5  Foam::motionSolver::newPoints() at ??:?
[3] #6  Foam::dynamicMotionSolverFvMesh::update() in  at ??:?
"/lib/x86_64-linux-gnu/libc.so. at ??:?
[13] #6  6"
Foam::dynamicMotionSolverFvMesh::update()[6] #3  [8] #4  Foam::displacementLaplacianFvMotionSolver::solve()Foam::tuibanwavegeneration::updateCoeffs() at ??:?
[1] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
[9] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
 at ??:?
[5] #7   at ??:?
[15] #7  [12] #4  Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
[0] #4   at ??:?
[3] #7   at ??:?
[7] #6  Foam::dynamicMotionSolverFvMesh::update()Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
[19] #7   at ??:?
[2] #4   at ??:?
[17] #6  Foam::dynamicMotionSolverFvMesh::update()Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
[16] #4  Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
[13] #7   at ??:?
[4] #4  Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
[9] #7   at ??:?
[1] #7  ??? at ??:?
[14] #4  Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
[7] #7  ? at ??:?
[8] #5  Foam::motionSolver::newPoints() at ??:?
[18] #2  ? at ??:?
[6] #4  Foam::displacementLaplacianFvMotionSolver::solve()? at ??:?
[17] #7  ?? at ??:?
[12] #5  Foam::motionSolver::newPoints() at ??:?
[5] #8  __libc_start_main at ??:?
[0] #5  Foam::motionSolver::newPoints() at ??:?
[16] #5  Foam::motionSolver::newPoints() at ??:?
[15] #8   at ??:?
__libc_start_main[4] #5  Foam::motionSolver::newPoints()?? at ??:?
[3] #8  __libc_start_main at ??:?
[1] #8  __libc_start_main at ??:?
 at ??:?
[19] #8  __libc_start_main[2] #5  Foam::motionSolver::newPoints() at ??:?
[9] #8  __libc_start_main at ??:?
[13] #8  __libc_start_main at ??:?
[14] #5  Foam::motionSolver::newPoints() in "/lib/x86_64-linux-gnu/libc.so.6"
[5] #9   at  in "/lib/x86_64-linux-gnu/libc.so.6"
[18] #3  Foam::tuibanwavegeneration::updateCoeffs() in "/lib/x86_64-linux-gnu/libc.so.6"
[15] #9   in "/lib/x86_64-linux-gnu/libc.so.6"
[3] #9   at ??:?
[17] #8  __libc_start_main??:?
[6] #5  Foam::motionSolver::newPoints() in "/lib/x86_64-linux-gnu/libc.so.6"
[1] #9   in "/lib/x86_64-linux-gnu/libc.so.6"
[19] #9   at ??:?
[7] #8  __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"
[9] #9   at ??:?
[8] #6  Foam::dynamicMotionSolverFvMesh::update() in "/lib/x86_64-linux-gnu/libc.so.6"
[13] #9  ???? at ??:?
[16] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
[4] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
[12] #6  Foam::dynamicMotionSolverFvMesh::update()? in "/lib/x86_64-linux-gnu/libc.so.6"
[7] #9   in "/lib/x86_64-linux-gnu/libc.so.6"
[17] #9  ? at ??:?
[shzx-T630:19791] *** Process received signal ***
[shzx-T630:19791] Signal: Floating point exception (8)
[shzx-T630:19791] Signal code:  (-6)
[shzx-T630:19791] Failing at address: 0x3e800004d4f
[shzx-T630:19791] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x354b0)[0x7f8a6fead4b0]
[shzx-T630:19791] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f8a6fead428]
[shzx-T630:19791] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x354b0)[0x7f8a6fead4b0]
[shzx-T630:19791] [ 3] /home/shzx/OpenFOAM/OpenFOAM-4.0/platforms/linux64GccDPInt32Opt/lib/libfvMotionSolvers.so(_ZN4Foam20tuibanwavegeneration12updateCoeffsEv+0x15ae)[0x7f8a4f6ad38e]
[shzx-T630:19791] [ 4] /home/shzx/OpenFOAM/OpenFOAM-4.0/platforms/linux64GccDPInt32Opt/lib/libfvMotionSolvers.so(_ZN4Foam35displacementLaplacianFvMotionSolver5solveEv+0xd9)[0x7f8a4f5ff179]
[shzx-T630:19791] [ 5] /home/shzx/OpenFOAM/OpenFOAM-4.0/platforms/linux64GccDPInt32Opt/lib/libdynamicMesh.so(_ZN4Foam12motionSolver9newPointsEv+0x25)[0x7f8a7310dc65]
[shzx-T630:19791] [ 6] /home/shzx/OpenFOAM/OpenFOAM-4.0/platforms/linux64GccDPInt32Opt/lib/libdynamicFvMesh.so(_ZN4Foam25dynamicMotionSolverFvMesh6updateEv+0x37)[0x7f8a72b0fd47]
[shzx-T630:19791] [ 7] interDyMFoam[0x439cf2]
[shzx-T630:19791] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f8a6fe98830]
[shzx-T630:19791] [ 9] interDyMFoam[0x446139]
[shzx-T630:19791] *** End of error message ***
? at ??:?
[shzx-T630:19801] *** Process received signal ***
[shzx-T630:19801] Signal: Floating point exception (8)
[shzx-T630:19801] Signal code:  (-6)
[shzx-T630:19801] Failing at address: 0x3e800004d59
[shzx-T630:19801] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x354b0)[0x7f983082e4b0]
[shzx-T630:19801] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f983082e428]
[shzx-T630:19801] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x354b0)[0x7f983082e4b0]
[shzx-T630:19801] [ 3] /home/shzx/OpenFOAM/OpenFOAM-4.0/platforms/linux64GccDPInt32Opt/lib/libfvMotionSolvers.so(_ZN4Foam20tuibanwavegeneration12updateCoeffsEv+0x15ae)[0x7f981010038e]
[shzx-T630:19801] [ 4] /home/shzx/OpenFOAM/OpenFOAM-4.0/platforms/linux64GccDPInt32Opt/lib/libfvMotionSolvers.so(_ZN4Foam35displacementLaplacianFvMotionSolver5solveEv+0xd9
[shzx-T630:19804] [ 7] interDyMFoam[0x439cf2]
[shzx-T630:19804] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fd6e0718830]
[shzx-T630:19804] [ 9] interDyMFoam[0x446139]
[shzx-T630:19804] *** End of error message ***
Simulation complete.```
  • C

    这个是什么：Foam::tuibanwavegeneration::updateCoeffs() ？
    成功运行一个时间步后没问题？

    C 1 Reply
  • C

    @cfd-china 我自己写的边界条件里面的，单核计算没有问题，如果不是昨天停电都快算完了，结果也还可以。

Log in to reply