单核计算没问题,mpi并行无法计算?



  • 弄了一个算例,单核计算没有问题,拿到服务器上并行计算就不能计算了,显示错误如下:

    /*---------------------------------------------------------------------------*\
    | =========                 |                                                 |
    | \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
    |  \\    /   O peration     | Version:  4.0                                   |
    |   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
    |    \\/     M anipulation  |                                                 |
    \*---------------------------------------------------------------------------*/
    Build  : 4.0
    Exec   : interDyMFoam -parallel
    Date   : Nov 28 2016
    Time   : 20:33:01
    Host   : "shzx-T630"
    PID    : 19786
    Case   : /home/shzx/ubuntu/wdx/yanzheng
    nProcs : 20
    Slaves : 
    19
    (
    "shzx-T630.19787"
    "shzx-T630.19788"
    "shzx-T630.19789"
    "shzx-T630.19790"
    "shzx-T630.19791"
    "shzx-T630.19792"
    "shzx-T630.19793"
    "shzx-T630.19794"
    "shzx-T630.19795"
    "shzx-T630.19796"
    "shzx-T630.19797"
    "shzx-T630.19798"
    "shzx-T630.19799"
    "shzx-T630.19800"
    "shzx-T630.19801"
    "shzx-T630.19802"
    "shzx-T630.19803"
    "shzx-T630.19804"
    "shzx-T630.19805"
    )
    
    Pstream initialized with:
        floatTransfer      : 0
        nProcsSimpleSum    : 0
        commsType          : nonBlocking
        polling iterations : 0
    sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
    fileModificationChecking : Monitoring run-time modified files using timeStampMaster
    allowSystemOperations : Allowing user-supplied system call operations
    
    // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
    Create time
    
    Create mesh for time = 0
    
    Selecting dynamicFvMesh dynamicMotionSolverFvMesh
    Selecting motion solver: displacementLaplacian
    Selecting motion diffusion: inverseDistance
    
    PIMPLE: Operating solver in PISO mode
    
    Reading field p_rgh
    
    Reading field U
    
    Reading/calculating face flux field phi
    
    Reading transportProperties
    
    Selecting incompressible transport model Newtonian
    Selecting incompressible transport model Newtonian
    Selecting turbulence model type laminar
    
    Reading g
    
    Reading hRef
    Calculating field g.h
    
    No MRF models present
    
    No finite volume options present
    
    DICPCG:  Solving for pcorr, Initial residual = 0, Final residual = 0, No Iterations 0
    time step continuity errors : sum local = 0, global = 0, cumulative = 0
    Reading/calculating face velocity Uf
    
    Courant Number mean: 0 max: 0
    
    Starting time loop
    
    Courant Number mean: 0 max: 0
    Interface Courant Number mean: 0 max: 0
    deltaT = 0.00119048
    Time = 0.00119048
    
    --------------------------------------------------------------------------
    mpirun noticed that process rank 15 with PID 19801 on node shzx-T630 exited on signal 8 (Floating point exception).
    --------------------------------------------------------------------------
    

    这个问题我以前也遇到过,不知道怎么解决,有的时候可以并行,有的时候就不可以。
    我怀疑是网格的问题,但是检查了几次,也重新定义了patch,还是不行。由于同学并行其他例子没有问题,因此目测mpi本身应该没问题。
    以下是我写的脚本:

    #!/bin/bash
    
    mkdir 0
    
    echo blockMesh meshing...
    blockMesh > blockMesh.log
    
    echo snappyHexMesh meshing...
    snappyHexMesh -overwrite > snappyHexMesh.log
    
    echo Preparing 0 folder...
    rm -fr 0
    cp -r 0.org 0
    
    echo Setting the fields...
    setFields > setFields.log
    
    echo decompose the fields...
    decomposePar > decomposePar.log
    
    echo Running...
    mpirun -np 20 interDyMFoam -parallel > interDyMFoam.log
    
    echo Simulation complete.
    

    在Running这一步出问题,其他都正常,running这一步不并行的话没任何问题,但是算的太慢了,所以还是想要并行。。。。

    请教大神们,这个类似于“浮点数溢出”的错误到底是咋回事。。



  • 这是控制台里面的一堆错误报告,由于太多,我取了一部分:

    Running...
    [5] #0  [7] #0  Foam::error::printStack(Foam::Ostream&)[13] #0  Foam::error::printStack(Foam::Ostream&)[16] #0  Foam::error::printStack(Foam::Ostream&)[17] #0  Foam::error::printStack(Foam::Ostream&)[18] #0  Foam::error::printStack(Foam::Ostream&)[0] #0  Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&)[8] #0  Foam::error::printStack(Foam::Ostream&)[9] #0  [12] #0  Foam::error::printStack(Foam::Ostream&)[15] #0  Foam::error::printStack(Foam::Ostream&)[1] #0  Foam::error::printStack(Foam::Ostream&)[2] #0  Foam::error::printStack(Foam::Ostream&)[3] #0  Foam::error::printStack(Foam::Ostream&)[4] #0  Foam::error::printStack(Foam::Ostream&)[6] #0  Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&)[14] #0  Foam::error::printStack(Foam::Ostream&)[19] #0  Foam::error::printStack(Foam::Ostream&) at ??:?
     at ??:?
     at ??:?
     at ??:?
    [5] #1  Foam::sigFpe::sigHandler(int) at ??:?
    [8] #1  Foam::sigFpe::sigHandler(int)[12] #1  Foam::sigFpe::sigHandler(int)[2] #1  Foam::sigFpe::sigHandler(int)[13] #1  Foam::sigFpe::sigHandler(int) at ??:?
    [0] #1  Foam::sigFpe::sigHandler(int) at ??:?
     at ??:?
     at ??:?
    [16] #1  Foam::sigFpe::sigHandler(int)[15] #1  Foam::sigFpe::sigHandler(int) at ??:?
    [4] #1  Foam::sigFpe::sigHandler(int)[14] #1  Foam::sigFpe::sigHandler(int) at ??:?
     at ??:?
    [19] #1  Foam::sigFpe::sigHandler(int) at ??:?
    [3] #1  Foam::sigFpe::sigHandler(int)[17] #1  Foam::sigFpe::sigHandler(int) at ??:?
    [7] #1  Foam::sigFpe::sigHandler(int) at ??:?
     at ??:?
     at ??:?
    [1] #1  Foam::sigFpe::sigHandler(int)[9] #1  Foam::sigFpe::sigHandler(int)[6] #1  Foam::sigFpe::sigHandler(int) at ??:?
    [5] #2  ? at ??:?
    [13] #2  ? at ??:?
    [19] #2  ? at ??:?
     at ??:?
    [15] #2  ? at ??:?
    [3] #2  ? at ??:?
    [8] #2  ?[18] #1  Foam::sigFpe::sigHandler(int) at ??:?
    [1] #2  ? in "/lib/x86_64-linux-gnu/libc.so.6"
    [13] #3  Foam::tuibanwavegeneration::updateCoeffs() in "/lib/x86_64-linux-gnu/libc.so.6"
    [19] #3  Foam::tuibanwavegeneration::updateCoeffs() in "/lib/x86_64-linux-gnu/libc.so.6"
    [5] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
    [9] #2  ? in "/lib/x86_64-linux-gnu/libc.so.6"
    [15] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
    [12] #2  ? at ??:?
    [17] #2  ? at ??:?
    [7] #2  ? in "/lib/x86_64-linux-gnu/libc.so.6"
    [3] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
    [4] #2  ? at ??:?
    [14] #2  ? at ??:?
    [0] #2  ? in "/lib/x86_64-linux-gnu/libc.so.6"
    [1] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
     at ??:?
    [2] #2  ?[16] #2  ? in "/lib/x86_64-linux-gnu/libc.so.6"
    [9] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
     in "/lib/x86_64-linux-gnu/libc.so.6"
     at ??:?
    [19] #4  Foam::displacementLaplacianFvMotionSolver::solve() in "/lib[13] #4  /x86_64-linux-gnu/libc.so.6Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
    "
    [17] #3  Foam::tuibanwavegeneration::updateCoeffs()[5] #4  Foam::displacementLaplacianFvMotionSolver::solve()[7] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
    [15] #4  Foam::displacementLaplacianFvMotionSolver::solve() in "/lib/x86_64-linux-gnu/libc.so.6"
    [8] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
    [6] #2  ? at ??:?
    [3] #4  Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
    [1] #4  Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
    [9] #4  Foam::displacementLaplacianFvMotionSolver::solve() in "/lib/x86_64-linux-gnu/libc.so.6"
    [12] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
    [5] #5  Foam::motionSolver::newPoints() in "/lib/x86_64-linux-gnu/libc.so.6"
     at ??:?
    [0] #3  [7] #4  Foam::displacementLaplacianFvMotionSolver::solve()Foam::tuibanwavegeneration::updateCoeffs() at ??:?
    [19] #5  Foam::motionSolver::newPoints() at ??:?
     at ??:?
     in "[17] #4  Foam::displacementLaplacianFvMotionSolver::solve()/lib/x86_64-linux-gnu/libc.so.6"
    [15] #5  Foam::motionSolver::newPoints() at ??:?
    [4] #3  Foam::tuibanwavegeneration::updateCoeffs()[13] #5  Foam::motionSolver::newPoints() in "/lib/x86_64-linux-gnu/libc.so.6"
     in "/lib/x86_64-linux-gnu/libc.so.6"
    [2] #3  Foam::tuibanwavegeneration::updateCoeffs()[16] #3   at ??:?
    Foam::tuibanwavegeneration::updateCoeffs()[3] #5  Foam::motionSolver::newPoints() at ??:?
    [1] #5  Foam::motionSolver::newPoints() at ??:?
    [9] #5  Foam::motionSolver::newPoints() in "/lib/x86_64-linux-gnu/libc.so.6"
    [14] #3  Foam::tuibanwavegeneration::updateCoeffs() at ??:?
    [7] #5  Foam::motionSolver::newPoints() at ??:?
    [5] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
    [19] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
    [15] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
    [17] #5  Foam::motionSolver::newPoints() at ??:?
    [3] #6  Foam::dynamicMotionSolverFvMesh::update() in  at ??:?
    "/lib/x86_64-linux-gnu/libc.so. at ??:?
    [13] #6  6"
    Foam::dynamicMotionSolverFvMesh::update()[6] #3  [8] #4  Foam::displacementLaplacianFvMotionSolver::solve()Foam::tuibanwavegeneration::updateCoeffs() at ??:?
    [1] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
    [9] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
     at ??:?
    [5] #7   at ??:?
    [15] #7  [12] #4  Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
    [0] #4   at ??:?
    [3] #7   at ??:?
    [7] #6  Foam::dynamicMotionSolverFvMesh::update()Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
    [19] #7   at ??:?
    [2] #4   at ??:?
    [17] #6  Foam::dynamicMotionSolverFvMesh::update()Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
    [16] #4  Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
    [13] #7   at ??:?
    [4] #4  Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
    [9] #7   at ??:?
    [1] #7  ??? at ??:?
    [14] #4  Foam::displacementLaplacianFvMotionSolver::solve() at ??:?
    [7] #7  ? at ??:?
    [8] #5  Foam::motionSolver::newPoints() at ??:?
    [18] #2  ? at ??:?
    [6] #4  Foam::displacementLaplacianFvMotionSolver::solve()? at ??:?
    [17] #7  ?? at ??:?
    [12] #5  Foam::motionSolver::newPoints() at ??:?
    [5] #8  __libc_start_main at ??:?
    [0] #5  Foam::motionSolver::newPoints() at ??:?
    [16] #5  Foam::motionSolver::newPoints() at ??:?
    [15] #8   at ??:?
    __libc_start_main[4] #5  Foam::motionSolver::newPoints()?? at ??:?
    [3] #8  __libc_start_main at ??:?
    [1] #8  __libc_start_main at ??:?
     at ??:?
    [19] #8  __libc_start_main[2] #5  Foam::motionSolver::newPoints() at ??:?
    [9] #8  __libc_start_main at ??:?
    [13] #8  __libc_start_main at ??:?
    [14] #5  Foam::motionSolver::newPoints() in "/lib/x86_64-linux-gnu/libc.so.6"
    [5] #9   at  in "/lib/x86_64-linux-gnu/libc.so.6"
    [18] #3  Foam::tuibanwavegeneration::updateCoeffs() in "/lib/x86_64-linux-gnu/libc.so.6"
    [15] #9   in "/lib/x86_64-linux-gnu/libc.so.6"
    [3] #9   at ??:?
    [17] #8  __libc_start_main??:?
    [6] #5  Foam::motionSolver::newPoints() in "/lib/x86_64-linux-gnu/libc.so.6"
    [1] #9   in "/lib/x86_64-linux-gnu/libc.so.6"
    [19] #9   at ??:?
    [7] #8  __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"
    [9] #9   at ??:?
    [8] #6  Foam::dynamicMotionSolverFvMesh::update() in "/lib/x86_64-linux-gnu/libc.so.6"
    [13] #9  ???? at ??:?
    [16] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
    [4] #6  Foam::dynamicMotionSolverFvMesh::update() at ??:?
    [12] #6  Foam::dynamicMotionSolverFvMesh::update()? in "/lib/x86_64-linux-gnu/libc.so.6"
    [7] #9   in "/lib/x86_64-linux-gnu/libc.so.6"
    [17] #9  ? at ??:?
    [shzx-T630:19791] *** Process received signal ***
    [shzx-T630:19791] Signal: Floating point exception (8)
    [shzx-T630:19791] Signal code:  (-6)
    [shzx-T630:19791] Failing at address: 0x3e800004d4f
    [shzx-T630:19791] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x354b0)[0x7f8a6fead4b0]
    [shzx-T630:19791] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f8a6fead428]
    [shzx-T630:19791] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x354b0)[0x7f8a6fead4b0]
    [shzx-T630:19791] [ 3] /home/shzx/OpenFOAM/OpenFOAM-4.0/platforms/linux64GccDPInt32Opt/lib/libfvMotionSolvers.so(_ZN4Foam20tuibanwavegeneration12updateCoeffsEv+0x15ae)[0x7f8a4f6ad38e]
    [shzx-T630:19791] [ 4] /home/shzx/OpenFOAM/OpenFOAM-4.0/platforms/linux64GccDPInt32Opt/lib/libfvMotionSolvers.so(_ZN4Foam35displacementLaplacianFvMotionSolver5solveEv+0xd9)[0x7f8a4f5ff179]
    [shzx-T630:19791] [ 5] /home/shzx/OpenFOAM/OpenFOAM-4.0/platforms/linux64GccDPInt32Opt/lib/libdynamicMesh.so(_ZN4Foam12motionSolver9newPointsEv+0x25)[0x7f8a7310dc65]
    [shzx-T630:19791] [ 6] /home/shzx/OpenFOAM/OpenFOAM-4.0/platforms/linux64GccDPInt32Opt/lib/libdynamicFvMesh.so(_ZN4Foam25dynamicMotionSolverFvMesh6updateEv+0x37)[0x7f8a72b0fd47]
    [shzx-T630:19791] [ 7] interDyMFoam[0x439cf2]
    [shzx-T630:19791] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f8a6fe98830]
    [shzx-T630:19791] [ 9] interDyMFoam[0x446139]
    [shzx-T630:19791] *** End of error message ***
    ? at ??:?
    [shzx-T630:19801] *** Process received signal ***
    [shzx-T630:19801] Signal: Floating point exception (8)
    [shzx-T630:19801] Signal code:  (-6)
    [shzx-T630:19801] Failing at address: 0x3e800004d59
    [shzx-T630:19801] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x354b0)[0x7f983082e4b0]
    [shzx-T630:19801] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f983082e428]
    [shzx-T630:19801] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x354b0)[0x7f983082e4b0]
    [shzx-T630:19801] [ 3] /home/shzx/OpenFOAM/OpenFOAM-4.0/platforms/linux64GccDPInt32Opt/lib/libfvMotionSolvers.so(_ZN4Foam20tuibanwavegeneration12updateCoeffsEv+0x15ae)[0x7f981010038e]
    [shzx-T630:19801] [ 4] /home/shzx/OpenFOAM/OpenFOAM-4.0/platforms/linux64GccDPInt32Opt/lib/libfvMotionSolvers.so(_ZN4Foam35displacementLaplacianFvMotionSolver5solveEv+0xd9
    [shzx-T630:19804] [ 7] interDyMFoam[0x439cf2]
    [shzx-T630:19804] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fd6e0718830]
    [shzx-T630:19804] [ 9] interDyMFoam[0x446139]
    [shzx-T630:19804] *** End of error message ***
    Simulation complete.```


  • 这个是什么:Foam::tuibanwavegeneration::updateCoeffs()
    成功运行一个时间步后没问题?



  • @cfd-china 我自己写的边界条件里面的,单核计算没有问题,如果不是昨天停电都快算完了,结果也还可以。


Log in to reply