从多个MPI输出合成VTK文件 [英] Composing VTK file from multiple MPI outputs

问题描述

对于盖驱动腔(CFD)的莱迪思·玻尔兹曼模拟，我将立方域分解为(也是立方)8个子域，这些子域由8个等级独立计算.每个MPI等级都为每个时间步生成一个VTK文件，由于我使用的是ParaView，因此我想将整个内容可视化为一个多维数据集.更具体地说明我要实现的目标:

For a Lattice Boltzmann simulation of a lid-driven cavity (CFD) I'm decomposing my cubic domain into (also cubic) 8 subdomains, which are computed independently by 8 ranks. Each MPI rank is producing a VTK file for each timestep and since I'm using ParaView I want to visualize the whole thing as one cube. To be more specific about what I am trying to achieve:

• 我有一个长为8(每个方向上的元素数)的多维数据集=> 8x8x8 = 512个元素.
• 每个维度分配给2个等级，即每个等级处理4x4x4 = 64个元素.
• 每个等级均以VTK StructuredGrid格式将结果写入名为lbm_out_<rank>.<timestep>.vts的文件中.
• 我想生成一个.pvts文件，该文件收集*.vts文件并将包含子域的文件合并为一个文件，ParaView可以将其视为整个域.

• I have a cube with length 8 (number of elements for each direction) => 8x8x8 = 512 elements.
• Each dimension is distributed to 2 ranks, i.e. every rank handles 4x4x4 = 64 elements.
• Every rank writes it's result to a file called lbm_out_<rank>.<timestep>.vts in a VTK StructuredGrid format.
• I want to produce a .pvts file that collects the *.vts files and combines the files containing the subdomains to a single file that ParaView can treat as whole domain.

不幸的是，我遇到了很多问题，因为我觉得ParaView和VTK的文档记录非常差，并且来自ParaView的错误消息完全没有用.

Unfortunately I'm facing many issues with that since I feel ParaView and VTK are extremely poorly documented and the error messages from ParaView are totally useless.

我有以下*.pvts文件，其中包括一个幽灵层和

I'm having the following *.pvts file, which includes a ghost layer and:

<?xml version="1.0"?>
<VTKFile type="PStructuredGrid" version="0.1" byte_order="LittleEndian">
    <PStructuredGrid WholeExtent="0 7 0 7 0 7 " GhostLevel="1">
        <PPoints>
            <PDataArray NumberOfComponents="3" type="Float32" />
        </PPoints>
        <Piece Extent="0 4 0 4 0 4" Source="lbm_out_0.0.vts"/>
        <Piece Extent="3 7 0 4 0 4" Source="lbm_out_1.0.vts"/>
        <Piece Extent="0 4 3 7 0 4" Source="lbm_out_2.0.vts"/>
        <Piece Extent="3 7 3 7 0 4" Source="lbm_out_3.0.vts"/>
        <Piece Extent="0 4 0 4 3 7" Source="lbm_out_4.0.vts"/>
        <Piece Extent="3 7 0 4 3 7" Source="lbm_out_5.0.vts"/>
        <Piece Extent="0 4 3 7 3 7" Source="lbm_out_6.0.vts"/>
        <Piece Extent="3 7 3 7 3 7" Source="lbm_out_7.0.vts"/>
    </PStructuredGrid>
</VTKFile>

具有该文件，我认为它应该可以正常工作(请注意，还没有参数，只有简单的几何信息)，我的域范围完全混乱了，尽管每个*.vts文件都可以正常工作.我附带了一个屏幕截图，以使事情更清楚:

Having that file, which I feel should work correctly (note that there are not parameters yet, just plain geometry information), my domain ranges are totally messed up, although each *.vts file works fine on its own. I have a screenshot attached to make things more clear:

可能是什么问题?是否可以将旧版VTK文件用于此任务?我可以做些完全错误的事情吗?我真的不知道如何完成这项任务，而我在Google中找到的资源非常有限.谢谢.

What may be the problem? Is it possible to use legacy VTK files for this tasks? May I be doing something totally wrong? I really don't know how to accomplish this task and the resources I find in google are very limited. Thank you.

推荐答案

不幸的是，没有 vtkXMLPStructuredGridWriter 类(示例中未使用的VTK类).因此，我决定编写最简单的代码来为结构化网格生成* .vts和.pvts文件，这与您要查找的情况非常相似.

Unfortunately there is no example for vtkXMLPStructuredGridWriter class (VTK Classes not used in the Examples). So I decided to write the simplest code to generate *.vts and .pvts files for a structured grid, very similar to the case you are looking for.

以下代码使用MPI和VTK编写并行结构化的网格文件.在此示例中，我们有两个进程创建自己的.vts文件，而vtkXMLPStructuredGridWriter类编写.pvts文件:

The following code uses MPI and VTK to write parallel structured grid files. In this example, we have two processes which create their own .vts files and the vtkXMLPStructuredGridWriter class writes the .pvts file:

// MPI Library
#include <mpi.h>

//VTK Library
#include <vtkXMLPStructuredGridWriter.h>
#include <vtkStructuredGrid.h>
#include <vtkSmartPointer.h>
#include <vtkFloatArray.h>
#include <vtkCellData.h>
#include <vtkMPIController.h>
#include <vtkProgrammableFilter.h>
#include <vtkInformation.h>

struct Args {
  vtkProgrammableFilter* pf;
  int local_extent[6];
};

// function to operate on the point attribute data
void execute (void* arg) {
  Args* args = reinterpret_cast<Args*>(arg);
  auto info = args->pf->GetOutputInformation(0);
  auto output_tmp = args->pf->GetOutput();
  auto input_tmp  = args->pf->GetInput();
  vtkStructuredGrid* output = dynamic_cast<vtkStructuredGrid*>(output_tmp);
  vtkStructuredGrid* input  = dynamic_cast<vtkStructuredGrid*>(input_tmp);
  output->ShallowCopy(input);
  output->SetExtent(args->local_extent);
}

int main (int argc, char *argv[]) {
  MPI_Init (&argc, &argv);
  int myrank;
  MPI_Comm_rank (MPI_COMM_WORLD, &myrank);

  int lx {10}, ly{10}, lz{10};        //local dimensions of the process's grid
  int dims[3] = {lx+1, ly+1, lz+1};
  int global_extent[6] = {0, 2*lx, 0, ly, 0, lz};
  int local_extent[6] = {myrank*lx, (myrank+1)*lx, 0, ly, 0, lz};

  // Create and Initialize vtkMPIController
  auto contr = vtkSmartPointer<vtkMPIController>::New();
  contr->Initialize(&argc, &argv, 1);
  int nranks = contr->GetNumberOfProcesses();
  int rank   = contr->GetLocalProcessId();

  // Create grid points, allocate memory and Insert them
  auto points = vtkSmartPointer<vtkPoints>::New();
  points->Allocate(dims[0]*dims[1]*dims[2]);
  for (int k=0; k<dims[2]; ++k)
    for (int j=0; j<dims[1]; ++j)
      for (int i=0; i<dims[0]; ++i)
        points->InsertPoint(i + j*dims[0] + k*dims[0]*dims[1],
                            i+rank*(dims[0]-1), j, k);

  // Create a density field. Note that the number of cells is always less than
  // number of grid points by an amount of one so we use dims[i]-1
  auto density = vtkSmartPointer<vtkFloatArray>::New();
  density->SetNumberOfComponents(1);
  density->SetNumberOfTuples((dims[0]-1)*(dims[1]-1)*(dims[2]-1));
  density->SetName ("density");
  int index;
  for (int k=0; k<lz; ++k)
    for (int j=0; j<ly; ++j)
      for (int i=0; i<lx; ++i) {
        index = i + j*(dims[0]-1) + k*(dims[0]-1)*(dims[1]-1);
        density->SetValue(index, i+j+k);
      }

  // Create a vtkProgrammableFilter
  auto pf = vtkSmartPointer<vtkProgrammableFilter>::New();

  // Initialize an instance of Args
  Args args;
  args.pf = pf;
  for(int i=0; i<6; ++i) args.local_extent[i] = local_extent[i];

  pf->SetExecuteMethod(execute, &args);

  // Create a structured grid and assign point data and cell data to it
  auto structuredGrid = vtkSmartPointer<vtkStructuredGrid>::New();
  structuredGrid->SetExtent(global_extent);
  pf->SetInputData(structuredGrid);
  structuredGrid->SetPoints(points);
  structuredGrid->GetCellData()->AddArray(density);

  // Create the parallel writer and call some functions
  auto parallel_writer = vtkSmartPointer<vtkXMLPStructuredGridWriter>::New();
  parallel_writer->SetInputConnection(pf->GetOutputPort());
  parallel_writer->SetController(contr);
  parallel_writer->SetFileName("test.pvts");
  parallel_writer->SetNumberOfPieces(nranks);
  parallel_writer->SetStartPiece(rank);
  parallel_writer->SetEndPiece(rank);
  parallel_writer->SetDataModeToBinary();
  parallel_writer->Update();
  parallel_writer->Write();

  contr->Finalize();

  // WARNING: it seems that MPI_Finalize is not necessary since we are using
  // Finalize() function from vtkMPIController class. Uncomment the following
  // line and see what happens.
//   MPI_Finalize ();
  return 0;
}

此代码仅将一些数据(在这种情况下为标量密度)写入文件中，而不呈现它.为了进行可视化，您将需要像Paraview这样的软件. 要运行此代码，可以使用以下CMake文件:

This code only writes some data (in this case density which is a scalar) into a file and do not render it. For visualization you will need a software like Paraview. To run this code, you can use this CMake file:

cmake_minimum_required(VERSION 2.8)

PROJECT(PXMLStructuredGridWriter)
add_executable(PXMLStructuredGridWriter parallel_vtk.cpp)

find_package(VTK REQUIRED)
include(${VTK_USE_FILE})
target_link_libraries(PXMLStructuredGridWriter ${VTK_LIBRARIES})

find_package(MPI REQUIRED)
include_directories(${MPI_INCLUDE_PATH})
target_link_libraries(PXMLStructuredGridWriter ${MPI_LIBRARIES})

最后，您将在可执行文件所在的目录中找到这样的xml文件:

At the end you will find an xml file like this in the same directory that you have the executable:

<?xml version="1.0"?>
<VTKFile type="PStructuredGrid" version="0.1" byte_order="LittleEndian" header_type="UInt32" compressor="vtkZLibDataCompressor">
  <PStructuredGrid WholeExtent="0 20 0 10 0 10" GhostLevel="2">
    <PCellData>
      <PDataArray type="Float32" Name="density"/>
    </PCellData>
    <PPoints>
      <PDataArray type="Float32" Name="Points" NumberOfComponents="3"/>
    </PPoints>
    <Piece Extent="0 10 0 10 0 10" Source="test_0.vts"/>
    <Piece Extent="10 20 0 10 0 10" Source="test_1.vts"/>
  </PStructuredGrid>
</VTKFile>

这篇关于从多个MPI输出合成VTK文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！