#include <deal.II/base/process_grid.h>

Detailed Description

A class taking care of setting up a two-dimensional processor grid. For example an MPI communicator with 5 processes can be arranged into a 2x2 grid with the 5-th processor being inactive:

     |   0     |   1
-----| ------- |-----
0    |   P0    |  P1
     |         |
-----| ------- |-----
1    |   P2    |  P3

A shared pointer to this class is provided to ScaLAPACKMatrix matrices to perform block-cyclic distribution.

Note that this class allows to set up a process grid which has fewer MPI cores than the total number of cores in the communicator.

Currently the only place where one would use a ProcessGrid object is in connection with a ScaLAPACKMatrix object.

Definition at line 60 of file process_grid.h.

Public Member Functions
	ProcessGrid (const MPI_Comm mpi_communicator, const unsigned int n_rows, const unsigned int n_columns)
	ProcessGrid (const MPI_Comm mpi_communicator, const unsigned int n_rows_matrix, const unsigned int n_columns_matrix, const unsigned int row_block_size, const unsigned int column_block_size)
	~ProcessGrid ()
unsigned int	get_process_grid_rows () const
unsigned int	get_process_grid_columns () const
int	get_this_process_row () const
int	get_this_process_column () const
template<typename NumberType>
void	send_to_inactive (NumberType *value, const int count=1) const
bool	is_process_active () const

Private Member Functions
	ProcessGrid (const MPI_Comm mpi_communicator, const std::pair< unsigned int, unsigned int > &grid_dimensions)

Private Attributes
MPI_Comm	mpi_communicator
MPI_Comm	mpi_communicator_inactive_with_root
int	blacs_context
const unsigned int	this_mpi_process
const unsigned int	n_mpi_processes
int	n_process_rows
int	n_process_columns
int	this_process_row
int	this_process_column
bool	mpi_process_is_active

Friends
template<typename NumberType>
class	::ScaLAPACKMatrix

Constructor & Destructor Documentation

◆ ProcessGrid() [1/3]

Utilities::MPI::ProcessGrid::ProcessGrid	(	const MPI_Comm	mpi_communicator,
		const unsigned int	n_rows,
		const unsigned int	n_columns )

Constructor for a process grid with n_rows and n_columns for a given mpi_communicator. The product of rows and columns should be less or equal to the total number of cores in the mpi_communicator.

Definition at line 223 of file process_grid.cc.

◆ ProcessGrid() [2/3]

Utilities::MPI::ProcessGrid::ProcessGrid	(	const MPI_Comm	mpi_communicator,
		const unsigned int	n_rows_matrix,
		const unsigned int	n_columns_matrix,
		const unsigned int	row_block_size,
		const unsigned int	column_block_size )

Constructor for a process grid for a given mpi_communicator. In this case the process grid is heuristically chosen based on the dimensions and block-cyclic distribution of a target matrix provided in n_rows_matrix, n_columns_matrix, row_block_size and column_block_size.

The maximum number of MPI cores one can utilize is \(\min\{\frac{M}{MB}\frac{N}{NB}, Np\}\), where \(M,N\) are the matrix dimension and \(MB,NB\) are the block sizes and \(Np\) is the number of processes in the mpi_communicator. This function then creates a 2d processor grid assuming the ratio between number of process row \(p\) and columns \(q\) to be equal the ratio between matrix dimensions \(M\) and \(N\).

For example, a square matrix \(640x640\) with the block size \(32\) and the mpi_communicator with 11 cores will result in the \(3x3\) process grid.

Definition at line 208 of file process_grid.cc.

◆ ~ProcessGrid()

Utilities::MPI::ProcessGrid::~ProcessGrid ( )

Destructor.

Definition at line 231 of file process_grid.cc.

◆ ProcessGrid() [3/3]

Utilities::MPI::ProcessGrid::ProcessGrid	(	const MPI_Comm	mpi_communicator,
		const std::pair< unsigned int, unsigned int > &	grid_dimensions )

private

A private constructor which takes grid dimensions as an std::pair.

Definition at line 101 of file process_grid.cc.

Member Function Documentation

◆ get_process_grid_rows()

unsigned int Utilities::MPI::ProcessGrid::get_process_grid_rows ( ) const

Return the number of rows in the processes grid.

◆ get_process_grid_columns()

unsigned int Utilities::MPI::ProcessGrid::get_process_grid_columns ( ) const

Return the number of columns in the processes grid.

◆ get_this_process_row()

int Utilities::MPI::ProcessGrid::get_this_process_row ( ) const

Return row of this process in the process grid.

It's negative for inactive processes.

◆ get_this_process_column()

int Utilities::MPI::ProcessGrid::get_this_process_column ( ) const

Return column of this process in the process grid.

It's negative for inactive processes.

◆ send_to_inactive()

template<typename NumberType>

template void Utilities::MPI::ProcessGrid::send_to_inactive< int >	(	NumberType *	value,
		const int	count = 1 ) const

Send count values stored consequently starting at value from the process with rank zero to processes which are not in the process grid.

Definition at line 244 of file process_grid.cc.

◆ is_process_active()

bool Utilities::MPI::ProcessGrid::is_process_active ( ) const

Return true if the process is active within the grid.

◆ ::ScaLAPACKMatrix

template<typename NumberType>

friend class ::ScaLAPACKMatrix

friend

Definition at line 65 of file process_grid.h.

Member Data Documentation

◆ mpi_communicator

MPI_Comm Utilities::MPI::ProcessGrid::mpi_communicator

private

An MPI communicator with all processes (active and inactive).

Definition at line 159 of file process_grid.h.

◆ mpi_communicator_inactive_with_root

MPI_Comm Utilities::MPI::ProcessGrid::mpi_communicator_inactive_with_root

private

An MPI communicator with inactive processes and the process with rank zero.

Definition at line 165 of file process_grid.h.

◆ blacs_context

int Utilities::MPI::ProcessGrid::blacs_context

private

BLACS context. This is equivalent to MPI communicators and is used by ScaLAPACK.

Definition at line 171 of file process_grid.h.

◆ this_mpi_process

const unsigned int Utilities::MPI::ProcessGrid::this_mpi_process

private

Rank of this MPI process.

Definition at line 176 of file process_grid.h.

◆ n_mpi_processes

const unsigned int Utilities::MPI::ProcessGrid::n_mpi_processes

private

Total number of MPI processes.

Definition at line 181 of file process_grid.h.

◆ n_process_rows

int Utilities::MPI::ProcessGrid::n_process_rows

private

Number of rows in the processes grid.

Definition at line 186 of file process_grid.h.

◆ n_process_columns

int Utilities::MPI::ProcessGrid::n_process_columns

private

Number of columns in the processes grid.

Definition at line 191 of file process_grid.h.

◆ this_process_row

int Utilities::MPI::ProcessGrid::this_process_row

private

Row of this process in the grid.

It's negative for in-active processes.

Definition at line 198 of file process_grid.h.

◆ this_process_column

int Utilities::MPI::ProcessGrid::this_process_column

private

Column of this process in the grid.

It's negative for in-active processes.

Definition at line 205 of file process_grid.h.

◆ mpi_process_is_active

bool Utilities::MPI::ProcessGrid::mpi_process_is_active

private

A flag which is true for processes within the 2d process grid.

Definition at line 210 of file process_grid.h.

The documentation for this class was generated from the following files:

include/deal.II/base/process_grid.h
source/base/process_grid.cc

Detailed Description

Public Member Functions

Private Member Functions

Private Attributes

Friends

Constructor & Destructor Documentation

◆ ProcessGrid() [1/3]

◆ ProcessGrid() [2/3]

◆ ~ProcessGrid()

◆ ProcessGrid() [3/3]

Member Function Documentation

◆ get_process_grid_rows()

◆ get_process_grid_columns()

◆ get_this_process_row()

◆ get_this_process_column()

◆ send_to_inactive()

◆ is_process_active()

Friends And Related Symbol Documentation

◆ ::ScaLAPACKMatrix

Member Data Documentation

◆ mpi_communicator

◆ mpi_communicator_inactive_with_root

◆ blacs_context

◆ this_mpi_process

◆ n_mpi_processes

◆ n_process_rows

◆ n_process_columns

◆ this_process_row

◆ this_process_column

◆ mpi_process_is_active