我用命令启动mpirun：
mpirun -np 2 prog
并获取下一个输出：
--------------------------------------------------------------------------
WARNING: It appears that your OpenFabrics subsystem is configured to only
allow registering part of your physical memory. This can cause MPI jobs to
run with erratic performance, hang, and/or crash.
This may be caused by your OpenFabrics vendor limiting the amount of
physical memory that can be registered. You should investigate the
relevant Linux kernel module parameters that control how much physical
memory can be registered, and increase them to allow registering all
physical memory on your machine.
See this Open MPI FAQ item for more information on these Linux kernel module
parameters:
http://www.open-mpi.org/faq/?category=openfabrics#ib-..
Local host: node107
Registerable memory: 32768 MiB
Total memory: 65459 MiB
Your MPI job will continue, but may be behave poorly and/or hang.
--------------------------------------------------------------------------
hello from 0
hello from 1
[node107:48993] 1 more process has sent help message help-mpi- btl-openib.txt / reg mem limit low
[node107:48993] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
其他安装的软件(英特尔 MPI 库(工作正常，没有任何错误，并且使用所有 64GB 内存。
对于OpenMPI，我不使用任何PBS管理器(扭矩，slurm等(，我在单节点上工作。我通过命令到达节点
ssh node107
对于命令
cat /etc/security/limits.conf
我得到下一个输出：
...
* soft rss 2000000
* soft stack 2000000
* hard stack unlimited
* soft data unlimited
* hard data unlimited
* soft memlock unlimited
* hard memlock unlimited
* soft nproc 10000
* hard nproc 10000
* soft nofile 10000
* hard nofile 10000
* hard cpu unlimited
* soft cpu unlimited
...
对于命令
cat /sys/module/mlx4_core/parameters/log_num_mtt
我得到输出：
0
命令：
cat /sys/module/mlx4_core/parameters/log_mtts_per_seg
输出：
3
命令：
getconf PAGESIZE
输出：
4096
使用此参数和公式
max_reg_mem = (2^log_num_mtt) * (2^log_mtts_per_seg) * PAGE_SIZE
max_reg_mem = 32768 字节，也不是 32GB，如何在 openmpi 警告中指定。
这是什么原因呢？openmpi可以不使用Mellanox和参数log_num_mtt，log_mtts_per_seg？如何配置 OpenFabrics 以使用所有 64GB 内存？
我通过安装最新版本的OpenMPI (2.0.2(来解决此问题。
在/etc/modprobe.d/mlx4_core.conf 中，输入以下模块参数：
选项 mlx4_core log_mtts_per_seg=5
重新加载mlx4_core模块：
RMMOD mlx4_ib;RMMOD mlx4_core;模组mlx4_ib
检查log_mtts_per_seg是否按上述配置显示：
cat/sys/module/mlx4_core/parameters/log_mtts_per_seg