ceph-deploy mon add node2 admin_socket: exception

Question

按照官方的快速 ceph-deploy 进行操作，在添加监视器步骤中，遇到以下问题：

[ubuntu@admin-node my-cluster]$ ceph-deploy mon add node2
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ubuntu/.cephdeploy.conf
[ceph_deploy.cli][INFO  ] Invoked (1.5.37): /usr/bin/ceph-deploy mon add node2
[ceph_deploy.cli][INFO  ] ceph-deploy options:
[ceph_deploy.cli][INFO  ]  username                      : None
[ceph_deploy.cli][INFO  ]  verbose                       : False
[ceph_deploy.cli][INFO  ]  overwrite_conf                : False
[ceph_deploy.cli][INFO  ]  subcommand                    : add
[ceph_deploy.cli][INFO  ]  quiet                         : False
[ceph_deploy.cli][INFO  ]  cd_conf                       : <ceph_deploy.conf.cephdeploy.Conf instance at 0x1b0fc20>
[ceph_deploy.cli][INFO  ]  cluster                       : ceph
[ceph_deploy.cli][INFO  ]  mon                           : ['node2']
[ceph_deploy.cli][INFO  ]  func                          : <function mon at 0x1b07320>
[ceph_deploy.cli][INFO  ]  address                       : None
[ceph_deploy.cli][INFO  ]  ceph_conf                     : None
[ceph_deploy.cli][INFO  ]  default_release               : False
[ceph_deploy.mon][INFO  ] ensuring configuration of new mon host: node2
[ceph_deploy.admin][DEBUG ] Pushing admin keys and conf to node2
[node2][DEBUG ] connection detected need for sudo
[node2][DEBUG ] connected to host: node2 
[node2][DEBUG ] detect platform information from remote host
[node2][DEBUG ] detect machine type
[node2][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.mon][DEBUG ] Adding mon to cluster ceph, host node2
[ceph_deploy.mon][DEBUG ] using mon address by resolving host: 172.24.2.232
[ceph_deploy.mon][DEBUG ] detecting platform for host node2 ...
[node2][DEBUG ] connection detected need for sudo
[node2][DEBUG ] connected to host: node2 
[node2][DEBUG ] detect platform information from remote host
[node2][DEBUG ] detect machine type
[node2][DEBUG ] find the location of an executable
[ceph_deploy.mon][INFO  ] distro info: CentOS Linux 7.3.1611 Core
[node2][DEBUG ] determining if provided host has same hostname in remote
[node2][DEBUG ] get remote short hostname
[node2][DEBUG ] adding mon to node2
[node2][DEBUG ] get remote short hostname
[node2][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[node2][DEBUG ] create the mon path if it does not exist
[node2][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-node2/done
[node2][DEBUG ] create a done file to avoid re-doing the mon deployment
[node2][DEBUG ] create the init path if it does not exist
[node2][INFO  ] Running command: sudo systemctl enable ceph.target
[node2][INFO  ] Running command: sudo systemctl enable ceph-mon@node2
[node2][INFO  ] Running command: sudo systemctl start ceph-mon@node2
[node2][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.node2.asok mon_status
[node2][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[node2][WARNIN] node2 is not defined in `mon initial members`
[node2][WARNIN] monitor node2 does not exist in monmap
[node2][WARNIN] neither `public_addr` nor `public_network` keys are defined for monitors
[node2][WARNIN] monitors may not be able to form quorum
[node2][INFO  ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.node2.asok mon_status
[node2][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[node2][WARNIN] monitor: mon.node2, might not be running yet

Ceph.conf：

[global]
fsid = 5ec213d4-ae42-44c2-81d1-d7bdbee7f36a
mon_initial_members = node1
mon_host = 172.24.2.230
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
# by honghe
osd pool default size = 2

Ceph 状态为：

[ubuntu@admin-node my-cluster]$ ceph status
    cluster 5ec213d4-ae42-44c2-81d1-d7bdbee7f36a
    health HEALTH_OK
    monmap e1: 1 mons at {node1=172.24.2.230:6789/0}
            election epoch 3, quorum 0 node1
    osdmap e25: 3 osds: 3 up, 3 in
            flags sortbitwise,require_jewel_osds
    pgmap v12180: 112 pgs, 7 pools, 1588 bytes data, 171 objects
            19243 MB used, 90095 MB / 106 GB avail
                112 active+clean
[ubuntu@admin-node my-cluster]$ ceph -v
ceph version 10.2.7 (50e863e0f4bc8f4b9e31156de690d765af245185)

任何帮助将不胜感激！

Answer 1

已解决。
我想念纽托克的配置。原因节点 X 有 2 个 IF：

[ubuntu@node1 ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
    valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:a8:33:13 brd ff:ff:ff:ff:ff:ff
    inet 172.24.2.230/24 brd 172.24.2.255 scope global eth0
    valid_lft forever preferred_lft forever
    inet6 fe80::9aae:a37b:289d:1745/64 scope link 
    valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:7f:98:49 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.210/24 brd 192.168.122.255 scope global dynamic eth1
    valid_lft 2754sec preferred_lft 2754sec
    inet6 fe80::19b4:5768:7469:5767/64 scope link 
    valid_lft forever preferred_lft forever

所以我们必须配置网络

如果您有多个网络接口，请在 Ceph 配置文件的 [global] 部分下添加公有网络设置。有关详细信息，请参阅网络配置参考。

如下：

[global]
fsid = 5ec213d4-ae42-44c2-81d1-d7bdbee7f36a
mon_initial_members = node1
mon_host = 172.24.2.230
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
# by honghe
osd pool default size = 2
public network = 172.24.2.0/24

此外，添加cluster network的配置以使用另一个IF可以获得更好的性能。