Error: unhealthy cluster | ETC node is not adding
 

ISSUE FACING
[root@PostgreSQL-HA1 etcd]#
[root@PostgreSQL-HA1 etcd]# ETCDCTL_API=3 etcdctl --endpoints http://192.168.2.174:2379,http://192.../192.168.2.181 member list
3466554c14b862ba, started, PostgreSQL-HA1, http://192.168.2.174:2380, http://192.168.2.174:2379
c3309c4b9c698f42, started, PostgreSQL-HA2, http://192.168.2.177:2380, http://192.168.2.177:2379

[root@PostgreSQL-HA1 etcd]# ETCDCTL_API=3 etcdctl --endpoints http://192.168.2.174:2379,http://192.../192.168.2.181 endpoint health
http://192.168.2.174:2379 is healthy: successfully committed proposal: took = 2.724403ms
http://192.168.2.177:2379 is healthy: successfully committed proposal: took = 2.72823ms
http://192.168.2.181 is unhealthy: failed to connect: dial tcp: address 192.168.2.181: missing port in address
Error: unhealthy cluster

HOST FILE FROM ALL THREE NODES
[root@PostgreSQL-HA3 etcd]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.2.174 PostgreSQL-HA1
192.168.2.177 PostgreSQL-HA2
192.168.2.181 PostgreSQL-HA3


ETCD CONFIGURATION FROM ALL THREE NODES
/etc/etcd/etcd.conf #(NODE-3)
ETCD_LISTEN_PEER_URLS="http://192.168.2.181:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.2.181:2379"
ETCD_NAME="PostgreSQL-HA3"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.2.181:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.2.181:2379"
ETCD_INITIAL_CLUSTER="PostgreSQL-HA1=http://192.168.2.174:2380,PostgreSQL-HA2=http://192.168.2.177:2380,PostgreSQL-HA3=http://192.168.2.181:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

/etc/etcd/etcd.conf #(NODE-2)
ETCD_LISTEN_PEER_URLS="http://192.168.2.177:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.2.177:2379"
ETCD_NAME="PostgreSQL-HA2"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.2.177:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.2.177:2379"
ETCD_INITIAL_CLUSTER="PostgreSQL-HA1=http://192.168.2.174:2380,PostgreSQL-HA2=http://192.168.2.177:2380,PostgreSQL-HA3=http://192.168.2.181:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

/etc/etcd/etcd.conf #(NODE-1)
ETCD_LISTEN_PEER_URLS="http://192.168.2.174:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.2.174:2379"
ETCD_NAME="PostgreSQL-HA1"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.2.174:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.2.174:2379"
ETCD_INITIAL_CLUSTER="PostgreSQL-HA1=http://192.168.2.174:2380,PostgreSQL-HA2=http://192.168.2.177:2380,PostgreSQL-HA3=http://192.168.2.181:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"


ETC SERVICE STATUS

etcd service is running on all nodes when I run systemctl status etcd

Firewall & selinux are disabled

I got a bit success. the 3rd node I was trying to add after creating the cluster. Now I am able to add the new node but unable to see the cluster as healthy

[root@PostgreSQL-HA2 ~]# ETCDCTL_API=3 etcdctl --endpoints http://192.168.2.174:2379,http://192.../192.168.2.181 member list
3466554c14b862ba, started, PostgreSQL-HA1, http://192.168.2.174:2380, http://192.168.2.174:2379
a457597fc069735a, started, PostgreSQL-HA3, http://192.168.2.181:2380, http://192.168.2.181:2379
c3309c4b9c698f42, started, PostgreSQL-HA2, http://192.168.2.177:2380, http://192.168.2.177:2379


[root@PostgreSQL-HA2 ~]# ETCDCTL_API=3 etcdctl --endpoints http://192.168.2.174:2379,http://192.../192.168.2.181 endpoint health
http://192.168.2.177:2379 is healthy: successfully committed proposal: took = 2.16678ms
http://192.168.2.174:2379 is healthy: successfully committed proposal: took = 1.887412ms
http://192.168.2.181 is unhealthy: failed to connect: dial tcp: address 192.168.2.181: missing port in address
Error: unhealthy cluster