keepalived 2 ips on same subnet routing question.

thaylin · 03-11-2016, 02:43 PM

I have a mariadb cluster that I am trying to use haproxy/keepalived. They are all on an internal network, but the problem i am having is the reverse looking for mariadb always looks at the primary IP, not the virtual floating IP. I have tried everything I can think of but route are not being added to send the packets out as from the secondary IP

Code:

Any help would be apprechiated:

# Managed by Puppet
vrrp_script chk_haproxy {
  script    "killall -0 haproxy"
  interval  2
  weight
}

global_defs {
  notification_email {
    thaylin@somedomain.org
  }
  notification_email_from keepalive@10.76.10.98
  smtp_server localhost
  smtp_connect_timeout 60
  router_id LVS_DEVEL
}

vrrp_instance VI_2 {
  interface                 eno16780032
  state                     BACKUP
  virtual_router_id         1
  priority                  140
  advert_int                1
  garp_master_delay         5



  # notify scripts and alerts are optional
  #
  # filenames of scripts to run on transitions
  # can be unquoted (if just filename)
  # or quoted (if has parameters)




  authentication {
    auth_type PASS
    auth_pass 1111
  }

  track_script {
    chk_haproxy
  }

  track_interface {
    eno16780032
  }

  virtual_ipaddress {
    10.76.10.105 dev eno16780032 brd 10.76.10.255
  }


  virtual_routes {
default src 10.76.10.105 via 10.76.10.1 dev eno16780032 
#      dev eno16780032 src 10.76.10.105 to 10.76.10.101/24 via 10.76.10.1
#      dev eno16780032 src 10.76.10.105 to 10.76.10.102/24 via 10.76.10.1
#      dev eno16780032 src 10.76.10.105 to 10.76.10.103/24 via 10.76.10.1

  }

[root@lvs2 ~]$ ip route show
default via 10.76.10.1 dev eno16780032 proto static metric 100
10.76.10.0/24 dev eno16780032 proto kernel scope link src 10.76.10.98 metric 100
152.1.14.37 via 10.76.10.1 dev eno16780032 proto dhcp metric 100
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1

lazydog · 03-12-2016, 07:42 AM

Quote:

Originally Posted by thaylin

the problem i am having is the reverse looking for mariadb always looks at the primary IP

Can you explain this more, not sure I understand what you are meaning.
Show an example.

thaylin · 03-12-2016, 10:03 AM

Quote:

Originally Posted by lazydog

Can you explain this more, not sure I understand what you are meaning.
Show an example.

Sure thing.

First I am running Rhel 7.2, haproxy 1.5.14 and keepalived 1.2.13
[root@lvs2 ~]$ haproxy -v
HA-Proxy version 1.5.14 2015/07/02
Copyright 2000-2015 Willy Tarreau <willy@haproxy.org>

[root@lvs2 ~]$ keepalived -v
Keepalived v1.2.13 (06/25,2015)
[root@lvs2 ~]$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.2 (Maipo)

In this test setup I have 2 IPs 10.76.10.98(lvs2), which is the hostname for the server itself, and 10.76.10.105(mysqlmnmt) the virtual IP that keepalived handles
2: eno16780032: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:50:56:a8:17:c2 brd ff:ff:ff:ff:ff:ff
inet 10.76.10.98/24 brd 10.76.10.255 scope global dynamic eno16780032
valid_lft 448057sec preferred_lft 448057sec
inet 10.76.10.105/32 brd 10.76.10.255 scope global eno16780032
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:fea8:17c2/64 scope link
valid_lft forever preferred_lft forever

The problem that I have is that when I do an mysql -u thaylin -p -h 10.76.10.105 from another machine I get the error

ERROR 1045 (28000): Access denied for user 'thaylin'@'lvs2' (using password: YES)

It should be looking for the hostname associated with

This is not a password error, the password is correct..

I know with mysql you would normally use hostname security of the webserver making the call, but that does not work when you are using haproxy to pass through, and therefore it homes in on the virtual ip keepalived manages.

I have also attempted 2 interfaces. It is easier to route properly, but keepalived tries to add the routes before their is an IP assigned to it, and therefore the network is unreachable, in addition even if you set track_interface to the always live server interface the VRRP requests are not checked on it, but on the managed interface.

lazydog · 03-12-2016, 11:57 PM

What does the following give you;

Code:

dig lvs2

thaylin · 03-13-2016, 06:45 AM

Quote:

Originally Posted by lazydog

What does the following give you;

Code:

dig lvs2

It gives me the proper 10.76.10.98 IP address for the machine. I should so add that the cluster is on different machines than the load balancers, I need them to see the IP as the 105 IP, not the 98.

lazydog · 03-14-2016, 11:15 AM

And there in lies the problem. If you want to use host names then those host names have to resolve to the ip addresses you want.

thaylin · 03-14-2016, 11:29 AM

Quote:

Originally Posted by lazydog

And there in lies the problem. If you want to use host names then those host names have to resolve to the ip addresses you want.

Thank you, but no, that is not actually the problem. The production has separate hostnames, btw.

Here is a basic diagram of the issue..

------------------- 10.76.10.98 -> Mysql/Mariadb cluster lookup 10.76.10.98

external traffic -> 10.76.10.105

I need this to change to

--------------------10.76.10.98

external traffic -> 10.76.10.105 -> Mysql/Mariadb cluster lookup 10.76.10.105

The .98 address is jsut there for system accessibility for when the .105 address is on lvs1, 10.76.10.97, and to do the VRRP checks. I am currently looking into maybe iptables SNAT?

thaylin · 03-14-2016, 01:55 PM

thanks for your help, I fixed it using SNAT:

-A POSTROUTING -s 10.76.10.99/32 -p tcp -m multiport --dports 3306 -m comment --comment "100 snat for mysqlcl" -j SNAT --to-source 10.76.10.100
-A POSTROUTING -s 10.76.10.98/32 -p tcp -m multiport --dports 3306 -m comment --comment "100 snat for mysqlcl" -j SNAT --to-source 10.76.10.100