The situation: We are building a cluster that implements an iptables firewall on every node. The firewall blocks all but a small number of specifid ports in the well-known ports range (1-1023) on the INPUT chain of the real IP address.

To handle failover for a vendor's daemon, we allocate a virtual IP address, and correspond through that. That daemon opens a random output port in the range from 512 to 1023.

When the firewall is down, or when we assign the daemon to use the ethernet port's "real" IP address, the vendor's daemon works fine.

However, when the firewall is up and the daemon is using the virtual IP address, the connection is prevented. It appears that traffic outbound on the virtual IP address is winding up on the INPUT chain for the real IP address?

In the following except from tcpdump,
the daemon is running on .46,
the daemon is using virtual IP .20
the client is running on .48
"tcpdump -ln -i eth1" (in promiscuous mode) is running on .48
----- snip ---------
11:16:58.703317 172.20.0.48.1023 > 172.20.0.20.6879: udp 16 (DF)
11:16:58.703489 172.20.0.46.6879 > 172.20.0.48.1023: udp 28 (DF)
11:16:58.703523 172.20.0.48 > 172.20.0.46: icmp: host 172.20.0.48 unreachable - admin prohibited [tos 0xc0]
11:17:03.703086 arp who-has 172.20.0.20 tell 172.20.0.48
11:17:03.703152 arp reply 172.20.0.20 is-at 0:30:6e:4a:82:b8
11:17:05.812507 172.20.0.48.35484 > 172.20.0.47.5666: S 1603799250:1603799250(0) win 5840 <mss 1460,sackOK,timestamp 8204997 0,nop,wscale 0> (DF)
11:17:05.812605 172.20.0.47.5666 > 172.20.0.48.35484: S 1615423288:1615423288(0) ack 1603799251 win 5792 <mss 1460,sackOK,timestamp 8191574 8204997,nop,wscale 0> (DF)
----- snip ---------

In any case, how can we allow unfettered outbound access on the virtual IP address while blocking unwanted inputs on the real IP address?

Just realized that "Linux -- network" took me to "wireless"; have moved the query to "security."

Sorry about that.