Multi-Homed Hosts and ARP
Why does Linux answer to ARP on incorrect interfaces?
Consider the following host configuration:
Joshua Fredrick
+---------------------+ +--------------------+
| eth0: NOT ASSIGNED | | |
| br0: 192.168.0.31 +-----+ eth0: 192.168.0.21 |
| tap25: 192.168.80.2 +---+ | eth0: 192.168.1.21 |
| tap29: 192.168.81.2 | | +--------------------+
+---------------------+ |
|
|
|
Riker |
+---------------------+ |
| eth0: 192.168.80.25 +---+
+---------------------+
All these hosts are normal Linux hosts and have the following settings:
/proc/sys/net/ipv4/conf/eth0/rp_filter - 1
/proc/sys/net/ipv4/ip_forward - 0
Who can talk to who?
It is not surprising that Riker cannot communicate with Fredrick on any IP address, for that to work Joshua would have to forward packets to Fredrick.
What may be surprising is that Fredrick can connect to Joshua on any of the configured interface addresses (assuming routing would allow it) the same is true for Riker: Riker may connect to Joshua using any of the addresses assigned to interfaces on Joshua.
The reason for this is that the Linux IP stack uses what is referred to as the Weak Host model. Explained by "The Cable Guy" on Microsoft TechNet (see References).
With the weak host model it does not matter on which interface a packet is received as long as the packet is destined for an address that is assigned on the receiving host it will be accepted and processed by the stack in the normal way.
Implications for Security
There is a certain class of attack that can be instigated by sending a packet to a local address when connected physically to an external interface.
In the example above a service intended only for Riker (or on Riker's subnet
192.168.80.0/24) could be bound to an address within that subnet on the
assumption that since Fredrick is not on that subnet Fredrick will not be able
to access the service bound on 192.168.80.2. This assumption is incorrect
for the weak host model as it is indeed possible to connect to a service
running on 192.168.80.2 from 192.168.0.21 without forwarding pakets.
ARP
Since the above outlined behaviour is a feature and not a bug then ARP also
provides the support required. When Riker requests that someone reply with the
MAC address for 192.168.0.31 Joshua will respond with the MAC address of
the tap25 interface allowing IP communication to begin. Of course this would
not be a required step for SLIP or PPP.
ARP Tweaking
As the SO post (see References) details there are some settings that can be customised to stop this first step. It is important to note that it will not prevent the IP stack from processing incoming data but it will force the client to know the MAC address of the server (easy) and also the IP address on which the service is bound (which is also probably easy).
The filenames in /proc/sys/net/ipv4/conf/?/ are:
arp_announcearp_ignore
Further details in sysctl variables and the SO post.
Final Note on rp_filter
Read the detail of what rp_filter does carefully. Sending a packet with a
source address on an interface that is not the optimal route will fail the
filter check if the flag is set in all OR interface:
The max value from conf/{all,interface}/rp_filter is used
when doing source validation on the {interface}.
See sysctl variables in References.