Skip to content

Commit 8c64d56

Browse files
Amaindexgvisor-bot
authored andcommitted
netstack: allow defaultHandler respond ICMPv4Echo in promiscuous mode
## Motivation The gVisor network stack is extensively employed in user-space tunneling software, often operating in promiscuous mode. In this configuration, the stack directly responds to all ICMPv4 Echo Request packets, irrespective of whether a [transport defaultHandler](https://github.com/google/gvisor/blob/6b2bcc44e061f48c6dd9cc6048ae17d389a2f22d/pkg/tcpip/stack/stack.go#L49) has already processed them. This behavior is often unintended in certain scenarios, as evidenced by issues such as: - #8657 - containers/gvisor-tap-vsock#428 - xjasonlyu/tun2socks#361 - etc. In these scenarios, users may prefer to utilize [SetTransportProtocolHandler](https://github.com/google/gvisor/blob/6b2bcc44e061f48c6dd9cc6048ae17d389a2f22d/pkg/tcpip/stack/stack.go#L517) to configure a custom `defaultHandler` for tailored processing of ICMPv4 Echo packets. In issue #8657, @kevinGC proposed a potential solution: ```go // If a customer ICMPv4 protocol handler has been set, use that in favor of default handling. if p == header.ICMPv4ProtocolNumber { if _, ok := e.protocol.stack.transportProtocols[header.ICMPv4ProtocolNumber]; !ok { // handle the "normal" way } } ``` However, this approach appears somewhat aggressive, as it could impair applications that depend on the existing gVisor stack behavior with ICMPv4 protocol handlers, such as [runsc itself](https://github.com/google/gvisor/blob/6b2bcc44e061f48c6dd9cc6048ae17d389a2f22d/runsc/boot/loader.go#L1536). These programs would require code adjustments to accommodate this change, as shown below: ```go // gvisor/runsc/boot/loader.go func newEmptySandboxNetworkStack(clock tcpip.Clock, allowPacketEndpointWrite bool) (*netstack.Stack, error) { netProtos := []stack.NetworkProtocolFactory{ipv4.NewProtocol, ipv6.NewProtocol, arp.NewProtocol} transProtos := []stack.TransportProtocolFactory{ tcp.NewProtocol, udp.NewProtocol, // icmp.NewProtocol4, runsc would need to remove this to allow stack ICMPv4Echo replies. icmp.NewProtocol6, } // ... } ``` This patch introduces an alternative by enabling the stack to refrain from directly responding to ICMPv4 Echo packets delivered locally due to promiscuous mode, thereby allowing the defaultHandler to [handle](https://github.com/google/gvisor/blob/6b2bcc44e061f48c6dd9cc6048ae17d389a2f22d/pkg/tcpip/stack/nic.go#L873) them independently. **This proposal is presented as an initial step for discussion, and insights from experts on potential refinements or superior alternatives are warmly welcomed.** **Testing and adjustments for ICMPv6 will be addressed once the approach is finalized.** ## Patch Details In [ipv4.go:handleValidatedPacket](https://github.com/google/gvisor/blob/6b2bcc44e061f48c6dd9cc6048ae17d389a2f22d/pkg/tcpip/network/ipv4/ipv4.go#L1108), the packet is evaluated based on `AcquireAssignedAddress` to determine local delivery or forwarding: ```go func (e *endpoint) handleValidatedPacket(h header.IPv4, pkt *stack.PacketBuffer, inNICName string) { // ... if addressEndpoint := e.AcquireAssignedAddress(dstAddr, e.nic.Promiscuous(), stack.CanBePrimaryEndpoint, true /* readOnly */); addressEndpoint != nil { subnet := addressEndpoint.AddressWithPrefix().Subnet() pkt.NetworkPacketInfo.LocalAddressBroadcast = subnet.IsBroadcast(dstAddr) || dstAddr == header.IPv4Broadcast e.deliverPacketLocally(h, pkt, inNICName) } else if e.Forwarding() { e.handleForwardingError(e.forwardUnicastPacket(pkt)) } else { stats.ip.InvalidDestinationAddressesReceived.Increment() } } ``` In promiscuous mode, packets destined for unknown addresses are assigned a temporary address and delivered locally: ```go /* handleValidatedPacket \----AcquireAssignedAddress \----AcquireAssignedAddressOrMatching \----addAndAcquireAddressLocked */ func (a *AddressableEndpointState) AcquireAssignedAddressOrMatching(localAddr tcpip.Address, f func(AddressEndpoint) bool, allowTemp bool, tempPEB PrimaryEndpointBehavior, readOnly bool) AddressEndpoint { // ... if !allowTemp { // e.nic.Promiscuous() return nil } // ... ep, err := a.addAndAcquireAddressLocked(addr, AddressProperties{PEB: tempPEB}, Temporary) // ... } ``` By leveraging the `Temporary` field in [AddressProperties](https://github.com/google/gvisor/blob/6b2bcc44e061f48c6dd9cc6048ae17d389a2f22d/pkg/tcpip/stack/registration.go#L492), we can identify packets delivered locally due to promiscuous mode. A new field, `LocalAddressTemporary`, is added to `NetworkPacketInfo` to record this status: ```go // pkg/tcpip/stack/addressable_endpoint_state.go func (a *AddressableEndpointState) AcquireAssignedAddressOrMatching(localAddr tcpip.Address, f func(AddressEndpoint) bool, allowTemp bool, tempPEB PrimaryEndpointBehavior, readOnly bool) AddressEndpoint { // ... if !allowTemp { // e.nic.Promiscuous() return nil } // ... ep, err := a.addAndAcquireAddressLocked(addr, AddressProperties{PEB: tempPEB, Temporary: true}, Temporary) // set AddressProperties.Temporary // ... } // pkg/tcpip/network/ipv4/ipv4.go func (e *endpoint) handleValidatedPacket(h header.IPv4, pkt *stack.PacketBuffer, inNICName string) { // ... if addressEndpoint := e.AcquireAssignedAddress(dstAddr, e.nic.Promiscuous(), stack.CanBePrimaryEndpoint, true /* readOnly */); addressEndpoint != nil { pkt.NetworkPacketInfo.LocalAddressTemporary = addressEndpoint.Temporary() // packets delivered locally due to promiscuous mode subnet := addressEndpoint.AddressWithPrefix().Subnet() pkt.NetworkPacketInfo.LocalAddressBroadcast = subnet.IsBroadcast(dstAddr) || dstAddr == header.IPv4Broadcast e.deliverPacketLocally(h, pkt, inNICName) } else if e.Forwarding() { e.handleForwardingError(e.forwardUnicastPacket(pkt)) } else { stats.ip.InvalidDestinationAddressesReceived.Increment() } } ``` Finally, in [icmp.go:handleICMP](https://github.com/google/gvisor/blob/6b2bcc44e061f48c6dd9cc6048ae17d389a2f22d/pkg/tcpip/network/ipv4/icmp.go#L282), direct replies to such packets are skipped: ```go func (e *endpoint) handleICMP(pkt *stack.PacketBuffer) { // ... switch h.Type() { case header.ICMPv4Echo: // ... localAddressTemporary := pkt.NetworkPacketInfo.LocalAddressTemporary localAddressBroadcast := pkt.NetworkPacketInfo.LocalAddressBroadcast // It's possible that a raw socket or custom defaultHandler expects to // receive this packet. e.dispatcher.DeliverTransportPacket(header.ICMPv4ProtocolNumber, pkt) pkt = nil // Skip direct ICMP echo reply if the packet was received with a temporary // address, allowing custom handlers to take over. if localAddressTemporary { return } // ... } } ``` ## Quick Testing To validate this patch, a simple test program and procedure are provided below: ```go package main import ( "fmt" "net" "gvisor.dev/gvisor/pkg/tcpip" "gvisor.dev/gvisor/pkg/tcpip/header" "gvisor.dev/gvisor/pkg/tcpip/link/fdbased" "gvisor.dev/gvisor/pkg/tcpip/link/tun" "gvisor.dev/gvisor/pkg/tcpip/network/ipv4" "gvisor.dev/gvisor/pkg/tcpip/network/ipv6" "gvisor.dev/gvisor/pkg/tcpip/stack" "gvisor.dev/gvisor/pkg/tcpip/transport/icmp" "gvisor.dev/gvisor/pkg/tcpip/transport/tcp" "gvisor.dev/gvisor/pkg/tcpip/transport/udp" ) func main() { fd, _ := tun.Open("tun0") ep, _ := fdbased.New(&fdbased.Options{ FDs: []int{fd}, MTU: 1500, EthernetHeader: false, PacketDispatchMode: fdbased.Readv, MaxSyscallHeaderBytes: 0x00, }) s := stack.New(stack.Options{ NetworkProtocols: []stack.NetworkProtocolFactory{ ipv4.NewProtocol, ipv6.NewProtocol, }, TransportProtocols: []stack.TransportProtocolFactory{ tcp.NewProtocol, udp.NewProtocol, icmp.NewProtocol4, icmp.NewProtocol6, }, }) nicID := s.NextNICID() s.CreateNICWithOptions(nicID, ep, stack.NICOptions{ Disabled: false, QDisc: nil, }, ) s.SetPromiscuousMode(nicID, true) s.SetSpoofing(nicID, true) s.SetRouteTable([]tcpip.Route{ {Destination: header.IPv4EmptySubnet, NIC: nicID}, {Destination: header.IPv6EmptySubnet, NIC: nicID}, }) s.SetTransportProtocolHandler(icmp.ProtocolNumber4, func(id stack.TransportEndpointID, pkt *stack.PacketBuffer) bool { h := header.ICMPv4(pkt.TransportHeader().Slice()) fmt.Printf("icmpv4: %s->%s, hType: %v, ", id.RemoteAddress, id.LocalAddress, h.Type()) if !pkt.NetworkPacketInfo.LocalAddressTemporary { fmt.Println("packet to permanent address, processed by stack") return true } fmt.Println("packet to temporary address, need to process by user") return true }) protocolAddr := tcpip.ProtocolAddress{ Protocol: ipv4.ProtocolNumber, AddressWithPrefix: tcpip.AddressWithPrefix{ Address: tcpip.AddrFromSlice(net.IPv4(11, 0, 0, 1).To4()), PrefixLen: 8, }, } s.AddProtocolAddress(nicID, protocolAddr, stack.AddressProperties{PEB: stack.CanBePrimaryEndpoint, Temporary: false}) fmt.Println("stack started ...") select {} } ``` Test procedure: ```shell $ ip tuntap add mode tun dev tun0 ip link set dev tun0 up $ go run main.go stack started ... icmpv4: 10.161.22.19->11.0.0.1, hType: 8, packet to permanent address, processed by stack icmpv4: 10.161.22.19->11.0.0.1, hType: 8, packet to permanent address, processed by stack icmpv4: 10.161.22.19->11.0.0.1, hType: 8, packet to permanent address, processed by stack icmpv4: 10.161.22.19->11.0.0.2, hType: 8, packet to temporary address, need to process by user icmpv4: 10.161.22.19->11.0.0.2, hType: 8, packet to temporary address, need to process by user icmpv4: 10.161.22.19->11.0.0.2, hType: 8, packet to temporary address, need to process by user ``` ```shell $ ping 11.0.0.1 PING 11.0.0.1 (11.0.0.1) 56(84) bytes of data. 64 bytes from 11.0.0.1: icmp_seq=1 ttl=64 time=0.441 ms 64 bytes from 11.0.0.1: icmp_seq=2 ttl=64 time=0.425 ms 64 bytes from 11.0.0.1: icmp_seq=3 ttl=64 time=0.437 ms ^C --- 11.0.0.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2012ms rtt min/avg/max/mdev = 0.425/0.434/0.441/0.006 ms $ ping 11.0.0.2 PING 11.0.0.2 (11.0.0.2) 56(84) bytes of data. ^C --- 11.0.0.2 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2052ms ``` FUTURE_COPYBARA_INTEGRATE_REVIEW=#11609 from Amaindex:master 868dfbc PiperOrigin-RevId: 753217427
1 parent e5f3875 commit 8c64d56

File tree

4 files changed

+15
-2
lines changed

4 files changed

+15
-2
lines changed

pkg/tcpip/network/ipv4/icmp.go

+9-1
Original file line numberDiff line numberDiff line change
@@ -356,12 +356,20 @@ func (e *endpoint) handleICMP(pkt *stack.PacketBuffer) {
356356
replyData := stack.PayloadSince(pkt.TransportHeader())
357357
defer replyData.Release()
358358
ipHdr := header.IPv4(pkt.NetworkHeader().Slice())
359+
localAddressTemporary := pkt.NetworkPacketInfo.LocalAddressTemporary
359360
localAddressBroadcast := pkt.NetworkPacketInfo.LocalAddressBroadcast
360361

361-
// It's possible that a raw socket expects to receive this.
362+
// It's possible that a raw socket or custom defaultHandler expects to
363+
// receive this packet.
362364
e.dispatcher.DeliverTransportPacket(header.ICMPv4ProtocolNumber, pkt)
363365
pkt = nil
364366

367+
// Skip direct ICMP echo reply if the packet was received with a temporary
368+
// address, allowing custom handlers to take over.
369+
if localAddressTemporary {
370+
return
371+
}
372+
365373
sent := e.stats.icmp.packetsSent
366374
if !e.protocol.allowICMPReply(header.ICMPv4EchoReply, header.ICMPv4UnusedCode) {
367375
sent.rateLimited.Increment()

pkg/tcpip/network/ipv4/ipv4.go

+1
Original file line numberDiff line numberDiff line change
@@ -1169,6 +1169,7 @@ func (e *endpoint) handleValidatedPacket(h header.IPv4, pkt *stack.PacketBuffer,
11691169
// If the packet is destined for this device, then it should be delivered
11701170
// locally. Otherwise, if forwarding is enabled, it should be forwarded.
11711171
if addressEndpoint := e.AcquireAssignedAddress(dstAddr, e.nic.Promiscuous(), stack.CanBePrimaryEndpoint, true /* readOnly */); addressEndpoint != nil {
1172+
pkt.NetworkPacketInfo.LocalAddressTemporary = addressEndpoint.Temporary()
11721173
subnet := addressEndpoint.AddressWithPrefix().Subnet()
11731174
pkt.NetworkPacketInfo.LocalAddressBroadcast = subnet.IsBroadcast(dstAddr) || dstAddr == header.IPv4Broadcast
11741175
e.deliverPacketLocally(h, pkt, inNICName)

pkg/tcpip/stack/addressable_endpoint_state.go

+1-1
Original file line numberDiff line numberDiff line change
@@ -602,7 +602,7 @@ func (a *AddressableEndpointState) AcquireAssignedAddressOrMatching(localAddr tc
602602

603603
// Proceed to add a new temporary endpoint.
604604
addr := localAddr.WithPrefix()
605-
ep, err := a.addAndAcquireAddressLocked(addr, AddressProperties{PEB: tempPEB}, Temporary)
605+
ep, err := a.addAndAcquireAddressLocked(addr, AddressProperties{PEB: tempPEB, Temporary: true}, Temporary)
606606
if err != nil {
607607
// addAndAcquireAddressLocked only returns an error if the address is
608608
// already assigned but we just checked above if the address exists so we

pkg/tcpip/stack/registration.go

+4
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,10 @@ type NetworkPacketInfo struct {
5858
// address.
5959
LocalAddressBroadcast bool
6060

61+
// LocalAddressTemporary is true if the packet's local address is a temporary
62+
// address.
63+
LocalAddressTemporary bool
64+
6165
// IsForwardedPacket is true if the packet is being forwarded.
6266
IsForwardedPacket bool
6367
}

0 commit comments

Comments
 (0)