[Midnightbsd-cvs] src [11983] trunk: One of the data structures that holds TCP segments uses an inefficient

[root at midnightbsd.org]

Revision: 11983
          http://svnweb.midnightbsd.org/src/?rev=11983
Author:   root
Date:     2018-08-06 18:14:56 -0400 (Mon, 06 Aug 2018)
Log Message:
-----------
One of the data structures that holds TCP segments uses an inefficient
algorithm to reassemble the data. This causes the CPU time spent on
segment processing to grow linearly with the number of segments in the
reassembly queue.

As a temporary solution to this problem, these patches limit the size
of each TCP connection's reassembly queue. The value is controlled by
a sysctl (net.inet.tcp.reass.maxqueuelen), which sets the maximum
number of TCP segments that can be outstanding on a session's
reassembly queue. This value defaults to 100.

Note that setting this value too low could impact the throughput of
TCP connections which experience significant loss or
reordering. However, the higher this number is set, the more resources
can be consumed on TCP reassembly processing.

Obtained from: FreeBSD

Modified Paths:
--------------
    trunk/share/man/man4/tcp.4
    trunk/sys/netinet/tcp_reass.c

Modified: trunk/share/man/man4/tcp.4
===================================================================
--- trunk/share/man/man4/tcp.4	2018-07-30 20:47:26 UTC (rev 11982)
+++ trunk/share/man/man4/tcp.4	2018-08-06 22:14:56 UTC (rev 11983)
@@ -437,6 +437,20 @@
 Reseeding should not be necessary, and will break
 .Dv TIME_WAIT
 recycling for a few minutes.
+.It Va reass.cursegments
+The current total number of segments present in all reassembly queues.
+.It Va reass.maxsegments
+The maximum limit on the total number of segments across all reassembly
+queues.
+The limit can be adjusted as a tunable.
+.It Va reass.maxqueuelen
+The maximum number of segments allowed in each reassembly queue.
+By default, the system chooses a limit based on each TCP connection's
+receive buffer size and maximum segment size (MSS).
+The actual limit applied to a session's reassembly queue will be the lower of
+the system-calculated automatic limit and the user-specified
+.Va reass.maxqueuelen
+limit.
 .It Va rexmit_min , rexmit_slop
 Adjust the retransmit timer calculation for
 .Tn TCP .

Modified: trunk/sys/netinet/tcp_reass.c
===================================================================
--- trunk/sys/netinet/tcp_reass.c	2018-07-30 20:47:26 UTC (rev 11982)
+++ trunk/sys/netinet/tcp_reass.c	2018-08-06 22:14:56 UTC (rev 11983)
@@ -97,6 +97,11 @@
 
 static uma_zone_t tcp_reass_zone;
 
+static u_int tcp_reass_maxqueuelen = 100;
+SYSCTL_UINT(_net_inet_tcp_reass, OID_AUTO, maxqueuelen, CTLFLAG_RWTUN,
+    &tcp_reass_maxqueuelen, 0,
+    "Maximum number of TCP Segments per Reassembly Queue");
+
 /* Initialize TCP reassembly queue */
 static void
 tcp_reass_zone_change(void *tag)
@@ -185,6 +190,10 @@
 	 * socket receive buffer determines our advertised window and grows
 	 * automatically when socket buffer autotuning is enabled. Use it as the
 	 * basis for our queue limit.
+	 *
+	 * However, allow the user to specify a ceiling for the number of
+	 * segments in each queue.
+	 *
 	 * Always let the missing segment through which caused this queue.
 	 * NB: Access to the socket buffer is left intentionally unlocked as we
 	 * can tolerate stale information here.
@@ -195,7 +204,8 @@
 	 * is understood.
 	 */
 	if ((th->th_seq != tp->rcv_nxt || !TCPS_HAVEESTABLISHED(tp->t_state)) &&
-	    tp->t_segqlen >= (so->so_rcv.sb_hiwat / tp->t_maxseg) + 1) {
+	    tp->t_segqlen >= min((so->so_rcv.sb_hiwat / tp->t_maxseg) + 1,
+	    tcp_reass_maxqueuelen)) {
 		tcp_reass_overflows++;
 		TCPSTAT_INC(tcps_rcvmemdrop);
 		m_freem(m);