General discussion on installation and configuration of SOGo

Text archives Help


Re: [SOGo] Large deployments


Chronological Thread 
  • From: Jean Benoit < >
  • To:
  • Subject: Re: [SOGo] Large deployments
  • Date: Mon, 24 Feb 2014 18:52:03 +0100

On Fri, Dec 20, 2013 at 07:35:40AM -0500, Ludovic Marcotte wrote:
> There are a couple of things you should look at for under-performing
> sogod child processes:
>
> 1. make sure your IMAP server is fast. Look at Cyrus IMAP Server or
> Dovecot with indexes. You might also want to use imapproxy in
> between SOGo and your imap server
> 2. make sure your LDAP server is properly indexed - see the SOGo
> installation guide for details
> 3. make sure the number of sogod workers X the value of SxVMemLimit is
> lower than the amount of RAM you have - so you don't start swapping
> 4. tune your database server - especially if you do use MySQL

Recently, we discovered another aspect that needs to be tuned
on the web side.

We had a performance problem crippling SOGo for several days.
The parent process was waiting a long time for unresponsive children.
Lots of kernel messages filled the log saying:
TCP: Possible SYN flooding on port 20000. Sending cookies.
Strace showed a strange behaviour: a child process accepts an Nginx
connection, and then blocks polling() on the socket for over a minute.
Capture trafic on the loopback interface, we saw TCP retransmissions
between Nginx and SOGo.

With the help of the following page
http://blog.dubbelboer.com/2012/04/09/syn-cookies.html
we understood the problem and found a solution.
Nginx was hammering SOGo too fast and saturating the listen queue.
It turned out that the listen backlock was too low;
Unless specified, WOListenQueueSize defaults to 5 :

(in _prepareListeningSocket,
sope/sope-appserver/NGObjWeb/WOWatchDogApplicationMain.m
backlog = [[NSUserDefaults standardUserDefaults]
integerForKey: @"WOListenQueueSize"];
if (!backlog)
backlog = 5;
[listeningSocket listenWithBacklog: backlog];

(in sope/sope-appserver/NGObjWeb/Defaults.plist)
WOListenQueueSize = 5;

This value is insufficient in a large SOGo installation :
we had enough workers to handle the load.
We set WOListenQueueSize to 511 in ~sogo/GNUstep/Defaults/.GNUstepDefaults
We also modified the following sysctl parameters :

net.core.somaxconn = 511
net.ipv4.tcp_max_syn_backlog = 4096

I am not sure about how much we should really allow
(32 ? 128 ?). But this fixed our problem.

--
Jean BENOIT


  • Re: [SOGo] Large deployments, Jean Benoit, 02/24/2014

Archive powered by MHonArc 2.6.18.

Top of page