[SDBUG] Anyone have a rackmount server laying around?

Sun Mar 11 22:35:17 PDT 2007

Mike,

Using, Testing, and Building Linux systems for VoIP systems, I can tell you
with a high level of confidence that your hardware interrupt rates can be
significantly reduced by a combination of the two following things:

1) New ethernet controller (specifically, switching the driver used)
2) New Northbridge / PCI controller (Motherboard)

In my experience, Intel based ethernet controllers perform poorly with very
high packet rates (10,000+ pps) due to cache optimization for very large
packets, at a low rate. The 20ms interval UDP packets just don't jive with
the Ethernet Express line, and I wouldn't reccomend them for something like
a DNS or VoIP box, but might work great under FTP / HTTP / SMTP load.

It takes about 20-50 motherboard combinations to find one that works well at
high load, and it seems to be a somewhat random mix. This is at the low end,
your typical sub $150 motherboards.

At the high end I have found HP to have an extraordinarily high success
rate, with any of the Proliant series being a great option.

As far as switching to an SMP kernel, all of the load for this kind of
application is due to the interrupt processing, changing the kernel from
XT-PIC to APIC helps tremendously in Linux, I assume there are similar
compile options for *BSD for choice of interrupt controllers.

Lastly, I found that using Celerons has a dramatic performance impact on
VoIP (very small, and frequent packet) performance. When upgrading to a P4 I
can see nearly a 4x performance increase (sometimes more), and this is with
the same clockspeed and chipset! This is likely attributable to the cache
size in the P4 allowing for faster flow of the packets through the L2 cache.
I would consider the very inexpensive upgrade to a processor with a larger
cache.

Hope all goes well with your firewall. Please let me (and the list if you
are willing) know how it goes.

Cheers!
Mike

On 3/11/07, Michael J McCafferty <mike at m5computersecurity.com> wrote:
>
> Ron,
>          Yes. I am very interested. See, I have OpenBSD with PF in
> production now as a transparent firewall, and looking to go to
> OpenBSD 4.0 with CARP and pfsync as a redundant pair of routing
> firewalls. I am concerned because
> http://www.tancsa.com/blast.html  shows that OpenBSD can't route as
> many pps as FreeBSD. I'd like to stay with OpenBSD if I can, but with
> our rate of growth, and with trying to guesstimate according the
> performance numbers on that page... I don't know.
>          Day in and day out, interrupts are currently using between
> 10% and 20% CPU on Celeron 2.8GHz on Intel 865 chipset, using a dual
> port Intel GigE server card. User and system time are nothing. My
> concern is that anomalous traffic will cause problems, as well as our
> growth rate indicates we will have more than double our current
> bandwidth with six months. I can throw more hardware at the problem,
> but it doesn't look like SMP is of any use. A P4 3.2GHz on Intel 945
> Chipset, same dual-port Intel GigE server network card is the planned
> hardware.
>
> Mike
>
>
> At 05:42 PM 3/11/2007, you wrote:
> >Well it s not on a soekris if that is what you are asking. The one
> >piece I left out was the CPU. It is a Hyper-threaded 3.0G P4. The
> >bandwidth it is handling at the moment is a DSL link but I would
> >definitely a production based installation. This is not my server. I
> >am running a practically a barebones version of pfsense on my
> >Soekris4801-60 with a hard drive.
> >
> >Have not ran an iperf test on it. If you are interested I will see
> >what I can do between 2 interfaces.
> >
> >-Ron
> >
> >P.S. The IDS is only watching the WAN link.
> >
> >On Mar 11, 2007, at 4:48 PM, Michael J McCafferty wrote:
> >
> >>Ron,
> >>         How many packets/sec or megabits/sec are you sending
> >>through this thing ? Is this just a home firewall or is it handling
> >>some production traffic ?
> >>Thanks,
> >>Mike
> >>
> >>
> >>At 02:21 PM 3/11/2007, you wrote:
> >>>Have you taken a look at pfsense (http://www.pfsense.com). I have it
> >>>running on a server with two drives running raid 1, 2 Gigs of Ram, 1
> >>>100Megbit interface, and three gigabit interfaces. It is a pretty
> >>>sweet setup Below is a few things I have it doing:
> >>>
> >>>- Captive portal on one of the interfaces (Airport network plugged
> >>>into here)
> >>>- Snort for detecting bad guys
> >>>- OpenVPN for raod warrior connections
> >>>- IPSEC for connecting networks via tunnels
> >>>- Pfflowd sending data to an internal server running nfsen for
> >>>anomaly detection
> >>>- Spamd to assist the internal mail server with fighting SPAM
> >>>- Traffic shaping to keep traffic under control and allow other app
> >>>the bandwidth when needed.
> >>>
> >>>The above are to just name a few. System has been rock solid since I
> >>>installed it.
> >>>
> >>>-Ron
> >>>
> >>>On Mar 11, 2007, at 2:22 PM, Kevin Stevens wrote:
> >>>
> >>>>Group member hooked me up with a nice DL36.  Now debating Free-
> >>>>(which I know) vs Open- (which I don't).  Probably go with Open-,
> >>>>since it's targeted at stuff like this and a good learning
> >>>>opportunity.
> >>>>
> >>>>Thanks all!
> >>>>
> >>>>KeS
> >>>>
> >>>>On Mar 9, 2007, at 21:28, Kevin Stevens wrote:
> >>>>
> >>>>>I'm tired of waiting for Juniper to provide IPv6 code for my
> >>>>>Netscreen GT, so I'm going to build a firewall/router appliance
> >>>>>(with a separate interface for my wireless).  I can get boxes off
> >>>>>of eBay for $100-$150 w/shipping, but if someone has one locally I
> >>>>>can grab this weekend, that would be great.
> >>>>>
> >>>>>Looking for a DL360 type of thing - single or dual 500-1000MHz
> >>>>>CPU, 256-512MB, 9-40GB drives (SCSI or IDE), CDROM.  USB and/or
> >>>>>gigabit would be pluses.  At least one PCI slot for additional
> >>>>>NIC.  Intent is to run FreeBSD.
> >>>>>
> >>>>>Thanks, let me know if you have a good candidate you want to get
> >>>>>rid of!
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>_______________________________________________
> >>>SDBUG mailing list
> >>>SDBUG at sdbug.org
> >>>http://lists.sdbug.org/mailman/listinfo/sdbug
> >>
> >>_______________________________________________
> >>SDBUG mailing list
> >>SDBUG at sdbug.org
> >>http://lists.sdbug.org/mailman/listinfo/sdbug
> >
> >
> >
> >_______________________________________________
> >SDBUG mailing list
> >SDBUG at sdbug.org
> >http://lists.sdbug.org/mailman/listinfo/sdbug
>
> _______________________________________________
> SDBUG mailing list
> SDBUG at sdbug.org
> http://lists.sdbug.org/mailman/listinfo/sdbug
>

-- 
Mike Joyce
mjoyce at obstinate.org