poor network speed outbound

[hart]

I've been getting poor network perfomance outbound since about 1100UTC (when I first noticed it). This is on both VM2 and VM10.

Downloads are maxing at about 50Kb/s and averaging less than 10Kb/s with the connection stopping for up to a minute or more. Just typing at the console is a real pain.

Anyone else noticing any problems?

--
David Hart

[limey]

Yes mate, problems here too

[matta]

It was another outbound DoS. We have plenty of bandwidth available, but the attacks that cripple us may sent out 20-50Mbit/s of bandwidth, but it is actually a high pps rate. It is our not router, but the upstreams that cannot handle the high pps it seems. Our IPS would detect and auto-block these, but i'm still working on the tuning after it caused downtime before.

[limey]

Hi Matt, I'm a noob and not scared to admit it!

Is an outgoing denial of service attack something which is orchastrated by an owner of a VPS maliciouslly, or is it something launched from a server without the owners consent. If so is there anything we can do to protect against it?

Cheers

Chris

[matta]

99% of the time it is a hacked web server and not a root compromise.

Examples include PHP <4.3.10, phpNuke, or phpBB installations that are out of date. Scripts scan the internet attempting the exploits and then auto-install a botkit into /tmp or /var/tmp (running as web server so write access is limited).

Nothing you guys can do (besides make sure yours servers are all patched!), but there are things I can do. Definitely working on it.

[joshmh]

I think I'm seeing this problem now. I'm getting about 80-100kb/s download speed, whereas i get 1mb/s download from one of my other servers. The connection also stops for a few seconds or more pretty frequently.

I'm on vm3.

-- Josh

[joshmh]

Update: I'm getting 1mb/s download speed now, which is pretty good. Is there something limiting the speed to 1mb/s? And any idea why the network was so slow earlier?

Best,
Josh

[matta]

Our upstream messed up and rate limited our line to 35mbit/s. We push a lot more than that so it hurt the performance. It was a billing confusion and has since been fixed.