[99s-extend] Cowboy load test

[Joe Freeman]

Hi,

I've started work on a project using Clojure, but I was wondering whether
(and secretly hoping that) Erlang would be a better fit, so I've been load
testing a few web server frameworks. I'm particularly interested in how the
server can handle a large number of concurrent WebSocket connections, and
the test I've been running is similar to Eric Moritz's [1].

I've setup a simple Cowboy 'echo' server running on an EC2 instance
(m1.medium, as in Eric's test) which could comfortably handle 10k
concurrent WebSocket requests (as in Eric's results), while echoing about
200 messages/second. The CPU usage of the VM at this point is about 99%,
but the server continues to handle up to 40k concurrent connections with a
consistent average response time (<30ms). Pushing the test beyond this
number results in a spike in response times and lots of connection timeouts.

40k connections seems pretty good, but when comparing this to the same test
against a couple of Clojure/JVM-based frameworks (specifically Aleph/Netty
and http-kit) I find I can get higher numbers of concurrent connections
with slightly better average response times (100k connections, <10ms
response time) using much less CPU (~20%). In fact, memory seems to be the
limiting factor.

So I have two questions:

1) Should I be concerned about the CPU usage in the Erlang/Cowboy test? I
have limited experience with Erlang so far, but 100% CPU feels like a bad
thing.

2) Is there any way to increase the performance of the cowboy server? Are
there any Erlang VM parameters I can change? The fact that the Clojure/JVM
tests (on the same machine) have managed to get to 100k connections
suggests that the limitation isn't being imposed by the operating system
(I've applied changes various changes to sysctl and ulimit).

(Perhaps an echo server isn't the best way to compare HTTP servers, but it
feels like a good starting point.)

Thanks for any help.

[1] https://github.com/ericmoritz/wsdemo/blob/results-v1/results.md - the
GitHub repo actually contains code for an Aleph server, but results from
this aren't included in the summary here.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ninenines.eu/archives/extend/attachments/20130915/c9a5340e/attachment.html>