1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML>
<HEAD>
<TITLE> [99s-extend] Cowboy load test
</TITLE>
<LINK REL="Index" HREF="index.html" >
<LINK REL="made" HREF="mailto:extend%40lists.ninenines.eu?Subject=Re%3A%20%5B99s-extend%5D%20Cowboy%20load%20test&In-Reply-To=%3C523604DE.6050600%40ninenines.eu%3E">
<META NAME="robots" CONTENT="index,nofollow">
<style type="text/css">
pre {
white-space: pre-wrap; /* css-2.1, curent FF, Opera, Safari */
}
</style>
<META http-equiv="Content-Type" content="text/html; charset=us-ascii">
<LINK REL="Previous" HREF="000228.html">
<LINK REL="Next" HREF="000230.html">
</HEAD>
<BODY BGCOLOR="#ffffff">
<H1>[99s-extend] Cowboy load test</H1>
<B>Loïc Hoguin</B>
<A HREF="mailto:extend%40lists.ninenines.eu?Subject=Re%3A%20%5B99s-extend%5D%20Cowboy%20load%20test&In-Reply-To=%3C523604DE.6050600%40ninenines.eu%3E"
TITLE="[99s-extend] Cowboy load test">essen at ninenines.eu
</A><BR>
<I>Sun Sep 15 21:05:02 CEST 2013</I>
<P><UL>
<LI>Previous message: <A HREF="000228.html">[99s-extend] Cowboy load test
</A></li>
<LI>Next message: <A HREF="000230.html">[99s-extend] how to send a message to all connections in cowboy
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#229">[ date ]</a>
<a href="thread.html#229">[ thread ]</a>
<a href="subject.html#229">[ subject ]</a>
<a href="author.html#229">[ author ]</a>
</LI>
</UL>
<HR>
<!--beginarticle-->
<PRE>This just hit me after hitting send.
Another possibility is that you are using text frames which are required
to be valid UTF-8. Cowboy checks that (as required by the RFC), and it's
fairly expensive, while most other servers don't. Try with binary frames
instead.
On 09/15/2013 09:02 PM, Loïc Hoguin wrote:
><i> This sounds like you have too many timers, or something else is using
</I>><i> the CPU. 40k connections is literally nothing. Also make sure the
</I>><i> clients are on a separate VM/machine.
</I>><i>
</I>><i> Either way 99% CPU for 10k sounds high, I've had that with 0 CPU (though
</I>><i> on real hardware).
</I>><i>
</I>><i> Someone a couple years back got above 1 million after fixing 'too many
</I>><i> timers issues' but I don't think it was on a medium instance.
</I>><i>
</I>><i> On 09/15/2013 07:01 PM, Joe Freeman wrote:
</I>>><i> Hi,
</I>>><i>
</I>>><i> I've started work on a project using Clojure, but I was wondering
</I>>><i> whether (and secretly hoping that) Erlang would be a better fit, so I've
</I>>><i> been load testing a few web server frameworks. I'm particularly
</I>>><i> interested in how the server can handle a large number of concurrent
</I>>><i> WebSocket connections, and the test I've been running is similar to Eric
</I>>><i> Moritz's [1].
</I>>><i>
</I>>><i> I've setup a simple Cowboy 'echo' server running on an EC2 instance
</I>>><i> (m1.medium, as in Eric's test) which could comfortably handle 10k
</I>>><i> concurrent WebSocket requests (as in Eric's results), while echoing
</I>>><i> about 200 messages/second. The CPU usage of the VM at this point is
</I>>><i> about 99%, but the server continues to handle up to 40k concurrent
</I>>><i> connections with a consistent average response time (<30ms). Pushing the
</I>>><i> test beyond this number results in a spike in response times and lots of
</I>>><i> connection timeouts.
</I>>><i>
</I>>><i> 40k connections seems pretty good, but when comparing this to the same
</I>>><i> test against a couple of Clojure/JVM-based frameworks (specifically
</I>>><i> Aleph/Netty and http-kit) I find I can get higher numbers of concurrent
</I>>><i> connections with slightly better average response times (100k
</I>>><i> connections, <10ms response time) using much less CPU (~20%). In fact,
</I>>><i> memory seems to be the limiting factor.
</I>>><i>
</I>>><i> So I have two questions:
</I>>><i>
</I>>><i> 1) Should I be concerned about the CPU usage in the Erlang/Cowboy test?
</I>>><i> I have limited experience with Erlang so far, but 100% CPU feels like a
</I>>><i> bad thing.
</I>>><i>
</I>>><i> 2) Is there any way to increase the performance of the cowboy server?
</I>>><i> Are there any Erlang VM parameters I can change? The fact that the
</I>>><i> Clojure/JVM tests (on the same machine) have managed to get to 100k
</I>>><i> connections suggests that the limitation isn't being imposed by the
</I>>><i> operating system (I've applied changes various changes to sysctl and
</I>>><i> ulimit).
</I>>><i>
</I>>><i> (Perhaps an echo server isn't the best way to compare HTTP servers, but
</I>>><i> it feels like a good starting point.)
</I>>><i>
</I>>><i> Thanks for any help.
</I>>><i>
</I>>><i> [1] <A HREF="https://github.com/ericmoritz/wsdemo/blob/results-v1/results.md">https://github.com/ericmoritz/wsdemo/blob/results-v1/results.md</A> -
</I>>><i> the GitHub repo actually contains code for an Aleph server, but results
</I>>><i> from this aren't included in the summary here.
</I>>><i>
</I>>><i>
</I>>><i>
</I>>><i> _______________________________________________
</I>>><i> Extend mailing list
</I>>><i> <A HREF="https://lists.ninenines.eu/listinfo/extend">Extend at lists.ninenines.eu</A>
</I>>><i> <A HREF="http://lists.ninenines.eu:81/listinfo/extend">http://lists.ninenines.eu:81/listinfo/extend</A>
</I>>><i>
</I>><i>
</I>><i>
</I>
--
Loïc Hoguin
Erlang Cowboy
Nine Nines
<A HREF="http://ninenines.eu">http://ninenines.eu</A>
</PRE>
<!--endarticle-->
<HR>
<P><UL>
<!--threads-->
<LI>Previous message: <A HREF="000228.html">[99s-extend] Cowboy load test
</A></li>
<LI>Next message: <A HREF="000230.html">[99s-extend] how to send a message to all connections in cowboy
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#229">[ date ]</a>
<a href="thread.html#229">[ thread ]</a>
<a href="subject.html#229">[ subject ]</a>
<a href="author.html#229">[ author ]</a>
</LI>
</UL>
<hr>
<a href="https://lists.ninenines.eu/listinfo/extend">More information about the Extend
mailing list</a><br>
</body></html>
|
