1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML>
<HEAD>
<TITLE> [99s-extend] Cowboy HTTPS connection memory usage
</TITLE>
<LINK REL="Index" HREF="index.html" >
<LINK REL="made" HREF="mailto:extend%40lists.ninenines.eu?Subject=Re%3A%20%5B99s-extend%5D%20Cowboy%20HTTPS%20connection%20memory%20usage&In-Reply-To=%3C50D49382.7020309%40ninenines.eu%3E">
<META NAME="robots" CONTENT="index,nofollow">
<style type="text/css">
pre {
white-space: pre-wrap; /* css-2.1, curent FF, Opera, Safari */
}
</style>
<META http-equiv="Content-Type" content="text/html; charset=us-ascii">
<LINK REL="Previous" HREF="000024.html">
<LINK REL="Next" HREF="000026.html">
</HEAD>
<BODY BGCOLOR="#ffffff">
<H1>[99s-extend] Cowboy HTTPS connection memory usage</H1>
<B>Loïc Hoguin</B>
<A HREF="mailto:extend%40lists.ninenines.eu?Subject=Re%3A%20%5B99s-extend%5D%20Cowboy%20HTTPS%20connection%20memory%20usage&In-Reply-To=%3C50D49382.7020309%40ninenines.eu%3E"
TITLE="[99s-extend] Cowboy HTTPS connection memory usage">essen at ninenines.eu
</A><BR>
<I>Fri Dec 21 17:51:14 CET 2012</I>
<P><UL>
<LI>Previous message: <A HREF="000024.html">[99s-extend] Cowboy HTTPS connection memory usage
</A></li>
<LI>Next message: <A HREF="000026.html">[99s-extend] Cowboy HTTPS connection memory usage
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#25">[ date ]</a>
<a href="thread.html#25">[ thread ]</a>
<a href="subject.html#25">[ subject ]</a>
<a href="author.html#25">[ author ]</a>
</LI>
</UL>
<HR>
<!--beginarticle-->
<PRE>Can you try enabling eprof to see where the VM spends its time?
On 12/21/2012 05:49 PM, rambocoder wrote:
><i> In my preliminary testing, I used Jmeter this morning since it's an
</I>><i> easy GUI load testing app and this is what I am seeing:
</I>><i>
</I>><i> With R15B03-01 [smp:4:4] [async-threads:4] [hipe] [kernel-poll:true],
</I>><i> when I establish 1K concurrent connections via HTTPS, each connection
</I>><i> takes up about 68K of memory.
</I>><i>
</I>><i> Unfortunately, after about 1050-1200 connections, on my test server the
</I>><i> Erlang scheduler jumps to 100% CPU utilization on all 4 schedulers,
</I>><i> while up to that point the scheduler's load was oscillating up and down.
</I>><i> Using the Observer, there is only 1 ssl_connection_sup in the ssl
</I>><i> application, having to deal with 1000+ gen_fsm workers, so that might be
</I>><i> the bottleneck. Since the ulimit on my server is 50000 I don't think I
</I>><i> am hitting any type of file handler's limit.
</I>><i>
</I>><i> Loïc and the group, am I missing some setting that is causing the
</I>><i> scheduler to go to 100% CPU and the run que in observer to be 99?
</I>><i>
</I>><i> Sincerely,
</I>><i>
</I>><i> rambocoder
</I>><i>
</I>><i>
</I>><i>
</I>><i> On Fri, Dec 21, 2012 at 6:45 AM, Loïc Hoguin <<A HREF="https://lists.ninenines.eu/listinfo/extend">essen at ninenines.eu</A>
</I>><i> <mailto:<A HREF="https://lists.ninenines.eu/listinfo/extend">essen at ninenines.eu</A>>> wrote:
</I>><i>
</I>><i> On 12/21/2012 04:34 AM, rambocoder wrote:
</I>><i>
</I>><i> Does anybody know either from benchmarks or real world data what
</I>><i> is the
</I>><i> average memory footprint of each concurrent HTTPS connection to
</I>><i> cowboy?
</I>><i>
</I>><i>
</I>><i> I don't have anything, sorry. I'm guessing it consumes a lot more
</I>><i> than TCP though.
</I>><i>
</I>><i>
</I>><i> SSL app in Erlang reuses SSL session-ids so I am not sure if the
</I>><i> Apache
</I>><i> Bench I test with reuses the session id or it does not.
</I>><i>
</I>><i>
</I>><i> I wouldn't know, but I wouldn't trust Apache Bench doing the right
</I>><i> thing. Any other benchmark tool usually works better in my experience.
</I>><i>
</I>><i>
</I>><i> BTW, what makes an erlang api "documented" vs "undocumented". For
</I>><i> example ssl:session_info/1 function here (
</I>><i> <A HREF="https://github.com/erlang/otp/__blob/maint/lib/ssl/src/ssl.__erl#L411">https://github.com/erlang/otp/__blob/maint/lib/ssl/src/ssl.__erl#L411</A>
</I>><i> <<A HREF="https://github.com/erlang/otp/blob/maint/lib/ssl/src/ssl.erl#L411">https://github.com/erlang/otp/blob/maint/lib/ssl/src/ssl.erl#L411</A>>
</I>><i> ) has
</I>><i> a spec and a short doc, but session_info is not described
</I>><i> <A HREF="http://www.erlang.org/doc/man/__ssl.html">http://www.erlang.org/doc/man/__ssl.html</A>
</I>><i> <<A HREF="http://www.erlang.org/doc/man/ssl.html">http://www.erlang.org/doc/man/ssl.html</A>> .ssl:session_info/1 is
</I>><i> a useful
</I>><i> function to be able to track if the load generator is reusing
</I>><i> the SSL
</I>><i> session_id or it is generating new one, because that would make
</I>><i> all the
</I>><i> difference during measurement due to Erlang caching SSL sessions
</I>><i> by default.
</I>><i>
</I>><i>
</I>><i> The documentation is separate (they're not using edoc). It's perhaps
</I>><i> not deemed useful enough for documenting it. I wouldn't worry about
</I>><i> using it for measurements though.
</I>><i>
</I>><i> Try asking Ingela on the ML about it, perhaps they just forgot to
</I>><i> document it.
</I>><i>
</I>><i> --
</I>><i> Loďc Hoguin
</I>><i> Erlang Cowboy
</I>><i> Nine Nines
</I>><i> <A HREF="http://ninenines.eu">http://ninenines.eu</A>
</I>><i>
</I>><i>
</I>><i>
</I>><i>
</I>><i> _______________________________________________
</I>><i> Extend mailing list
</I>><i> <A HREF="https://lists.ninenines.eu/listinfo/extend">Extend at lists.ninenines.eu</A>
</I>><i> <A HREF="http://lists.ninenines.eu:81/listinfo/extend">http://lists.ninenines.eu:81/listinfo/extend</A>
</I>><i>
</I>
--
Loïc Hoguin
Erlang Cowboy
Nine Nines
<A HREF="http://ninenines.eu">http://ninenines.eu</A>
</PRE>
<!--endarticle-->
<HR>
<P><UL>
<!--threads-->
<LI>Previous message: <A HREF="000024.html">[99s-extend] Cowboy HTTPS connection memory usage
</A></li>
<LI>Next message: <A HREF="000026.html">[99s-extend] Cowboy HTTPS connection memory usage
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#25">[ date ]</a>
<a href="thread.html#25">[ thread ]</a>
<a href="subject.html#25">[ subject ]</a>
<a href="author.html#25">[ author ]</a>
</LI>
</UL>
<hr>
<a href="https://lists.ninenines.eu/listinfo/extend">More information about the Extend
mailing list</a><br>
</body></html>
|
