<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML>
<HEAD>
<TITLE> [99s-extend] Cowboy HTTPS connection memory usage
</TITLE>
<LINK REL="Index" HREF="index.html" >
<LINK REL="made" HREF="mailto:extend%40lists.ninenines.eu?Subject=Re%3A%20%5B99s-extend%5D%20Cowboy%20HTTPS%20connection%20memory%20usage&In-Reply-To=%3C50D49382.7020309%40ninenines.eu%3E">
<META NAME="robots" CONTENT="index,nofollow">
<style type="text/css">
pre {
white-space: pre-wrap; /* css-2.1, curent FF, Opera, Safari */
}
</style>
<META http-equiv="Content-Type" content="text/html; charset=us-ascii">
<LINK REL="Previous" HREF="000024.html">
<LINK REL="Next" HREF="000026.html">
</HEAD>
<BODY BGCOLOR="#ffffff">
<H1>[99s-extend] Cowboy HTTPS connection memory usage</H1>
<B>Loïc Hoguin</B>
<A HREF="mailto:extend%40lists.ninenines.eu?Subject=Re%3A%20%5B99s-extend%5D%20Cowboy%20HTTPS%20connection%20memory%20usage&In-Reply-To=%3C50D49382.7020309%40ninenines.eu%3E"
TITLE="[99s-extend] Cowboy HTTPS connection memory usage">essen at ninenines.eu
</A><BR>
<I>Fri Dec 21 17:51:14 CET 2012</I>
<P><UL>
<LI>Previous message: <A HREF="000024.html">[99s-extend] Cowboy HTTPS connection memory usage
</A></li>
<LI>Next message: <A HREF="000026.html">[99s-extend] Cowboy HTTPS connection memory usage
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#25">[ date ]</a>
<a href="thread.html#25">[ thread ]</a>
<a href="subject.html#25">[ subject ]</a>
<a href="author.html#25">[ author ]</a>
</LI>
</UL>
<HR>
<!--beginarticle-->
<PRE>Can you try enabling eprof to see where the VM spends its time?
On 12/21/2012 05:49 PM, rambocoder wrote:
><i> In my preliminary testing, I used Jmeter this morning since it's an
</I>><i> easy GUI load testing app and this is what I am seeing:
</I>><i>
</I>><i> With R15B03-01 [smp:4:4] [async-threads:4] [hipe] [kernel-poll:true],
</I>><i> when I establish 1K concurrent connections via HTTPS, each connection
</I>><i> takes up about 68K of memory.
</I>><i>
</I>><i> Unfortunately, after about 1050-1200 connections, on my test server the
</I>><i> Erlang scheduler jumps to 100% CPU utilization on all 4 schedulers,
</I>><i> while up to that point the scheduler's load was oscillating up and down.
</I>><i> Using the Observer, there is only 1 ssl_connection_sup in the ssl
</I>><i> application, having to deal with 1000+ gen_fsm workers, so that might be
</I>><i> the bottleneck. Since the ulimit on my server is 50000 I don't think I
</I>><i> am hitting any type of file handler's limit.
</I>><i>
</I>><i> Loïc and the group, am I missing some setting that is causing the
</I>><i> scheduler to go to 100% CPU and the run que in observer to be 99?
</I>><i>
</I>><i> Sincerely,
</I>><i>
</I>><i> rambocoder
</I>><i>
</I>><i>
</I>><i>
</I>><i> On Fri, Dec 21, 2012 at 6:45 AM, Loïc Hoguin <<A HREF="https://lists.ninenines.eu/listinfo/extend">essen at ninenines.eu</A>
</I>><i> <mailto:<A HREF="https://lists.ninenines.eu/listinfo/extend">essen at ninenines.eu</A>>> wrote:
</I>><i>
</I>><i> On 12/21/2012 04:34 AM, rambocoder wrote:
</I>><i>
</I>><i> Does anybody know either from benchmarks or real world data what
</I>><i> is the
</I>><i> average memory footprint of each concurrent HTTPS connection to
</I>><i> cowboy?
</I>><i>
</I>><i>
</I>><i> I don't have anything, sorry. I'm guessing it consumes a lot more
</I>><i> than TCP though.
</I>><i>
</I>><i>
</I>><i> SSL app in Erlang reuses SSL session-ids so I am not sure if the
</I>><i> Apache
</I>><i> Bench I test with reuses the session id or it does not.
</I>><i>
</I>><i>
</I>><i> I wouldn't know, but I wouldn't trust Apache Bench doing the right
</I>><i> thing. Any other benchmark tool usually works better in my experience.
</I>><i>
</I>><i>
</I>><i> BTW, what makes an erlang api "documented" vs "undocumented". For
</I>><i> example ssl:session_info/1 function here (
</I>><i> <A HREF="https://github.com/erlang/otp/__blob/maint/lib/ssl/src/ssl.__erl#L411">https://github.com/erlang/otp/__blob/maint/lib/ssl/src/ssl.__erl#L411</A>
</I>><i> <<A HREF="https://github.com/erlang/otp/blob/maint/lib/ssl/src/ssl.erl#L411">https://github.com/erlang/otp/blob/maint/lib/ssl/src/ssl.erl#L411</A>>
</I>><i> ) has
</I>><i> a spec and a short doc, but session_info is not described
</I>><i> <A HREF="http://www.erlang.org/doc/man/__ssl.html">http://www.erlang.org/doc/man/__ssl.html</A>
</I>><i> <<A HREF="http://www.erlang.org/doc/man/ssl.html">http://www.erlang.org/doc/man/ssl.html</A>> .ssl:session_info/1 is
</I>><i> a useful
</I>><i> function to be able to track if the load generator is reusing
</I>><i> the SSL
</I>><i> session_id or it is generating new one, because that would make
</I>><i> all the
</I>><i> difference during measurement due to Erlang caching SSL sessions
</I>><i> by default.
</I>><i>
</I>><i>
</I>><i> The documentation is separate (they're not using edoc). It's perhaps
</I>><i> not deemed useful enough for documenting it. I wouldn't worry about
</I>><i> using it for measurements though.
</I>><i>
</I>><i> Try asking Ingela on the ML about it, perhaps they just forgot to
</I>><i> document it.
</I>><i>
</I>><i> --
</I>><i> Loďc Hoguin
</I>><i> Erlang Cowboy
</I>><i> Nine Nines
</I>><i> <A HREF="http://ninenines.eu">http://ninenines.eu</A>
</I>><i>
</I>><i>
</I>><i>
</I>><i>
</I>><i> _______________________________________________
</I>><i> Extend mailing list
</I>><i> <A HREF="https://lists.ninenines.eu/listinfo/extend">Extend at lists.ninenines.eu</A>
</I>><i> <A HREF="http://lists.ninenines.eu:81/listinfo/extend">http://lists.ninenines.eu:81/listinfo/extend</A>
</I>><i>
</I>
--
Loïc Hoguin
Erlang Cowboy
Nine Nines
<A HREF="http://ninenines.eu">http://ninenines.eu</A>
</PRE>
<!--endarticle-->
<HR>
<P><UL>
<!--threads-->
<LI>Previous message: <A HREF="000024.html">[99s-extend] Cowboy HTTPS connection memory usage
</A></li>
<LI>Next message: <A HREF="000026.html">[99s-extend] Cowboy HTTPS connection memory usage
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#25">[ date ]</a>
<a href="thread.html#25">[ thread ]</a>
<a href="subject.html#25">[ subject ]</a>
<a href="author.html#25">[ author ]</a>
</LI>
</UL>
<hr>
<a href="https://lists.ninenines.eu/listinfo/extend">More information about the Extend
mailing list</a><br>
</body></html>