<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML>
<HEAD>
<TITLE> [99s-extend] Cowboy handler linked processes
</TITLE>
<LINK REL="Index" HREF="index.html" >
<LINK REL="made" HREF="mailto:extend%40lists.ninenines.eu?Subject=Re%3A%20%5B99s-extend%5D%20Cowboy%20handler%20linked%20processes&In-Reply-To=%3C66E9A8B267A946AF8F527593BBFD26B0%40id3as.co.uk%3E">
<META NAME="robots" CONTENT="index,nofollow">
<style type="text/css">
pre {
white-space: pre-wrap; /* css-2.1, curent FF, Opera, Safari */
}
</style>
<META http-equiv="Content-Type" content="text/html; charset=us-ascii">
<LINK REL="Previous" HREF="000158.html">
<LINK REL="Next" HREF="000160.html">
</HEAD>
<BODY BGCOLOR="#ffffff">
<H1>[99s-extend] Cowboy handler linked processes</H1>
<B>Adrian Roe</B>
<A HREF="mailto:extend%40lists.ninenines.eu?Subject=Re%3A%20%5B99s-extend%5D%20Cowboy%20handler%20linked%20processes&In-Reply-To=%3C66E9A8B267A946AF8F527593BBFD26B0%40id3as.co.uk%3E"
TITLE="[99s-extend] Cowboy handler linked processes">adrian at id3as.co.uk
</A><BR>
<I>Thu Jul 18 12:37:30 CEST 2013</I>
<P><UL>
<LI>Previous message: <A HREF="000158.html">[99s-extend] Cowboy handler linked processes
</A></li>
<LI>Next message: <A HREF="000160.html">[99s-extend] Cowboy handler linked processes
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#159">[ date ]</a>
<a href="thread.html#159">[ thread ]</a>
<a href="subject.html#159">[ subject ]</a>
<a href="author.html#159">[ author ]</a>
</LI>
</UL>
<HR>
<!--beginarticle-->
<PRE>That would be perfect! Do you want me to make the change and issue a pull request?
--
Dr Adrian Roe
Director
On Thursday, 18 July 2013 at 11:36, Loïc Hoguin wrote:
><i> I don't think the problem is that the handler is reused, we don't reuse
</I>><i> them if there's an error. However we do catch errors to print them in
</I>><i> the logs, and then the process stops normally. If you link without
</I>><i> trap_exit you receive a normal exit signal which is ignored and doesn't
</I>><i> kill your process. I suppose we should throw an exit signal when we got
</I>><i> an error, after logging everything, instead of stopping normally.
</I>><i>
</I>><i> On 07/18/2013 12:31 PM, Adrian Roe wrote:
</I>><i> > My issue is the other way round. My handler crashes - and terminate
</I>><i> > gets called, but the linked process is NOT stopped (unless I stop it in
</I>><i> > terminate having stashed any processes I need to stop in the process
</I>><i> > dictionary - this is what I'm currently doing, but yuck!)
</I>><i> >
</I>><i> > . My question is whether it wouldn't be better to no re-use the handler
</I>><i> > process that has crashed and replace it so that handler's can use the
</I>><i> > canonical erlang way of stopping related processes rather than having to
</I>><i> > do it by hand.
</I>><i> >
</I>><i> > Obviously if the handler does not crash there's no need to kill the
</I>><i> > process, so the current efficiency saving works in the "normal" case/
</I>><i> >
</I>><i> > --
</I>><i> > Dr Adrian Roe
</I>><i> > Director
</I>><i> >
</I>><i> > On Thursday, 18 July 2013 at 11:20, Loïc Hoguin wrote:
</I>><i> >
</I>><i> > > I don't know what happens but there's two things I know:
</I>><i> > >
</I>><i> > > * Handlers don't trap_exit, so if the linked process crashes, they
</I>><i> > > crash too
</I>><i> > > * If the handler crashes, we close the connection and stop the
</I>><i> > > handler; if not this is a bug
</I>><i> > >
</I>><i> > > After your log message the handler should stop unless there's a bug
</I>><i> > > somewhere.
</I>><i> > >
</I>><i> > > On 07/18/2013 12:15 PM, Adrian Roe wrote:
</I>><i> > > > We have been using spawn_linked workers to handle tasks that live for
</I>><i> > > > the lifetime of a single HTTP request
</I>><i> > > >
</I>><i> > > > Although in the cowboy guide it is clear that Cowboy can use "One
</I>><i> > > > Process of Many Requests" I am surprised that this is the case even if
</I>><i> > > > the handler crashes. For example, our use case is to copy a large file
</I>><i> > > > to the server over HTTP where a worker process relays the file contents
</I>><i> > > > to long term storage. The worker process is spawn_linked from the HTTP
</I>><i> > > > handler and (for our use case) should die if the handler stops.
</I>><i> > > >
</I>><i> > > > If the client stops the upload (for example by browsing away, or losing
</I>><i> > > > connectivity) we correctly receive an error (see sample Lager trace
</I>><i> > > > below), but what we are seeing is that spawn_linked processes are NOT
</I>><i> > > > being killed.
</I>><i> > > >
</I>><i> > > > Is this intended behaviour - I accept it makes sense to reuse the
</I>><i> > > > processes but should this continue to be the case even if the previous
</I>><i> > > > use of the process crashed? If it is intended behaviour I think the
</I>><i> > > > docs should highlight this as we've been leaking processes for some time
</I>><i> > > > now, but I've always seen it as erlang's job to look after related
</I>><i> > > > process trees in the event of error. Our current workaround is to hold
</I>><i> > > > a list of linked processes in process storage and then kill them in the
</I>><i> > > > terminate handler which is ugly in the extreme!! We don't know the PIDS
</I>><i> > > > of the linked processes until it is too late to return State to Cowboy
</I>><i> > > > (i.e. we are already in our handle code)...
</I>><i> > > >
</I>><i> > > > Kind regards
</I>><i> > > >
</I>><i> > > > Adrian
</I>><i> > > >
</I>><i> > > > 16:09:32.347 [info] Trailer upload failed with reason
</I>><i> > > > {case_clause,{error,closed}}
</I>><i> > > > 16:09:32.348 [error] ** Cowboy handler upload_trailer_resource
</I>><i> > > > terminating in handle/2
</I>><i> > > > for the reason error:{case_clause,{error,closed}}
</I>><i> > > > ** Handler state was {state,undefined,0,undefined,undefined,undefined}
</I>><i> > > > ** Request was
</I>><i> > > > [{socket,#Port<0.11230>},{transport,ranch_tcp},{connection,keepalive},{pid,<0.1987.0>},{method,<<"POST">>},{version,'HTTP/1.1'},{peer,{{84,92,32,116},64136}},{host,<<"54.225.117.108">>},{host_info,undefined},{port,8000},{path,<<"/upload_trailer">>},{path_info,undef
</I>><i> > > > ined},{qs,<<"name=linux-7.4.21.zip&size=54015414">>},{qs_vals,undefined},{bindings,[]},{headers,[{<<"host">>,<<"54.225.117.108:8000">>},{<<"connection">>,<<"keep-alive">>},{<<"content-length">>,<<"54015414">>},{<<"origin">>,<<"<A HREF="http://54.225.117.108:8000">http://54.225.117.108:8000</A>">>},{<<"user-agent">>,<<"M
</I>><i> > > > ozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML,
</I>><i> > > > like Gecko) Chrome/28.0.1500.71
</I>><i> > > > Safari/537.36">>},{<<"content-type">>,<<>>},{<<"accept">>,<<"*/*">>},{<<"referer">>,<<"<A HREF="http://54.225.117.108:8000/">http://54.225.117.108:8000/</A>">>},{<<"accept-encoding">>,<<"gzip,deflate,sdch">>},{<<"acce
</I>><i> > > > pt-language">>,<<"en-US,en;q=0.8">>},{<<"cookie">>,<<"__jwpusr=cbc133d7-1b49-443c-8a13-364660cc93e5;
</I>><i> > > > id3as_manager=f4803c004d71dde3b64394f6e6f44faa54970e93">>}]},{p_headers,[{<<"connection">>,[<<"keep-alive">>]}]},{cookies,undefined},{meta,[]},{body_state,waiting},{multipart,unde
</I>><i> > > > fined},{buffer,<<>>},{resp_compress,true},{resp_state,waiting},{resp_headers,[]},{resp_body,<<>>},{onresponse,undefined}]
</I>><i> > > > ** Stacktrace:
</I>><i> > > > [{i_cowboy,stream_body,0,[{file,"src/i_cowboy.erl"},{line,76}]},{upload_trailer_resource,stream_upload_file,4,[{file,"src/endpoints/upload_trailer_resource.erl"},{line,247}]},{upload_trailer_resource,upload_file,1,[{file,"src/endpoints/upload_trailer_resource.erl"}
</I>><i> > > > ,{line,237}]},{upload_trailer_resource,head_or_post,1,[{file,"src/endpoints/upload_trailer_resource.erl"},{line,202}]},{upload_trailer_resource,sequence,2,[{file,"src/endpoints/upload_trailer_resource.erl"},{line,106}]},{upload_trailer_resource,process_request,1,[{file,"src/endpo
</I>><i> > > > ints/upload_trailer_resource.erl"},{line,212}]},{i_cowboy,do,3,[{file,"src/i_cowboy.erl"},{line,29}]},{cowboy_handler,handler_handle,4,[{file,"src/cowboy_handler.erl"},{line,119}]}]
</I>><i> > > >
</I>><i> > > >
</I>><i> > > > --
</I>><i> > > > Dr Adrian Roe
</I>><i> > > > Director
</I>><i> > > >
</I>><i> > > >
</I>><i> > > >
</I>><i> > > > _______________________________________________
</I>><i> > > > Extend mailing list
</I>><i> > > > <A HREF="https://lists.ninenines.eu/listinfo/extend">Extend at lists.ninenines.eu</A> <mailto:<A HREF="https://lists.ninenines.eu/listinfo/extend">Extend at lists.ninenines.eu</A>>
</I>><i> > > > <A HREF="http://lists.ninenines.eu:81/listinfo/extend">http://lists.ninenines.eu:81/listinfo/extend</A>
</I>><i> > > >
</I>><i> > >
</I>><i> > >
</I>><i> > >
</I>><i> > > --
</I>><i> > > Loïc Hoguin
</I>><i> > > Erlang Cowboy
</I>><i> > > Nine Nines
</I>><i> > > <A HREF="http://ninenines.eu">http://ninenines.eu</A>
</I>><i> > >
</I>><i> >
</I>><i> >
</I>><i>
</I>><i>
</I>><i>
</I>><i> --
</I>><i> Loïc Hoguin
</I>><i> Erlang Cowboy
</I>><i> Nine Nines
</I>><i> <A HREF="http://ninenines.eu">http://ninenines.eu</A>
</I>><i>
</I>><i>
</I>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <<A HREF="http://lists.ninenines.eu/archives/extend/attachments/20130718/79e075b8/attachment.html">http://lists.ninenines.eu/archives/extend/attachments/20130718/79e075b8/attachment.html</A>>
</PRE>
<!--endarticle-->
<HR>
<P><UL>
<!--threads-->
<LI>Previous message: <A HREF="000158.html">[99s-extend] Cowboy handler linked processes
</A></li>
<LI>Next message: <A HREF="000160.html">[99s-extend] Cowboy handler linked processes
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#159">[ date ]</a>
<a href="thread.html#159">[ thread ]</a>
<a href="subject.html#159">[ subject ]</a>
<a href="author.html#159">[ author ]</a>
</LI>
</UL>
<hr>
<a href="https://lists.ninenines.eu/listinfo/extend">More information about the Extend
mailing list</a><br>
</body></html>
