1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML>
<HEAD>
<TITLE> [99s-extend] Cowboy handler linked processes
</TITLE>
<LINK REL="Index" HREF="index.html" >
<LINK REL="made" HREF="mailto:extend%40lists.ninenines.eu?Subject=Re%3A%20%5B99s-extend%5D%20Cowboy%20handler%20linked%20processes&In-Reply-To=%3C51E7C595.5050905%40ninenines.eu%3E">
<META NAME="robots" CONTENT="index,nofollow">
<style type="text/css">
pre {
white-space: pre-wrap; /* css-2.1, curent FF, Opera, Safari */
}
</style>
<META http-equiv="Content-Type" content="text/html; charset=us-ascii">
<LINK REL="Previous" HREF="000159.html">
<LINK REL="Next" HREF="000161.html">
</HEAD>
<BODY BGCOLOR="#ffffff">
<H1>[99s-extend] Cowboy handler linked processes</H1>
<B>Loïc Hoguin</B>
<A HREF="mailto:extend%40lists.ninenines.eu?Subject=Re%3A%20%5B99s-extend%5D%20Cowboy%20handler%20linked%20processes&In-Reply-To=%3C51E7C595.5050905%40ninenines.eu%3E"
TITLE="[99s-extend] Cowboy handler linked processes">essen at ninenines.eu
</A><BR>
<I>Thu Jul 18 12:38:13 CEST 2013</I>
<P><UL>
<LI>Previous message: <A HREF="000159.html">[99s-extend] Cowboy handler linked processes
</A></li>
<LI>Next message: <A HREF="000161.html">[99s-extend] Cowboy handler linked processes
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#160">[ date ]</a>
<a href="thread.html#160">[ thread ]</a>
<a href="subject.html#160">[ subject ]</a>
<a href="author.html#160">[ author ]</a>
</LI>
</UL>
<HR>
<!--beginarticle-->
<PRE>If you got time sure, I won't have much time until Monday. Have fun!
On 07/18/2013 12:37 PM, Adrian Roe wrote:
><i> That would be perfect! Do you want me to make the change and issue a
</I>><i> pull request?
</I>><i>
</I>><i> --
</I>><i> Dr Adrian Roe
</I>><i> Director
</I>><i>
</I>><i> On Thursday, 18 July 2013 at 11:36, Loïc Hoguin wrote:
</I>><i>
</I>>><i> I don't think the problem is that the handler is reused, we don't reuse
</I>>><i> them if there's an error. However we do catch errors to print them in
</I>>><i> the logs, and then the process stops normally. If you link without
</I>>><i> trap_exit you receive a normal exit signal which is ignored and doesn't
</I>>><i> kill your process. I suppose we should throw an exit signal when we got
</I>>><i> an error, after logging everything, instead of stopping normally.
</I>>><i>
</I>>><i> On 07/18/2013 12:31 PM, Adrian Roe wrote:
</I>>>><i> My issue is the other way round. My handler crashes - and terminate
</I>>>><i> gets called, but the linked process is NOT stopped (unless I stop it in
</I>>>><i> terminate having stashed any processes I need to stop in the process
</I>>>><i> dictionary - this is what I'm currently doing, but yuck!)
</I>>>><i>
</I>>>><i> . My question is whether it wouldn't be better to no re-use the handler
</I>>>><i> process that has crashed and replace it so that handler's can use the
</I>>>><i> canonical erlang way of stopping related processes rather than having to
</I>>>><i> do it by hand.
</I>>>><i>
</I>>>><i> Obviously if the handler does not crash there's no need to kill the
</I>>>><i> process, so the current efficiency saving works in the "normal" case/
</I>>>><i>
</I>>>><i> --
</I>>>><i> Dr Adrian Roe
</I>>>><i> Director
</I>>>><i>
</I>>>><i> On Thursday, 18 July 2013 at 11:20, Loïc Hoguin wrote:
</I>>>><i>
</I>>>>><i> I don't know what happens but there's two things I know:
</I>>>>><i>
</I>>>>><i> * Handlers don't trap_exit, so if the linked process crashes, they
</I>>>>><i> crash too
</I>>>>><i> * If the handler crashes, we close the connection and stop the
</I>>>>><i> handler; if not this is a bug
</I>>>>><i>
</I>>>>><i> After your log message the handler should stop unless there's a bug
</I>>>>><i> somewhere.
</I>>>>><i>
</I>>>>><i> On 07/18/2013 12:15 PM, Adrian Roe wrote:
</I>>>>>><i> We have been using spawn_linked workers to handle tasks that live for
</I>>>>>><i> the lifetime of a single HTTP request
</I>>>>>><i>
</I>>>>>><i> Although in the cowboy guide it is clear that Cowboy can use "One
</I>>>>>><i> Process of Many Requests" I am surprised that this is the case even if
</I>>>>>><i> the handler crashes. For example, our use case is to copy a large file
</I>>>>>><i> to the server over HTTP where a worker process relays the file contents
</I>>>>>><i> to long term storage. The worker process is spawn_linked from the HTTP
</I>>>>>><i> handler and (for our use case) should die if the handler stops.
</I>>>>>><i>
</I>>>>>><i> If the client stops the upload (for example by browsing away, or losing
</I>>>>>><i> connectivity) we correctly receive an error (see sample Lager trace
</I>>>>>><i> below), but what we are seeing is that spawn_linked processes are NOT
</I>>>>>><i> being killed.
</I>>>>>><i>
</I>>>>>><i> Is this intended behaviour - I accept it makes sense to reuse the
</I>>>>>><i> processes but should this continue to be the case even if the previous
</I>>>>>><i> use of the process crashed? If it is intended behaviour I think the
</I>>>>>><i> docs should highlight this as we've been leaking processes for some
</I>>>>>><i> time
</I>>>>>><i> now, but I've always seen it as erlang's job to look after related
</I>>>>>><i> process trees in the event of error. Our current workaround is to hold
</I>>>>>><i> a list of linked processes in process storage and then kill them in the
</I>>>>>><i> terminate handler which is ugly in the extreme!! We don't know the PIDS
</I>>>>>><i> of the linked processes until it is too late to return State to Cowboy
</I>>>>>><i> (i.e. we are already in our handle code)...
</I>>>>>><i>
</I>>>>>><i> Kind regards
</I>>>>>><i>
</I>>>>>><i> Adrian
</I>>>>>><i>
</I>>>>>><i> 16:09:32.347 [info] Trailer upload failed with reason
</I>>>>>><i> {case_clause,{error,closed}}
</I>>>>>><i> 16:09:32.348 [error] ** Cowboy handler upload_trailer_resource
</I>>>>>><i> terminating in handle/2
</I>>>>>><i> for the reason error:{case_clause,{error,closed}}
</I>>>>>><i> ** Handler state was {state,undefined,0,undefined,undefined,undefined}
</I>>>>>><i> ** Request was
</I>>>>>><i> [{socket,#Port<0.11230>},{transport,ranch_tcp},{connection,keepalive},{pid,<0.1987.0>},{method,<<"POST">>},{version,'HTTP/1.1'},{peer,{{84,92,32,116},64136}},{host,<<"54.225.117.108">>},{host_info,undefined},{port,8000},{path,<<"/upload_trailer">>},{path_info,undef
</I>>>>>><i> ined},{qs,<<"name=linux-7.4.21.zip&size=54015414">>},{qs_vals,undefined},{bindings,[]},{headers,[{<<"host">>,<<"54.225.117.108:8000">>},{<<"connection">>,<<"keep-alive">>},{<<"content-length">>,<<"54015414">>},{<<"origin">>,<<"<A HREF="http://54.225.117.108:8000">http://54.225.117.108:8000</A>">>},{<<"user-agent">>,<<"M
</I>>>>>><i> ozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36
</I>>>>>><i> (KHTML,
</I>>>>>><i> like Gecko) Chrome/28.0.1500.71
</I>>>>>><i> Safari/537.36">>},{<<"content-type">>,<<>>},{<<"accept">>,<<"*/*">>},{<<"referer">>,<<"<A HREF="http://54.225.117.108:8000/">http://54.225.117.108:8000/</A>">>},{<<"accept-encoding">>,<<"gzip,deflate,sdch">>},{<<"acce
</I>>>>>><i> pt-language">>,<<"en-US,en;q=0.8">>},{<<"cookie">>,<<"__jwpusr=cbc133d7-1b49-443c-8a13-364660cc93e5;
</I>>>>>><i> id3as_manager=f4803c004d71dde3b64394f6e6f44faa54970e93">>}]},{p_headers,[{<<"connection">>,[<<"keep-alive">>]}]},{cookies,undefined},{meta,[]},{body_state,waiting},{multipart,unde
</I>>>>>><i> fined},{buffer,<<>>},{resp_compress,true},{resp_state,waiting},{resp_headers,[]},{resp_body,<<>>},{onresponse,undefined}]
</I>>>>>><i> ** Stacktrace:
</I>>>>>><i> [{i_cowboy,stream_body,0,[{file,"src/i_cowboy.erl"},{line,76}]},{upload_trailer_resource,stream_upload_file,4,[{file,"src/endpoints/upload_trailer_resource.erl"},{line,247}]},{upload_trailer_resource,upload_file,1,[{file,"src/endpoints/upload_trailer_resource.erl"}
</I>>>>>><i> ,{line,237}]},{upload_trailer_resource,head_or_post,1,[{file,"src/endpoints/upload_trailer_resource.erl"},{line,202}]},{upload_trailer_resource,sequence,2,[{file,"src/endpoints/upload_trailer_resource.erl"},{line,106}]},{upload_trailer_resource,process_request,1,[{file,"src/endpo
</I>>>>>><i> ints/upload_trailer_resource.erl"},{line,212}]},{i_cowboy,do,3,[{file,"src/i_cowboy.erl"},{line,29}]},{cowboy_handler,handler_handle,4,[{file,"src/cowboy_handler.erl"},{line,119}]}]
</I>>>>>><i>
</I>>>>>><i>
</I>>>>>><i> --
</I>>>>>><i> Dr Adrian Roe
</I>>>>>><i> Director
</I>>>>>><i>
</I>>>>>><i>
</I>>>>>><i>
</I>>>>>><i> _______________________________________________
</I>>>>>><i> Extend mailing list
</I>>>>>><i> <A HREF="https://lists.ninenines.eu/listinfo/extend">Extend at lists.ninenines.eu</A> <mailto:<A HREF="https://lists.ninenines.eu/listinfo/extend">Extend at lists.ninenines.eu</A>>
</I>>>>>><i> <A HREF="http://lists.ninenines.eu:81/listinfo/extend">http://lists.ninenines.eu:81/listinfo/extend</A>
</I>>>>><i>
</I>>>>><i>
</I>>>>><i> --
</I>>>>><i> Loïc Hoguin
</I>>>>><i> Erlang Cowboy
</I>>>>><i> Nine Nines
</I>>>>><i> <A HREF="http://ninenines.eu">http://ninenines.eu</A>
</I>>><i>
</I>>><i>
</I>>><i> --
</I>>><i> Loïc Hoguin
</I>>><i> Erlang Cowboy
</I>>><i> Nine Nines
</I>>><i> <A HREF="http://ninenines.eu">http://ninenines.eu</A>
</I>><i>
</I>
--
Loïc Hoguin
Erlang Cowboy
Nine Nines
<A HREF="http://ninenines.eu">http://ninenines.eu</A>
</PRE>
<!--endarticle-->
<HR>
<P><UL>
<!--threads-->
<LI>Previous message: <A HREF="000159.html">[99s-extend] Cowboy handler linked processes
</A></li>
<LI>Next message: <A HREF="000161.html">[99s-extend] Cowboy handler linked processes
</A></li>
<LI> <B>Messages sorted by:</B>
<a href="date.html#160">[ date ]</a>
<a href="thread.html#160">[ thread ]</a>
<a href="subject.html#160">[ subject ]</a>
<a href="author.html#160">[ author ]</a>
</LI>
</UL>
<hr>
<a href="https://lists.ninenines.eu/listinfo/extend">More information about the Extend
mailing list</a><br>
</body></html>
|
