[99s-extend] Distributed model?
Lee Sylvester
lee.sylvester at gmail.com
Thu Apr 11 14:55:29 CEST 2013
Thank you, Jeremy, that's good advice. It's not so much a chat platform, but I guess it would resemble one in architecture. The part I'm concerned about, though, is should I be avoiding the internal Erlang messaging between connections (over many nodes) for heavy messaging?
Thanks,
Lee
On 11 Apr 2013, at 08:04, Jeremy Ong <jeremy at quarkgames.com> wrote:
> I see. I assume this is for a chat server of some sort?
>
> You don't want the user process sending all these messages because the
> user process wouldn't be able to do anything useful (like receive
> messages) in the meantime.
>
> Better is to implement a pubsub process for each channel of
> communication (i.e. one process per room) or rely on Redis pubsub or
> something if speed is extremely important.
>
> There is no way to get around the O(N) complexity of broadcasting.
>
> On Wed, Apr 10, 2013 at 11:49 PM, Lee Sylvester <lee.sylvester at gmail.com> wrote:
>> Thanks Jeremy, but what about inter-node communication? If I have a user on node A sending a message to 10k users located on 10 other nodes, what is the best way to handle that? Especially if this user is sending several messages and expecting replies. Should I use the standard Erlang inter-process messaging or should I implement an MQ on top to handle this?
>>
>> Thanks,
>> Lee
>>
>>
>> On 11 Apr 2013, at 07:29, Jeremy Ong <jeremy at quarkgames.com> wrote:
>>
>>> Make all the machines identically and add an haproxy (or equivalent)
>>> machine to load balance between all of them. Haproxy can handle many
>>> many requests. Keep in mind that with tcp, the load balancer is just
>>> accepting the socket but then the client communicates with the actual
>>> application server directly afterwards.
>>>
>>> On Wed, Apr 10, 2013 at 10:51 PM, Lee Sylvester <lee.sylvester at gmail.com> wrote:
>>>> Hi guys,
>>>>
>>>> So, I have my Cowboy / Bullet server working nicely, now, with much thanks to members on this list. I'm now looking at the best means of clustering this app. I want to set this up so that, should the connection count get very high (which it will), then I should only have to throw more machines at this problem and it'll all go away.
>>>>
>>>> I've got most of the logic working for this, but what I'm worried about is sending a lot of content over the erlang inter-node connection. I've heard hogging this line can be both a bottleneck and can potentially interrupt the heartbeat between nodes. With this in mind, should I look at adding a ZMQ layer or some such to facilitate this? What is the general solution to high traffic between nodes?
>>>>
>>>> Thanks,
>>>> Lee
>>>> _______________________________________________
>>>> Extend mailing list
>>>> Extend at lists.ninenines.eu
>>>> http://lists.ninenines.eu:81/listinfo/extend
>>
More information about the Extend
mailing list