| Age | Commit message (Collapse) | Author |
|---|
|
* anders/diameter/codec/OTP-14511: (26 commits)
Limit SCTP testing in traffic suite
Increase init_per_group timetrap in traffic suite
Add diameter_util:eprof/1 for test
Don't search forms unnecessarily in diameter_exprecs parse transform
Increase init_per_suite timetrap in traffic suite
Don't count AVPs unnecessarily at encode
Test decode_format record_from_map in traffic suite
Tweak limiting of testcases in traffic suite
Don't take length of AVP lists unnecessarily at encode
Tweak map-valued decode
Rearrange group names in traffic suite
Randomly wrap answers in diameter_packet in transport suite
Don't exercise client/server encoding independently in traffic suite
Add decode_format record_from_map
Rename record_decode -> decode_format
Create fewer client connections in traffic suite
Test record_decode in traffic suite
Map answers to maps in traffic suite
Test map encoding in traffic suite
Let messages and grouped AVPs be decoded to lists
...
|
|
{record_decode, map} is a bit too quirky.
|
|
That is, decode to the same format that encode already accepts. Only a
message has its name at the head of the list since AVPs are already
name/value pairs.
|
|
With {record_decode, map}. The option name is arguably a bit misleading
now, but not too objectionable given that the encode/decode in question
has historically only been of records.
One advantage of the map decode is that the map only contains values for
those AVPs existing in the message or grouped AVP in question. The name
of the message or grouped AVP is stored in with key ':name', the leading
colon ensuring that the key isn't a diameter-name.
Decoding to maps makes the hrl files generated from dictionary files
largely irrelevant. There are value defines generated into these, but
they're typically so long as to be unusable.
|
|
To control whether or not messages and grouped AVPs are decoded to
records, in #diameter_packet.msg and #diameter_avp.value respectively.
The decode became unnecessary for diameter's needs in parent commit,
which decoupled it from the checking of AVP arities.
|
|
Service configuration share_peers and use_shared_peers is used to share
peer connections with other connected nodes having a service of the same
name: a service process asks its neighbours about existing connections
when it starts, and pushes new connections as they're established.
The problem is that the mechanics assume that nodes() doesn't change. In
particular, if a neighbour isn't connected when a service starts then it
doesn't receive the request to share connections. Solve by having each
service process monitor nodes, a nodeup notification causing it to
request connections of its neighbours. Nodes going down is already
handled, by remote connections being monitored in diameter_service.
|
|
* anders/diameter/capx_vs_dpr/OTP-14338:
Let candidate peers be passed to diameter:call/4
Comment on RFC ambiguity regarding application identifiers
Remove trailing whitespace
|
|
By accepting an MFA that is applied to the fun that is otherwise spawned
for each incoming request, to allow handler processes to be reused. This
is not yet documented and may change, but the motivation is to let spawn
be replaced by process pool, from which the MFA selects. A list-valued
spawn_opt is equivalent to {erlang, spawn_opt, [Opts]}.
|
|
To solve the problem of being able to send messages to a peer that
hasn't advertised support for the application in question, as discussed
in the parent commit. diameter:call/4 can be passed 'peer' options to
identify candidates, and the only requirement is that an appropriate
dictionary be configured for encode. Filters are applied as if
candidates had been selected by advertised application.
|
|
The tuple is returned from and passed to callbacks, so retain the tuple
instead of its elements.
|
|
By passing additional arguments through it.
|
|
This and subsequent commits are destined for OTP 20.0.
|
|
The table has existed forever, to route incoming answers to a waiting
request process: each outgoing request writes to the table, and each
incoming answer reads. This has been seen to suffer from lock contention
at high load however, so this commit moves the routing into the
diameter_peer_fsm processes that are diameter's conduit to transport
processes. The request table is still used for failover detection, but
entries are only written when a watchdog state transitions leaves or
enters state OKAY.
|
|
A replacement accepting transport could be started after the service
process received a shutdown message from diameter_config, if a
connection was accepted before the transport process in question was
terminated. The replacement lived on until the service needed to restart
it.
|
|
Letters are cheap.
|
|
* anders/diameter/info/OTP-13508:
Add diameter:peer_find/1
Add diameter:peer_info/1
|
|
To return information about a single peer_ref(), to avoid having to
retrieve more than is needed with service_info/2.
|
|
|
|
Whether making record declarations unreadable to compensate for
dialyzer's ignorance of match specs is worth it is truly debatable.
|
|
|
|
* anders/diameter/retransmission/OTP-13342:
Fix handling of shared peer connections in watchdog state SUSPECT
Remove unnecessary parentheses
Remove dead export
|
|
A peer connection shared from a remote node was regarded as being
available for peer selection (aka up) as long as its peer_fsm process
was alive; that is, for the lifetime of the peer connection. In
particular, it didn't take note of transitions into watchdog state
SUSPECT, when the connection remains. As a result, retransmissions could
select the same peer connection whose watchdog transition caused the
retransmission.
A service process now broadcasts a peer_down event just as it
does a peer_up event.
The fault predates the table rearrangements of commit 8fd4e5f4.
|
|
Each service process maintains a dictionary of peers, mapping an
application alias to a {pid(), #diameter_caps{}} list of connected
peers. These lists are potentially large, peers were appended to the end
of the list for no particular reason, and these long lists were
constructed/deconstructed when filtering them for pick_peer callbacks.
Many simultaneous outgoing request could then slow the VM to a crawl,
with many scheduled processes mired in list manipulation.
The pseudo-dicts are now replaced by plain ets tables. The reason for
them was (once upon a time) to have an interface interchangeable with a
plain dict for debugging purposes, but strict swapablity hasn't been the
case for some time now, and in practice a swap has never taken place.
Additional tables mapping Origin-Host/Realm have also been introduced,
to minimize the size of the peers lists when peers are filtered on
host/realm. For example, a filter like
{any, [{all, [realm, host]}, realm]}
is probably a very common case: preferring a Destination-Realm/Host
match before falling back on Destination-Realm alone. This is now more
efficiently (but not equivalently) expressed as
{first, [{all, [realm, host]}, realm]}
to stop the search when the best match is made, and extracts peers from
host/realm tables instead of searching through the list of all peers
supporting the application in question. The code to try and start with a
lookup isn't exhaustive, and the 'any' filter is still as inefficient as
previously.
|
|
See commit 862af31d.
|
|
|
|
|
|
Each service process maintains a dictionary of peers, mapping an
application alias to a {pid(), #diameter_caps{}} list of connected
peers. These lists are potentially large, peers were appended to the end
of the list for no particular reason, and these long lists were
constructed/deconstructed when filtering them for pick_peer callbacks.
Many simultaneous outgoing request could then slow the VM to a crawl,
with many scheduled processes mired in list manipulation.
The pseudo-dicts are now replaced by plain ets tables. The reason for
them was (once upon a time) to have an interface interchangeable with a
plain dict for debugging purposes, but strict swapablity hasn't been the
case for some time now, and in practice a swap has never taken place.
Additional tables mapping Origin-Host/Realm have also been introduced,
to minimize the size of the peers lists when peers are filtered on
host/realm. For example, a filter like
{any, [{all, [realm, host]}, realm]}
is probably a very common case: preferring a Destination-Realm/Host
match before falling back on Destination-Realm alone. This is now more
efficiently (but not equivalently) expressed as
{first, [{all, [realm, host]}, realm]}
to stop the search when the best match is made, and extracts peers from
host/realm tables instead of searching through the list of all peers
supporting the application in question. The code to try and start with a
lookup isn't exhaustive, and the 'any' filter is still as inefficient as
previously.
|
|
See commit 862af31d.
|
|
* anders/diameter/17.5.6.7/OTP-13211:
vsn -> 1.9.2.2
Update/fix appup for 17.5.6.7
Be resilient to diameter_service state upgrades
|
|
By not failing in code that looks up state: pick_peer and service_info.
|
|
Record field types have been modified due to commit 8ce35b2:
"Take out automatic insertion of 'undefined' from typed record fields".
|
|
* anders/diameter/M-bit/OTP-12947:
Add service_opt() strict_mbit
|
|
There are differing opinions on whether or not reception of an arbitrary
AVP setting the M-bit is an error. 1.3.4 of RFC 6733 says this about
how an existing Diameter application may be modified:
o The M-bit allows the sender to indicate to the receiver whether or
not understanding the semantics of an AVP and its content is
mandatory. If the M-bit is set by the sender and the receiver
does not understand the AVP or the values carried within that AVP,
then a failure is generated (see Section 7).
It is the decision of the protocol designer when to develop a new
Diameter application rather than extending Diameter in other ways.
However, a new Diameter application MUST be created when one or more
of the following criteria are met:
M-bit Setting
An AVP with the M-bit in the MUST column of the AVP flag table is
added to an existing Command/Application. An AVP with the M-bit
in the MAY column of the AVP flag table is added to an existing
Command/Application.
The point here is presumably interoperability: that the command grammar
should specify explicitly what mandatory AVPs much be understood, and
that anything more is an error.
On the other hand, 3.2 says thus about command grammars:
avp-name = avp-spec / "AVP"
; The string "AVP" stands for *any* arbitrary AVP
; Name, not otherwise listed in that Command Code
; definition. The inclusion of this string
; is recommended for all CCFs to allow for
; extensibility.
This renders 1.3.4 pointless unless "*any* AVP" is qualified by "not
setting the M-bit", since the sender can effectively violate 1.3.4
without this necessitating an error at the receiver. If clients add
arbitrary AVPs setting the M-bit then request handling becomes more
implementation-dependent.
The current interpretation in diameter is strict: if a command grammar
doesn't explicitly allow an AVP setting the M-bit then reception of such
an AVP is regarded as an error. The strict_mbit option now allows this
behaviour to be changed, false turning all responsibility for the M-bit
over to the user.
|
|
The diffs are all about adapting to the OTP 18 time interface. The code
was previously backwards compatible, falling back on the erlang:now/0 if
erlang:monotonic_time/0 is unavailable, but this was seen to be a bad
thing in commit 9c0f2f2c. Use of erlang:now/0 is now removed.
|
|
By doing away with more wrapping that the parent commit started to
remove.
|
|
|
|
To bound the length of incoming messages that will be decoded. A message
longer than the specified number of bytes is discarded. An
incoming_maxlen_exceeded counter is incremented to make note of the
occurrence.
The motivation is to prevent a sufficiently malicious peer from
generating significant load by sending long messages with many AVPs for
diameter to decode. The 24-bit message length header accomodates
(16#FFFFFF - 20) div 12 = 1398099
Unsigned32 AVPs for example, which the current record-valued decode is
too slow with in practice. A bound of 16#FFFF bytes allows for 5461
small AVPs, which is probably more than enough for the majority of
applications, but the default is the full 16#FFFFFF.
|
|
To control whether stringish Diameter types are decoded to string or
left as binary. The motivation is the same as in the parent commit: to
avoid large strings being copied when incoming Diameter messages are
passed between processes; or *if* in the case of messages destined for
handle_request and handle_answer callbacks, since these are decoded in
the dedicated processes that the callbacks take place in. It would be
possible to do something about other messages without requiring an
option, but disabling the decode is the most effective.
The value is a boolean(), true being the default for backwards
compatibility. Setting false causes both diameter_caps records and
decoded messages to contain binary() in relevant places that previously
had string(): diameter_app(3) callbacks need to be prepared for the
change.
The Diameter types affected are OctetString and the derived types that
can contain arbitrarily large values: OctetString, UTF8String,
DiameterIdentity, DiameterURI, IPFilterRule, and QoSFilterRule. Time and
Address are unaffected.
The DiameterURI decode has been redone using re(3), which both
simplifies and does away with a vulnerability resulting from the
conversion of arbitrary strings to atom.
The solution continues the use and abuse of the process dictionary for
encode/decode purposes, last seen in commit 0f9cdba.
|
|
* anders/diameter/time/OTP-12439:
Use new time api in test suites
Use new time api in implementation
|
|
* anders/diameter/pool/OTP-12428:
Fix SCTP match blunder in suites
Be backwards compatible with diameter_sctp listener state
Add gen_tcp testcase that fails sporadically
Simplify transport suite
Remove (ancient) dead code
Don't orphan slave nodes in example suite
Refresh example code
Improve language consistency in diameter(1)
Add pool suite to test transport_opt() pool_size
Adapt tcp/sctp transport modules for pool_size > 1
Add transport_opt() pool_size
|
|
In particular, deal with the deprecation of erlang:now/0 in OTP 18. Be
backwards compatible with older releases: the new api is only used when
available.
The test suites have not been modified.
|
|
Transport processes are started by diameter one at a time. In the
listening case, a transport process accepts a connection, tells the
peer_fsm process, which tells its watchdog process, which tells its
service process, which then starts a new watchdog, which starts a new
peer_fsm, which starts a new transport process, which (finally) goes
about accepting another connection. In other words, not particularly
aggressive in accepting new connections. This behaviour doesn't do
particularly well with a large number of concurrent connections: with
TCP and 250 connecting peers we see connections being refused.
This commit adds the possibilty of configuring a pool of accepting
processes, by way of a new transport option, pool_size. Instead of
diameter:add_transport/2 starting just a single process, it now starts
the configured number, so that instead of a single process waiting for a
connection there's now a pool.
The option is even available for connecting processes, which provides an
alternate to adding multiple transports when multiple connections to the
same peer are required. In practice this also means configuring
{restrict_connections, false}: this is not implicit.
For backwards compatibility, the form of
diameter:service_info(_,transport) differs in the connecting case,
depending on whether or not pool_size is configured.
Note that transport processes for the same transport_ref() can be
started concurrently when pool_size > 1. This places additional
requirements on diameter_{tcp,sctp}, that will be dealt with in a
subsequent commit.
|
|
There's no need for building a pid list only to map it to a list of
monitor references. Also, monitoring before banging the shutdown
message makes for better trace, avoiding unnecessary noproc reasons when
the process dies before the monitor is created.
|
|
The order of peers presented to a diameter_app(3) pick_peer callback has
previously not been documented, but there are use cases that are
simplified by an ordering. For example, consider preferring a direct
connection to a specified Destination-Host/Realm to any host in the
realm. The implementation previously treated this as a special case by
placing matching hosts at the head of the peers list, but the
documentation made no guarantees. Now present peers in match-order, so
that the desired sorting is the result of the following filter.
{any, [{all, [host, realm]}, realm]}
The implementation is not backwards compatible in the sense that a realm
filter alone is no longer equivalent in this case. However, as stated,
the documentation never made any guarantees regarding the sorting.
|
|
That is, instead of including the list in a diameter:service_info/2 info
tuple, only include the number of references and the number of bytes
referenced. The list itself can be quite large and typically isn't that
interesting, at least not to a diameter user.
|
|
To extract only process info from connections info, which can be useful
to reduce the amount of information returned.
Choose 'info' for the item since process_info is more than one word: all
others are one. Don't choose memory since it's too specific: might want
to use it for more.
|
|
To show process_info of interest. This is not yet documented since it
may well change.
|
|
The former were a little over-enthusiastic and could cause a node to be
logged to death if a peer Diameter node was sufficiently ill-willed.
The function calls are to diameter_lib:log/4, the arguments of which
identify the happening in question, and which does nothing but provide a
function to trace on. Many existing log calls have been shrunk.
The only remaining traffic-related report (hopefully) is that resulting
from {answer_errors, report} config, and this has been slimmed.
|
|
* anders/diameter/watchdog_leak/OTP-11934:
Simplify sending of 'close' to watchdog
Fix watchdog table leak
|
|
* anders/diameter/request_leak/OTP-11893:
Fix leaking request table
Add check that request table is empty to failover suite
Comment fix
|
