Don't leave extra bit in decoded AVP data.

An extra bit could be communicated in the data field of a diameter_avp record in the case of length errors. Of no consequence for code using the record encoding of Diameter messages, but code examining diameter_avp records would see this bit.

Dictionary files must be recompiled for the fix to have effect.

Own Id: OTP-12074

Fix counting of outgoing requests and answers setting the E-bit.

OTP-11721 broke these counters for all outgoing requests except DWR, and caused answers setting the E-bit to be counted as unknown messages.

Own Id: OTP-12080

Fix Failed-AVP decode.

The best-effort decode only worked for AVPs in the common dictionary, not for those in the dictionary of the application identified in the Diameter Header of the answer message in question.

Failed-AVP in an answer decoded with the RFC 3588 common dictionary (diameter_gen_base_rfc3588) was regarded as an error. The RFC 6733 dictionary was unaffected.

Dictionary files must be recompiled for the fix to have effect.

Own Id: OTP-12094

Improve robustness.

Counters returned by diameter:service_info/2 now only count messages known to the dictionary in question, so that an attacker cannot cause arbitrarily many counters to be created.

Messages to the Erlang log have been minimized, and those related to traffic have been removed entirely since an attacker could cause a node to be logged to death. Consequently, the default answer_errors configuration has been changed from report to discard. A service needs to be restarted for the change in default to take effect.

Own Id: OTP-11721

Fix request table leak.

Outgoing Diameter requests are stored in a table until an answer is received or times out. Calling diameter:stop_service/1 before this took place would orphan the entries, resulting in a memory leak.

Own Id: OTP-11893

Fix broken SCTP transport.

OTP-11593 caused the sending of answer messages over SCTP to fail.

Own Id: OTP-11901 Aux Id: OTP-11593

Fix watchdog process leak.

A failed capabilities exchange on a listening transport would orphan a process, causing a memory leak.

Own Id: OTP-11934

Fix incorrect handling of incoming DPR.

In the case of a listening transport, a reconnection by a peer following DPR could transition the watchdog state to REOPEN instead of OKAY.

Own Id: OTP-11938

Fix handling of AVP length errors on unknown AVPs.

An AVP (Header) length that pointed past the end of the message was not flagged as a 5014 error in this case. Moreover, encoding such an AVP in the Failed-AVP of an answer message as a consequence of other errors (eg. M-bit, resulting in 5001) failed if the AVP contained a complete header.

Dictionary files must be recompiled for the fix to have effect.

Own Id: OTP-11946

Fix broken check in dictionary compilation.

That an AVP specified in the content of a @codecs or @custom_types section was undefined went undetected, causing compilation to fail when attempting to lookup the AVP's type.

Own Id: OTP-11958

Add result code counters for CEA, DWA, and DPA.

In addition to the existing result code counters on other answer messages.

Own Id: OTP-11891

Add best-effort decode of AVPs within Failed-AVP.

OTP-11007 disabled the decode of AVPs in Failed-AVP since errors could cause the decode of Failed-AVP itself to fail. Component AVPs are now decoded if possible, otherwise not. AVPs of type Grouped are decoded as much as possible, as deeply as possible.

Dictionary files must be recompiled for the fix to have effect.

Own Id: OTP-11936 Aux Id: OTP-11007

Add counters for encode errors in outgoing Diameter messages.

In addition to the existing counters on decode errors. The latter now count independently of result codes in answer messages since decode errors do not preclude the presence of a result code.

Own Id: OTP-11937

Add missing check at dictionary compilation.

In particular, that an AVP defined as having type Grouped in an @avp_types section has a corresponding definition in a @grouped section.

Own Id: OTP-11561

Correct documentation on the setting of Origin-State-Id

It was incorrectly stated that the AVP would be set in an outgoing DPR/DPA.

Own Id: OTP-11583

Change interface for communicating outbound stream id to diameter_sctp

The module uses the transport_data field of record diameter_packet to communicate the stream on which the an incoming message is received and on which an outgoing message should be sent, the previous interface being that both are communicated as a tuple of the form {stream, Id}. However, since diameter retains the value of an incoming request's transport_data unless the corresponding answer message specifies otherwise, the behaviour in this case is to send an answer on the outbound stream with the same identifier as the that of the inbound stream on which the request was received. If the inbound stream id is greater than or equal to the number of outbound streams then this is guaranteed to fail, causing the transport process in question to terminate. There is no relationship between inbound and outbound stream identifiers so diameter_sctp's imposition of one is simply wrong.

Outbound stream ids are now communicated with a different tuple: {outstream, Id}, interpreted modulo the number of outbound streams. Thus, retention of an inbound request's transport_data has no effect on the selection of an outbound stream.

The change in interface is not strictly backwards compatible because of the new atom for the outbound stream. However, as there is currently no documented way of obtaining the available number of outbound streams for a peer connection, there is no way for a client to have known the range of ids from which it could reliably have chosen with the previous interface, so any setting of the outbound stream has probably been unintentional. Not explicitly specifying an outbound stream now results in a round-robin selection.

Thanks to Sharmila Pillai for reporting the problem.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-11593

Fix unicode path failure in diameter_make:codec/2.

A dictionary path containing a unicode codepoint > 255 caused the function to fail.

Own Id: OTP-11655

Fix 'accept' config to diameter_sctp.

OTP-10893 added support for {accept, Match} tuples to specify addresses or regexps that should be matched against peer addresses to decide whether or not a newly established association should be retained, but this hasn't been functional in the SCTP case because of missing support in inet(3).

The display of both local and peer addresses in diameter:service_info/2 output has also been corrected.

Own Id: OTP-11661 Aux Id: OTP-10229

Be lenient with the M-bit in Grouped AVPs.

RFC 6733 says this, in 4.4:

Receivers of a Grouped AVP that does not have the 'M' (mandatory) bit set and one or more of the encapsulated AVPs within the group has the 'M' (mandatory) bit set MAY simply be ignored if the Grouped AVP itself is unrecognized. The rule applies even if the encapsulated AVP with its 'M' (mandatory) bit set is further encapsulated within other sub-groups, i.e., other Grouped AVPs embedded within the Grouped AVP.

Fix pick_peer case clause failure.

In the case of {call_mutates_state, true} configuration on the service in question, any peer selection that failed to select a peer resulted in a case clause failure. This was noticed in the case of a peer failover in which an alternate peer wasn't available.

Own Id: OTP-11789

Rename reconnect_timer to connect_timer.

The former is still accepted for backwards compatibility, but the name is misleading given the semantics of the timer.

Own Id: OTP-11168

Extend diameter_make(3).

Dictionaries can now be compiled from strings, not just filesystem paths, and results can be returned instead of written to the filesystem.

Own Id: OTP-11348

Remove hardcoding of diameter_base as @prefix on dictionaries for application id 0.

Own Id: OTP-11361

Fix setting of End-to-End and Hop-by-Hop Identifiers in outgoing DWA.

Broken by OTP-11184, which caused the identifiers to be set anew, discarding the values from the incoming DWR.

Own Id: OTP-11367

Fix handling of 5014, DIAMETER_INVALID_AVP_LENGTH.

The error was detected as 5004, DIAMETER_INVALID_AVP_VALUE, for some Diameter types, in which case an AVP length that pointed past the end of a message resulted in encode failure.

Own Id: OTP-11395

Fix UTF8String encode.

Encode now accepts any nested list of codepoints and binaries. A list containing a binary was previously misinterpreted and the documentation was incomplete.

Own Id: OTP-11172

Ensure DWR isn't sent immediately after DWA.

This was possible if the timing was unfortunate. An incoming DWR now properly resets the watchdog timer.

Own Id: OTP-11184

Fix faulty encode of Failed-AVP

Reception of a CER, DWR or DPR that has decode failures caused encode of the corresponding answer message to fail.

Own Id: OTP-11293

Fix broken service_opt() spawn_opt.

The option was ignored.

Own Id: OTP-11299

Fix handling of 5014 (INVALID_AVP_LENGTH) errors.

This was in some cases reported as 3009 (INVALID_AVP_BITS).

Note that the correction is partially implemented in modules generated by diameterc(1): a dictionary file must be recompiled for the correction to apply to any messages it defines.

Own Id: OTP-11007

Fix faulty capitalization in release notes.

Diameter = the protocol.
diameter = the Erlang application.

Own Id: OTP-11014

Fix watchdog memory leak.

Entries were not removed from a service-specific ets table, causing them to be orphaned at connection reestablishment for listening transports, and diameter:remove_transport/2 for both listening and connecting transports.

The fault was introduced by OTP-10692 in diameter-1.4.1 (R16B).

Own Id: OTP-11019 Aux Id: OTP-10692

Fix decode failure on AVP Length < 8.

The failure caused the message in question to be discarded.

Own Id: OTP-11026

Respect Host-IP-Address configuration.

Addresses returned from a transport module were always used to populate Host-IP-Address AVP's in an outgoing CER/CEA, which precluded the sending of a VIP address. Transport addresses are now only used if Host-IP-Address is unspecified.

Own Id: OTP-11045

Fix mkdir race.

Install could fail if examples/code and examples/dict were created in parallel. Noticed on FreeBSD.

Own Id: OTP-11051

Fix recognition of 5001 on mandatory AVP's.

An AVP setting the M-bit was not regarded as erroneous if it was defined in the dictionary in question and its container (message or Grouped AVP) had an 'AVP' field. It's now regarded as a 5001 error (AVP_UNSUPPORTED), as in the case that the AVP is not defined.

Note that the correction is partially implemented in modules generated by diameterc(1): a dictionary file must be recompiled for the correction to apply to any messages it defines.

Own Id: OTP-11087

Fix setting of Failed-AVP on handle_request {answer_message, 5xxx} return.

Failed-AVP was never in the outgoing answer-message. It is now set with the AVP from the first entry with the specified Result-Code in the errors field of the incoming diameter_packet, if found.

Own Id: OTP-11092

Fix watchdog function_clause

A listening transport on a service that allowed multiple connections to the same peer could result in a function_clause error in module diameter_watchdog. The resulting crash was harmless but unseemly.

Thanks to Aleksander Nycz.

Own Id: OTP-11115

Fix population of Failed-AVP.

In cases in which diameter populated this AVP, many values were sent instead of one as suggested by RFC 6733. This was partially corrected by OTP-11007.

Own Id: OTP-11127 Aux Id: OTP-11007

Fix list-valued Vendor-Specific-Application-Id config

R16B (specifically, OTP-10760) broke the handling of such configuration, resulting in a function clause error if the list was not of length 3, and faulty interpretation of the list's contents otherwise. Only record-valued configuration was properly interpreted.

Own Id: OTP-11165

Allow peer connections to be shared between Erlang nodes for the purpose of sending outgoing requests.

A diameter_app(3) pick_peer/4 callback gets a list of remote candidates as argument, allowing a callback on one node to select a transport connection established on another node. The service_opt() share_peers controls the extent to which local connections are shared with remote nodes. The service_opt() use_shared_peers controls the extent to which connections shared from remote nodes are utilized on the local node.

Own Id: OTP-9610

Allow listening diameter_{tcp,sctp} transports to be configured with remote addresses.

Option 'accept' allows remote addresses to be configured as tuples or regular expressions. Remote addresses are matched against the configured values at connection establishment, any non-matching address causing the connection to be aborted.

Own Id: OTP-10893

Detect more transport_opt() configuration errors at diameter:add_transport/2.

Many errors would previously not be detected until transport start, diameter:add_transport/2 returning 'ok' but transport connections failing to be established. An error tuple is now returned.

Own Id: OTP-10972

Make explicit local address configuration optional in diameter_tcp:start/3.

The default address (as determined by gen_tcp) is now used when a local address is not explicitly configured.

Own Id: OTP-10986

Improve handling of unrecognized service options.

Such options were silently ignored by diameter:start_service/2. An error tuple is now returned.

Own Id: OTP-11017

Don't send default Inband-Security-Id in CER/CEA.

RFC 6733 recommends against the use of Inband-Security-Id. Only send a value that differs from the default, NO_INBAND_SECURITY = 0.

Own Id: OTP-11050

Make spawn options for request processes configurable.

Own Id: OTP-11060

Fix broken Vendor-Specific-Application-Id configuration.

RFC 6733 changed the definition of this Grouped AVP, changing the arity of Vendor-Id from 1* to 1. A component Vendor-Id can now be either list- or integer-valued in service and transport configuration, allowing it to be used with both RFC 3588 and RFC 6733 dictionaries.

Own Id: OTP-10942

Add transport_opt() watchdog_config to allow non-standard behaviour of the watchdog state machine.

This can be useful during test but should not be used on nodes that must conform to RFC 3539.

Own Id: OTP-10898

Fix erroneous watchdog transition from DOWN to INITIAL.

This transition took place when a peer connection was reestablished following a failed capabilities exchange. RFC 3539 requires DOWN to transition into REOPEN.

Own Id: OTP-10692

Add application_opt() request_errors to make the handling of incoming requests containing decode errors configurable.

The value 'callback' ensures that a handle_request callback takes place for all such requests, the default being for diameter to answer 3xxx series errors itself.

Own Id: OTP-10686

Add transport_opt() length_errors.

The value determines how messages received over the transport interface with an incorrect Message Length are dealt with.

Own Id: OTP-10687

Add commentary on RFC 6733 to Standards Compliance chapter of the User's Guide.

Own Id: OTP-10688

Allow a 5xxx result code in an answer-message on peer connections using the RFC 6733 common dictionary.

RFC 6733 allows this while RFC 3588 does not. A handle_request callback can return {answer_message, 3000..3999|5000..5999} in the simplest case.

Own Id: OTP-10759

Add dictionaries for RFC 6733.

Both the common and accounting dictionaries differ from their RFC 3588 counterparts, which is reflected in generated record definitions. Application configuration on a service or transport determines the dictionary that will be used on a given peer connection.

Own Id: OTP-10760

Allow a handle_request callback to control diameter's setting of Result-Code and Failed-AVP.

Setting errors = false in a returned #diameter_packet{} disables the setting.

Own Id: OTP-10761

Add registered server names to the app file.

Own Id: OTP-10442

Fix #diameter_header{} handling broken by OTP-10445.

The fault caused the the header of a [Header | Avps] request to be ignored if both end_to_end_id and hop_by_hop_id were undefined.

Own Id: OTP-10609

Fix error handling for handle_request callback.

A callback that returned a #diameter_packet{} would fail if the incoming request had decode errors.

Own Id: OTP-10614

Fix timing of service start event.

The event did not necessarily precede other events as documented.

Own Id: OTP-10618

Fix setting of header T flag at peer failover.

The flag is now set in the diameter_header record passed to a prepare_retransmit callback.

Own Id: OTP-10619

Fix sending of CER/CEA timeout event at capx_timeout.

The event was not sent as documented.

Own Id: OTP-10628

Fix improper setting of Application-ID in the Diameter header of an answer message whose E flag is set.

The value should be that of the request in question. The fault caused it always to be 0.

Own Id: OTP-10655

Fix faulty handling of AVP length errors.

An incorrect AVP length but no other errors caused an incoming request to fail.

Own Id: OTP-10693

Fix function clause resulting from use of an eval callback.

Own Id: OTP-10685

Fix faulty handling of Origin-State-Id and faulty config values.

The former was expected in a list despite the documentation requiring (correctly) an integer. A bare value for a list-valued capability was not handled.

Own Id: OTP-10440

Fix timing of up/down events.

Previously, a call to diameter:call/4 following a peer_up callback might incorrectly return {error, no_connection}, depending on timing. Both events now follow the corresponding callbacks.

Own Id: OTP-10459

Make diameter:service_info/2 usable in peer_up, peer_down and pick_peer callbacks.

Except for in pick_peer when {call_mutates_state, false}, it would previously hang indefinitely.

Own Id: OTP-10460

Verify that End-to-End and Hop-by-Hop Identifiers in an incoming CEA/DPA match those sent in the corresponding CER/DPR.

The values were previously ignored. Answers whose identifiers do not match are handled as unexpected.

Own Id: OTP-10565

Fix formatting problems in PDF documentation.

In particular, text corresponding to links in HTML was omitted in preformatted blocks. There are still issues with indentation but this is not diameter-specific.

Own Id: OTP-10583

Let prepare_request, prepare_retransmit and handle_request callbacks return a function to be invoked on outgoing messages after encode.

This allows encoded messages to be logged for example.

Own Id: OTP-10441

Add service_opt() 'restrict_connections' to allow multiple transport connections with the same peer.

Own Id: OTP-10443

Add service_opt() 'sequence' to allow the masking of a constant onto the topmost bits of End-to-End and Hop-by-Hop identifiers.

This allows the same service on different nodes to use distinct values in outgoing request messages.

Own Id: OTP-10445

Add diameter:service_info(PeerRef) to return the transport_ref() and transport_opt() list of the corresponding transport.

This allows easy access to these from diameter_app callbacks that only get peer_ref() as an argument.

Own Id: OTP-10470

Add reference pages diameter_codec(3) and diameter_make(3).

Own Id: OTP-10471

Add events for service start and stop.

Own Id: OTP-10492

Add transport_opt() 'disconnect_cb' to make the sending of DPR configurable.

Whether or not DPR should be sent at application stop, service stop or transport removal is determined by the value returned by the callback, as is the Disconnect-Cause and timeout if DPA is not received.

Own Id: OTP-10493

Add transport_opt() 'capx_timeout' for the timeout associated with non-reception of CER/CEA.

Own Id: OTP-10554

Allow a handle_request callback to return a #diameter_packet{}.

This allows an answer to set transport_data and header fields.

Own Id: OTP-10566

Update documentation for RFC 6733.

RFC 3588 is now obsolete.

Own Id: OTP-10568

Fix broken Result-Code setting and Destination-Host/Realm extraction.

Result-Code was assumed to have arity 1 when setting this value in an answer to a request containing AVP decode errors. Destination-Host/Realm were only correctly extracted from messages in the common application.

Own Id: OTP-10202

Handle insufficient capabilities configuration more gracefully.

A transport that does not have sufficient capabilities configuration in order to encode CER/CEA will now emit an error report noting the configuration error and exit instead of failing. The error is not detected at diameter:add_transport/2 since there is no requirement that a service be configured before its transports.

Own Id: OTP-10203

Ensure a failing peer_up/down callback does not affect transport connections to other peers.

Such a failure would previously have taken down all of a service's connections.

Own Id: OTP-10215

Statistics related to Diameter messages can be retrieved using diameter:service_info/2.

Both Diameter and socket-level statistics are available, for both incoming and outgoing messages.

Own Id: OTP-9608

Allow multiple transport_module/config to diameter:add_transport/2.

Multiple values are attempted in sequence until one results in an established connection. This provides a way for a connecting transport to specify configuration in order of preference. (For example, SCTP before TCP.)

Own Id: OTP-9885

Add events for state transitions in the RFC 3539 watchdog state machine.

The watchdog state is also available through diameter:service_info/2.

Own Id: OTP-10212

Add diameter:service_info(SvcName, connections).

This provides an alternative to diameter:service_info(SvcName, transport) that presents information per established connection instead of per transport reference.

Own Id: OTP-10213

Assorted documentation corrections/improvements.

Own Id: OTP-10216

Fix fault in sending of 'closed' events.

The fault made it possible for the 'closed' event not to be sent following a failed capabilities exchange.

Own Id: OTP-9824

Fix faulty diameterc -name/-prefix.

A minor blunder when introducing the new dictionary parser in diameter-1.0 broke these options.

Own Id: OTP-9826

Fix faulty cleanup after diameter:remove_transport/2.

Removing a transport removed the configuration but did not prevent the transport process from being restarted.

Own Id: OTP-9756

Add support for TLS over TCP.

RFC 3588 requires that a Diameter server support TLS. In practice this seems to mean TLS over SCTP since there are limitations with running over SCTP: see RFC 6083 (DTLS over SCTP), which is a response to RFC 3436 (TLS over SCTP). The current RFC 3588 draft acknowledges this by equating TLS with TLS/TCP and DTLS/SCTP.

TLS handshaking can take place either following a CER/CEA that negotiates TLS using the Inband-Security-Id AVP (the method documented in RFC 3588) or immediately following connection establishment (the method added to the current draft).

Own Id: OTP-9605

Improvements to the dictionary parser.

The dictionary parser now emits useful error messages in case of faults in the input file, also identifying the line number at which the fault was detected. There are semantic checks that were missing in the previous parser, a fault in the interpretation of vendor id's in combination with @inherits has been fixed and @end can be used to terminate parsing explicitly instead of always parsing to end of file.

Own Id: OTP-9639

Improve dictionary reusability.

Reusing a dictionary just to get a different generated module name or prefix previously required taking a copy of the source, which may consist of several files if inheritance is used, just to edit a couple of lines which don't affect the semantics of the Diameter application being defined. Options --name, --prefix and --inherits have been added to diameterc to allow corresponding values to be set at compile time.

Own Id: OTP-9641

Add capabilities_cb transport option.

Its value is a function that's applied to the transport reference and capabilities record after capabilities exchange. If a callback returns anything but 'ok' then the connection is closed. In the case of an incoming CER, the callback can return a result code with which to answer. Multiple callbacks can be specified and are applied until either all return 'ok' or one doesn't.

This provides a way to reject a peer connection.

Own Id: OTP-9654

Add @codecs to dictionary format.

The semantics are similar to @custom_types but results in codec functions of the form TypeName(encode|decode, AvpName, Data) rather than AvpName(encode|decode, TypeName, Data). That is, the role of the AVP name and Diameter type name are reversed. This eliminates the need for exporting one function for each AVP sharing a common specialized encode/decode.

Own Id: OTP-9708 Aux Id: OTP-9639

Add #diameter_callback{} for more flexible callback configuration.

The record allows individual functions to be configured for each of the diameter_app(3) callbacks, as well as a default callback.

Own Id: OTP-9777

Handle #sctp_paddr_change and #sctp_pdapi_event from gen_sctp.

The events are enabled by default but diameter_sctp neither disabled nor dealt with them. Reception of such an event caused a transport process to crash.

Own Id: OTP-9538

Fix header folding bug.

A prepare_request callback from diameter can return a diameter_header record in order to set values in the header of an outgoing request. A fault in diameter_lib:fold_tuple/3 caused the subsequent encode of the outgoing request to fail.

Own Id: OTP-9577

Fix bugs in sending of answer-message replies.

3001 (DIAMETER_COMMAND_UNSUPPORTED) was not sent since the decode placed the AVP list in the wrong field of the diameter_packet, causing the subsequent encode to fail. Session-Id was also set improperly, causing encode to fail even in this case.

Own Id: OTP-9578

Fix improper use of error_logger:info_report/2.

Function doesn't take a format string and arguments as it was called. Instead use error_logger:info_report/1 and use the same report format as used for warning and error reports.

Own Id: OTP-9579

Fix and clarify semantics of peer filters.

An eval filter returning a non-true value caused the call process to fail and the doc was vague on how an exception was treated. Clarify that the non-tuple host/realm filters assume messages of a certain form.

Own Id: OTP-9580

Fix and clarify relay behaviour.

Implicit filtering of the sending peer in relaying a request could cause loop detection to be preempted in a manner not specified by RFC3588. Reply with 3002 (DIAMETER_UNABLE_TO_DELIVER) on anything but an answer to a relayed request.

Own Id: OTP-9583

@id required in dictionary files only when @messages is specified.

@id defines an application identifier and this is used only when sending or receiving messages. A dictionary can define only AVP's however, to be included by other dictionaries using @inherits, in which case it makes no sense to require @id.

Note that message definitions are not inherited with @inherits, only AVP's

Own Id: OTP-9467

Allow @enum when AVP is defined in an inherited dictionary.

3GPP standards (for one) extend the values allowed for RFC 3588 AVP's of type Enumerated. Previously, extending an AVP was only possible by completely redefining the AVP.

Own Id: OTP-9469

Migrate testsuites to pure common test and add both suites and testcases.

Own Id: OTP-9553

Requests of arbitrary form.

diameter:call/4 can be passed anything, as long as the subsequent prepare_request callback returns a term that can be encoded.

Own Id: OTP-9581