Conversion of (agent) Audit Trail Log (ATL) failed due to invalid log entries.

The conversion aborted completely midway because the ATL contained invalid entries. The conversion has been improved so that it now firstly handles encountered errors and write an informative message (into the converted stream) and secondly keeps count of the number of successful or failed entry conversions. See log_to_txt for more info.

The reason the ATL contained invalid entries have also been fixed. The reason was that for some outgoing messages (not response):

Own Id: OTP-15287 Aux Id: ERIERL-206

[compiler] Spurious version message removed. The snmp mib compiler printed an spurious version message if the 'version' option was provided.

Own Id: OTP-15290

[snmp|agent] Add a get-mechanism callback module (and a corresponding behaviour). The agent calls this module to handle each get (get, get-next and get-bulk) request.

Own Id: OTP-15691 Aux Id: ERIERL-324

The Snmp MIB compiler now allows using a TEXTUAL-CONVENTION type before defining it.

Own Id: OTP-14196 Aux Id: ERIERL-161

The example MIB EX1-MIB in the SNMP application has been corrected to match its example.

Own Id: OTP-14204 Aux Id: PR-1726

Removed all old unused files in the documentation.

Own Id: OTP-14475 Aux Id: ERL-409, PR-1493

The recbuf configuration option was not propagated correctly to the socket for the SNMP Manager.

Own Id: OTP-13372 Aux Id: ERIERL-73

A bug in the SNMP MIB compiler has been fixed. An AUGMENTS referring to a table defined later in the MIB did not work.

Own Id: OTP-13014 Aux Id: ERL-375

Internal code change: Calls to catch followed by a call to erlang:get_stacktrace/0 has been rewritten to use try instead of catch to make the code future-proof.

Own Id: OTP-14400

The SNMP MIB compiler has been fixed to compile MIBS with refinements on user types such as in RFC 4669 RADIUS-AUTH-SERVER-MIB.mib. Problem reported and researched by Kenneth Lakin and Daniel Goertzen.

See also: https://bugs.erlang.org/browse/ERL-325

Own Id: OTP-14145 Aux Id: ERL-325

Correct bugs when path to mib or idl spec files contains UTF-8 characters.

Own Id: OTP-13718 Aux Id: ERL-179

Solves snmp config string handling as reported by ERL-164 and solved by PR-1100

Own Id: OTP-13706

Internal changes

Own Id: OTP-13551

Snmp agent now properly handles vacmViewTreeFamily masks.

Own Id: OTP-13264

Small documentation fixes

Own Id: OTP-13017

Update configuration check of imask ( list of ones and zeros) to allow the empty list.

Own Id: OTP-13101

The runtime dependencies in the application resource file have been updated.

Own Id: OTP-12762

A bug in the SNMP Agent has been corrected; when opening a port using the command line argument -snmpa_fd the Port should be 0 when calling gen_udp:open.

A bug in the SNMP manager has been corrected; it should not look at the -snmp_fd command line argument, but instead at -snmpm_fd.

Own Id: OTP-12669 Aux Id: seq12841

Improved cryptocraphic capability.

Own Id: OTP-12452

[compiler] Refinement of type Opaque was not allowed.

MIB constructs such as 'SYNTAX Opaque (SIZE(0..65535))' was previously not allowed, see the standard ALARM-MIB for eaxmple.

Own Id: OTP-12066

Aux Id: Seq 12669

The SNMP manager has been enhanced with dual stack IPv4+IPv6, as the agent just was. The documentation is also now updated for both the agent and the manager.

Own Id: OTP-12108 Aux Id: OTP-12020

SNMP has been improved to handle IPv6. The agent can handle dual stack IPv4 + IPv6, but not yet the manager. The documentation also still lags behind... If you do such advanced stuff like writing a custom net_if module, the interface for it has changed, but other than that SNMP is backwards compatible.

*** POTENTIAL INCOMPATIBILITY ***

Own Id: OTP-12020 Aux Id: OTP-11518

Application upgrade (appup) files are corrected for the following applications:

asn1, common_test, compiler, crypto, debugger, dialyzer, edoc, eldap, erl_docgen, et, eunit, gs, hipe, inets, observer, odbc, os_mon, otp_mibs, parsetools, percept, public_key, reltool, runtime_tools, ssh, syntax_tools, test_server, tools, typer, webtool, wx, xmerl

A new test utility for testing appup files is added to test_server. This is now used by most applications in OTP.

(Thanks to Tobias Schlager)

Own Id: OTP-11744

Updated doc files to utf8.

Own Id: OTP-10907

Fixed test suite to support UTF-8 paths.

Own Id: OTP-10877

[agent] Enable SNMP to create missing database directories.

Add {db_init_error, create_db_and_dir} option to SNMP manager and agent. This allows them to create any missing parent directories for db_dir, rather than treating any missing directories as a fatal error. The default for db_init_error, which is terminate, is unchanged.

Steve Vinoski

Own Id: OTP-11352

[manager] Improved handling of unexpected return values from snmpm_user callback functions.

Violations of the documented API (crashes or invalid return values) will now result in an error message.

Own Id: OTP-11307

Add (atl) log conversion block option.

It is now possible to request that the Audit Trail Log should be blocked during conversion (log_to_txt or log_to_io). This could be usefull when coverting an entire large log (when there is a chance it may otherwise wrap during conversion).

See agent log_to_txt and log_to_io and also manager log_to_txt and log_to_io for details.

Own Id: OTP-11396

Own Id: seq12433

When converting an Audit Trail Log to text, a corrupt log entry could cause the entire conversion to fail.

Also, for a log with sequence numbers, failing to decode a log entry would cause the conversion to fail (not because of the failed decode, but because of the failure to write the error message).

Own Id: OTP-111453

Aux Id: Seq 12459

Wrong block cypher type used for AES ('aes_cbf128' instead of 'aes_cfb128') when performing AES block encrypt/decrypt which breaks SNMP usmAesCfb128Protocol in agent and manager.

Own Id: OTP-11412

[manager] When performing the AES encryption, invalid values for the EngineBoots and EngineTime was used.

The values of the local agent was used, which would have produced "some" values if an agent was actually running. If not it would have caused a crash.

Own Id: OTP-11413

[agent] Improved documentation for the functions for loading and unloading mibs, see load_mibs and unload_mibs for more info.

Also added new functions for loading and unloading a single mib, see load_mib and unload_mib for more info.

Own Id: OTP-11216

[agent] Reading the value of the vacmViewTreeFamilyMask returns it in the wrong (internal bitlist) format.

The vacmViewTreeFamilyMask is defined as a bit string in the MIB (OCTET STRING). Internally a bitlist (list of 1's and 0's, see vacm config file for more info) is used. However, the MIB implementation assumed the latter, effectively rendering all attempts to read/set masks via SNMP unsuccessful.

Since the mask is used in hot paths (e.g. access permission checks for each SNMP operation, the bitlist representation of the mask has benefits (e.g. faster processing). Reading/writing the view mask objects is less time-critical. Therefore, to fix the issue, convert between the bitlist (internal) representation and bitstring (external) when the vacmViewTreeFamilyMask objects are accessed.

Also, the check of the vacm config file was invalid with regard to the mask value. It was assumed to be a proper oid, which is not strictly the case (see bitlist above).

Own Id: OTP-11177

Stefan Zegenhagen

[agent] The counter increment function in the local-db was incorrect. It did not handle counter wrap correctly.

Own Id: OTP-11192

[agent,manager] Updated to support the new crypto interface.

Own Id: OTP-11009

[agent] Introduced a documented behaviour for the mib-server mib-data backend. At present only the default module (snmpa_mib_data_tttn) is provided.

A config option for the (agent) mib-servers mib-data backend module has been added to the agent config options, data_module.

Own Id: OTP-11101

[agent] Introduced a documented behaviour for the mib storage. At present there are three simple modules (snmpa_mib_storage_ets, snmpa_mib_storage_dets and snmpa_mib_storage_mnesia) implementing this behaviour, provided with the app.

A config option for the (agent) mib storage has been added to the agent config options.

Own Id: OTP-11107

[compiler] Now handles MIBs importing the pesudotype BITS.

Own Id: OTP-10799

[compiler] The MIB compiler could not handle a table index that was defined later in the MIB.

Own Id: OTP-10808

[manager] Polish return values of snmpm_user_default according to snmpm_user doc.

Luca Favatella

Own Id: OTP-10671

[agent] Remove runtime warning in snmpa_agent because of tuple fun usage.

Luca Favatella

Own Id: OTP-10672

[manager] SNMP manager performance optimization.

Ivan Dubovik

Own Id: OTP-10673

[manager] The old Addr-and-Port based API functions, previously long deprecated and marked for deletion in R16B, has now been removed.

Own Id: OTP-10027

[agent] Sematic fixes to SNMP-USER-BASED-SM-MIB. The semantics allow the usmUserAuthKeyChange and usmUserPrivKeyChange objects to be written to in the same set requests that also creates and clones the user. This was not possible beforehand, causing test tools checking semantic SNMPv3 behaviour to fail on a lot of test cases.

Furthermore, once the user has been cloned by writing to an instance of usmUserCloneFrom, further set-operations to the same object will not return an error, but be no-ops. Especially, it must be avoided to copy security parameters again (possibly even from a different user).

Stefan Zegenhagen

Own Id: OTP-10166

[agent] Errors in vacmAccessTable RowStatus handling. There are problems with the handling of vacmAccessTableStatus that cause some SNMP test suites to report errors. Most notably, erroneous set operations frequently cause "genErr" errors to be returned. These "genErr" errors are usually caused by badmatch exceptions coming from {ok, Row} = snmpa_vacm:get_row(RowIndex) if the row does not exist.

The semantics of the RowStatus handling in that table has been adjusted to be compliant with the RowStatus textual description of SNPMv2-TC MIB.

Stefan Zegenhagen

Own Id: OTP-10164

[agent] Fix walk over vacmAccessTable. Fix the get_next implementation of vacmAccessTable to return all table entries.

The get_next implementation of vacmAccessTable did not return all available table data. Instead, it only returned the first column for each row, and all columns for the last row available.

Stefan Zegenhagen

Own Id: OTP-10165

[manager] snmpm:log_to_io/6 did not use the LogName argument.

Own Id: OTP-10066

Incorrect TimeTicks decode. Also bad handling of invalid encode (value outside of value range) for both TimeTicks and Unsigned32.

Own Id: OTP-10132

[compiler] The table information the MIB compiler provides with augmented tables has been extended with nbr_of_cols, first_accessible and not_accessible.

Own Id: OTP-9969

Added the log_to_io audit-trail-log converter function to the api modules of both the manager and agent.

Own Id: OTP-9940

[manager] Introduced a new transport module, snmpm_net_if_mt, which handles all incomming and outgoing traffic in newly created processes. The message/request is processed and then the process exits.

Own Id: OTP-9876

[agent] Documenting previously existing but undocumented function, snmp_generic:get_table_info/2.

Own Id: OTP-9942

[agent] Improve error handling while reading agent config files. Some files contain mandatory information and is therefor themself mandatory.

Own Id: OTP-9943

[agent] Simultaneous snmpa:backup/1,2 calls can interfere. The master agent did not check if a backup was already in progress when a backup request was accepted.

Own Id: OTP-9884

Aux Id: Seq 11995

[agent] DoS attack using GET-BULK with large value of MaxRepetitions. A preventive method has been implementing by simply limit the number of varbinds that can be included in a Get-BULK response message. This is specified by the new config option, gb_max_vbs.

Own Id: OTP-9700

[agent] Mib server cache gclimit update function incorrectly calls age update function. The gclimit update function, update_mibs_cache_gclimit/1, incorrectly called the age update function, update_mibs_cache_age/2.

Johan Claesson

Own Id: OTP-9868

[agent] Removed (more) use of old style tuple funs.

Own Id: OTP-9783

[agent] Repeated vacm table dumping fails due to file name conflict. When dumping the vacm table to disk, a temoporary file with a fixed name was used. If the table dumping (snmpa_vacm:dump_table/0) was initiated from several different processes in rapid succesion, the dumping could fail because the different processes was simultaniously trying to write to the same file. This problem has been eliminated by creating a unique name for the temporary file.

Own Id: OTP-9851

Aux Id: Seq 11980

[agent] Removed use of old style tuple funs.

Own Id: OTP-9779

[compiler] Improved version info printout from the MIB compiler frontend escript.

Own Id: OTP-9618

[agent] Version 4.20 introduced a change that broke trap sending from subagents. Due to a bug in the test code, this was not discovered, until that bug was fixed.

Own Id: OTP-9745

[agent] When sending an error message (reply) regarding snmpUnknownPDUHandlers, the agent used the wrong OID.

Own Id: OTP-9747

[compiler] Fix the --warnings/--W option parsing in the snmpc wrapper (e)script. The short warning option was incorrectly --w, instead of as documented --W. This has now been corrected.

*** POTENTIAL INCOMPATIBILITY ***

Tuncer Ayaz

Own Id: OTP-9718

[compiler] The short warning option has been changed from --w to --W to comply with the documentation.

Tuncer Ayaz

Own Id: OTP-9718

Bad note store GC timer deactivation. Wrong field in the state record was set (timeout instead active).

Stefan Grundmann

Own Id: OTP-9690

[compiler] Used wrong variable name (for warnings-as-errors variable), which caused the compiler to crash when using the snmpc (e)script.

Also added the option --Werror for the SNMP MIB compiler (escript) frontend (to mimic erlc), which specifies whether warnings should be treated as errors.

Own Id: OTP-9447

[agent] Some very minor debugging improvements.

Own Id: OTP-9446

[manager] There was no way to specify transport domain. The transport domains was assumed to be IPv4 (transportDomainUdpIpv4). This has now been changed so that it can also be IPv6 (transportDomainUdpIpv6). To facilitate this, the transport domain, tdomain, is now a (new) valid option when registering a new agent (and updating agent info).

This also mean that the transport behaviour has changed.

Own Id: OTP-9305

Aux Id: Seq 11847

[compiler] Added the option warnings_as_errors (for the SNMP MIB compiler (escript) frontend, the option --wae is used) which specifies whether warnings should be treated as errors.

Tuncer Ayaz

Own Id: OTP-9437

The snmp config tool could not handle (manager) audit trail config because the option seqno was not handled.

Own Id: OTP-9354

[agent] The SNMP ACM cache was not properly updated when changes where made to the VACM security-to-group, access and view-tree-family tables.

Own Id: OTP-9367

Aux Id: Seq 11858

Fixed install directory typo for man3.

Peter Lemenkov

Hans Ulrich Niedermann

Own Id: OTP-9442

[agent] Did not handle transport domains properly in some cases, for instance trap sending.

Own Id: OTP-9400

[agent] Wrong default transport domain, snmpUDPDomain, instead of transportDomainUdpIpv4.

Own Id: OTP-9425

Aux Id: Seq 11874

[agent] Added support for sending traps to IPv6 targets.

See the target address config file, the target_addr_entry/11 function or add_addr/11 for more info.

Own Id: OTP-9088

Aux Id: Seq 11790

[agent] To be able to handle multiple engine-id(s) when sending trap(s), the function add_community/6 has been added.

Own Id: OTP-9119

Aux Id: Seq 11792

[manager] The API for snmp requests has been augmented to allow the caller to override some configuration.

This has been done by introducing a new set of API functions, see sync_get2/3,4, async_get2/3,4, sync_get_next2/3,4, async_get_next2/3,4, sync_get_bulk2/5,6, async_get_bulk2/5,6, sync_set2/3,4 and async_set2/3,4 for more info.

Own Id: OTP-9162

[manager] The old API functions (for get and set requests: snmpm:g/3,4,5,6,7, snmpm:ag/3,4,5,6,7, snmpm:gn/3,4,5,6,7, snmpm:agn/3,4,5,6,7, snmpm:s/3,4,5,6,7, snmpm:s/3,4,5,6,7, snmpm:gb/5,6,7,8,9 and snmpm:agb/5,6,7,8,9) are now officially deprecated. They will be removed as of R16B.

Own Id: OTP-9174

[agent] Pass extra info through the agent to the net-if process when sending notifications.

See snmpa:send_notification2/3 for more info. See also the incomming net-if messages when sending a trap (send_pdu message) and notification (send_pdu_req message).

Own Id: OTP-9183

Aux Id: Seq 11817

Added type specs for functions that do not return.

Kostis Sagonas

Own Id: OTP-9208

Fixed endode/decode of values of type Counter32.

This type (Counter32) is an unsigned integer 32, but is actually encoded as an signed integer 32. The encode/decode functions however, treated it as if it was encodeded as an unsigned integer 32.

Own Id: OTP-9022