diff options
331 files changed, 2667 insertions, 1868 deletions
diff --git a/OTP_VERSION b/OTP_VERSION index 07256210c7..ab1e4f6a31 100644 --- a/OTP_VERSION +++ b/OTP_VERSION @@ -1 +1 @@ -20.0-rc1 +20.0-rc2 diff --git a/bootstrap/bin/start.boot b/bootstrap/bin/start.boot Binary files differindex 6a42dfe423..61a255e086 100644 --- a/bootstrap/bin/start.boot +++ b/bootstrap/bin/start.boot diff --git a/bootstrap/bin/start_clean.boot b/bootstrap/bin/start_clean.boot Binary files differindex 6a42dfe423..61a255e086 100644 --- a/bootstrap/bin/start_clean.boot +++ b/bootstrap/bin/start_clean.boot diff --git a/bootstrap/lib/compiler/ebin/beam_a.beam b/bootstrap/lib/compiler/ebin/beam_a.beam Binary files differindex bc936a29f4..1459567485 100644 --- a/bootstrap/lib/compiler/ebin/beam_a.beam +++ b/bootstrap/lib/compiler/ebin/beam_a.beam diff --git a/bootstrap/lib/compiler/ebin/beam_asm.beam b/bootstrap/lib/compiler/ebin/beam_asm.beam Binary files differindex bdd4b00b3f..7335dbab61 100644 --- a/bootstrap/lib/compiler/ebin/beam_asm.beam +++ b/bootstrap/lib/compiler/ebin/beam_asm.beam diff --git a/bootstrap/lib/compiler/ebin/beam_block.beam b/bootstrap/lib/compiler/ebin/beam_block.beam Binary files differindex 3c286dffbf..fa708d60a0 100644 --- a/bootstrap/lib/compiler/ebin/beam_block.beam +++ b/bootstrap/lib/compiler/ebin/beam_block.beam diff --git a/bootstrap/lib/compiler/ebin/beam_bs.beam b/bootstrap/lib/compiler/ebin/beam_bs.beam Binary files differindex 7840bbdf47..61b1b9b055 100644 --- a/bootstrap/lib/compiler/ebin/beam_bs.beam +++ b/bootstrap/lib/compiler/ebin/beam_bs.beam diff --git a/bootstrap/lib/compiler/ebin/beam_bsm.beam b/bootstrap/lib/compiler/ebin/beam_bsm.beam Binary files differindex 568c2f3ec4..4b4412532a 100644 --- a/bootstrap/lib/compiler/ebin/beam_bsm.beam +++ b/bootstrap/lib/compiler/ebin/beam_bsm.beam diff --git a/bootstrap/lib/compiler/ebin/beam_clean.beam b/bootstrap/lib/compiler/ebin/beam_clean.beam Binary files differindex 178f660d7a..8b97d6e227 100644 --- a/bootstrap/lib/compiler/ebin/beam_clean.beam +++ b/bootstrap/lib/compiler/ebin/beam_clean.beam diff --git a/bootstrap/lib/compiler/ebin/beam_dead.beam b/bootstrap/lib/compiler/ebin/beam_dead.beam Binary files differindex 605607445f..6960217ac0 100644 --- a/bootstrap/lib/compiler/ebin/beam_dead.beam +++ b/bootstrap/lib/compiler/ebin/beam_dead.beam diff --git a/bootstrap/lib/compiler/ebin/beam_dict.beam b/bootstrap/lib/compiler/ebin/beam_dict.beam Binary files differindex 0eafd9c839..b8cc501dd6 100644 --- a/bootstrap/lib/compiler/ebin/beam_dict.beam +++ b/bootstrap/lib/compiler/ebin/beam_dict.beam diff --git a/bootstrap/lib/compiler/ebin/beam_disasm.beam b/bootstrap/lib/compiler/ebin/beam_disasm.beam Binary files differindex 1028fd7e6c..d68a8b7fc5 100644 --- a/bootstrap/lib/compiler/ebin/beam_disasm.beam +++ b/bootstrap/lib/compiler/ebin/beam_disasm.beam diff --git a/bootstrap/lib/compiler/ebin/beam_except.beam b/bootstrap/lib/compiler/ebin/beam_except.beam Binary files differindex 4cdbb91dd5..8987254a04 100644 --- a/bootstrap/lib/compiler/ebin/beam_except.beam +++ b/bootstrap/lib/compiler/ebin/beam_except.beam diff --git a/bootstrap/lib/compiler/ebin/beam_flatten.beam b/bootstrap/lib/compiler/ebin/beam_flatten.beam Binary files differindex 2db71eafdf..5c019f7ed1 100644 --- a/bootstrap/lib/compiler/ebin/beam_flatten.beam +++ b/bootstrap/lib/compiler/ebin/beam_flatten.beam diff --git a/bootstrap/lib/compiler/ebin/beam_jump.beam b/bootstrap/lib/compiler/ebin/beam_jump.beam Binary files differindex d92c009533..51761fff35 100644 --- a/bootstrap/lib/compiler/ebin/beam_jump.beam +++ b/bootstrap/lib/compiler/ebin/beam_jump.beam diff --git a/bootstrap/lib/compiler/ebin/beam_listing.beam b/bootstrap/lib/compiler/ebin/beam_listing.beam Binary files differindex bd4a5760f3..19af25e71a 100644 --- a/bootstrap/lib/compiler/ebin/beam_listing.beam +++ b/bootstrap/lib/compiler/ebin/beam_listing.beam diff --git a/bootstrap/lib/compiler/ebin/beam_opcodes.beam b/bootstrap/lib/compiler/ebin/beam_opcodes.beam Binary files differindex 9c7029898e..f8d1e43f1d 100644 --- a/bootstrap/lib/compiler/ebin/beam_opcodes.beam +++ b/bootstrap/lib/compiler/ebin/beam_opcodes.beam diff --git a/bootstrap/lib/compiler/ebin/beam_peep.beam b/bootstrap/lib/compiler/ebin/beam_peep.beam Binary files differindex 892d8a67c5..0f8397da3e 100644 --- a/bootstrap/lib/compiler/ebin/beam_peep.beam +++ b/bootstrap/lib/compiler/ebin/beam_peep.beam diff --git a/bootstrap/lib/compiler/ebin/beam_receive.beam b/bootstrap/lib/compiler/ebin/beam_receive.beam Binary files differindex 9b6ee8692e..1d4a96e02d 100644 --- a/bootstrap/lib/compiler/ebin/beam_receive.beam +++ b/bootstrap/lib/compiler/ebin/beam_receive.beam diff --git a/bootstrap/lib/compiler/ebin/beam_record.beam b/bootstrap/lib/compiler/ebin/beam_record.beam Binary files differindex 96dbaa4338..1a5127146c 100644 --- a/bootstrap/lib/compiler/ebin/beam_record.beam +++ b/bootstrap/lib/compiler/ebin/beam_record.beam diff --git a/bootstrap/lib/compiler/ebin/beam_reorder.beam b/bootstrap/lib/compiler/ebin/beam_reorder.beam Binary files differindex 7500c0f725..ba3759c2d9 100644 --- a/bootstrap/lib/compiler/ebin/beam_reorder.beam +++ b/bootstrap/lib/compiler/ebin/beam_reorder.beam diff --git a/bootstrap/lib/compiler/ebin/beam_split.beam b/bootstrap/lib/compiler/ebin/beam_split.beam Binary files differindex def7cd3314..c26800e0ab 100644 --- a/bootstrap/lib/compiler/ebin/beam_split.beam +++ b/bootstrap/lib/compiler/ebin/beam_split.beam diff --git a/bootstrap/lib/compiler/ebin/beam_trim.beam b/bootstrap/lib/compiler/ebin/beam_trim.beam Binary files differindex 31e2a2c1c2..6f9725b7c1 100644 --- a/bootstrap/lib/compiler/ebin/beam_trim.beam +++ b/bootstrap/lib/compiler/ebin/beam_trim.beam diff --git a/bootstrap/lib/compiler/ebin/beam_type.beam b/bootstrap/lib/compiler/ebin/beam_type.beam Binary files differindex 76a2a176f7..e2b26e648a 100644 --- a/bootstrap/lib/compiler/ebin/beam_type.beam +++ b/bootstrap/lib/compiler/ebin/beam_type.beam diff --git a/bootstrap/lib/compiler/ebin/beam_utils.beam b/bootstrap/lib/compiler/ebin/beam_utils.beam Binary files differindex 1d88c6a6e7..88e8398c31 100644 --- a/bootstrap/lib/compiler/ebin/beam_utils.beam +++ b/bootstrap/lib/compiler/ebin/beam_utils.beam diff --git a/bootstrap/lib/compiler/ebin/beam_validator.beam b/bootstrap/lib/compiler/ebin/beam_validator.beam Binary files differindex 97c24118b1..28fbf7ea01 100644 --- a/bootstrap/lib/compiler/ebin/beam_validator.beam +++ b/bootstrap/lib/compiler/ebin/beam_validator.beam diff --git a/bootstrap/lib/compiler/ebin/beam_z.beam b/bootstrap/lib/compiler/ebin/beam_z.beam Binary files differindex 6a95f140f1..b55e3c2383 100644 --- a/bootstrap/lib/compiler/ebin/beam_z.beam +++ b/bootstrap/lib/compiler/ebin/beam_z.beam diff --git a/bootstrap/lib/compiler/ebin/cerl.beam b/bootstrap/lib/compiler/ebin/cerl.beam Binary files differindex b9f10145c0..d66a51d75e 100644 --- a/bootstrap/lib/compiler/ebin/cerl.beam +++ b/bootstrap/lib/compiler/ebin/cerl.beam diff --git a/bootstrap/lib/compiler/ebin/cerl_clauses.beam b/bootstrap/lib/compiler/ebin/cerl_clauses.beam Binary files differindex 4c2e4e7f97..3d7e9e4d7a 100644 --- a/bootstrap/lib/compiler/ebin/cerl_clauses.beam +++ b/bootstrap/lib/compiler/ebin/cerl_clauses.beam diff --git a/bootstrap/lib/compiler/ebin/cerl_inline.beam b/bootstrap/lib/compiler/ebin/cerl_inline.beam Binary files differindex 605f17f057..a703e1759b 100644 --- a/bootstrap/lib/compiler/ebin/cerl_inline.beam +++ b/bootstrap/lib/compiler/ebin/cerl_inline.beam diff --git a/bootstrap/lib/compiler/ebin/cerl_sets.beam b/bootstrap/lib/compiler/ebin/cerl_sets.beam Binary files differindex 72bcb3527a..b5ec77d84b 100644 --- a/bootstrap/lib/compiler/ebin/cerl_sets.beam +++ b/bootstrap/lib/compiler/ebin/cerl_sets.beam diff --git a/bootstrap/lib/compiler/ebin/cerl_trees.beam b/bootstrap/lib/compiler/ebin/cerl_trees.beam Binary files differindex 6193ea155f..47f46fa679 100644 --- a/bootstrap/lib/compiler/ebin/cerl_trees.beam +++ b/bootstrap/lib/compiler/ebin/cerl_trees.beam diff --git a/bootstrap/lib/compiler/ebin/compile.beam b/bootstrap/lib/compiler/ebin/compile.beam Binary files differindex 65809b44de..e5ed9dfd3e 100644 --- a/bootstrap/lib/compiler/ebin/compile.beam +++ b/bootstrap/lib/compiler/ebin/compile.beam diff --git a/bootstrap/lib/compiler/ebin/core_lib.beam b/bootstrap/lib/compiler/ebin/core_lib.beam Binary files differindex eaca62b945..ba2f8ff573 100644 --- a/bootstrap/lib/compiler/ebin/core_lib.beam +++ b/bootstrap/lib/compiler/ebin/core_lib.beam diff --git a/bootstrap/lib/compiler/ebin/core_lint.beam b/bootstrap/lib/compiler/ebin/core_lint.beam Binary files differindex 5ed8c04322..fa31a6555f 100644 --- a/bootstrap/lib/compiler/ebin/core_lint.beam +++ b/bootstrap/lib/compiler/ebin/core_lint.beam diff --git a/bootstrap/lib/compiler/ebin/core_parse.beam b/bootstrap/lib/compiler/ebin/core_parse.beam Binary files differindex 0555025360..07abc0dd36 100644 --- a/bootstrap/lib/compiler/ebin/core_parse.beam +++ b/bootstrap/lib/compiler/ebin/core_parse.beam diff --git a/bootstrap/lib/compiler/ebin/core_pp.beam b/bootstrap/lib/compiler/ebin/core_pp.beam Binary files differindex 0513126ae0..a7cf3140ff 100644 --- a/bootstrap/lib/compiler/ebin/core_pp.beam +++ b/bootstrap/lib/compiler/ebin/core_pp.beam diff --git a/bootstrap/lib/compiler/ebin/core_scan.beam b/bootstrap/lib/compiler/ebin/core_scan.beam Binary files differindex 1f3d88e797..2ce3a55b85 100644 --- a/bootstrap/lib/compiler/ebin/core_scan.beam +++ b/bootstrap/lib/compiler/ebin/core_scan.beam diff --git a/bootstrap/lib/compiler/ebin/erl_bifs.beam b/bootstrap/lib/compiler/ebin/erl_bifs.beam Binary files differindex a385352783..e027c6a303 100644 --- a/bootstrap/lib/compiler/ebin/erl_bifs.beam +++ b/bootstrap/lib/compiler/ebin/erl_bifs.beam diff --git a/bootstrap/lib/compiler/ebin/rec_env.beam b/bootstrap/lib/compiler/ebin/rec_env.beam Binary files differindex 1830d079b4..2f6b4f8af4 100644 --- a/bootstrap/lib/compiler/ebin/rec_env.beam +++ b/bootstrap/lib/compiler/ebin/rec_env.beam diff --git a/bootstrap/lib/compiler/ebin/sys_core_dsetel.beam b/bootstrap/lib/compiler/ebin/sys_core_dsetel.beam Binary files differindex 6e9bd6255c..59c470fbae 100644 --- a/bootstrap/lib/compiler/ebin/sys_core_dsetel.beam +++ b/bootstrap/lib/compiler/ebin/sys_core_dsetel.beam diff --git a/bootstrap/lib/compiler/ebin/sys_core_fold.beam b/bootstrap/lib/compiler/ebin/sys_core_fold.beam Binary files differindex 90296baf60..98be753e31 100644 --- a/bootstrap/lib/compiler/ebin/sys_core_fold.beam +++ b/bootstrap/lib/compiler/ebin/sys_core_fold.beam diff --git a/bootstrap/lib/compiler/ebin/sys_core_fold_lists.beam b/bootstrap/lib/compiler/ebin/sys_core_fold_lists.beam Binary files differindex 373debfc27..ba5e540511 100644 --- a/bootstrap/lib/compiler/ebin/sys_core_fold_lists.beam +++ b/bootstrap/lib/compiler/ebin/sys_core_fold_lists.beam diff --git a/bootstrap/lib/compiler/ebin/sys_core_inline.beam b/bootstrap/lib/compiler/ebin/sys_core_inline.beam Binary files differindex 92d636005d..714e073b9f 100644 --- a/bootstrap/lib/compiler/ebin/sys_core_inline.beam +++ b/bootstrap/lib/compiler/ebin/sys_core_inline.beam diff --git a/bootstrap/lib/compiler/ebin/sys_pre_attributes.beam b/bootstrap/lib/compiler/ebin/sys_pre_attributes.beam Binary files differindex 52391ec2c9..c418856bad 100644 --- a/bootstrap/lib/compiler/ebin/sys_pre_attributes.beam +++ b/bootstrap/lib/compiler/ebin/sys_pre_attributes.beam diff --git a/bootstrap/lib/compiler/ebin/v3_codegen.beam b/bootstrap/lib/compiler/ebin/v3_codegen.beam Binary files differindex df54c8cf70..a40e6f9447 100644 --- a/bootstrap/lib/compiler/ebin/v3_codegen.beam +++ b/bootstrap/lib/compiler/ebin/v3_codegen.beam diff --git a/bootstrap/lib/compiler/ebin/v3_core.beam b/bootstrap/lib/compiler/ebin/v3_core.beam Binary files differindex 73a5a6c48b..eb0865231a 100644 --- a/bootstrap/lib/compiler/ebin/v3_core.beam +++ b/bootstrap/lib/compiler/ebin/v3_core.beam diff --git a/bootstrap/lib/compiler/ebin/v3_kernel.beam b/bootstrap/lib/compiler/ebin/v3_kernel.beam Binary files differindex fac8b05528..cebb49597c 100644 --- a/bootstrap/lib/compiler/ebin/v3_kernel.beam +++ b/bootstrap/lib/compiler/ebin/v3_kernel.beam diff --git a/bootstrap/lib/compiler/ebin/v3_kernel_pp.beam b/bootstrap/lib/compiler/ebin/v3_kernel_pp.beam Binary files differindex 54af18ed4c..331e6ba032 100644 --- a/bootstrap/lib/compiler/ebin/v3_kernel_pp.beam +++ b/bootstrap/lib/compiler/ebin/v3_kernel_pp.beam diff --git a/bootstrap/lib/compiler/ebin/v3_life.beam b/bootstrap/lib/compiler/ebin/v3_life.beam Binary files differindex cbfc1f4f82..23a894f62a 100644 --- a/bootstrap/lib/compiler/ebin/v3_life.beam +++ b/bootstrap/lib/compiler/ebin/v3_life.beam diff --git a/bootstrap/lib/kernel/ebin/application.beam b/bootstrap/lib/kernel/ebin/application.beam Binary files differindex 8746279ae8..d1d381e09b 100644 --- a/bootstrap/lib/kernel/ebin/application.beam +++ b/bootstrap/lib/kernel/ebin/application.beam diff --git a/bootstrap/lib/kernel/ebin/application_controller.beam b/bootstrap/lib/kernel/ebin/application_controller.beam Binary files differindex cccab712e7..b3674c2cd5 100644 --- a/bootstrap/lib/kernel/ebin/application_controller.beam +++ b/bootstrap/lib/kernel/ebin/application_controller.beam diff --git a/bootstrap/lib/kernel/ebin/application_master.beam b/bootstrap/lib/kernel/ebin/application_master.beam Binary files differindex 94b902f08b..6253ba71e4 100644 --- a/bootstrap/lib/kernel/ebin/application_master.beam +++ b/bootstrap/lib/kernel/ebin/application_master.beam diff --git a/bootstrap/lib/kernel/ebin/application_starter.beam b/bootstrap/lib/kernel/ebin/application_starter.beam Binary files differindex 996accedea..851951db38 100644 --- a/bootstrap/lib/kernel/ebin/application_starter.beam +++ b/bootstrap/lib/kernel/ebin/application_starter.beam diff --git a/bootstrap/lib/kernel/ebin/auth.beam b/bootstrap/lib/kernel/ebin/auth.beam Binary files differindex 0daa7ca83a..4d7652d9a9 100644 --- a/bootstrap/lib/kernel/ebin/auth.beam +++ b/bootstrap/lib/kernel/ebin/auth.beam diff --git a/bootstrap/lib/kernel/ebin/code.beam b/bootstrap/lib/kernel/ebin/code.beam Binary files differindex f197c8a5b5..026dccb205 100644 --- a/bootstrap/lib/kernel/ebin/code.beam +++ b/bootstrap/lib/kernel/ebin/code.beam diff --git a/bootstrap/lib/kernel/ebin/code_server.beam b/bootstrap/lib/kernel/ebin/code_server.beam Binary files differindex b6e8faf492..26bfba1e62 100644 --- a/bootstrap/lib/kernel/ebin/code_server.beam +++ b/bootstrap/lib/kernel/ebin/code_server.beam diff --git a/bootstrap/lib/kernel/ebin/disk_log.beam b/bootstrap/lib/kernel/ebin/disk_log.beam Binary files differindex 1a15e400c5..1bf6220227 100644 --- a/bootstrap/lib/kernel/ebin/disk_log.beam +++ b/bootstrap/lib/kernel/ebin/disk_log.beam diff --git a/bootstrap/lib/kernel/ebin/disk_log_1.beam b/bootstrap/lib/kernel/ebin/disk_log_1.beam Binary files differindex 5efe11b918..3a281e9f5a 100644 --- a/bootstrap/lib/kernel/ebin/disk_log_1.beam +++ b/bootstrap/lib/kernel/ebin/disk_log_1.beam diff --git a/bootstrap/lib/kernel/ebin/disk_log_server.beam b/bootstrap/lib/kernel/ebin/disk_log_server.beam Binary files differindex e212e82d15..58fe6f7f39 100644 --- a/bootstrap/lib/kernel/ebin/disk_log_server.beam +++ b/bootstrap/lib/kernel/ebin/disk_log_server.beam diff --git a/bootstrap/lib/kernel/ebin/disk_log_sup.beam b/bootstrap/lib/kernel/ebin/disk_log_sup.beam Binary files differindex 939e6f4d3d..2faecea6bd 100644 --- a/bootstrap/lib/kernel/ebin/disk_log_sup.beam +++ b/bootstrap/lib/kernel/ebin/disk_log_sup.beam diff --git a/bootstrap/lib/kernel/ebin/dist_ac.beam b/bootstrap/lib/kernel/ebin/dist_ac.beam Binary files differindex 61f988cb71..2c08e04eb7 100644 --- a/bootstrap/lib/kernel/ebin/dist_ac.beam +++ b/bootstrap/lib/kernel/ebin/dist_ac.beam diff --git a/bootstrap/lib/kernel/ebin/dist_util.beam b/bootstrap/lib/kernel/ebin/dist_util.beam Binary files differindex 00d4d5198c..5f21527ec9 100644 --- a/bootstrap/lib/kernel/ebin/dist_util.beam +++ b/bootstrap/lib/kernel/ebin/dist_util.beam diff --git a/bootstrap/lib/kernel/ebin/erl_boot_server.beam b/bootstrap/lib/kernel/ebin/erl_boot_server.beam Binary files differindex e19853bb22..bd838b7c7f 100644 --- a/bootstrap/lib/kernel/ebin/erl_boot_server.beam +++ b/bootstrap/lib/kernel/ebin/erl_boot_server.beam diff --git a/bootstrap/lib/kernel/ebin/erl_ddll.beam b/bootstrap/lib/kernel/ebin/erl_ddll.beam Binary files differindex c464b92096..13aeb8fecb 100644 --- a/bootstrap/lib/kernel/ebin/erl_ddll.beam +++ b/bootstrap/lib/kernel/ebin/erl_ddll.beam diff --git a/bootstrap/lib/kernel/ebin/erl_distribution.beam b/bootstrap/lib/kernel/ebin/erl_distribution.beam Binary files differindex 1bd3c7111c..4b304c3678 100644 --- a/bootstrap/lib/kernel/ebin/erl_distribution.beam +++ b/bootstrap/lib/kernel/ebin/erl_distribution.beam diff --git a/bootstrap/lib/kernel/ebin/erl_epmd.beam b/bootstrap/lib/kernel/ebin/erl_epmd.beam Binary files differindex 8af37f8bf4..1025a920b2 100644 --- a/bootstrap/lib/kernel/ebin/erl_epmd.beam +++ b/bootstrap/lib/kernel/ebin/erl_epmd.beam diff --git a/bootstrap/lib/kernel/ebin/erl_reply.beam b/bootstrap/lib/kernel/ebin/erl_reply.beam Binary files differindex 24ee703ca2..527052b53f 100644 --- a/bootstrap/lib/kernel/ebin/erl_reply.beam +++ b/bootstrap/lib/kernel/ebin/erl_reply.beam diff --git a/bootstrap/lib/kernel/ebin/erl_signal_handler.beam b/bootstrap/lib/kernel/ebin/erl_signal_handler.beam Binary files differindex c5b85d9b69..2bc5d929b3 100644 --- a/bootstrap/lib/kernel/ebin/erl_signal_handler.beam +++ b/bootstrap/lib/kernel/ebin/erl_signal_handler.beam diff --git a/bootstrap/lib/kernel/ebin/error_handler.beam b/bootstrap/lib/kernel/ebin/error_handler.beam Binary files differindex f2df2a189e..2249d2c98c 100644 --- a/bootstrap/lib/kernel/ebin/error_handler.beam +++ b/bootstrap/lib/kernel/ebin/error_handler.beam diff --git a/bootstrap/lib/kernel/ebin/error_logger.beam b/bootstrap/lib/kernel/ebin/error_logger.beam Binary files differindex a0a8f8b121..0de0bc3ab3 100644 --- a/bootstrap/lib/kernel/ebin/error_logger.beam +++ b/bootstrap/lib/kernel/ebin/error_logger.beam diff --git a/bootstrap/lib/kernel/ebin/erts_debug.beam b/bootstrap/lib/kernel/ebin/erts_debug.beam Binary files differindex 3c77d5a20f..ac33275203 100644 --- a/bootstrap/lib/kernel/ebin/erts_debug.beam +++ b/bootstrap/lib/kernel/ebin/erts_debug.beam diff --git a/bootstrap/lib/kernel/ebin/file.beam b/bootstrap/lib/kernel/ebin/file.beam Binary files differindex 4a50f9de78..2575312622 100644 --- a/bootstrap/lib/kernel/ebin/file.beam +++ b/bootstrap/lib/kernel/ebin/file.beam diff --git a/bootstrap/lib/kernel/ebin/file_io_server.beam b/bootstrap/lib/kernel/ebin/file_io_server.beam Binary files differindex 7245ec2695..9b7a33bd36 100644 --- a/bootstrap/lib/kernel/ebin/file_io_server.beam +++ b/bootstrap/lib/kernel/ebin/file_io_server.beam diff --git a/bootstrap/lib/kernel/ebin/file_server.beam b/bootstrap/lib/kernel/ebin/file_server.beam Binary files differindex ace5489a3c..b58c1a2710 100644 --- a/bootstrap/lib/kernel/ebin/file_server.beam +++ b/bootstrap/lib/kernel/ebin/file_server.beam diff --git a/bootstrap/lib/kernel/ebin/gen_sctp.beam b/bootstrap/lib/kernel/ebin/gen_sctp.beam Binary files differindex 3fa0409227..8478258754 100644 --- a/bootstrap/lib/kernel/ebin/gen_sctp.beam +++ b/bootstrap/lib/kernel/ebin/gen_sctp.beam diff --git a/bootstrap/lib/kernel/ebin/gen_tcp.beam b/bootstrap/lib/kernel/ebin/gen_tcp.beam Binary files differindex c154687a9b..1356853176 100644 --- a/bootstrap/lib/kernel/ebin/gen_tcp.beam +++ b/bootstrap/lib/kernel/ebin/gen_tcp.beam diff --git a/bootstrap/lib/kernel/ebin/gen_udp.beam b/bootstrap/lib/kernel/ebin/gen_udp.beam Binary files differindex 447d4fdc49..7a79ecb6b4 100644 --- a/bootstrap/lib/kernel/ebin/gen_udp.beam +++ b/bootstrap/lib/kernel/ebin/gen_udp.beam diff --git a/bootstrap/lib/kernel/ebin/global.beam b/bootstrap/lib/kernel/ebin/global.beam Binary files differindex 98c21a15f3..db8034ffbe 100644 --- a/bootstrap/lib/kernel/ebin/global.beam +++ b/bootstrap/lib/kernel/ebin/global.beam diff --git a/bootstrap/lib/kernel/ebin/global_group.beam b/bootstrap/lib/kernel/ebin/global_group.beam Binary files differindex 2e1fc5d542..fe618952d7 100644 --- a/bootstrap/lib/kernel/ebin/global_group.beam +++ b/bootstrap/lib/kernel/ebin/global_group.beam diff --git a/bootstrap/lib/kernel/ebin/global_search.beam b/bootstrap/lib/kernel/ebin/global_search.beam Binary files differindex ae2a767188..4fc4d2414b 100644 --- a/bootstrap/lib/kernel/ebin/global_search.beam +++ b/bootstrap/lib/kernel/ebin/global_search.beam diff --git a/bootstrap/lib/kernel/ebin/group.beam b/bootstrap/lib/kernel/ebin/group.beam Binary files differindex 8bf4af542f..53546fb74c 100644 --- a/bootstrap/lib/kernel/ebin/group.beam +++ b/bootstrap/lib/kernel/ebin/group.beam diff --git a/bootstrap/lib/kernel/ebin/group_history.beam b/bootstrap/lib/kernel/ebin/group_history.beam Binary files differnew file mode 100644 index 0000000000..51855d33ed --- /dev/null +++ b/bootstrap/lib/kernel/ebin/group_history.beam diff --git a/bootstrap/lib/kernel/ebin/heart.beam b/bootstrap/lib/kernel/ebin/heart.beam Binary files differindex 6eadb43fe4..104956f663 100644 --- a/bootstrap/lib/kernel/ebin/heart.beam +++ b/bootstrap/lib/kernel/ebin/heart.beam diff --git a/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam b/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam Binary files differindex 6e35a4f5cf..44d14383ec 100644 --- a/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam +++ b/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam diff --git a/bootstrap/lib/kernel/ebin/inet.beam b/bootstrap/lib/kernel/ebin/inet.beam Binary files differindex 473d6154da..eb0d71d97f 100644 --- a/bootstrap/lib/kernel/ebin/inet.beam +++ b/bootstrap/lib/kernel/ebin/inet.beam diff --git a/bootstrap/lib/kernel/ebin/inet6_sctp.beam b/bootstrap/lib/kernel/ebin/inet6_sctp.beam Binary files differindex 2199828859..079b21b6e5 100644 --- a/bootstrap/lib/kernel/ebin/inet6_sctp.beam +++ b/bootstrap/lib/kernel/ebin/inet6_sctp.beam diff --git a/bootstrap/lib/kernel/ebin/inet6_tcp.beam b/bootstrap/lib/kernel/ebin/inet6_tcp.beam Binary files differindex 5a5bbfea43..0ce20bc589 100644 --- a/bootstrap/lib/kernel/ebin/inet6_tcp.beam +++ b/bootstrap/lib/kernel/ebin/inet6_tcp.beam diff --git a/bootstrap/lib/kernel/ebin/inet6_tcp_dist.beam b/bootstrap/lib/kernel/ebin/inet6_tcp_dist.beam Binary files differindex af6ce5f042..dd4c54cee6 100644 --- a/bootstrap/lib/kernel/ebin/inet6_tcp_dist.beam +++ b/bootstrap/lib/kernel/ebin/inet6_tcp_dist.beam diff --git a/bootstrap/lib/kernel/ebin/inet6_udp.beam b/bootstrap/lib/kernel/ebin/inet6_udp.beam Binary files differindex 0563b5db26..092cae80f2 100644 --- a/bootstrap/lib/kernel/ebin/inet6_udp.beam +++ b/bootstrap/lib/kernel/ebin/inet6_udp.beam diff --git a/bootstrap/lib/kernel/ebin/inet_config.beam b/bootstrap/lib/kernel/ebin/inet_config.beam Binary files differindex d9482ebcb0..4b6899be8a 100644 --- a/bootstrap/lib/kernel/ebin/inet_config.beam +++ b/bootstrap/lib/kernel/ebin/inet_config.beam diff --git a/bootstrap/lib/kernel/ebin/inet_db.beam b/bootstrap/lib/kernel/ebin/inet_db.beam Binary files differindex dc505f54da..2b9990aea5 100644 --- a/bootstrap/lib/kernel/ebin/inet_db.beam +++ b/bootstrap/lib/kernel/ebin/inet_db.beam diff --git a/bootstrap/lib/kernel/ebin/inet_dns.beam b/bootstrap/lib/kernel/ebin/inet_dns.beam Binary files differindex d3872cebbd..6bee965f89 100644 --- a/bootstrap/lib/kernel/ebin/inet_dns.beam +++ b/bootstrap/lib/kernel/ebin/inet_dns.beam diff --git a/bootstrap/lib/kernel/ebin/inet_gethost_native.beam b/bootstrap/lib/kernel/ebin/inet_gethost_native.beam Binary files differindex 883cfe6cdf..ea2bdf2a26 100644 --- a/bootstrap/lib/kernel/ebin/inet_gethost_native.beam +++ b/bootstrap/lib/kernel/ebin/inet_gethost_native.beam diff --git a/bootstrap/lib/kernel/ebin/inet_hosts.beam b/bootstrap/lib/kernel/ebin/inet_hosts.beam Binary files differindex 4d9377b124..d607df4f2f 100644 --- a/bootstrap/lib/kernel/ebin/inet_hosts.beam +++ b/bootstrap/lib/kernel/ebin/inet_hosts.beam diff --git a/bootstrap/lib/kernel/ebin/inet_parse.beam b/bootstrap/lib/kernel/ebin/inet_parse.beam Binary files differindex f78e30cbb0..595a23397e 100644 --- a/bootstrap/lib/kernel/ebin/inet_parse.beam +++ b/bootstrap/lib/kernel/ebin/inet_parse.beam diff --git a/bootstrap/lib/kernel/ebin/inet_res.beam b/bootstrap/lib/kernel/ebin/inet_res.beam Binary files differindex 3147ff5dbc..59c641f85c 100644 --- a/bootstrap/lib/kernel/ebin/inet_res.beam +++ b/bootstrap/lib/kernel/ebin/inet_res.beam diff --git a/bootstrap/lib/kernel/ebin/inet_sctp.beam b/bootstrap/lib/kernel/ebin/inet_sctp.beam Binary files differindex 4eb47f1f50..5128048f88 100644 --- a/bootstrap/lib/kernel/ebin/inet_sctp.beam +++ b/bootstrap/lib/kernel/ebin/inet_sctp.beam diff --git a/bootstrap/lib/kernel/ebin/inet_tcp.beam b/bootstrap/lib/kernel/ebin/inet_tcp.beam Binary files differindex 48e3986b0b..ebbbb55972 100644 --- a/bootstrap/lib/kernel/ebin/inet_tcp.beam +++ b/bootstrap/lib/kernel/ebin/inet_tcp.beam diff --git a/bootstrap/lib/kernel/ebin/inet_tcp_dist.beam b/bootstrap/lib/kernel/ebin/inet_tcp_dist.beam Binary files differindex 666b2b5beb..13237c1421 100644 --- a/bootstrap/lib/kernel/ebin/inet_tcp_dist.beam +++ b/bootstrap/lib/kernel/ebin/inet_tcp_dist.beam diff --git a/bootstrap/lib/kernel/ebin/inet_udp.beam b/bootstrap/lib/kernel/ebin/inet_udp.beam Binary files differindex 804ee860dd..153456e968 100644 --- a/bootstrap/lib/kernel/ebin/inet_udp.beam +++ b/bootstrap/lib/kernel/ebin/inet_udp.beam diff --git a/bootstrap/lib/kernel/ebin/kernel.app b/bootstrap/lib/kernel/ebin/kernel.app index af771b205c..c499734bb1 100644 --- a/bootstrap/lib/kernel/ebin/kernel.app +++ b/bootstrap/lib/kernel/ebin/kernel.app @@ -44,6 +44,7 @@ global_group, global_search, group, + group_history, heart, hipe_unified_loader, inet6_tcp, diff --git a/bootstrap/lib/kernel/ebin/kernel.beam b/bootstrap/lib/kernel/ebin/kernel.beam Binary files differindex 6bb295a61a..d4170b3652 100644 --- a/bootstrap/lib/kernel/ebin/kernel.beam +++ b/bootstrap/lib/kernel/ebin/kernel.beam diff --git a/bootstrap/lib/kernel/ebin/kernel_config.beam b/bootstrap/lib/kernel/ebin/kernel_config.beam Binary files differindex 90b62dd556..ddd47e7ad8 100644 --- a/bootstrap/lib/kernel/ebin/kernel_config.beam +++ b/bootstrap/lib/kernel/ebin/kernel_config.beam diff --git a/bootstrap/lib/kernel/ebin/local_tcp.beam b/bootstrap/lib/kernel/ebin/local_tcp.beam Binary files differindex f3b3291ad1..7d4e10d93f 100644 --- a/bootstrap/lib/kernel/ebin/local_tcp.beam +++ b/bootstrap/lib/kernel/ebin/local_tcp.beam diff --git a/bootstrap/lib/kernel/ebin/local_udp.beam b/bootstrap/lib/kernel/ebin/local_udp.beam Binary files differindex 0ddae3a57a..798eeeb7f6 100644 --- a/bootstrap/lib/kernel/ebin/local_udp.beam +++ b/bootstrap/lib/kernel/ebin/local_udp.beam diff --git a/bootstrap/lib/kernel/ebin/net.beam b/bootstrap/lib/kernel/ebin/net.beam Binary files differindex a02b17cae2..772535169c 100644 --- a/bootstrap/lib/kernel/ebin/net.beam +++ b/bootstrap/lib/kernel/ebin/net.beam diff --git a/bootstrap/lib/kernel/ebin/net_adm.beam b/bootstrap/lib/kernel/ebin/net_adm.beam Binary files differindex 6bb14d425d..94d6a5ce97 100644 --- a/bootstrap/lib/kernel/ebin/net_adm.beam +++ b/bootstrap/lib/kernel/ebin/net_adm.beam diff --git a/bootstrap/lib/kernel/ebin/net_kernel.beam b/bootstrap/lib/kernel/ebin/net_kernel.beam Binary files differindex 7ca6040b70..d78e9b6c05 100644 --- a/bootstrap/lib/kernel/ebin/net_kernel.beam +++ b/bootstrap/lib/kernel/ebin/net_kernel.beam diff --git a/bootstrap/lib/kernel/ebin/os.beam b/bootstrap/lib/kernel/ebin/os.beam Binary files differindex fb65ca94e8..2c4a0caa9d 100644 --- a/bootstrap/lib/kernel/ebin/os.beam +++ b/bootstrap/lib/kernel/ebin/os.beam diff --git a/bootstrap/lib/kernel/ebin/pg2.beam b/bootstrap/lib/kernel/ebin/pg2.beam Binary files differindex 1de40824fd..2d219d03d5 100644 --- a/bootstrap/lib/kernel/ebin/pg2.beam +++ b/bootstrap/lib/kernel/ebin/pg2.beam diff --git a/bootstrap/lib/kernel/ebin/ram_file.beam b/bootstrap/lib/kernel/ebin/ram_file.beam Binary files differindex 0dc6ad73e0..1b23e89143 100644 --- a/bootstrap/lib/kernel/ebin/ram_file.beam +++ b/bootstrap/lib/kernel/ebin/ram_file.beam diff --git a/bootstrap/lib/kernel/ebin/rpc.beam b/bootstrap/lib/kernel/ebin/rpc.beam Binary files differindex 7578e601ff..c73b59779b 100644 --- a/bootstrap/lib/kernel/ebin/rpc.beam +++ b/bootstrap/lib/kernel/ebin/rpc.beam diff --git a/bootstrap/lib/kernel/ebin/seq_trace.beam b/bootstrap/lib/kernel/ebin/seq_trace.beam Binary files differindex 5ba162831d..88406b3e2e 100644 --- a/bootstrap/lib/kernel/ebin/seq_trace.beam +++ b/bootstrap/lib/kernel/ebin/seq_trace.beam diff --git a/bootstrap/lib/kernel/ebin/standard_error.beam b/bootstrap/lib/kernel/ebin/standard_error.beam Binary files differindex 3c4cba4edd..f0860d3ae8 100644 --- a/bootstrap/lib/kernel/ebin/standard_error.beam +++ b/bootstrap/lib/kernel/ebin/standard_error.beam diff --git a/bootstrap/lib/kernel/ebin/user.beam b/bootstrap/lib/kernel/ebin/user.beam Binary files differindex 6d59ec7971..73be8706a4 100644 --- a/bootstrap/lib/kernel/ebin/user.beam +++ b/bootstrap/lib/kernel/ebin/user.beam diff --git a/bootstrap/lib/kernel/ebin/user_drv.beam b/bootstrap/lib/kernel/ebin/user_drv.beam Binary files differindex 615bd1bb15..ca29be601b 100644 --- a/bootstrap/lib/kernel/ebin/user_drv.beam +++ b/bootstrap/lib/kernel/ebin/user_drv.beam diff --git a/bootstrap/lib/kernel/ebin/user_sup.beam b/bootstrap/lib/kernel/ebin/user_sup.beam Binary files differindex 7277b79a76..c81217476b 100644 --- a/bootstrap/lib/kernel/ebin/user_sup.beam +++ b/bootstrap/lib/kernel/ebin/user_sup.beam diff --git a/bootstrap/lib/kernel/ebin/wrap_log_reader.beam b/bootstrap/lib/kernel/ebin/wrap_log_reader.beam Binary files differindex f1486678f0..dc30a0e212 100644 --- a/bootstrap/lib/kernel/ebin/wrap_log_reader.beam +++ b/bootstrap/lib/kernel/ebin/wrap_log_reader.beam diff --git a/bootstrap/lib/kernel/include/inet.hrl b/bootstrap/lib/kernel/include/inet.hrl index df788aca08..daa2e14b46 100644 --- a/bootstrap/lib/kernel/include/inet.hrl +++ b/bootstrap/lib/kernel/include/inet.hrl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2016. All Rights Reserved. +%% Copyright Ericsson AB 1997-2017. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/bootstrap/lib/stdlib/ebin/array.beam b/bootstrap/lib/stdlib/ebin/array.beam Binary files differindex 6992a24d98..e9b416b68b 100644 --- a/bootstrap/lib/stdlib/ebin/array.beam +++ b/bootstrap/lib/stdlib/ebin/array.beam diff --git a/bootstrap/lib/stdlib/ebin/base64.beam b/bootstrap/lib/stdlib/ebin/base64.beam Binary files differindex f89e3be956..8c42e9f22d 100644 --- a/bootstrap/lib/stdlib/ebin/base64.beam +++ b/bootstrap/lib/stdlib/ebin/base64.beam diff --git a/bootstrap/lib/stdlib/ebin/beam_lib.beam b/bootstrap/lib/stdlib/ebin/beam_lib.beam Binary files differindex 4c82a8a6c5..8cc54cf7b3 100644 --- a/bootstrap/lib/stdlib/ebin/beam_lib.beam +++ b/bootstrap/lib/stdlib/ebin/beam_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/binary.beam b/bootstrap/lib/stdlib/ebin/binary.beam Binary files differindex 0863c4cf8d..ec03caa782 100644 --- a/bootstrap/lib/stdlib/ebin/binary.beam +++ b/bootstrap/lib/stdlib/ebin/binary.beam diff --git a/bootstrap/lib/stdlib/ebin/c.beam b/bootstrap/lib/stdlib/ebin/c.beam Binary files differindex 5a69b635cb..541ae0e54a 100644 --- a/bootstrap/lib/stdlib/ebin/c.beam +++ b/bootstrap/lib/stdlib/ebin/c.beam diff --git a/bootstrap/lib/stdlib/ebin/calendar.beam b/bootstrap/lib/stdlib/ebin/calendar.beam Binary files differindex ca9e73c245..5ba087e5e9 100644 --- a/bootstrap/lib/stdlib/ebin/calendar.beam +++ b/bootstrap/lib/stdlib/ebin/calendar.beam diff --git a/bootstrap/lib/stdlib/ebin/dets.beam b/bootstrap/lib/stdlib/ebin/dets.beam Binary files differindex 586b0c35cd..10675b4bf2 100644 --- a/bootstrap/lib/stdlib/ebin/dets.beam +++ b/bootstrap/lib/stdlib/ebin/dets.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_server.beam b/bootstrap/lib/stdlib/ebin/dets_server.beam Binary files differindex de315c0517..2019fda39d 100644 --- a/bootstrap/lib/stdlib/ebin/dets_server.beam +++ b/bootstrap/lib/stdlib/ebin/dets_server.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_sup.beam b/bootstrap/lib/stdlib/ebin/dets_sup.beam Binary files differindex 34732d5a99..05da1822d3 100644 --- a/bootstrap/lib/stdlib/ebin/dets_sup.beam +++ b/bootstrap/lib/stdlib/ebin/dets_sup.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_utils.beam b/bootstrap/lib/stdlib/ebin/dets_utils.beam Binary files differindex 576992c128..0d90896c26 100644 --- a/bootstrap/lib/stdlib/ebin/dets_utils.beam +++ b/bootstrap/lib/stdlib/ebin/dets_utils.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_v9.beam b/bootstrap/lib/stdlib/ebin/dets_v9.beam Binary files differindex 55e37c99a4..273211cf78 100644 --- a/bootstrap/lib/stdlib/ebin/dets_v9.beam +++ b/bootstrap/lib/stdlib/ebin/dets_v9.beam diff --git a/bootstrap/lib/stdlib/ebin/dict.beam b/bootstrap/lib/stdlib/ebin/dict.beam Binary files differindex 0c38ee1717..9bd8311a9b 100644 --- a/bootstrap/lib/stdlib/ebin/dict.beam +++ b/bootstrap/lib/stdlib/ebin/dict.beam diff --git a/bootstrap/lib/stdlib/ebin/digraph.beam b/bootstrap/lib/stdlib/ebin/digraph.beam Binary files differindex 7b1ce6b4ce..237a5d3b30 100644 --- a/bootstrap/lib/stdlib/ebin/digraph.beam +++ b/bootstrap/lib/stdlib/ebin/digraph.beam diff --git a/bootstrap/lib/stdlib/ebin/digraph_utils.beam b/bootstrap/lib/stdlib/ebin/digraph_utils.beam Binary files differindex 2b82c5f2d9..29e26659ce 100644 --- a/bootstrap/lib/stdlib/ebin/digraph_utils.beam +++ b/bootstrap/lib/stdlib/ebin/digraph_utils.beam diff --git a/bootstrap/lib/stdlib/ebin/edlin.beam b/bootstrap/lib/stdlib/ebin/edlin.beam Binary files differindex dbc9a668b0..40dd9561c4 100644 --- a/bootstrap/lib/stdlib/ebin/edlin.beam +++ b/bootstrap/lib/stdlib/ebin/edlin.beam diff --git a/bootstrap/lib/stdlib/ebin/edlin_expand.beam b/bootstrap/lib/stdlib/ebin/edlin_expand.beam Binary files differindex f79a4044e6..591f6ab195 100644 --- a/bootstrap/lib/stdlib/ebin/edlin_expand.beam +++ b/bootstrap/lib/stdlib/ebin/edlin_expand.beam diff --git a/bootstrap/lib/stdlib/ebin/epp.beam b/bootstrap/lib/stdlib/ebin/epp.beam Binary files differindex 5fc1d337bb..990cc1ab26 100644 --- a/bootstrap/lib/stdlib/ebin/epp.beam +++ b/bootstrap/lib/stdlib/ebin/epp.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_abstract_code.beam b/bootstrap/lib/stdlib/ebin/erl_abstract_code.beam Binary files differindex 83f5ceac01..c98a937bc0 100644 --- a/bootstrap/lib/stdlib/ebin/erl_abstract_code.beam +++ b/bootstrap/lib/stdlib/ebin/erl_abstract_code.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_anno.beam b/bootstrap/lib/stdlib/ebin/erl_anno.beam Binary files differindex 212fe27e28..4dae00603f 100644 --- a/bootstrap/lib/stdlib/ebin/erl_anno.beam +++ b/bootstrap/lib/stdlib/ebin/erl_anno.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_bits.beam b/bootstrap/lib/stdlib/ebin/erl_bits.beam Binary files differindex 794162a95e..c0dc007589 100644 --- a/bootstrap/lib/stdlib/ebin/erl_bits.beam +++ b/bootstrap/lib/stdlib/ebin/erl_bits.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_compile.beam b/bootstrap/lib/stdlib/ebin/erl_compile.beam Binary files differindex 07f78c8599..be7381012b 100644 --- a/bootstrap/lib/stdlib/ebin/erl_compile.beam +++ b/bootstrap/lib/stdlib/ebin/erl_compile.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_eval.beam b/bootstrap/lib/stdlib/ebin/erl_eval.beam Binary files differindex a15d6d1b39..faf992ee0b 100644 --- a/bootstrap/lib/stdlib/ebin/erl_eval.beam +++ b/bootstrap/lib/stdlib/ebin/erl_eval.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_expand_records.beam b/bootstrap/lib/stdlib/ebin/erl_expand_records.beam Binary files differindex e38a940352..ab6f576b81 100644 --- a/bootstrap/lib/stdlib/ebin/erl_expand_records.beam +++ b/bootstrap/lib/stdlib/ebin/erl_expand_records.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_internal.beam b/bootstrap/lib/stdlib/ebin/erl_internal.beam Binary files differindex 7f4519c7b7..bcf53a392a 100644 --- a/bootstrap/lib/stdlib/ebin/erl_internal.beam +++ b/bootstrap/lib/stdlib/ebin/erl_internal.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_lint.beam b/bootstrap/lib/stdlib/ebin/erl_lint.beam Binary files differindex c839164faf..98477b5d8f 100644 --- a/bootstrap/lib/stdlib/ebin/erl_lint.beam +++ b/bootstrap/lib/stdlib/ebin/erl_lint.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_parse.beam b/bootstrap/lib/stdlib/ebin/erl_parse.beam Binary files differindex ad45f99496..a2a236fe7d 100644 --- a/bootstrap/lib/stdlib/ebin/erl_parse.beam +++ b/bootstrap/lib/stdlib/ebin/erl_parse.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_posix_msg.beam b/bootstrap/lib/stdlib/ebin/erl_posix_msg.beam Binary files differindex d2be4727e1..c6a52797f3 100644 --- a/bootstrap/lib/stdlib/ebin/erl_posix_msg.beam +++ b/bootstrap/lib/stdlib/ebin/erl_posix_msg.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_pp.beam b/bootstrap/lib/stdlib/ebin/erl_pp.beam Binary files differindex 19e5fb932b..fc65556a86 100644 --- a/bootstrap/lib/stdlib/ebin/erl_pp.beam +++ b/bootstrap/lib/stdlib/ebin/erl_pp.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_scan.beam b/bootstrap/lib/stdlib/ebin/erl_scan.beam Binary files differindex 608d8c2788..db02f2f493 100644 --- a/bootstrap/lib/stdlib/ebin/erl_scan.beam +++ b/bootstrap/lib/stdlib/ebin/erl_scan.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_tar.beam b/bootstrap/lib/stdlib/ebin/erl_tar.beam Binary files differindex bcb8a3aa72..81adab4fad 100644 --- a/bootstrap/lib/stdlib/ebin/erl_tar.beam +++ b/bootstrap/lib/stdlib/ebin/erl_tar.beam diff --git a/bootstrap/lib/stdlib/ebin/error_logger_file_h.beam b/bootstrap/lib/stdlib/ebin/error_logger_file_h.beam Binary files differindex 78dfc523b2..63e350df89 100644 --- a/bootstrap/lib/stdlib/ebin/error_logger_file_h.beam +++ b/bootstrap/lib/stdlib/ebin/error_logger_file_h.beam diff --git a/bootstrap/lib/stdlib/ebin/error_logger_tty_h.beam b/bootstrap/lib/stdlib/ebin/error_logger_tty_h.beam Binary files differindex 9c913ce5fb..77a5216e39 100644 --- a/bootstrap/lib/stdlib/ebin/error_logger_tty_h.beam +++ b/bootstrap/lib/stdlib/ebin/error_logger_tty_h.beam diff --git a/bootstrap/lib/stdlib/ebin/escript.beam b/bootstrap/lib/stdlib/ebin/escript.beam Binary files differindex 1411e99456..ac03b99ee4 100644 --- a/bootstrap/lib/stdlib/ebin/escript.beam +++ b/bootstrap/lib/stdlib/ebin/escript.beam diff --git a/bootstrap/lib/stdlib/ebin/ets.beam b/bootstrap/lib/stdlib/ebin/ets.beam Binary files differindex 935a132190..1b3fc9c9cd 100644 --- a/bootstrap/lib/stdlib/ebin/ets.beam +++ b/bootstrap/lib/stdlib/ebin/ets.beam diff --git a/bootstrap/lib/stdlib/ebin/eval_bits.beam b/bootstrap/lib/stdlib/ebin/eval_bits.beam Binary files differindex ecfedab133..251c1946ff 100644 --- a/bootstrap/lib/stdlib/ebin/eval_bits.beam +++ b/bootstrap/lib/stdlib/ebin/eval_bits.beam diff --git a/bootstrap/lib/stdlib/ebin/file_sorter.beam b/bootstrap/lib/stdlib/ebin/file_sorter.beam Binary files differindex 182eb8d648..d6bab6459e 100644 --- a/bootstrap/lib/stdlib/ebin/file_sorter.beam +++ b/bootstrap/lib/stdlib/ebin/file_sorter.beam diff --git a/bootstrap/lib/stdlib/ebin/filelib.beam b/bootstrap/lib/stdlib/ebin/filelib.beam Binary files differindex 733b60a71d..249e7a3410 100644 --- a/bootstrap/lib/stdlib/ebin/filelib.beam +++ b/bootstrap/lib/stdlib/ebin/filelib.beam diff --git a/bootstrap/lib/stdlib/ebin/filename.beam b/bootstrap/lib/stdlib/ebin/filename.beam Binary files differindex 516ec1fcdd..85d0c53a53 100644 --- a/bootstrap/lib/stdlib/ebin/filename.beam +++ b/bootstrap/lib/stdlib/ebin/filename.beam diff --git a/bootstrap/lib/stdlib/ebin/gb_sets.beam b/bootstrap/lib/stdlib/ebin/gb_sets.beam Binary files differindex 2e1a95b10c..18acb2891c 100644 --- a/bootstrap/lib/stdlib/ebin/gb_sets.beam +++ b/bootstrap/lib/stdlib/ebin/gb_sets.beam diff --git a/bootstrap/lib/stdlib/ebin/gb_trees.beam b/bootstrap/lib/stdlib/ebin/gb_trees.beam Binary files differindex ea9d7c7a66..6f5368d1d6 100644 --- a/bootstrap/lib/stdlib/ebin/gb_trees.beam +++ b/bootstrap/lib/stdlib/ebin/gb_trees.beam diff --git a/bootstrap/lib/stdlib/ebin/gen.beam b/bootstrap/lib/stdlib/ebin/gen.beam Binary files differindex 8f36971bb5..c500cb1af1 100644 --- a/bootstrap/lib/stdlib/ebin/gen.beam +++ b/bootstrap/lib/stdlib/ebin/gen.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_event.beam b/bootstrap/lib/stdlib/ebin/gen_event.beam Binary files differindex aa01da81eb..aeb8aaa42f 100644 --- a/bootstrap/lib/stdlib/ebin/gen_event.beam +++ b/bootstrap/lib/stdlib/ebin/gen_event.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_fsm.beam b/bootstrap/lib/stdlib/ebin/gen_fsm.beam Binary files differindex 73eff4700f..404ec6735b 100644 --- a/bootstrap/lib/stdlib/ebin/gen_fsm.beam +++ b/bootstrap/lib/stdlib/ebin/gen_fsm.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_server.beam b/bootstrap/lib/stdlib/ebin/gen_server.beam Binary files differindex f4fd0c415c..cc30bcbbb8 100644 --- a/bootstrap/lib/stdlib/ebin/gen_server.beam +++ b/bootstrap/lib/stdlib/ebin/gen_server.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_statem.beam b/bootstrap/lib/stdlib/ebin/gen_statem.beam Binary files differindex 09cf9cbd50..847042c84a 100644 --- a/bootstrap/lib/stdlib/ebin/gen_statem.beam +++ b/bootstrap/lib/stdlib/ebin/gen_statem.beam diff --git a/bootstrap/lib/stdlib/ebin/io.beam b/bootstrap/lib/stdlib/ebin/io.beam Binary files differindex 5ac3d1f069..d774ba98a4 100644 --- a/bootstrap/lib/stdlib/ebin/io.beam +++ b/bootstrap/lib/stdlib/ebin/io.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib.beam b/bootstrap/lib/stdlib/ebin/io_lib.beam Binary files differindex 4ab1fc644a..be6c16582a 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib_format.beam b/bootstrap/lib/stdlib/ebin/io_lib_format.beam Binary files differindex 27ea37e332..58a6b6fb1f 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib_format.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib_format.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib_fread.beam b/bootstrap/lib/stdlib/ebin/io_lib_fread.beam Binary files differindex 78afdd522c..0fcefbd620 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib_fread.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib_fread.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam b/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam Binary files differindex b0756eecdf..9f00efb00a 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam diff --git a/bootstrap/lib/stdlib/ebin/lib.beam b/bootstrap/lib/stdlib/ebin/lib.beam Binary files differindex 158154cc15..8fe032f818 100644 --- a/bootstrap/lib/stdlib/ebin/lib.beam +++ b/bootstrap/lib/stdlib/ebin/lib.beam diff --git a/bootstrap/lib/stdlib/ebin/lists.beam b/bootstrap/lib/stdlib/ebin/lists.beam Binary files differindex 3331a5808b..5fd6f76a23 100644 --- a/bootstrap/lib/stdlib/ebin/lists.beam +++ b/bootstrap/lib/stdlib/ebin/lists.beam diff --git a/bootstrap/lib/stdlib/ebin/log_mf_h.beam b/bootstrap/lib/stdlib/ebin/log_mf_h.beam Binary files differindex f305ba99f5..fbc8c8853e 100644 --- a/bootstrap/lib/stdlib/ebin/log_mf_h.beam +++ b/bootstrap/lib/stdlib/ebin/log_mf_h.beam diff --git a/bootstrap/lib/stdlib/ebin/maps.beam b/bootstrap/lib/stdlib/ebin/maps.beam Binary files differindex 3a0fa8ec1d..abb18a4e36 100644 --- a/bootstrap/lib/stdlib/ebin/maps.beam +++ b/bootstrap/lib/stdlib/ebin/maps.beam diff --git a/bootstrap/lib/stdlib/ebin/math.beam b/bootstrap/lib/stdlib/ebin/math.beam Binary files differindex de17278918..aaa6cb3e4d 100644 --- a/bootstrap/lib/stdlib/ebin/math.beam +++ b/bootstrap/lib/stdlib/ebin/math.beam diff --git a/bootstrap/lib/stdlib/ebin/ms_transform.beam b/bootstrap/lib/stdlib/ebin/ms_transform.beam Binary files differindex 1214b3591b..377d02bf80 100644 --- a/bootstrap/lib/stdlib/ebin/ms_transform.beam +++ b/bootstrap/lib/stdlib/ebin/ms_transform.beam diff --git a/bootstrap/lib/stdlib/ebin/orddict.beam b/bootstrap/lib/stdlib/ebin/orddict.beam Binary files differindex 1fc4685614..f6f6627291 100644 --- a/bootstrap/lib/stdlib/ebin/orddict.beam +++ b/bootstrap/lib/stdlib/ebin/orddict.beam diff --git a/bootstrap/lib/stdlib/ebin/ordsets.beam b/bootstrap/lib/stdlib/ebin/ordsets.beam Binary files differindex 736605b90f..2797b73ee4 100644 --- a/bootstrap/lib/stdlib/ebin/ordsets.beam +++ b/bootstrap/lib/stdlib/ebin/ordsets.beam diff --git a/bootstrap/lib/stdlib/ebin/otp_internal.beam b/bootstrap/lib/stdlib/ebin/otp_internal.beam Binary files differindex e71354d8d4..76430a9d24 100644 --- a/bootstrap/lib/stdlib/ebin/otp_internal.beam +++ b/bootstrap/lib/stdlib/ebin/otp_internal.beam diff --git a/bootstrap/lib/stdlib/ebin/pool.beam b/bootstrap/lib/stdlib/ebin/pool.beam Binary files differindex da31679d1f..2bd926690a 100644 --- a/bootstrap/lib/stdlib/ebin/pool.beam +++ b/bootstrap/lib/stdlib/ebin/pool.beam diff --git a/bootstrap/lib/stdlib/ebin/proc_lib.beam b/bootstrap/lib/stdlib/ebin/proc_lib.beam Binary files differindex 25eb3b652d..934c3d96a1 100644 --- a/bootstrap/lib/stdlib/ebin/proc_lib.beam +++ b/bootstrap/lib/stdlib/ebin/proc_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/proplists.beam b/bootstrap/lib/stdlib/ebin/proplists.beam Binary files differindex d0496eb80f..cdc79d74ea 100644 --- a/bootstrap/lib/stdlib/ebin/proplists.beam +++ b/bootstrap/lib/stdlib/ebin/proplists.beam diff --git a/bootstrap/lib/stdlib/ebin/qlc.beam b/bootstrap/lib/stdlib/ebin/qlc.beam Binary files differindex e2428a9559..b5689663ac 100644 --- a/bootstrap/lib/stdlib/ebin/qlc.beam +++ b/bootstrap/lib/stdlib/ebin/qlc.beam diff --git a/bootstrap/lib/stdlib/ebin/qlc_pt.beam b/bootstrap/lib/stdlib/ebin/qlc_pt.beam Binary files differindex 74f933fb47..bf321e4047 100644 --- a/bootstrap/lib/stdlib/ebin/qlc_pt.beam +++ b/bootstrap/lib/stdlib/ebin/qlc_pt.beam diff --git a/bootstrap/lib/stdlib/ebin/queue.beam b/bootstrap/lib/stdlib/ebin/queue.beam Binary files differindex 5a2cd23b83..95402c8d48 100644 --- a/bootstrap/lib/stdlib/ebin/queue.beam +++ b/bootstrap/lib/stdlib/ebin/queue.beam diff --git a/bootstrap/lib/stdlib/ebin/rand.beam b/bootstrap/lib/stdlib/ebin/rand.beam Binary files differindex c8a6a97976..3129fa7aa9 100644 --- a/bootstrap/lib/stdlib/ebin/rand.beam +++ b/bootstrap/lib/stdlib/ebin/rand.beam diff --git a/bootstrap/lib/stdlib/ebin/random.beam b/bootstrap/lib/stdlib/ebin/random.beam Binary files differindex 0073fc5281..42ba923e19 100644 --- a/bootstrap/lib/stdlib/ebin/random.beam +++ b/bootstrap/lib/stdlib/ebin/random.beam diff --git a/bootstrap/lib/stdlib/ebin/re.beam b/bootstrap/lib/stdlib/ebin/re.beam Binary files differindex 7de890492f..0d493e2a5c 100644 --- a/bootstrap/lib/stdlib/ebin/re.beam +++ b/bootstrap/lib/stdlib/ebin/re.beam diff --git a/bootstrap/lib/stdlib/ebin/sets.beam b/bootstrap/lib/stdlib/ebin/sets.beam Binary files differindex 2cc06fb919..eb7eb95f4e 100644 --- a/bootstrap/lib/stdlib/ebin/sets.beam +++ b/bootstrap/lib/stdlib/ebin/sets.beam diff --git a/bootstrap/lib/stdlib/ebin/shell.beam b/bootstrap/lib/stdlib/ebin/shell.beam Binary files differindex 231440311b..ff0fd46dc8 100644 --- a/bootstrap/lib/stdlib/ebin/shell.beam +++ b/bootstrap/lib/stdlib/ebin/shell.beam diff --git a/bootstrap/lib/stdlib/ebin/shell_default.beam b/bootstrap/lib/stdlib/ebin/shell_default.beam Binary files differindex 0d4e21e544..d13255b49d 100644 --- a/bootstrap/lib/stdlib/ebin/shell_default.beam +++ b/bootstrap/lib/stdlib/ebin/shell_default.beam diff --git a/bootstrap/lib/stdlib/ebin/slave.beam b/bootstrap/lib/stdlib/ebin/slave.beam Binary files differindex cb606a0736..b976a1fd19 100644 --- a/bootstrap/lib/stdlib/ebin/slave.beam +++ b/bootstrap/lib/stdlib/ebin/slave.beam diff --git a/bootstrap/lib/stdlib/ebin/sofs.beam b/bootstrap/lib/stdlib/ebin/sofs.beam Binary files differindex 8ef3635454..7d45fbc2ad 100644 --- a/bootstrap/lib/stdlib/ebin/sofs.beam +++ b/bootstrap/lib/stdlib/ebin/sofs.beam diff --git a/bootstrap/lib/stdlib/ebin/stdlib.app b/bootstrap/lib/stdlib/ebin/stdlib.app index ed73fa6985..33e8de4d90 100644 --- a/bootstrap/lib/stdlib/ebin/stdlib.app +++ b/bootstrap/lib/stdlib/ebin/stdlib.app @@ -2,7 +2,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2017. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. diff --git a/bootstrap/lib/stdlib/ebin/string.beam b/bootstrap/lib/stdlib/ebin/string.beam Binary files differindex 65fb114b50..5d48ed2bcd 100644 --- a/bootstrap/lib/stdlib/ebin/string.beam +++ b/bootstrap/lib/stdlib/ebin/string.beam diff --git a/bootstrap/lib/stdlib/ebin/supervisor.beam b/bootstrap/lib/stdlib/ebin/supervisor.beam Binary files differindex e11d7c9375..7833f18605 100644 --- a/bootstrap/lib/stdlib/ebin/supervisor.beam +++ b/bootstrap/lib/stdlib/ebin/supervisor.beam diff --git a/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam b/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam Binary files differindex a6019f18af..714f92dc7c 100644 --- a/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam +++ b/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam diff --git a/bootstrap/lib/stdlib/ebin/sys.beam b/bootstrap/lib/stdlib/ebin/sys.beam Binary files differindex de5443003f..37dceb7660 100644 --- a/bootstrap/lib/stdlib/ebin/sys.beam +++ b/bootstrap/lib/stdlib/ebin/sys.beam diff --git a/bootstrap/lib/stdlib/ebin/timer.beam b/bootstrap/lib/stdlib/ebin/timer.beam Binary files differindex a1803b06b5..24145e81d6 100644 --- a/bootstrap/lib/stdlib/ebin/timer.beam +++ b/bootstrap/lib/stdlib/ebin/timer.beam diff --git a/bootstrap/lib/stdlib/ebin/unicode.beam b/bootstrap/lib/stdlib/ebin/unicode.beam Binary files differindex d58ad7564e..359e1b9fdf 100644 --- a/bootstrap/lib/stdlib/ebin/unicode.beam +++ b/bootstrap/lib/stdlib/ebin/unicode.beam diff --git a/bootstrap/lib/stdlib/ebin/unicode_util.beam b/bootstrap/lib/stdlib/ebin/unicode_util.beam Binary files differindex 3251926a9c..7771cb469c 100644 --- a/bootstrap/lib/stdlib/ebin/unicode_util.beam +++ b/bootstrap/lib/stdlib/ebin/unicode_util.beam diff --git a/bootstrap/lib/stdlib/ebin/win32reg.beam b/bootstrap/lib/stdlib/ebin/win32reg.beam Binary files differindex 289ca96f59..c5369b13c3 100644 --- a/bootstrap/lib/stdlib/ebin/win32reg.beam +++ b/bootstrap/lib/stdlib/ebin/win32reg.beam diff --git a/bootstrap/lib/stdlib/ebin/zip.beam b/bootstrap/lib/stdlib/ebin/zip.beam Binary files differindex 09f873d8fa..032df45451 100644 --- a/bootstrap/lib/stdlib/ebin/zip.beam +++ b/bootstrap/lib/stdlib/ebin/zip.beam diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml index e1aa5ce76e..638e88ca31 100644 --- a/erts/doc/src/erl.xml +++ b/erts/doc/src/erl.xml @@ -379,6 +379,16 @@ <c><![CDATA[Host]]></c> is the fully qualified host name of the current host. For short names, use flag <c><![CDATA[-sname]]></c> instead.</p> + <warning> + <p> + Starting a distributed node without also specifying + <seealso marker="#proto_dist"><c>-proto_dist inet_tls</c></seealso> + will expose the node to attacks that may give the attacker + complete access to the node and in extension the cluster. + When using un-secure distributed nodes, make sure that the + network is configured to keep potential attackers out. + </p> + </warning> </item> <tag><c><![CDATA[-noinput]]></c></tag> <item> @@ -428,12 +438,17 @@ </item> <tag><c><![CDATA[-proto_dist Proto]]></c></tag> <item> + <marker id="proto_dist"/> <p>Specifies a protocol for Erlang distribution:</p> <taglist> <tag><c>inet_tcp</c></tag> <item>TCP over IPv4 (the default)</item> <tag><c>inet_tls</c></tag> - <item>Distribution over TLS/SSL</item> + <item>Distribution over TLS/SSL, See the + <seealso marker="ssl:ssl_distribution"> + Using SSL for Erlang Distribution</seealso> User's Guide + for details on how to setup a secure distributed node. + </item> <tag><c>inet6_tcp</c></tag> <item>TCP over IPv6</item> </taglist> @@ -497,6 +512,16 @@ exist between nodes running with flag <c><![CDATA[-sname]]></c> and those running with flag <c><![CDATA[-name]]></c>, as node names must be unique in distributed Erlang systems.</p> + <warning> + <p> + Starting a distributed node without also specifying + <seealso marker="#proto_dist"><c>-proto_dist inet_tls</c></seealso> + will expose the node to attacks that may give the attacker + complete access to the node and in extension the cluster. + When using un-secure distributed nodes, make sure that the + network is configured to keep potential attackers out. + </p> + </warning> </item> <tag><marker id="start_epmd"/><c>-start_epmd true | false</c></tag> <item> @@ -1528,32 +1553,27 @@ <item> <p><em>Unix systems</em>: This variable gives the number of seconds that the emulator is allowed to spend writing a crash dump. When the - given number of seconds have elapsed, the emulator is terminated by a - <c>SIGALRM</c> signal.</p> - <p>If the variable is <em>not</em> set or set to <c>0</c> seconds - (<c><![CDATA[ERL_CRASH_DUMP_SECONDS=0]]></c>), the runtime system does - not even attempt to write the crash dump file. It only terminates.</p> - <p>If the variable is set to negative value, such as - <c><![CDATA[ERL_CRASH_DUMP_SECONDS=-1]]></c>, the runtime system - waits indefinitely for the crash dump file to be written.</p> - <p>This variable is used with <seealso marker="kernel:heart"> - <c>heart(3)</c></seealso> if <c>heart</c> is running:</p> + given number of seconds have elapsed, the emulator is terminated.</p> <taglist> <tag><c><![CDATA[ERL_CRASH_DUMP_SECONDS=0]]></c></tag> - <item>Suppresses the writing a crash dump file entirely, thus - rebooting the runtime system immediately. This is the same as not - setting the environment variable. + <item>If the variable is set to <c>0</c> seconds, the runtime system does + not even attempt to write the crash dump file. It only terminates. + This is the default if option <c>-heart</c> is passed to <c>erl</c> + and <c>ERL_CRASH_DUMP_SECONDS</c> is not set. + </item> + <tag><c><![CDATA[ERL_CRASH_DUMP_SECONDS=S]]></c></tag> + <item>If the variable is set to a positive value <c>S</c>, + wait for <c>S</c> seconds to complete the crash dump file and + then terminates the runtime system with a <c>SIGALRM</c> signal. </item> <tag><c><![CDATA[ERL_CRASH_DUMP_SECONDS=-1]]></c></tag> - <item>Setting the environment variable to a negative value causes the - termination of the runtime system to wait until the crash dump file - has been completly written. - </item> - <tag><c><![CDATA[ERL_CRASH_DUMP_SECONDS=S]]></c></tag> - <item>Waits for <c>S</c> seconds to complete the crash dump file and - then terminates the runtime system. + <item>A negative value causes the termination of the runtime system + to wait indefinitely until the crash dump file has been completly + written. This is the default if option <c>-heart</c> is <em>not</em> + passed to <c>erl</c> and <c>ERL_CRASH_DUMP_SECONDS</c> is not set. </item> </taglist> + <p>See also <seealso marker="kernel:heart"><c>heart(3)</c></seealso>.</p> </item> <tag><c><![CDATA[ERL_CRASH_DUMP_BYTES]]></c></tag> <item> diff --git a/erts/doc/src/erl_dist_protocol.xml b/erts/doc/src/erl_dist_protocol.xml index ee74983730..8391408a2e 100644 --- a/erts/doc/src/erl_dist_protocol.xml +++ b/erts/doc/src/erl_dist_protocol.xml @@ -70,6 +70,17 @@ <p>The integers in all multibyte fields are in big-endian order.</p> + <warning> + <p> + The Erlang Distribution protocol is not by itself secure and does not + aim to be so. In order to get secure distribution the distributed nodes + should be configured to use distribution over tls. + See the <seealso marker="ssl:ssl_distribution"> + Using SSL for Erlang Distribution</seealso> User's Guide + for details on how to setup a secure distributed node. + </p> + </warning> + <section> <title>EPMD Protocol</title> <p>The requests served by the EPMD are summarized in the following diff --git a/erts/doc/src/erl_nif.xml b/erts/doc/src/erl_nif.xml index 51ba70994a..3eb3e04f33 100644 --- a/erts/doc/src/erl_nif.xml +++ b/erts/doc/src/erl_nif.xml @@ -714,6 +714,7 @@ typedef struct { typedef struct { ErlNifResourceDtor* dtor; ErlNifResourceStop* stop; + ErlNifResourceDown* down; } ErlNifResourceTypeInit;</code> <p>Initialization structure read by <seealso marker="#enif_open_resource_type_x"> enif_open_resource_type_x</seealso>.</p> @@ -1395,13 +1396,12 @@ typedef enum { <p>Returns <c>true</c> if a pending exception is associated with the environment <c>env</c>. If <c>reason</c> is a <c>NULL</c> pointer, ignore it. Otherwise, if a pending exception associated with - <c>env</c> exists, set <c>ERL_NIF_TERM</c> to which <c>reason</c> - points to the value of the exception's term. For example, if - <seealso marker="#enif_make_badarg"> + <c>env</c> exists, set <c>*reason</c> to the value of the exception + term. For example, if <seealso marker="#enif_make_badarg"> <c>enif_make_badarg</c></seealso> is called to set a pending <c>badarg</c> exception, a later call to <c>enif_has_pending_exception(env, &reason)</c> sets - <c>reason</c> to the atom <c>badarg</c>, then return <c>true</c>.</p> + <c>*reason</c> to the atom <c>badarg</c>, then return <c>true</c>.</p> <p>See also <seealso marker="#enif_make_badarg"> <c>enif_make_badarg</c></seealso> and <seealso marker="#enif_raise_exception"> @@ -2384,20 +2384,23 @@ enif_map_iterator_destroy(env, &iter);</code> called in the two callbacks <seealso marker="#load"><c>load</c></seealso> and <seealso marker="#upgrade"><c>upgrade</c></seealso>.</p> + <p>See also <seealso marker="#enif_open_resource_type_x"> + <c>enif_open_resource_type_x</c></seealso>.</p> </desc> </func> <func> <name><ret>ErlNifResourceType *</ret> <nametext>enif_open_resource_type_x(ErlNifEnv* env, const char* name, - ErlNifResourceTypeInit* init, + const ErlNifResourceTypeInit* init, ErlNifResourceFlags flags, ErlNifResourceFlags* tried)</nametext> </name> <fsummary>Create or takeover a resource type.</fsummary> <desc> <p>Same as <seealso marker="#enif_open_resource_type"><c>enif_open_resource_type</c></seealso> - except is also accept a <c>stop</c> callback for resource types that are - used together with <seealso marker="#enif_select"><c>enif_select</c></seealso>.</p> + except it accepts additional callback functions for resource types that are + used together with <seealso marker="#enif_select"><c>enif_select</c></seealso> + and <seealso marker="#enif_monitor_process"><c>enif_monitor_process</c></seealso>.</p> <p>Argument <c>init</c> is a pointer to an <seealso marker="#ErlNifResourceTypeInit"><c>ErlNifResourceTypeInit</c></seealso> structure that contains the function pointers for destructor, down and stop callbacks @@ -2695,6 +2698,21 @@ enif_map_iterator_destroy(env, &iter);</code> the event object. This safe way of closing event objects must be used even if all notifications have been received and no further calls to <c>enif_select</c> have been made.</p> + <p>The first call to <c>enif_select</c> for a specific OS <c>event</c> will establish + a relation between the event object and the containing resource. All subsequent calls + for an <c>event</c> must pass its containing resource as argument + <c>obj</c>. The relation is dissolved when <c>enif_select</c> has + been called with <c>mode</c> as <c>ERL_NIF_SELECT_STOP</c> and the + corresponding <c>stop</c> callback has returned. A resource can contain + several event objects but one event object can only be contained within + one resource. A resource will not be destructed until all its contained relations + have been dissolved.</p> + <note> + <p>Use <seealso marker="#enif_monitor_process"><c>enif_monitor_process</c></seealso> + together with <c>enif_select</c> to detect failing Erlang + processes and prevent them from causing permanent leakage of resources + and their contained OS event objects.</p> + </note> <p>Returns a non-negative value on success where the following bits can be set:</p> <taglist> <tag><c>ERL_NIF_SELECT_STOP_CALLED</c></tag> diff --git a/erts/doc/src/erl_tracer.xml b/erts/doc/src/erl_tracer.xml index 63feebb0b5..fd3c17f337 100644 --- a/erts/doc/src/erl_tracer.xml +++ b/erts/doc/src/erl_tracer.xml @@ -653,7 +653,7 @@ ok <0.37.0> 3> erlang:trace(new, true, [send,{tracer, erl_msg_tracer, Tracer}]). 0 -{<0.39.0>,<0.27.0>} +{trace,<0.39.0>,<0.27.0>} 4> {ok, D} = file:open("/tmp/tmp.data",[write]). {trace,#Port<0.486>,<0.40.0>} {trace,<0.40.0>,<0.21.0>} @@ -758,18 +758,21 @@ static ERL_NIF_TERM enabled(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) /* * argv[0]: TraceTag, should only be 'send' - * argv[1]: TracerState, process to send {argv[2], argv[4]} to + * argv[1]: TracerState, process to send {Tracee, Recipient} to * argv[2]: Tracee - * argv[3]: Recipient - * argv[4]: Options, ignored + * argv[3]: Message + * argv[4]: Options, map containing Recipient */ static ERL_NIF_TERM trace(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { ErlNifPid to_pid; + ERL_NIF_TERM recipient, msg; if (enif_get_local_pid(env, argv[1], &to_pid)) { - ERL_NIF_TERM msg = enif_make_tuple3(env, enif_make_atom(env, "trace"), argv[2], argv[4]); + if (enif_get_map_value(env, argv[4], enif_make_atom(env, "extra"), &recipient)) { + msg = enif_make_tuple3(env, enif_make_atom(env, "trace"), argv[2], recipient); enif_send(env, &to_pid, NULL, msg); + } } return enif_make_atom(env, "ok"); diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 6d165e9eff..d9cc5ef936 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -6148,28 +6148,60 @@ true</pre> <fsummary>Information about active processes and ports.</fsummary> <desc> <marker id="statistics_active_tasks"></marker> + <p>Returns the same as + <seealso marker="#statistics_active_tasks_all"> + <c>statistics(active_tasks_all)</c></seealso> + with the exception that no information about the dirty + IO run queue and its associated schedulers is part of + the result. That is, only tasks that are expected to be + CPU bound are part of the result.</p> + </desc> + </func> + + <func> + <name name="statistics" arity="1" clause_i="2"/> + <fsummary>Information about active processes and ports.</fsummary> + <desc> + <marker id="statistics_active_tasks_all"></marker> <p>Returns a list where each element represents the amount of active processes and ports on each run queue and its - associated scheduler. That is, the number of processes and - ports that are ready to run, or are currently running. The - element location in the list corresponds to the scheduler - and its run queue. The first element corresponds to scheduler - number 1 and so on. The information is <em>not</em> gathered - atomically. That is, the result is not necessarily a - consistent snapshot of the state, but instead quite - efficiently gathered.</p> + associated schedulers. That is, the number of processes and + ports that are ready to run, or are currently running. + Values for normal run queues and their associated schedulers + are located first in the resulting list. The first element + corresponds to scheduler number 1 and so on. If support for + dirty schedulers exist, an element with the value for the + dirty CPU run queue and its associated dirty CPU schedulers + follow and then as last element the value for the the dirty + IO run queue and its associated dirty IO schedulers follow. + The information is <em>not</em> gathered atomically. That is, + the result is not necessarily a consistent snapshot of the + state, but instead quite efficiently gathered.</p> + <note><p>Each normal scheduler has one run queue that it + manages. If dirty schedulers schedulers are supported, all + dirty CPU schedulers share one run queue, and all dirty IO + schedulers share one run queue. That is, we have multiple + normal run queues, one dirty CPU run queue and one dirty + IO run queue. Work can <em>not</em> migrate between the + different types of run queues. Only work in normal run + queues can migrate to other normal run queues. This has + to be taken into account when evaluating the result.</p></note> <p>See also <seealso marker="#statistics_total_active_tasks"> <c>statistics(total_active_tasks)</c></seealso>, <seealso marker="#statistics_run_queue_lengths"> - <c>statistics(run_queue_lengths)</c></seealso>, and + <c>statistics(run_queue_lengths)</c></seealso>, + <seealso marker="#statistics_run_queue_lengths_all"> + <c>statistics(run_queue_lengths_all)</c></seealso>, <seealso marker="#statistics_total_run_queue_lengths"> - <c>statistics(total_run_queue_lengths)</c></seealso>.</p> + <c>statistics(total_run_queue_lengths)</c></seealso>, and + <seealso marker="#statistics_total_run_queue_lengths_all"> + <c>statistics(total_run_queue_lengths_all)</c></seealso>.</p> </desc> </func> <func> - <name name="statistics" arity="1" clause_i="2"/> + <name name="statistics" arity="1" clause_i="3"/> <fsummary>Information about context switches.</fsummary> <desc> <p>Returns the total number of context switches since the @@ -6178,7 +6210,7 @@ true</pre> </func> <func> - <name name="statistics" arity="1" clause_i="3"/> + <name name="statistics" arity="1" clause_i="4"/> <fsummary>Information about exact reductions.</fsummary> <desc> <marker id="statistics_exact_reductions"></marker> @@ -6194,7 +6226,7 @@ true</pre> </func> <func> - <name name="statistics" arity="1" clause_i="4"/> + <name name="statistics" arity="1" clause_i="5"/> <fsummary>Information about garbage collection.</fsummary> <desc> <p>Returns information about garbage collection, for example:</p> @@ -6206,7 +6238,7 @@ true</pre> </func> <func> - <name name="statistics" arity="1" clause_i="5"/> + <name name="statistics" arity="1" clause_i="6"/> <fsummary>Information about I/O.</fsummary> <desc> <p>Returns <c><anno>Input</anno></c>, @@ -6217,7 +6249,7 @@ true</pre> </func> <func> - <name name="statistics" arity="1" clause_i="6"/> + <name name="statistics" arity="1" clause_i="7"/> <fsummary>Information about microstate accounting.</fsummary> <desc> <marker id="statistics_microstate_accounting"></marker> @@ -6353,7 +6385,7 @@ lists:map( </func> <func> - <name name="statistics" arity="1" clause_i="7"/> + <name name="statistics" arity="1" clause_i="8"/> <fsummary>Information about reductions.</fsummary> <desc> <marker id="statistics_reductions"></marker> @@ -6372,12 +6404,13 @@ lists:map( </func> <func> - <name name="statistics" arity="1" clause_i="8"/> + <name name="statistics" arity="1" clause_i="9"/> <fsummary>Information about the run-queues.</fsummary> <desc><marker id="statistics_run_queue"></marker> - <p>Returns the total length of the run-queues. That is, the number + <p>Returns the total length of all normal run-queues. That is, the number of processes and ports that are ready to run on all available - run-queues. The information is gathered atomically. That + normal run-queues. Dirty run queues are not part of the + result. The information is gathered atomically. That is, the result is a consistent snapshot of the state, but this operation is much more expensive compared to <seealso marker="#statistics_total_run_queue_lengths"> @@ -6387,29 +6420,63 @@ lists:map( </func> <func> - <name name="statistics" arity="1" clause_i="9"/> + <name name="statistics" arity="1" clause_i="10"/> <fsummary>Information about the run-queue lengths.</fsummary> <desc><marker id="statistics_run_queue_lengths"></marker> + <p>Returns the same as + <seealso marker="#statistics_run_queue_lengths_all"> + <c>statistics(run_queue_lengths_all)</c></seealso> + with the exception that no information about the dirty + IO run queue is part of the result. That is, only + run queues with work that is expected to be CPU bound + is part of the result.</p> + </desc> + </func> + + <func> + <name name="statistics" arity="1" clause_i="11"/> + <fsummary>Information about the run-queue lengths.</fsummary> + <desc><marker id="statistics_run_queue_lengths_all"></marker> <p>Returns a list where each element represents the amount - of processes and ports ready to run for each run queue. The - element location in the list corresponds to the run queue - of a scheduler. The first element corresponds to the run - queue of scheduler number 1 and so on. The information is - <em>not</em> gathered atomically. That is, the result is - not necessarily a consistent snapshot of the state, but - instead quite efficiently gathered.</p> + of processes and ports ready to run for each run queue. + Values for normal run queues are located first in the + resulting list. The first element corresponds to the + normal run queue of scheduler number 1 and so on. If + support for dirty schedulers exist, values for the dirty + CPU run queue and the dirty IO run queue follow (in that + order) at the end. The information is <em>not</em> + gathered atomically. That is, the result is not + necessarily a consistent snapshot of the state, but + instead quite efficiently gathered.</p> + <note><p>Each normal scheduler has one run queue that it + manages. If dirty schedulers schedulers are supported, all + dirty CPU schedulers share one run queue, and all dirty IO + schedulers share one run queue. That is, we have multiple + normal run queues, one dirty CPU run queue and one dirty + IO run queue. Work can <em>not</em> migrate between the + different types of run queues. Only work in normal run + queues can migrate to other normal run queues. This has + to be taken into account when evaluating the result.</p></note> <p>See also + <seealso marker="#statistics_run_queue_lengths"> + <c>statistics(run_queue_lengths)</c></seealso>, + <seealso marker="#statistics_total_run_queue_lengths_all"> + <c>statistics(total_run_queue_lengths_all)</c></seealso>, <seealso marker="#statistics_total_run_queue_lengths"> <c>statistics(total_run_queue_lengths)</c></seealso>, <seealso marker="#statistics_active_tasks"> - <c>statistics(active_tasks)</c></seealso>, and + <c>statistics(active_tasks)</c></seealso>, + <seealso marker="#statistics_active_tasks_all"> + <c>statistics(active_tasks_all)</c></seealso>, and <seealso marker="#statistics_total_active_tasks"> - <c>statistics(total_active_tasks)</c></seealso>.</p> + <c>statistics(total_active_tasks)</c></seealso>, + <seealso marker="#statistics_total_active_tasks_all"> + <c>statistics(total_active_tasks_all)</c></seealso>.</p> </desc> </func> <func> - <name name="statistics" arity="1" clause_i="10"/> + <name name="statistics" arity="1" clause_i="12"/> <fsummary>Information about runtime.</fsummary> <desc> <p>Returns information about runtime, in milliseconds.</p> @@ -6424,7 +6491,7 @@ lists:map( </func> <func> - <name name="statistics" arity="1" clause_i="11"/> + <name name="statistics" arity="1" clause_i="13"/> <fsummary>Information about each schedulers work time.</fsummary> <desc> <marker id="statistics_scheduler_wall_time"></marker> @@ -6545,7 +6612,7 @@ ok </func> <func> - <name name="statistics" arity="1" clause_i="12"/> + <name name="statistics" arity="1" clause_i="14"/> <fsummary>Information about each schedulers work time.</fsummary> <desc> <marker id="statistics_scheduler_wall_time_all"></marker> @@ -6570,47 +6637,47 @@ ok </desc> </func> <func> - <name name="statistics" arity="1" clause_i="13"/> + <name name="statistics" arity="1" clause_i="15"/> <fsummary>Information about active processes and ports.</fsummary> <desc><marker id="statistics_total_active_tasks"></marker> - <p>Returns the total amount of active processes and ports in - the system. That is, the number of processes and ports that - are ready to run, or are currently running. The information - is <em>not</em> gathered atomically. That is, the result - is not necessarily a consistent snapshot of the state, but - instead quite efficiently gathered.</p> - <p>See also - <seealso marker="#statistics_active_tasks"> - <c>statistics(active_tasks)</c></seealso>, - <seealso marker="#statistics_run_queue_lengths"> - <c>statistics(run_queue_lengths)</c></seealso>, and - <seealso marker="#statistics_total_run_queue_lengths"> - <c>statistics(total_run_queue_lengths)</c></seealso>.</p> + <p>The same as calling + <c>lists:sum(</c><seealso marker="#statistics_active_tasks"><c>statistics(active_tasks)</c></seealso><c>)</c>, + but more efficient.</p> </desc> </func> <func> - <name name="statistics" arity="1" clause_i="14"/> + <name name="statistics" arity="1" clause_i="16"/> + <fsummary>Information about active processes and ports.</fsummary> + <desc><marker id="statistics_total_active_tasks_all"></marker> + <p>The same as calling + <c>lists:sum(</c><seealso marker="#statistics_active_tasks_all"><c>statistics(active_tasks_all)</c></seealso><c>)</c>, + but more efficient.</p> + </desc> + </func> + + <func> + <name name="statistics" arity="1" clause_i="17"/> <fsummary>Information about the run-queue lengths.</fsummary> <desc><marker id="statistics_total_run_queue_lengths"></marker> - <p>Returns the total length of the run queues. That is, the number - of processes and ports that are ready to run on all available - run queues. The information is <em>not</em> gathered atomically. - That is, the result is not necessarily a consistent snapshot of - the state, but much more efficiently gathered compared to - <seealso marker="#statistics_run_queue"> - <c>statistics(run_queue)</c></seealso>.</p> - <p>See also <seealso marker="#statistics_run_queue_lengths"> - <c>statistics(run_queue_lengths)</c></seealso>, - <seealso marker="#statistics_total_active_tasks"> - <c>statistics(total_active_tasks)</c></seealso>, and - <seealso marker="#statistics_active_tasks"> - <c>statistics(active_tasks)</c></seealso>.</p> + <p>The same as calling + <c>lists:sum(</c><seealso marker="#statistics_run_queue_lengths"><c>statistics(run_queue_lengths)</c></seealso><c>)</c>, + but more efficient.</p> </desc> </func> <func> - <name name="statistics" arity="1" clause_i="15"/> + <name name="statistics" arity="1" clause_i="18"/> + <fsummary>Information about the run-queue lengths.</fsummary> + <desc><marker id="statistics_total_run_queue_lengths_all"></marker> + <p>The same as calling + <c>lists:sum(</c><seealso marker="#statistics_run_queue_lengths_all"><c>statistics(run_queue_lengths_all)</c></seealso><c>)</c>, + but more efficient.</p> + </desc> + </func> + + <func> + <name name="statistics" arity="1" clause_i="19"/> <fsummary>Information about wall clock.</fsummary> <desc> <p>Returns information about wall clock. <c>wall_clock</c> can diff --git a/erts/emulator/beam/atom.names b/erts/emulator/beam/atom.names index 477a7676d6..a44d23b181 100644 --- a/erts/emulator/beam/atom.names +++ b/erts/emulator/beam/atom.names @@ -76,6 +76,7 @@ atom ac atom accessor atom active atom active_tasks +atom active_tasks_all atom alive atom all atom all_but_first @@ -567,6 +568,7 @@ atom return_to atom return_trace atom run_queue atom run_queue_lengths +atom run_queue_lengths_all atom runnable atom runnable_ports atom runnable_procs @@ -656,8 +658,10 @@ atom Times='*' atom timestamp atom total atom total_active_tasks +atom total_active_tasks_all atom total_heap_size atom total_run_queue_lengths +atom total_run_queue_lengths_all atom tpkt atom trace trace_ts traced atom trace_control_word diff --git a/erts/emulator/beam/beam_emu.c b/erts/emulator/beam/beam_emu.c index a90e6a0ba8..79d751d13e 100644 --- a/erts/emulator/beam/beam_emu.c +++ b/erts/emulator/beam/beam_emu.c @@ -158,7 +158,9 @@ do { \ /* * Register target (X or Y register). */ -#define REG_TARGET(Target) (*(((Target) & 1) ? &yb(Target-1) : &xb(Target))) + +#define REG_TARGET_PTR(Target) (((Target) & 1) ? &yb(Target-1) : &xb(Target)) +#define REG_TARGET(Target) (*REG_TARGET_PTR(Target)) /* * Store a result into a register given a destination descriptor. @@ -172,8 +174,6 @@ do { \ REG_TARGET(stb_reg) = (Result); \ } while (0) -#define StoreSimpleDest(Src, Dest) Dest = (Src) - /* * Store a result into a register and execute the next instruction. * Dst points to the word with a destination descriptor, which MUST @@ -420,7 +420,7 @@ void** beam_ops; #define TestHeapPutList(Need, Reg) \ do { \ TestHeap((Need), 1); \ - PutList(Reg, r(0), r(0), StoreSimpleDest); \ + PutList(Reg, r(0), r(0)); \ CHECK_TERM(r(0)); \ } while (0) @@ -547,11 +547,11 @@ void** beam_ops; GetR((N)+1, Dst2); \ } while (0) -#define PutList(H, T, Dst, Store) \ - do { \ - HTOP[0] = (H); HTOP[1] = (T); \ - Store(make_list(HTOP), Dst); \ - HTOP += 2; \ +#define PutList(H, T, Dst) \ + do { \ + HTOP[0] = (H); HTOP[1] = (T); \ + Dst = make_list(HTOP); \ + HTOP += 2; \ } while (0) #define Swap(R1, R2) \ @@ -568,11 +568,7 @@ void** beam_ops; R2 = Tmp = V; \ } while (0) -#define Move(Src, Dst, Store) \ - do { \ - Eterm term = (Src); \ - Store(term, Dst); \ - } while (0) +#define Move(Src, Dst) Dst = (Src) #define Move2Par(S1, D1, S2, D2) \ do { \ @@ -911,7 +907,7 @@ do { \ Target = _uint_size * Unit; \ } while (0) -#define BsGetFloat2(Ms, Live, Sz, Flags, Dst, Store, Fail) \ +#define BsGetFloat2(Ms, Live, Sz, Flags, Dst, Fail) \ do { \ ErlBinMatchBuffer *_mb; \ Eterm _result; Sint _size; \ @@ -922,12 +918,12 @@ do { \ LIGHT_SWAPOUT; \ _result = erts_bs_get_float_2(c_p, _size, (Flags), _mb); \ LIGHT_SWAPIN; \ - HEAP_SPACE_VERIFIED(0); \ + HEAP_SPACE_VERIFIED(0); \ if (is_non_value(_result)) { Fail; } \ - else { Store(_result, Dst); } \ + else { Dst = _result; } \ } while (0) -#define BsGetBinaryImm_2(Ms, Live, Sz, Flags, Dst, Store, Fail) \ +#define BsGetBinaryImm_2(Ms, Live, Sz, Flags, Dst, Fail) \ do { \ ErlBinMatchBuffer *_mb; \ Eterm _result; \ @@ -936,12 +932,12 @@ do { \ LIGHT_SWAPOUT; \ _result = erts_bs_get_binary_2(c_p, (Sz), (Flags), _mb); \ LIGHT_SWAPIN; \ - HEAP_SPACE_VERIFIED(0); \ + HEAP_SPACE_VERIFIED(0); \ if (is_non_value(_result)) { Fail; } \ - else { Store(_result, Dst); } \ + else { Dst = _result; } \ } while (0) -#define BsGetBinary_2(Ms, Live, Sz, Flags, Dst, Store, Fail) \ +#define BsGetBinary_2(Ms, Live, Sz, Flags, Dst, Fail) \ do { \ ErlBinMatchBuffer *_mb; \ Eterm _result; Uint _size; \ @@ -951,27 +947,27 @@ do { \ LIGHT_SWAPOUT; \ _result = erts_bs_get_binary_2(c_p, _size, (Flags), _mb); \ LIGHT_SWAPIN; \ - HEAP_SPACE_VERIFIED(0); \ + HEAP_SPACE_VERIFIED(0); \ if (is_non_value(_result)) { Fail; } \ - else { Store(_result, Dst); } \ + else { Dst = _result; } \ } while (0) -#define BsGetBinaryAll_2(Ms, Live, Unit, Dst, Store, Fail) \ - do { \ - ErlBinMatchBuffer *_mb; \ - Eterm _result; \ - TestHeap(ERL_SUB_BIN_SIZE, Live); \ - _mb = ms_matchbuffer(Ms); \ - if (((_mb->size - _mb->offset) % Unit) == 0) { \ - LIGHT_SWAPOUT; \ - _result = erts_bs_get_binary_all_2(c_p, _mb); \ - LIGHT_SWAPIN; \ - HEAP_SPACE_VERIFIED(0); \ - ASSERT(is_value(_result)); \ - Store(_result, Dst); \ - } else { \ - HEAP_SPACE_VERIFIED(0); \ - Fail; } \ +#define BsGetBinaryAll_2(Ms, Live, Unit, Dst, Fail) \ + do { \ + ErlBinMatchBuffer *_mb; \ + Eterm _result; \ + TestHeap(ERL_SUB_BIN_SIZE, Live); \ + _mb = ms_matchbuffer(Ms); \ + if (((_mb->size - _mb->offset) % Unit) == 0) { \ + LIGHT_SWAPOUT; \ + _result = erts_bs_get_binary_all_2(c_p, _mb); \ + LIGHT_SWAPIN; \ + HEAP_SPACE_VERIFIED(0); \ + ASSERT(is_value(_result)); \ + Dst = _result; \ + } else { \ + HEAP_SPACE_VERIFIED(0); \ + Fail; } \ } while (0) #define BsSkipBits2(Ms, Bits, Unit, Fail) \ diff --git a/erts/emulator/beam/bif.c b/erts/emulator/beam/bif.c index 7134d2da56..40dd4129d2 100644 --- a/erts/emulator/beam/bif.c +++ b/erts/emulator/beam/bif.c @@ -2047,8 +2047,6 @@ ebif_bang_2(BIF_ALIST_2) #define SEND_YIELD_CONTINUE (-8) -Sint do_send(Process *p, Eterm to, Eterm msg, Eterm *refp, ErtsSendContext*); - static Sint remote_send(Process *p, DistEntry *dep, Eterm to, Eterm full_to, Eterm msg, ErtsSendContext* ctx) @@ -2102,8 +2100,8 @@ static Sint remote_send(Process *p, DistEntry *dep, return res; } -Sint -do_send(Process *p, Eterm to, Eterm msg, Eterm *refp, ErtsSendContext* ctx) +static Sint +do_send(Process *p, Eterm to, Eterm msg, Eterm *refp, ErtsSendContext *ctx) { Eterm portid; Port *pt; diff --git a/erts/emulator/beam/erl_alloc_util.c b/erts/emulator/beam/erl_alloc_util.c index 4889fac923..6fddba4b34 100644 --- a/erts/emulator/beam/erl_alloc_util.c +++ b/erts/emulator/beam/erl_alloc_util.c @@ -821,7 +821,7 @@ static void clear_literal_range(void* start, Uint size) #if HAVE_ERTS_MSEG -void* +static void* erts_alcu_mseg_alloc(Allctr_t *allctr, Uint *size_p, Uint flags) { void *res; @@ -832,7 +832,7 @@ erts_alcu_mseg_alloc(Allctr_t *allctr, Uint *size_p, Uint flags) return res; } -void* +static void* erts_alcu_mseg_realloc(Allctr_t *allctr, void *seg, Uint old_size, Uint *new_size_p) { @@ -845,7 +845,7 @@ erts_alcu_mseg_realloc(Allctr_t *allctr, void *seg, return res; } -void +static void erts_alcu_mseg_dealloc(Allctr_t *allctr, void *seg, Uint size, Uint flags) { erts_mseg_dealloc_opt(allctr->alloc_no, seg, (UWord) size, flags, &allctr->mseg_opt); @@ -996,7 +996,7 @@ erts_alcu_exec_mseg_dealloc(Allctr_t *allctr, void *seg, Uint size, Uint flags) #endif /* HAVE_ERTS_MSEG */ -void* +static void* erts_alcu_sys_alloc(Allctr_t *allctr, Uint* size_p, int superalign) { void *res; @@ -1013,7 +1013,7 @@ erts_alcu_sys_alloc(Allctr_t *allctr, Uint* size_p, int superalign) return res; } -void* +static void* erts_alcu_sys_realloc(Allctr_t *allctr, void *ptr, Uint *size_p, Uint old_size, int superalign) { void *res; @@ -1035,7 +1035,7 @@ erts_alcu_sys_realloc(Allctr_t *allctr, void *ptr, Uint *size_p, Uint old_size, return res; } -void +static void erts_alcu_sys_dealloc(Allctr_t *allctr, void *ptr, Uint size, int superalign) { #if ERTS_SA_MB_CARRIERS && ERTS_HAVE_ERTS_SYS_ALIGNED_ALLOC diff --git a/erts/emulator/beam/erl_alloc_util.h b/erts/emulator/beam/erl_alloc_util.h index f570703f25..7a96bd0319 100644 --- a/erts/emulator/beam/erl_alloc_util.h +++ b/erts/emulator/beam/erl_alloc_util.h @@ -195,10 +195,6 @@ extern UWord erts_literal_vspace_map[]; # define ERTS_VSPACE_WORD_BITS (sizeof(UWord)*8) #endif -void* erts_alcu_mseg_alloc(Allctr_t*, Uint *size_p, Uint flags); -void* erts_alcu_mseg_realloc(Allctr_t*, void *seg, Uint old_size, Uint *new_size_p); -void erts_alcu_mseg_dealloc(Allctr_t*, void *seg, Uint size, Uint flags); - #if HAVE_ERTS_MSEG # if defined(ARCH_32) void* erts_alcu_literal_32_mseg_alloc(Allctr_t*, Uint *size_p, Uint flags); @@ -218,9 +214,6 @@ void erts_alcu_exec_mseg_dealloc(Allctr_t*, void *seg, Uint size, Uint flags); # endif #endif /* HAVE_ERTS_MSEG */ -void* erts_alcu_sys_alloc(Allctr_t*, Uint *size_p, int superalign); -void* erts_alcu_sys_realloc(Allctr_t*, void *ptr, Uint *size_p, Uint old_size, int superalign); -void erts_alcu_sys_dealloc(Allctr_t*, void *ptr, Uint size, int superalign); #ifdef ARCH_32 void* erts_alcu_literal_32_sys_alloc(Allctr_t*, Uint *size_p, int superalign); void* erts_alcu_literal_32_sys_realloc(Allctr_t*, void *ptr, Uint *size_p, Uint old_size, int superalign); diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index 5fc70dfc02..e2773475b0 100644 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -3450,9 +3450,15 @@ BIF_RETTYPE statistics_1(BIF_ALIST_1) if (is_non_value(res)) BIF_RET(am_undefined); BIF_TRAP1(gather_sched_wall_time_res_trap, BIF_P, res); - } else if (BIF_ARG_1 == am_total_active_tasks - || BIF_ARG_1 == am_total_run_queue_lengths) { - Uint no = erts_run_queues_len(NULL, 0, BIF_ARG_1 == am_total_active_tasks); + } else if ((BIF_ARG_1 == am_total_active_tasks) + | (BIF_ARG_1 == am_total_run_queue_lengths) + | (BIF_ARG_1 == am_total_active_tasks_all) + | (BIF_ARG_1 == am_total_run_queue_lengths_all)) { + Uint no = erts_run_queues_len(NULL, 0, + ((BIF_ARG_1 == am_total_active_tasks) + | (BIF_ARG_1 == am_total_active_tasks_all)), + ((BIF_ARG_1 == am_total_active_tasks_all) + | (BIF_ARG_1 == am_total_run_queue_lengths_all))); if (IS_USMALL(0, no)) res = make_small(no); else { @@ -3460,13 +3466,21 @@ BIF_RETTYPE statistics_1(BIF_ALIST_1) res = uint_to_big(no, hp); } BIF_RET(res); - } else if (BIF_ARG_1 == am_active_tasks - || BIF_ARG_1 == am_run_queue_lengths) { + } else if ((BIF_ARG_1 == am_active_tasks) + | (BIF_ARG_1 == am_run_queue_lengths) + | (BIF_ARG_1 == am_active_tasks_all) + | (BIF_ARG_1 == am_run_queue_lengths_all)) { Eterm res, *hp, **hpp; Uint sz, *szp; - int no_qs = erts_no_run_queues; + int incl_dirty_io = ((BIF_ARG_1 == am_active_tasks_all) + | (BIF_ARG_1 == am_run_queue_lengths_all)); + int no_qs = (erts_no_run_queues + ERTS_NUM_DIRTY_CPU_RUNQS + + (incl_dirty_io ? ERTS_NUM_DIRTY_IO_RUNQS : 0)); Uint *qszs = erts_alloc(ERTS_ALC_T_TMP,sizeof(Uint)*no_qs*2); - (void) erts_run_queues_len(qszs, 0, BIF_ARG_1 == am_active_tasks); + (void) erts_run_queues_len(qszs, 0, + ((BIF_ARG_1 == am_active_tasks) + | (BIF_ARG_1 == am_active_tasks_all)), + incl_dirty_io); sz = 0; szp = &sz; hpp = NULL; @@ -3539,7 +3553,7 @@ BIF_RETTYPE statistics_1(BIF_ALIST_1) res = TUPLE2(hp, b1, b2); BIF_RET(res); } else if (BIF_ARG_1 == am_run_queue) { - res = erts_run_queues_len(NULL, 1, 0); + res = erts_run_queues_len(NULL, 1, 0, 0); BIF_RET(make_small(res)); } else if (BIF_ARG_1 == am_wall_clock) { UWord w1, w2; @@ -3557,9 +3571,9 @@ BIF_RETTYPE statistics_1(BIF_ALIST_1) else if (ERTS_IS_ATOM_STR("run_queues", BIF_ARG_1)) { Eterm res, *hp, **hpp; Uint sz, *szp; - int no_qs = erts_no_run_queues; + int no_qs = erts_no_run_queues + ERTS_NUM_DIRTY_RUNQS; Uint *qszs = erts_alloc(ERTS_ALC_T_TMP,sizeof(Uint)*no_qs*2); - (void) erts_run_queues_len(qszs, 0, 0); + (void) erts_run_queues_len(qszs, 0, 0, 1); sz = 0; szp = &sz; hpp = NULL; diff --git a/erts/emulator/beam/erl_bits.c b/erts/emulator/beam/erl_bits.c index eea55b3239..71c64997c1 100644 --- a/erts/emulator/beam/erl_bits.c +++ b/erts/emulator/beam/erl_bits.c @@ -1636,7 +1636,7 @@ erts_bs_get_unaligned_uint32(ErlBinMatchBuffer* mb) return LSB[0] | (LSB[1]<<8) | (LSB[2]<<16) | (LSB[3]<<24); } -void +static void erts_align_utf8_bytes(ErlBinMatchBuffer* mb, byte* buf) { Uint bits = mb->size - mb->offset; diff --git a/erts/emulator/beam/erl_bits.h b/erts/emulator/beam/erl_bits.h index 632855255e..5da2b28a89 100644 --- a/erts/emulator/beam/erl_bits.h +++ b/erts/emulator/beam/erl_bits.h @@ -202,7 +202,6 @@ void erts_new_bs_put_string(ERL_BITS_PROTO_2(byte* iptr, Uint num_bytes)); Uint erts_bits_bufs_size(void); Uint32 erts_bs_get_unaligned_uint32(ErlBinMatchBuffer* mb); -void erts_align_utf8_bytes(ErlBinMatchBuffer* mb, byte* buf); Eterm erts_bs_get_utf8(ErlBinMatchBuffer* mb); Eterm erts_bs_get_utf16(ErlBinMatchBuffer* mb, Uint flags); Eterm erts_bs_append(Process* p, Eterm* reg, Uint live, Eterm build_size_term, diff --git a/erts/emulator/beam/erl_cpu_topology.c b/erts/emulator/beam/erl_cpu_topology.c index 4347f9f2b7..d0fd763798 100644 --- a/erts/emulator/beam/erl_cpu_topology.c +++ b/erts/emulator/beam/erl_cpu_topology.c @@ -2243,45 +2243,6 @@ add_cpu_groups(int groups, return cgm; } -static void -remove_cpu_groups(erts_cpu_groups_callback_t callback, void *arg) -{ - erts_cpu_groups_map_t *prev_cgm, *cgm; - erts_cpu_groups_callback_list_t *prev_cgcl, *cgcl; - - ERTS_SMP_LC_ASSERT(erts_lc_rwmtx_is_rwlocked(&cpuinfo_rwmtx)); - - no_cpu_groups_callbacks--; - - prev_cgm = NULL; - for (cgm = cpu_groups_maps; cgm; cgm = cgm->next) { - prev_cgcl = NULL; - for (cgcl = cgm->callback_list; cgcl; cgcl = cgcl->next) { - if (cgcl->callback == callback && cgcl->arg == arg) { - if (prev_cgcl) - prev_cgcl->next = cgcl->next; - else - cgm->callback_list = cgcl->next; - erts_free(ERTS_ALC_T_CPU_GRPS_MAP, cgcl); - if (!cgm->callback_list) { - if (prev_cgm) - prev_cgm->next = cgm->next; - else - cpu_groups_maps = cgm->next; - if (cgm->array) - erts_free(ERTS_ALC_T_CPU_GRPS_MAP, cgm->array); - erts_free(ERTS_ALC_T_CPU_GRPS_MAP, cgm); - } - return; - } - prev_cgcl = cgcl; - } - prev_cgm = cgm; - } - - erts_exit(ERTS_ABORT_EXIT, "Cpu groups not found\n"); -} - static int cpu_groups_lookup(erts_cpu_groups_map_t *map, ErtsSchedulerData *esdp) @@ -2321,21 +2282,3 @@ update_cpu_groups_maps(void) for (cgm = cpu_groups_maps; cgm; cgm = cgm->next) make_cpu_groups_map(cgm, 0); } - -void -erts_add_cpu_groups(int groups, - erts_cpu_groups_callback_t callback, - void *arg) -{ - erts_smp_rwmtx_rwlock(&cpuinfo_rwmtx); - add_cpu_groups(groups, callback, arg); - erts_smp_rwmtx_rwunlock(&cpuinfo_rwmtx); -} - -void erts_remove_cpu_groups(erts_cpu_groups_callback_t callback, - void *arg) -{ - erts_smp_rwmtx_rwlock(&cpuinfo_rwmtx); - remove_cpu_groups(callback, arg); - erts_smp_rwmtx_rwunlock(&cpuinfo_rwmtx); -} diff --git a/erts/emulator/beam/erl_db_hash.c b/erts/emulator/beam/erl_db_hash.c index 08a0f0e83b..0addfaa3c7 100644 --- a/erts/emulator/beam/erl_db_hash.c +++ b/erts/emulator/beam/erl_db_hash.c @@ -676,7 +676,7 @@ int db_create_hash(Process *p, DbTable *tbl) sizeof(DbTableHashFineLocks)); for (i=0; i<DB_HASH_LOCK_CNT; ++i) { erts_smp_rwmtx_init_opt_x(&tb->locks->lck_vec[i].lck, &rwmtx_opt, - "db_hash_slot", make_small(i)); + "db_hash_slot", tb->common.the_name); } /* This important property is needed to guarantee the two buckets * involved in a grow/shrink operation it protected by the same lock: diff --git a/erts/emulator/beam/erl_init.c b/erts/emulator/beam/erl_init.c index ac99f043e6..6172595552 100644 --- a/erts/emulator/beam/erl_init.c +++ b/erts/emulator/beam/erl_init.c @@ -431,7 +431,7 @@ erl_first_process_otp(char* modname, void* code, unsigned size, int argc, char** } static Eterm -erl_system_process_otp(Eterm parent_pid, char* modname) +erl_system_process_otp(Eterm parent_pid, char* modname, int off_heap_msgq) { Eterm start_mod; Process* parent; @@ -447,6 +447,8 @@ erl_system_process_otp(Eterm parent_pid, char* modname) parent = erts_pid2proc(NULL, 0, parent_pid, ERTS_PROC_LOCK_MAIN); so.flags = erts_default_spo_flags|SPO_SYSTEM_PROC; + if (off_heap_msgq) + so.flags |= SPO_OFF_HEAP_MSGQ; res = erl_create_process(parent, start_mod, am_start, NIL, &so); erts_smp_proc_unlock(parent, ERTS_PROC_LOCK_MAIN); return res; @@ -2326,14 +2328,14 @@ erl_start(int argc, char **argv) */ Eterm pid; - pid = erl_system_process_otp(otp_ring0_pid, "erts_code_purger"); + pid = erl_system_process_otp(otp_ring0_pid, "erts_code_purger", !0); erts_code_purger = (Process *) erts_ptab_pix2intptr_ddrb(&erts_proc, internal_pid_index(pid)); ASSERT(erts_code_purger && erts_code_purger->common.id == pid); erts_proc_inc_refc(erts_code_purger); - pid = erl_system_process_otp(otp_ring0_pid, "erts_literal_area_collector"); + pid = erl_system_process_otp(otp_ring0_pid, "erts_literal_area_collector", !0); erts_literal_area_collector = (Process *) erts_ptab_pix2intptr_ddrb(&erts_proc, internal_pid_index(pid)); @@ -2342,7 +2344,7 @@ erl_start(int argc, char **argv) erts_proc_inc_refc(erts_literal_area_collector); #ifdef ERTS_DIRTY_SCHEDULERS - pid = erl_system_process_otp(otp_ring0_pid, "erts_dirty_process_code_checker"); + pid = erl_system_process_otp(otp_ring0_pid, "erts_dirty_process_code_checker", !0); erts_dirty_process_code_checker = (Process *) erts_ptab_pix2intptr_ddrb(&erts_proc, internal_pid_index(pid)); diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index da27c7e7c6..d4385e3987 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -405,13 +405,59 @@ static erts_atomic_t runq_supervisor_sleeping; ErtsSchedulerData *erts_scheduler_data; #endif -ErtsAlignedRunQueue *erts_aligned_run_queues; -Uint erts_no_run_queues; +ErtsAlignedRunQueue * ERTS_WRITE_UNLIKELY(erts_aligned_run_queues); +Uint ERTS_WRITE_UNLIKELY(erts_no_run_queues); -ErtsAlignedSchedulerData *erts_aligned_scheduler_data; #ifdef ERTS_DIRTY_SCHEDULERS -ErtsAlignedSchedulerData *erts_aligned_dirty_cpu_scheduler_data; -ErtsAlignedSchedulerData *erts_aligned_dirty_io_scheduler_data; + +struct { + union { + erts_smp_atomic32_t active; + char align__[ERTS_CACHE_LINE_SIZE]; + } cpu; + union { + erts_smp_atomic32_t active; + char align__[ERTS_CACHE_LINE_SIZE]; + } io; +} dirty_count erts_align_attribute(ERTS_CACHE_LINE_SIZE); + +#endif + +static ERTS_INLINE void +dirty_active(ErtsSchedulerData *esdp, erts_aint32_t add) +{ +#ifdef ERTS_DIRTY_SCHEDULERS + erts_aint32_t val; + erts_smp_atomic32_t *ap; + switch (esdp->type) { + case ERTS_SCHED_DIRTY_CPU: + ap = &dirty_count.cpu.active; + break; + case ERTS_SCHED_DIRTY_IO: + ap = &dirty_count.io.active; + break; + default: + ap = NULL; + ERTS_INTERNAL_ERROR("Not a dirty scheduler"); + break; + } + + /* + * All updates done under run-queue lock, so + * no inc or dec needed... + */ + ERTS_SMP_ASSERT(erts_smp_lc_runq_is_locked(esdp->run_queue)); + + val = erts_smp_atomic32_read_nob(ap); + val += add; + erts_smp_atomic32_set_nob(ap, val); +#endif +} + +ErtsAlignedSchedulerData * ERTS_WRITE_UNLIKELY(erts_aligned_scheduler_data); +#ifdef ERTS_DIRTY_SCHEDULERS +ErtsAlignedSchedulerData * ERTS_WRITE_UNLIKELY(erts_aligned_dirty_cpu_scheduler_data); +ErtsAlignedSchedulerData * ERTS_WRITE_UNLIKELY(erts_aligned_dirty_io_scheduler_data); typedef union { Process dsp; char align[ERTS_ALC_CACHE_LINE_ALIGN_SIZE(sizeof(Process))]; @@ -539,22 +585,28 @@ do { \ } \ } while (0) -#define ERTS_ATOMIC_FOREACH_RUNQ_X(RQVAR, DO, DOX) \ +#define ERTS_ATOMIC_FOREACH_RUNQ_X(RQVAR, NRQS, DO, DOX) \ do { \ ErtsRunQueue *RQVAR; \ + int nrqs = (NRQS); \ int ix__; \ - for (ix__ = 0; ix__ < erts_no_run_queues; ix__++) { \ + for (ix__ = 0; ix__ < nrqs; ix__++) { \ RQVAR = ERTS_RUNQ_IX(ix__); \ erts_smp_runq_lock(RQVAR); \ { DO; } \ } \ { DOX; } \ - for (ix__ = 0; ix__ < erts_no_run_queues; ix__++) \ + for (ix__ = 0; ix__ < nrqs; ix__++) \ erts_smp_runq_unlock(ERTS_RUNQ_IX(ix__)); \ } while (0) -#define ERTS_ATOMIC_FOREACH_RUNQ(RQVAR, DO) \ - ERTS_ATOMIC_FOREACH_RUNQ_X(RQVAR, DO, ) +#define ERTS_ATOMIC_FOREACH_RUNQ(RQVAR, DO) \ + ERTS_ATOMIC_FOREACH_RUNQ_X(RQVAR, erts_no_run_queues + ERTS_NUM_DIRTY_RUNQS, DO, ) + +#define ERTS_ATOMIC_FOREACH_NORMAL_RUNQ(RQVAR, DO) \ + ERTS_ATOMIC_FOREACH_RUNQ_X(RQVAR, erts_no_run_queues, DO, ) + + /* * Local functions. */ @@ -2971,7 +3023,7 @@ erts_active_schedulers(void) { Uint as = erts_no_schedulers; - ERTS_ATOMIC_FOREACH_RUNQ(rq, as -= abs(rq->waiting)); + ERTS_ATOMIC_FOREACH_NORMAL_RUNQ(rq, as -= abs(rq->waiting)); return as; } @@ -3383,6 +3435,7 @@ scheduler_wait(int *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) rq->sleepers.list->prev = ssi; rq->sleepers.list = ssi; erts_smp_spin_unlock(&rq->sleepers.lock); + dirty_active(esdp, -1); } #endif @@ -3724,6 +3777,9 @@ scheduler_wait(int *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) sched_active_sys(esdp->no, rq); } + if (ERTS_SCHEDULER_IS_DIRTY(esdp)) + dirty_active(esdp, 1); + ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(rq)); } @@ -6123,7 +6179,7 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online #endif ) { - int ix, n, no_ssi; + int ix, n, no_ssi, tot_rqs; char *daww_ptr; size_t daww_sz; size_t size_runqs; @@ -6156,26 +6212,19 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online /* Create and initialize run queues */ n = no_schedulers; - size_runqs = sizeof(ErtsAlignedRunQueue) * (n + ERTS_NUM_DIRTY_RUNQS); + tot_rqs = (n + ERTS_NUM_DIRTY_RUNQS); + size_runqs = sizeof(ErtsAlignedRunQueue) * tot_rqs; erts_aligned_run_queues = erts_alloc_permanent_cache_aligned(ERTS_ALC_T_RUNQS, size_runqs); #ifdef ERTS_SMP -#ifdef ERTS_DIRTY_SCHEDULERS - erts_aligned_run_queues += ERTS_NUM_DIRTY_RUNQS; -#endif erts_smp_atomic32_init_nob(&no_empty_run_queues, 0); #endif erts_no_run_queues = n; - for (ix = -(ERTS_NUM_DIRTY_RUNQS); ix < n; ix++) { + for (ix = 0; ix < tot_rqs; ix++) { int pix, rix; -#ifdef ERTS_DIRTY_SCHEDULERS - ErtsRunQueue *rq = ERTS_RUNQ_IX_IS_DIRTY(ix) ? - ERTS_DIRTY_RUNQ_IX(ix) : ERTS_RUNQ_IX(ix); -#else ErtsRunQueue *rq = ERTS_RUNQ_IX(ix); -#endif rq->ix = ix; @@ -6448,6 +6497,11 @@ erts_init_scheduling(int no_schedulers, int no_schedulers_online ERTS_SCHED_DIRTY_IO, no_dirty_io_schedulers); + erts_smp_atomic32_init_nob(&dirty_count.cpu.active, + (erts_aint32_t) no_dirty_cpu_schedulers); + erts_smp_atomic32_init_nob(&dirty_count.io.active, + (erts_aint32_t) no_dirty_io_schedulers); + #endif if (set_schdlr_sspnd_change_flags) @@ -7822,6 +7876,7 @@ suspend_scheduler(ErtsSchedulerData *esdp) #endif if (sched_type != ERTS_SCHED_NORMAL) { + dirty_active(esdp, -1); erts_smp_runq_unlock(esdp->run_queue); dirty_sched_wall_time_change(esdp, 0); } @@ -8130,7 +8185,9 @@ suspend_scheduler(ErtsSchedulerData *esdp) erts_smp_runq_lock(esdp->run_queue); non_empty_runq(esdp->run_queue); - if (sched_type == ERTS_SCHED_NORMAL) { + if (sched_type != ERTS_SCHED_NORMAL) + dirty_active(esdp, 1); + else { schedule_bound_processes(esdp->run_queue, &sbp); erts_sched_check_cpu_bind_post_suspend(esdp); @@ -9764,38 +9821,69 @@ erts_internal_is_process_executing_dirty_1(BIF_ALIST_1) BIF_RET(am_false); } +static ERTS_INLINE void +run_queues_len_aux(ErtsRunQueue *rq, Uint *tot_len, Uint *qlen, int *ip, int incl_active_sched, int locked) +{ + Sint rq_len; + + if (locked) + rq_len = (Sint) erts_smp_atomic32_read_dirty(&rq->len); + else + rq_len = (Sint) erts_smp_atomic32_read_nob(&rq->len); + ASSERT(rq_len >= 0); + + if (incl_active_sched) { +#ifdef ERTS_DIRTY_SCHEDULERS + if (ERTS_RUNQ_IX_IS_DIRTY(rq->ix)) { + erts_aint32_t dcnt; + if (ERTS_RUNQ_IS_DIRTY_CPU_RUNQ(rq)) { + dcnt = erts_smp_atomic32_read_nob(&dirty_count.cpu.active); + ASSERT(0 <= dcnt && dcnt <= erts_no_dirty_cpu_schedulers); + } + else { + ASSERT(ERTS_RUNQ_IS_DIRTY_IO_RUNQ(rq)); + dcnt = erts_smp_atomic32_read_nob(&dirty_count.io.active); + ASSERT(0 <= dcnt && dcnt <= erts_no_dirty_io_schedulers); + } + rq_len += (Sint) dcnt; + } + else +#endif + { + if (ERTS_RUNQ_FLGS_GET_NOB(rq) & ERTS_RUNQ_FLG_EXEC) + rq_len++; + } + } + if (qlen) + qlen[(*ip)++] = rq_len; + *tot_len += (Uint) rq_len; +} Uint -erts_run_queues_len(Uint *qlen, int atomic_queues_read, int incl_active_sched) +erts_run_queues_len(Uint *qlen, int atomic_queues_read, int incl_active_sched, + int incl_dirty_io) { - int i = 0; + int i = 0, j = 0; Uint len = 0; - if (atomic_queues_read) - ERTS_ATOMIC_FOREACH_RUNQ(rq, - { - Sint rq_len = (Sint) erts_smp_atomic32_read_dirty(&rq->len); - ASSERT(rq_len >= 0); - if (incl_active_sched - && (ERTS_RUNQ_FLGS_GET_NOB(rq) & ERTS_RUNQ_FLG_EXEC)) { - rq_len++; - } - if (qlen) - qlen[i++] = rq_len; - len += (Uint) rq_len; - } - ); + int no_rqs = erts_no_run_queues; + +#ifdef ERTS_DIRTY_SCHEDULERS + if (incl_dirty_io) + no_rqs += ERTS_NUM_DIRTY_RUNQS; + else + no_rqs += ERTS_NUM_DIRTY_CPU_RUNQS; +#endif + + if (atomic_queues_read) { + ERTS_ATOMIC_FOREACH_RUNQ_X(rq, no_rqs, + run_queues_len_aux(rq, &len, qlen, &j, + incl_active_sched, 1), + /* Nothing... */); + } else { - for (i = 0; i < erts_no_run_queues; i++) { + for (i = 0; i < no_rqs; i++) { ErtsRunQueue *rq = ERTS_RUNQ_IX(i); - Sint rq_len = (Sint) erts_smp_atomic32_read_nob(&rq->len); - ASSERT(rq_len >= 0); - if (incl_active_sched - && (ERTS_RUNQ_FLGS_GET_NOB(rq) & ERTS_RUNQ_FLG_EXEC)) { - rq_len++; - } - if (qlen) - qlen[i] = rq_len; - len += (Uint) rq_len; + run_queues_len_aux(rq, &len, qlen, &j, incl_active_sched, 0); } } @@ -11243,7 +11331,9 @@ cleanup_sys_tasks(Process *c_p, erts_aint32_t in_state, int in_reds) erts_aint32_t state = in_state; int max_reds = in_reds; int reds = 0; - int qmask = 0; + int qmask = 1; /* Set to 1 to force looping as long as there + * are dirty tasks. + */ ERTS_SMP_LC_ASSERT(erts_proc_lc_my_proc_locks(c_p) == ERTS_PROC_LOCK_MAIN); @@ -12097,6 +12187,8 @@ erts_get_total_reductions(Uint *redsp, Uint *diffp) Uint reds = 0; ERTS_ATOMIC_FOREACH_RUNQ_X(rq, + erts_no_run_queues + ERTS_NUM_DIRTY_RUNQS, + reds += rq->procs.reductions, if (redsp) *redsp = reds; @@ -13845,6 +13937,16 @@ erts_continue_exit_process(Process *p) goto yield; } +#ifdef DEBUG + erts_smp_proc_lock(p, ERTS_PROC_LOCK_STATUS); + ASSERT(p->sys_task_qs == NULL); + ASSERT(ERTS_PROC_GET_DELAYED_GC_TASK_QS(p) == NULL); +#ifdef ERTS_DIRTY_SCHEDULERS + ASSERT(p->dirty_sys_tasks == NULL); +#endif + erts_smp_proc_unlock(p, ERTS_PROC_LOCK_STATUS); +#endif + if (p->flags & F_USING_DDLL) { erts_ddll_proc_dead(p, ERTS_PROC_LOCK_MAIN); p->flags &= ~F_USING_DDLL; diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index 2b169bb9ce..d44e8c252d 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -1541,24 +1541,29 @@ extern int erts_system_profile_ts_type; } while (0) #if defined(ERTS_DIRTY_SCHEDULERS) && defined(ERTS_SMP) -#define ERTS_NUM_DIRTY_RUNQS 2 +#define ERTS_NUM_DIRTY_CPU_RUNQS 1 +#define ERTS_NUM_DIRTY_IO_RUNQS 1 #else -#define ERTS_NUM_DIRTY_RUNQS 0 +#define ERTS_NUM_DIRTY_CPU_RUNQS 0 +#define ERTS_NUM_DIRTY_IO_RUNQS 0 #endif +#define ERTS_NUM_DIRTY_RUNQS (ERTS_NUM_DIRTY_CPU_RUNQS+ERTS_NUM_DIRTY_IO_RUNQS) + #define ERTS_RUNQ_IX(IX) \ - (ASSERT(0 <= (IX) && (IX) < erts_no_run_queues), \ + (ASSERT(0 <= (IX) && (IX) < erts_no_run_queues+ERTS_NUM_DIRTY_RUNQS), \ &erts_aligned_run_queues[(IX)].runq) #ifdef ERTS_DIRTY_SCHEDULERS #define ERTS_RUNQ_IX_IS_DIRTY(IX) \ - (-(ERTS_NUM_DIRTY_RUNQS) <= (IX) && (IX) < 0) + (ASSERT(0 <= (IX) && (IX) < erts_no_run_queues+ERTS_NUM_DIRTY_RUNQS), \ + (erts_no_run_queues <= (IX))) #define ERTS_DIRTY_RUNQ_IX(IX) \ (ASSERT(ERTS_RUNQ_IX_IS_DIRTY(IX)), \ &erts_aligned_run_queues[(IX)].runq) -#define ERTS_DIRTY_CPU_RUNQ (&erts_aligned_run_queues[-1].runq) -#define ERTS_DIRTY_IO_RUNQ (&erts_aligned_run_queues[-2].runq) -#define ERTS_RUNQ_IS_DIRTY_CPU_RUNQ(RQ) ((RQ)->ix == -1) -#define ERTS_RUNQ_IS_DIRTY_IO_RUNQ(RQ) ((RQ)->ix == -2) +#define ERTS_DIRTY_CPU_RUNQ (&erts_aligned_run_queues[erts_no_run_queues].runq) +#define ERTS_DIRTY_IO_RUNQ (&erts_aligned_run_queues[erts_no_run_queues+1].runq) +#define ERTS_RUNQ_IS_DIRTY_CPU_RUNQ(RQ) ((RQ) == ERTS_DIRTY_CPU_RUNQ) +#define ERTS_RUNQ_IS_DIRTY_IO_RUNQ(RQ) ((RQ) == ERTS_DIRTY_IO_RUNQ) #else #define ERTS_RUNQ_IX_IS_DIRTY(IX) 0 #endif @@ -1836,7 +1841,7 @@ Uint erts_active_schedulers(void); void erts_init_process(int, int, int); Eterm erts_process_state2status(erts_aint32_t); Eterm erts_process_status(Process *, Eterm); -Uint erts_run_queues_len(Uint *, int, int); +Uint erts_run_queues_len(Uint *, int, int, int); void erts_add_to_runq(Process *); Eterm erts_bound_schedulers_term(Process *c_p); Eterm erts_get_cpu_topology_term(Process *c_p, Eterm which); diff --git a/erts/emulator/beam/ops.tab b/erts/emulator/beam/ops.tab index 9b5bd7a749..8abe871c14 100644 --- a/erts/emulator/beam/ops.tab +++ b/erts/emulator/beam/ops.tab @@ -182,15 +182,7 @@ i_jump_on_val x f I I i_jump_on_val y f I I %macro: get_list GetList -pack -get_list x x x -get_list x x y -get_list x y x -get_list x y y - -get_list y x x -get_list y x y -get_list y y x -get_list y y y +get_list xy xy xy # The following get_list instructions using x(0) are frequently used. get_list r x x @@ -218,12 +210,10 @@ set_tuple_element s d P # Get tuple element %macro: i_get_tuple_element GetTupleElement -pack -i_get_tuple_element x P x -i_get_tuple_element y P x +i_get_tuple_element xy P x %cold -i_get_tuple_element x P y -i_get_tuple_element y P y +i_get_tuple_element xy P y %hot %macro: i_get_tuple_element2 GetTupleElement2 -pack @@ -270,11 +260,7 @@ system_limit j move C=cxy x==0 | jump Lbl => move_jump Lbl C %macro: move_jump MoveJump -nonext -move_jump f n -move_jump f c -move_jump f x -move_jump f y - +move_jump f ncxy # Movement to and from the stack is common # Try to pack as much as we can into one instruction @@ -326,12 +312,10 @@ swap_temp R1 R2 Tmp | line Loc | call_ext_last Live Addr D | \ is_killed(Tmp, Live) => swap R1 R2 | line Loc | call_ext_last Live Addr D %macro: swap_temp SwapTemp -pack -swap_temp x x x -swap_temp x y x +swap_temp x xy x %macro: swap Swap -pack -swap x x -swap x y +swap x xy move Src=x D1=x | move Src=x D2=x => move_dup Src D1 D2 move Src=x SD=x | move SD=x D=x => move_dup Src SD D @@ -381,10 +365,7 @@ move_shift x y x move_shift x x y %macro: move_dup MoveDup -pack -move_dup x x x -move_dup x x y -move_dup y x x -move_dup y x y +move_dup xy x xy %macro: move2_par Move2Par -pack @@ -409,7 +390,7 @@ move3 x x x x x x move S=n D=y => init D move S=c D=y => move S x | move x D -%macro:move Move -pack -gen_dest +%macro:move Move -pack move x x move x y move y x @@ -483,8 +464,7 @@ i_is_ne_exact_literal f y c is_eq_exact Lbl Y=y X=x => is_eq_exact Lbl X Y %macro: is_eq_exact EqualExact -fail_action -pack -is_eq_exact f x x -is_eq_exact f x y +is_eq_exact f x xy is_eq_exact f s s %macro: is_lt IsLessThan -fail_action @@ -538,7 +518,7 @@ i_put_tuple y I # put_list Const=c n Dst => move Const x | put_list x n Dst -%macro:put_list PutList -pack -gen_dest +%macro:put_list PutList -pack put_list x n x put_list y n x @@ -630,9 +610,7 @@ is_tagged_tuple Fail=f c Arity Atom => jump Fail %macro:is_tagged_tuple IsTaggedTuple -fail_action -is_tagged_tuple f r A a -is_tagged_tuple f x A a -is_tagged_tuple f y A a +is_tagged_tuple f rxy A a # Test tuple & arity (head) @@ -642,21 +620,16 @@ is_tuple Fail=f S=xy | test_arity Fail=f S=xy Arity => is_tuple_of_arity Fail S %macro:is_tuple_of_arity IsTupleOfArity -fail_action -is_tuple_of_arity f r A -is_tuple_of_arity f x A -is_tuple_of_arity f y A +is_tuple_of_arity f rxy A %macro: is_tuple IsTuple -fail_action -is_tuple f r -is_tuple f x -is_tuple f y +is_tuple f rxy test_arity Fail Literal=q Arity => move Literal x | test_arity Fail x Arity test_arity Fail=f c Arity => jump Fail %macro: test_arity IsArity -fail_action -test_arity f x A -test_arity f y A +test_arity f xy A get_tuple_element Reg=x P1 D1=x | get_tuple_element Reg=x P2 D2=x | \ get_tuple_element Reg=x P3 D3=x | \ @@ -681,8 +654,7 @@ is_integer Fail=f S=x | allocate Need Regs => is_integer_allocate Fail S Need Re is_integer_allocate f x I I %macro: is_integer IsInteger -fail_action -is_integer f x -is_integer f y +is_integer f xy is_list Fail=f n => is_list Fail Literal=q => move Literal x | is_list Fail x @@ -696,8 +668,7 @@ is_list f y is_nonempty_list Fail=f S=x | allocate Need Rs => is_nonempty_list_allocate Fail S Need Rs %macro:is_nonempty_list_allocate IsNonemptyListAllocate -fail_action -pack -is_nonempty_list_allocate f r I t -is_nonempty_list_allocate f x I t +is_nonempty_list_allocate f rx I t is_nonempty_list F=f x==0 | test_heap I1 I2 => is_non_empty_list_test_heap F I1 I2 @@ -708,12 +679,10 @@ is_nonempty_list Fail=f S=x | get_list S D1=x D2=x => \ is_nonempty_list_get_list Fail S D1 D2 %macro: is_nonempty_list_get_list IsNonemptyListGetList -fail_action -pack -is_nonempty_list_get_list f r x x -is_nonempty_list_get_list f x x x +is_nonempty_list_get_list f rx x x %macro: is_nonempty_list IsNonemptyList -fail_action -is_nonempty_list f x -is_nonempty_list f y +is_nonempty_list f xy %macro: is_atom IsAtom -fail_action is_atom f x @@ -735,8 +704,7 @@ is_nil Fail=f n => is_nil Fail=f qia => jump Fail %macro: is_nil IsNil -fail_action -is_nil f x -is_nil f y +is_nil f xy is_binary Fail Literal=q => move Literal x | is_binary Fail x is_binary Fail=f c => jump Fail @@ -784,8 +752,7 @@ is_boolean Fail=f ac => jump Fail %cold %macro: is_boolean IsBoolean -fail_action -is_boolean f x -is_boolean f y +is_boolean f xy %hot is_function2 Fail=f acq Arity => jump Fail @@ -1079,8 +1046,7 @@ i_get_hash c I d i_get s d %macro: self Self -self x -self y +self xy %macro: node Node node x @@ -1091,8 +1057,7 @@ node y i_fast_element j x I d i_fast_element j y I d -i_element j x s d -i_element j y s d +i_element j xy s d bif1 f b s d bif1_body b s d @@ -1111,8 +1076,7 @@ i_move_call c f %macro:move_call MoveCall -arg_f -size -nonext move_call/2 -move_call x f -move_call y f +move_call xy f move S=c x==0 | call_last Ar P=f D => i_move_call_last P D S move S x==0 | call_last Ar P=f D => move_call_last S P D @@ -1122,8 +1086,7 @@ i_move_call_last f P c %macro:move_call_last MoveCallLast -arg_f -nonext -pack move_call_last/3 -move_call_last x f Q -move_call_last y f Q +move_call_last xy f Q move S=c x==0 | call_only Ar P=f => i_move_call_only P S move S=x x==0 | call_only Ar P=f => move_call_only S P @@ -1167,8 +1130,7 @@ i_make_fun I t %hot %macro: is_function IsFunction -fail_action -is_function f x -is_function f y +is_function f xy is_function Fail=f c => jump Fail func_info M F A => i_func_info u M F A @@ -1180,8 +1142,7 @@ func_info M F A => i_func_info u M F A %cold bs_start_match2 Fail=f ica X Y D => jump Fail bs_start_match2 Fail Bin X Y D => i_bs_start_match2 Bin Fail X Y D -i_bs_start_match2 x f I I d -i_bs_start_match2 y f I I d +i_bs_start_match2 xy f I I d bs_save2 Reg Index => gen_bs_save(Reg, Index) i_bs_save2 x I @@ -1209,9 +1170,9 @@ i_bs_get_integer_32 x f I d bs_get_binary2 Fail=f Ms=x Live=u Sz=sq Unit=u Flags=u Dst=d => \ gen_get_binary2(Fail, Ms, Live, Sz, Unit, Flags, Dst) -%macro: i_bs_get_binary_imm2 BsGetBinaryImm_2 -fail_action -gen_dest -%macro: i_bs_get_binary2 BsGetBinary_2 -fail_action -gen_dest -%macro: i_bs_get_binary_all2 BsGetBinaryAll_2 -fail_action -gen_dest +%macro: i_bs_get_binary_imm2 BsGetBinaryImm_2 -fail_action +%macro: i_bs_get_binary2 BsGetBinary_2 -fail_action +%macro: i_bs_get_binary_all2 BsGetBinaryAll_2 -fail_action i_bs_get_binary_imm2 f x I I I d i_bs_get_binary2 f x I s I d @@ -1224,7 +1185,7 @@ bs_get_float2 Fail=f Ms=x Live=u Sz=s Unit=u Flags=u Dst=d => \ bs_get_float2 Fail=f Ms=x Live=u Sz=q Unit=u Flags=u Dst=d => jump Fail -%macro: i_bs_get_float2 BsGetFloat2 -fail_action -gen_dest +%macro: i_bs_get_float2 BsGetFloat2 -fail_action i_bs_get_float2 f x I s I d # Miscellanous @@ -1236,8 +1197,7 @@ bs_skip_bits2 Fail=f Ms=x Sz=sq Unit=u Flags=u => \ i_bs_skip_bits_imm2 f x I %macro: i_bs_skip_bits2 BsSkipBits2 -fail_action -i_bs_skip_bits2 f x x I -i_bs_skip_bits2 f x y I +i_bs_skip_bits2 f x xy I %macro: i_bs_skip_bits_all2 BsSkipBitsAll2 -fail_action i_bs_skip_bits_all2 f x I @@ -1302,8 +1262,7 @@ bs_init2 Fail Sz Words=u==0 Regs Flags Dst => \ bs_init2 Fail Sz Words Regs Flags Dst => \ i_bs_init_fail_heap Sz Words Fail Regs Dst -i_bs_init_fail x j I d -i_bs_init_fail y j I d +i_bs_init_fail xy j I d i_bs_init_fail_heap s I j I d @@ -1324,8 +1283,7 @@ bs_init_bits Fail Sz Words=u==0 Regs Flags Dst => \ bs_init_bits Fail Sz Words Regs Flags Dst => \ i_bs_init_bits_fail_heap Sz Words Fail Regs Dst -i_bs_init_bits_fail x j I d -i_bs_init_bits_fail y j I d +i_bs_init_bits_fail xy j I d i_bs_init_bits_fail_heap s I j I d @@ -1493,8 +1451,7 @@ is_map Fail Lit=q | literal_is_map(Lit) => is_map Fail cq => jump Fail %macro: is_map IsMap -fail_action -is_map f x -is_map f y +is_map f xy ## Transform has_map_fields #{ K1 := _, K2 := _ } to has_map_elements @@ -1514,16 +1471,10 @@ i_get_map_element Fail Src=xy Key=y Dst => \ move Key x | i_get_map_element Fail Src x Dst %macro: i_get_map_element_hash GetMapElementHash -fail_action -i_get_map_element_hash f x c I x -i_get_map_element_hash f y c I x -i_get_map_element_hash f x c I y -i_get_map_element_hash f y c I y +i_get_map_element_hash f xy c I xy %macro: i_get_map_element GetMapElement -fail_action -i_get_map_element f x x x -i_get_map_element f y x x -i_get_map_element f x x y -i_get_map_element f y x y +i_get_map_element f xy x xy # # Convert the plus operations to a generic plus instruction. @@ -1589,12 +1540,9 @@ gc_bif2 Fail Live u$bif:erlang:bxor/2 S1 S2 Dst => \ gc_bif1 Fail I u$bif:erlang:bnot/1 Src Dst=d => i_int_bnot Fail Src I Dst -i_increment r I I d -i_increment x I I d -i_increment y I I d +i_increment rxy I I d -i_plus j I x x d -i_plus j I x y d +i_plus j I x xy d i_plus j I s s d i_minus j I x x d diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 2de908ffb3..bfebff5706 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -604,18 +604,6 @@ static size_t my_strnlen(const char *s, size_t maxlen) #endif -#if defined(HAVE_SYS_UN_H) - -/* Check that some character in the buffer != '\0' */ -static int is_nonzero(const char *s, size_t n) -{ - size_t i; - for (i = 0; i < n; i++) if (s[i] != '\0') return !0; - return 0; -} - -#endif - #ifdef VALGRIND # include <valgrind/memcheck.h> #else @@ -4025,13 +4013,30 @@ static char* inet_set_address(int family, inet_address* dst, int n; if (*len == 0) return str_einval; n = *((unsigned char*)(*src)); /* Length field */ - if ((*len < 1+n) || (sizeof(dst->sal.sun_path) < n+1)) { + if (*len < 1+n) return str_einval; + if (n + +#ifdef __linux__ + /* Make sure the address gets zero terminated + * except when the first byte is \0 because then it is + * sort of zero terminated although the zero termination + * comes before the address... + * This fix handles Linux's nonportable + * abstract socket address extension. + */ + ((*len) > 1 && (*src)[1] == '\0' ? 0 : 1) +#else + 1 +#endif + > sizeof(dst->sal.sun_path)) { return str_einval; } sys_memzero((char*)dst, sizeof(struct sockaddr_un)); dst->sal.sun_family = family; sys_memcpy(dst->sal.sun_path, (*src)+1, n); *len = offsetof(struct sockaddr_un, sun_path) + n; +#ifndef NO_SA_LEN + dst->sal.sun_len = *len; +#endif *src += 1 + n; return NULL; } @@ -4176,15 +4181,16 @@ static int inet_get_address(char* dst, inet_address* src, unsigned int* len) if (*len < offsetof(struct sockaddr_un, sun_path)) return -1; n = *len - offsetof(struct sockaddr_un, sun_path); if (255 < n) return -1; - /* Portability fix: Assume that the address is a zero terminated - * string, except when the first byte is \0 i.e the - * string length is 0. Then use the reported length instead. - * This fix handles Linux's abstract socket address - * nonportable extension. - */ m = my_strnlen(src->sal.sun_path, n); - if ((m == 0) && is_nonzero(src->sal.sun_path, n)) - m = n; +#ifdef __linux__ + /* Assume that the address is a zero terminated string, + * except when the first byte is \0 i.e the string length is 0, + * then use the reported length instead. + * This fix handles Linux's nonportable + * abstract socket address extension. + */ + if (m == 0) m = n; +#endif dst[0] = INET_AF_LOCAL; dst[1] = (char) ((unsigned char) m); sys_memcpy(dst+2, src->sal.sun_path, m); @@ -4241,15 +4247,16 @@ inet_address_to_erlang(char *dst, inet_address **src, SOCKLEN_T sz) { if (sz < offsetof(struct sockaddr_un, sun_path)) return -1; n = sz - offsetof(struct sockaddr_un, sun_path); if (255 < n) return -1; - /* Portability fix: Assume that the address is a zero terminated - * string, except when the first byte is \0 i.e the - * string length is 0. Then use the reported length instead. - * This fix handles Linux's abstract socket address - * nonportable extension. - */ m = my_strnlen((*src)->sal.sun_path, n); - if ((m == 0) && is_nonzero((*src)->sal.sun_path, n)) - m = n; +#ifdef __linux__ + /* Assume that the address is a zero terminated string, + * except when the first byte is \0 i.e the string length is 0, + * Then use the reported length instead. + * This fix handles Linux's nonportable + * abstract socket address extension. + */ + if (m == 0) m = n; +#endif if (dst) { dst[0] = INET_AF_LOCAL; dst[1] = (char) ((unsigned char) m); @@ -8680,6 +8687,7 @@ static ErlDrvSSizeT inet_ctl(inet_descriptor* desc, int cmd, char* buf, else { ptr = &peer; sz = sizeof(peer); + sys_memzero((char *) &peer, sz); if (IS_SOCKET_ERROR (sock_peer (desc->s, (struct sockaddr*)ptr, &sz))) @@ -10830,10 +10838,11 @@ static int tcp_inet_output(tcp_descriptor* desc, HANDLE event) #ifndef SO_ERROR { - int sz = sizeof(desc->inet.remote); - int code = sock_peer(desc->inet.s, - (struct sockaddr*) &desc->inet.remote, &sz); - + int sz, code; + sz = sizeof(desc->inet.remote); + sys_memzero((char *) &desc->inet.remote, sz); + code = sock_peer(desc->inet.s, + (struct sockaddr*) &desc->inet.remote, &sz); if (IS_SOCKET_ERROR(code)) { desc->inet.state = INET_STATE_OPEN; /* restore state */ ret = async_error(INETP(desc), sock_errno()); diff --git a/erts/emulator/sys/unix/sys.c b/erts/emulator/sys/unix/sys.c index 0079912b10..5cf0a49972 100644 --- a/erts/emulator/sys/unix/sys.c +++ b/erts/emulator/sys/unix/sys.c @@ -885,7 +885,6 @@ void erts_replace_intr(void) { void init_break_handler(void) { sys_signal(SIGINT, request_break); - sys_signal(SIGHUP, generic_signal_handler); #ifndef ETHR_UNUSABLE_SIGUSRX sys_signal(SIGUSR1, generic_signal_handler); #endif /* #ifndef ETHR_UNUSABLE_SIGUSRX */ @@ -1457,12 +1456,12 @@ erts_sys_main_thread(void) erts_thread_disable_fpe(); #ifdef __DARWIN__ initialize_darwin_main_thread_pipes(); -#endif +#else /* Become signal receiver thread... */ #ifdef ERTS_ENABLE_LOCK_CHECK erts_lc_set_thread_name("signal_receiver"); #endif - +#endif smp_sig_notify(0); /* Notify initialized */ /* Wait for a signal to arrive... */ diff --git a/erts/emulator/test/bs_construct_SUITE.erl b/erts/emulator/test/bs_construct_SUITE.erl index 95042ac802..ed03284a5b 100644 --- a/erts/emulator/test/bs_construct_SUITE.erl +++ b/erts/emulator/test/bs_construct_SUITE.erl @@ -537,6 +537,8 @@ huge_binary(Config) when is_list(Config) -> ct:timetrap({seconds, 60}), 16777216 = size(<<0:(id(1 bsl 26)),(-1):(id(1 bsl 26))>>), garbage_collect(), + FreeMem = free_mem(), + io:format("Free memory (Mb): ~p\n", [FreeMem]), {Shift,Return} = case free_mem() of undefined -> %% This test has to be inlined inside the case to @@ -552,10 +554,14 @@ huge_binary(Config) when is_list(Config) -> garbage_collect(), id(<<0:((1 bsl 31)-1)>>), {31,"Limit huge binaries to 256 Mb"}; - _ -> + Mb when Mb > 200 -> garbage_collect(), id(<<0:((1 bsl 30)-1)>>), - {30,"Limit huge binary to 128 Mb"} + {30,"Limit huge binary to 128 Mb"}; + _ -> + garbage_collect(), + id(<<0:((1 bsl 29)-1)>>), + {29,"Limit huge binary to 64 Mb"} end, garbage_collect(), id(<<0:((1 bsl Shift)-1)>>), @@ -567,13 +573,14 @@ huge_binary(Config) when is_list(Config) -> Comment -> {comment, Comment} end. +%% Return the amount of free memory in Mb. free_mem() -> {ok,Apps} = application:ensure_all_started(os_mon), Mem = memsup:get_system_memory_data(), [ok = application:stop(App)||App <- Apps], case proplists:get_value(free_memory,Mem) of undefined -> undefined; - Val -> Val div 1024 + Val -> Val div (1024*1024) end. system_limit(Config) when is_list(Config) -> diff --git a/erts/emulator/test/call_trace_SUITE.erl b/erts/emulator/test/call_trace_SUITE.erl index 1216863c51..95171d04ce 100644 --- a/erts/emulator/test/call_trace_SUITE.erl +++ b/erts/emulator/test/call_trace_SUITE.erl @@ -60,7 +60,7 @@ all() -> init_per_testcase(Func, Config) when is_atom(Func), is_list(Config) -> Config. -end_per_testcase(_Func, Config) -> +end_per_testcase(_Func, _Config) -> %% Reloading the module will clear all trace patterns, and %% in a debug-compiled emulator run assertions of the counters %% for the number of traced exported functions in this module. diff --git a/erts/emulator/test/code_SUITE.erl b/erts/emulator/test/code_SUITE.erl index d07166ed98..35f7baf2cf 100644 --- a/erts/emulator/test/code_SUITE.erl +++ b/erts/emulator/test/code_SUITE.erl @@ -229,27 +229,6 @@ multi_proc_purge(Config) when is_list(Config) -> Pid9, Pid10, Pid11, Pid12, Pid13, Pid14, Pid15, Pid16]), ok. -body(F, Fakes) -> - receive - jog -> - 40 = F(3), - erlang:garbage_collect(), - body(F, Fakes); - drop_funs -> - dropped_body() - end. - -dropped_body() -> - receive - X -> exit(X) - end. - -gc() -> - erlang:garbage_collect(), - gc1(). -gc1() -> ok. - - %% Test the erlang:check_old_code/1 BIF. t_check_old_code(Config) when is_list(Config) -> Data = proplists:get_value(data_dir, Config), diff --git a/erts/emulator/test/driver_SUITE.erl b/erts/emulator/test/driver_SUITE.erl index 2fbf6eae61..e854a5f945 100644 --- a/erts/emulator/test/driver_SUITE.erl +++ b/erts/emulator/test/driver_SUITE.erl @@ -127,7 +127,7 @@ init_per_testcase(Case, Config) when is_atom(Case), is_list(Config) -> 0 = element(1, erts_debug:get_internal_state(check_io_debug)), [{testcase, Case}|Config]. -end_per_testcase(Case, Config) -> +end_per_testcase(Case, _Config) -> erlang:display({end_per_testcase, Case}), 0 = element(1, erts_debug:get_internal_state(check_io_debug)), ok. diff --git a/erts/emulator/test/gc_SUITE.erl b/erts/emulator/test/gc_SUITE.erl index 35dd147550..2c2cb9c32d 100644 --- a/erts/emulator/test/gc_SUITE.erl +++ b/erts/emulator/test/gc_SUITE.erl @@ -203,89 +203,90 @@ long_receive() -> end. minor_major_gc_option_self(_Config) -> - Endless = fun Endless() -> - receive - {gc, Type} -> erlang:garbage_collect(self(), [{type, Type}]) - after 100 -> ok end, - Endless() - end, - - %% Try as major, a test process will self-trigger GC - P1 = spawn(Endless), - erlang:garbage_collect(P1, []), - erlang:trace(P1, true, [garbage_collection]), - P1 ! {gc, major}, - expect_trace_messages(P1, [gc_major_start, gc_major_end]), - erlang:trace(P1, false, [garbage_collection]), - erlang:exit(P1, kill), - - %% Try as minor, a test process will self-trigger GC - P2 = spawn(Endless), - erlang:garbage_collect(P2, []), - erlang:trace(P2, true, [garbage_collection]), - P2 ! {gc, minor}, - expect_trace_messages(P2, [gc_minor_start, gc_minor_end]), - erlang:trace(P2, false, [garbage_collection]), - erlang:exit(P2, kill). + %% Try as major, the test process will self-trigger GC + check_gc_tracing_around( + fun(Pid, Ref) -> + Pid ! {gc, Ref, major} + end, [gc_major_start, gc_major_end]), + + %% Try as minor, the test process will self-trigger GC + check_gc_tracing_around( + fun(Pid, Ref) -> + Pid ! {gc, Ref, minor} + end, [gc_minor_start, gc_minor_end]). minor_major_gc_option_async(_Config) -> - Endless = fun Endless() -> - receive after 100 -> ok end, - Endless() - end, - - %% Try with default option, must be major gc - P1 = spawn(Endless), - erlang:garbage_collect(P1, []), - erlang:trace(P1, true, [garbage_collection]), - erlang:garbage_collect(P1, []), - expect_trace_messages(P1, [gc_major_start, gc_major_end]), - erlang:trace(P1, false, [garbage_collection]), - erlang:exit(P1, kill), + %% Try with default option, must be major GC + check_gc_tracing_around( + fun(Pid, _Ref) -> + erlang:garbage_collect(Pid, []) + end, [gc_major_start, gc_major_end]), %% Try with the 'major' type - P2 = spawn(Endless), - erlang:garbage_collect(P2, []), - erlang:trace(P2, true, [garbage_collection]), - erlang:garbage_collect(P2, [{type, major}]), - expect_trace_messages(P2, [gc_major_start, gc_major_end]), - erlang:trace(P2, false, [garbage_collection]), - erlang:exit(P2, kill), + check_gc_tracing_around( + fun(Pid, _Ref) -> + erlang:garbage_collect(Pid, [{type, major}]) + end, [gc_major_start, gc_major_end]), %% Try with 'minor' option, once - P3 = spawn(Endless), - erlang:garbage_collect(P3, []), - erlang:trace(P3, true, [garbage_collection]), - erlang:garbage_collect(P3, [{type, minor}]), - expect_trace_messages(P3, [gc_minor_start, gc_minor_end]), - erlang:trace(P3, false, [garbage_collection]), - erlang:exit(P3, kill), + check_gc_tracing_around( + fun(Pid, _Ref) -> + erlang:garbage_collect(Pid, [{type, minor}]) + end, [gc_minor_start, gc_minor_end]), %% Try with 'minor' option, once, async - P4 = spawn(Endless), + check_gc_tracing_around( + fun(Pid, Ref) -> + ?assertEqual(async, + erlang:garbage_collect(Pid, [{type, minor}, {async, Ref}])), + + receive + {garbage_collect, Ref, true} -> + ok + after 10000 -> + ct:fail("Did not receive a completion notification on async GC") + end + end, [gc_minor_start, gc_minor_end]). + +%% Traces garbage collection around the given operation, and fails the test if +%% it results in any unexpected messages or if the expected trace tags are not +%% received. +check_gc_tracing_around(Fun, ExpectedTraceTags) -> Ref = erlang:make_ref(), - erlang:garbage_collect(P4, []), - erlang:trace(P4, true, [garbage_collection]), - ?assertEqual(async, - erlang:garbage_collect(P4, [{type, minor}, {async, Ref}])), - expect_trace_messages(P4, [gc_minor_start, gc_minor_end]), - erlang:trace(P4, false, [garbage_collection]), - receive {garbage_collect, Ref, true} -> ok; - Other4 -> ct:pal("Unexpected message: ~p~n" - ++ "while waiting for async gc result", [Other4]) - after 2000 -> ?assert(false) - end, - erlang:exit(P4, kill). - -%% Given a list of atoms, trace tags - receives messages and checks if they are -%% trace events, and if the tag matches. Else will crash failing the test. -expect_trace_messages(_Pid, []) -> ok; + Pid = spawn( + fun Endless() -> + receive + {gc, Ref, Type} -> + erlang:garbage_collect(self(), [{type, Type}]) + after 100 -> + ok + end, + Endless() + end), + erlang:garbage_collect(Pid, []), + erlang:trace(Pid, true, [garbage_collection]), + Fun(Pid, Ref), + expect_trace_messages(Pid, ExpectedTraceTags), + erlang:trace(Pid, false, [garbage_collection]), + erlang:exit(Pid, kill), + check_no_unexpected_messages(). + +%% Ensures that trace messages with the provided tags have all been received +%% within a reasonable timeframe. +expect_trace_messages(_Pid, []) -> + ok; expect_trace_messages(Pid, [Tag | TraceTags]) -> receive - {trace, Pid, Tag, _Data} -> ok; - AnythingElse -> - ct:pal("Unexpected message: ~p~nWhile expected {trace, _, ~p, _}", - [AnythingElse, Tag]), - ?assert(false) - end, - expect_trace_messages(Pid, TraceTags). + {trace, Pid, Tag, _Data} -> + expect_trace_messages(Pid, TraceTags) + after 4000 -> + ct:fail("Didn't receive tag ~p within 4000ms", [Tag]) + end. + +check_no_unexpected_messages() -> + receive + Anything -> + ct:fail("Unexpected message: ~p", [Anything]) + after 0 -> + ok + end. diff --git a/erts/emulator/test/mtx_SUITE.erl b/erts/emulator/test/mtx_SUITE.erl index 12928ed6d8..0d6ab5cdb2 100644 --- a/erts/emulator/test/mtx_SUITE.erl +++ b/erts/emulator/test/mtx_SUITE.erl @@ -97,7 +97,7 @@ init_per_testcase(_Case, Config) -> wait_deallocations(), Config. -end_per_testcase(_Func, Config) -> +end_per_testcase(_Func, _Config) -> ok. wait_deallocations() -> diff --git a/erts/emulator/test/nif_SUITE.erl b/erts/emulator/test/nif_SUITE.erl index 1eb58699b2..bcea9e3539 100644 --- a/erts/emulator/test/nif_SUITE.erl +++ b/erts/emulator/test/nif_SUITE.erl @@ -488,7 +488,7 @@ select(Config) when is_list(Config) -> %% Wait for read eagain = read_nif(R, 3), 0 = select_nif(R,?ERL_NIF_SELECT_READ,R,null,Ref), - [] = flush(), + [] = flush(0), ok = write_nif(W, <<"hej">>), [{select, R, Ref, ready_input}] = flush(), 0 = select_nif(R,?ERL_NIF_SELECT_READ,R,self(),Ref2), @@ -505,7 +505,7 @@ select(Config) when is_list(Config) -> %% Wait for write Written = write_full(W, $a), 0 = select_nif(W,?ERL_NIF_SELECT_WRITE,W,self(),Ref), - [] = flush(), + [] = flush(0), Written = read_nif(R,byte_size(Written)), [{select, W, Ref, ready_output}] = flush(), @@ -515,7 +515,7 @@ select(Config) when is_list(Config) -> [{fd_resource_stop, W_ptr, _}] = flush(), {1, {W_ptr,_}} = last_fd_stop_call(), true = is_closed_nif(W), - [] = flush(), + [] = flush(0), 0 = select_nif(R,?ERL_NIF_SELECT_READ,R,self(),Ref), [{select, R, Ref, ready_input}] = flush(), eof = read_nif(R,1), @@ -540,7 +540,7 @@ select_2(Config) -> 0 = select_nif(R,?ERL_NIF_SELECT_READ,R,null,Ref1), 0 = select_nif(R,?ERL_NIF_SELECT_READ,R,self(),Ref2), - [] = flush(), + [] = flush(0), ok = write_nif(W, <<"hej">>), [{select, R, Ref2, ready_input}] = flush(), <<"hej">> = read_nif(R, 3), @@ -551,7 +551,7 @@ select_2(Config) -> Papa = self(), spawn_link(fun() -> 0 = select_nif(R,?ERL_NIF_SELECT_READ,R,null,Ref1), - [] = flush(), + [] = flush(0), Papa ! sync, [{select, R, Ref1, ready_input}] = flush(), <<"hej">> = read_nif(R, 3), @@ -560,7 +560,7 @@ select_2(Config) -> sync = receive_any(), ok = write_nif(W, <<"hej">>), done = receive_any(), - [] = flush(), + [] = flush(0), check_stop_ret(select_nif(R,?ERL_NIF_SELECT_STOP,R,null,Ref1)), [{fd_resource_stop, R_ptr, _}] = flush(), @@ -629,6 +629,15 @@ monitor_process_a(Config) -> monitor_process_b(Config) -> ensure_lib_loaded(Config), + monitor_process_b_do(false), + case erlang:system_info(threads) of + true -> monitor_process_b_do(true); + false -> ok + end, + ok. + + +monitor_process_b_do(FromThread) -> Pid = spawn_link(fun() -> receive return -> ok @@ -637,8 +646,11 @@ monitor_process_b(Config) -> R_ptr = alloc_monitor_resource_nif(), {0,_} = monitor_process_nif(R_ptr, Pid, true, self()), [R_ptr] = monitored_by(Pid), - ok = release_resource(R_ptr), - [] = flush(), + case FromThread of + false -> ok = release_resource(R_ptr); + true -> ok = release_resource_from_thread(R_ptr) + end, + [] = flush(0), {R_ptr, _, 1} = last_resource_dtor_call(), [] = monitored_by(Pid), Pid ! return, @@ -660,7 +672,7 @@ monitor_process_c(Config) -> exit end), [{Pid, done, R_ptr, Mon1}, - {monitor_resource_down, R_ptr, Pid, Mon2}] = flush(), + {monitor_resource_down, R_ptr, Pid, Mon2}] = flush(2), compare_monitors_nif(Mon1, Mon2), {R_ptr, _, 1} = last_resource_dtor_call(), ok. @@ -708,7 +720,7 @@ demonitor_process(Config) -> 1 = demonitor_process_nif(R_ptr, MonBin2), ok = release_resource(R_ptr), - [] = flush(), + [] = flush(0), {R_ptr, _, 1} = last_resource_dtor_call(), [] = monitored_by(Pid), Pid ! return, @@ -2307,10 +2319,16 @@ receive_any(Timeout) -> after Timeout -> timeout end. flush() -> - flush(10). -flush(Timeout) -> + flush(1). + +flush(0) -> + flush(0, 10); % don't waste too much time waiting for nothing +flush(N) -> + flush(N, 1000). + +flush(N, Timeout) -> receive M -> - [M | flush(Timeout)] + [M | flush(N-1)] after Timeout -> [] end. @@ -2619,9 +2637,9 @@ nif_snprintf(Config) -> nif_internal_hash(Config) -> ensure_lib_loaded(Config), HashValueBitSize = nif_hash_result_bitsize(internal), - Terms = unique([random_term() || _ <- lists:seq(1, 5000)]), + Terms = unique([random_term() || _ <- lists:seq(1, 500)]), HashValues = [hash_nif(internal, Term, 0) || Term <- Terms], - test_bit_distribution_fitness(HashValues, HashValueBitSize, 0.05). + test_bit_distribution_fitness(HashValues, HashValueBitSize). nif_internal_hash_salted(Config) -> ensure_lib_loaded(Config), @@ -2630,7 +2648,7 @@ nif_internal_hash_salted(Config) -> nif_phash2(Config) -> ensure_lib_loaded(Config), HashValueBitSize = nif_hash_result_bitsize(phash2), - Terms = unique([random_term() || _ <- lists:seq(1, 5000)]), + Terms = unique([random_term() || _ <- lists:seq(1, 500)]), HashValues = lists:map( fun (Term) -> @@ -2643,12 +2661,12 @@ nif_phash2(Config) -> HashValue end, Terms), - test_bit_distribution_fitness(HashValues, HashValueBitSize, 0.05). + test_bit_distribution_fitness(HashValues, HashValueBitSize). test_salted_nif_hash(HashType) -> HashValueBitSize = nif_hash_result_bitsize(HashType), - Terms = unique([random_term() || _ <- lists:seq(1, 5000)]), - Salts = unique([random_uint32() || _ <- lists:seq(1, 100)]), + Terms = unique([random_term() || _ <- lists:seq(1, 500)]), + Salts = unique([random_uint32() || _ <- lists:seq(1, 50)]), {HashValuesPerSalt, HashValuesPerTerm} = lists:mapfoldl( fun (Salt, Acc) -> @@ -2669,22 +2687,20 @@ test_salted_nif_hash(HashType) -> % Test per-salt hash distribution of different terms lists:foreach( fun ({_Salt, HashValues}) -> - test_bit_distribution_fitness(HashValues, HashValueBitSize, 0.05) + test_bit_distribution_fitness(HashValues, HashValueBitSize) end, HashValuesPerSalt), % Test per-term hash distribution of different salts dict:fold( fun (_Term, HashValues, Acc) -> - % Be more tolerant of relative deviation, - % as there's fewer hash values here. - test_bit_distribution_fitness(HashValues, HashValueBitSize, 0.30), + test_bit_distribution_fitness(HashValues, HashValueBitSize), Acc end, ok, HashValuesPerTerm). -test_bit_distribution_fitness(Integers, BitSize, MaxRelativeDeviation) -> +test_bit_distribution_fitness(Integers, BitSize) -> MaxInteger = (1 bsl BitSize) - 1, OnesPerBit = lists:foldl( @@ -2700,19 +2716,29 @@ test_bit_distribution_fitness(Integers, BitSize, MaxRelativeDeviation) -> orddict:new(), Integers), - ExpectedNrOfOnes = length(Integers) div 2, + N = length(Integers), + ExpectedNrOfOnes = N div 2, + %% ExpectedNrOfOnes should have a binomial distribution + %% with a standard deviation as: + ExpectedStdDev = math:sqrt(N) / 2, + %% which can be approximated as a normal distribution + %% where we allow a deviation of 6 std.devs + %% for a fail probability of 0.000000002: + MaxStdDevs = 6, + FailureText = orddict:fold( fun (BitIndex, NrOfOnes, Acc) -> - RelativeDeviation = abs(NrOfOnes - ExpectedNrOfOnes) / length(Integers), - case RelativeDeviation >= MaxRelativeDeviation of - false -> Acc; + Deviation = abs(NrOfOnes - ExpectedNrOfOnes) / ExpectedStdDev, + case Deviation >= MaxStdDevs of + false -> + Acc; true -> [Acc, io_lib:format( "Unreasonable deviation on number of set bits (i=~p): " - "expected ~p, got ~p (relative dev. ~.3f)~n", - [BitIndex, ExpectedNrOfOnes, NrOfOnes, RelativeDeviation])] + "expected ~p, got ~p (# std.dev ~.3f > ~p)~n", + [BitIndex, ExpectedNrOfOnes, NrOfOnes, Deviation, MaxStdDevs])] end end, [], @@ -2789,6 +2815,7 @@ alloc_resource(_,_) -> ?nif_stub. make_resource(_) -> ?nif_stub. get_resource(_,_) -> ?nif_stub. release_resource(_) -> ?nif_stub. +release_resource_from_thread(_) -> ?nif_stub. last_resource_dtor_call() -> ?nif_stub. make_new_resource(_,_) -> ?nif_stub. check_is(_,_,_,_,_,_,_,_,_,_,_) -> ?nif_stub. diff --git a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c index 3747291e7e..15d31162ed 100644 --- a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c +++ b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c @@ -972,6 +972,30 @@ static ERL_NIF_TERM release_resource(ErlNifEnv* env, int argc, const ERL_NIF_TER return enif_make_atom(env,"ok"); } +static void* threaded_release_resource(void* resource) +{ + enif_release_resource(resource); +} + +static ERL_NIF_TERM release_resource_from_thread(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + void* resource; + ErlNifTid tid; + int err; + + if (!get_pointer(env, argv[0], &resource)) { + return enif_make_badarg(env); + } + if (enif_thread_create("nif_SUITE:release_resource_from_thread", &tid, + threaded_release_resource, resource, NULL) != 0) { + return enif_make_badarg(env); + } + err = enif_thread_join(tid, NULL); + assert(err == 0); + return atom_ok; +} + + /* * argv[0] an atom * argv[1] a binary @@ -2537,6 +2561,7 @@ static ERL_NIF_TERM monitor_frenzy_nif(ErlNifEnv* env, int argc, const ERL_NIF_T static unsigned long spawn_cnt = 0; static unsigned long kill_cnt = 0; static unsigned long proc_histogram[FRENZY_PROCS_MAX]; + static int initialized = 0; static const unsigned int primes[] = {7, 13, 17, 19}; @@ -2556,7 +2581,7 @@ static ERL_NIF_TERM monitor_frenzy_nif(ErlNifEnv* env, int argc, const ERL_NIF_T if (enif_is_atom(env, Op)) { if (Op == atom_init) { - if (procs_lock || !enif_get_uint(env, Rnd, &frenzy_rand_bits_max)) + if (initialized || !enif_get_uint(env, Rnd, &frenzy_rand_bits_max)) return enif_make_badarg(env); procs_lock = enif_mutex_create("nif_SUITE:monitor_frenzy.procs"); @@ -2583,6 +2608,7 @@ static ERL_NIF_TERM monitor_frenzy_nif(ErlNifEnv* env, int argc, const ERL_NIF_T spawn_cnt = 1; kill_cnt = 0; + initialized = 1; return enif_make_uint(env, 0); /* SelfPix */ } else if (Op == atom_stats) { @@ -2613,7 +2639,7 @@ static ERL_NIF_TERM monitor_frenzy_nif(ErlNifEnv* env, int argc, const ERL_NIF_T enif_make_ulong(env, res_dtor_cnt))); } - else if (Op == atom_stop && procs_lock) { /* stop all */ + else if (Op == atom_stop && initialized) { /* stop all */ /* Release all resources */ for (rix = 0; rix < FRENZY_RESOURCES_MAX; rix++) { @@ -2903,6 +2929,7 @@ static ErlNifFunc nif_funcs[] = {"make_resource", 1, make_resource}, {"get_resource", 2, get_resource}, {"release_resource", 1, release_resource}, + {"release_resource_from_thread", 1, release_resource_from_thread}, {"last_resource_dtor_call", 0, last_resource_dtor_call}, {"make_new_resource", 2, make_new_resource}, {"check_is", 11, check_is}, diff --git a/erts/emulator/test/signal_SUITE.erl b/erts/emulator/test/signal_SUITE.erl index 7e516176f7..d788360812 100644 --- a/erts/emulator/test/signal_SUITE.erl +++ b/erts/emulator/test/signal_SUITE.erl @@ -53,7 +53,7 @@ init_per_testcase(Func, Config) when is_atom(Func), is_list(Config) -> available_internal_state(true), [{testcase, Func}|Config]. -end_per_testcase(_Func, Config) -> +end_per_testcase(_Func, _Config) -> ok. init_per_suite(Config) -> diff --git a/erts/emulator/test/statistics_SUITE.erl b/erts/emulator/test/statistics_SUITE.erl index 3057905f4c..7690557fda 100644 --- a/erts/emulator/test/statistics_SUITE.erl +++ b/erts/emulator/test/statistics_SUITE.erl @@ -396,7 +396,7 @@ msb_swt_hog(false) -> count(1000000), msb_swt_hog(false). -msb_scheduler_wall_time(Config) -> +msb_scheduler_wall_time(_Config) -> erlang:system_flag(scheduler_wall_time, true), Dirty = erlang:system_info(dirty_cpu_schedulers) /= 0, Hogs = lists:map(fun (_) -> @@ -493,7 +493,7 @@ badarg(Config) when is_list(Config) -> tok_loop() -> tok_loop(). -run_queues_lengths_active_tasks(Config) -> +run_queues_lengths_active_tasks(_Config) -> TokLoops = lists:map(fun (_) -> spawn_opt(fun () -> tok_loop() @@ -502,20 +502,37 @@ run_queues_lengths_active_tasks(Config) -> end, lists:seq(1,10)), + + TRQLs0 = statistics(total_run_queue_lengths), + TRQLAs0 = statistics(total_run_queue_lengths_all), TATs0 = statistics(total_active_tasks), + TATAs0 = statistics(total_active_tasks_all), true = is_integer(TRQLs0), true = is_integer(TATs0), true = TRQLs0 >= 0, + true = TRQLAs0 >= 0, true = TATs0 >= 11, + true = TATAs0 >= 11, NoScheds = erlang:system_info(schedulers), + {DefRqs, + AllRqs} = case erlang:system_info(dirty_cpu_schedulers) of + 0 -> {NoScheds, NoScheds}; + _ -> {NoScheds+1, NoScheds+2} + end, RQLs0 = statistics(run_queue_lengths), + RQLAs0 = statistics(run_queue_lengths_all), ATs0 = statistics(active_tasks), - NoScheds = length(RQLs0), - NoScheds = length(ATs0), + ATAs0 = statistics(active_tasks_all), + DefRqs = length(RQLs0), + AllRqs = length(RQLAs0), + DefRqs = length(ATs0), + AllRqs = length(ATAs0), true = lists:sum(RQLs0) >= 0, + true = lists:sum(RQLAs0) >= 0, true = lists:sum(ATs0) >= 11, + true = lists:sum(ATAs0) >= 11, SO = erlang:system_flag(schedulers_online, 1), @@ -531,8 +548,8 @@ run_queues_lengths_active_tasks(Config) -> RQLs1 = statistics(run_queue_lengths), ATs1 = statistics(active_tasks), - NoScheds = length(RQLs1), - NoScheds = length(ATs1), + DefRqs = length(RQLs1), + DefRqs = length(ATs1), TRQLs2 = lists:sum(RQLs1), TATs2 = lists:sum(ATs1), true = TRQLs2 >= 10, diff --git a/erts/emulator/test/trace_SUITE.erl b/erts/emulator/test/trace_SUITE.erl index f846b0f4b9..643c2e0472 100644 --- a/erts/emulator/test/trace_SUITE.erl +++ b/erts/emulator/test/trace_SUITE.erl @@ -733,7 +733,7 @@ set_on_first_spawn(Config) when is_list(Config) -> %% Tests trace(Pid, How, [set_on_link]). -set_on_link(Config) -> +set_on_link(_Config) -> Listener = fun_spawn(fun process/0), %% Create and trace a process with the set_on_link flag. @@ -756,7 +756,7 @@ set_on_link(Config) -> %% Tests trace(Pid, How, [set_on_first_spawn]). -set_on_first_link(Config) -> +set_on_first_link(_Config) -> ct:timetrap({seconds, 10}), Listener = fun_spawn(fun process/0), diff --git a/erts/emulator/test/trace_meta_SUITE.erl b/erts/emulator/test/trace_meta_SUITE.erl index b6a6fd5404..ef58978749 100644 --- a/erts/emulator/test/trace_meta_SUITE.erl +++ b/erts/emulator/test/trace_meta_SUITE.erl @@ -74,7 +74,7 @@ config(priv_dir,_) -> init_per_testcase(_Case, Config) -> Config. -end_per_testcase(_Case, Config) -> +end_per_testcase(_Case, _Config) -> shutdown(), ok. diff --git a/erts/emulator/utils/beam_makeops b/erts/emulator/utils/beam_makeops index 9813142585..05cd48d434 100755 --- a/erts/emulator/utils/beam_makeops +++ b/erts/emulator/utils/beam_makeops @@ -332,9 +332,9 @@ while (<>) { if (/^\%macro:(.*)/) { my($op, $macro, @flags) = split(' ', $1); defined($macro) and $macro =~ /^-/ and - &error("A macro must not start with a hyphen"); + error("A macro must not start with a hyphen"); foreach (@flags) { - /^-/ or &error("Flags for macros should start with a hyphen"); + /^-/ or error("Flags for macros should start with a hyphen"); } error("Macro for '$op' is already defined") if defined $macro{$op}; @@ -347,7 +347,7 @@ while (<>) { # Handle transformations. # if (/=>/) { - &parse_transformation($_); + parse_transformation($_); next; } @@ -357,8 +357,8 @@ while (<>) { $op_num = undef; if (s/^(\d+):\s*//) { $op_num = $1; - $op_num != 0 or &error("Opcode 0 invalid"); - &error("Opcode $op_num already defined") + $op_num != 0 or error("Opcode 0 invalid"); + error("Opcode $op_num already defined") if defined $gen_opname[$op_num]; } @@ -369,11 +369,11 @@ while (<>) { my($obsolete) = $1; my($name) = $2; my($arity) = $3; - $name =~ /^[a-z]/ or &error("Opname must start with a lowercase letter"); + $name =~ /^[a-z]/ or error("Opname must start with a lowercase letter"); defined $gen_arity{$name} and $gen_arity{$name} != $arity and - &error("Opname $name already defined with arity $gen_arity{$name}"); + error("Opname $name already defined with arity $gen_arity{$name}"); defined $unnumbered{$name,$arity} and - &error("Opname $name already defined with arity $gen_arity{$name}"); + error("Opname $name already defined with arity $gen_arity{$name}"); if (defined $op_num) { # Numbered generic operation $gen_opname[$op_num] = $name; @@ -395,16 +395,16 @@ while (<>) { # Name Arg1 Arg2... # my($name, @args) = split; - &error("too many operands") + error("too many operands") if @args > $max_spec_operands; - &syntax_check($name, @args); + syntax_check($name, @args); my $arity = @args; if (defined $gen_opnum{$name,$arity} and $obsolete[$gen_opnum{$name,$arity}]) { error("specific instructions may not be specified for obsolete instructions"); } - push(@{$specific_op{"$name/$arity"}}, [$name, $hot, @args]); + save_specific_ops($name, $arity, $hot, @args); if (defined $op_num) { - &error("specific instructions must not be numbered"); + error("specific instructions must not be numbered"); } elsif (!defined($gen_arity{$name}) && !defined($unnumbered{$name,$arity})) { # # Create an unumbered generic instruction too. @@ -446,7 +446,7 @@ $num_file_opcodes = @gen_opname; # Produce output for the chosen target. # -&$target; +&$target(); # # Produce output needed by the emulator/loader. @@ -462,7 +462,7 @@ sub emulator_output { # $name = "$outdir/beam_opcodes.c"; open(STDOUT, ">$name") || die "Failed to open $name for writing: $!\n"; - &comment('C'); + comment('C'); print "#ifdef HAVE_CONFIG_H\n"; print "# include \"config.h\"\n"; print "#endif\n\n"; @@ -525,7 +525,7 @@ sub emulator_output { # Call a generator to calculate size and generate macros # for the emulator. # - my($size, $code, $pack) = &basic_generator($name, $hot, @args); + my($size, $code, $pack) = basic_generator($name, $hot, @args); # # Save the generated $code for later. @@ -587,7 +587,7 @@ sub emulator_output { # Generate transformations. # - &tr_gen(@transformations); + tr_gen(@transformations); # # Print the generic instruction table. @@ -602,7 +602,7 @@ sub emulator_output { my($arity) = $gen_arity[$i]; printf "/* %3d */ ", $i; if (!defined $name) { - &init_item("", 0, 0, 0, -1); + init_item("", 0, 0, 0, -1); } else { my($key) = "$name/$arity"; my($tr) = defined $gen_transform_offset{$key} ? @@ -614,7 +614,7 @@ sub emulator_output { $is_transformed{$name,$arity} or error("instruction $key has no specific instruction"); $spec_op = -1 unless defined $spec_op; - &init_item($name, $arity, $spec_op, $num_specific, $tr); + init_item($name, $arity, $spec_op, $num_specific, $tr); } } print "};\n"; @@ -624,7 +624,7 @@ sub emulator_output { # $name = "$outdir/beam_opcodes.h"; open(STDOUT, ">$name") || die "Failed to open $name for writing: $!\n"; - &comment('C'); + comment('C'); print "#ifndef __OPCODES_H__\n"; print "#define __OPCODES_H__\n\n"; @@ -662,14 +662,14 @@ sub emulator_output { my $letter; my $tag_num = 0; - &comment('C', "The following operand types for generic instructions", + comment('C', "The following operand types for generic instructions", "occur in beam files."); foreach $letter (split('', $compiler_types)) { print "#define TAG_$letter $tag_num\n"; $tag_num++; } print "\n"; - &comment('C', "The following operand types are only used in the loader."); + comment('C', "The following operand types are only used in the loader."); foreach $letter (split('', $loader_types)) { print "#define TAG_$letter $tag_num\n"; $tag_num++; @@ -736,26 +736,26 @@ sub emulator_output { $name = "$outdir/beam_tr_funcs.h"; open(STDOUT, ">$name") || die "Failed to open $name for writing: $!\n"; - &comment('C'); - &tr_gen_call(@call_table); + comment('C'); + tr_gen_call(@call_table); $name = "$outdir/beam_pred_funcs.h"; open(STDOUT, ">$name") || die "Failed to open $name for writing: $!\n"; - &comment('C'); - &tr_gen_call(@pred_table); + comment('C'); + tr_gen_call(@pred_table); # # Implementation of operations for emulator. # $name = "$outdir/beam_hot.h"; open(STDOUT, ">$name") || die "Failed to open $name for writing: $!\n"; - &comment('C'); - &print_code(\%hot_code); + comment('C'); + print_code(\%hot_code); $name = "$outdir/beam_cold.h"; open(STDOUT, ">$name") || die "Failed to open $name for writing: $!\n"; - &comment('C'); - &print_code(\%cold_code); + comment('C'); + print_code(\%cold_code); } @@ -818,7 +818,7 @@ sub compiler_output { open(STDOUT, ">$outdir/$name") || die "Failed to open $name for writing: $!\n"; print "-module($module).\n"; - &comment('erlang'); + comment('erlang'); print "-export([format_number/0]).\n"; print "-export([opcode/2,opname/1]).\n"; @@ -830,7 +830,7 @@ sub compiler_output { for ($i = 0; $i < @gen_opname; $i++) { next unless defined $gen_opname[$i]; print "%%" if $obsolete[$i]; - print "opcode(", "e($gen_opname[$i]), ", $gen_arity[$i]) -> $i;\n"; + print "opcode(", quote($gen_opname[$i]), ", $gen_arity[$i]) -> $i;\n"; } print "opcode(Name, Arity) -> erlang:error(badarg, [Name,Arity]).\n\n"; @@ -838,7 +838,7 @@ sub compiler_output { for ($i = 0; $i < @gen_opname; $i++) { next unless defined $gen_opname[$i]; print "opname($i) -> {", - "e($gen_opname[$i]), ",$gen_arity[$i]};\n"; + quote($gen_opname[$i]), ",$gen_arity[$i]};\n"; } print "opname(Number) -> erlang:error(badarg, [Number]).\n"; @@ -847,7 +847,7 @@ sub compiler_output { # my($hrl_name) = "$outdir/${module}.hrl"; open(STDOUT, ">$hrl_name") || die "Failed to open $hrl_name for writing: $!\n"; - &comment('erlang'); + comment('erlang'); for ($i = 0; $i < @tag_type && $i < 8; $i++) { print "-define(tag_$tag_type[$i], $i).\n"; @@ -863,11 +863,33 @@ sub syntax_check { my($name, @args) = @_; my($i); - &error("Bad opcode name '$name'") + error("Bad opcode name '$name'") unless $name =~ /^[a-z][\w\d_]*$/; for ($i = 0; $i < @args; $i++) { - &error("Argument " . ($i+1) . ": invalid type '$args[$i]'") - unless defined $arg_size{$args[$i]}; + foreach my $type (split(//, $args[$i])) { + error("Argument " . ($i+1) . ": invalid type '$type'") + unless defined $arg_size{$type}; + } + } +} + +sub save_specific_ops { + my($name,$arity,$hot,@args) = @_; + my(@res) = (""); + + foreach my $arg (@args) { + my @new_res = (); + foreach my $type (split(//, $arg)) { + foreach my $args (@res) { + push @new_res, "$args$type"; + } + } + @res = @new_res; + } + my $key = "$name/$arity"; + foreach my $args (@res) { + @args = split //, $args; + push @{$specific_op{$key}}, [$name,$hot,@args]; } } @@ -928,7 +950,6 @@ sub basic_generator { my($tmp_arg_num) = 1; my($pack_spec) = ''; my($var_decls) = ''; - my($gen_dest_arg) = 'StoreSimpleDest'; my($i); my($no_prefetch) = 0; @@ -964,7 +985,7 @@ sub basic_generator { # if ($flags =~ /-pack/ && $hot) { - ($prefix, $pack_spec, @args) = &do_pack(@args); + ($prefix, $pack_spec, @args) = do_pack(@args); } # @@ -994,11 +1015,11 @@ sub basic_generator { push(@f_types, $_); $prefix .= "GetR($size, $tmp);\n"; last SWITCH; }; - /d/ and do { $var_decls .= "Eterm dst; "; - push(@f, "dst"); + /d/ and do { $var_decls .= "Eterm dst; Eterm* dst_ptr; "; + push(@f, "*dst_ptr"); push(@f_types, $_); $prefix .= "dst = Arg($size);\n"; - $gen_dest_arg = 'StoreResult'; + $prefix .= "dst_ptr = REG_TARGET_PTR(dst);\n"; last SWITCH; }; defined($incl_arg{$_}) @@ -1017,14 +1038,6 @@ sub basic_generator { } # - # If requested, pass a pointer to the destination register. - # The destination must be the last operand. - # - if ($flags =~ /-gen_dest/) { - push(@f, $gen_dest_arg); - } - - # # Add a fail action macro if requested. # @@ -1195,7 +1208,7 @@ sub do_pack { } $down = "$instr[$ap]$down"; - my($unpack) = &make_unpack($tmpnum, $shift[$ap], $mask[$ap]); + my($unpack) = make_unpack($tmpnum, $shift[$ap], $mask[$ap]); $args[$i] = "pack:$this_size:$reg" . "b($unpack)"; if (++$ap == $args_per_word) { @@ -1259,7 +1272,7 @@ sub parse_transformation { foreach (@from) { if (/^(\w+)\((.*?)\)/) { my($name, $arglist) = ($1, $2); - $_ = (&compile_transform_function($name, split(/\s*,\s*/, $arglist))); + $_ = (compile_transform_function($name, split(/\s*,\s*/, $arglist))); } else { (@op) = split; ($rest_var,$_) = compile_transform(1, $rest_var, @op); @@ -1275,7 +1288,7 @@ sub parse_transformation { my @to; if ($to =~ /^(\w+)\((.*?)\)/) { my($name, $arglist) = ($1, $2); - @to = (&compile_transform_function($name, split(/\s*,\s*/, $arglist))); + @to = (compile_transform_function($name, split(/\s*,\s*/, $arglist))); } else { @to = split(/\s*\|\s*/, $to); foreach (@to) { @@ -1297,7 +1310,7 @@ sub compile_transform { my $arity = 0; foreach (@ops) { - my(@list) = &tr_parse_op($src, $_); + my(@list) = tr_parse_op($src, $_); if ($list[1] eq '*') { $rest_var = $list[0]; } elsif (defined $rest_var and $list[0] eq $rest_var) { @@ -1334,7 +1347,7 @@ sub tr_parse_op { if (/^([A-Z]\w*)(.*)/) { $var = $1; $_ = $2; - &error("garbage after variable") + error("garbage after variable") unless /^=(.*)/ or /^(\s*)$/; $_ = $1; } @@ -1345,7 +1358,7 @@ sub tr_parse_op { $type = $1; $_ = $2; foreach (split('', $type)) { - &error("bad type in $op") + error("bad type in $op") unless defined $type_bit{$_} or $type eq '*'; $_ eq 'r' and error("$op: 'r' is not allowed in transformations") @@ -1392,7 +1405,7 @@ sub tr_parse_op { # Nothing more is allowed after the command. - &error("garbage '$_' after operand: $op") + error("garbage '$_' after operand: $op") unless /^\s*$/; # Test that destination has no conditions. @@ -1519,7 +1532,7 @@ sub tr_gen_from { # Check that $name/$arity refers to a valid generic instruction. # - &error($where, "invalid generic op $name/$arity") + error($where, "invalid generic op $name/$arity") unless defined $gen_opnum{$name,$arity}; $opnum = $gen_opnum{$name,$arity}; @@ -1547,11 +1560,11 @@ sub tr_gen_from { $type_mask |= $type_bit{$_}; } if ($cond ne 'is_eq') { - push(@code, &make_op($types, 'is_type', $type_mask)); + push(@code, make_op($types, 'is_type', $type_mask)); } else { $cond = ''; - push(@code, &make_op("$types== $val", 'is_type_eq', - $type_mask, $val)); + push(@code, make_op("$types== $val", 'is_type_eq', + $type_mask, $val)); } } } @@ -1560,12 +1573,12 @@ sub tr_gen_from { my($m, $f, $a) = split(/:/, $val); $ignored_var = ''; $may_fail = 1; - push(@code, &make_op('', "$cond", "am_$m", + push(@code, make_op('', "$cond", "am_$m", "am_$f", $a)); } elsif ($cond ne '') { $ignored_var = ''; $may_fail = 1; - push(@code, &make_op('', "$cond", $val)); + push(@code, make_op('', "$cond", $val)); } if ($var ne '') { @@ -1591,7 +1604,7 @@ sub tr_gen_from { $var_type{$var} = 'scalar'; $var{$var} = $var_num; $var_num++; - push(@code, &make_op($var, 'set_var', $var{$var})); + push(@code, make_op($var, 'set_var', $var{$var})); } } if (is_instr($code[$#code], 'set_var')) { @@ -1600,7 +1613,7 @@ sub tr_gen_from { my $var = $ref->[1][1]; push(@code, make_op($comment, 'set_var_next_arg', $var)); } else { - push(@code, &make_op($ignored_var, 'next_arg')); + push(@code, make_op($ignored_var, 'next_arg')); } } @@ -1645,7 +1658,7 @@ sub tr_gen_to { my(@args); foreach $var (@ops) { - &error($where, "variable '$var' unbound") + error($where, "variable '$var' unbound") unless defined $var{$var}; if ($var_type{$var} eq 'scalar') { push(@args, "var[$var{$var}]"); @@ -1668,7 +1681,7 @@ sub tr_gen_to { # my($key) = "$name/$arity"; - &error($where, "invalid generic op $name/$arity") + error($where, "invalid generic op $name/$arity") unless defined $gen_opnum{$name,$arity}; my $opnum = $gen_opnum{$name,$arity}; @@ -1683,15 +1696,15 @@ sub tr_gen_to { if ($type eq '*') { push(@code, make_op($var, 'store_rest_args')); } elsif ($var ne '') { - &error($where, "variable '$var' unbound") + error($where, "variable '$var' unbound") unless defined $var{$var}; my $op = make_op($var, 'store_var_next_arg', $var{$var}); op_slot_usage($op, $var{$var}); push(@code, $op); } elsif ($type ne '') { - push(@code, &make_op('', 'store_type', "TAG_$type")); + push(@code, make_op('', 'store_type', "TAG_$type")); if ($type_val) { - push(@code, &make_op('', 'store_val', $type_val)); + push(@code, make_op('', 'store_val', $type_val)); } push(@code, make_op('', 'next_arg')); } diff --git a/erts/etc/common/erlexec.c b/erts/etc/common/erlexec.c index 70520eea15..51ed2d0dff 100644 --- a/erts/etc/common/erlexec.c +++ b/erts/etc/common/erlexec.c @@ -555,7 +555,7 @@ int main(int argc, char **argv) if(s) { add_Eargs(s); /* argv[0] = scriptname*/ } else { - add_Eargs(progname); /* argv[0] = erl or cerl */ + add_Eargs(emu); /* argv[0] = erl or cerl */ } /* * Add the bindir to the path (unless it is there already). diff --git a/erts/etc/common/escript.c b/erts/etc/common/escript.c index 630e241882..7f0af77a4c 100644 --- a/erts/etc/common/escript.c +++ b/erts/etc/common/escript.c @@ -433,7 +433,7 @@ main(int argc, char** argv) char* emulator; char* env; char* basename; - char* absname; + char* def_emu_lookup_path; char scriptname[PMAX]; char** last_opt; char** first_opt; @@ -480,6 +480,7 @@ main(int argc, char** argv) #else if (strcmp(basename, "escript") == 0) { #endif + def_emu_lookup_path = argv[0]; /* * Locate all options before the script name. */ @@ -498,27 +499,24 @@ main(int argc, char** argv) argc--; argv++; } else { + char *absname = find_prog(argv[0]); #ifdef __WIN32__ - int len; -#endif - absname = find_prog(argv[0]); -#ifdef __WIN32__ - len = strlen(absname); + int len = strlen(absname); if (len >= 4 && _stricmp(absname+len-4, ".exe") == 0) { absname[len-4] = '\0'; } #endif - erts_snprintf(scriptname, sizeof(scriptname), "%s.escript", absname); - efree(absname); + efree(absname); + def_emu_lookup_path = scriptname; } /* Determine path to emulator */ emulator = env = get_env("ESCRIPT_EMULATOR"); if (emulator == NULL) { - emulator = get_default_emulator(scriptname); + emulator = get_default_emulator(def_emu_lookup_path); } if (strlen(emulator) >= PMAX) @@ -528,7 +526,7 @@ main(int argc, char** argv) * Push initial arguments. */ - push_words(emulator); + PUSH(emulator); free_env_val(env); PUSH("+B"); diff --git a/erts/etc/unix/etp-commands.in b/erts/etc/unix/etp-commands.in index b7b3a2ae99..8f70f879d5 100644 --- a/erts/etc/unix/etp-commands.in +++ b/erts/etc/unix/etp-commands.in @@ -1,3 +1,4 @@ +# -*- gdb-script -*- # # %CopyrightBegin% # @@ -2153,13 +2154,22 @@ define etp-processes printf "No processes, since system isn't initialized!\n" else set $proc_ix = 0 - while $proc_ix < erts_proc.r.o.max - set $proc = (Process *) *((UWord *) &erts_proc.r.o.tab[$proc_ix]) - if ($proc != ((Process *) 0) && $proc != &erts_invalid_process) + set $proc_max_ix = erts_proc.r.o.max + set $proc_tab = erts_proc.r.o.tab + set $invalid_proc = &erts_invalid_process + set $proc_decentile = $proc_max_ix / 10 + set $proc_printile = $proc_decentile + while $proc_ix < $proc_max_ix + set $proc = (Process *) *((UWord *) ($proc_tab + $proc_ix)) + if ($proc != ((Process *) 0) && $proc != $invalid_proc) printf "---\n" printf " Pix: %d\n", $proc_ix etp-process-info $proc end + if $proc_ix == $proc_printile + printf "--- %d%% (%d / %d) searched\n", $proc_printile / $proc_decentile * 10, $proc_ix, $proc_max_ix + set $proc_printile += $proc_decentile + end set $proc_ix++ end printf "---\n", @@ -2479,15 +2489,19 @@ document etp-port-info %--------------------------------------------------------------------------- end - define etp-ports if (!erts_initialized) printf "No ports, since system isn't initialized!\n" else set $port_ix = 0 - while $port_ix < erts_port.r.o.max - set $port = (Port *) *((UWord *) &erts_port.r.o.tab[$port_ix]) - if ($port != ((Port *) 0) && $port != &erts_invalid_port) + set $port_max_ix = erts_port.r.o.max + set $port_tab = erts_port.r.o.tab + set $invalid_port = &erts_invalid_port + set $port_decentile = $port_max_ix / 10 + set $port_printile = $port_decentile + while $port_ix < $port_max_ix + set $port = (Port *) *((UWord *) ($port_tab + $port_ix)) + if ($port != ((Port *) 0) && $port != $invalid_port) if (*(((Uint32 *) &(((Port *) $port)->state))) & 0x100) == 0 # I.e, not free printf "---\n" @@ -2495,6 +2509,10 @@ define etp-ports etp-port-info $port end end + if $port_ix == $port_printile + printf "--- %d%% (%d / %d) searched\n", $port_printile / $port_decentile * 10, $port_ix, $port_max_ix + set $port_printile += $port_decentile + end set $port_ix++ end printf "---\n", @@ -2810,10 +2828,10 @@ define etp-run-queue-info-internal else if ($sched_type == 1) printf "\n--- Dirty CPU Run Queue ---\n" - set $runq = &erts_aligned_run_queues[-1].runq + set $runq = &erts_aligned_run_queues[erts_no_run_queues].runq else printf "\n--- Dirty I/O Run Queue ---\n" - set $runq = &erts_aligned_run_queues[-2].runq + set $runq = &erts_aligned_run_queues[erts_no_run_queues+1].runq end end printf " Length: total=%d", *((Uint32 *) &($runq->len)) diff --git a/erts/preloaded/ebin/erlang.beam b/erts/preloaded/ebin/erlang.beam Binary files differindex 980df873ca..63518ed6e1 100644 --- a/erts/preloaded/ebin/erlang.beam +++ b/erts/preloaded/ebin/erlang.beam diff --git a/erts/preloaded/src/erlang.erl b/erts/preloaded/src/erlang.erl index fcb27ef575..72dd804412 100644 --- a/erts/preloaded/src/erlang.erl +++ b/erts/preloaded/src/erlang.erl @@ -2306,6 +2306,8 @@ spawn_opt(_Tuple) -> -spec statistics(active_tasks) -> [ActiveTasks] when ActiveTasks :: non_neg_integer(); + (active_tasks_all) -> [ActiveTasks] when + ActiveTasks :: non_neg_integer(); (context_switches) -> {ContextSwitches,0} when ContextSwitches :: non_neg_integer(); (exact_reductions) -> {Total_Exact_Reductions, @@ -2335,6 +2337,8 @@ spawn_opt(_Tuple) -> (run_queue) -> non_neg_integer(); (run_queue_lengths) -> [RunQueueLength] when RunQueueLength :: non_neg_integer(); + (run_queue_lengths_all) -> [RunQueueLength] when + RunQueueLength :: non_neg_integer(); (runtime) -> {Total_Run_Time, Time_Since_Last_Call} when Total_Run_Time :: non_neg_integer(), Time_Since_Last_Call :: non_neg_integer(); @@ -2347,9 +2351,13 @@ spawn_opt(_Tuple) -> ActiveTime :: non_neg_integer(), TotalTime :: non_neg_integer(); (total_active_tasks) -> ActiveTasks when + ActiveTasks :: non_neg_integer(); + (total_active_tasks_all) -> ActiveTasks when ActiveTasks :: non_neg_integer(); (total_run_queue_lengths) -> TotalRunQueueLengths when TotalRunQueueLengths :: non_neg_integer(); + (total_run_queue_lengths_all) -> TotalRunQueueLengths when + TotalRunQueueLengths :: non_neg_integer(); (wall_clock) -> {Total_Wallclock_Time, Wallclock_Time_Since_Last_Call} when Total_Wallclock_Time :: non_neg_integer(), diff --git a/lib/common_test/doc/src/ct_ssh.xml b/lib/common_test/doc/src/ct_ssh.xml index 137e4c3f1d..63627d8840 100644 --- a/lib/common_test/doc/src/ct_ssh.xml +++ b/lib/common_test/doc/src/ct_ssh.xml @@ -1034,6 +1034,33 @@ ChannelId, 0, Data, End, Timeout)</c></seealso>.</p> </func> <func> + <name>shell(SSH, ChannelId) -> ok | {error, Reason}</name> + <fsummary>Equivalent to shell(SSH, ChannelId, DefaultTimeout).</fsummary> + <desc><marker id="shell-2"/> + <p>Equivalent to + <seealso marker="#shell-3"><c>ct_ssh:shell(SSH, ChannelId, + DefaultTimeout)</c></seealso>.</p> + </desc> + </func> + + <func> + <name>shell(SSH, ChannelId, Timeout) -> ok | {error, Reason}</name> + <fsummary>Requests that the user default shell is executed at the + server end.</fsummary> + <type> + <v>SSH = connection()</v> + <v>ChannelId = integer()</v> + <v>Timeout = integer()</v> + <v>Reason = term()</v> + </type> + <desc><marker id="shell-3"/> + <p>Requests that the user default shell (typically defined in + <c>/etc/passwd</c> in Unix systems) is executed at the + server end.</p> + </desc> + </func> + + <func> <name>subsystem(SSH, ChannelId, Subsystem) -> Status | {error, Reason}</name> <fsummary>Equivalent to subsystem(SSH, ChannelId, Subsystem, DefaultTimeout).</fsummary> diff --git a/lib/common_test/src/ct_ssh.erl b/lib/common_test/src/ct_ssh.erl index 6ab3bf036c..5ac91f396b 100644 --- a/lib/common_test/src/ct_ssh.erl +++ b/lib/common_test/src/ct_ssh.erl @@ -68,7 +68,8 @@ send_and_receive/3, send_and_receive/4, send_and_receive/5, send_and_receive/6, exec/2, exec/3, exec/4, - subsystem/3, subsystem/4]). + subsystem/3, subsystem/4, + shell/2, shell/3]). %% STFP Functions -export([sftp_connect/1, @@ -94,6 +95,7 @@ -record(state, {ssh_ref, conn_type, target}). +-type handle() :: pid(). %%%----------------------------------------------------------------- %%%------------------------ SSH COMMANDS --------------------------- @@ -490,6 +492,22 @@ subsystem(SSH, ChannelId, Subsystem, Timeout) -> call(SSH, {subsystem,ChannelId,Subsystem,Timeout}). +-spec shell(SSH, ChannelId) -> Result when + SSH :: handle() | ct:target_name(), + ChannelId :: ssh:ssh_channel_id(), + Result :: ok | {error,term()}. +shell(SSH, ChannelId) -> + shell(SSH, ChannelId, ?DEFAULT_TIMEOUT). + +-spec shell(SSH, ChannelId, Timeout) -> Result when + SSH :: handle() | ct:target_name(), + ChannelId :: ssh:ssh_channel_id(), + Timeout :: timeout(), + Result :: ok | {error,term()}. +shell(SSH, ChannelId, Timeout) -> + call(SSH, {shell,ChannelId,Timeout}). + + %%%----------------------------------------------------------------- %%%------------------------ SFTP COMMANDS -------------------------- @@ -1067,6 +1085,14 @@ handle_msg({subsystem,Chn,Subsystem,TO}, State) -> Result = ssh_connection:subsystem(SSHRef, Chn, Subsystem, TO), {Result,State}; +handle_msg({shell,Chn,TO}, State) -> + #state{ssh_ref=SSHRef, target=Target} = State, + try_log(heading(shell,Target), + "SSH Ref: ~p, Chn: ~p, Timeout: ~p", + [SSHRef,Chn,TO]), + Result = ssh_connection:shell(SSHRef, Chn), + {Result,State}; + %% --- SFTP Commands --- handle_msg({read_file,Srv,File}=Cmd, S=#state{ssh_ref=SSHRef}) -> diff --git a/lib/common_test/test/ct_keep_logs_SUITE.erl b/lib/common_test/test/ct_keep_logs_SUITE.erl index f0c85a88a2..6b7aaa57ac 100644 --- a/lib/common_test/test/ct_keep_logs_SUITE.erl +++ b/lib/common_test/test/ct_keep_logs_SUITE.erl @@ -70,8 +70,7 @@ keep_logs(Config) -> Opts = [{suite,Suite},{label,keep_logs} | Opts0], LogDir=?config(logdir,Opts), - KeepLogsDir = filename:join(LogDir,unique_name("keep_logs-")), - ok = file:make_dir(KeepLogsDir), + KeepLogsDir = create_dir(filename:join(LogDir,"keep_logs-")), Opts1 = lists:keyreplace(logdir,1,Opts,{logdir,KeepLogsDir}), ct:log("New LogDir = ~s", [KeepLogsDir]), @@ -121,7 +120,11 @@ keep_logs(Config) -> 9 = length(L10), 0 = ct_test_support:run_ct_script_start([{keep_logs,10}|Opts1], Config), L11 = filelib:wildcard(WC), - 10 = length(L11). + 10 = length(L11), + + {ok,Content} = file:list_dir(KeepLogsDir), + ct:log("Deleting dir: ~p~nContent: ~p~n",[KeepLogsDir,Content]), + ct_test_support:rm_dir(KeepLogsDir). %% Test the keep_logs option togwther with the refresh_logs option refresh_logs(Config) -> @@ -129,8 +132,7 @@ refresh_logs(Config) -> Suite = filename:join(DataDir, "keep_logs_SUITE"), Opts0 = ct_test_support:get_opts(Config), LogDir=?config(logdir,Opts0), - KeepLogsDir = filename:join(LogDir,unique_name("refresh_logs-")), - ok = file:make_dir(KeepLogsDir), + KeepLogsDir = create_dir(filename:join(LogDir,"refresh_logs-")), Opts1 = lists:keyreplace(logdir,1,Opts0,{logdir,KeepLogsDir}), ct:log("New LogDir = ~s", [KeepLogsDir]), @@ -178,9 +180,20 @@ refresh_logs(Config) -> L10 = filelib:wildcard(WC), 4 = length(L10), - ok. + {ok,Content} = file:list_dir(KeepLogsDir), + ct:log("Deleting dir: ~p~nContent: ~p~n",[KeepLogsDir,Content]), + ct_test_support:rm_dir(KeepLogsDir). + %%%----------------------------------------------------------------- %%% Internal -unique_name(Prefix) -> +create_dir(Prefix) -> I = erlang:unique_integer([positive]), - Prefix ++ integer_to_list(I). + Dir = Prefix ++ integer_to_list(I), + case filelib:is_dir(Dir) of + true -> + %% Try again + create_dir(Prefix); + false -> + ok = file:make_dir(Dir), + Dir + end. diff --git a/lib/common_test/test/ct_test_support.erl b/lib/common_test/test/ct_test_support.erl index 935d032667..ba7aadfeec 100644 --- a/lib/common_test/test/ct_test_support.erl +++ b/lib/common_test/test/ct_test_support.erl @@ -45,6 +45,8 @@ -export([unique_timestamp/0]). +-export([rm_dir/1]). + -include_lib("kernel/include/file.hrl"). %%%----------------------------------------------------------------- diff --git a/lib/compiler/src/compile.erl b/lib/compiler/src/compile.erl index b3c8c42af7..c6e61d543e 100644 --- a/lib/compiler/src/compile.erl +++ b/lib/compiler/src/compile.erl @@ -467,8 +467,10 @@ mpf(Ms) -> passes(Type, Opts) -> {Ext,Passes0} = passes_1(Opts), Passes1 = case Type of - file -> Passes0; - forms -> tl(Passes0) + file -> + Passes0; + forms -> + fix_first_pass(Passes0) end, Passes = select_passes(Passes1, Opts), @@ -505,6 +507,22 @@ pass(from_beam) -> {".beam",[?pass(read_beam_file)|binary_passes()]}; pass(_) -> none. +%% For compilation from forms, replace the first pass with a pass +%% that retrieves the module name. The module name is needed for +%% proper diagnostics and for compilation to native code. + +fix_first_pass([{parse_core,_}|Passes]) -> + [?pass(get_module_name_from_core)|Passes]; +fix_first_pass([{beam_consult_asm,_}|Passes]) -> + [?pass(get_module_name_from_asm)|Passes]; +fix_first_pass([{read_beam_file,_}|Passes]) -> + [?pass(get_module_name_from_beam)|Passes]; +fix_first_pass([_|Passes]) -> + %% When compiling from abstract code, the module name + %% will be set after running the v3_core pass. + Passes. + + %% select_passes([Command], Opts) -> [{Name,Function}] %% Interpret the lists of commands to return a pure list of passes. %% @@ -836,6 +854,12 @@ beam_consult_asm(_Code, St) -> {error,St#compile{errors=St#compile.errors ++ Es}} end. +get_module_name_from_asm({Mod,_,_,_,_}=Asm, St) -> + {ok,Asm,St#compile{module=Mod}}; +get_module_name_from_asm(Asm, St) -> + %% Invalid Beam assembly code. Let it crash in a later pass. + {ok,Asm,St}. + read_beam_file(_Code, St) -> case file:read_file(St#compile.ifile) of {ok,Beam} -> @@ -853,6 +877,16 @@ read_beam_file(_Code, St) -> {error,St#compile{errors=St#compile.errors ++ Es}} end. +get_module_name_from_beam(Beam, St) -> + case beam_lib:info(Beam) of + {error,beam_lib,Error} -> + Es = [{"((forms))",[{none,beam_lib,Error}]}], + {error,St#compile{errors=St#compile.errors ++ Es}}; + Info -> + {module,Mod} = keyfind(module, 1, Info), + {ok,Beam,St#compile{module=Mod}} + end. + no_native_compilation(BeamFile, #compile{options=Opts0}) -> case beam_lib:chunks(BeamFile, ["CInf"]) of {ok,{_,[{"CInf",Term0}]}} -> @@ -940,6 +974,16 @@ parse_core(_Code, St) -> {error,St#compile{errors=St#compile.errors ++ Es}} end. +get_module_name_from_core(Core, St) -> + try + Mod = cerl:concrete(cerl:module_name(Core)), + {ok,Core,St#compile{module=Mod}} + catch + _:_ -> + %% Invalid Core Erlang code. Let it crash in a later pass. + {ok,Core,St} + end. + compile_options([{attribute,_L,compile,C}|Fs]) when is_list(C) -> C ++ compile_options(Fs); compile_options([{attribute,_L,compile,C}|Fs]) -> diff --git a/lib/compiler/src/genop.tab b/lib/compiler/src/genop.tab index 5e0c2b3ebf..9efb9ad6f8 100755 --- a/lib/compiler/src/genop.tab +++ b/lib/compiler/src/genop.tab @@ -538,6 +538,8 @@ BEAM_FORMAT_NUMBER=0 157: has_map_fields/3 158: get_map_elements/3 +# OTP 20 + ## @spec is_tagged_tuple Lbl Reg N Atom ## @doc Test the type of Reg and jumps to Lbl if it is not a tuple. ## Test the arity of Reg and jumps to Lbl if it is not N. diff --git a/lib/compiler/test/beam_validator_SUITE.erl b/lib/compiler/test/beam_validator_SUITE.erl index ca85eef688..75bfbf68cc 100644 --- a/lib/compiler/test/beam_validator_SUITE.erl +++ b/lib/compiler/test/beam_validator_SUITE.erl @@ -446,7 +446,7 @@ do_bin_opt(Mod, Asm) -> do_bin_opt(Transform, Mod, Asm0) -> Asm = Transform(Asm0), case compile:forms(Asm, [from_asm,no_postopt,return]) of - {ok,[],Code,_Warnings} when is_binary(Code) -> + {ok,Mod,Code,_Warnings} when is_binary(Code) -> ok; {error,Errors0,_} -> %% beam_validator must return errors, not simply crash, diff --git a/lib/compiler/test/compile_SUITE.erl b/lib/compiler/test/compile_SUITE.erl index f9bcb044ac..4e2753ba5f 100644 --- a/lib/compiler/test/compile_SUITE.erl +++ b/lib/compiler/test/compile_SUITE.erl @@ -32,7 +32,7 @@ binary/1, makedep/1, cond_and_ifdef/1, listings/1, listings_big/1, other_output/1, kernel_listing/1, encrypted_abstr/1, strict_record/1, utf8_atoms/1, utf8_functions/1, extra_chunks/1, - cover/1, env/1, core/1, + cover/1, env/1, core_pp/1, core_roundtrip/1, asm/1, optimized_guards/1, sys_pre_attributes/1, dialyzer/1, warnings/1, pre_load_check/1, env_compiler_options/1, @@ -51,7 +51,7 @@ all() -> binary, makedep, cond_and_ifdef, listings, listings_big, other_output, kernel_listing, encrypted_abstr, strict_record, utf8_atoms, utf8_functions, extra_chunks, - cover, env, core, core_roundtrip, asm, optimized_guards, + cover, env, core_pp, core_roundtrip, asm, optimized_guards, sys_pre_attributes, dialyzer, warnings, pre_load_check, env_compiler_options, custom_debug_info, bc_options]. @@ -163,6 +163,24 @@ forms_2(Config) when is_list(Config) -> ok end, + {ok,simple,Core} = compile:forms(SimpleCode, [to_core0,binary]), + forms_compile_and_load(Core, [from_core]), + forms_compile_and_load(Core, [from_core,native]), + + {ok,simple,Asm} = compile:forms(SimpleCode, [to_asm,binary]), + forms_compile_and_load(Asm, [from_asm]), + forms_compile_and_load(Asm, [from_asm,native]), + + {ok,simple,Beam} = compile:forms(SimpleCode, []), + forms_compile_and_load(Beam, [from_beam]), + forms_compile_and_load(Beam, [from_beam,native]), + + %% Cover the error handling code. + error = compile:forms(bad_core, [from_core,report]), + error = compile:forms(bad_asm, [from_asm,report]), + error = compile:forms(<<"bad_beam">>, [from_beam,report]), + error = compile:forms(<<"bad_beam">>, [from_beam,native,report]), + ok. @@ -180,6 +198,14 @@ forms_load_code(Mod, Src, Bin) -> SourceOption. +forms_compile_and_load(Code, Opts) -> + Mod = simple, + {ok,Mod,Bin} = compile:forms(Code, Opts), + {module,Mod} = code:load_binary(Mod, "ignore", Bin), + _ = Mod:module_info(), + true = code:delete(simple), + false = code:purge(simple), + ok. module_mismatch(Config) when is_list(Config) -> DataDir = proplists:get_value(data_dir, Config), @@ -795,9 +821,9 @@ env_1(Simple, Target) -> %% Test pretty-printing in Core Erlang format and then try to %% compile the generated Core Erlang files. -core(Config) when is_list(Config) -> +core_pp(Config) when is_list(Config) -> PrivDir = proplists:get_value(priv_dir, Config), - Outdir = filename:join(PrivDir, "core"), + Outdir = filename:join(PrivDir, atom_to_list(?FUNCTION_NAME)), ok = file:make_dir(Outdir), TestBeams = get_unique_beam_files(), @@ -805,11 +831,11 @@ core(Config) when is_list(Config) -> {raw_abstract_v1,Abstr}}]}} = beam_lib:chunks(Beam, [abstract_code]), {Mod,Abstr} end || Beam <- TestBeams], - test_lib:p_run(fun(F) -> do_core(F, Outdir) end, Abstr). + test_lib:p_run(fun(F) -> do_core_pp(F, Outdir) end, Abstr). -do_core({M,A}, Outdir) -> +do_core_pp({M,A}, Outdir) -> try - do_core_1(M, A, Outdir) + do_core_pp_1(M, A, Outdir) catch throw:{error,Error} -> io:format("*** compilation failure '~p' for module ~s\n", @@ -821,7 +847,7 @@ do_core({M,A}, Outdir) -> error end. -do_core_1(M, A, Outdir) -> +do_core_pp_1(M, A, Outdir) -> {ok,M,Core0} = compile:forms(A, [to_core]), CoreFile = filename:join(Outdir, atom_to_list(M)++".core"), CorePP = core_pp:format(Core0), @@ -835,7 +861,7 @@ do_core_1(M, A, Outdir) -> ok = file:delete(CoreFile), %% Compile as usual (including optimizations). - compile_forms(Core, [clint,from_core,binary]), + compile_forms(M, Core, [clint,from_core,binary]), %% Don't optimize to test that we are not dependent %% on the Core Erlang optmimization passes. @@ -844,13 +870,13 @@ do_core_1(M, A, Outdir) -> %% records; if sys_core_fold was run it would fix %% that; if sys_core_fold was not run v3_kernel would %% crash.) - compile_forms(Core, [clint,from_core,no_copt,binary]), + compile_forms(M, Core, [clint,from_core,no_copt,binary]), ok. -compile_forms(Forms, Opts) -> +compile_forms(Mod, Forms, Opts) -> case compile:forms(Forms, [report_errors|Opts]) of - {ok,[],_} -> ok; + {ok,Mod,_} -> ok; Other -> throw({error,Other}) end. diff --git a/lib/crypto/c_src/crypto.c b/lib/crypto/c_src/crypto.c index 793cff166c..688ec339aa 100644 --- a/lib/crypto/c_src/crypto.c +++ b/lib/crypto/c_src/crypto.c @@ -609,7 +609,7 @@ struct digest_type_t { }md; }; -struct digest_type_t digest_types[] = +static struct digest_type_t digest_types[] = { {{"md4"}, {&EVP_md4}}, {{"md5"}, {&EVP_md5}}, @@ -666,7 +666,7 @@ struct cipher_type_t { #define COND_NO_DES_PTR(Ptr) (Ptr) #endif -struct cipher_type_t cipher_types[] = +static struct cipher_type_t cipher_types[] = { {{"rc2_cbc"}, #ifndef OPENSSL_NO_RC2 @@ -1845,7 +1845,7 @@ static ERL_NIF_TERM aes_cfb_128_crypt_nif(ErlNifEnv* env, int argc, const ERL_NI AES_cfb128_encrypt((unsigned char *) text.data, enif_make_new_binary(env, text.size, &ret), text.size, &aes_key, ivec_clone, &new_ivlen, - (argv[3] != atom_true)); + (argv[3] == atom_true)); CONSUME_REDS(env,text); return ret; } diff --git a/lib/crypto/test/Makefile b/lib/crypto/test/Makefile index 5a81c84558..138081d386 100644 --- a/lib/crypto/test/Makefile +++ b/lib/crypto/test/Makefile @@ -77,6 +77,7 @@ release_spec: release_tests_spec: $(TEST_TARGET) $(INSTALL_DIR) "$(RELSYSDIR)" $(INSTALL_DATA) crypto.spec crypto.cover $(RELTEST_FILES) "$(RELSYSDIR)" + @tar cfh - crypto_SUITE_data | (cd "$(RELSYSDIR)"; tar xf -) chmod -R u+w "$(RELSYSDIR)" release_docs_spec: diff --git a/lib/crypto/test/crypto_SUITE.erl b/lib/crypto/test/crypto_SUITE.erl index 8cb08cade0..30ee782fe9 100644 --- a/lib/crypto/test/crypto_SUITE.erl +++ b/lib/crypto/test/crypto_SUITE.erl @@ -169,6 +169,12 @@ groups() -> %%------------------------------------------------------------------- init_per_suite(Config) -> + file:set_cwd(datadir(Config)), + {ok, _} = zip:unzip("KAT_AES.zip"), + {ok, _} = zip:unzip("aesmmt.zip"), + {ok, _} = zip:unzip("cmactestvectors.zip"), + {ok, _} = zip:unzip("gcmtestvectors.zip"), + try crypto:start() of ok -> try crypto:strong_rand_bytes(1) of @@ -330,7 +336,7 @@ no_hmac(Config) when is_list(Config) -> cmac() -> [{doc, "Test all different cmac functions"}]. cmac(Config) when is_list(Config) -> - Pairs = proplists:get_value(cmac, Config), + Pairs = lazy_eval(proplists:get_value(cmac, Config)), lists:foreach(fun cmac_check/1, Pairs), lists:foreach(fun cmac_check/1, cmac_iolistify(Pairs)). %%-------------------------------------------------------------------- @@ -350,7 +356,7 @@ block(Config) when is_list(Config) -> ok end, - Blocks = proplists:get_value(block, Config), + Blocks = lazy_eval(proplists:get_value(block, Config)), lists:foreach(fun block_cipher/1, Blocks), lists:foreach(fun block_cipher/1, block_iolistify(Blocks)), lists:foreach(fun block_cipher_increment/1, block_iolistify(Blocks)). @@ -359,7 +365,7 @@ block(Config) when is_list(Config) -> no_block() -> [{doc, "Test disabled block ciphers"}]. no_block(Config) when is_list(Config) -> - Blocks = proplists:get_value(block, Config), + Blocks = lazy_eval(proplists:get_value(block, Config)), Args = case Blocks of [{_Type, _Key, _PlainText} = A | _] -> tuple_to_list(A); @@ -376,7 +382,7 @@ no_aead() -> [{doc, "Test disabled aead ciphers"}]. no_aead(Config) when is_list(Config) -> [{Type, Key, PlainText, Nonce, AAD, CipherText, CipherTag} | _] = - proplists:get_value(aead, Config), + lazy_eval(proplists:get_value(aead, Config)), EncryptArgs = [Type, Key, Nonce, {AAD, PlainText}], DecryptArgs = [Type, Key, Nonce, {AAD, CipherText, CipherTag}], notsup(fun crypto:block_encrypt/4, EncryptArgs), @@ -628,10 +634,15 @@ block_cipher({Type, Key, IV, PlainText, CipherText}) -> block_cipher_increment({Type, Key, IV, PlainTexts}) when Type == des_cbc; Type == aes_cbc; Type == des3_cbc -> block_cipher_increment(Type, Key, IV, IV, PlainTexts, iolist_to_binary(PlainTexts), []); +block_cipher_increment({Type, Key, IV, PlainTexts, CipherText}) + when Type == des_cbc; Type == des3_cbc -> + block_cipher_increment(Type, Key, IV, IV, PlainTexts, iolist_to_binary(PlainTexts), CipherText, []); block_cipher_increment({Type, Key, IV, PlainTexts, _CipherText}) when Type == aes_cbc -> Plain = iolist_to_binary(PlainTexts), Blocks = [iolistify(Block) || << Block:128/bitstring >> <= Plain], block_cipher_increment(Type, Key, IV, IV, Blocks, Plain, []); +block_cipher_increment({_Type, _, _, _, _}) -> + ok; block_cipher_increment({_Type, _, _, _}) -> ok; block_cipher_increment({_,_,_}) -> @@ -648,6 +659,17 @@ block_cipher_increment(Type, Key, IV0, IV, [PlainText | PlainTexts], Plain, Acc) CipherText = crypto:block_encrypt(Type, Key, IV, PlainText), NextIV = crypto:next_iv(Type, CipherText), block_cipher_increment(Type, Key, IV0, NextIV, PlainTexts, Plain, [CipherText | Acc]). +block_cipher_increment(Type, Key, IV0, _IV, [], _Plain, CipherText, Acc) -> + case iolist_to_binary(lists:reverse(Acc)) of + CipherText -> + ok; + Other -> + ct:fail({{crypto, block_decrypt, [Type, Key, IV0, CipherText]}, {expected, CipherText}, {got, Other}}) + end; +block_cipher_increment(Type, Key, IV0, IV, [PlainText | PlainTexts], Plain, CipherText, Acc) -> + CT = crypto:block_encrypt(Type, Key, IV, PlainText), + NextIV = crypto:next_iv(Type, CT), + block_cipher_increment(Type, Key, IV0, NextIV, PlainTexts, Plain, CipherText, [CT | Acc]). stream_cipher({Type, Key, PlainText}) -> Plain = iolist_to_binary(PlainText), @@ -812,6 +834,8 @@ notsup(Fun, Args) -> hexstr2point(X, Y) -> <<4:8, (hexstr2bin(X))/binary, (hexstr2bin(Y))/binary>>. +hexstr2bin(S) when is_binary(S) -> + list_to_binary(hexstr2list(binary_to_list(S))); hexstr2bin(S) -> list_to_binary(hexstr2list(S)). @@ -1181,24 +1205,24 @@ group_config(rc2_cbc, Config) -> Block = rc2_cbc(), [{block, Block} | Config]; group_config(aes_cbc128 = Type, Config) -> - Block = aes_cbc128(), - Pairs = cmac_nist(Type), + Block = fun() -> aes_cbc128(Config) end, + Pairs = fun() -> cmac_nist(Config, Type) end, [{block, Block}, {cmac, Pairs} | Config]; group_config(aes_cbc256 = Type, Config) -> - Block = aes_cbc256(), - Pairs = cmac_nist(Type), + Block = fun() -> aes_cbc256(Config) end, + Pairs = fun() -> cmac_nist(Config, Type) end, [{block, Block}, {cmac, Pairs} | Config]; group_config(aes_ecb, Config) -> - Block = aes_ecb(), - [{block, Block} | Config]; + Block = fun() -> aes_ecb(Config) end, + [{block, Block} | Config]; group_config(aes_ige256, Config) -> Block = aes_ige256(), [{block, Block} | Config]; group_config(aes_cfb8, Config) -> - Block = aes_cfb8(), + Block = fun() -> aes_cfb8(Config) end, [{block, Block} | Config]; group_config(aes_cfb128, Config) -> - Block = aes_cfb128(), + Block = fun() -> aes_cfb128(Config) end, [{block, Block} | Config]; group_config(blowfish_cbc, Config) -> Block = blowfish_cbc(), @@ -1219,13 +1243,13 @@ group_config(aes_ctr, Config) -> Stream = aes_ctr(), [{stream, Stream} | Config]; group_config(aes_gcm, Config) -> - AEAD = aes_gcm(), + AEAD = fun() -> aes_gcm(Config) end, [{aead, AEAD} | Config]; group_config(chacha20_poly1305, Config) -> AEAD = chacha20_poly1305(), [{aead, AEAD} | Config]; group_config(aes_cbc, Config) -> - Block = aes_cbc(), + Block = aes_cbc(Config), [{block, Block} | Config]; group_config(_, Config) -> Config. @@ -1311,9 +1335,10 @@ rfc_4634_sha512_digests() -> long_msg() -> fun() -> lists:duplicate(1000000, $a) end. -%% Building huge terms (like long_msg/0) in init_per_group seems to cause -%% test_server crash with 'no_answer_from_tc_supervisor' sometimes on some -%% machines. Therefore lazy evaluation when test case has started. +%% Passing huge terms (like long_msg/0) through config causes excessive memory +%% consumption and long runtimes in the test server. This results in test_server +%% crash with 'no_answer_from_tc_supervisor' sometimes on some machines. +%% Therefore lazy evaluation when test case has started. lazy_eval(F) when is_function(F) -> F(); lazy_eval(Lst) when is_list(Lst) -> lists:map(fun lazy_eval/1, Lst); lazy_eval(Tpl) when is_tuple(Tpl) -> list_to_tuple(lists:map(fun lazy_eval/1, tuple_to_list(Tpl))); @@ -1601,209 +1626,30 @@ rc2_cbc() -> }]. %% AES CBC test vectors from http://csrc.nist.gov/publications/nistpubs/800-38a/sp800-38a.pdf -aes_cbc() -> - [ - %% F.2.1 CBC-AES128.Encrypt, F.2.2 CBC-AES128.Decrypt - {aes_cbc, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), %% Key - hexstr2bin("000102030405060708090a0b0c0d0e0f"), %% IV - hexstr2bin("6bc1bee22e409f96e93d7e117393172a" %% PlainText - "ae2d8a571e03ac9c9eb76fac45af8e51" - "30c81c46a35ce411e5fbc1191a0a52ef" - "f69f2445df4f9b17ad2b417be66c3710"), - hexstr2bin("7649abac8119b246cee98e9b12e9197d" %% CipherText - "5086cb9b507219ee95db113a917678b2" - "73bed6b8e3c1743b7116e69e22229516" - "3ff1caa1681fac09120eca307586e1a7")}, - %% F.2.3 CBC-AES192.Encrypt, F.2.4 CBC-AES192.Decrypt - {aes_cbc, - hexstr2bin("8e73b0f7da0e6452c810f32b809079e5" %% Key - "62f8ead2522c6b7b"), - hexstr2bin("000102030405060708090a0b0c0d0e0f"), %% IV - hexstr2bin("6bc1bee22e409f96e93d7e117393172a" %% PlainText - "ae2d8a571e03ac9c9eb76fac45af8e51" - "30c81c46a35ce411e5fbc1191a0a52ef" - "f69f2445df4f9b17ad2b417be66c3710"), - hexstr2bin("4f021db243bc633d7178183a9fa071e8" %% CipherText - "b4d9ada9ad7dedf4e5e738763f69145a" - "571b242012fb7ae07fa9baac3df102e0" - "08b0e27988598881d920a9e64f5615cd")}, - %% F.2.5 CBC-AES256.Encrypt, F.2.6 CBC-AES256.Decrypt - {aes_cbc, - hexstr2bin("603deb1015ca71be2b73aef0857d7781" %% Key - "1f352c073b6108d72d9810a30914dff4"), - hexstr2bin("000102030405060708090a0b0c0d0e0f"), %% IV - hexstr2bin("6bc1bee22e409f96e93d7e117393172a" %% PlainText - "ae2d8a571e03ac9c9eb76fac45af8e51" - "30c81c46a35ce411e5fbc1191a0a52ef" - "f69f2445df4f9b17ad2b417be66c3710"), - hexstr2bin("f58c4c04d6e5f1ba779eabfb5f7bfbd6" %% CipherText - "9cfc4e967edb808d679f777bc6702c7d" - "39f23369a9d9bacfa530e26304231461" - "b2eb05e2c39be9fcda6c19078c6a9d1b")} - ]. - -aes_cbc128() -> - [{aes_cbc128, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("000102030405060708090a0b0c0d0e0f"), - hexstr2bin("6bc1bee22e409f96e93d7e117393172a")}, - {aes_cbc128, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("7649ABAC8119B246CEE98E9B12E9197D"), - hexstr2bin("ae2d8a571e03ac9c9eb76fac45af8e51")}, - {aes_cbc128, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("5086CB9B507219EE95DB113A917678B2"), - hexstr2bin("30c81c46a35ce411e5fbc1191a0a52ef")}, - {aes_cbc128, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("73BED6B8E3C1743B7116E69E22229516"), - hexstr2bin("f69f2445df4f9b17ad2b417be66c3710")} - ]. - -aes_cbc256() -> - [{aes_cbc256, - hexstr2bin("603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4"), - hexstr2bin("000102030405060708090A0B0C0D0E0F"), - hexstr2bin("6bc1bee22e409f96e93d7e117393172a")}, - {aes_cbc256, - hexstr2bin("603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4"), - hexstr2bin("F58C4C04D6E5F1BA779EABFB5F7BFBD6"), - hexstr2bin("ae2d8a571e03ac9c9eb76fac45af8e51")}, - {aes_cbc256, - hexstr2bin("603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4"), - hexstr2bin("9CFC4E967EDB808D679F777BC6702C7D"), - hexstr2bin("30c81c46a35ce411e5fbc1191a0a52ef")}, - {aes_cbc256, - hexstr2bin("603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4"), - hexstr2bin("39F23369A9D9BACFA530E26304231461"), - hexstr2bin("f69f2445df4f9b17ad2b417be66c3710")} - ]. - -aes_ecb() -> - [ - {aes_ecb, - <<"YELLOW SUBMARINE">>, - <<"YELLOW SUBMARINE">>}, - {aes_ecb, - <<"0000000000000000">>, - <<"0000000000000000">>}, - {aes_ecb, - <<"FFFFFFFFFFFFFFFF">>, - <<"FFFFFFFFFFFFFFFF">>}, - {aes_ecb, - <<"3000000000000000">>, - <<"1000000000000001">>}, - {aes_ecb, - <<"1111111111111111">>, - <<"1111111111111111">>}, - {aes_ecb, - <<"0123456789ABCDEF">>, - <<"1111111111111111">>}, - {aes_ecb, - <<"0000000000000000">>, - <<"0000000000000000">>}, - {aes_ecb, - <<"FEDCBA9876543210">>, - <<"0123456789ABCDEF">>}, - {aes_ecb, - <<"7CA110454A1A6E57">>, - <<"01A1D6D039776742">>}, - {aes_ecb, - <<"0131D9619DC1376E">>, - <<"5CD54CA83DEF57DA">>}, - {aes_ecb, - <<"07A1133E4A0B2686">>, - <<"0248D43806F67172">>}, - {aes_ecb, - <<"3849674C2602319E">>, - <<"51454B582DDF440A">>}, - {aes_ecb, - <<"04B915BA43FEB5B6">>, - <<"42FD443059577FA2">>}, - {aes_ecb, - <<"0113B970FD34F2CE">>, - <<"059B5E0851CF143A">>}, - {aes_ecb, - <<"0170F175468FB5E6">>, - <<"0756D8E0774761D2">>}, - {aes_ecb, - <<"43297FAD38E373FE">>, - <<"762514B829BF486A">>}, - {aes_ecb, - <<"07A7137045DA2A16">>, - <<"3BDD119049372802">>}, - {aes_ecb, - <<"04689104C2FD3B2F">>, - <<"26955F6835AF609A">>}, - {aes_ecb, - <<"37D06BB516CB7546">>, - <<"164D5E404F275232">>}, - {aes_ecb, - <<"1F08260D1AC2465E">>, - <<"6B056E18759F5CCA">>}, - {aes_ecb, - <<"584023641ABA6176">>, - <<"004BD6EF09176062">>}, - {aes_ecb, - <<"025816164629B007">>, - <<"480D39006EE762F2">>}, - {aes_ecb, - <<"49793EBC79B3258F">>, - <<"437540C8698F3CFA">>}, - {aes_ecb, - <<"018310DC409B26D6">>, - <<"1D9D5C5018F728C2">>}, - {aes_ecb, - <<"1C587F1C13924FEF">>, - <<"305532286D6F295A">>}, - {aes_ecb, - <<"0101010101010101">>, - <<"0123456789ABCDEF">>}, - {aes_ecb, - <<"1F1F1F1F0E0E0E0E">>, - <<"0123456789ABCDEF">>}, - {aes_ecb, - <<"E0FEE0FEF1FEF1FE">>, - <<"0123456789ABCDEF">>}, - {aes_ecb, - <<"0000000000000000">>, - <<"FFFFFFFFFFFFFFFF">>}, - {aes_ecb, - <<"FFFFFFFFFFFFFFFF">>, - <<"0000000000000000">>}, - {aes_ecb, - <<"0123456789ABCDEF">>, - <<"0000000000000000">>}, - {aes_ecb, - <<"FEDCBA9876543210">>, - <<"FFFFFFFFFFFFFFFF">>}, - %% AES ECB test vectors from http://csrc.nist.gov/publications/nistpubs/800-38a/sp800-38a.pdf - %% F.1.1 ECB-AES128.Encrypt, F.1.2 ECB-AES128.Decrypt - {aes_ecb, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("6bc1bee22e409f96e93d7e117393172a" - "ae2d8a571e03ac9c9eb76fac45af8e51" - "30c81c46a35ce411e5fbc1191a0a52ef" - "f69f2445df4f9b17ad2b417be66c3710")}, - %% F.1.3 ECB-AES192.Encrypt, F.1.4 ECB-AES192.Decrypt - {aes_ecb, - hexstr2bin("8e73b0f7da0e6452c810f32b809079e5" - "62f8ead2522c6b7b"), - hexstr2bin("6bc1bee22e409f96e93d7e117393172a" - "ae2d8a571e03ac9c9eb76fac45af8e51" - "30c81c46a35ce411e5fbc1191a0a52ef" - "f69f2445df4f9b17ad2b417be66c3710")}, - %% F.1.5 ECB-AES256.Encrypt, F.1.6 ECB-AES256.Decrypt - {aes_ecb, - hexstr2bin("603deb1015ca71be2b73aef0857d7781" - "1f352c073b6108d72d9810a30914dff4"), - hexstr2bin("6bc1bee22e409f96e93d7e117393172a" - "ae2d8a571e03ac9c9eb76fac45af8e51" - "30c81c46a35ce411e5fbc1191a0a52ef" - "f69f2445df4f9b17ad2b417be66c3710")} - ]. +aes_cbc(Config) -> + read_rsp(Config, aes_cbc, + ["CBCVarTxt128.rsp", "CBCVarKey128.rsp", "CBCGFSbox128.rsp", "CBCKeySbox128.rsp", + "CBCVarTxt192.rsp", "CBCVarKey192.rsp", "CBCGFSbox192.rsp", "CBCKeySbox192.rsp", + "CBCVarTxt256.rsp", "CBCVarKey256.rsp", "CBCGFSbox256.rsp", "CBCKeySbox256.rsp", + "CBCMMT128.rsp", "CBCMMT192.rsp", "CBCMMT256.rsp" + ]). + +aes_cbc128(Config) -> + read_rsp(Config, aes_cbc128, + ["CBCVarTxt128.rsp", "CBCVarKey128.rsp", "CBCGFSbox128.rsp", "CBCKeySbox128.rsp", + "CBCMMT128.rsp"]). + +aes_cbc256(Config) -> + read_rsp(Config, aes_cbc256, + ["CBCVarTxt256.rsp", "CBCVarKey256.rsp", "CBCGFSbox256.rsp", "CBCKeySbox256.rsp", + "CBCMMT256.rsp"]). + +aes_ecb(Config) -> + read_rsp(Config, aes_ecb, + ["ECBVarTxt128.rsp", "ECBVarKey128.rsp", "ECBGFSbox128.rsp", "ECBKeySbox128.rsp", + "ECBVarTxt192.rsp", "ECBVarKey192.rsp", "ECBGFSbox192.rsp", "ECBKeySbox192.rsp", + "ECBVarTxt256.rsp", "ECBVarKey256.rsp", "ECBGFSbox256.rsp", "ECBKeySbox256.rsp", + "ECBMMT128.rsp", "ECBMMT192.rsp", "ECBMMT256.rsp"]). aes_ige256() -> [{aes_ige256, @@ -1824,107 +1670,19 @@ aes_ige256() -> hexstr2bin("f69f2445df4f9b17ad2b417be66c3710")} ]. -aes_cfb8() -> - [{aes_cfb8, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("000102030405060708090a0b0c0d0e0f"), - hexstr2bin("6bc1bee22e409f96e93d7e117393172a")}, - {aes_cfb8, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("3B3FD92EB72DAD20333449F8E83CFB4A"), - hexstr2bin("ae2d8a571e03ac9c9eb76fac45af8e51")}, - {aes_cfb8, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("C8A64537A0B3A93FCDE3CDAD9F1CE58B"), - hexstr2bin("30c81c46a35ce411e5fbc1191a0a52ef")}, - {aes_cfb8, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("26751F67A3CBB140B1808CF187A4F4DF"), - hexstr2bin("f69f2445df4f9b17ad2b417be66c3710")}, - {aes_cfb8, - hexstr2bin("8e73b0f7da0e6452c810f32b809079e562f8ead2522c6b7b"), - hexstr2bin("000102030405060708090a0b0c0d0e0f"), - hexstr2bin("6bc1bee22e409f96e93d7e117393172a")}, - {aes_cfb8, - hexstr2bin("8e73b0f7da0e6452c810f32b809079e562f8ead2522c6b7b"), - hexstr2bin("cdc80d6fddf18cab34c25909c99a4174"), - hexstr2bin("ae2d8a571e03ac9c9eb76fac45af8e51")}, - {aes_cfb8, - hexstr2bin("8e73b0f7da0e6452c810f32b809079e562f8ead2522c6b7b"), - hexstr2bin("67ce7f7f81173621961a2b70171d3d7a"), - hexstr2bin("30c81c46a35ce411e5fbc1191a0a52ef")}, - {aes_cfb8, - hexstr2bin("8e73b0f7da0e6452c810f32b809079e562f8ead2522c6b7b"), - hexstr2bin("2e1e8a1dd59b88b1c8e60fed1efac4c9"), - hexstr2bin("f69f2445df4f9b17ad2b417be66c3710")}, - {aes_cfb8, - hexstr2bin("603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4"), - hexstr2bin("000102030405060708090a0b0c0d0e0f"), - hexstr2bin("6bc1bee22e409f96e93d7e117393172a")}, - {aes_cfb8, - hexstr2bin("603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4"), - hexstr2bin("dc7e84bfda79164b7ecd8486985d3860"), - hexstr2bin("ae2d8a571e03ac9c9eb76fac45af8e51")}, - {aes_cfb8, - hexstr2bin("603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4"), - hexstr2bin("39ffed143b28b1c832113c6331e5407b"), - hexstr2bin("30c81c46a35ce411e5fbc1191a0a52ef")}, - {aes_cfb8, - hexstr2bin("603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4"), - hexstr2bin("df10132415e54b92a13ed0a8267ae2f9"), - hexstr2bin("f69f2445df4f9b17ad2b417be66c3710")} - ]. +aes_cfb8(Config) -> + read_rsp(Config, aes_cfb8, + ["CFB8VarTxt128.rsp", "CFB8VarKey128.rsp", "CFB8GFSbox128.rsp", "CFB8KeySbox128.rsp", + "CFB8VarTxt192.rsp", "CFB8VarKey192.rsp", "CFB8GFSbox192.rsp", "CFB8KeySbox192.rsp", + "CFB8VarTxt256.rsp", "CFB8VarKey256.rsp", "CFB8GFSbox256.rsp", "CFB8KeySbox256.rsp", + "CFB8MMT128.rsp", "CFB8MMT192.rsp", "CFB8MMT256.rsp"]). -aes_cfb128() -> - [{aes_cfb128, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("000102030405060708090a0b0c0d0e0f"), - hexstr2bin("6bc1bee22e409f96e93d7e117393172a")}, - {aes_cfb128, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("3B3FD92EB72DAD20333449F8E83CFB4A"), - hexstr2bin("ae2d8a571e03ac9c9eb76fac45af8e51")}, - {aes_cfb128, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("C8A64537A0B3A93FCDE3CDAD9F1CE58B"), - hexstr2bin("30c81c46a35ce411e5fbc1191a0a52ef")}, - {aes_cfb128, - hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - hexstr2bin("26751F67A3CBB140B1808CF187A4F4DF"), - hexstr2bin("f69f2445df4f9b17ad2b417be66c3710")}, - {aes_cfb128, - hexstr2bin("8e73b0f7da0e6452c810f32b809079e562f8ead2522c6b7b"), - hexstr2bin("000102030405060708090a0b0c0d0e0f"), - hexstr2bin("6bc1bee22e409f96e93d7e117393172a")}, - {aes_cfb128, - hexstr2bin("8e73b0f7da0e6452c810f32b809079e562f8ead2522c6b7b"), - hexstr2bin("cdc80d6fddf18cab34c25909c99a4174"), - hexstr2bin("ae2d8a571e03ac9c9eb76fac45af8e51")}, - {aes_cfb128, - hexstr2bin("8e73b0f7da0e6452c810f32b809079e562f8ead2522c6b7b"), - hexstr2bin("67ce7f7f81173621961a2b70171d3d7a"), - hexstr2bin("30c81c46a35ce411e5fbc1191a0a52ef")}, - {aes_cfb128, - hexstr2bin("8e73b0f7da0e6452c810f32b809079e562f8ead2522c6b7b"), - hexstr2bin("2e1e8a1dd59b88b1c8e60fed1efac4c9"), - hexstr2bin("f69f2445df4f9b17ad2b417be66c3710")}, - {aes_cfb128, - hexstr2bin("603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4"), - hexstr2bin("000102030405060708090a0b0c0d0e0f"), - hexstr2bin("6bc1bee22e409f96e93d7e117393172a")}, - {aes_cfb128, - hexstr2bin("603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4"), - hexstr2bin("dc7e84bfda79164b7ecd8486985d3860"), - hexstr2bin("ae2d8a571e03ac9c9eb76fac45af8e51")}, - {aes_cfb128, - hexstr2bin("603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4"), - hexstr2bin("39ffed143b28b1c832113c6331e5407b"), - hexstr2bin("30c81c46a35ce411e5fbc1191a0a52ef")}, - {aes_cfb128, - hexstr2bin("603deb1015ca71be2b73aef0857d77811f352c073b6108d72d9810a30914dff4"), - hexstr2bin("df10132415e54b92a13ed0a8267ae2f9"), - hexstr2bin("f69f2445df4f9b17ad2b417be66c3710")} - ]. +aes_cfb128(Config) -> + read_rsp(Config, aes_cfb128, + ["CFB128VarTxt128.rsp", "CFB128VarKey128.rsp", "CFB128GFSbox128.rsp", "CFB128KeySbox128.rsp", + "CFB128VarTxt192.rsp", "CFB128VarKey192.rsp", "CFB128GFSbox192.rsp", "CFB128KeySbox192.rsp", + "CFB128VarTxt256.rsp", "CFB128VarKey256.rsp", "CFB128GFSbox256.rsp", "CFB128KeySbox256.rsp", + "CFB128MMT128.rsp", "CFB128MMT192.rsp", "CFB128MMT256.rsp"]). blowfish_cbc() -> [{blowfish_cbc, @@ -2098,284 +1856,14 @@ aes_ctr() -> ]. -%% AES GCM test vectors from http://csrc.nist.gov/groups/ST/toolkit/BCM/documents/proposedmodes/gcm/gcm-spec.pdf -aes_gcm() -> - [ - %% Test Case 1 - {aes_gcm, hexstr2bin("00000000000000000000000000000000"), %% Key - hexstr2bin(""), %% PlainText - hexstr2bin("000000000000000000000000"), %% IV - hexstr2bin(""), %% AAD - hexstr2bin(""), %% CipherText - hexstr2bin("58e2fccefa7e3061367f1d57a4e7455a")}, %% CipherTag - - %% Test Case 2 - {aes_gcm, hexstr2bin("00000000000000000000000000000000"), %% Key - hexstr2bin("00000000000000000000000000000000"), %% PlainText - hexstr2bin("000000000000000000000000"), %% IV - hexstr2bin(""), %% AAD - hexstr2bin("0388dace60b6a392f328c2b971b2fe78"), %% CipherText - hexstr2bin("ab6e47d42cec13bdf53a67b21257bddf")}, %% CipherTag - - %% Test Case 3 - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308"), %% Key - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b391aafd255"), - hexstr2bin("cafebabefacedbaddecaf888"), %% IV - hexstr2bin(""), %% AAD - hexstr2bin("42831ec2217774244b7221b784d0d49c" %% CipherText - "e3aa212f2c02a4e035c17e2329aca12e" - "21d514b25466931c7d8f6a5aac84aa05" - "1ba30b396a0aac973d58e091473f5985"), - hexstr2bin("4d5c2af327cd64a62cf35abd2ba6fab4")}, %% CipherTag - - %% Test Case 4 - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308"), %% Key - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b39"), - hexstr2bin("cafebabefacedbaddecaf888"), %% IV - hexstr2bin("feedfacedeadbeeffeedfacedeadbeef" %% AAD - "abaddad2"), - hexstr2bin("42831ec2217774244b7221b784d0d49c" %% CipherText - "e3aa212f2c02a4e035c17e2329aca12e" - "21d514b25466931c7d8f6a5aac84aa05" - "1ba30b396a0aac973d58e091"), - hexstr2bin("5bc94fbc3221a5db94fae95ae7121a47")}, %% CipherTag - - %% Test Case 5 - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308"), %% Key - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b39"), - hexstr2bin("cafebabefacedbad"), %% IV - hexstr2bin("feedfacedeadbeeffeedfacedeadbeef" %% AAD - "abaddad2"), - hexstr2bin("61353b4c2806934a777ff51fa22a4755" %% CipherText - "699b2a714fcdc6f83766e5f97b6c7423" - "73806900e49f24b22b097544d4896b42" - "4989b5e1ebac0f07c23f4598"), - hexstr2bin("3612d2e79e3b0785561be14aaca2fccb")}, %% CipherTag - - %% Test Case 6" - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308"), %% Key - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b39"), - hexstr2bin("9313225df88406e555909c5aff5269aa" %% IV - "6a7a9538534f7da1e4c303d2a318a728" - "c3c0c95156809539fcf0e2429a6b5254" - "16aedbf5a0de6a57a637b39b"), - hexstr2bin("feedfacedeadbeeffeedfacedeadbeef" %% AAD - "abaddad2"), - hexstr2bin("8ce24998625615b603a033aca13fb894" %% CipherText - "be9112a5c3a211a8ba262a3cca7e2ca7" - "01e4a9a4fba43c90ccdcb281d48c7c6f" - "d62875d2aca417034c34aee5"), - hexstr2bin("619cc5aefffe0bfa462af43c1699d050")}, %% CipherTag - - %% Test Case 7 - {aes_gcm, hexstr2bin("00000000000000000000000000000000" %% Key - "0000000000000000"), - hexstr2bin(""), %% PlainText - hexstr2bin("000000000000000000000000"), %% IV - hexstr2bin(""), %% AAD - hexstr2bin(""), %% CipherText - hexstr2bin("cd33b28ac773f74ba00ed1f312572435")}, %% CipherTag - - %% Test Case 8 - {aes_gcm, hexstr2bin("00000000000000000000000000000000" %% Key - "0000000000000000"), - hexstr2bin("00000000000000000000000000000000"), %% PlainText - hexstr2bin("000000000000000000000000"), %% IV - hexstr2bin(""), %% AAD - hexstr2bin("98e7247c07f0fe411c267e4384b0f600"), %% CipherText - hexstr2bin("2ff58d80033927ab8ef4d4587514f0fb")}, %% CipherTag - - %% Test Case 9 - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308" %% Key - "feffe9928665731c"), - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b391aafd255"), - hexstr2bin("cafebabefacedbaddecaf888"), %% IV - hexstr2bin(""), %% ADD - hexstr2bin("3980ca0b3c00e841eb06fac4872a2757" %% CipherText - "859e1ceaa6efd984628593b40ca1e19c" - "7d773d00c144c525ac619d18c84a3f47" - "18e2448b2fe324d9ccda2710acade256"), - hexstr2bin("9924a7c8587336bfb118024db8674a14")}, %% CipherTag - - %% Test Case 10 - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308" %% Key - "feffe9928665731c"), - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b39"), - hexstr2bin("cafebabefacedbaddecaf888"), %% IV - hexstr2bin("feedfacedeadbeeffeedfacedeadbeef" %% AAD - "abaddad2"), - hexstr2bin("3980ca0b3c00e841eb06fac4872a2757" %% CipherText - "859e1ceaa6efd984628593b40ca1e19c" - "7d773d00c144c525ac619d18c84a3f47" - "18e2448b2fe324d9ccda2710"), - hexstr2bin("2519498e80f1478f37ba55bd6d27618c")}, %% CipherTag - - %% Test Case 11 - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308" %% Key - "feffe9928665731c"), - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b39"), - hexstr2bin("cafebabefacedbad"), %% IV - hexstr2bin("feedfacedeadbeeffeedfacedeadbeef" %% AAD - "abaddad2"), - hexstr2bin("0f10f599ae14a154ed24b36e25324db8" %% CipherText - "c566632ef2bbb34f8347280fc4507057" - "fddc29df9a471f75c66541d4d4dad1c9" - "e93a19a58e8b473fa0f062f7"), - hexstr2bin("65dcc57fcf623a24094fcca40d3533f8")}, %% CipherTag - - %% Test Case 12 - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308" %% Key - "feffe9928665731c"), - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b39"), - hexstr2bin("9313225df88406e555909c5aff5269aa" %% IV - "6a7a9538534f7da1e4c303d2a318a728" - "c3c0c95156809539fcf0e2429a6b5254" - "16aedbf5a0de6a57a637b39b"), - hexstr2bin("feedfacedeadbeeffeedfacedeadbeef" %% AAD - "abaddad2"), - hexstr2bin("d27e88681ce3243c4830165a8fdcf9ff" %% CipherText - "1de9a1d8e6b447ef6ef7b79828666e45" - "81e79012af34ddd9e2f037589b292db3" - "e67c036745fa22e7e9b7373b"), - hexstr2bin("dcf566ff291c25bbb8568fc3d376a6d9")}, %% CipherTag - - %% Test Case 13 - {aes_gcm, hexstr2bin("00000000000000000000000000000000" %% Key - "00000000000000000000000000000000"), - hexstr2bin(""), %% PlainText - hexstr2bin("000000000000000000000000"), %% IV - hexstr2bin(""), %% AAD - hexstr2bin(""), %% CipherText - hexstr2bin("530f8afbc74536b9a963b4f1c4cb738b")}, %% CipherTag - - %% Test Case 14 - {aes_gcm, hexstr2bin("00000000000000000000000000000000" %% Key - "00000000000000000000000000000000"), - hexstr2bin("00000000000000000000000000000000"), %% PlainText - hexstr2bin("000000000000000000000000"), %% IV - hexstr2bin(""), %% AAD - hexstr2bin("cea7403d4d606b6e074ec5d3baf39d18"), %% CipherText - hexstr2bin("d0d1c8a799996bf0265b98b5d48ab919")}, %% CipherTag - - %% Test Case 15 - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308" %% Key - "feffe9928665731c6d6a8f9467308308"), - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b391aafd255"), - hexstr2bin("cafebabefacedbaddecaf888"), %% IV - hexstr2bin(""), %% AAD - hexstr2bin("522dc1f099567d07f47f37a32a84427d" %% CipherText - "643a8cdcbfe5c0c97598a2bd2555d1aa" - "8cb08e48590dbb3da7b08b1056828838" - "c5f61e6393ba7a0abcc9f662898015ad"), - hexstr2bin("b094dac5d93471bdec1a502270e3cc6c")}, %% CipherTag - - %% Test Case 16 - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308" %% Key - "feffe9928665731c6d6a8f9467308308"), - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b39"), - hexstr2bin("cafebabefacedbaddecaf888"), %% IV - hexstr2bin("feedfacedeadbeeffeedfacedeadbeef" %% AAD - "abaddad2"), - hexstr2bin("522dc1f099567d07f47f37a32a84427d" %% CipherText - "643a8cdcbfe5c0c97598a2bd2555d1aa" - "8cb08e48590dbb3da7b08b1056828838" - "c5f61e6393ba7a0abcc9f662"), - hexstr2bin("76fc6ece0f4e1768cddf8853bb2d551b")}, %% CipherTag - - %% Test Case 17 - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308" %% Key - "feffe9928665731c6d6a8f9467308308"), - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b39"), - hexstr2bin("cafebabefacedbad"), %% IV - hexstr2bin("feedfacedeadbeeffeedfacedeadbeef" %% AAD - "abaddad2"), - hexstr2bin("c3762df1ca787d32ae47c13bf19844cb" %% CipherText - "af1ae14d0b976afac52ff7d79bba9de0" - "feb582d33934a4f0954cc2363bc73f78" - "62ac430e64abe499f47c9b1f"), - hexstr2bin("3a337dbf46a792c45e454913fe2ea8f2")}, %% CipherTag - - %% Test Case 18 - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308" %% Key - "feffe9928665731c6d6a8f9467308308"), - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b39"), - hexstr2bin("9313225df88406e555909c5aff5269aa" %% IV - "6a7a9538534f7da1e4c303d2a318a728" - "c3c0c95156809539fcf0e2429a6b5254" - "16aedbf5a0de6a57a637b39b"), - hexstr2bin("feedfacedeadbeeffeedfacedeadbeef" %% AAD - "abaddad2"), - hexstr2bin("5a8def2f0c9e53f1f75d7853659e2a20" %% CipherText - "eeb2b22aafde6419a058ab4f6f746bf4" - "0fc0c3b780f244452da3ebf1c5d82cde" - "a2418997200ef82e44ae7e3f"), - hexstr2bin("a44a8266ee1c8eb0c8b5d4cf5ae9f19a")}, %% CipherTag - - %% Test Case 0 for TagLength = 1 - {aes_gcm, hexstr2bin("00000000000000000000000000000000"), %% Key - hexstr2bin(""), %% PlainText - hexstr2bin("000000000000000000000000"), %% IV - hexstr2bin(""), %% AAD - hexstr2bin(""), %% CipherText - hexstr2bin("58"), %% CipherTag - 1}, %% TagLength - - %% Test Case 18 for TagLength = 1 - {aes_gcm, hexstr2bin("feffe9928665731c6d6a8f9467308308" %% Key - "feffe9928665731c6d6a8f9467308308"), - hexstr2bin("d9313225f88406e5a55909c5aff5269a" %% PlainText - "86a7a9531534f7da2e4c303d8a318a72" - "1c3c0c95956809532fcf0e2449a6b525" - "b16aedf5aa0de657ba637b39"), - hexstr2bin("9313225df88406e555909c5aff5269aa" %% IV - "6a7a9538534f7da1e4c303d2a318a728" - "c3c0c95156809539fcf0e2429a6b5254" - "16aedbf5a0de6a57a637b39b"), - hexstr2bin("feedfacedeadbeeffeedfacedeadbeef" %% AAD - "abaddad2"), - hexstr2bin("5a8def2f0c9e53f1f75d7853659e2a20" %% CipherText - "eeb2b22aafde6419a058ab4f6f746bf4" - "0fc0c3b780f244452da3ebf1c5d82cde" - "a2418997200ef82e44ae7e3f"), - hexstr2bin("a4"), %% CipherTag - 1} %% TagLength - ]. +aes_gcm(Config) -> + read_rsp(Config, aes_gcm, + ["gcmDecrypt128.rsp", + "gcmDecrypt192.rsp", + "gcmDecrypt256.rsp", + "gcmEncryptExtIV128.rsp", + "gcmEncryptExtIV192.rsp", + "gcmEncryptExtIV256.rsp"]). %% https://tools.ietf.org/html/rfc7539#appendix-A.5 chacha20_poly1305() -> @@ -2750,49 +2238,13 @@ ecc() -> end, TestCases). -%% Test data from Appendix D of NIST Special Publication 800-38B -%% http://csrc.nist.gov/publications/nistpubs/800-38B/Updated_CMAC_Examples.pdf -%% The same AES128 test data are also in the RFC 4493 -%% https://tools.ietf.org/html/rfc4493 -cmac_nist(aes_cbc128 = Type) -> - Key = hexstr2bin("2b7e151628aed2a6abf7158809cf4f3c"), - [{Type, Key, <<"">>, - hexstr2bin("bb1d6929e95937287fa37d129b756746")}, - {Type, Key, hexstr2bin("6bc1bee22e409f96e93d7e117393172a"), - hexstr2bin("070a16b46b4d4144f79bdd9dd04a287c")}, - {Type, Key, hexstr2bin("6bc1bee22e409f96e93d7e117393172a" - "ae2d8a571e03ac9c9eb76fac45af8e51" - "30c81c46a35ce411"), - hexstr2bin("dfa66747de9ae63030ca32611497c827")}, - {Type, Key, hexstr2bin("6bc1bee22e409f96e93d7e117393172a" - "ae2d8a571e03ac9c9eb76fac45af8e51" - "30c81c46a35ce411e5fbc1191a0a52ef" - "f69f2445df4f9b17ad2b417be66c3710"), - hexstr2bin("51f0bebf7e3b9d92fc49741779363cfe")}, - % truncation - {Type, Key, <<"">>, 4, - hexstr2bin("bb1d6929")}]; - -cmac_nist(aes_cbc256 = Type) -> - Key = hexstr2bin("603deb1015ca71be2b73aef0857d7781" - "1f352c073b6108d72d9810a30914dff4"), - [{Type, Key, <<"">>, - hexstr2bin("028962f61b7bf89efc6b551f4667d983")}, - {Type, Key, hexstr2bin("6bc1bee22e409f96e93d7e117393172a"), - hexstr2bin("28a7023f452e8f82bd4bf28d8c37c35c")}, - {Type, Key, hexstr2bin("6bc1bee22e409f96e93d7e117393172a" - "ae2d8a571e03ac9c9eb76fac45af8e51" - "30c81c46a35ce411"), - hexstr2bin("aaf3d8f1de5640c232f5b169b9c911e6")}, - {Type, Key, hexstr2bin("6bc1bee22e409f96e93d7e117393172a" - "ae2d8a571e03ac9c9eb76fac45af8e51" - "30c81c46a35ce411e5fbc1191a0a52ef" - "f69f2445df4f9b17ad2b417be66c3710"), - hexstr2bin("e1992190549f6ed5696a2c056c315410")}, - % truncation - {Type, Key, <<"">>, 4, - hexstr2bin("028962f6")}]. +cmac_nist(Config, aes_cbc128 = Type) -> + read_rsp(Config, Type, + ["CMACGenAES128.rsp", "CMACVerAES128.rsp"]); +cmac_nist(Config, aes_cbc256 = Type) -> + read_rsp(Config, Type, + ["CMACGenAES256.rsp", "CMACVerAES256.rsp"]). no_padding() -> Public = [_, Mod] = rsa_public_stronger(), @@ -2813,3 +2265,123 @@ int_to_bin_neg(-1, Ds=[MSB|_]) when MSB >= 16#80 -> list_to_binary(Ds); int_to_bin_neg(X,Ds) -> int_to_bin_neg(X bsr 8, [(X band 255)|Ds]). + +datadir(Config) -> + proplists:get_value(data_dir, Config). + +-define(KiB, 1024). +-define(MiB, (1024 * 1024)). +-define(GiB, (1024 * 1024 * 1024)). + +fmt_words(Words) -> + BSize = Words * erlang:system_info(wordsize), + if BSize < ?KiB -> + integer_to_list(BSize); + BSize < ?MiB -> + io_lib:format("~8.2fKiB (~8w)", [BSize / ?KiB, BSize]); + BSize < ?GiB -> + io_lib:format("~8.2fMiB (~8w)", [BSize / ?MiB, BSize]); + true -> + io_lib:format("~8.2fGiB (~8w)", [BSize / ?GiB, BSize]) + end. + +log_rsp_size(Label, Term) -> + S = erts_debug:size(Term), + ct:pal("~s: ~w test(s), Memory used: ~s", + [Label, length(Term), fmt_words(S)]). + +read_rsp(Config, Type, Files) -> + Tests = + lists:foldl( + fun(FileName, Acc) -> + read_rsp_file(filename:join(datadir(Config), FileName), + Type, Acc) + end, [], Files), + log_rsp_size(Type, Tests), + Tests. + +read_rsp_file(FileName, Type, Acc) -> + {ok, Raw} = file:read_file(FileName), + Split = binary:split(Raw, [<<"\r">>, <<"\n">>], [global, trim_all]), + parse_rsp(Type, Split, Acc). + +parse_rsp(_Type, [], Acc) -> + Acc; +parse_rsp(_Type, [<<"DECRYPT">>|_], Acc) -> + Acc; +%% AES format +parse_rsp(Type, [<<"COUNT = ", _/binary>>, + <<"KEY = ", Key/binary>>, + <<"IV = ", IV/binary>>, + <<"PLAINTEXT = ", PlainText/binary>>, + <<"CIPHERTEXT = ", CipherText/binary>>|Next], Acc) -> + parse_rsp(Type, Next, [{Type, hexstr2bin(Key), hexstr2bin(IV), + hexstr2bin(PlainText), hexstr2bin(CipherText)}|Acc]); +%% CMAC format +parse_rsp(Type, [<<"Count = ", _/binary>>, + <<"Klen = ", _/binary>>, + <<"Mlen = ", Mlen/binary>>, + <<"Tlen = ", Tlen/binary>>, + <<"Key = ", Key/binary>>, + <<"Msg = ", Msg/binary>>, + <<"Mac = ", MAC/binary>>|Rest], Acc) -> + case Rest of + [<<"Result = P">>|Next] -> + parse_rsp_cmac(Type, Key, Msg, Mlen, Tlen, MAC, Next, Acc); + [<<"Result = ", _/binary>>|Next] -> + parse_rsp(Type, Next, Acc); + _ -> + parse_rsp_cmac(Type, Key, Msg, Mlen, Tlen, MAC, Rest, Acc) + end; +%% GCM format decode format +parse_rsp(Type, [<<"Count = ", _/binary>>, + <<"Key = ", Key/binary>>, + <<"IV = ", IV/binary>>, + <<"CT = ", CipherText/binary>>, + <<"AAD = ", AAD/binary>>, + <<"Tag = ", CipherTag0/binary>>, + <<"PT = ", PlainText/binary>>|Next], Acc) -> + CipherTag = hexstr2bin(CipherTag0), + TestCase = {Type, + hexstr2bin(Key), + hexstr2bin(PlainText), + hexstr2bin(IV), + hexstr2bin(AAD), + hexstr2bin(CipherText), + CipherTag, + size(CipherTag)}, + parse_rsp(Type, Next, [TestCase|Acc]); +%% GCM format encode format +parse_rsp(Type, [<<"Count = ", _/binary>>, + <<"Key = ", Key/binary>>, + <<"IV = ", IV/binary>>, + <<"PT = ", PlainText/binary>>, + <<"AAD = ", AAD/binary>>, + <<"CT = ", CipherText/binary>>, + <<"Tag = ", CipherTag0/binary>>|Next], Acc) -> + CipherTag = hexstr2bin(CipherTag0), + TestCase = {Type, + hexstr2bin(Key), + hexstr2bin(PlainText), + hexstr2bin(IV), + hexstr2bin(AAD), + hexstr2bin(CipherText), + CipherTag, + size(CipherTag)}, + parse_rsp(Type, Next, [TestCase|Acc]); + +parse_rsp(Type, [_|Next], Acc) -> + parse_rsp(Type, Next, Acc). + +parse_rsp_cmac(Type, Key0, Msg0, Mlen0, Tlen, MAC0, Next, Acc) -> + Key = hexstr2bin(Key0), + Mlen = binary_to_integer(Mlen0), + <<Msg:Mlen/bytes, _/binary>> = hexstr2bin(Msg0), + MAC = hexstr2bin(MAC0), + + case binary_to_integer(Tlen) of + 0 -> + parse_rsp(Type, Next, [{Type, Key, Msg, MAC}|Acc]); + I -> + parse_rsp(Type, Next, [{Type, Key, Msg, I, MAC}|Acc]) + end. diff --git a/lib/crypto/test/crypto_SUITE_data/KAT_AES.zip b/lib/crypto/test/crypto_SUITE_data/KAT_AES.zip Binary files differnew file mode 100644 index 0000000000..128a74c52e --- /dev/null +++ b/lib/crypto/test/crypto_SUITE_data/KAT_AES.zip diff --git a/lib/crypto/test/crypto_SUITE_data/aesmmt.zip b/lib/crypto/test/crypto_SUITE_data/aesmmt.zip Binary files differnew file mode 100644 index 0000000000..5024de1d06 --- /dev/null +++ b/lib/crypto/test/crypto_SUITE_data/aesmmt.zip diff --git a/lib/crypto/test/crypto_SUITE_data/cmactestvectors.zip b/lib/crypto/test/crypto_SUITE_data/cmactestvectors.zip Binary files differnew file mode 100644 index 0000000000..0d52444e57 --- /dev/null +++ b/lib/crypto/test/crypto_SUITE_data/cmactestvectors.zip diff --git a/lib/crypto/test/crypto_SUITE_data/gcmtestvectors.zip b/lib/crypto/test/crypto_SUITE_data/gcmtestvectors.zip Binary files differnew file mode 100644 index 0000000000..81eaa6c2f0 --- /dev/null +++ b/lib/crypto/test/crypto_SUITE_data/gcmtestvectors.zip diff --git a/lib/erl_docgen/priv/xsl/db_html.xsl b/lib/erl_docgen/priv/xsl/db_html.xsl index 8a1a70448e..d863c056e9 100644 --- a/lib/erl_docgen/priv/xsl/db_html.xsl +++ b/lib/erl_docgen/priv/xsl/db_html.xsl @@ -95,7 +95,7 @@ <xsl:value-of select="$elem/@name"/> </xsl:when> <xsl:otherwise> - <xsl:value-of select="$elem"/> + <xsl:value-of select="substring-before($elem, '(')"/> </xsl:otherwise> </xsl:choose> </xsl:otherwise> diff --git a/lib/hipe/cerl/erl_types.erl b/lib/hipe/cerl/erl_types.erl index ea8cc1677d..4cfa80f153 100644 --- a/lib/hipe/cerl/erl_types.erl +++ b/lib/hipe/cerl/erl_types.erl @@ -5171,9 +5171,9 @@ cache_put(Key, Type, DeltaL, #cache{types = Types} = Cache) -> NewTypes = maps:put(Key, {Type, DeltaL}, Types), Cache#cache{types = NewTypes}. --spec t_var_names([erl_type()]) -> [atom()]. +-spec t_var_names([parse_form()]) -> [atom()]. -t_var_names([{var, _, Name}|L]) when L =/= '_' -> +t_var_names([{var, _, Name}|L]) when Name =/= '_' -> [Name|t_var_names(L)]; t_var_names([]) -> []. diff --git a/lib/inets/src/http_client/httpc_handler.erl b/lib/inets/src/http_client/httpc_handler.erl index c99200777b..89c17a8679 100644 --- a/lib/inets/src/http_client/httpc_handler.erl +++ b/lib/inets/src/http_client/httpc_handler.erl @@ -1224,7 +1224,7 @@ close_socket(#session{socket = Socket, socket_type = SocketType}) -> http_transport:close(SocketType, Socket). activate_request_timeout( - #state{request = #request{timer = undefined} = Request} = State) -> + #state{request = #request{timer = OldRef} = Request} = State) -> Timeout = (Request#request.settings)#http_options.timeout, case Timeout of infinity -> @@ -1232,17 +1232,21 @@ activate_request_timeout( _ -> ReqId = Request#request.id, Msg = {timeout, ReqId}, + case OldRef of + undefined -> + ok; + _ -> + %% Timer is already running! This is the case for a redirect or retry + %% We need to restart the timer because the handler pid has changed + cancel_timer(OldRef, Msg) + end, Ref = erlang:send_after(Timeout, self(), Msg), Request2 = Request#request{timer = Ref}, ReqTimers = [{Request#request.id, Ref} | (State#state.timers)#timers.request_timers], Timers = #timers{request_timers = ReqTimers}, State#state{request = Request2, timers = Timers} - end; - -%% Timer is already running! This is the case for a redirect or retry -activate_request_timeout(State) -> - State. + end. activate_queue_timeout(infinity, State) -> State; diff --git a/lib/inets/test/httpc_SUITE.erl b/lib/inets/test/httpc_SUITE.erl index fc7f01245b..e6dcd2285f 100644 --- a/lib/inets/test/httpc_SUITE.erl +++ b/lib/inets/test/httpc_SUITE.erl @@ -108,6 +108,7 @@ only_simulated() -> tolerate_missing_CR, userinfo, bad_response, + timeout_redirect, internal_server_error, invalid_http, invalid_chunk_size, @@ -785,6 +786,14 @@ bad_response(Config) when is_list(Config) -> ct:print("Wrong Statusline: ~p~n", [Reason]). %%------------------------------------------------------------------------- +timeout_redirect() -> + [{doc, "Test that timeout works for redirects, check ERL-420."}]. +timeout_redirect(Config) when is_list(Config) -> + URL = url(group_name(Config), "/redirect_to_missing_crlf.html", Config), + {error, timeout} = httpc:request(get, {URL, []}, [{timeout, 400}], []). + +%%------------------------------------------------------------------------- + internal_server_error(doc) -> ["Test 50X codes"]; internal_server_error(Config) when is_list(Config) -> @@ -1915,6 +1924,16 @@ handle_uri(_,"/missing_crlf.html",_,_,_,_) -> "Content-Length:32\r\n" ++ "<HTML><BODY>foobar</BODY></HTML>"; +handle_uri(_,"/redirect_to_missing_crlf.html",Port,_,Socket,_) -> + NewUri = url_start(Socket) ++ + integer_to_list(Port) ++ "/missing_crlf.html", + Body = "<HTML><BODY><a href=" ++ NewUri ++ + ">New place</a></BODY></HTML>", + "HTTP/1.1 303 See Other \r\n" ++ + "Location:" ++ NewUri ++ "\r\n" ++ + "Content-Length:" ++ integer_to_list(length(Body)) + ++ "\r\n\r\n" ++ Body; + handle_uri(_,"/wrong_statusline.html",_,_,_,_) -> "ok 200 HTTP/1.1\r\n\r\n" ++ "Content-Length:32\r\n\r\n" ++ diff --git a/lib/kernel/doc/src/error_logger.xml b/lib/kernel/doc/src/error_logger.xml index 814e8eac46..27db00819f 100644 --- a/lib/kernel/doc/src/error_logger.xml +++ b/lib/kernel/doc/src/error_logger.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2016</year> + <year>1996</year><year>2017</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -163,6 +163,19 @@ ok</pre> </desc> </func> <func> + <name name="get_format_depth" arity="0"/> + <fsummary>Get the value of the Kernel application variable + <c>error_logger_format_depth</c>.</fsummary> + <desc> + <p>Returns <c>max(10, Depth)</c>, where <c>Depth</c> is the + value of + <seealso marker="kernel:kernel_app#error_logger_format_depth"> + error_logger_format_depth</seealso> + in the Kernel application, if Depth is an integer. Otherwise, + <c>unlimited</c> is returned.</p> + </desc> + </func> + <func> <name name="info_msg" arity="1"/> <name name="info_msg" arity="2"/> <fsummary>Send a standard information event to the error logger.</fsummary> diff --git a/lib/kernel/doc/src/inet.xml b/lib/kernel/doc/src/inet.xml index b7c904ff45..b71e8a1e5d 100644 --- a/lib/kernel/doc/src/inet.xml +++ b/lib/kernel/doc/src/inet.xml @@ -75,8 +75,8 @@ Address ip_address() ------- ------------ ::1 {0,0,0,0,0,0,0,1} ::192.168.42.2 {0,0,0,0,0,0,(192 bsl 8) bor 168,(42 bsl 8) bor 2} -FFFF::192.168.42.2 - {16#FFFF,0,0,0,0,0,(192 bsl 8) bor 168,(42 bsl 8) bor 2} +::FFFF:192.168.42.2 + {0,0,0,0,0,16#FFFF,(192 bsl 8) bor 168,(42 bsl 8) bor 2} 3ffe:b80:1f8d:2:204:acff:fe17:bf38 {16#3ffe,16#b80,16#1f8d,16#2,16#204,16#acff,16#fe17,16#bf38} fe80::204:acff:fe17:bf38 @@ -87,8 +87,8 @@ fe80::204:acff:fe17:bf38 <pre> 1> <input>inet:parse_address("192.168.42.2").</input> {ok,{192,168,42,2}} -2> <input>inet:parse_address("FFFF::192.168.42.2").</input> -{ok,{65535,0,0,0,0,0,49320,10754}}</pre> +2> <input>inet:parse_address("::FFFF:192.168.42.2").</input> +{ok,{0,0,0,0,0,65535,49320,10754}}</pre> </description> <datatypes> diff --git a/lib/kernel/doc/src/kernel_app.xml b/lib/kernel/doc/src/kernel_app.xml index 9fccb4c7ac..75e1e18d86 100644 --- a/lib/kernel/doc/src/kernel_app.xml +++ b/lib/kernel/doc/src/kernel_app.xml @@ -58,6 +58,7 @@ </section> <section> + <marker id="erl_signal_server"/> <title>OS Signal Event Handler</title> <p>Asynchronous OS signals may be subscribed to via the Kernel applications event manager (see <seealso marker="doc/design_principles:des_princ">OTP Design Principles</seealso> and diff --git a/lib/kernel/doc/src/net_kernel.xml b/lib/kernel/doc/src/net_kernel.xml index 4e2b0c69db..7ddb849824 100644 --- a/lib/kernel/doc/src/net_kernel.xml +++ b/lib/kernel/doc/src/net_kernel.xml @@ -64,6 +64,19 @@ $ <input>erl -sname foobar</input></pre> by the magic cookie system, see section <seealso marker="doc/reference_manual:distributed">Distributed Erlang</seealso> in the Erlang Reference Manual.</p> + <warning> + <p> + Starting a distributed node without also specifying + <seealso marker="erts:erl#proto_dist"><c>-proto_dist inet_tls</c></seealso> + will expose the node to attacks that may give the attacker + complete access to the node and in extension the cluster. + When using un-secure distributed nodes, make sure that the + network is configured to keep potential attackers out. + See the <seealso marker="ssl:ssl_distribution"> + Using SSL for Erlang Distribution</seealso> User's Guide + for details on how to setup a secure distributed node. + </p> + </warning> </description> <funcs> diff --git a/lib/kernel/doc/src/os.xml b/lib/kernel/doc/src/os.xml index 64c5cbe571..0e9add4161 100644 --- a/lib/kernel/doc/src/os.xml +++ b/lib/kernel/doc/src/os.xml @@ -174,8 +174,9 @@ DirOut = os:cmd("dir"), % on Win32 platform</code> <tag><c>handle</c></tag> <item> - This signal will notify <c>erl_signal_server</c> when it is received by - the Erlang runtime system. + This signal will notify + <seealso marker="kernel_app#erl_signal_server"><c>erl_signal_server</c></seealso> + when it is received by the Erlang runtime system. </item> </taglist> </desc> diff --git a/lib/kernel/src/dist_util.erl b/lib/kernel/src/dist_util.erl index 1c326afca8..b3507e5d13 100644 --- a/lib/kernel/src/dist_util.erl +++ b/lib/kernel/src/dist_util.erl @@ -572,12 +572,25 @@ recv_name(#hs_data{socket = Socket, f_recv = Recv}) -> ?shutdown(no_node) end. -get_name([$n,VersionA, VersionB, Flag1, Flag2, Flag3, Flag4 | OtherNode]) -> - {?u32(Flag1, Flag2, Flag3, Flag4), list_to_atom(OtherNode), - ?u16(VersionA,VersionB)}; +get_name([$n,VersionA, VersionB, Flag1, Flag2, Flag3, Flag4 | OtherNode] = Data) -> + case is_valid_name(OtherNode) of + true -> + {?u32(Flag1, Flag2, Flag3, Flag4), list_to_atom(OtherNode), + ?u16(VersionA,VersionB)}; + false -> + ?shutdown(Data) + end; get_name(Data) -> ?shutdown(Data). +is_valid_name(OtherNodeName) -> + case string:lexemes(OtherNodeName,"@") of + [_OtherNodeName,_OtherNodeHost] -> + true; + _else -> + false + end. + publish_type(Flags) -> case Flags band ?DFLAG_PUBLISHED of 0 -> diff --git a/lib/kernel/src/error_logger.erl b/lib/kernel/src/error_logger.erl index 81f1bf8d97..9bf8547745 100644 --- a/lib/kernel/src/error_logger.erl +++ b/lib/kernel/src/error_logger.erl @@ -31,6 +31,8 @@ handle_event/2, handle_call/2, handle_info/2, terminate/2]). +-export([get_format_depth/0, limit_term/1]). + -define(buffer_size, 10). %%----------------------------------------------------------------- @@ -518,3 +520,21 @@ string_p1([H|T]) when is_list(H) -> end; string_p1([]) -> true; string_p1(_) -> false. + +-spec limit_term(term()) -> term(). + +limit_term(Term) -> + case get_format_depth() of + unlimited -> Term; + D -> io_lib:limit_term(Term, D) + end. + +-spec get_format_depth() -> 'unlimited' | pos_integer(). + +get_format_depth() -> + case application:get_env(kernel, error_logger_format_depth) of + {ok, Depth} when is_integer(Depth) -> + max(10, Depth); + undefined -> + unlimited + end. diff --git a/lib/kernel/src/global.erl b/lib/kernel/src/global.erl index 3d6415036c..a9e92b28b8 100644 --- a/lib/kernel/src/global.erl +++ b/lib/kernel/src/global.erl @@ -447,7 +447,8 @@ info() -> init([]) -> process_flag(trap_exit, true), _ = ets:new(global_locks, [set, named_table, protected]), - _ = ets:new(global_names, [set, named_table, protected]), + _ = ets:new(global_names, [set, named_table, protected, + {read_concurrency, true}]), _ = ets:new(global_names_ext, [set, named_table, protected]), _ = ets:new(global_pid_names, [bag, named_table, protected]), diff --git a/lib/kernel/test/code_SUITE.erl b/lib/kernel/test/code_SUITE.erl index 7831777726..6f8e949aac 100644 --- a/lib/kernel/test/code_SUITE.erl +++ b/lib/kernel/test/code_SUITE.erl @@ -107,14 +107,19 @@ init_per_testcase(big_boot_embedded, Config) -> _Else -> {skip, "Needs crypto!"} end; -init_per_testcase(on_load_embedded, Config) -> +init_per_testcase(on_load_embedded, Config0) -> LibRoot = code:lib_dir(), LinkName = filename:join(LibRoot, "on_load_app-1.0"), - [{link_name,LinkName}|Config]; + Config = [{link_name,LinkName}|Config0], + init_per_testcase(Config); init_per_testcase(_Func, Config) -> + init_per_testcase(Config). + +init_per_testcase(Config) -> P = code:get_path(), [{code_path, P}|Config]. + end_per_testcase(module_status, Config) -> code:purge(?TESTMOD), code:delete(?TESTMOD), @@ -1786,6 +1791,19 @@ do_normalized_paths([]) -> %% Test that module_status/1 behaves as expected module_status(_Config) -> + case test_server:is_cover() of + true -> + module_status(); + false -> + %% Make sure that we terminate the cover server. + try + module_status() + after + cover:stop() + end + end. + +module_status() -> %% basics not_loaded = code:module_status(fubar), % nonexisting {file, preloaded} = code:is_loaded(erlang), diff --git a/lib/kernel/test/erl_distribution_SUITE.erl b/lib/kernel/test/erl_distribution_SUITE.erl index d7a9ac39a3..bbfaa9d147 100644 --- a/lib/kernel/test/erl_distribution_SUITE.erl +++ b/lib/kernel/test/erl_distribution_SUITE.erl @@ -230,7 +230,7 @@ legal(Name) -> end. illegal(Name) -> - case test_node(Name) of + case test_node(Name, true) of not_started -> ok; started -> @@ -238,12 +238,20 @@ illegal(Name) -> end. test_node(Name) -> + test_node(Name, false). +test_node(Name, Illigal) -> ProgName = atom_to_list(lib:progname()), Command = ProgName ++ " -noinput " ++ long_or_short() ++ Name ++ - " -eval \"net_adm:ping('" ++ atom_to_list(node()) ++ "')\"", + " -eval \"net_adm:ping('" ++ atom_to_list(node()) ++ "')\"" ++ + case Illigal of + true -> + " -eval \"timer:sleep(10000),init:stop().\""; + false -> + "" + end, net_kernel:monitor_nodes(true), BinCommand = unicode:characters_to_binary(Command, utf8), - open_port({spawn, BinCommand}, [stream]), + Prt = open_port({spawn, BinCommand}, [stream]), Node = list_to_atom(Name), receive {nodeup, Node} -> diff --git a/lib/kernel/test/gen_tcp_api_SUITE.erl b/lib/kernel/test/gen_tcp_api_SUITE.erl index 92a74465b7..3f11e25b93 100644 --- a/lib/kernel/test/gen_tcp_api_SUITE.erl +++ b/lib/kernel/test/gen_tcp_api_SUITE.erl @@ -302,9 +302,9 @@ t_implicit_inet6(Config) when is_list(Config) -> end. t_implicit_inet6(Host, Addr) -> - case gen_tcp:listen(0, [inet6]) of + Loopback = {0,0,0,0,0,0,0,1}, + case gen_tcp:listen(0, [inet6, {ip,Loopback}]) of {ok,S1} -> - Loopback = {0,0,0,0,0,0,0,1}, io:format("~s ~p~n", ["::1",Loopback]), implicit_inet6(S1, Loopback), ok = gen_tcp:close(S1), @@ -524,10 +524,10 @@ local_handshake(S, SAddr, C, CAddr) -> t_accept_inet6_tclass(Config) when is_list(Config) -> TClassOpt = {tclass,8#56 bsl 2}, % Expedited forwarding - case gen_tcp:listen(0, [inet6,TClassOpt]) of + Loopback = {0,0,0,0,0,0,0,1}, + case gen_tcp:listen(0, [inet6, {ip, Loopback}, TClassOpt]) of {ok,L} -> LPort = ok(inet:port(L)), - Loopback = {0,0,0,0,0,0,0,1}, Sa = ok(gen_tcp:connect(Loopback, LPort, [])), Sb = ok(gen_tcp:accept(L)), [TClassOpt] = ok(inet:getopts(Sb, [tclass])), diff --git a/lib/kernel/test/gen_udp_SUITE.erl b/lib/kernel/test/gen_udp_SUITE.erl index 1029d7ef0a..836e0c5a05 100644 --- a/lib/kernel/test/gen_udp_SUITE.erl +++ b/lib/kernel/test/gen_udp_SUITE.erl @@ -717,9 +717,9 @@ implicit_inet6(Config) when is_list(Config) -> implicit_inet6(Host, Addr) -> Active = {active,false}, - case gen_udp:open(0, [inet6,Active]) of + Loopback = {0,0,0,0,0,0,0,1}, + case gen_udp:open(0, [inet6,Active,{ip, Loopback}]) of {ok,S1} -> - Loopback = {0,0,0,0,0,0,0,1}, io:format("~s ~p~n", ["::1",Loopback]), implicit_inet6(S1, Active, Loopback), ok = gen_udp:close(S1), diff --git a/lib/kernel/test/inet_sockopt_SUITE.erl b/lib/kernel/test/inet_sockopt_SUITE.erl index 322b9f30fe..9413cbd976 100644 --- a/lib/kernel/test/inet_sockopt_SUITE.erl +++ b/lib/kernel/test/inet_sockopt_SUITE.erl @@ -620,7 +620,7 @@ ipv6_v6only_close(Module, Socket) -> %% Test using socket option ipv6_v6only for UDP. use_ipv6_v6only_udp(Config) when is_list(Config) -> - case gen_udp:open(0, [inet6,{ipv6_v6only,true}]) of + case gen_udp:open(0, [inet6,{ip,{0,0,0,0,0,0,0,1}}, {ipv6_v6only,true}]) of {ok,S6} -> case inet:getopts(S6, [ipv6_v6only]) of {ok,[{ipv6_v6only,true}]} -> diff --git a/lib/mnesia/src/mnesia.erl b/lib/mnesia/src/mnesia.erl index dece995d39..3b771e8c5b 100644 --- a/lib/mnesia/src/mnesia.erl +++ b/lib/mnesia/src/mnesia.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2017. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -166,7 +166,7 @@ -type select_continuation() :: term(). -type snmp_struct() :: [{atom(), snmp_type() | tuple_of(snmp_type())}]. -type snmp_type() :: 'fix_string' | 'string' | 'integer'. --type tuple_of(_) :: tuple(). +-type tuple_of(_T) :: tuple(). -define(DEFAULT_ACCESS, ?MODULE). diff --git a/lib/observer/src/etop.erl b/lib/observer/src/etop.erl index 925f4456bb..2093e6a0d7 100644 --- a/lib/observer/src/etop.erl +++ b/lib/observer/src/etop.erl @@ -180,10 +180,16 @@ stop(Opts) -> end, unregister(etop_server). -update(#opts{store=Store,node=Node,tracing=Tracing}=Opts) -> +update(#opts{store=Store,node=Node,tracing=Tracing,intv=Interval}=Opts) -> Pid = spawn_link(Node,observer_backend,etop_collect,[self()]), Info = receive {Pid,I} -> I - after 1000 -> exit(connection_lost) + after Interval -> + %% Took more than the update interval to fetch + %% data. Either the connection is lost or the + %% fetching took too long... + io:format("Timeout when waiting for process info from " + "node ~p; exiting~n", [Node]), + exit(timeout) end, #etop_info{procinfo=ProcInfo} = Info, ProcInfo1 = diff --git a/lib/observer/src/observer_lib.erl b/lib/observer/src/observer_lib.erl index 4145a8d961..7352af936c 100644 --- a/lib/observer/src/observer_lib.erl +++ b/lib/observer/src/observer_lib.erl @@ -173,13 +173,13 @@ fill_info([{Str,Attrib,Key}|Rest], Data) when is_atom(Key); is_function(Key) -> Value -> [{Str,Attrib,Value} | fill_info(Rest, Data)] end; fill_info([{Str, {Format, Key}}|Rest], Data) - when is_atom(Key); is_function(Key), is_atom(Format) -> + when is_atom(Key); is_function(Key) -> case get_value(Key, Data) of undefined -> [undefined | fill_info(Rest, Data)]; Value -> [{Str, {Format, Value}} | fill_info(Rest, Data)] end; fill_info([{Str, Attrib, {Format, Key}}|Rest], Data) - when is_atom(Key); is_function(Key), is_atom(Format) -> + when is_atom(Key); is_function(Key) -> case get_value(Key, Data) of undefined -> [undefined | fill_info(Rest, Data)]; Value -> [{Str, Attrib, {Format, Value}} | fill_info(Rest, Data)] @@ -252,6 +252,8 @@ to_str({bytes, B}) -> KB > 0 -> integer_to_list(KB) ++ " kB"; true -> integer_to_list(B) ++ " B" end; +to_str({{words,WSz}, Sz}) -> + to_str({bytes, WSz*Sz}); to_str({time_ms, MS}) -> S = MS div 1000, Min = S div 60, diff --git a/lib/observer/src/observer_procinfo.erl b/lib/observer/src/observer_procinfo.erl index 8d19d77488..10decd8b62 100644 --- a/lib/observer/src/observer_procinfo.erl +++ b/lib/observer/src/observer_procinfo.erl @@ -198,10 +198,11 @@ code_change(_, _, State) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% init_process_page(Panel, Pid) -> - Fields0 = process_info_fields(Pid), + WSz = observer_wx:try_rpc(node(Pid), erlang, system_info,[wordsize]), + Fields0 = process_info_fields(Pid, WSz), {FPanel, _, UpFields} = observer_lib:display_info(Panel, Fields0), {FPanel, fun() -> - Fields = process_info_fields(Pid), + Fields = process_info_fields(Pid, WSz), observer_lib:update_info(UpFields, Fields) end}. @@ -359,7 +360,7 @@ create_menus(MenuBar) -> {"View", [#create_menu{id=?REFRESH, text="Refresh\tCtrl-R"}]}], observer_lib:create_menus(Menus, MenuBar, new_window). -process_info_fields(Pid) -> +process_info_fields(Pid, WSz) -> Struct = [{"Overview", [{"Initial Call", initial_call}, {"Current Function", current_function}, @@ -383,10 +384,10 @@ process_info_fields(Pid) -> {"Monitored by", {click, monitored_by}}]}, {"Memory and Garbage Collection", right, [{"Memory", {bytes, memory}}, - {"Stack and Heaps", {bytes, total_heap_size}}, - {"Heap Size", {bytes, heap_size}}, - {"Stack Size", {bytes, stack_size}}, - {"GC Min Heap Size", {bytes, get_gc_info(min_heap_size)}}, + {"Stack and Heaps", {{words,WSz}, total_heap_size}}, + {"Heap Size", {{words,WSz}, heap_size}}, + {"Stack Size", {{words,WSz}, stack_size}}, + {"GC Min Heap Size", {{words,WSz}, get_gc_info(min_heap_size)}}, {"GC FullSweep After", get_gc_info(fullsweep_after)} ]}], case observer_wx:try_rpc(node(Pid), erlang, process_info, [Pid, item_list()]) of diff --git a/lib/orber/test/multi_ORB_SUITE.erl b/lib/orber/test/multi_ORB_SUITE.erl index d739e47cc1..8becc11d6a 100644 --- a/lib/orber/test/multi_ORB_SUITE.erl +++ b/lib/orber/test/multi_ORB_SUITE.erl @@ -135,13 +135,12 @@ cases() -> setup_multi_connection_timeout_api, setup_multi_connection_timeout_attempts_api, setup_multi_connection_timeout_random_api, - ssl_1_multi_orber_api, ssl_1_multi_orber_generation_3_api, - ssl_2_multi_orber_api, ssl_2_multi_orber_generation_3_api, - ssl_reconfigure_api, ssl_reconfigure_generation_3_api]. +% ssl_1_multi_orber_api,ssl_2_multi_orber_api,ssl_reconfigure_api, + %%----------------------------------------------------------------- %% Init and cleanup functions. %%----------------------------------------------------------------- diff --git a/lib/orber/test/orber_test_lib.erl b/lib/orber/test/orber_test_lib.erl index 95ab26cd30..4a247ce492 100644 --- a/lib/orber/test/orber_test_lib.erl +++ b/lib/orber/test/orber_test_lib.erl @@ -356,11 +356,17 @@ get_options(ssl, Role, Level) -> get_options(ssl, Role, 2, Options) -> Dir = filename:join([code:lib_dir(ssl), "examples", "certs", "etc"]), - [{depth, 2}, - {verify, 2}, - {keyfile, filename:join([Dir, Role, "key.pem"])}, - {cacertfile, filename:join([Dir, Role, "cacerts.pem"])}, - {certfile, filename:join([Dir, Role, "cert.pem"])} |Options]; + Options1 = [{depth, 2}, + {verify, 2}, + {keyfile, filename:join([Dir, Role, "key.pem"])}, + {cacertfile, filename:join([Dir, Role, "cacerts.pem"])}, + {certfile, filename:join([Dir, Role, "cert.pem"])} |Options], + case Role of + client -> + [{server_name_indication, disable} |Options1]; + server -> + Options1 + end; get_options(iiop_ssl, _Role, 2, Options) -> Dir = filename:join([code:lib_dir(ssl), "examples", "certs", "etc"]), [{ssl_server_options, [{depth, 2}, @@ -369,10 +375,11 @@ get_options(iiop_ssl, _Role, 2, Options) -> {cacertfile, filename:join([Dir, "server", "cacerts.pem"])}, {keyfile, filename:join([Dir, "server", "key.pem"])}]}, {ssl_client_options, [{depth, 2}, - {verify, 2}, - {certfile, filename:join([Dir, "client", "cert.pem"])}, - {cacertfile, filename:join([Dir, "client", "cacerts.pem"])}, - {keyfile, filename:join([Dir, "client", "key.pem"])}]}, + {verify, 2}, + {server_name_indication, disable}, + {certfile, filename:join([Dir, "client", "cert.pem"])}, + {cacertfile, filename:join([Dir, "client", "cacerts.pem"])}, + {keyfile, filename:join([Dir, "client", "key.pem"])}]}, {secure, ssl} |Options]; get_options(iiop_ssl, _Role, 1, Options) -> Dir = filename:join([code:lib_dir(ssl), "examples", "certs", "etc"]), @@ -382,10 +389,11 @@ get_options(iiop_ssl, _Role, 1, Options) -> {cacertfile, filename:join([Dir, "server", "cacerts.pem"])}, {keyfile, filename:join([Dir, "server", "key.pem"])}]}, {ssl_client_options, [{depth, 1}, - {verify, 0}, - {certfile, filename:join([Dir, "client", "cert.pem"])}, - {cacertfile, filename:join([Dir, "client", "cacerts.pem"])}, - {keyfile, filename:join([Dir, "client", "key.pem"])}]}, + {verify, 0}, + {server_name_indication, disable}, + {certfile, filename:join([Dir, "client", "cert.pem"])}, + {cacertfile, filename:join([Dir, "client", "cacerts.pem"])}, + {keyfile, filename:join([Dir, "client", "key.pem"])}]}, {secure, ssl} |Options]. create_paths() -> diff --git a/lib/public_key/src/pubkey_ssh.erl b/lib/public_key/src/pubkey_ssh.erl index 6974afa992..9bda76d670 100644 --- a/lib/public_key/src/pubkey_ssh.erl +++ b/lib/public_key/src/pubkey_ssh.erl @@ -408,10 +408,11 @@ comma_list_encode([Option | Rest], Acc) -> comma_list_encode(Rest, Acc ++ "," ++ Option). -%% An experimental fix adding the signature algorithm name as the last element in a tuple... - ssh2_pubkey_encode(#'RSAPublicKey'{modulus = N, publicExponent = E}) -> ssh2_pubkey_encode({#'RSAPublicKey'{modulus = N, publicExponent = E}, 'ssh-rsa'}); + +ssh2_pubkey_encode({Key, 'rsa-sha2-256'}) -> ssh2_pubkey_encode({Key, 'ssh-rsa'}); +ssh2_pubkey_encode({Key, 'rsa-sha2-512'}) -> ssh2_pubkey_encode({Key, 'ssh-rsa'}); ssh2_pubkey_encode({#'RSAPublicKey'{modulus = N, publicExponent = E}, SignAlg}) -> SignAlgName = list_to_binary(atom_to_list(SignAlg)), StrLen = size(SignAlgName), @@ -448,16 +449,12 @@ ssh2_pubkey_encode(Key={#'ECPoint'{point = Q}, {namedCurve,OID}}) -> ssh2_pubkey_decode(Bin = <<?UINT32(Len), Type:Len/binary, _/binary>>) -> ssh2_pubkey_decode(Type, Bin). -%% An experimental fix with the Signature Algorithm Name -ssh2_pubkey_decode(SignAlgName, +ssh2_pubkey_decode(<<"rsa-sha2-256">>, Bin) -> ssh2_pubkey_decode(<<"ssh-rsa">>, Bin); +ssh2_pubkey_decode(<<"rsa-sha2-512">>, Bin) -> ssh2_pubkey_decode(<<"ssh-rsa">>, Bin); +ssh2_pubkey_decode(<<"ssh-rsa">>, <<?UINT32(Len), _:Len/binary, ?UINT32(SizeE), E:SizeE/binary, - ?UINT32(SizeN), N:SizeN/binary>>) - when SignAlgName == <<"ssh-rsa">> ; - SignAlgName == <<"rsa-sha2-256">> ; - SignAlgName == <<"rsa-sha2-384">> ; - SignAlgName == <<"rsa-sha2-512">> - -> + ?UINT32(SizeN), N:SizeN/binary>>) -> #'RSAPublicKey'{modulus = erlint(SizeN, N), publicExponent = erlint(SizeE, E)}; @@ -471,6 +468,7 @@ ssh2_pubkey_decode(<<"ssh-dss">>, #'Dss-Parms'{p = erlint(SizeP, P), q = erlint(SizeQ, Q), g = erlint(SizeG, G)}}; + ssh2_pubkey_decode(<<"ecdsa-sha2-",Id/binary>>, <<?UINT32(Len), ECDSA_SHA2_etc:Len/binary, ?UINT32(SizeId), Id:SizeId/binary, diff --git a/lib/sasl/src/sasl_report.erl b/lib/sasl/src/sasl_report.erl index 14702b0ad2..eb454155d5 100644 --- a/lib/sasl/src/sasl_report.erl +++ b/lib/sasl/src/sasl_report.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2017. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -68,12 +68,12 @@ write_report2(IO, Fd, Head, progress, Report) -> Format = format_key_val(Report), write_report_action(IO, Fd, Head, "~s", [Format]); write_report2(IO, Fd, Head, crash_report, Report) -> - Depth = get_depth(), + Depth = error_logger:get_format_depth(), Format = proc_lib:format(Report, latin1, Depth), write_report_action(IO, Fd, Head, "~s", [Format]). supervisor_format(Args0) -> - case get_depth() of + case error_logger:get_format_depth() of unlimited -> {" Supervisor: ~p~n" " Context: ~p~n" @@ -102,14 +102,6 @@ format_key_val([{Tag,Data}|Rep]) -> format_key_val(_) -> []. -get_depth() -> - case application:get_env(kernel, error_logger_format_depth) of - {ok, Depth} when is_integer(Depth) -> - max(10, Depth); - undefined -> - unlimited - end. - sup_get(Tag, Report) -> case lists:keysearch(Tag, 1, Report) of {value, {_, Value}} -> diff --git a/lib/ssh/src/ssh_auth.erl b/lib/ssh/src/ssh_auth.erl index 6cf659f830..ac64a7bf14 100644 --- a/lib/ssh/src/ssh_auth.erl +++ b/lib/ssh/src/ssh_auth.erl @@ -28,7 +28,8 @@ -include("ssh_auth.hrl"). -include("ssh_transport.hrl"). --export([publickey_msg/1, password_msg/1, keyboard_interactive_msg/1, +-export([get_public_key/2, + publickey_msg/1, password_msg/1, keyboard_interactive_msg/1, service_request_msg/1, init_userauth_request_msg/1, userauth_request_msg/1, handle_userauth_request/3, handle_userauth_info_request/2, handle_userauth_info_response/2 @@ -136,41 +137,49 @@ keyboard_interactive_msg([#ssh{user = User, Ssh) end. -publickey_msg([SigAlg, #ssh{user = User, - session_id = SessionId, - service = Service, - opts = Opts} = Ssh]) -> - Hash = ssh_transport:sha(SigAlg), + +get_public_key(SigAlg, #ssh{opts = Opts}) -> KeyAlg = key_alg(SigAlg), {KeyCb,KeyCbOpts} = ?GET_OPT(key_cb, Opts), UserOpts = ?GET_OPT(user_options, Opts), case KeyCb:user_key(KeyAlg, [{key_cb_private,KeyCbOpts}|UserOpts]) of - {ok, PrivKey} -> - SigAlgStr = atom_to_list(SigAlg), + {ok, PrivKey} -> try Key = ssh_transport:extract_public_key(PrivKey), public_key:ssh_encode(Key, ssh2_pubkey) of - PubKeyBlob -> - SigData = build_sig_data(SessionId, User, Service, - PubKeyBlob, SigAlgStr), - Sig = ssh_transport:sign(SigData, Hash, PrivKey), - SigBlob = list_to_binary([?string(SigAlgStr), - ?binary(Sig)]), - ssh_transport:ssh_packet( - #ssh_msg_userauth_request{user = User, - service = Service, - method = "publickey", - data = [?TRUE, - ?string(SigAlgStr), - ?binary(PubKeyBlob), - ?binary(SigBlob)]}, - Ssh) + PubKeyBlob -> {ok,{PrivKey,PubKeyBlob}} catch _:_ -> - {not_ok, Ssh} + not_ok end; - _Error -> + _Error -> + not_ok + end. + + +publickey_msg([SigAlg, #ssh{user = User, + session_id = SessionId, + service = Service} = Ssh]) -> + case get_public_key(SigAlg, Ssh) of + {ok, {PrivKey,PubKeyBlob}} -> + SigAlgStr = atom_to_list(SigAlg), + SigData = build_sig_data(SessionId, User, Service, + PubKeyBlob, SigAlgStr), + Hash = ssh_transport:sha(SigAlg), + Sig = ssh_transport:sign(SigData, Hash, PrivKey), + SigBlob = list_to_binary([?string(SigAlgStr), + ?binary(Sig)]), + ssh_transport:ssh_packet( + #ssh_msg_userauth_request{user = User, + service = Service, + method = "publickey", + data = [?TRUE, + ?string(SigAlgStr), + ?binary(PubKeyBlob), + ?binary(SigBlob)]}, + Ssh); + _ -> {not_ok, Ssh} end. diff --git a/lib/ssh/src/ssh_connection_handler.erl b/lib/ssh/src/ssh_connection_handler.erl index 6a6b9896cb..8d3ddb09a4 100644 --- a/lib/ssh/src/ssh_connection_handler.erl +++ b/lib/ssh/src/ssh_connection_handler.erl @@ -346,7 +346,7 @@ renegotiate_data(ConnectionHandler) -> | undefined, last_size_rekey = 0 :: non_neg_integer(), event_queue = [] :: list(), - opts :: ssh_options:options(), +% opts :: ssh_options:options(), inet_initial_recbuf_size :: pos_integer() | undefined }). @@ -398,8 +398,7 @@ init([Role,Socket,Opts]) -> transport_protocol = Protocol, transport_cb = Callback, transport_close_tag = CloseTag, - ssh_params = init_ssh_record(Role, Socket, PeerAddr, Opts), - opts = Opts + ssh_params = init_ssh_record(Role, Socket, PeerAddr, Opts) }, D = case Role of client -> @@ -454,16 +453,20 @@ init_ssh_record(Role, _Socket, PeerAddr, Opts) -> PeerName0 when is_list(PeerName0) -> PeerName0 end, - S0#ssh{c_vsn = Vsn, - c_version = Version, - io_cb = case ?GET_OPT(user_interaction, Opts) of - true -> ssh_io; - false -> ssh_no_io - end, - userauth_pubkeys = ?GET_OPT(pref_public_key_algs, Opts), - userauth_quiet_mode = ?GET_OPT(quiet_mode, Opts), - peer = {PeerName, PeerAddr} - }; + S1 = + S0#ssh{c_vsn = Vsn, + c_version = Version, + io_cb = case ?GET_OPT(user_interaction, Opts) of + true -> ssh_io; + false -> ssh_no_io + end, + userauth_quiet_mode = ?GET_OPT(quiet_mode, Opts), + peer = {PeerName, PeerAddr} + }, + S1#ssh{userauth_pubkeys = [K || K <- ?GET_OPT(pref_public_key_algs, Opts), + is_usable_user_pubkey(K, S1) + ] + }; server -> S0#ssh{s_vsn = Vsn, @@ -1012,7 +1015,7 @@ handle_event(cast, renegotiate, _, _) -> handle_event(cast, data_size, {connected,Role}, D) -> {ok, [{send_oct,Sent0}]} = inet:getstat(D#data.socket, [send_oct]), Sent = Sent0 - D#data.last_size_rekey, - MaxSent = ?GET_OPT(rekey_limit, D#data.opts), + MaxSent = ?GET_OPT(rekey_limit, (D#data.ssh_params)#ssh.opts), timer:apply_after(?REKEY_DATA_TIMOUT, gen_statem, cast, [self(), data_size]), case Sent >= MaxSent of true -> @@ -1701,24 +1704,49 @@ handle_ssh_msg_ext_info(#ssh_msg_ext_info{data=Data}, D0) -> lists:foldl(fun ext_info/2, D0, Data). -ext_info({"server-sig-algs",SigAlgs}, D0 = #data{ssh_params=#ssh{role=client, - userauth_pubkeys=ClientSigAlgs}=Ssh0}) -> - %% Make strings to eliminate risk of beeing bombed with odd strings that fills the atom table: - SupportedAlgs = lists:map(fun erlang:atom_to_list/1, ssh_transport:supported_algorithms(public_key)), - ServerSigAlgs = [list_to_atom(SigAlg) || SigAlg <- string:tokens(SigAlgs,","), - %% length of SigAlg is implicitly checked by the comparison - %% in member/2: - lists:member(SigAlg, SupportedAlgs) - ], - CommonAlgs = [Alg || Alg <- ServerSigAlgs, - lists:member(Alg, ClientSigAlgs)], - D0#data{ssh_params = Ssh0#ssh{userauth_pubkeys = CommonAlgs} }; +ext_info({"server-sig-algs",SigAlgsStr}, + D0 = #data{ssh_params=#ssh{role=client, + userauth_pubkeys=ClientSigAlgs}=Ssh0}) -> + %% ClientSigAlgs are the pub_key algortithms that: + %% 1) is usable, that is, the user has such a public key and + %% 2) is either the default list or set by the caller + %% with the client option 'pref_public_key_algs' + %% + %% The list is already checked for duplicates. + + SigAlgs = [A || Astr <- string:tokens(SigAlgsStr, ","), + A <- try [list_to_existing_atom(Astr)] + %% list_to_existing_atom will fail for unknown algorithms + catch _:_ -> [] + end], + + CommonAlgs = [A || A <- SigAlgs, + lists:member(A, ClientSigAlgs)], + + %% Re-arrange the client supported public-key algorithms so that the server + %% preferred ones are tried first. + %% Trying algorithms not mentioned by the server is ok, since the server can't know + %% if the client supports 'server-sig-algs' or not. + + D0#data{ + ssh_params = + Ssh0#ssh{ + userauth_pubkeys = + CommonAlgs ++ (ClientSigAlgs -- CommonAlgs) + }}; ext_info(_, D0) -> %% Not implemented D0. %%%---------------------------------------------------------------- +is_usable_user_pubkey(A, Ssh) -> + case ssh_auth:get_public_key(A, Ssh) of + {ok,_} -> true; + _ -> false + end. + +%%%---------------------------------------------------------------- handle_request(ChannelPid, ChannelId, Type, Data, WantReply, From, D) -> case ssh_channel:cache_lookup(cache(D), ChannelId) of #channel{remote_id = Id} = Channel -> @@ -1862,7 +1890,7 @@ get_repl(X, Acc) -> exit({get_repl,X,Acc}). %%%---------------------------------------------------------------- --define(CALL_FUN(Key,D), catch (?GET_OPT(Key, D#data.opts)) ). +-define(CALL_FUN(Key,D), catch (?GET_OPT(Key, (D#data.ssh_params)#ssh.opts)) ). disconnect_fun({disconnect,Msg}, D) -> ?CALL_FUN(disconnectfun,D)(Msg); disconnect_fun(Reason, D) -> ?CALL_FUN(disconnectfun,D)(Reason). @@ -1912,7 +1940,7 @@ retry_fun(User, Reason, #data{ssh_params = #ssh{opts = Opts, %%% channels open for a while. cache_init_idle_timer(D) -> - case ?GET_OPT(idle_time, D#data.opts) of + case ?GET_OPT(idle_time, (D#data.ssh_params)#ssh.opts) of infinity -> D#data{idle_timer_value = infinity, idle_timer_ref = infinity % A flag used later... diff --git a/lib/ssh/src/ssh_dbg.erl b/lib/ssh/src/ssh_dbg.erl index 7dfbfc3b4b..3f742ad9b6 100644 --- a/lib/ssh/src/ssh_dbg.erl +++ b/lib/ssh/src/ssh_dbg.erl @@ -22,9 +22,8 @@ -module(ssh_dbg). --export([messages/0, - messages/1, - messages/2, +-export([messages/0, messages/1, messages/2, messages/3, + auth/0, auth/1, auth/2, auth/3, stop/0 ]). @@ -36,98 +35,193 @@ -include("ssh_connect.hrl"). -include("ssh_auth.hrl"). --record(data, { - writer, - acc = []}). %%%================================================================ -messages() -> - messages(fun(String,_D) -> io:format(String) end). +messages() -> start(msg). +messages(F) -> start(msg,F). +messages(F,X) -> start(msg,F,X). +messages(F,M,I) -> start(msg,F,M,I). -messages(Write) when is_function(Write,2) -> - messages(Write, fun(X) -> X end). +auth() -> start(auth). +auth(F) -> start(auth,F). +auth(F,X) -> start(auth,F,X). +auth(F,M,I) -> start(auth,F,M,I). -messages(Write, MangleArg) when is_function(Write,2), - is_function(MangleArg,1) -> - catch dbg:start(), - setup_tracer(Write, MangleArg), - dbg:p(new,[c,timestamp]), - dbg_ssh_messages(). +stop() -> dbg:stop(). -dbg_ssh_messages() -> +%%%---------------------------------------------------------------- +start(Type) -> start(Type, fun io:format/2). + +start(Type, F) when is_function(F,2) -> start(Type, fmt_fun(F)); +start(Type, F) when is_function(F,3) -> start(Type, F, id_fun()). + +start(Type, WriteFun, MangleArgFun) when is_function(WriteFun, 3), + is_function(MangleArgFun, 1) -> + start(Type, WriteFun, MangleArgFun, []); +start(Type, WriteFun, InitValue) -> + start(Type, WriteFun, id_fun(), InitValue). + +start(Type, WriteFun, MangleArgFun, InitValue) when is_function(WriteFun, 3), + is_function(MangleArgFun, 1) -> + cond_start(Type, WriteFun, MangleArgFun, InitValue), + dbg_ssh(Type). + +%%%---------------------------------------------------------------- +fmt_fun(F) -> fun(Fmt,Args,Data) -> F(Fmt,Args), Data end. + +id_fun() -> fun(X) -> X end. + +%%%---------------------------------------------------------------- +dbg_ssh(msg) -> + dbg_ssh(auth), dbg:tp(ssh_message,encode,1, x), dbg:tp(ssh_message,decode,1, x), dbg:tpl(ssh_transport,select_algorithm,4, x), dbg:tp(ssh_transport,hello_version_msg,1, x), - dbg:tp(ssh_transport,handle_hello_version,1, x). + dbg:tp(ssh_transport,handle_hello_version,1, x), + dbg:tpl(ssh_connection_handler,ext_info,2, x); + +dbg_ssh(auth) -> + dbg:tp(ssh_transport,hello_version_msg,1, x), + dbg:tp(ssh_transport,handle_hello_version,1, x), + dbg:tp(ssh_message,encode,1, x), + dbg:tpl(ssh_transport,select_algorithm,4, x), + dbg:tpl(ssh_connection_handler,ext_info,2, x), + lists:foreach(fun(F) -> dbg:tp(ssh_auth, F, x) end, + [publickey_msg, password_msg, keyboard_interactive_msg]). -%%%---------------------------------------------------------------- -stop() -> - dbg:stop(). - %%%================================================================ -msg_formater({trace_ts,Pid,call,{ssh_message,encode,[Msg]},TS}, D) -> +cond_start(Type, WriteFun, MangleArgFun, Init) -> + try + dbg:start(), + setup_tracer(Type, WriteFun, MangleArgFun, Init), + dbg:p(new,[c,timestamp]) + catch + _:_ -> ok + end. + + +msg_formater(msg, {trace_ts,Pid,call,{ssh_message,encode,[Msg]},TS}, D) -> fmt("~n~s SEND ~p ~s~n", [ts(TS),Pid,wr_record(shrink_bin(Msg))], D); -msg_formater({trace_ts,_Pid,return_from,{ssh_message,encode,1},_Res,_TS}, D) -> +msg_formater(msg, {trace_ts,_Pid,return_from,{ssh_message,encode,1},_Res,_TS}, D) -> D; -msg_formater({trace_ts,_Pid,call,{ssh_message,decode,_},_TS}, D) -> +msg_formater(msg, {trace_ts,_Pid,call,{ssh_message,decode,_},_TS}, D) -> D; -msg_formater({trace_ts,Pid,return_from,{ssh_message,decode,1},Msg,TS}, D) -> +msg_formater(msg, {trace_ts,Pid,return_from,{ssh_message,decode,1},Msg,TS}, D) -> fmt("~n~s ~p RECV ~s~n", [ts(TS),Pid,wr_record(shrink_bin(Msg))], D); + +msg_formater(auth, {trace_ts,Pid,return_from,{ssh_message,decode,1},#ssh_msg_userauth_failure{authentications=As},TS}, D) -> + fmt("~n~s ~p Client login FAILURE. Try ~s~n", [ts(TS),Pid,As], D); -msg_formater({trace_ts,_Pid,call,{ssh_transport,select_algorithm,_},_TS}, D) -> +msg_formater(auth, {trace_ts,Pid,return_from,{ssh_message,decode,1},#ssh_msg_userauth_success{},TS}, D) -> + fmt("~n~s ~p Client login SUCCESS~n", [ts(TS),Pid], D); + + +msg_formater(_, {trace_ts,_Pid,call,{ssh_transport,select_algorithm,_},_TS}, D) -> D; -msg_formater({trace_ts,Pid,return_from,{ssh_transport,select_algorithm,_},{ok,Alg},TS}, D) -> +msg_formater(_, {trace_ts,Pid,return_from,{ssh_transport,select_algorithm,_},{ok,Alg},TS}, D) -> fmt("~n~s ~p ALGORITHMS~n~s~n", [ts(TS),Pid, wr_record(Alg)], D); -msg_formater({trace_ts,_Pid,call,{ssh_transport,hello_version_msg,_},_TS}, D) -> +msg_formater(_, {trace_ts,_Pid,call,{ssh_transport,hello_version_msg,_},_TS}, D) -> D; -msg_formater({trace_ts,Pid,return_from,{ssh_transport,hello_version_msg,1},Hello,TS}, D) -> +msg_formater(_, {trace_ts,Pid,return_from,{ssh_transport,hello_version_msg,1},Hello,TS}, D) -> fmt("~n~s ~p TCP SEND HELLO~n ~p~n", [ts(TS),Pid,lists:flatten(Hello)], D); -msg_formater({trace_ts,Pid,call,{ssh_transport,handle_hello_version,[Hello]},TS}, D) -> +msg_formater(_, {trace_ts,Pid,call,{ssh_transport,handle_hello_version,[Hello]},TS}, D) -> fmt("~n~s ~p RECV HELLO~n ~p~n", [ts(TS),Pid,lists:flatten(Hello)], D); -msg_formater({trace_ts,_Pid,return_from,{ssh_transport,handle_hello_version,1},_,_TS}, D) -> +msg_formater(_, {trace_ts,_Pid,return_from,{ssh_transport,handle_hello_version,1},_,_TS}, D) -> D; -msg_formater({trace_ts,Pid,send,{tcp,Sock,Bytes},Pid,TS}, D) -> +msg_formater(_, {trace_ts,Pid,call,{ssh_connection_handler,ext_info,[{"server-sig-algs",SigAlgs},State]},TS}, D) -> + try lists:keyfind(ssh, 1, tuple_to_list(State)) of + false -> + D; + #ssh{userauth_pubkeys = PKs} -> + fmt("~n~s ~p Client got suggestion to use user public key sig-algs~n ~p~n and can use~n ~p~n", + [ts(TS),Pid,string:tokens(SigAlgs,","),PKs], D) + catch + _:_ -> + D + end; + +msg_formater(_, {trace_ts,Pid,return_from,{ssh_connection_handler,ext_info,2},State,TS}, D) -> + try lists:keyfind(ssh, 1, tuple_to_list(State)) of + false -> + D; + #ssh{userauth_pubkeys = PKs} -> + fmt("~n~s ~p Client will try user public key sig-algs~n ~p~n", [ts(TS),Pid,PKs], D) + catch + _:_ -> + D + end; + +msg_formater(_, {trace_ts,Pid,call,{ssh_auth,publickey_msg,[[SigAlg,#ssh{user=User}]]},TS}, D) -> + fmt("~n~s ~p Client will try to login user ~p with public key algorithm ~p~n", [ts(TS),Pid,User,SigAlg], D); +msg_formater(_, {trace_ts,Pid,return_from,{ssh_auth,publickey_msg,1},{not_ok,#ssh{user=User}},TS}, D) -> + fmt("~s ~p User ~p can't login with that kind of public key~n", [ts(TS),Pid,User], D); + +msg_formater(_, {trace_ts,Pid,call,{ssh_auth,password_msg,[[#ssh{user=User}]]},TS}, D) -> + fmt("~n~s ~p Client will try to login user ~p with password~n", [ts(TS),Pid,User], D); +msg_formater(_, {trace_ts,Pid,return_from,{ssh_auth,password_msg,1},{not_ok,#ssh{user=User}},TS}, D) -> + fmt("~s ~p User ~p can't login with password~n", [ts(TS),Pid,User], D); + +msg_formater(_, {trace_ts,Pid,call,{ssh_auth,keyboard_interactive_msg,[[#ssh{user=User}]]},TS}, D) -> + fmt("~n~s ~p Client will try to login user ~p with password~n", [ts(TS),Pid,User], D); +msg_formater(_, {trace_ts,Pid,return_from,{ssh_auth,keyboard_interactive_msg,1},{not_ok,#ssh{user=User}},TS}, D) -> + fmt("~s ~p User ~p can't login with keyboard_interactive password~n", [ts(TS),Pid,User], D); + +msg_formater(msg, {trace_ts,Pid,send,{tcp,Sock,Bytes},Pid,TS}, D) -> fmt("~n~s ~p TCP SEND on ~p~n ~p~n", [ts(TS),Pid,Sock, shrink_bin(Bytes)], D); -msg_formater({trace_ts,Pid,send,{tcp,Sock,Bytes},Dest,TS}, D) -> +msg_formater(msg, {trace_ts,Pid,send,{tcp,Sock,Bytes},Dest,TS}, D) -> fmt("~n~s ~p TCP SEND from ~p TO ~p~n ~p~n", [ts(TS),Pid,Sock,Dest, shrink_bin(Bytes)], D); -msg_formater({trace_ts,Pid,send,ErlangMsg,Dest,TS}, D) -> +msg_formater(msg, {trace_ts,Pid,send,ErlangMsg,Dest,TS}, D) -> fmt("~n~s ~p ERL MSG SEND TO ~p~n ~p~n", [ts(TS),Pid,Dest, shrink_bin(ErlangMsg)], D); -msg_formater({trace_ts,Pid,'receive',{tcp,Sock,Bytes},TS}, D) -> +msg_formater(msg, {trace_ts,Pid,'receive',{tcp,Sock,Bytes},TS}, D) -> fmt("~n~s ~p TCP RECEIVE on ~p~n ~p~n", [ts(TS),Pid,Sock,shrink_bin(Bytes)], D); -msg_formater({trace_ts,Pid,'receive',ErlangMsg,TS}, D) -> +msg_formater(msg, {trace_ts,Pid,'receive',ErlangMsg,TS}, D) -> fmt("~n~s ~p ERL MSG RECEIVE~n ~p~n", [ts(TS),Pid,shrink_bin(ErlangMsg)], D); -msg_formater(M, D) -> - fmt("~nDBG ~n~p~n", [shrink_bin(M)], D). +%% msg_formater(_, {trace_ts,_Pid,return_from,MFA,_Ret,_TS}=M, D) -> +%% case lists:member(MFA, [{ssh_auth,keyboard_interactive_msg,1}, +%% {ssh_auth,password_msg,1}, +%% {ssh_auth,publickey_msg,1}]) of +%% true -> +%% D; +%% false -> +%% fmt("~nDBG ~n~p~n", [shrink_bin(M)], D) +%% end; + +%% msg_formater(_, M, D) -> +%% fmt("~nDBG ~n~p~n", [shrink_bin(M)], D). -%% msg_formater(_, D) -> -%% D. +msg_formater(_, _, D) -> + D. +%%%---------------------------------------------------------------- +-record(data, {writer, + acc}). -fmt(Fmt, Args, D=#data{writer=Write,acc=Acc}) -> - D#data{acc = Write(io_lib:format(Fmt, Args), Acc)}. +fmt(Fmt, Args, D=#data{writer=Write, acc=Acc}) -> + D#data{acc = Write(Fmt,Args,Acc)}. ts({_,_,Usec}=Now) -> {_Date,{HH,MM,SS}} = calendar:now_to_local_time(Now), io_lib:format("~.2.0w:~.2.0w:~.2.0w.~.6.0w",[HH,MM,SS,Usec]); ts(_) -> "-". -%%%---------------------------------------------------------------- -setup_tracer(Write, MangleArg) -> + +setup_tracer(Type, WriteFun, MangleArgFun, Init) -> Handler = fun(Arg, D) -> - msg_formater(MangleArg(Arg), D) + msg_formater(Type, MangleArgFun(Arg), D) end, - InitialData = #data{writer = Write}, + InitialData = #data{writer = WriteFun, + acc = Init}, {ok,_} = dbg:tracer(process, {Handler, InitialData}), ok. diff --git a/lib/ssh/src/ssh_message.erl b/lib/ssh/src/ssh_message.erl index 609040826f..b1fc05ae33 100644 --- a/lib/ssh/src/ssh_message.erl +++ b/lib/ssh/src/ssh_message.erl @@ -598,8 +598,8 @@ decode_kex_init(<<?DEC_BIN(Data,__0), Rest/binary>>, Acc, N) -> %%% Signature decode/encode %%% -decode_signature(<<?DEC_BIN(_Alg,__0), ?UINT32(_), Signature/binary>>) -> - Signature. +decode_signature(<<?DEC_BIN(Alg,__0), ?UINT32(_), Signature/binary>>) -> + {binary_to_list(Alg), Signature}. encode_signature({#'RSAPublicKey'{},Sign}, Signature) -> @@ -611,13 +611,3 @@ encode_signature({{#'ECPoint'{}, {namedCurve,OID}},_}, Signature) -> CurveName = public_key:oid2ssh_curvename(OID), <<?Ebinary(<<"ecdsa-sha2-",CurveName/binary>>), ?Ebinary(Signature)>>. -%% encode_signature(#'RSAPublicKey'{}, Signature) -> -%% SignName = <<"ssh-rsa">>, -%% <<?Ebinary(SignName), ?Ebinary(Signature)>>; -%% encode_signature({_, #'Dss-Parms'{}}, Signature) -> -%% <<?Ebinary(<<"ssh-dss">>), ?Ebinary(Signature)>>; -%% encode_signature({#'ECPoint'{}, {namedCurve,OID}}, Signature) -> -%% CurveName = public_key:oid2ssh_curvename(OID), -%% <<?Ebinary(<<"ecdsa-sha2-",CurveName/binary>>), ?Ebinary(Signature)>>. - - diff --git a/lib/ssh/src/ssh_options.erl b/lib/ssh/src/ssh_options.erl index 12c0190082..7eeed70739 100644 --- a/lib/ssh/src/ssh_options.erl +++ b/lib/ssh/src/ssh_options.erl @@ -392,12 +392,6 @@ default(server) -> class => user_options }, - {preferred_algorithms, def} => - #{default => ssh:default_algorithms(), - chk => fun check_preferred_algorithms/1, - class => user_options - }, - %%%%% Undocumented {infofun, def} => #{default => fun(_,_,_) -> void end, @@ -436,26 +430,11 @@ default(client) -> }, {pref_public_key_algs, def} => - #{default => ssh_transport:default_algorithms(public_key) -- ['rsa-sha2-256', - 'rsa-sha2-512'], + #{default => ssh_transport:default_algorithms(public_key), chk => fun check_pref_public_key_algs/1, class => user_options }, - {preferred_algorithms, def} => - #{default => [{K,Vs} || {K,Vs0} <- ssh:default_algorithms(), - Vs <- [case K of - public_key -> - Vs0 -- ['rsa-sha2-256', - 'rsa-sha2-512']; - _ -> - Vs0 - end] - ], - chk => fun check_preferred_algorithms/1, - class => user_options - }, - {dh_gex_limits, def} => #{default => {1024, 6144, 8192}, % FIXME: Is this true nowadays? chk => fun({Min,I,Max}) -> @@ -521,6 +500,12 @@ default(common) -> class => user_options }, + {preferred_algorithms, def} => + #{default => ssh:default_algorithms(), + chk => fun check_preferred_algorithms/1, + class => user_options + }, + {id_string, def} => #{default => undefined, % FIXME: see ssh_transport:ssh_vsn/0 chk => fun(random) -> @@ -689,7 +674,11 @@ check_pref_public_key_algs(V) -> PKs = ssh_transport:supported_algorithms(public_key), CHK = fun(A, Ack) -> case lists:member(A, PKs) of - true -> [A|Ack]; + true -> + case lists:member(A,Ack) of + false -> [A|Ack]; + true -> Ack % Remove duplicates + end; false -> error_in_check(A, "Not supported public key") end end, diff --git a/lib/ssh/src/ssh_transport.erl b/lib/ssh/src/ssh_transport.erl index 1a15798080..412f5de9de 100644 --- a/lib/ssh/src/ssh_transport.erl +++ b/lib/ssh/src/ssh_transport.erl @@ -776,16 +776,20 @@ extract_public_key(#'ECPrivateKey'{parameters = {namedCurve,OID}, {#'ECPoint'{point=Q}, {namedCurve,OID}}. -verify_host_key(#ssh{algorithms=Alg}=SSH, PublicKey, Digest, Signature) -> - case verify(Digest, sha(Alg#alg.hkey), Signature, PublicKey) of - false -> - {error, bad_signature}; - true -> - known_host_key(SSH, PublicKey, public_algo(PublicKey)) +verify_host_key(#ssh{algorithms=Alg}=SSH, PublicKey, Digest, {AlgStr,Signature}) -> + case atom_to_list(Alg#alg.hkey) of + AlgStr -> + case verify(Digest, sha(Alg#alg.hkey), Signature, PublicKey) of + false -> + {error, bad_signature}; + true -> + known_host_key(SSH, PublicKey, public_algo(PublicKey)) + end; + _ -> + {error, bad_signature_name} end. - accepted_host(Ssh, PeerName, Public, Opts) -> case ?GET_OPT(silently_accept_hosts, Opts) of diff --git a/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl b/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl index 0995182623..165274241c 100644 --- a/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl +++ b/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl @@ -280,21 +280,21 @@ msg_code(Num) -> Name -include_lib("ssh/src/ssh_transport.hrl"). %%% Encoding and decodeing is asymetric so out=binary in=string. Sometimes. :( +-define(fix_asym_Xdh_reply(S), + fix_asym(#S{public_host_key = Key, h_sig = {Alg,Sig}} = M) -> + M#S{public_host_key = {Key, list_to_atom(Alg)}, h_sig = Sig} +). + + fix_asym(#ssh_msg_global_request{name=N} = M) -> M#ssh_msg_global_request{name = binary_to_list(N)}; fix_asym(#ssh_msg_debug{message=D,language=L} = M) -> M#ssh_msg_debug{message = binary_to_list(D), language = binary_to_list(L)}; fix_asym(#ssh_msg_kexinit{cookie=C} = M) -> M#ssh_msg_kexinit{cookie = <<C:128>>}; - -fix_asym(#ssh_msg_kexdh_reply{public_host_key = Key} = M) -> M#ssh_msg_kexdh_reply{public_host_key = key_sigalg(Key)}; -fix_asym(#ssh_msg_kex_dh_gex_reply{public_host_key = Key} = M) -> M#ssh_msg_kex_dh_gex_reply{public_host_key = key_sigalg(Key)}; -fix_asym(#ssh_msg_kex_ecdh_reply{public_host_key = Key} = M) -> M#ssh_msg_kex_ecdh_reply{public_host_key = key_sigalg(Key)}; - +?fix_asym_Xdh_reply(ssh_msg_kexdh_reply); +?fix_asym_Xdh_reply(ssh_msg_kex_dh_gex_reply); +?fix_asym_Xdh_reply(ssh_msg_kex_ecdh_reply); fix_asym(M) -> M. -%%% Keys now contains an sig-algorithm name -key_sigalg(#'RSAPublicKey'{} = Key) -> {Key,'ssh-rsa'}; -key_sigalg({_, #'Dss-Parms'{}} = Key) -> {Key,'ssh-dss'}; -key_sigalg({#'ECPoint'{}, {namedCurve,OID}} = Key) -> {Key,"ecdsa-sha2-256"}. %%% Message codes 30 and 31 are overloaded depending on kex family so arrange the decoder %%% input as the test object does diff --git a/lib/ssh/test/ssh.spec b/lib/ssh/test/ssh.spec index 68268cb20d..b4e3d36072 100644 --- a/lib/ssh/test/ssh.spec +++ b/lib/ssh/test/ssh.spec @@ -1,6 +1,7 @@ {suites,"../ssh_test",all}. -{skip_suites, "../ssh_test", [ssh_bench_SUITE +{skip_suites, "../ssh_test", [ssh_bench_SUITE, + ssh_upgrade_SUITE ], "Benchmarks run separately"}. diff --git a/lib/ssh/test/ssh_algorithms_SUITE.erl b/lib/ssh/test/ssh_algorithms_SUITE.erl index 736461624d..98964a2c8a 100644 --- a/lib/ssh/test/ssh_algorithms_SUITE.erl +++ b/lib/ssh/test/ssh_algorithms_SUITE.erl @@ -131,9 +131,14 @@ init_per_group(public_key=Tag, Alg, Config) -> ct:log("Init tests for public_key ~p",[Alg]), PrefAlgs = {preferred_algorithms,[{Tag,[Alg]}]}, %% Daemon started later in init_per_testcase - [{pref_algs,PrefAlgs}, - {tag_alg,{Tag,Alg}} - | Config]; + try + setup_pubkey(Alg, + [{pref_algs,PrefAlgs}, + {tag_alg,{Tag,Alg}} + | Config]) + catch + _:_ -> {skip, io_lib:format("Unsupported: ~p",[Alg])} + end; init_per_group(Tag, Alg, Config) -> PA = @@ -167,17 +172,24 @@ init_per_testcase(TC, Config) -> init_per_testcase(TC, proplists:get_value(tag_alg,Config), Config). -init_per_testcase(_, {public_key,Alg}, Config) -> - Opts = pubkey_opts(Config), +init_per_testcase(TC, {public_key,Alg}, Config) -> + ExtraOpts = case TC of + simple_connect -> + [{user_dir, proplists:get_value(priv_dir,Config)}]; + _ -> + [] + end, + Opts = pubkey_opts(Config) ++ ExtraOpts, case {ssh_file:user_key(Alg,Opts), ssh_file:host_key(Alg,Opts)} of {{ok,_}, {ok,_}} -> - start_pubkey_daemon([proplists:get_value(pref_algs,Config)], + start_pubkey_daemon([proplists:get_value(pref_algs,Config) + | ExtraOpts], [{extra_daemon,true}|Config]); - {{ok,_}, _} -> - {skip, "No host key"}; + {{ok,_}, {error,Err}} -> + {skip, io_lib:format("No host key: ~p",[Err])}; - {_, {ok,_}} -> - {skip, "No user key"}; + {{error,Err}, {ok,_}} -> + {skip, io_lib:format("No user key: ~p",[Err])}; _ -> {skip, "Neither host nor user key"} @@ -221,6 +233,19 @@ simple_exec(Config) -> ssh_test_lib:std_simple_exec(Host, Port, Config). %%-------------------------------------------------------------------- +%% A simple exec call +simple_connect(Config) -> + {Host,Port} = proplists:get_value(srvr_addr, Config), + Opts = + case proplists:get_value(tag_alg, Config) of + {public_key,Alg} -> [{pref_public_key_algs,[Alg]}]; + _ -> [] + end, + ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, Opts), + ct:log("~p:~p connected! ~p",[?MODULE,?LINE,ConnectionRef]), + ssh:close(ConnectionRef). + +%%-------------------------------------------------------------------- %% Testing if no group matches simple_exec_groups_no_match_too_small(Config) -> try_exec_simple_group({400,500,600}, Config). @@ -302,9 +327,15 @@ sshc_simple_exec_os_cmd(Config) -> %%-------------------------------------------------------------------- %% Connect to the ssh server of the OS sshd_simple_exec(Config) -> + ClientPubKeyOpts = + case proplists:get_value(tag_alg,Config) of + {public_key,Alg} -> [{pref_public_key_algs,[Alg]}]; + _ -> [] + end, ConnectionRef = ssh_test_lib:connect(22, [{silently_accept_hosts, true}, proplists:get_value(pref_algs,Config), - {user_interaction, false}]), + {user_interaction, false} + | ClientPubKeyOpts]), {ok, ChannelId0} = ssh_connection:session_channel(ConnectionRef, infinity), success = ssh_connection:exec(ConnectionRef, ChannelId0, "echo testing", infinity), @@ -361,8 +392,8 @@ split(Alg) -> ssh_test_lib:to_atoms(string:tokens(atom_to_list(Alg), " + ")). specific_test_cases(Tag, Alg, SshcAlgos, SshdAlgos, TypeSSH) -> case Tag of - public_key -> []; - _ -> [simple_exec, simple_sftp] + public_key -> [simple_connect]; + _ -> [simple_connect, simple_exec, simple_sftp] end ++ case supports(Tag, Alg, SshcAlgos) of true when TypeSSH == openSSH -> @@ -437,10 +468,26 @@ setup_pubkey(Config) -> Keys = [ssh_test_lib:setup_dsa(DataDir, UserDir), ssh_test_lib:setup_rsa(DataDir, UserDir), - ssh_test_lib:setup_ecdsa("256", DataDir, UserDir)], + ssh_test_lib:setup_ecdsa("256", DataDir, UserDir) + ], ssh_test_lib:write_auth_keys(Keys, UserDir), % 'authorized_keys' shall contain ALL pub keys Config. +setup_pubkey(Alg, Config) -> + DataDir = proplists:get_value(data_dir, Config), + UserDir = proplists:get_value(priv_dir, Config), + ct:log("Setup keys for ~p",[Alg]), + case Alg of + 'ssh-dss' -> ssh_test_lib:setup_dsa(DataDir, UserDir); + 'ssh-rsa' -> ssh_test_lib:setup_rsa(DataDir, UserDir); + 'rsa-sha2-256' -> ssh_test_lib:setup_rsa(DataDir, UserDir); + 'rsa-sha2-512' -> ssh_test_lib:setup_rsa(DataDir, UserDir); + 'ecdsa-sha2-nistp256' -> ssh_test_lib:setup_ecdsa("256", DataDir, UserDir); + 'ecdsa-sha2-nistp384' -> ssh_test_lib:setup_ecdsa("384", DataDir, UserDir); + 'ecdsa-sha2-nistp521' -> ssh_test_lib:setup_ecdsa("521", DataDir, UserDir) + end, + Config. + simple_exec_group(I, Config) when is_integer(I) -> simple_exec_group({I,I,I}, Config); diff --git a/lib/ssh/test/ssh_algorithms_SUITE_data/id_ecdsa384 b/lib/ssh/test/ssh_algorithms_SUITE_data/id_ecdsa384 new file mode 100644 index 0000000000..4c39e916e9 --- /dev/null +++ b/lib/ssh/test/ssh_algorithms_SUITE_data/id_ecdsa384 @@ -0,0 +1,6 @@ +-----BEGIN EC PRIVATE KEY----- +MIGkAgEBBDAughXu55DNyhxe6x+MNjv4oZKWUDh7bhi4CqjvxhCp9KMpsybltcq+ +lsuKTarzTdKgBwYFK4EEACKhZANiAASu1vvDL0SQoXGtzlltaPHPyDfEVMG/sKLA +pqv8vfRN5Wcs7+yaRKw92nYEKGXfZLbhVX8ArFPMtXPWHcRHCntvL1Acn2kJQ8Gc +7iL4NAr8JhTIUBv4YMhHDa9Pv/CH2zk= +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_algorithms_SUITE_data/id_ecdsa384.pub b/lib/ssh/test/ssh_algorithms_SUITE_data/id_ecdsa384.pub new file mode 100644 index 0000000000..caa9604c84 --- /dev/null +++ b/lib/ssh/test/ssh_algorithms_SUITE_data/id_ecdsa384.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK7W+8MvRJChca3OWW1o8c/IN8RUwb+wosCmq/y99E3lZyzv7JpErD3adgQoZd9ktuFVfwCsU8y1c9YdxEcKe28vUByfaQlDwZzuIvg0CvwmFMhQG/hgyEcNr0+/8IfbOQ== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_algorithms_SUITE_data/id_ecdsa521 b/lib/ssh/test/ssh_algorithms_SUITE_data/id_ecdsa521 new file mode 100644 index 0000000000..1e16fcbd57 --- /dev/null +++ b/lib/ssh/test/ssh_algorithms_SUITE_data/id_ecdsa521 @@ -0,0 +1,7 @@ +-----BEGIN EC PRIVATE KEY----- +MIHbAgEBBEEWXGoVLiNwQVUwAGZWxOu6uxtU8ntxyZNlcWU4Z8pze9kq3eK7a9XH +l/wxL75Vk1QdOiR/rE3s/L/zOuChp44o1aAHBgUrgQQAI6GBiQOBhgAEAfCrtwjO +kQYKr4/F3uanS7Eby1+SYDdRl1ABuDFhNC3CivVBFt4CnRneV+Mf0viDAxD+HEpd +/GaE2CdsFoVpglN5AVG+fEePY2PiCLHmjc4/pBuR+tWhErzcWAd0KLBCBuc4OAvl +aLLYV1NAJI6COnnfGTCVvYYE5nKMG4LLX0zaWtWl +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_algorithms_SUITE_data/id_ecdsa521.pub b/lib/ssh/test/ssh_algorithms_SUITE_data/id_ecdsa521.pub new file mode 100644 index 0000000000..069683eba7 --- /dev/null +++ b/lib/ssh/test/ssh_algorithms_SUITE_data/id_ecdsa521.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHwq7cIzpEGCq+Pxd7mp0uxG8tfkmA3UZdQAbgxYTQtwor1QRbeAp0Z3lfjH9L4gwMQ/hxKXfxmhNgnbBaFaYJTeQFRvnxHj2Nj4gix5o3OP6QbkfrVoRK83FgHdCiwQgbnODgL5Wiy2FdTQCSOgjp53xkwlb2GBOZyjBuCy19M2lrVpQ== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_algorithms_SUITE_data/ssh_host_ecdsa_key384 b/lib/ssh/test/ssh_algorithms_SUITE_data/ssh_host_ecdsa_key384 new file mode 100644 index 0000000000..5835bcd74c --- /dev/null +++ b/lib/ssh/test/ssh_algorithms_SUITE_data/ssh_host_ecdsa_key384 @@ -0,0 +1,6 @@ +-----BEGIN EC PRIVATE KEY----- +MIGkAgEBBDB+l0+SMLYgQ3ZRzg2Pn5u+1ZwKbEnJzXsTKTJM9QSJbKkbA7uCnjdS +CvEW+66CoHqgBwYFK4EEACKhZANiAAT6awCCIrcCr9H4wq0bJ/rQou3tpLHyyf33 +c8D6FPn48/hNqinpx7b0le/0D+Rrhdl9edIplAf6oki7yoFFGl4yuzWtv7rag9jB +vv6w1508ChOmyQ094rFt/xj4KVBhEHI= +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_algorithms_SUITE_data/ssh_host_ecdsa_key384.pub b/lib/ssh/test/ssh_algorithms_SUITE_data/ssh_host_ecdsa_key384.pub new file mode 100644 index 0000000000..714fc4eb89 --- /dev/null +++ b/lib/ssh/test/ssh_algorithms_SUITE_data/ssh_host_ecdsa_key384.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPprAIIitwKv0fjCrRsn+tCi7e2ksfLJ/fdzwPoU+fjz+E2qKenHtvSV7/QP5GuF2X150imUB/qiSLvKgUUaXjK7Na2/utqD2MG+/rDXnTwKE6bJDT3isW3/GPgpUGEQcg== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_algorithms_SUITE_data/ssh_host_ecdsa_key521 b/lib/ssh/test/ssh_algorithms_SUITE_data/ssh_host_ecdsa_key521 new file mode 100644 index 0000000000..81aa8df39f --- /dev/null +++ b/lib/ssh/test/ssh_algorithms_SUITE_data/ssh_host_ecdsa_key521 @@ -0,0 +1,7 @@ +-----BEGIN EC PRIVATE KEY----- +MIHbAgEBBEHHxgYEfDclsu5bW+pZfg+bkaqWpgEpXtuzLVm++FFPjhAPhMkurSRj +WQ+CuI2TxgYkBbYFNjn9JqgdMF7FzaiojKAHBgUrgQQAI6GBiQOBhgAEAFTM8TKG +xexxmfAGuyl/Tpk4wytB/OyuVfkF+Q3H1v17HLcpMacA5xUFr80+D5XnjxGttBsS ++X0uexR7QbPbhhPqADgQzFqvTsB1mUNAZnJBD6QNCZkfWwRRwFYQWSmisb43H6G3 +iUTKqiCXMXO8drKLA+Wi+L7VyfoI1CvatBBlDHbV +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_algorithms_SUITE_data/ssh_host_ecdsa_key521.pub b/lib/ssh/test/ssh_algorithms_SUITE_data/ssh_host_ecdsa_key521.pub new file mode 100644 index 0000000000..17b9a1d834 --- /dev/null +++ b/lib/ssh/test/ssh_algorithms_SUITE_data/ssh_host_ecdsa_key521.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBABUzPEyhsXscZnwBrspf06ZOMMrQfzsrlX5BfkNx9b9exy3KTGnAOcVBa/NPg+V548RrbQbEvl9LnsUe0Gz24YT6gA4EMxar07AdZlDQGZyQQ+kDQmZH1sEUcBWEFkporG+Nx+ht4lEyqoglzFzvHayiwPlovi+1cn6CNQr2rQQZQx21Q== uabhnil@elxadlj3q32 diff --git a/lib/ssl/src/dtls_connection.erl b/lib/ssl/src/dtls_connection.erl index f078b87bce..2de947d8b4 100644 --- a/lib/ssl/src/dtls_connection.erl +++ b/lib/ssl/src/dtls_connection.erl @@ -750,31 +750,58 @@ next_event(connection = StateName, no_record, {#ssl_tls{epoch = Epoch, type = ?HANDSHAKE, version = _Version}, State1} = _Record when Epoch == CurrentEpoch-1 -> - {State, MoreActions} = send_handshake_flight(State1, Epoch), + {State, MoreActions} = send_handshake_flight(State1, CurrentEpoch), + {next_state, StateName, State, Actions ++ MoreActions}; + %% From FLIGHT perspective CHANGE_CIPHER_SPEC is treated as a handshake + {#ssl_tls{epoch = Epoch, + type = ?CHANGE_CIPHER_SPEC, + version = _Version}, State1} = _Record when Epoch == CurrentEpoch-1 -> + {State, MoreActions} = send_handshake_flight(State1, CurrentEpoch), {next_state, StateName, State, Actions ++ MoreActions}; {#ssl_tls{epoch = _Epoch, - version = _Version}, State} -> + version = _Version}, State1} -> %% TODO maybe buffer later epoch - {next_state, StateName, State, Actions}; + {Record, State} = next_record(State1), + next_event(StateName, Record, State, Actions); {#alert{} = Alert, State} -> {next_state, StateName, State, [{next_event, internal, Alert} | Actions]} end; +next_event(connection = StateName, Record, + #state{connection_states = #{current_read := #{epoch := CurrentEpoch}}} = State0, Actions) -> + case Record of + #ssl_tls{epoch = CurrentEpoch} -> + {next_state, StateName, State0, [{next_event, internal, {protocol_record, Record}} | Actions]}; + #ssl_tls{epoch = Epoch, + type = ?HANDSHAKE, + version = _Version} when Epoch == CurrentEpoch-1 -> + {State, MoreActions} = send_handshake_flight(State0, CurrentEpoch), + {next_state, StateName, State, Actions ++ MoreActions}; + %% From FLIGHT perspective CHANGE_CIPHER_SPEC is treated as a handshake + #ssl_tls{epoch = Epoch, + type = ?CHANGE_CIPHER_SPEC, + version = _Version} when Epoch == CurrentEpoch-1 -> + {State, MoreActions} = send_handshake_flight(State0, CurrentEpoch), + {next_state, StateName, State, Actions ++ MoreActions}; + _ -> + next_event(StateName, no_record, State0, Actions) + end; next_event(StateName, Record, - #state{connection_states = #{current_read := #{epoch := CurrentEpoch}}} = State, Actions) -> + #state{connection_states = #{current_read := #{epoch := CurrentEpoch}}} = State0, Actions) -> case Record of no_record -> - {next_state, StateName, State, Actions}; + {next_state, StateName, State0, Actions}; #ssl_tls{epoch = CurrentEpoch, version = Version} = Record -> {next_state, StateName, - dtls_version(StateName, Version, State), + dtls_version(StateName, Version, State0), [{next_event, internal, {protocol_record, Record}} | Actions]}; #ssl_tls{epoch = _Epoch, version = _Version} = _Record -> %% TODO maybe buffer later epoch - {next_state, StateName, State, Actions}; + {Record, State} = next_record(State0), + next_event(StateName, Record, State, Actions); #alert{} = Alert -> - {next_state, StateName, State, [{next_event, internal, Alert} | Actions]} + {next_state, StateName, State0, [{next_event, internal, Alert} | Actions]} end. decode_cipher_text(#state{protocol_buffers = #protocol_buffers{dtls_cipher_texts = [ CT | Rest]} = Buffers, diff --git a/lib/ssl/src/dtls_udp_listener.erl b/lib/ssl/src/dtls_udp_listener.erl index f0ace2d887..29380e3008 100644 --- a/lib/ssl/src/dtls_udp_listener.erl +++ b/lib/ssl/src/dtls_udp_listener.erl @@ -121,6 +121,18 @@ handle_info({udp, Socket, IP, InPortNo, _} = Msg, #state{listner = Socket} = Sta next_datagram(Socket), {noreply, State}; +%% UDP socket does not have a connection and should not receive an econnreset +%% This does however happens on on some windows versions. Just ignoring it +%% appears to make things work as expected! +handle_info({udp_error, Socket, econnreset = Error}, #state{listner = Socket} = State) -> + Report = io_lib:format("Ignore SSL UDP Listener: Socket error: ~p ~n", [Error]), + error_logger:info_report(Report), + {noreply, State}; +handle_info({udp_error, Socket, Error}, #state{listner = Socket} = State) -> + Report = io_lib:format("SSL UDP Listener shutdown: Socket error: ~p ~n", [Error]), + error_logger:info_report(Report), + {noreply, State#state{close=true}}; + handle_info({'DOWN', _, process, Pid, _}, #state{clients = Clients, dtls_processes = Processes0, close = ListenClosed} = State) -> diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index 58db8449d6..3cf466e78f 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -2295,6 +2295,8 @@ is_acceptable_hash_sign({_, dsa} = Algos, dsa, _, srp_dss, SupportedHashSigns) - is_acceptable_hash_sign(Algos, SupportedHashSigns); is_acceptable_hash_sign({_, ecdsa} = Algos, ecdsa, _, dhe_ecdsa, SupportedHashSigns) -> is_acceptable_hash_sign(Algos, SupportedHashSigns); +is_acceptable_hash_sign({_, ecdsa} = Algos, ecdsa, ecdsa, ecdh_ecdsa, SupportedHashSigns) -> + is_acceptable_hash_sign(Algos, SupportedHashSigns); is_acceptable_hash_sign({_, ecdsa} = Algos, ecdsa, ecdsa, ecdhe_ecdsa, SupportedHashSigns) -> is_acceptable_hash_sign(Algos, SupportedHashSigns); is_acceptable_hash_sign(_, _, _, KeyExAlgo, _) when diff --git a/lib/ssl/src/tls_connection.erl b/lib/ssl/src/tls_connection.erl index 96c3ab86e9..352874c77d 100644 --- a/lib/ssl/src/tls_connection.erl +++ b/lib/ssl/src/tls_connection.erl @@ -600,8 +600,12 @@ next_record(#state{protocol_buffers = next_record(#state{protocol_buffers = #protocol_buffers{tls_packets = [], tls_cipher_texts = []}, socket = Socket, transport_cb = Transport} = State) -> - tls_socket:setopts(Transport, Socket, [{active,once}]), - {no_record, State}; + case tls_socket:setopts(Transport, Socket, [{active,once}]) of + ok -> + {no_record, State}; + _ -> + {socket_closed, State} + end; next_record(State) -> {no_record, State}. @@ -626,10 +630,15 @@ passive_receive(State0 = #state{user_data_buffer = Buffer}, StateName) -> next_event(StateName, Record, State) -> next_event(StateName, Record, State, []). +next_event(StateName, socket_closed, State, _) -> + ssl_connection:handle_normal_shutdown(?ALERT_REC(?FATAL, ?CLOSE_NOTIFY), StateName, State), + {stop, {shutdown, transport_closed}, State}; next_event(connection = StateName, no_record, State0, Actions) -> case next_record_if_active(State0) of {no_record, State} -> ssl_connection:hibernate_after(StateName, State, Actions); + {socket_closed, State} -> + next_event(StateName, socket_closed, State, Actions); {#ssl_tls{} = Record, State} -> {next_state, StateName, State, [{next_event, internal, {protocol_record, Record}} | Actions]}; {#alert{} = Alert, State} -> diff --git a/lib/ssl/test/ssl_ECC_SUITE.erl b/lib/ssl/test/ssl_ECC_SUITE.erl index 1149513417..0fbb0bb79a 100644 --- a/lib/ssl/test/ssl_ECC_SUITE.erl +++ b/lib/ssl/test/ssl_ECC_SUITE.erl @@ -44,27 +44,74 @@ groups() -> {'tlsv1.2', [], all_versions_groups()}, {'tlsv1.1', [], all_versions_groups()}, {'tlsv1', [], all_versions_groups()}, - {'erlang_server', [], key_cert_combinations()}, - {'erlang_client', [], key_cert_combinations()}, + {'erlang_server', [], openssl_key_cert_combinations()}, + %%{'erlang_client', [], openssl_key_cert_combinations()}, {'erlang', [], key_cert_combinations() ++ misc() ++ ecc_negotiation()} ]. all_versions_groups ()-> [{group, 'erlang_server'}, - {group, 'erlang_client'}, + %%{group, 'erlang_client'}, {group, 'erlang'} ]. + +openssl_key_cert_combinations() -> + ECDH_RSA = case ssl_test_lib:openssl_filter("ECDH-RSA") of + [] -> + []; + _ -> + server_ecdh_rsa() + end, + + ECDHE_RSA = case ssl_test_lib:openssl_filter("ECDHE-RSA") of + [] -> + []; + _ -> + server_ecdhe_rsa() + end, + ECDH_ECDSA = case ssl_test_lib:openssl_filter("ECDH-ECDSA") of + [] -> + []; + _ -> + server_ecdhe_ecdsa() + end, + + ECDHE_ECDSA = case ssl_test_lib:openssl_filter("ECDHE-ECDSA") of + [] -> + []; + _ -> + server_ecdhe_ecdsa() + end, + ECDH_RSA ++ ECDHE_RSA ++ ECDH_ECDSA ++ ECDHE_ECDSA. + key_cert_combinations() -> + server_ecdh_rsa() ++ + server_ecdhe_rsa() ++ + server_ecdh_ecdsa() ++ + server_ecdhe_ecdsa(). + +server_ecdh_rsa() -> [client_ecdh_rsa_server_ecdh_rsa, - client_ecdhe_rsa_server_ecdh_rsa, - client_ecdh_rsa_server_ecdhe_rsa, + client_ecdhe_rsa_server_ecdh_rsa, + client_ecdhe_ecdsa_server_ecdh_rsa]. + +server_ecdhe_rsa() -> + [client_ecdh_rsa_server_ecdhe_rsa, client_ecdhe_rsa_server_ecdhe_rsa, - client_ecdhe_ecdsa_server_ecdhe_rsa, - client_ecdhe_ecdsa_server_ecdhe_ecdsa, - client_ecdh_rsa_server_ecdhe_ecdsa - ]. + client_ecdhe_ecdsa_server_ecdhe_rsa]. + +server_ecdh_ecdsa() -> + [client_ecdh_ecdsa_server_ecdh_ecdsa, + client_ecdhe_rsa_server_ecdh_ecdsa, + client_ecdhe_ecdsa_server_ecdh_ecdsa]. + +server_ecdhe_ecdsa() -> + [client_ecdh_rsa_server_ecdhe_ecdsa, + client_ecdh_ecdsa_server_ecdhe_ecdsa, + client_ecdhe_ecdsa_server_ecdhe_ecdsa]. + misc()-> [client_ecdsa_server_ecdsa_with_raw_key]. @@ -175,37 +222,63 @@ end_per_testcase(_TestCase, Config) -> %% ECDH_RSA client_ecdh_rsa_server_ecdh_rsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdh_rsa, ecdh_rsa, Config), - basic_test(COpts, SOpts, Config). - + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], + ecdh_rsa, ecdh_rsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdh_rsa} | proplists:delete(check_keyex, Config)]). client_ecdhe_rsa_server_ecdh_rsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_rsa, ecdh_rsa, Config), - basic_test(COpts, SOpts, Config). - + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdhe_rsa, ecdh_rsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdh_rsa} | proplists:delete(check_keyex, Config)]). +client_ecdhe_ecdsa_server_ecdh_rsa(Config) when is_list(Config) -> + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdh_rsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdh_rsa} | proplists:delete(check_keyex, Config)]). + %% ECDHE_RSA client_ecdh_rsa_server_ecdhe_rsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdh_rsa, ecdhe_rsa, Config), - basic_test(COpts, SOpts, Config). - + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdh_rsa, ecdhe_rsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdhe_rsa} | proplists:delete(check_keyex, Config)]). client_ecdhe_rsa_server_ecdhe_rsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_rsa, ecdhe_rsa, Config), - basic_test(COpts, SOpts, Config). - + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdhe_rsa, ecdhe_rsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdhe_rsa} | proplists:delete(check_keyex, Config)]). client_ecdhe_ecdsa_server_ecdhe_rsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdh_ecdsa, ecdhe_rsa, Config), - basic_test(COpts, SOpts, Config). + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdh_ecdsa, ecdhe_rsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdhe_rsa} | proplists:delete(check_keyex, Config)]). -%% ECDHE_ECDSA -client_ecdhe_ecdsa_server_ecdhe_ecdsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), - basic_test(COpts, SOpts, Config). +%% ECDH_ECDSA +client_ecdh_ecdsa_server_ecdh_ecdsa(Config) when is_list(Config) -> + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([{server_peer_opts, + [{extensions, [{key_usage, [keyEncipherment] + }]}]}], + ecdh_ecdsa, ecdh_ecdsa, Config), + basic_test(COpts, SOpts, + [{check_keyex, ecdh_ecdsa} | proplists:delete(check_keyex, Config)]). +client_ecdhe_rsa_server_ecdh_ecdsa(Config) when is_list(Config) -> + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([{server_peer_opts, + [{extensions, [{key_usage, [keyEncipherment] + }]}]}], + ecdhe_rsa, ecdh_ecdsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdh_ecdsa} | proplists:delete(check_keyex, Config)]). + +client_ecdhe_ecdsa_server_ecdh_ecdsa(Config) when is_list(Config) -> + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([{server_peer_opts, + [{extensions, [{key_usage, [keyEncipherment] + }]}]}], + ecdhe_ecdsa, ecdh_ecdsa, Config), + basic_test(COpts, SOpts, + [{check_keyex, ecdh_ecdsa} | proplists:delete(check_keyex, Config)]). +%% ECDHE_ECDSA client_ecdh_rsa_server_ecdhe_ecdsa(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdh_rsa, ecdhe_ecdsa, Config), - basic_test(COpts, SOpts, Config). + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdh_rsa, ecdhe_ecdsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdhe_ecdsa} | proplists:delete(check_keyex, Config)]). +client_ecdh_ecdsa_server_ecdhe_ecdsa(Config) when is_list(Config) -> + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdh_ecdsa, ecdhe_ecdsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdhe_ecdsa} | proplists:delete(check_keyex, Config)]). +client_ecdhe_ecdsa_server_ecdhe_ecdsa(Config) when is_list(Config) -> + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdhe_ecdsa, ecdhe_ecdsa, Config), + basic_test(COpts, SOpts, [{check_keyex, ecdhe_ecdsa} | proplists:delete(check_keyex, Config)]). client_ecdsa_server_ecdsa_with_raw_key(Config) when is_list(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdhe_ecdsa, ecdhe_ecdsa, Config), ServerKeyFile = proplists:get_value(keyfile, SOpts), {ok, PemBin} = file:read_file(ServerKeyFile), PemEntries = public_key:pem_decode(PemBin), @@ -221,7 +294,7 @@ client_ecdsa_server_ecdsa_with_raw_key(Config) when is_list(Config) -> close(Server, Client). ecc_default_order(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [], case supported_eccs([{eccs, [sect571r1]}]) of true -> ecc_test(sect571r1, COpts, SOpts, [], ECCOpts, Config); @@ -229,7 +302,7 @@ ecc_default_order(Config) -> end. ecc_default_order_custom_curves(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [{eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(sect571r1, COpts, SOpts, [], ECCOpts, Config); @@ -237,7 +310,7 @@ ecc_default_order_custom_curves(Config) -> end. ecc_client_order(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [{honor_ecc_order, false}], case supported_eccs([{eccs, [sect571r1]}]) of true -> ecc_test(sect571r1, COpts, SOpts, [], ECCOpts, Config); @@ -245,7 +318,7 @@ ecc_client_order(Config) -> end. ecc_client_order_custom_curves(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [{honor_ecc_order, false}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(sect571r1, COpts, SOpts, [], ECCOpts, Config); @@ -253,12 +326,12 @@ ecc_client_order_custom_curves(Config) -> end. ecc_unknown_curve(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [{eccs, ['123_fake_curve']}], ecc_test_error(COpts, SOpts, [], ECCOpts, Config). client_ecdh_rsa_server_ecdhe_ecdsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdh_rsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdh_rsa, ecdhe_ecdsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); @@ -266,7 +339,7 @@ client_ecdh_rsa_server_ecdhe_ecdsa_server_custom(Config) -> end. client_ecdh_rsa_server_ecdhe_rsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdh_rsa, ecdhe_rsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdh_rsa, ecdhe_rsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(undefined, COpts, SOpts, [], ECCOpts, Config); @@ -274,7 +347,7 @@ client_ecdh_rsa_server_ecdhe_rsa_server_custom(Config) -> end. client_ecdhe_rsa_server_ecdhe_ecdsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_rsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_rsa, ecdhe_ecdsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); @@ -282,14 +355,16 @@ client_ecdhe_rsa_server_ecdhe_ecdsa_server_custom(Config) -> end. client_ecdhe_rsa_server_ecdhe_rsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_rsa, ecdhe_rsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdhe_rsa, ecdhe_rsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(undefined, COpts, SOpts, [], ECCOpts, Config); false -> {skip, "unsupported named curves"} end. client_ecdhe_rsa_server_ecdh_rsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_rsa, ecdh_rsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([{server_peer_opts, + [{extensions, [{key_usage, [keyEncipherment] + }]}]}], ecdhe_rsa, ecdh_rsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(undefined, COpts, SOpts, [], ECCOpts, Config); @@ -297,7 +372,7 @@ client_ecdhe_rsa_server_ecdh_rsa_server_custom(Config) -> end. client_ecdhe_ecdsa_server_ecdhe_ecdsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([], ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(secp256r1, COpts, SOpts, [], ECCOpts, Config); @@ -305,7 +380,7 @@ client_ecdhe_ecdsa_server_ecdhe_ecdsa_server_custom(Config) -> end. client_ecdhe_ecdsa_server_ecdhe_rsa_server_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_rsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_rsa, Config), ECCOpts = [{honor_ecc_order, true}, {eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(undefined, COpts, SOpts, [], ECCOpts, Config); @@ -313,7 +388,7 @@ client_ecdhe_ecdsa_server_ecdhe_rsa_server_custom(Config) -> end. client_ecdhe_ecdsa_server_ecdhe_ecdsa_client_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_ecdsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_ecdsa, ecdhe_ecdsa, Config), ECCOpts = [{eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(secp256r1, COpts, SOpts, ECCOpts, [], Config); @@ -321,7 +396,7 @@ client_ecdhe_ecdsa_server_ecdhe_ecdsa_client_custom(Config) -> end. client_ecdhe_rsa_server_ecdhe_ecdsa_client_custom(Config) -> - {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains(ecdhe_rsa, ecdhe_ecdsa, Config), + {COpts, SOpts} = ssl_test_lib:make_ec_cert_chains([],ecdhe_rsa, ecdhe_ecdsa, Config), ECCOpts = [{eccs, [secp256r1, sect571r1]}], case supported_eccs(ECCOpts) of true -> ecc_test(secp256r1, COpts, SOpts, ECCOpts, [], Config); @@ -370,10 +445,11 @@ start_client(openssl, Port, ClientOpts, _Config) -> start_client(erlang, Port, ClientOpts, Config) -> {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), + KeyEx = proplists:get_value(check_keyex, Config, false), ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, {host, Hostname}, {from, self()}, - {mfa, {ssl_test_lib, send_recv_result_active, []}}, + {mfa, {ssl_test_lib, check_key_exchange_send_active, [KeyEx]}}, {options, [{verify, verify_peer} | ClientOpts]}]). @@ -412,11 +488,12 @@ start_server(openssl, ServerOpts, _Config) -> {OpenSslPort, Port}; start_server(erlang, ServerOpts, Config) -> {_, ServerNode, _} = ssl_test_lib:run_where(Config), + KeyEx = proplists:get_value(check_keyex, Config, false), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, {from, self()}, {mfa, {ssl_test_lib, - send_recv_result_active, - []}}, + check_key_exchange_send_active, + [KeyEx]}}, {options, [{verify, verify_peer} | ServerOpts]}]), {Server, ssl_test_lib:inet_port(Server)}. diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index 58870a3419..d13b1b3f2a 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -240,6 +240,7 @@ error_handling_tests()-> error_handling_tests_tls()-> [controller_dies, tls_client_closes_socket, + tls_closed_in_active_once, tls_tcp_error_propagation_in_active_mode, tls_tcp_connect, tls_tcp_connect_big, @@ -430,6 +431,7 @@ init_per_testcase(prf, Config) -> init_per_testcase(TestCase, Config) when TestCase == tls_ssl_accept_timeout; TestCase == tls_client_closes_socket; + TestCase == tls_closed_in_active_once; TestCase == tls_downgrade -> ssl_test_lib:ct_log_supported_protocol_versions(Config), ct:timetrap({seconds, 15}), @@ -961,6 +963,48 @@ tls_client_closes_socket(Config) when is_list(Config) -> ssl_test_lib:check_result(Server, {error,closed}). %%-------------------------------------------------------------------- +tls_closed_in_active_once() -> + [{doc, "Test that ssl_closed is delivered in active once with non-empty buffer, check ERL-420."}]. + +tls_closed_in_active_once(Config) when is_list(Config) -> + ClientOpts = ssl_test_lib:ssl_options(client_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_opts, Config), + {_ClientNode, _ServerNode, Hostname} = ssl_test_lib:run_where(Config), + TcpOpts = [binary, {reuseaddr, true}], + Port = ssl_test_lib:inet_port(node()), + Server = fun() -> + {ok, Listen} = gen_tcp:listen(Port, TcpOpts), + {ok, TcpServerSocket} = gen_tcp:accept(Listen), + {ok, ServerSocket} = ssl:ssl_accept(TcpServerSocket, ServerOpts), + lists:foreach( + fun(_) -> + ssl:send(ServerSocket, "some random message\r\n") + end, lists:seq(1, 20)), + %% Close TCP instead of SSL socket to trigger the bug: + gen_tcp:close(TcpServerSocket), + gen_tcp:close(Listen) + end, + spawn_link(Server), + {ok, Socket} = ssl:connect(Hostname, Port, [{active, false} | ClientOpts]), + Result = tls_closed_in_active_once_loop(Socket), + ssl:close(Socket), + case Result of + ok -> ok; + _ -> ct:fail(Result) + end. + +tls_closed_in_active_once_loop(Socket) -> + ssl:setopts(Socket, [{active, once}]), + receive + {ssl, Socket, _} -> + tls_closed_in_active_once_loop(Socket); + {ssl_closed, Socket} -> + ok + after 5000 -> + no_ssl_closed_received + end. + +%%-------------------------------------------------------------------- connect_dist() -> [{doc,"Test a simple connect as is used by distribution"}]. diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index 68a99c28dd..77c21d9b57 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -498,12 +498,12 @@ make_rsa_cert_chains(ChainConf, Config, Suffix) -> [{reuseaddr, true}, {verify, verify_peer} | ServerConf] }. -make_ec_cert_chains(ClientChainType, ServerChainType, Config) -> +make_ec_cert_chains(ChainConf, ClientChainType, ServerChainType, Config) -> CryptoSupport = crypto:supports(), KeyGenSpec = key_gen_info(ClientChainType, ServerChainType), ClientFileBase = filename:join([proplists:get_value(priv_dir, Config), atom_to_list(ClientChainType)]), ServerFileBase = filename:join([proplists:get_value(priv_dir, Config), atom_to_list(ServerChainType)]), - GenCertData = x509_test:gen_test_certs([{digest, appropriate_sha(CryptoSupport)} | KeyGenSpec]), + GenCertData = x509_test:gen_test_certs([{digest, appropriate_sha(CryptoSupport)} | KeyGenSpec] ++ ChainConf), [{server_config, ServerConf}, {client_config, ClientConf}] = x509_test:gen_pem_config_files(GenCertData, ClientFileBase, ServerFileBase), @@ -1009,6 +1009,12 @@ openssl_ecdh_rsa_suites() -> lists:filter(fun(Str) -> string_regex_filter(Str, "ECDH-RSA") end, Ciphers). +openssl_filter(FilterStr) -> + Ciphers = string:tokens(os:cmd("openssl ciphers"), ":"), + lists:filter(fun(Str) -> string_regex_filter(Str, FilterStr) + end, Ciphers). + + string_regex_filter(Str, Search) when is_list(Str) -> case re:run(Str, Search, []) of nomatch -> @@ -1174,6 +1180,21 @@ sufficient_crypto_support(Group) when Group == ciphers_ec; %% From ssl_basic sufficient_crypto_support(_) -> true. +check_key_exchange_send_active(Socket, false) -> + send_recv_result_active(Socket); +check_key_exchange_send_active(Socket, KeyEx) -> + {ok, [{cipher_suite, Suite}]} = ssl:connection_information(Socket, [cipher_suite]), + true = check_key_exchange(Suite, KeyEx), + send_recv_result_active(Socket). + +check_key_exchange({KeyEx,_, _}, KeyEx) -> + true; +check_key_exchange({KeyEx,_,_,_}, KeyEx) -> + true; +check_key_exchange(KeyEx1, KeyEx2) -> + ct:pal("Negotiated ~p Expected ~p", [KeyEx1, KeyEx2]), + false. + send_recv_result_active(Socket) -> ssl:send(Socket, "Hello world"), receive diff --git a/lib/stdlib/doc/src/gen_statem.xml b/lib/stdlib/doc/src/gen_statem.xml index 1aac88c308..ad7f2f2e95 100644 --- a/lib/stdlib/doc/src/gen_statem.xml +++ b/lib/stdlib/doc/src/gen_statem.xml @@ -346,7 +346,7 @@ ok <p> To compare styles, here follows the same example using <seealso marker="#type-callback_mode"><em>callback mode</em></seealso> - <c>state_functions</c>, or rather the code to replace + <c>handle_event_function</c>, or rather the code to replace after function <c>init/1</c> of the <c>pushbutton.erl</c> example file above: </p> diff --git a/lib/stdlib/doc/src/string.xml b/lib/stdlib/doc/src/string.xml index 343904a49a..9d5edd9ecf 100644 --- a/lib/stdlib/doc/src/string.xml +++ b/lib/stdlib/doc/src/string.xml @@ -311,7 +311,9 @@ true</pre> <desc> <p> Returns the first codepoint in <c><anno>String</anno></c> - and the rest of <c><anno>String</anno></c> in the tail. + and the rest of <c><anno>String</anno></c> in the tail. Returns + an empty list if <c><anno>String</anno></c> is empty or an + <c>{error, String}</c> tuple if the next byte is invalid. </p> <p><em>Example:</em></p> <pre> @@ -326,7 +328,9 @@ true</pre> <desc> <p> Returns the first grapheme cluster in <c><anno>String</anno></c> - and the rest of <c><anno>String</anno></c> in the tail. + and the rest of <c><anno>String</anno></c> in the tail. Returns + an empty list if <c><anno>String</anno></c> is empty or an + <c>{error, String}</c> tuple if the next byte is invalid. </p> <p><em>Example:</em></p> <pre> diff --git a/lib/stdlib/src/erl_parse.yrl b/lib/stdlib/src/erl_parse.yrl index 2dcddeb8c2..733932e711 100644 --- a/lib/stdlib/src/erl_parse.yrl +++ b/lib/stdlib/src/erl_parse.yrl @@ -1052,6 +1052,9 @@ build_typed_attribute({atom,Aa,record}, build_typed_attribute({atom,Aa,Attr}, {type_def, {call,_,{atom,_,TypeName},Args}, Type}) when Attr =:= 'type' ; Attr =:= 'opaque' -> + lists:foreach(fun({var, A, '_'}) -> ret_err(A, "bad type variable"); + (_) -> ok + end, Args), case lists:all(fun({var, _, _}) -> true; (_) -> false end, Args) of diff --git a/lib/stdlib/src/erl_tar.erl b/lib/stdlib/src/erl_tar.erl index 168ea4002c..76f0b38108 100644 --- a/lib/stdlib/src/erl_tar.erl +++ b/lib/stdlib/src/erl_tar.erl @@ -176,7 +176,7 @@ check_extract(Name, #read_opts{files=Files}) -> -type tar_entry() :: {filename(), typeflag(), non_neg_integer(), - calendar:datetime(), + tar_time(), mode(), uid(), gid()}. @@ -274,8 +274,13 @@ mode_to_string(Mode, [_|T], Acc) -> mode_to_string(_, [], Acc) -> Acc. -%% Converts a datetime tuple to a readable string -time_to_string({{Y, Mon, Day}, {H, Min, _}}) -> +%% Converts a tar_time() (POSIX time) to a readable string +time_to_string(Secs0) -> + Epoch = calendar:datetime_to_gregorian_seconds(?EPOCH), + Secs = Epoch + Secs0, + DateTime0 = calendar:gregorian_seconds_to_datetime(Secs), + DateTime = calendar:universal_time_to_local_time(DateTime0), + {{Y, Mon, Day}, {H, Min, _}} = DateTime, io_lib:format("~s ~2w ~s:~s ~w", [month(Mon), Day, two_d(H), two_d(Min), Y]). two_d(N) -> @@ -452,7 +457,8 @@ add(Reader, NameOrBin, NameInArchive, Options) do_add(#reader{access=write}=Reader, Name, NameInArchive, Options) when is_list(NameInArchive), is_list(Options) -> - Opts = #add_opts{read_info=fun(F) -> file:read_link_info(F) end}, + RF = fun(F) -> file:read_link_info(F, [{time, posix}]) end, + Opts = #add_opts{read_info=RF}, add1(Reader, Name, NameInArchive, add_opts(Options, Opts)); do_add(#reader{access=read},_,_,_) -> {error, eacces}; @@ -460,7 +466,8 @@ do_add(Reader,_,_,_) -> {error, {badarg, Reader}}. add_opts([dereference|T], Opts) -> - add_opts(T, Opts#add_opts{read_info=fun(F) -> file:read_file_info(F) end}); + RF = fun(F) -> file:read_file_info(F, [{time, posix}]) end, + add_opts(T, Opts#add_opts{read_info=RF}); add_opts([verbose|T], Opts) -> add_opts(T, Opts#add_opts{verbose=true}); add_opts([{chunks,N}|T], Opts) -> @@ -503,7 +510,7 @@ add1(#reader{}=Reader, Name, NameInArchive, #add_opts{read_info=ReadInfo}=Opts) end; add1(Reader, Bin, NameInArchive, Opts) when is_binary(Bin) -> add_verbose(Opts, "a ~ts~n", [NameInArchive]), - Now = calendar:now_to_local_time(erlang:timestamp()), + Now = os:system_time(seconds), Header = #tar_header{ name = NameInArchive, size = byte_size(Bin), @@ -612,7 +619,7 @@ build_header(#tar_header{}=Header, Opts) -> devmajor=Devmaj, devminor=Devmin } = Header, - Mtime = datetime_to_posix(Header#tar_header.mtime), + Mtime = Header#tar_header.mtime, Block0 = ?ZERO_BLOCK, {Block1, Pax0} = write_string(Block0, ?V7_NAME, ?V7_NAME_LEN, Name, ?PAX_PATH, #{}), @@ -770,14 +777,6 @@ join_split_ustar_path([Part|Rest], {ok, Name, nil}) -> join_split_ustar_path([Part|Rest], {ok, Name, Acc}) -> join_split_ustar_path(Rest, {ok, Name, <<Acc/binary,$/,Part/binary>>}). -datetime_to_posix(DateTime) -> - Epoch = calendar:datetime_to_gregorian_seconds(?EPOCH), - Secs = calendar:datetime_to_gregorian_seconds(DateTime), - case Secs - Epoch of - N when N < 0 -> 0; - N -> N - end. - write_octal(Block, Pos, Size, X) -> Octal = zero_pad(format_octal(X), Size-1), if byte_size(Octal) < Size -> @@ -984,7 +983,7 @@ do_get_format(#header_v7{}=V7, Bin) unpack_format(Format, #header_v7{}=V7, Bin, Reader) when is_binary(Bin), byte_size(Bin) =:= ?BLOCK_SIZE -> - Mtime = posix_to_erlang_time(parse_numeric(V7#header_v7.mtime)), + Mtime = parse_numeric(V7#header_v7.mtime), Header0 = #tar_header{ name=parse_string(V7#header_v7.name), mode=parse_numeric(V7#header_v7.mode), @@ -1051,9 +1050,9 @@ unpack_modern(Format, #header_v7{}=V7, Bin, #tar_header{}=Header0) Star = to_star(V7, Bin), Prefix0 = parse_string(Star#header_star.prefix), Atime0 = Star#header_star.atime, - Atime = posix_to_erlang_time(parse_numeric(Atime0)), + Atime = parse_numeric(Atime0), Ctime0 = Star#header_star.ctime, - Ctime = posix_to_erlang_time(parse_numeric(Ctime0)), + Ctime = parse_numeric(Ctime0), {Prefix0, H1#tar_header{ atime=Atime, ctime=Ctime @@ -1313,11 +1312,6 @@ is_header_only_type(?TYPE_LINK) -> true; is_header_only_type(?TYPE_DIR) -> true; is_header_only_type(_) -> false. -posix_to_erlang_time(Sec) -> - OneMillion = 1000000, - Time = calendar:now_to_datetime({Sec div OneMillion, Sec rem OneMillion, 0}), - erlang:universaltime_to_localtime(Time). - foldl_read(#reader{access=read}=Reader, Fun, Accu, #read_opts{}=Opts) when is_function(Fun,4) -> case foldl_read0(Reader, Fun, Accu, Opts) of @@ -1423,7 +1417,7 @@ do_merge_pax(Header, [_Ignore|Rest]) -> do_merge_pax(Header, Rest). %% Returns the time since UNIX epoch as a datetime --spec parse_pax_time(binary()) -> calendar:datetime(). +-spec parse_pax_time(binary()) -> tar_time(). parse_pax_time(Bin) when is_binary(Bin) -> TotalNano = case binary:split(Bin, [<<$.>>]) of [SecondsStr, NanoStr0] -> @@ -1450,8 +1444,7 @@ parse_pax_time(Bin) when is_binary(Bin) -> Micro = TotalNano div 1000, Mega = Micro div 1000000000000, Secs = Micro div 1000000 - (Mega*1000000), - Micro2 = Micro rem 1000000, - calendar:now_to_datetime({Mega, Secs, Micro2}). + Secs. %% Given a regular file reader, reads the whole file and %% parses all extended attributes it contains. @@ -1671,7 +1664,7 @@ set_extracted_file_info(Name, #tar_header{typeflag = ?TYPE_BLOCK}=Header) -> set_device_info(Name, Header); set_extracted_file_info(Name, #tar_header{mtime=Mtime,mode=Mode}) -> Info = #file_info{mode=Mode, mtime=Mtime}, - file:write_file_info(Name, Info). + file:write_file_info(Name, Info, [{time, posix}]). set_device_info(Name, #tar_header{}=Header) -> Mtime = Header#tar_header.mtime, diff --git a/lib/stdlib/src/erl_tar.hrl b/lib/stdlib/src/erl_tar.hrl index d646d02989..cff0c2f500 100644 --- a/lib/stdlib/src/erl_tar.hrl +++ b/lib/stdlib/src/erl_tar.hrl @@ -55,6 +55,8 @@ {string(), binary()} | {string(), file:filename()}]. +-type tar_time() :: non_neg_integer(). + %% The tar header, once fully parsed. -record(tar_header, { name = "" :: string(), %% name of header file entry @@ -62,15 +64,15 @@ uid = 0 :: non_neg_integer(), %% user id of owner gid = 0 :: non_neg_integer(), %% group id of owner size = 0 :: non_neg_integer(), %% length in bytes - mtime :: calendar:datetime(), %% modified time + mtime :: tar_time(), %% modified time typeflag :: char(), %% type of header entry linkname = "" :: string(), %% target name of link uname = "" :: string(), %% user name of owner gname = "" :: string(), %% group name of owner devmajor = 0 :: non_neg_integer(), %% major number of character or block device devminor = 0 :: non_neg_integer(), %% minor number of character or block device - atime :: calendar:datetime(), %% access time - ctime :: calendar:datetime() %% status change time + atime :: tar_time(), %% access time + ctime :: tar_time() %% status change time }). -type tar_header() :: #tar_header{}. diff --git a/lib/stdlib/src/error_logger_file_h.erl b/lib/stdlib/src/error_logger_file_h.erl index 0b262de3ab..76f89841b9 100644 --- a/lib/stdlib/src/error_logger_file_h.erl +++ b/lib/stdlib/src/error_logger_file_h.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2017. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -57,7 +57,7 @@ init(File, PrevHandler) -> process_flag(trap_exit, true), case file:open(File, [write]) of {ok,Fd} -> - Depth = get_depth(), + Depth = error_logger:get_format_depth(), State = #st{fd=Fd,filename=File,prev_handler=PrevHandler, depth=Depth}, {ok, State}; @@ -65,14 +65,6 @@ init(File, PrevHandler) -> Error end. -get_depth() -> - case application:get_env(kernel, error_logger_format_depth) of - {ok, Depth} when is_integer(Depth) -> - max(10, Depth); - undefined -> - unlimited - end. - handle_event({_Type, GL, _Msg}, State) when node(GL) =/= node() -> {ok, State}; handle_event(Event, State) -> diff --git a/lib/stdlib/src/error_logger_tty_h.erl b/lib/stdlib/src/error_logger_tty_h.erl index 2f2fd65252..8f0d7b0362 100644 --- a/lib/stdlib/src/error_logger_tty_h.erl +++ b/lib/stdlib/src/error_logger_tty_h.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2017. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -44,7 +44,7 @@ %% This one is used when we takeover from the simple error_logger. init({[], {error_logger, Buf}}) -> User = set_group_leader(), - Depth = get_depth(), + Depth = error_logger:get_format_depth(), State = #st{user=User,prev_handler=error_logger,depth=Depth}, write_events(State, Buf), {ok, State}; @@ -56,17 +56,9 @@ init({[], {error_logger_tty_h, PrevHandler}}) -> %% This one is used when we are started directly. init([]) -> User = set_group_leader(), - Depth = get_depth(), + Depth = error_logger:get_format_depth(), {ok, #st{user=User,prev_handler=[],depth=Depth}}. -get_depth() -> - case application:get_env(kernel, error_logger_format_depth) of - {ok, Depth} when is_integer(Depth) -> - max(10, Depth); - undefined -> - unlimited - end. - handle_event({_Type, GL, _Msg}, State) when node(GL) =/= node() -> {ok, State}; handle_event(Event, State) -> diff --git a/lib/stdlib/src/gen_server.erl b/lib/stdlib/src/gen_server.erl index ba0a7ae8e5..a3d53efd0d 100644 --- a/lib/stdlib/src/gen_server.erl +++ b/lib/stdlib/src/gen_server.erl @@ -107,7 +107,9 @@ %% Internal exports -export([init_it/6]). --import(error_logger, [format/2]). +-define( + STACKTRACE(), + try throw(ok) catch _ -> erlang:get_stacktrace() end). %%%========================================================================= %%% API @@ -326,15 +328,16 @@ init_it(Starter, self, Name, Mod, Args, Options) -> init_it(Starter, Parent, Name0, Mod, Args, Options) -> Name = gen:name(Name0), Debug = gen:debug_options(Name, Options), - HibernateAfterTimeout = gen:hibernate_after(Options), - case catch Mod:init(Args) of - {ok, State} -> + HibernateAfterTimeout = gen:hibernate_after(Options), + + case init_it(Mod, Args) of + {ok, {ok, State}} -> proc_lib:init_ack(Starter, {ok, self()}), loop(Parent, Name, State, Mod, infinity, HibernateAfterTimeout, Debug); - {ok, State, Timeout} -> + {ok, {ok, State, Timeout}} -> proc_lib:init_ack(Starter, {ok, self()}), loop(Parent, Name, State, Mod, Timeout, HibernateAfterTimeout, Debug); - {stop, Reason} -> + {ok, {stop, Reason}} -> %% For consistency, we must make sure that the %% registered name (if any) is unregistered before %% the parent process is notified about the failure. @@ -344,18 +347,25 @@ init_it(Starter, Parent, Name0, Mod, Args, Options) -> gen:unregister_name(Name0), proc_lib:init_ack(Starter, {error, Reason}), exit(Reason); - ignore -> + {ok, ignore} -> gen:unregister_name(Name0), proc_lib:init_ack(Starter, ignore), exit(normal); - {'EXIT', Reason} -> - gen:unregister_name(Name0), - proc_lib:init_ack(Starter, {error, Reason}), - exit(Reason); - Else -> + {ok, Else} -> Error = {bad_return_value, Else}, proc_lib:init_ack(Starter, {error, Error}), - exit(Error) + exit(Error); + {'EXIT', Class, Reason, Stacktrace} -> + gen:unregister_name(Name0), + proc_lib:init_ack(Starter, {error, terminate_reason(Class, Reason, Stacktrace)}), + erlang:raise(Class, Reason, Stacktrace) + end. +init_it(Mod, Args) -> + try + {ok, Mod:init(Args)} + catch + throw:R -> {ok, R}; + Class:R -> {'EXIT', Class, R, erlang:get_stacktrace()} end. %%%======================================================================== @@ -397,7 +407,7 @@ decode_msg(Msg, Parent, Name, State, Mod, Time, HibernateAfterTimeout, Debug, Hi sys:handle_system_msg(Req, From, Parent, ?MODULE, Debug, [Name, State, Mod, Time, HibernateAfterTimeout], Hib); {'EXIT', Parent, Reason} -> - terminate(Reason, Name, undefined, Msg, Mod, State, Debug); + terminate(Reason, ?STACKTRACE(), Name, undefined, Msg, Mod, State, Debug); _Msg when Debug =:= [] -> handle_msg(Msg, Parent, Name, State, Mod, HibernateAfterTimeout); _Msg -> @@ -589,17 +599,11 @@ start_monitor(Node, Name) when is_atom(Node), is_atom(Name) -> %% --------------------------------------------------- %% Helper functions for try-catch of callbacks. %% Returns the return value of the callback, or -%% {'EXIT', ExitReason, ReportReason} (if an exception occurs) -%% -%% ExitReason is the reason that shall be used when the process -%% terminates. +%% {'EXIT', Class, Reason, Stack} (if an exception occurs) %% -%% ReportReason is the reason that shall be printed in the error -%% report. -%% -%% These functions are introduced in order to add the stack trace in -%% the error report produced when a callback is terminated with -%% erlang:exit/1 (OTP-12263). +%% The Class, Reason and Stack are given to erlang:raise/3 +%% to make sure proc_lib receives the proper reasons and +%% stacktraces. %% --------------------------------------------------- try_dispatch({'$gen_cast', Msg}, Mod, State) -> @@ -621,15 +625,10 @@ try_dispatch(Mod, Func, Msg, State) -> [Mod, Msg]), {ok, {noreply, State}}; true -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', {R, Stacktrace}, {R, Stacktrace}} + {'EXIT', error, R, erlang:get_stacktrace()} end; - error:R -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', {R, Stacktrace}, {R, Stacktrace}}; - exit:R -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', R, {R, Stacktrace}} + Class:R -> + {'EXIT', Class, R, erlang:get_stacktrace()} end. try_handle_call(Mod, Msg, From, State) -> @@ -638,12 +637,8 @@ try_handle_call(Mod, Msg, From, State) -> catch throw:R -> {ok, R}; - error:R -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', {R, Stacktrace}, {R, Stacktrace}}; - exit:R -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', R, {R, Stacktrace}} + Class:R -> + {'EXIT', Class, R, erlang:get_stacktrace()} end. try_terminate(Mod, Reason, State) -> @@ -654,12 +649,8 @@ try_terminate(Mod, Reason, State) -> catch throw:R -> {ok, R}; - error:R -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', {R, Stacktrace}, {R, Stacktrace}}; - exit:R -> - Stacktrace = erlang:get_stacktrace(), - {'EXIT', R, {R, Stacktrace}} + Class:R -> + {'EXIT', Class, R, erlang:get_stacktrace()} end; false -> {ok, ok} @@ -684,10 +675,11 @@ handle_msg({'$gen_call', From, Msg}, Parent, Name, State, Mod, HibernateAfterTim {ok, {noreply, NState, Time1}} -> loop(Parent, Name, NState, Mod, Time1, HibernateAfterTimeout, []); {ok, {stop, Reason, Reply, NState}} -> - {'EXIT', R} = - (catch terminate(Reason, Name, From, Msg, Mod, NState, [])), - reply(From, Reply), - exit(R); + try + terminate(Reason, ?STACKTRACE(), Name, From, Msg, Mod, NState, []) + after + reply(From, Reply) + end; Other -> handle_common_reply(Other, Parent, Name, From, Msg, Mod, HibernateAfterTimeout, State) end; handle_msg(Msg, Parent, Name, State, Mod, HibernateAfterTimeout) -> @@ -712,10 +704,11 @@ handle_msg({'$gen_call', From, Msg}, Parent, Name, State, Mod, HibernateAfterTim {noreply, NState}), loop(Parent, Name, NState, Mod, Time1, HibernateAfterTimeout, Debug1); {ok, {stop, Reason, Reply, NState}} -> - {'EXIT', R} = - (catch terminate(Reason, Name, From, Msg, Mod, NState, Debug)), - _ = reply(Name, From, Reply, NState, Debug), - exit(R); + try + terminate(Reason, ?STACKTRACE(), Name, From, Msg, Mod, NState, Debug) + after + _ = reply(Name, From, Reply, NState, Debug) + end; Other -> handle_common_reply(Other, Parent, Name, From, Msg, Mod, HibernateAfterTimeout, State, Debug) end; @@ -730,11 +723,11 @@ handle_common_reply(Reply, Parent, Name, From, Msg, Mod, HibernateAfterTimeout, {ok, {noreply, NState, Time1}} -> loop(Parent, Name, NState, Mod, Time1, HibernateAfterTimeout, []); {ok, {stop, Reason, NState}} -> - terminate(Reason, Name, From, Msg, Mod, NState, []); - {'EXIT', ExitReason, ReportReason} -> - terminate(ExitReason, ReportReason, Name, From, Msg, Mod, State, []); + terminate(Reason, ?STACKTRACE(), Name, From, Msg, Mod, NState, []); + {'EXIT', Class, Reason, Stacktrace} -> + terminate(Class, Reason, Stacktrace, Name, From, Msg, Mod, State, []); {ok, BadReply} -> - terminate({bad_return_value, BadReply}, Name, From, Msg, Mod, State, []) + terminate({bad_return_value, BadReply}, ?STACKTRACE(), Name, From, Msg, Mod, State, []) end. handle_common_reply(Reply, Parent, Name, From, Msg, Mod, HibernateAfterTimeout, State, Debug) -> @@ -748,11 +741,11 @@ handle_common_reply(Reply, Parent, Name, From, Msg, Mod, HibernateAfterTimeout, {noreply, NState}), loop(Parent, Name, NState, Mod, Time1, HibernateAfterTimeout, Debug1); {ok, {stop, Reason, NState}} -> - terminate(Reason, Name, From, Msg, Mod, NState, Debug); - {'EXIT', ExitReason, ReportReason} -> - terminate(ExitReason, ReportReason, Name, From, Msg, Mod, State, Debug); + terminate(Reason, ?STACKTRACE(), Name, From, Msg, Mod, NState, Debug); + {'EXIT', Class, Reason, Stacktrace} -> + terminate(Class, Reason, Stacktrace, Name, From, Msg, Mod, State, Debug); {ok, BadReply} -> - terminate({bad_return_value, BadReply}, Name, From, Msg, Mod, State, Debug) + terminate({bad_return_value, BadReply}, ?STACKTRACE(), Name, From, Msg, Mod, State, Debug) end. reply(Name, {To, Tag}, Reply, State, Debug) -> @@ -770,7 +763,7 @@ system_continue(Parent, Debug, [Name, State, Mod, Time, HibernateAfterTimeout]) -spec system_terminate(_, _, _, [_]) -> no_return(). system_terminate(Reason, _Parent, Debug, [Name, State, Mod, _Time, _HibernateAfterTimeout]) -> - terminate(Reason, Name, undefined, [], Mod, State, Debug). + terminate(Reason, ?STACKTRACE(), Name, undefined, [], Mod, State, Debug). system_code_change([Name, State, Mod, Time, HibernateAfterTimeout], _Module, OldVsn, Extra) -> case catch Mod:code_change(OldVsn, State, Extra) of @@ -811,35 +804,58 @@ print_event(Dev, Event, Name) -> %%% --------------------------------------------------- %%% Terminate the server. +%%% +%%% terminate/8 is triggered by {stop, Reason} or bad +%%% return values. The stacktrace is generated via the +%%% ?STACKTRACE() macro and the ReportReason must not +%%% be wrapped in tuples. +%%% +%%% terminate/9 is triggered in case of error/exit in +%%% the user callback. In this case the report reason +%%% always includes the user stacktrace. +%%% +%%% The reason received in the terminate/2 callbacks +%%% always includes the stacktrace for errors and never +%%% for exits. %%% --------------------------------------------------- - --spec terminate(_, _, _, _, _, _, _) -> no_return(). -terminate(Reason, Name, From, Msg, Mod, State, Debug) -> - terminate(Reason, Reason, Name, From, Msg, Mod, State, Debug). -spec terminate(_, _, _, _, _, _, _, _) -> no_return(). -terminate(ExitReason, ReportReason, Name, From, Msg, Mod, State, Debug) -> - Reply = try_terminate(Mod, ExitReason, State), +terminate(Reason, Stacktrace, Name, From, Msg, Mod, State, Debug) -> + terminate(exit, Reason, Stacktrace, Reason, Name, From, Msg, Mod, State, Debug). + +-spec terminate(_, _, _, _, _, _, _, _, _) -> no_return(). +terminate(Class, Reason, Stacktrace, Name, From, Msg, Mod, State, Debug) -> + ReportReason = {Reason, Stacktrace}, + terminate(Class, Reason, Stacktrace, ReportReason, Name, From, Msg, Mod, State, Debug). + +-spec terminate(_, _, _, _, _, _, _, _, _, _) -> no_return(). +terminate(Class, Reason, Stacktrace, ReportReason, Name, From, Msg, Mod, State, Debug) -> + Reply = try_terminate(Mod, terminate_reason(Class, Reason, Stacktrace), State), case Reply of - {'EXIT', ExitReason1, ReportReason1} -> + {'EXIT', C, R, S} -> FmtState = format_status(terminate, Mod, get(), State), - error_info(ReportReason1, Name, From, Msg, FmtState, Debug), - exit(ExitReason1); + error_info({R, S}, Name, From, Msg, FmtState, Debug), + erlang:raise(C, R, S); _ -> - case ExitReason of - normal -> - exit(normal); - shutdown -> - exit(shutdown); - {shutdown,_}=Shutdown -> - exit(Shutdown); + case {Class, Reason} of + {exit, normal} -> ok; + {exit, shutdown} -> ok; + {exit, {shutdown,_}} -> ok; _ -> FmtState = format_status(terminate, Mod, get(), State), - error_info(ReportReason, Name, From, Msg, FmtState, Debug), - exit(ExitReason) + error_info(ReportReason, Name, From, Msg, FmtState, Debug) end + end, + case Stacktrace of + [] -> + erlang:Class(Reason); + _ -> + erlang:raise(Class, Reason, Stacktrace) end. +terminate_reason(error, Reason, Stacktrace) -> {Reason, Stacktrace}; +terminate_reason(exit, Reason, _Stacktrace) -> Reason. + error_info(_Reason, application_controller, _From, _Msg, _State, _Debug) -> %% OTP-5811 Don't send an error report if it's the system process %% application_controller which is terminating - let init take care @@ -861,14 +877,15 @@ error_info(Reason, Name, From, Msg, State, Debug) -> end end; _ -> - Reason + error_logger:limit_term(Reason) end, {ClientFmt, ClientArgs} = client_stacktrace(From), - format("** Generic server ~p terminating \n" - "** Last message in was ~p~n" - "** When Server state == ~p~n" - "** Reason for termination == ~n** ~p~n" ++ ClientFmt, - [Name, Msg, State, Reason1] ++ ClientArgs), + LimitedState = error_logger:limit_term(State), + error_logger:format("** Generic server ~p terminating \n" + "** Last message in was ~p~n" + "** When Server state == ~p~n" + "** Reason for termination == ~n** ~p~n" ++ ClientFmt, + [Name, Msg, LimitedState, Reason1] ++ ClientArgs), sys:print_log(Debug), ok. client_stacktrace(undefined) -> diff --git a/lib/stdlib/src/gen_statem.erl b/lib/stdlib/src/gen_statem.erl index 86109f04b4..b5e9da1e66 100644 --- a/lib/stdlib/src/gen_statem.erl +++ b/lib/stdlib/src/gen_statem.erl @@ -1722,6 +1722,8 @@ error_info( end; _ -> {Reason,Stacktrace} end, + [LimitedP, LimitedFmtData, LimitedFixedReason] = + [error_logger:limit_term(D) || D <- [P, FmtData, FixedReason]], CBMode = case StateEnter of true -> @@ -1755,8 +1757,8 @@ error_info( [] -> []; [Event|_] -> [Event] end] ++ - [FmtData, - Class,FixedReason, + [LimitedFmtData, + Class,LimitedFixedReason, CBMode] ++ case Q of [_|[_|_] = Events] -> [Events]; @@ -1764,7 +1766,7 @@ error_info( end ++ case P of [] -> []; - _ -> [P] + _ -> [LimitedP] end ++ case FixedStacktrace of [] -> []; diff --git a/lib/stdlib/src/io_lib.erl b/lib/stdlib/src/io_lib.erl index 5ed2f4d888..9d447418f8 100644 --- a/lib/stdlib/src/io_lib.erl +++ b/lib/stdlib/src/io_lib.erl @@ -84,6 +84,8 @@ -export([write_unicode_string/1, write_unicode_char/1, deep_unicode_char_list/1]). +-export([limit_term/2]). + -export_type([chars/0, latin1_string/0, continuation/0, fread_error/0, fread_item/0, format_spec/0]). @@ -911,3 +913,116 @@ binrev(L) -> binrev(L, T) -> list_to_binary(lists:reverse(L, T)). + +-spec limit_term(term(), non_neg_integer()) -> term(). + +%% The intention is to mimic the depth limitation of io_lib:write() +%% and io_lib_pretty:print(). The leaves ('...') should never be +%% seen when printed with the same depth. Bitstrings are never +%% truncated, which is OK as long as they are not sent to other nodes. +limit_term(Term, Depth) -> + try test_limit(Term, Depth) of + ok -> Term + catch + throw:limit -> + limit(Term, Depth) + end. + +limit(_, 0) -> '...'; +limit([H|T]=L, D) -> + if + D =:= 1 -> '...'; + true -> + case printable_list(L) of + true -> L; + false -> + [limit(H, D-1)|limit_tail(T, D-1)] + end + end; +limit(Term, D) when is_map(Term) -> + limit_map(Term, D); +limit({}=T, _D) -> T; +limit(T, D) when is_tuple(T) -> + if + D =:= 1 -> '...'; + true -> + list_to_tuple([limit(element(1, T), D-1)| + limit_tail(tl(tuple_to_list(T)), D-1)]) + end; +limit(<<_/bitstring>>=Term, D) -> limit_bitstring(Term, D); +limit(Term, _D) -> Term. + +limit_tail([], _D) -> []; +limit_tail(_, 1) -> ['...']; +limit_tail([H|T], D) -> + [limit(H, D-1)|limit_tail(T, D-1)]; +limit_tail(Other, D) -> + limit(Other, D-1). + +%% Cannot limit maps properly since there is no guarantee that +%% maps:from_list() creates a map with the same internal ordering of +%% the selected associations as in Map. +limit_map(Map, D) -> + maps:from_list(erts_internal:maps_to_list(Map, D)). +%% maps:from_list(limit_map_body(erts_internal:maps_to_list(Map, D), D)). + +%% limit_map_body(_, 0) -> [{'...', '...'}]; +%% limit_map_body([], _) -> []; +%% limit_map_body([{K,V}], D) -> [limit_map_assoc(K, V, D)]; +%% limit_map_body([{K,V}|KVs], D) -> +%% [limit_map_assoc(K, V, D) | limit_map_body(KVs, D-1)]. + +%% limit_map_assoc(K, V, D) -> +%% {limit(K, D-1), limit(V, D-1)}. + +limit_bitstring(B, _D) -> B. %% Keeps all printable binaries. + +test_limit(_, 0) -> throw(limit); +test_limit([H|T]=L, D) when is_integer(D) -> + if + D =:= 1 -> throw(limit); + true -> + case printable_list(L) of + true -> ok; + false -> + test_limit(H, D-1), + test_limit_tail(T, D-1) + end + end; +test_limit(Term, D) when is_map(Term) -> + test_limit_map(Term, D); +test_limit({}, _D) -> ok; +test_limit(T, D) when is_tuple(T) -> + test_limit_tuple(T, 1, tuple_size(T), D); +test_limit(<<_/bitstring>>=Term, D) -> test_limit_bitstring(Term, D); +test_limit(_Term, _D) -> ok. + +test_limit_tail([], _D) -> ok; +test_limit_tail(_, 1) -> throw(limit); +test_limit_tail([H|T], D) -> + test_limit(H, D-1), + test_limit_tail(T, D-1); +test_limit_tail(Other, D) -> + test_limit(Other, D-1). + +test_limit_tuple(_T, I, Sz, _D) when I > Sz -> ok; +test_limit_tuple(_, _, _, 1) -> throw(limit); +test_limit_tuple(T, I, Sz, D) -> + test_limit(element(I, T), D-1), + test_limit_tuple(T, I+1, Sz, D-1). + +test_limit_map(_Map, _D) -> ok. +%% test_limit_map_body(erts_internal:maps_to_list(Map, D), D). + +%% test_limit_map_body(_, 0) -> throw(limit); +%% test_limit_map_body([], _) -> ok; +%% test_limit_map_body([{K,V}], D) -> test_limit_map_assoc(K, V, D); +%% test_limit_map_body([{K,V}|KVs], D) -> +%% test_limit_map_assoc(K, V, D), +%% test_limit_map_body(KVs, D-1). + +%% test_limit_map_assoc(K, V, D) -> +%% test_limit(K, D-1), +%% test_limit(V, D-1). + +test_limit_bitstring(_, _) -> ok. diff --git a/lib/stdlib/src/proc_lib.erl b/lib/stdlib/src/proc_lib.erl index 2219467a8d..3fa54cd0d5 100644 --- a/lib/stdlib/src/proc_lib.erl +++ b/lib/stdlib/src/proc_lib.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2016. All Rights Reserved. +%% Copyright Ericsson AB 1996-2017. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -519,7 +519,8 @@ my_info_1(Class, Reason, Stacktrace) -> get_process_info(self(), registered_name), {error_info, {Class,Reason,Stacktrace}}, get_ancestors(self()), - get_process_info(self(), messages), + get_process_info(self(), message_queue_len), + get_messages(self()), get_process_info(self(), links), get_cleaned_dictionary(self()), get_process_info(self(), trap_exit), @@ -539,12 +540,49 @@ get_ancestors(Pid) -> {ancestors,[]} end. +%% The messages and the dictionary are possibly limited too much if +%% some error handles output the messages or the dictionary using ~P +%% or ~W with depth greater than the depth used here (the depth of +%% control characters P and W takes precedence over the depth set by +%% application variable error_logger_format_depth). However, it is +%% assumed that all report handlers call proc_lib:format(). +get_messages(Pid) -> + Messages = get_process_messages(Pid), + {messages, error_logger:limit_term(Messages)}. + +get_process_messages(Pid) -> + Depth = error_logger:get_format_depth(), + case Pid =/= self() orelse Depth =:= unlimited of + true -> + {messages, Messages} = get_process_info(Pid, messages), + Messages; + false -> + %% If there are more messages than Depth, garbage + %% collection can sometimes be avoided by collecting just + %% enough messages for the crash report. It is assumed the + %% process is about to die anyway. + receive_messages(Depth) + end. + +receive_messages(0) -> []; +receive_messages(N) -> + receive + M -> + [M|receive_messages(N - 1)] + after 0 -> + [] + end. + get_cleaned_dictionary(Pid) -> case get_process_info(Pid,dictionary) of - {dictionary,Dict} -> {dictionary,clean_dict(Dict)}; + {dictionary,Dict} -> {dictionary,cleaned_dict(Dict)}; _ -> {dictionary,[]} end. +cleaned_dict(Dict) -> + CleanDict = clean_dict(Dict), + error_logger:limit_term(CleanDict). + clean_dict([{'$ancestors',_}|Dict]) -> clean_dict(Dict); clean_dict([{'$initial_call',_}|Dict]) -> @@ -582,20 +620,24 @@ make_neighbour_reports1([P|Ps]) -> make_neighbour_reports1([]) -> []. +%% Do not include messages or process dictionary, even if +%% error_logger_format_depth is unlimited. make_neighbour_report(Pid) -> [{pid, Pid}, get_process_info(Pid, registered_name), get_initial_call(Pid), get_process_info(Pid, current_function), get_ancestors(Pid), - get_process_info(Pid, messages), + get_process_info(Pid, message_queue_len), + %% get_messages(Pid), get_process_info(Pid, links), - get_cleaned_dictionary(Pid), + %% get_cleaned_dictionary(Pid), get_process_info(Pid, trap_exit), get_process_info(Pid, status), get_process_info(Pid, heap_size), get_process_info(Pid, stack_size), - get_process_info(Pid, reductions) + get_process_info(Pid, reductions), + get_process_info(Pid, current_stacktrace) ]. get_initial_call(Pid) -> @@ -722,24 +764,37 @@ format(CrashReport, Encoding) -> format([OwnReport,LinkReport], Encoding, Depth) -> Extra = {Encoding,Depth}, - OwnFormat = format_report(OwnReport, Extra), - LinkFormat = format_report(LinkReport, Extra), + MyIndent = " ", + OwnFormat = format_report(OwnReport, MyIndent, Extra), + LinkFormat = format_link_report(LinkReport, MyIndent, Extra), Str = io_lib:format(" crasher:~n~ts neighbours:~n~ts", [OwnFormat, LinkFormat]), lists:flatten(Str). -format_report(Rep, Extra) when is_list(Rep) -> - format_rep(Rep, Extra); -format_report(Rep, {Enc,_}) -> - io_lib:format("~"++modifier(Enc)++"p~n", [Rep]). - -format_rep([{initial_call,InitialCall}|Rep], {_Enc,Depth}=Extra) -> - [format_mfa(InitialCall, Depth)|format_rep(Rep, Extra)]; -format_rep([{error_info,{Class,Reason,StackTrace}}|Rep], Extra) -> - [format_exception(Class, Reason, StackTrace, Extra)|format_rep(Rep, Extra)]; -format_rep([{Tag,Data}|Rep], Extra) -> - [format_tag(Tag, Data, Extra)|format_rep(Rep, Extra)]; -format_rep(_, _Extra) -> +format_link_report([Link|Reps], Indent, Extra) -> + Rep = case Link of + {neighbour,Rep0} -> Rep0; + _ -> Link + end, + LinkIndent = [" ",Indent], + [Indent,"neighbour:\n",format_report(Rep, LinkIndent, Extra)| + format_link_report(Reps, Indent, Extra)]; +format_link_report([], _, _) -> + []. + +format_report(Rep, Indent, Extra) when is_list(Rep) -> + format_rep(Rep, Indent, Extra); +format_report(Rep, Indent, {Enc,Depth}) -> + io_lib:format("~s~"++modifier(Enc)++"P~n", [Indent, Rep, Depth]). + +format_rep([{initial_call,InitialCall}|Rep], Indent, Extra) -> + [format_mfa(Indent, InitialCall, Extra)|format_rep(Rep, Indent, Extra)]; +format_rep([{error_info,{Class,Reason,StackTrace}}|Rep], Indent, Extra) -> + [format_exception(Class, Reason, StackTrace, Extra)| + format_rep(Rep, Indent, Extra)]; +format_rep([{Tag,Data}|Rep], Indent, Extra) -> + [format_tag(Indent, Tag, Data, Extra)|format_rep(Rep, Indent, Extra)]; +format_rep(_, _, _Extra) -> []. format_exception(Class, Reason, StackTrace, {Enc,_}=Extra) -> @@ -750,14 +805,14 @@ format_exception(Class, Reason, StackTrace, {Enc,_}=Extra) -> [EI, lib:format_exception(1+length(EI), Class, Reason, StackTrace, StackFun, PF, Enc), "\n"]. -format_mfa({M,F,Args}=StartF, Depth) -> +format_mfa(Indent, {M,F,Args}=StartF, Extra) -> try A = length(Args), - [" initial call: ",atom_to_list(M),$:,atom_to_list(F),$/, + [Indent,"initial call: ",atom_to_list(M),$:,atom_to_list(F),$/, integer_to_list(A),"\n"] catch error:_ -> - format_tag(initial_call, StartF, Depth) + format_tag(Indent, initial_call, StartF, Extra) end. pp_fun({Enc,Depth}) -> @@ -770,12 +825,12 @@ pp_fun({Enc,Depth}) -> io_lib:format("~." ++ integer_to_list(I) ++ P, [Term|Tl]) end. -format_tag(Tag, Data, {_Enc,Depth}) -> +format_tag(Indent, Tag, Data, {_Enc,Depth}) -> case Depth of unlimited -> - io_lib:format(" ~p: ~80.18p~n", [Tag, Data]); + io_lib:format("~s~p: ~80.18p~n", [Indent, Tag, Data]); _ -> - io_lib:format(" ~p: ~80.18P~n", [Tag, Data, Depth]) + io_lib:format("~s~p: ~80.18P~n", [Indent, Tag, Data, Depth]) end. modifier(latin1) -> ""; diff --git a/lib/stdlib/src/string.erl b/lib/stdlib/src/string.erl index 17135dd64a..6f7009b5d9 100644 --- a/lib/stdlib/src/string.erl +++ b/lib/stdlib/src/string.erl @@ -486,12 +486,14 @@ find(String, SearchPattern, trailing) -> %% Fetch first codepoint and return rest in tail -spec next_grapheme(String::unicode:chardata()) -> - maybe_improper_list(grapheme_cluster(),unicode:chardata()). + maybe_improper_list(grapheme_cluster(),unicode:chardata()) | + {error,unicode:chardata()}. next_grapheme(CD) -> unicode_util:gc(CD). %% Fetch first grapheme cluster and return rest in tail -spec next_codepoint(String::unicode:chardata()) -> - maybe_improper_list(char(),unicode:chardata()). + maybe_improper_list(char(),unicode:chardata()) | + {error,unicode:chardata()}. next_codepoint(CD) -> unicode_util:cp(CD). %% Internals @@ -508,7 +510,7 @@ equal_1(A0,B0) -> case {unicode_util:cp(A0), unicode_util:cp(B0)} of {[CP|A],[CP|B]} -> equal_1(A,B); {[], []} -> true; - _ -> false + {L1,L2} when is_list(L1), is_list(L2) -> false end. equal_nocase(A, A) -> true; @@ -517,7 +519,7 @@ equal_nocase(A0, B0) -> unicode_util:cp(unicode_util:casefold(B0))} of {[CP|A],[CP|B]} -> equal_nocase(A,B); {[], []} -> true; - _ -> false + {L1,L2} when is_list(L1), is_list(L2) -> false end. equal_norm(A, A, _Norm) -> true; @@ -526,7 +528,7 @@ equal_norm(A0, B0, Norm) -> unicode_util:cp(unicode_util:Norm(B0))} of {[CP|A],[CP|B]} -> equal_norm(A,B, Norm); {[], []} -> true; - _ -> false + {L1,L2} when is_list(L1), is_list(L2) -> false end. equal_norm_nocase(A, A, _Norm) -> true; @@ -535,7 +537,7 @@ equal_norm_nocase(A0, B0, Norm) -> unicode_util:cp(unicode_util:casefold(unicode_util:Norm(B0)))} of {[CP|A],[CP|B]} -> equal_norm_nocase(A,B, Norm); {[], []} -> true; - _ -> false + {L1,L2} when is_list(L1), is_list(L2) -> false end. reverse_1(CD, Acc) -> diff --git a/lib/stdlib/src/unicode.erl b/lib/stdlib/src/unicode.erl index aa1da400ce..fbe8a94074 100644 --- a/lib/stdlib/src/unicode.erl +++ b/lib/stdlib/src/unicode.erl @@ -250,89 +250,110 @@ encoding_to_bom(latin1) -> -define(GC_N, 200). %% arbitrary number %% Canonical decompose string to list of chars --spec characters_to_nfd_list(chardata()) -> [char()]. +-spec characters_to_nfd_list(chardata()) -> [char()] | {error, [char()], chardata()}. characters_to_nfd_list(CD) -> + characters_to_nfd_list(CD, []). +characters_to_nfd_list(CD, Acc) -> case unicode_util:nfd(CD) of - [GC|Str] when is_list(GC) -> GC++characters_to_nfd_list(Str); - [CP|Str] -> [CP|characters_to_nfd_list(Str)]; - [] -> [] + [GC|Str] when is_list(GC) -> characters_to_nfd_list(Str, lists:reverse(GC, Acc)); + [CP|Str] -> characters_to_nfd_list(Str, [CP | Acc]); + [] -> lists:reverse(Acc); + {error,Error} -> {error, lists:reverse(Acc), Error} end. --spec characters_to_nfd_binary(chardata()) -> unicode_binary(). +-spec characters_to_nfd_binary(chardata()) -> unicode_binary() | {error, unicode_binary(), chardata()}. characters_to_nfd_binary(CD) -> - list_to_binary(characters_to_nfd_binary(CD, ?GC_N, [])). + characters_to_nfd_binary(CD, ?GC_N, [], []). -characters_to_nfd_binary(CD, N, Row) when N > 0 -> +characters_to_nfd_binary(CD, N, Row, Acc) when N > 0 -> case unicode_util:nfd(CD) of - [GC|Str] -> characters_to_nfd_binary(Str, N-1, [GC|Row]); - [] -> [characters_to_binary(lists:reverse(Row))] + [GC|Str] -> characters_to_nfd_binary(Str, N-1, [GC|Row], Acc); + [] -> acc_to_binary(prepend_row_to_acc(Row, Acc)); + {error, Error} -> {error, acc_to_binary(prepend_row_to_acc(Row, Acc)), Error} end; -characters_to_nfd_binary(CD, _, Row) -> - [characters_to_binary(lists:reverse(Row))|characters_to_nfd_binary(CD,?GC_N,[])]. +characters_to_nfd_binary(CD, _, Row, Acc) -> + characters_to_nfd_binary(CD, ?GC_N, [], prepend_row_to_acc(Row, Acc)). %% Compability Canonical decompose string to list of chars. --spec characters_to_nfkd_list(chardata()) -> [char()]. +-spec characters_to_nfkd_list(chardata()) -> [char()] | {error, [char()], chardata()}. characters_to_nfkd_list(CD) -> + characters_to_nfkd_list(CD, []). +characters_to_nfkd_list(CD, Acc) -> case unicode_util:nfkd(CD) of - [GC|Str] when is_list(GC) -> GC++characters_to_nfkd_list(Str); - [CP|Str] -> [CP|characters_to_nfkd_list(Str)]; - [] -> [] + [GC|Str] when is_list(GC) -> characters_to_nfkd_list(Str, lists:reverse(GC, Acc)); + [CP|Str] -> characters_to_nfkd_list(Str, [CP | Acc]); + [] -> lists:reverse(Acc); + {error,Error} -> {error, lists:reverse(Acc), Error} end. --spec characters_to_nfkd_binary(chardata()) -> unicode_binary(). +-spec characters_to_nfkd_binary(chardata()) -> unicode_binary() | {error, unicode_binary(), chardata()}. characters_to_nfkd_binary(CD) -> - list_to_binary(characters_to_nfkd_binary(CD, ?GC_N, [])). + characters_to_nfkd_binary(CD, ?GC_N, [], []). -characters_to_nfkd_binary(CD, N, Row) when N > 0 -> +characters_to_nfkd_binary(CD, N, Row, Acc) when N > 0 -> case unicode_util:nfkd(CD) of - [GC|Str] -> characters_to_nfkd_binary(Str, N-1, [GC|Row]); - [] -> [characters_to_binary(lists:reverse(Row))] + [GC|Str] -> characters_to_nfkd_binary(Str, N-1, [GC|Row], Acc); + [] -> acc_to_binary(prepend_row_to_acc(Row, Acc)); + {error, Error} -> {error, acc_to_binary(prepend_row_to_acc(Row, Acc)), Error} end; -characters_to_nfkd_binary(CD, _, Row) -> - [characters_to_binary(lists:reverse(Row))|characters_to_nfkd_binary(CD,?GC_N,[])]. +characters_to_nfkd_binary(CD, _, Row, Acc) -> + characters_to_nfkd_binary(CD, ?GC_N, [], prepend_row_to_acc(Row, Acc)). %% Canonical compose string to list of chars --spec characters_to_nfc_list(chardata()) -> [char()]. +-spec characters_to_nfc_list(chardata()) -> [char()] | {error, [char()], chardata()}. characters_to_nfc_list(CD) -> + characters_to_nfc_list(CD, []). +characters_to_nfc_list(CD, Acc) -> case unicode_util:nfc(CD) of - [CPs|Str] when is_list(CPs) -> CPs ++ characters_to_nfc_list(Str); - [CP|Str] -> [CP|characters_to_nfc_list(Str)]; - [] -> [] + [GC|Str] when is_list(GC) -> characters_to_nfc_list(Str, lists:reverse(GC, Acc)); + [CP|Str] -> characters_to_nfc_list(Str, [CP | Acc]); + [] -> lists:reverse(Acc); + {error,Error} -> {error, lists:reverse(Acc), Error} end. --spec characters_to_nfc_binary(chardata()) -> unicode_binary(). +-spec characters_to_nfc_binary(chardata()) -> unicode_binary() | {error, unicode_binary(), chardata()}. characters_to_nfc_binary(CD) -> - list_to_binary(characters_to_nfc_binary(CD, ?GC_N, [])). + characters_to_nfc_binary(CD, ?GC_N, [], []). -characters_to_nfc_binary(CD, N, Row) when N > 0 -> +characters_to_nfc_binary(CD, N, Row, Acc) when N > 0 -> case unicode_util:nfc(CD) of - [GC|Str] -> characters_to_nfc_binary(Str, N-1, [GC|Row]); - [] -> [characters_to_binary(lists:reverse(Row))] + [GC|Str] -> characters_to_nfc_binary(Str, N-1, [GC|Row], Acc); + [] -> acc_to_binary(prepend_row_to_acc(Row, Acc)); + {error, Error} -> {error, acc_to_binary(prepend_row_to_acc(Row, Acc)), Error} end; -characters_to_nfc_binary(CD, _, Row) -> - [characters_to_binary(lists:reverse(Row))|characters_to_nfc_binary(CD,?GC_N,[])]. +characters_to_nfc_binary(CD, _, Row, Acc) -> + characters_to_nfc_binary(CD, ?GC_N, [], prepend_row_to_acc(Row, Acc)). %% Compability Canonical compose string to list of chars --spec characters_to_nfkc_list(chardata()) -> [char()]. +-spec characters_to_nfkc_list(chardata()) -> [char()] | {error, [char()], chardata()}. characters_to_nfkc_list(CD) -> + characters_to_nfkc_list(CD, []). +characters_to_nfkc_list(CD, Acc) -> case unicode_util:nfkc(CD) of - [CPs|Str] when is_list(CPs) -> CPs ++ characters_to_nfkc_list(Str); - [CP|Str] -> [CP|characters_to_nfkc_list(Str)]; - [] -> [] + [GC|Str] when is_list(GC) -> characters_to_nfkc_list(Str, lists:reverse(GC, Acc)); + [CP|Str] -> characters_to_nfkc_list(Str, [CP | Acc]); + [] -> lists:reverse(Acc); + {error,Error} -> {error, lists:reverse(Acc), Error} end. --spec characters_to_nfkc_binary(chardata()) -> unicode_binary(). +-spec characters_to_nfkc_binary(chardata()) -> unicode_binary() | {error, unicode_binary(), chardata()}. characters_to_nfkc_binary(CD) -> - list_to_binary(characters_to_nfkc_binary(CD, ?GC_N, [])). + characters_to_nfkc_binary(CD, ?GC_N, [], []). -characters_to_nfkc_binary(CD, N, Row) when N > 0 -> +characters_to_nfkc_binary(CD, N, Row, Acc) when N > 0 -> case unicode_util:nfkc(CD) of - [GC|Str] -> characters_to_nfkc_binary(Str, N-1, [GC|Row]); - [] -> [characters_to_binary(lists:reverse(Row))] + [GC|Str] -> characters_to_nfkc_binary(Str, N-1, [GC|Row], Acc); + [] -> acc_to_binary(prepend_row_to_acc(Row, Acc)); + {error, Error} -> {error, acc_to_binary(prepend_row_to_acc(Row, Acc)), Error} end; -characters_to_nfkc_binary(CD, _, Row) -> - [characters_to_binary(lists:reverse(Row))|characters_to_nfkc_binary(CD,?GC_N,[])]. +characters_to_nfkc_binary(CD, _, Row, Acc) -> + characters_to_nfkc_binary(CD, ?GC_N, [], prepend_row_to_acc(Row, Acc)). + +acc_to_binary(Acc) -> + list_to_binary(lists:reverse(Acc)). +prepend_row_to_acc(Row, Acc) -> + [characters_to_binary(lists:reverse(Row))|Acc]. %% internals diff --git a/lib/stdlib/test/erl_lint_SUITE.erl b/lib/stdlib/test/erl_lint_SUITE.erl index 02524679fa..cc3d605840 100644 --- a/lib/stdlib/test/erl_lint_SUITE.erl +++ b/lib/stdlib/test/erl_lint_SUITE.erl @@ -3978,7 +3978,11 @@ otp_14323(Config) -> {13,erl_lint,{undefined_function,{a,1}}}, {14,erl_lint,{bad_dialyzer_attribute, {nowarn_function,{a,-1}}}}], - []}}], + []}}, + {otp_14323_2, + <<"-type t(_) :: atom().">>, + [], + {errors,[{1,erl_parse,"bad type variable"}],[]}}], [] = run(Config, Ts), ok. diff --git a/lib/stdlib/test/gen_server_SUITE.erl b/lib/stdlib/test/gen_server_SUITE.erl index 7e3c71715e..2e9dc4d4fb 100644 --- a/lib/stdlib/test/gen_server_SUITE.erl +++ b/lib/stdlib/test/gen_server_SUITE.erl @@ -404,7 +404,7 @@ crash(Config) when is_list(Config) -> end, receive {error_report,_,{Pid4,crash_report,[List4|_]}} -> - {exit,crashed,_} = proplists:get_value(error_info, List4), + {exit,crashed,[{?MODULE, handle_call, 3, _}|_]} = proplists:get_value(error_info, List4), Pid4 = proplists:get_value(pid, List4); Other4 -> io:format("Unexpected: ~p", [Other4]), diff --git a/lib/stdlib/test/io_SUITE.erl b/lib/stdlib/test/io_SUITE.erl index ef3f0be5d7..e2c73371cd 100644 --- a/lib/stdlib/test/io_SUITE.erl +++ b/lib/stdlib/test/io_SUITE.erl @@ -31,7 +31,7 @@ otp_10836/1, io_lib_width_too_small/1, io_with_huge_message_queue/1, format_string/1, maps/1, coverage/1, otp_14178_unicode_atoms/1, otp_14175/1, - otp_14285/1]). + otp_14285/1, limit_term/1]). -export([pretty/2]). @@ -63,7 +63,7 @@ all() -> io_lib_print_binary_depth_one, otp_10302, otp_10755, otp_10836, io_lib_width_too_small, io_with_huge_message_queue, format_string, maps, coverage, otp_14178_unicode_atoms, otp_14175, - otp_14285]. + otp_14285, limit_term]. %% Error cases for output. error_1(Config) when is_list(Config) -> @@ -2373,3 +2373,58 @@ otp_14285(_Config) -> latin1_fmt(Fmt, Args) -> L = fmt(Fmt, Args), true = lists:all(fun is_latin1/1, L). + +limit_term(_Config) -> + {_, 2} = limt([a,b,c], 2), + {_, 2} = limt([a,b,c], 3), + {_, 2} = limt([a,b|c], 2), + {_, 2} = limt([a,b|c], 3), + {_, 2} = limt({a,b,c,[d,e]}, 2), + {_, 2} = limt({a,b,c,[d,e]}, 3), + {_, 2} = limt({a,b,c,[d,e]}, 4), + {_, 1} = limt(<<"foo">>, 18), + ok = blimt(<<"123456789012345678901234567890">>), + {_, 1} = limt(<<7:3>>, 2), + {_, 1} = limt(<<7:21>>, 2), + {_, 1} = limt([], 2), + {_, 1} = limt({}, 2), + {_, 1} = limt(#{}, 2), + {_, 1} = limt(#{[] => {}}, 2), + {_, 1} = limt(#{[] => {}}, 3), + T = #{[] => {},[a] => [b]}, + {_, 1} = limt(T, 2), + {_, 1} = limt(T, 3), + {_, 1} = limt(T, 4), + ok. + +blimt(Binary) -> + blimt(Binary, byte_size(Binary)). + +blimt(_B, 1) -> ok; +blimt(B, D) -> + {_, 1} = limt(B, D), + blimt(B, D - 1). + +limt(Term, Depth) when is_integer(Depth) -> + T1 = io_lib:limit_term(Term, Depth), + S = form(Term, Depth), + S1 = form(T1, Depth), + OK1 = S1 =:= S, + + T2 = io_lib:limit_term(Term, Depth+1), + S2 = form(T2, Depth), + OK2 = S2 =:= S, + + T3 = io_lib:limit_term(Term, Depth-1), + S3 = form(T3, Depth), + OK3 = S3 =/= S, + + R = case {OK1, OK2, OK3} of + {true, true, true} -> 2; + {true, true, false} -> 1; + _ -> 0 + end, + {{S, S1, S2}, R}. + +form(Term, Depth) -> + lists:flatten(io_lib:format("~W", [Term, Depth])). diff --git a/lib/stdlib/test/string_SUITE.erl b/lib/stdlib/test/string_SUITE.erl index 4320b735ac..90f980c0e5 100644 --- a/lib/stdlib/test/string_SUITE.erl +++ b/lib/stdlib/test/string_SUITE.erl @@ -582,6 +582,8 @@ cd_gc(_) -> [$e,778] = string:next_codepoint([$e,778]), [$e|<<204,138>>] = string:next_codepoint(<<$e,778/utf8>>), [778|_] = string:next_codepoint(tl(string:next_codepoint(<<$e,778/utf8>>))), + [0|<<128,1>>] = string:next_codepoint(<<0,128,1>>), + {error,<<128,1>>} = string:next_codepoint(<<128,1>>), [] = string:next_grapheme(""), [] = string:next_grapheme(<<>>), @@ -589,6 +591,8 @@ cd_gc(_) -> "abcd" = string:next_grapheme("abcd"), [[$e,778]] = string:next_grapheme([$e,778]), [[$e,778]] = string:next_grapheme(<<$e,778/utf8>>), + [0|<<128,1>>] = string:next_grapheme(<<0,128,1>>), + {error,<<128,1>>} = string:next_grapheme(<<128,1>>), ok. diff --git a/lib/stdlib/test/tar_SUITE.erl b/lib/stdlib/test/tar_SUITE.erl index e9ab12e061..4061008812 100644 --- a/lib/stdlib/test/tar_SUITE.erl +++ b/lib/stdlib/test/tar_SUITE.erl @@ -27,7 +27,8 @@ extract_from_binary_compressed/1, extract_filtered/1, extract_from_open_file/1, symlinks/1, open_add_close/1, cooked_compressed/1, memory/1,unicode/1,read_other_implementations/1, - sparse/1, init/1, leading_slash/1, dotdot/1]). + sparse/1, init/1, leading_slash/1, dotdot/1, + roundtrip_metadata/1]). -include_lib("common_test/include/ct.hrl"). -include_lib("kernel/include/file.hrl"). @@ -41,7 +42,7 @@ all() -> extract_filtered, symlinks, open_add_close, cooked_compressed, memory, unicode, read_other_implementations, - sparse,init,leading_slash,dotdot]. + sparse,init,leading_slash,dotdot,roundtrip_metadata]. groups() -> []. @@ -953,6 +954,42 @@ dotdot(Config) -> ok. +roundtrip_metadata(Config) -> + PrivDir = proplists:get_value(priv_dir, Config), + Dir = filename:join(PrivDir, ?FUNCTION_NAME), + ok = file:make_dir(Dir), + + do_roundtrip_metadata(Dir, "name-does-not-matter"), + ok. + +do_roundtrip_metadata(Dir, File) -> + Tar = filename:join(Dir, atom_to_list(?FUNCTION_NAME)++".tar"), + BeamFile = code:which(compile), + {ok,Fd} = erl_tar:open(Tar, [write]), + ok = erl_tar:add(Fd, BeamFile, File, []), + ok = erl_tar:close(Fd), + + ok = erl_tar:extract(Tar, [{cwd,Dir}]), + + %% Make sure that size and modification times are the same + %% on all platforms. + {ok,OrigInfo} = file:read_file_info(BeamFile), + ExtractedFile = filename:join(Dir, File), + {ok,ExtractedInfo} = file:read_file_info(ExtractedFile), + #file_info{size=Size,mtime=Mtime,type=regular} = OrigInfo, + #file_info{size=Size,mtime=Mtime,type=regular} = ExtractedInfo, + + %% On Unix platforms more fields are expected to be the same. + case os:type() of + {unix,_} -> + #file_info{access=Access,mode=Mode} = OrigInfo, + #file_info{access=Access,mode=Mode} = ExtractedInfo, + ok; + _ -> + ok + end. + + %% Delete the given list of files. delete_files([]) -> ok; delete_files([Item|Rest]) -> diff --git a/lib/stdlib/test/unicode_SUITE.erl b/lib/stdlib/test/unicode_SUITE.erl index 3d97ab93f1..e01ba3fbb0 100644 --- a/lib/stdlib/test/unicode_SUITE.erl +++ b/lib/stdlib/test/unicode_SUITE.erl @@ -998,6 +998,30 @@ normalize(_) -> true = unicode:characters_to_nfkc_list("ホンダ") =:= unicode:characters_to_nfkc_list("ﾎﾝﾀﾞ"), true = unicode:characters_to_nfkd_list("32") =:= unicode:characters_to_nfkd_list("３２"), + + {error, [0], <<128>>} = unicode:characters_to_nfc_list(<<0, 128>>), + {error, [0], <<128>>} = unicode:characters_to_nfkc_list(<<0, 128>>), + {error, [0], <<128>>} = unicode:characters_to_nfd_list(<<0, 128>>), + {error, [0], <<128>>} = unicode:characters_to_nfkd_list(<<0, 128>>), + + {error, <<0>>, <<128>>} = unicode:characters_to_nfc_binary(<<0, 128>>), + {error, <<0>>, <<128>>} = unicode:characters_to_nfkc_binary(<<0, 128>>), + {error, <<0>>, <<128>>} = unicode:characters_to_nfd_binary(<<0, 128>>), + {error, <<0>>, <<128>>} = unicode:characters_to_nfkd_binary(<<0, 128>>), + + LargeBin = binary:copy(<<"abcde">>, 50), + LargeList = binary_to_list(LargeBin), + + {error, LargeList, <<128>>} = unicode:characters_to_nfc_list(<<LargeBin/binary, 128>>), + {error, LargeList, <<128>>} = unicode:characters_to_nfkc_list(<<LargeBin/binary, 128>>), + {error, LargeList, <<128>>} = unicode:characters_to_nfd_list(<<LargeBin/binary, 128>>), + {error, LargeList, <<128>>} = unicode:characters_to_nfkd_list(<<LargeBin/binary, 128>>), + + {error, LargeBin, <<128>>} = unicode:characters_to_nfc_binary(<<LargeBin/binary, 128>>), + {error, LargeBin, <<128>>} = unicode:characters_to_nfkc_binary(<<LargeBin/binary, 128>>), + {error, LargeBin, <<128>>} = unicode:characters_to_nfd_binary(<<LargeBin/binary, 128>>), + {error, LargeBin, <<128>>} = unicode:characters_to_nfkd_binary(<<LargeBin/binary, 128>>), + ok. diff --git a/lib/stdlib/test/unicode_util_SUITE.erl b/lib/stdlib/test/unicode_util_SUITE.erl index e9b3d7f98d..03c24c7027 100644 --- a/lib/stdlib/test/unicode_util_SUITE.erl +++ b/lib/stdlib/test/unicode_util_SUITE.erl @@ -97,6 +97,8 @@ cp(_) -> "hejsan" = fetch(<<"hejsan">>, Get), "hejsan" = fetch(["hej",<<"san">>], Get), "hejsan" = fetch(["hej"|<<"san">>], Get), + {error, <<128>>} = Get(<<128>>), + {error, [<<128>>, 0]} = Get([<<128>>, 0]), ok. gc(Config) -> @@ -106,6 +108,8 @@ gc(Config) -> "hejsan" = fetch(<<"hejsan">>, Get), "hejsan" = fetch(["hej",<<"san">>], Get), "hejsan" = fetch(["hej"|<<"san">>], Get), + {error, <<128>>} = Get(<<128>>), + {error, [<<128>>, 0]} = Get([<<128>>, 0]), 0 = fold(fun verify_gc/3, 0, DataDir ++ "/GraphemeBreakTest.txt"), ok. diff --git a/lib/stdlib/uc_spec/gen_unicode_mod.escript b/lib/stdlib/uc_spec/gen_unicode_mod.escript index c8b815e435..fefd7d3b70 100755 --- a/lib/stdlib/uc_spec/gen_unicode_mod.escript +++ b/lib/stdlib/uc_spec/gen_unicode_mod.escript @@ -170,7 +170,7 @@ gen_header(Fd) -> io:put_chars(Fd, "-export([spec_version/0, lookup/1, get_case/1]).\n"), io:put_chars(Fd, "-inline([class/1]).\n"), io:put_chars(Fd, "-compile(nowarn_unused_vars).\n"), - io:put_chars(Fd, "-dialyzer({no_improper_lists, cp/1}).\n"), + io:put_chars(Fd, "-dialyzer({no_improper_lists, [cp/1, gc_prepend/2, gc_e_cont/2]}).\n"), io:put_chars(Fd, "-type gc() :: char()|[char()].\n\n\n"), ok. @@ -237,39 +237,43 @@ gen_static(Fd) -> gen_norm(Fd) -> io:put_chars(Fd, - "-spec nfd(unicode:chardata()) -> maybe_improper_list(gc(),unicode:chardata()).\n" + "-spec nfd(unicode:chardata()) -> maybe_improper_list(gc(),unicode:chardata()) | {error, unicode:chardata()}.\n" "nfd(Str0) ->\n" " case gc(Str0) of\n" " [GC|R] when GC < 127 -> [GC|R];\n" " [GC|Str] -> [decompose(GC)|Str];\n" - " [] -> []\n end.\n\n" + " [] -> [];\n" + " {error,_}=Error -> Error\n end.\n\n" ), io:put_chars(Fd, - "-spec nfkd(unicode:chardata()) -> maybe_improper_list(gc(),unicode:chardata()).\n" + "-spec nfkd(unicode:chardata()) -> maybe_improper_list(gc(),unicode:chardata()) | {error, unicode:chardata()}.\n" "nfkd(Str0) ->\n" " case gc(Str0) of\n" " [GC|R] when GC < 127 -> [GC|R];\n" " [GC|Str] -> [decompose_compat(GC)|Str];\n" - " [] -> []\n end.\n\n" + " [] -> [];\n" + " {error,_}=Error -> Error\n end.\n\n" ), io:put_chars(Fd, - "-spec nfc(unicode:chardata()) -> maybe_improper_list(gc(),unicode:chardata()).\n" + "-spec nfc(unicode:chardata()) -> maybe_improper_list(gc(),unicode:chardata()) | {error, unicode:chardata()}.\n" "nfc(Str0) ->\n" " case gc(Str0) of\n" " [GC|R] when GC < 255 -> [GC|R];\n" " [GC|Str] -> [compose(decompose(GC))|Str];\n" - " [] -> []\n end.\n\n" + " [] -> [];\n" + " {error,_}=Error -> Error\n end.\n\n" ), io:put_chars(Fd, - "-spec nfkc(unicode:chardata()) -> maybe_improper_list(gc(),unicode:chardata()).\n" + "-spec nfkc(unicode:chardata()) -> maybe_improper_list(gc(),unicode:chardata()) | {error, unicode:chardata()}.\n" "nfkc(Str0) ->\n" " case gc(Str0) of\n" " [GC|R] when GC < 127 -> [GC|R];\n" " [GC|Str] -> [compose_compat_0(decompose_compat(GC))|Str];\n" - " [] -> []\n end.\n\n" + " [] -> [];\n" + " {error,_}=Error -> Error\n end.\n\n" ), io:put_chars(Fd, @@ -448,18 +452,20 @@ gen_ws(Fd, Props) -> gen_cp(Fd) -> io:put_chars(Fd, "-spec cp(String::unicode:chardata()) ->" - " maybe_improper_list().\n"), + " maybe_improper_list() | {error, unicode:chardata()}.\n"), io:put_chars(Fd, "cp([C|_]=L) when is_integer(C) -> L;\n"), io:put_chars(Fd, "cp([List]) -> cp(List);\n"), io:put_chars(Fd, "cp([List|R]) ->\n"), io:put_chars(Fd, " case cp(List) of\n"), io:put_chars(Fd, " [] -> cp(R);\n"), io:put_chars(Fd, " [CP] -> [CP|R];\n"), - io:put_chars(Fd, " [C|R0] -> [C|[R0|R]]\n"), + io:put_chars(Fd, " [C|R0] -> [C|[R0|R]];\n"), + io:put_chars(Fd, " {error,Error} -> {error,[Error|R]}\n"), io:put_chars(Fd, " end;\n"), io:put_chars(Fd, "cp([]) -> [];\n"), io:put_chars(Fd, "cp(<<C/utf8, R/binary>>) -> [C|R];\n"), - io:put_chars(Fd, "cp(<<>>) -> [].\n\n"), + io:put_chars(Fd, "cp(<<>>) -> [];\n"), + io:put_chars(Fd, "cp(<<R/binary>>) -> {error,R}.\n\n"), ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -468,7 +474,7 @@ gen_gc(Fd, GBP) -> %% see http://www.unicode.org/reports/tr29/#Grapheme_Cluster_Boundary_Rules io:put_chars(Fd, "-spec gc(String::unicode:chardata()) ->" - " maybe_improper_list().\n"), + " maybe_improper_list() | {error, unicode:chardata()}.\n"), io:put_chars(Fd, "gc(Str) ->\n" " gc_1(cp(Str)).\n\n" @@ -521,7 +527,8 @@ gen_gc(Fd, GBP) -> [GenEBG(CP) || CP <- merge_ranges(maps:get(e_base_gaz,GBP))], io:put_chars(Fd, "gc_1([CP|R]) -> gc_extend(R, CP);\n"), - io:put_chars(Fd, "gc_1([]) -> [].\n\n"), + io:put_chars(Fd, "gc_1([]) -> [];\n"), + io:put_chars(Fd, "gc_1({error,_}=Error) -> Error.\n\n"), io:put_chars(Fd, "%% Handle Prepend\n"), io:put_chars(Fd, @@ -536,7 +543,8 @@ gen_gc(Fd, GBP) -> " [GC|R1] -> [[CP0|GC]|R1]\n" " end\n" " end;\n" - " [] -> [CP0]\n" + " [] -> [CP0];\n" + " {error,R} -> [CP0|R]\n" " end.\n\n"), IsCtrl = fun(Range) -> io:format(Fd, "is_control~s true;\n", [gen_single_clause(Range)]) end, @@ -574,7 +582,10 @@ gen_gc(Fd, GBP) -> " [_]=Acc -> Acc;\n" " [_|_]=Acc -> [lists:reverse(Acc)];\n" " Acc -> [Acc]\n" - " end.\n\n"), + " end;\n" + "gc_extend({error,R}, T, Acc0) ->\n" + " gc_extend([], T, Acc0) ++ [R].\n\n" + ), [ZWJ] = maps:get(zwj, GBP), GenExtend = fun(R) when R =:= ZWJ -> io:format(Fd, "is_extend~s zwj;\n", [gen_single_clause(ZWJ)]); (Range) -> io:format(Fd, "is_extend~s true;\n", [gen_single_clause(Range)]) @@ -604,6 +615,11 @@ gen_gc(Fd, GBP) -> " case Acc of\n" " [A] -> [A];\n" " _ -> [lists:reverse(Acc)]\n" + " end;\n" + " {error,R} ->\n" + " case Acc of\n" + " [A] -> [A|R];\n" + " _ -> [lists:reverse(Acc)|R]\n" " end\n" " end.\n\n"), @@ -660,6 +676,7 @@ gen_gc(Fd, GBP) -> [GenHangulT_1(CP) || CP <- merge_ranges(maps:get(t,GBP))], io:put_chars(Fd, " R1 -> gc_extend(R1, R0, Acc)\n end.\n\n"), + io:put_chars(Fd, "gc_h_lv_lvt({error,_}=Error, Acc) -> gc_extend(Error, [], Acc);\n"), io:put_chars(Fd, "%% Handle Hangul LV\n"), GenHangulLV = fun(Range) -> io:format(Fd, "gc_h_lv_lvt~s gc_h_V(R1,[CP|Acc]);\n", [gen_clause2(Range)]) end, diff --git a/lib/wx/c_src/wxe_impl.cpp b/lib/wx/c_src/wxe_impl.cpp index 7817e7fa8a..1510866f09 100644 --- a/lib/wx/c_src/wxe_impl.cpp +++ b/lib/wx/c_src/wxe_impl.cpp @@ -267,7 +267,7 @@ int WxeApp::dispatch_cmds() return more; } -#define BREAK_BATCH 200 +#define BREAK_BATCH 10000 int WxeApp::dispatch(wxeFifo * batch) { @@ -284,7 +284,7 @@ int WxeApp::dispatch(wxeFifo * batch) if(blevel>0) { blevel--; if(blevel==0) - wait += BREAK_BATCH*100; + wait += BREAK_BATCH/4; } break; case WXE_BATCH_BEGIN: @@ -317,7 +317,7 @@ int WxeApp::dispatch(wxeFifo * batch) erl_drv_mutex_lock(wxe_batch_locker_m); batch->Cleanup(); } - if(blevel <= 0 || wait > BREAK_BATCH) { + if(blevel <= 0 || wait >= BREAK_BATCH) { erl_drv_mutex_unlock(wxe_batch_locker_m); if(blevel > 0) { return 1; // We are still in a batch but we can let wx check for events diff --git a/system/doc/design_principles/statem.xml b/system/doc/design_principles/statem.xml index 0667af7868..7febe31df3 100644 --- a/system/doc/design_principles/statem.xml +++ b/system/doc/design_principles/statem.xml @@ -1582,10 +1582,12 @@ format_status(Opt, [_PDict,State,Data]) -> for example, a complex state term like a tuple. </p> <p> - One reason to use this is when you have - a state item that affects the event handling, - in particular in combination with postponing events. - We complicate the previous example + One reason to use this is when you have a state item + that when changed should cancel the + <seealso marker="#State Time-Outs">state time-out</seealso>, + or one that affects the event handling + in combination with postponing events. + We will complicate the previous example by introducing a configurable lock button (this is the state item in question), which in the <c>open</c> state immediately locks the door, diff --git a/system/doc/getting_started/conc_prog.xml b/system/doc/getting_started/conc_prog.xml index f3136898ad..4b19095d95 100644 --- a/system/doc/getting_started/conc_prog.xml +++ b/system/doc/getting_started/conc_prog.xml @@ -355,8 +355,8 @@ pong ! {ping, self()},</code> <p>Let us rewrite the ping pong program with "ping" and "pong" on different computers. First a few things are needed to set up to get this to work. The distributed Erlang - implementation provides a basic security mechanism to prevent - unauthorized access to an Erlang system on another computer. + implementation provides a very basic authentication mechanism to prevent + unintentional access to an Erlang system on another computer. Erlang systems which talk to each other must have the same <em>magic cookie</em>. The easiest way to achieve this is by having a file called <c>.erlang.cookie</c> in your home diff --git a/system/doc/reference_manual/distributed.xml b/system/doc/reference_manual/distributed.xml index 0a4a323fe9..01d78436c5 100644 --- a/system/doc/reference_manual/distributed.xml +++ b/system/doc/reference_manual/distributed.xml @@ -42,6 +42,19 @@ <p>The distribution mechanism is implemented using TCP/IP sockets. How to implement an alternative carrier is described in the <seealso marker="erts:alt_dist">ERTS User's Guide</seealso>.</p> + <warning> + <p> + Starting a distributed node without also specifying + <seealso marker="erts:erl#proto_dist"><c>-proto_dist inet_tls</c></seealso> + will expose the node to attacks that may give the attacker + complete access to the node and in extension the cluster. + When using un-secure distributed nodes, make sure that the + network is configured to keep potential attackers out. + See the <seealso marker="ssl:ssl_distribution"> + Using SSL for Erlang Distribution</seealso> User's Guide + for details on how to setup a secure distributed node. + </p> + </warning> </section> <section> |