fix sys:get_state/1,2 and sys:replace_state/2,3 when sys suspended

Add two new system callbacks Module:system_get_state/1 and
Module:system_replace_state/2 to allow sys:get_state/1,2 and
sys:replace_state/2,3 to operate correctly even if a process is sys
suspended. Modify gen_server, gen_fsm, and gen_event to support the new
callbacks.

If a callback module does not export these functions, then by default the
Misc value (the same as that passed as the final argument to
sys:handle_system_msg/6, and returned as part of the return value of
sys:get_status/1,2) is treated as the callback module's state.

The previous behaviour of intercepting the system message and passing a
tuple of size 2 as the last argument to sys:handle_system_msg/6 is no
longer supported.

Add tests to verify the correctness of sys:get_state/1,2 and
sys:replace_state/2,3 when processes are sys suspended. Add two tests for
modules that implement special processes, one that exports
system_get_state/1 and system_replace_state/2 and one that doesn't.

Much of the credit for this patch goes to James Fish, who reported the
initial problem and implemented much of the fix.

13 files changed, 547 insertions, 57 deletions

diff --git a/lib/stdlib/doc/src/sys.xml b/lib/stdlib/doc/src/sys.xml
index ab8b380f49..a46fa1289f 100644
--- a/lib/stdlib/doc/src/sys.xml
+++ b/lib/stdlib/doc/src/sys.xml
@@ -246,6 +246,22 @@
           <c>{Module, Id, HandlerState}</c>, where <c>Module</c> is the event handler's module name,
           <c>Id</c> is the handler's ID (which is the value <c>false</c> if it was registered without
           an ID), and <c>HandlerState</c> is the handler's state.</p>
+        <p>If the callback module exports a <c>system_get_state/1</c> function, it will be called in the
+          target process to get its state. Its argument is the same as the <c>Misc</c> value returned by
+          <seealso marker="#get_status-1">get_status/1,2</seealso>, and the <c>system_get_state/1</c>
+          function is expected to extract the callback module's state from it. The <c>system_get_state/1</c>
+          function must return <c>{ok, State}</c> where <c>State</c> is the callback module's state.</p>
+        <p>If the callback module does not export a <c>system_get_state/1</c> function, <c>get_state/1,2</c>
+          assumes the <c>Misc</c> value is the callback module's state and returns it directly instead.</p>
+        <p>If the callback module's <c>system_get_state/1</c> function crashes or throws an exception, the
+          caller exits with error <c>{callback_failed, {Module, system_get_state}, {Class, Reason}}</c> where
+          <c>Module</c> is the name of the callback module and <c>Class</c> and <c>Reason</c> indicate
+          details of the exception.</p>
+        <p>The <c>system_get_state/1</c> function is primarily useful for user-defined
+          behaviours and modules that implement OTP <seealso marker="#special_process">special
+          processes</seealso>. The <c>gen_server</c>, <c>gen_fsm</c>, and <c>gen_event</c> OTP
+          behaviour modules export this function, and so callback modules for those behaviours
+          need not supply their own.</p>
         <p>To obtain more information about a process, including its state, see
           <seealso marker="#get_status-1">get_status/1</seealso> and
           <seealso marker="#get_status-2">get_status/2</seealso>.</p>
@@ -289,6 +305,28 @@
           function means that only the state of the particular event handler it was working on when it
           failed or crashed is unchanged; it can still succeed in changing the states of other event
           handlers registered in the same <c>gen_event</c> process.</p>
+        <p>If the callback module exports a <c>system_replace_state/2</c> function, it will be called in the
+          target process to replace its state using <c>StateFun</c>. Its two arguments are <c>StateFun</c>
+          and <c>Misc</c>, where <c>Misc</c> is the same as the <c>Misc</c> value returned by
+          <seealso marker="#get_status-1">get_status/1,2</seealso>. A <c>system_replace_state/2</c> function
+          is expected to return <c>{ok, NewState, NewMisc}</c> where <c>NewState</c> is the callback module's
+          new state obtained by calling <c>StateFun</c>, and <c>NewMisc</c> is a possibly new value used to
+          replace the original <c>Misc</c> (required since <c>Misc</c> often contains the callback
+          module's state within it).</p>
+        <p>If the callback module does not export a <c>system_replace_state/2</c> function,
+          <c>replace_state/2,3</c> assumes the <c>Misc</c> value is the callback module's state, passes it
+          to <c>StateFun</c> and uses the return value as both the new state and as the new value of
+          <c>Misc</c>.</p>
+        <p>If the callback module's <c>system_replace_state/2</c> function crashes or throws an exception,
+          the caller exits with error <c>{callback_failed, {Module, system_replace_state}, {Class, Reason}}</c>
+          where <c>Module</c> is the name of the callback module and <c>Class</c> and <c>Reason</c> indicate details
+          of the exception. If the callback module does not provide a <c>system_replace_state/2</c> function and
+          <c>StateFun</c> crashes or throws an exception, the caller exits with error
+          <c>{callback_failed, StateFun, {Class, Reason}}</c>.</p>
+        <p>The <c>system_replace_state/2</c> function is primarily useful for user-defined behaviours and
+          modules that implement OTP <seealso marker="#special_process">special processes</seealso>. The
+          <c>gen_server</c>, <c>gen_fsm</c>, and <c>gen_event</c> OTP behaviour modules export this function,
+          and so callback modules for those behaviours need not supply their own.</p>
       </desc>
     </func>
     <func>
@@ -322,7 +360,7 @@
 
   <section>
     <title>Process Implementation Functions</title>
-    <p>The following functions are used when implementing a
+    <p><marker id="special_process"/>The following functions are used when implementing a
       special process. This is an ordinary process which does not use a
       standard behaviour, but a process which understands the standard system messages.</p>
   </section>
@@ -375,8 +413,9 @@
           process continues the execution, or
           <c><anno>Module</anno>:system_terminate(Reason, <anno>Parent</anno>, <anno>Debug</anno>, <anno>Misc</anno>)</c> if
           the process should terminate. The <c><anno>Module</anno></c> must export
-          <c>system_continue/3</c>, <c>system_terminate/4</c>, and
-          <c>system_code_change/4</c> (see below).
+          <c>system_continue/3</c>, <c>system_terminate/4</c>,
+          <c>system_code_change/4</c>, <c>system_get_state/1</c> and
+          <c>system_replace_state/2</c> (see below).
           </p>
         <p>The <c><anno>Misc</anno></c> argument can be used to save internal data
           in a process, for example its state. It is sent to
@@ -444,6 +483,34 @@
           defined, the atom <c>undefined</c> is sent.</p>
       </desc>
     </func>
+    <func>
+      <name>Mod:system_get_state(Misc) -> {ok, State}</name>
+      <fsummary>Called when the process should return its current state</fsummary>
+      <type>
+        <v>Misc = term()</v>
+        <v>State = term()</v>
+    </type>
+      <desc>
+        <p>This function is called from <c>sys:handle_system_msg/6</c> when the process
+          should return a term that reflects its current state. <c>State</c> is the
+          value returned by <c>sys:get_state/2</c>.</p>
+      </desc>
+    </func>
+    <func>
+      <name>Mod:system_replace_state(StateFun, Misc) -> {ok, NState, NMisc}</name>
+      <fsummary>Called when the process should replace its current state</fsummary>
+      <type>
+        <v>StateFun = fun((State :: term()) -> NState)</v>
+        <v>Misc = term()</v>
+        <v>NState = term()</v>
+        <v>NMisc = term()</v>
+    </type>
+      <desc>
+        <p>This function is called from <c>sys:handle_system_msg/6</c> when the process
+         should replace its current state. <c>NState</c> is the value returned by
+         <c>sys:replace_state/3</c>.</p>
+      </desc>
+    </func>
   </funcs>
 </erlref>
 
diff --git a/lib/stdlib/src/gen_event.erl b/lib/stdlib/src/gen_event.erl
index 7629e88fbf..d39dd89d3a 100644
--- a/lib/stdlib/src/gen_event.erl
+++ b/lib/stdlib/src/gen_event.erl
@@ -40,6 +40,8 @@
 	 system_continue/3,
 	 system_terminate/4,
 	 system_code_change/4,
+	 system_get_state/1,
+	 system_replace_state/2,
 	 format_status/2]).
 
 -export_type([handler/0, handler_args/0, add_handler_ret/0,
@@ -229,24 +231,6 @@ wake_hib(Parent, ServerName, MSL, Debug) ->
 
 fetch_msg(Parent, ServerName, MSL, Debug, Hib) ->
     receive
-	{system, From, get_state} ->
-	    States = [{Mod,Id,State} || #handler{module=Mod, id=Id, state=State} <- MSL],
-	    sys:handle_system_msg(get_state, From, Parent, ?MODULE, Debug,
-				  {States, [ServerName, MSL, Hib]}, Hib);
-	{system, From, {replace_state, StateFun}} ->
-	    {NMSL, NStates} =
-		lists:unzip([begin
-				 Cur = {Mod,Id,State},
-				 try
-				     NState = {Mod,Id,NS} = StateFun(Cur),
-				     {HS#handler{state=NS}, NState}
-				 catch
-				     _:_ ->
-					 {HS, Cur}
-				 end
-			     end || #handler{module=Mod, id=Id, state=State}=HS <- MSL]),
-	    sys:handle_system_msg(replace_state, From, Parent, ?MODULE, Debug,
-				  {NStates, [ServerName, NMSL, Hib]}, Hib);
 	{system, From, Req} ->
 	    sys:handle_system_msg(Req, From, Parent, ?MODULE, Debug,
 				  [ServerName, MSL, Hib],Hib);
@@ -383,6 +367,23 @@ system_code_change([ServerName, MSL, Hib], Module, OldVsn, Extra) ->
 		    MSL),
     {ok, [ServerName, MSL1, Hib]}.
 
+system_get_state([_ServerName, MSL, _Hib]) ->
+    {ok, [{Mod,Id,State} || #handler{module=Mod, id=Id, state=State} <- MSL]}.
+
+system_replace_state(StateFun, [ServerName, MSL, Hib]) ->
+    {NMSL, NStates} =
+		lists:unzip([begin
+				 Cur = {Mod,Id,State},
+				 try
+				     NState = {Mod,Id,NS} = StateFun(Cur),
+				     {HS#handler{state=NS}, NState}
+				 catch
+				     _:_ ->
+					 {HS, Cur}
+				 end
+			     end || #handler{module=Mod, id=Id, state=State}=HS <- MSL]),
+    {ok, NStates, [ServerName, NMSL, Hib]}.
+
 %%-----------------------------------------------------------------
 %% Format debug messages.  Print them as the call-back module sees
 %% them, not as the real erlang messages.  Use trace for that.
diff --git a/lib/stdlib/src/gen_fsm.erl b/lib/stdlib/src/gen_fsm.erl
index e9654322f1..e914f7d0b2 100644
--- a/lib/stdlib/src/gen_fsm.erl
+++ b/lib/stdlib/src/gen_fsm.erl
@@ -118,6 +118,8 @@
 	 system_continue/3,
 	 system_terminate/4,
 	 system_code_change/4,
+	 system_get_state/1,
+	 system_replace_state/2,
 	 format_status/2]).
 
 -import(error_logger, [format/2]).
@@ -422,17 +424,6 @@ wake_hib(Parent, Name, StateName, StateData, Mod, Debug) ->
 
 decode_msg(Msg,Parent, Name, StateName, StateData, Mod, Time, Debug, Hib) ->
     case Msg of
-	{system, From, get_state} ->
-	    Misc = [Name, StateName, StateData, Mod, Time],
-	    sys:handle_system_msg(get_state, From, Parent, ?MODULE, Debug,
-				  {{StateName, StateData}, Misc}, Hib);
-	{system, From, {replace_state, StateFun}} ->
-	    State = {StateName, StateData},
-	    NState = {NStateName, NStateData} = try StateFun(State)
-						catch _:_ -> State end,
-	    NMisc = [Name, NStateName, NStateData, Mod, Time],
-	    sys:handle_system_msg(replace_state, From, Parent, ?MODULE, Debug,
-				  {NState, NMisc}, Hib);
         {system, From, Req} ->
 	    sys:handle_system_msg(Req, From, Parent, ?MODULE, Debug,
 				  [Name, StateName, StateData, Mod, Time], Hib);
@@ -467,6 +458,13 @@ system_code_change([Name, StateName, StateData, Mod, Time],
 	Else -> Else
     end.
 
+system_get_state([_Name, StateName, StateData, _Mod, _Time]) ->
+    {ok, {StateName, StateData}}.
+
+system_replace_state(StateFun, [Name, StateName, StateData, Mod, Time]) ->
+    Result = {NStateName, NStateData} = StateFun({StateName, StateData}),
+    {ok, Result, [Name, NStateName, NStateData, Mod, Time]}.
+
 %%-----------------------------------------------------------------
 %% Format debug messages.  Print them as the call-back module sees
 %% them, not as the real erlang messages.  Use trace for that.
diff --git a/lib/stdlib/src/gen_server.erl b/lib/stdlib/src/gen_server.erl
index 5f14e48b0a..202a931fae 100644
--- a/lib/stdlib/src/gen_server.erl
+++ b/lib/stdlib/src/gen_server.erl
@@ -98,6 +98,8 @@
 -export([system_continue/3,
 	 system_terminate/4,
 	 system_code_change/4,
+	 system_get_state/1,
+	 system_replace_state/2,
 	 format_status/2]).
 
 %% Internal exports
@@ -372,13 +374,6 @@ wake_hib(Parent, Name, State, Mod, Debug) ->
 
 decode_msg(Msg, Parent, Name, State, Mod, Time, Debug, Hib) ->
     case Msg of
-	{system, From, get_state} ->
-	    sys:handle_system_msg(get_state, From, Parent, ?MODULE, Debug,
-				  {State, [Name, State, Mod, Time]}, Hib);
-	{system, From, {replace_state, StateFun}} ->
-	    NState = try StateFun(State) catch _:_ -> State end,
-	    sys:handle_system_msg(replace_state, From, Parent, ?MODULE, Debug,
-				  {NState, [Name, NState, Mod, Time]}, Hib);
 	{system, From, Req} ->
 	    sys:handle_system_msg(Req, From, Parent, ?MODULE, Debug,
 				  [Name, State, Mod, Time], Hib);
@@ -687,6 +682,13 @@ system_code_change([Name, State, Mod, Time], _Module, OldVsn, Extra) ->
 	Else -> Else
     end.
 
+system_get_state([_Name, State, _Mod, _Time]) ->
+    {ok, State}.
+
+system_replace_state(StateFun, [Name, State, Mod, Time]) ->
+    NState = StateFun(State),
+    {ok, NState, [Name, NState, Mod, Time]}.
+
 %%-----------------------------------------------------------------
 %% Format debug messages.  Print them as the call-back module sees
 %% them, not as the real erlang messages.  Use trace for that.
diff --git a/lib/stdlib/src/sys.erl b/lib/stdlib/src/sys.erl
index 04f8dfb61b..e25cc25f57 100644
--- a/lib/stdlib/src/sys.erl
+++ b/lib/stdlib/src/sys.erl
@@ -102,20 +102,31 @@ get_status(Name, Timeout) -> send_system_msg(Name, get_status, Timeout).
 -spec get_state(Name) -> State when
       Name :: name(),
       State :: term().
-get_state(Name) -> send_system_msg(Name, get_state).
+get_state(Name) ->
+    case send_system_msg(Name, get_state) of
+	{error, Reason} -> error(Reason);
+	State -> State
+    end.
 
 -spec get_state(Name, Timeout) -> State when
       Name :: name(),
       Timeout :: timeout(),
       State :: term().
-get_state(Name, Timeout) -> send_system_msg(Name, get_state, Timeout).
+get_state(Name, Timeout) ->
+    case send_system_msg(Name, get_state, Timeout) of
+	{error, Reason} -> error(Reason);
+	State -> State
+    end.
 
 -spec replace_state(Name, StateFun) -> NewState when
       Name :: name(),
       StateFun :: fun((State :: term()) -> NewState :: term()),
       NewState :: term().
 replace_state(Name, StateFun) ->
-    send_system_msg(Name, {replace_state, StateFun}).
+    case send_system_msg(Name, {replace_state, StateFun}) of
+	{error, Reason} -> error(Reason);
+	State -> State
+    end.
 
 -spec replace_state(Name, StateFun, Timeout) -> NewState when
       Name :: name(),
@@ -123,7 +134,10 @@ replace_state(Name, StateFun) ->
       Timeout :: timeout(),
       NewState :: term().
 replace_state(Name, StateFun, Timeout) ->
-    send_system_msg(Name, {replace_state, StateFun}, Timeout).
+    case send_system_msg(Name, {replace_state, StateFun}, Timeout) of
+	{error, Reason} -> error(Reason);
+	State -> State
+    end.
 
 -spec change_code(Name, Module, OldVsn, Extra) -> 'ok' | {error, Reason} when
       Name :: name(),
@@ -390,10 +404,11 @@ do_cmd(_, suspend, _Parent, _Mod, Debug, Misc) ->
     {suspended, ok, Debug, Misc};
 do_cmd(_, resume, _Parent, _Mod, Debug, Misc) ->
     {running, ok, Debug, Misc};
-do_cmd(SysState, get_state, _Parent, _Mod, Debug, {State, Misc}) ->
-    {SysState, State, Debug, Misc};
-do_cmd(SysState, replace_state, _Parent, _Mod, Debug, {State, Misc}) ->
-    {SysState, State, Debug, Misc};
+do_cmd(SysState, get_state, _Parent, Mod, Debug, Misc) ->
+    {SysState, do_get_state(Mod, Misc), Debug, Misc};
+do_cmd(SysState, {replace_state, StateFun},  _Parent, Mod, Debug, Misc) ->
+    {Res, NMisc} = do_replace_state(StateFun, Mod, Misc),
+    {SysState, Res, Debug, NMisc};
 do_cmd(SysState, get_status, Parent, Mod, Debug, Misc) ->
     Res = get_status(SysState, Parent, Mod, Debug, Misc),
     {SysState, Res, Debug, Misc};
@@ -407,6 +422,40 @@ do_cmd(suspended, {change_code, Module, Vsn, Extra}, _Parent,
 do_cmd(SysState, Other, _Parent, _Mod, Debug, Misc) ->
     {SysState, {error, {unknown_system_msg, Other}}, Debug, Misc}.
 
+do_get_state(Mod, Misc) ->
+    case erlang:function_exported(Mod, system_get_state, 1) of
+	true ->
+	    try
+		{ok, State} = Mod:system_get_state(Misc),
+		State
+	    catch
+		Cl:Exc ->
+		    {error, {callback_failed,{Mod,system_get_state},{Cl,Exc}}}
+	    end;
+	false ->
+	    Misc
+    end.
+
+do_replace_state(StateFun, Mod, Misc) ->
+    case erlang:function_exported(Mod, system_replace_state, 2) of
+	true ->
+	    try
+		{ok, State, NMisc} = Mod:system_replace_state(StateFun, Misc),
+		{State, NMisc}
+	    catch
+		Cl:Exc ->
+		    {{error, {callback_failed,{Mod,system_replace_state},{Cl,Exc}}}, Misc}
+	    end;
+	false ->
+	    try
+		NMisc = StateFun(Misc),
+		{NMisc, NMisc}
+	    catch
+		Cl:Exc ->
+		    {{error, {callback_failed,StateFun,{Cl,Exc}}}, Misc}
+	    end
+    end.
+
 get_status(SysState, Parent, Mod, Debug, Misc) ->
     PDict = get(),
     FmtMisc =
diff --git a/lib/stdlib/test/Makefile b/lib/stdlib/test/Makefile
index af82f22b21..39f6ce423a 100644
--- a/lib/stdlib/test/Makefile
+++ b/lib/stdlib/test/Makefile
@@ -73,6 +73,8 @@ MODULES= \
 	supervisor_SUITE \
 	supervisor_bridge_SUITE \
 	sys_SUITE \
+	sys_sp1 \
+	sys_sp2 \
 	tar_SUITE \
 	timer_SUITE \
 	timer_simple_SUITE \
diff --git a/lib/stdlib/test/gen_event_SUITE.erl b/lib/stdlib/test/gen_event_SUITE.erl
index 5819ef3890..60a1ba8c60 100644
--- a/lib/stdlib/test/gen_event_SUITE.erl
+++ b/lib/stdlib/test/gen_event_SUITE.erl
@@ -974,6 +974,10 @@ get_state(Config) when is_list(Config) ->
     [{dummy1_h,false,State1},{dummy1_h,id,State2}] = lists:sort(Result1),
     Result2 = sys:get_state(Pid, 5000),
     [{dummy1_h,false,State1},{dummy1_h,id,State2}] = lists:sort(Result2),
+    ok = sys:suspend(Pid),
+    Result3 = sys:get_state(Pid),
+    [{dummy1_h,false,State1},{dummy1_h,id,State2}] = lists:sort(Result3),
+    ok = sys:resume(Pid),
     ok = gen_event:stop(Pid),
     ok.
 
@@ -998,4 +1002,11 @@ replace_state(Config) when is_list(Config) ->
     Replace3 = fun(_) -> exit(fail) end,
     [{dummy1_h,false,NState2}] = sys:replace_state(Pid, Replace3),
     [{dummy1_h,false,NState2}] = sys:get_state(Pid),
+    %% verify state replaced if process sys suspended
+    NState3 = "replaced again and again",
+    Replace4 = fun({dummy1_h,false,_}=S) -> setelement(3,S,NState3) end,
+    ok = sys:suspend(Pid),
+    [{dummy1_h,false,NState3}] = sys:replace_state(Pid, Replace4),
+    ok = sys:resume(Pid),
+    [{dummy1_h,false,NState3}] = sys:get_state(Pid),
     ok.
diff --git a/lib/stdlib/test/gen_fsm_SUITE.erl b/lib/stdlib/test/gen_fsm_SUITE.erl
index fd15838b7d..8aeec07ae8 100644
--- a/lib/stdlib/test/gen_fsm_SUITE.erl
+++ b/lib/stdlib/test/gen_fsm_SUITE.erl
@@ -426,6 +426,14 @@ get_state(Config) when is_list(Config) ->
     {idle, State} = sys:get_state(gfsm),
     {idle, State} = sys:get_state(gfsm, 5000),
     stop_it(Pid2),
+
+    %% check that get_state works when pid is sys suspended
+    {ok, Pid3} = gen_fsm:start(gen_fsm_SUITE, {state_data, State}, []),
+    {idle, State} = sys:get_state(Pid3),
+    ok = sys:suspend(Pid3),
+    {idle, State} = sys:get_state(Pid3, 5000),
+    ok = sys:resume(Pid3),
+    stop_it(Pid3),
     ok.
 
 replace_state(Config) when is_list(Config) ->
@@ -442,8 +450,18 @@ replace_state(Config) when is_list(Config) ->
     {state0, NState2} = sys:get_state(Pid),
     %% verify no change in state if replace function crashes
     Replace3 = fun(_) -> error(fail) end,
-    {state0, NState2} = sys:replace_state(Pid, Replace3),
+    {'EXIT',{{callback_failed,
+	      {gen_fsm,system_replace_state},{error,fail}},_}} =
+	(catch sys:replace_state(Pid, Replace3)),
     {state0, NState2} = sys:get_state(Pid),
+    %% verify state replaced if process sys suspended
+    ok = sys:suspend(Pid),
+    Suffix2 = " and again",
+    NState3 = NState2 ++ Suffix2,
+    Replace4 = fun({StateName, _}) -> {StateName, NState3} end,
+    {state0, NState3} = sys:replace_state(Pid, Replace4),
+    ok = sys:resume(Pid),
+    {state0, NState3} = sys:get_state(Pid, 5000),
     stop_it(Pid),
     ok.
 
diff --git a/lib/stdlib/test/gen_server_SUITE.erl b/lib/stdlib/test/gen_server_SUITE.erl
index a360a0809b..960e7f60e7 100644
--- a/lib/stdlib/test/gen_server_SUITE.erl
+++ b/lib/stdlib/test/gen_server_SUITE.erl
@@ -1049,6 +1049,9 @@ get_state(Config) when is_list(Config) ->
     {ok, Pid} = gen_server:start_link(?MODULE, {state,State}, []),
     State = sys:get_state(Pid),
     State = sys:get_state(Pid, 5000),
+    ok = sys:suspend(Pid),
+    State = sys:get_state(Pid),
+    ok = sys:resume(Pid),
     ok.
 
 %% Verify that sys:replace_state correctly replaces gen_server state
@@ -1075,8 +1078,18 @@ replace_state(Config) when is_list(Config) ->
     NState2 = sys:get_state(Pid, 5000),
     %% verify no change in state if replace function crashes
     Replace3 = fun(_) -> throw(fail) end,
-    NState2 = sys:replace_state(Pid, Replace3),
+    {'EXIT',{{callback_failed,
+	      {gen_server,system_replace_state},{throw,fail}},_}} =
+	(catch sys:replace_state(Pid, Replace3)),
     NState2 = sys:get_state(Pid, 5000),
+    %% verify state replaced if process sys suspended
+    ok = sys:suspend(Pid),
+    Suffix2 = " and again",
+    NState3 = NState2 ++ Suffix2,
+    Replace4 = fun(S) -> S ++ Suffix2 end,
+    NState3 = sys:replace_state(Pid, Replace4),
+    ok = sys:resume(Pid),
+    NState3 = sys:get_state(Pid, 5000),
     ok.
 
 %% Test that the time for a huge message queue is not
diff --git a/lib/stdlib/test/sys_SUITE.erl b/lib/stdlib/test/sys_SUITE.erl
index c06ba545e7..f38bc87ae5 100644
--- a/lib/stdlib/test/sys_SUITE.erl
+++ b/lib/stdlib/test/sys_SUITE.erl
@@ -19,7 +19,7 @@
 -module(sys_SUITE).
 -export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, 
 	 init_per_group/2,end_per_group/2,log/1,log_to_file/1,
-	 stats/1,trace/1,suspend/1,install/1]).
+	 stats/1,trace/1,suspend/1,install/1,special_process/1]).
 -export([handle_call/3,terminate/2,init/1]).
 -include_lib("test_server/include/test_server.hrl").
 
@@ -27,14 +27,12 @@
 
 
 %% Doesn't look into change_code at all
-%% Doesn't address writing your own process that understands
-%% system messages at all.
 
 
 suite() -> [{ct_hooks,[ts_install_cth]}].
 
 all() -> 
-    [log, log_to_file, stats, trace, suspend, install].
+    [log, log_to_file, stats, trace, suspend, install, special_process].
 
 groups() -> 
     [].
@@ -157,6 +155,84 @@ install(Config) when is_list(Config) ->
     ?line stop(),
     ok.
 
+special_process(suite) -> [];
+special_process(Config) when is_list(Config) ->
+    ok = spec_proc(sys_sp1),
+    ok = spec_proc(sys_sp2).
+
+spec_proc(Mod) ->
+    {ok,_} = Mod:start_link(100),
+    ok = sys:statistics(Mod,true),
+    ok = sys:trace(Mod,true),
+    1 = Ch = Mod:alloc(),
+    Free = lists:seq(2,100),
+    Replace = case sys:get_state(Mod) of
+		  {[Ch],Free} ->
+		      fun({A,F}) ->
+			      Free = F,
+			      {A,[2,3,4]}
+		      end;
+		  {state,[Ch],Free} ->
+		      fun({state,A,F}) ->
+			      Free = F,
+			      {state,A,[2,3,4]}
+		      end
+	      end,
+    case sys:replace_state(Mod, Replace) of
+	{[Ch],[2,3,4]} -> ok;
+	{state,[Ch],[2,3,4]} -> ok
+    end,
+    ok = Mod:free(Ch),
+    case sys:get_state(Mod) of
+	{[],[1,2,3,4]} -> ok;
+	{state,[],[1,2,3,4]} -> ok
+    end,
+    {ok,[{start_time,_},
+	 {current_time,_},
+	 {reductions,_},
+	 {messages_in,2},
+	 {messages_out,1}]} = sys:statistics(Mod,get),
+    ok = sys:statistics(Mod,false),
+    [] = sys:replace_state(Mod, fun(_) -> [] end),
+    process_flag(trap_exit,true),
+    ok = case catch sys:get_state(Mod) of
+	     [] ->
+		 ok;
+	     {'EXIT',{{callback_failed,
+		       {Mod,system_get_state},{throw,fail}},_}} ->
+		 ok
+	 end,
+    Mod:stop(),
+    WaitForUnregister = fun W() ->
+		       case whereis(Mod) of
+			   undefined -> ok;
+			   _ -> timer:sleep(10), W()
+		       end
+	       end,
+    WaitForUnregister(),
+    {ok,_} = Mod:start_link(4),
+    ok = case catch sys:replace_state(Mod, fun(_) -> {} end) of
+	     {} ->
+		 ok;
+	     {'EXIT',{{callback_failed,
+		       {Mod,system_replace_state},{throw,fail}},_}} ->
+		 ok
+	 end,
+    Mod:stop(),
+    WaitForUnregister(),
+    {ok,_} = Mod:start_link(4),
+    StateFun = fun(_) -> error(fail) end,
+    ok = case catch sys:replace_state(Mod, StateFun) of
+	     {} ->
+		 ok;
+	     {'EXIT',{{callback_failed,
+		       {Mod,system_replace_state},{error,fail}},_}} ->
+		 ok;
+	     {'EXIT',{{callback_failed,StateFun,{error,fail}},_}} ->
+		 ok
+	 end,
+    Mod:stop().
+
 %%%%%%%%%%%%%%%%%%%%
 %% Dummy server
 
diff --git a/lib/stdlib/test/sys_sp1.erl b/lib/stdlib/test/sys_sp1.erl
new file mode 100644
index 0000000000..e84ffcfa12
--- /dev/null
+++ b/lib/stdlib/test/sys_sp1.erl
@@ -0,0 +1,114 @@
+%%
+%% %CopyrightBegin%
+%%
+%% Copyright Ericsson AB 1996-2013. All Rights Reserved.
+%%
+%% The contents of this file are subject to the Erlang Public License,
+%% Version 1.1, (the "License"); you may not use this file except in
+%% compliance with the License. You should have received a copy of the
+%% Erlang Public License along with this software. If not, it can be
+%% retrieved online at http://www.erlang.org/.
+%%
+%% Software distributed under the License is distributed on an "AS IS"
+%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
+%% the License for the specific language governing rights and limitations
+%% under the License.
+%%
+%% %CopyrightEnd%
+%%
+-module(sys_sp1).
+-export([start_link/1, stop/0]).
+-export([alloc/0, free/1]).
+-export([init/1]).
+-export([system_continue/3, system_terminate/4,
+         write_debug/3,
+         system_get_state/1, system_replace_state/2]).
+
+%% Implements the ch4 example from the Design Principles doc.  Same as
+%% sys_sp2 except this module exports system_get_state/1 and
+%% system_replace_state/2
+
+start_link(NumCh) ->
+    proc_lib:start_link(?MODULE, init, [[self(),NumCh]]).
+
+stop() ->
+    ?MODULE ! stop,
+    ok.
+
+alloc() ->
+    ?MODULE ! {self(), alloc},
+    receive
+        {?MODULE, Res} ->
+            Res
+    end.
+
+free(Ch) ->
+    ?MODULE ! {free, Ch},
+    ok.
+
+init([Parent,NumCh]) ->
+    register(?MODULE, self()),
+    Chs = channels(NumCh),
+    Deb = sys:debug_options([]),
+    proc_lib:init_ack(Parent, {ok, self()}),
+    loop(Chs, Parent, Deb).
+
+loop(Chs, Parent, Deb) ->
+    receive
+        {From, alloc} ->
+            Deb2 = sys:handle_debug(Deb, fun write_debug/3,
+                                    ?MODULE, {in, alloc, From}),
+            {Ch, Chs2} = alloc(Chs),
+            From ! {?MODULE, Ch},
+            Deb3 = sys:handle_debug(Deb2, fun write_debug/3,
+                                    ?MODULE, {out, {?MODULE, Ch}, From}),
+            loop(Chs2, Parent, Deb3);
+        {free, Ch} ->
+            Deb2 = sys:handle_debug(Deb, fun write_debug/3,
+                                    ?MODULE, {in, {free, Ch}}),
+            Chs2 = free(Ch, Chs),
+            loop(Chs2, Parent, Deb2);
+        {system, From, Request} ->
+            sys:handle_system_msg(Request, From, Parent,
+                                  ?MODULE, Deb, Chs);
+        stop ->
+            sys:handle_debug(Deb, fun write_debug/3,
+                             ?MODULE, {in, stop}),
+            ok
+    end.
+
+system_continue(Parent, Deb, Chs) ->
+    loop(Chs, Parent, Deb).
+
+system_terminate(Reason, _Parent, _Deb, _Chs) ->
+    exit(Reason).
+
+system_get_state([]) ->
+    throw(fail);
+system_get_state(Chs) ->
+    {ok, Chs}.
+
+system_replace_state(_StateFun, {}) ->
+    throw(fail);
+system_replace_state(StateFun, Chs) ->
+    NChs = StateFun(Chs),
+    {ok, NChs, NChs}.
+
+write_debug(Dev, Event, Name) ->
+    io:format(Dev, "~p event = ~p~n", [Name, Event]).
+
+channels(NumCh) ->
+    {_Allocated=[], _Free=lists:seq(1,NumCh)}.
+
+alloc({_, []}) ->
+    {error, "no channels available"};
+alloc({Allocated, [H|T]}) ->
+    {H, {[H|Allocated], T}}.
+
+free(Ch, {Alloc, Free}=Channels) ->
+    case lists:member(Ch, Alloc) of
+        true ->
+            {lists:delete(Ch, Alloc), [Ch|Free]};
+        false ->
+            Channels
+    end.
diff --git a/lib/stdlib/test/sys_sp2.erl b/lib/stdlib/test/sys_sp2.erl
new file mode 100644
index 0000000000..56a5e4d071
--- /dev/null
+++ b/lib/stdlib/test/sys_sp2.erl
@@ -0,0 +1,107 @@
+%%
+%% %CopyrightBegin%
+%%
+%% Copyright Ericsson AB 1996-2013. All Rights Reserved.
+%%
+%% The contents of this file are subject to the Erlang Public License,
+%% Version 1.1, (the "License"); you may not use this file except in
+%% compliance with the License. You should have received a copy of the
+%% Erlang Public License along with this software. If not, it can be
+%% retrieved online at http://www.erlang.org/.
+%%
+%% Software distributed under the License is distributed on an "AS IS"
+%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
+%% the License for the specific language governing rights and limitations
+%% under the License.
+%%
+%% %CopyrightEnd%
+%%
+-module(sys_sp2).
+-export([start_link/1, stop/0]).
+-export([alloc/0, free/1]).
+-export([init/1]).
+-export([system_continue/3, system_terminate/4,
+         write_debug/3]).
+
+%% Implements the ch4 example from the Design Principles doc.  Same as
+%% sys_sp1 except this module does not export system_get_state/1 or
+%% system_replace_state/2
+
+start_link(NumCh) ->
+    proc_lib:start_link(?MODULE, init, [[self(),NumCh]]).
+
+stop() ->
+    ?MODULE ! stop,
+    ok.
+
+alloc() ->
+    ?MODULE ! {self(), alloc},
+    receive
+        {?MODULE, Res} ->
+            Res
+    end.
+
+free(Ch) ->
+    ?MODULE ! {free, Ch},
+    ok.
+
+%% can't use 2-tuple for state here as we do in sys_sp1, since the 2-tuple
+%% is not compatible with the backward compatibility handling for
+%% sys:get_state in sys.erl
+-record(state, {alloc,free}).
+
+init([Parent,NumCh]) ->
+    register(?MODULE, self()),
+    Chs = channels(NumCh),
+    Deb = sys:debug_options([]),
+    proc_lib:init_ack(Parent, {ok, self()}),
+    loop(Chs, Parent, Deb).
+
+loop(Chs, Parent, Deb) ->
+    receive
+        {From, alloc} ->
+            Deb2 = sys:handle_debug(Deb, fun write_debug/3,
+                                    ?MODULE, {in, alloc, From}),
+            {Ch, Chs2} = alloc(Chs),
+            From ! {?MODULE, Ch},
+            Deb3 = sys:handle_debug(Deb2, fun write_debug/3,
+                                    ?MODULE, {out, {?MODULE, Ch}, From}),
+            loop(Chs2, Parent, Deb3);
+        {free, Ch} ->
+            Deb2 = sys:handle_debug(Deb, fun write_debug/3,
+                                    ?MODULE, {in, {free, Ch}}),
+            Chs2 = free(Ch, Chs),
+            loop(Chs2, Parent, Deb2);
+        {system, From, Request} ->
+            sys:handle_system_msg(Request, From, Parent,
+                                  ?MODULE, Deb, Chs);
+        stop ->
+            sys:handle_debug(Deb, fun write_debug/3,
+                             ?MODULE, {in, stop}),
+            ok
+    end.
+
+system_continue(Parent, Deb, Chs) ->
+    loop(Chs, Parent, Deb).
+
+system_terminate(Reason, _Parent, _Deb, _Chs) ->
+    exit(Reason).
+
+write_debug(Dev, Event, Name) ->
+    io:format(Dev, "~p event = ~p~n", [Name, Event]).
+
+channels(NumCh) ->
+    #state{alloc=[], free=lists:seq(1,NumCh)}.
+
+alloc(#state{free=[]}=Channels) ->
+    {{error, "no channels available"}, Channels};
+alloc(#state{alloc=Allocated, free=[H|T]}) ->
+    {H, #state{alloc=[H|Allocated], free=T}}.
+
+free(Ch, #state{alloc=Alloc, free=Free}=Channels) ->
+    case lists:member(Ch, Alloc) of
+        true ->
+            #state{alloc=lists:delete(Ch, Alloc), free=[Ch|Free]};
+        false ->
+            Channels
+    end.
diff --git a/system/doc/design_principles/spec_proc.xml b/system/doc/design_principles/spec_proc.xml
index 8de7a5fe03..d55427aa10 100644
--- a/system/doc/design_principles/spec_proc.xml
+++ b/system/doc/design_principles/spec_proc.xml
@@ -130,7 +130,8 @@ ok
 -export([alloc/0, free/1]).
 -export([init/1]).
 -export([system_continue/3, system_terminate/4,
-         write_debug/3]).
+         write_debug/3,
+         system_get_state/1, system_replace_state/2]).
 
 start_link() ->
     proc_lib:start_link(ch4, init, [self()]).
@@ -177,9 +178,21 @@ loop(Chs, Parent, Deb) ->
 system_continue(Parent, Deb, Chs) ->
     loop(Chs, Parent, Deb).
 
-system_terminate(Reason, Parent, Deb, Chs) ->
+system_terminate(Reason, _Parent, _Deb, _Chs) ->
     exit(Reason).
 
+system_get_state(Chs) ->
+    {ok, Chs}.
+
+system_replace_state(StateFun, Chs) ->
+    try
+        NChs = StateFun(Chs),
+        {ok, NChs, NChs}
+    catch
+        _:_ ->
+            {ok, Chs, Chs}
+    end.
+
 write_debug(Dev, Event, Name) ->
     io:format(Dev, "~p event = ~p~n", [Name, Event]).</pre>
       <p>Example on how the simple debugging functions in <c>sys</c> can
@@ -366,8 +379,15 @@ Module:system_terminate(Reason, Parent, Deb, State)</code>
         <item><c>Module</c> is the name of the module.</item>
         <item><c>Deb</c> is the debug structure.</item>
         <item><c>State</c> is a term describing the internal state and
-         is passed to <c>system_continue</c>/<c>system_terminate</c>.</item>
+         is passed to <c>system_continue</c>/<c>system_terminate</c>/
+         <c>system_get_state</c>/<c>system_replace_state</c>.</item>
       </list>
+      <p>If the process should return its state <c>handle_system_msg</c> will call:</p>
+      <code type="none">
+Module:system_get_state(State)</code>
+      <p>or if the process should replace its state using the fun <c>StateFun</c>:</p>
+      <code type="none">
+Module:system_replace_state(StateFun, State)</code>
       <p>In the example:</p>
       <code type="none">
 loop(Chs, Parent, Deb) ->
@@ -383,7 +403,19 @@ system_continue(Parent, Deb, Chs) ->
     loop(Chs, Parent, Deb).
 
 system_terminate(Reason, Parent, Deb, Chs) ->
-    exit(Reason).</code>
+    exit(Reason).
+
+system_get_state(Chs) ->
+    {ok, Chs, Chs}.
+
+system_replace_state(StateFun, Chs) ->
+    try
+        NChs = StateFun(Chs),
+        {ok, NChs, NChs}
+    catch
+        _:_ ->
+            {ok, Chs, Chs}
+    end.</code>
       <p>If the special process is set to trap exits, note that if
         the parent process terminates, the expected behavior is to
         terminate with the same reason:</p>