Optimize continuation pointer management

The BEAM instructions for calling a function don't save the
continuation pointer (return address) on the stack, but to a special
BEAM register called CP. It is the responsibility of the called
function to save CP to the stack frame before calling other functions.

In the earlier implementations of BEAM on Sparc, CP was located in a
CPU register. That meant that the continuation pointer was never
written to memory when calling simple functions that didn't call
other functions at all or ended in a tail-call to another function.

The modern BEAM no longer keeps CP in CPU register. Instead, it is
kept in the `process` struct (in `p->cp`). That means the continuation
pointer must be written to the memory on every call, and if the called
function will call other functions, it will must read the continuation
pointer from `p->cp` and store it on the stack.

This commit eliminates the concept of the CP register and modifies
the call instructions to directly store the continuation pointer on
the stack. That makes allocation and trimming of stack frames slightly
faster. A more important benefit is simplification of code that handles
continuation pointers. Because all continuation pointers are now stored
on the stack, the special case of handling `p->cp` disappears.

Co-authored-by: John Högberg <[email protected]>

1 files changed, 42 insertions, 4 deletions

diff --git a/erts/emulator/beam/macros.tab b/erts/emulator/beam/macros.tab
index 1b5e5f66b0..9d183e1f41 100644
--- a/erts/emulator/beam/macros.tab
+++ b/erts/emulator/beam/macros.tab
@@ -104,14 +104,52 @@ GC_TEST_PRESERVE(NeedHeap, Live, PreserveTerm) {
 
 
 // Make sure that there are NeedStack + NeedHeap + 1 words available
-// on the combined heap/stack segment, then allocates NeedHeap + 1
-// words on the stack and saves CP.
+// on the combined heap/stack segment, then decrement the stack
+// pointer by (NeedStack + 1) words. Finally clear the word reserved
+// for the continuation pointer at the top of the stack.
+//
+// Stack frame layout:
+//
+//       +-----------+
+// y(N)  | Term      |
+//       +-----------+
+//            .
+//            .
+//            .
+//       +-----------+
+// y(0)  | Term      |
+//       +-----------+
+// E ==> | NIL or CP |
+//       +-----------+
+//
+// When the function owning the stack frame is the currently executing
+// function, the word at the top of the stack is NIL. When calling
+// another function, the continuation pointer will be stored in the
+// word at the top of the stack. When returning to the function
+// owning the stack frame, the word at the stack top will again be set
+// to NIL.
+
 AH(NeedStack, NeedHeap, Live) {
     unsigned needed = $NeedStack + 1;
     $GC_TEST(needed, $NeedHeap, $Live);
     E -= needed;
-    *E = make_cp(c_p->cp);
-    c_p->cp = 0;
+    *E = NIL;
+}
+
+// Save the continuation pointer in the reserved slot at the
+// top of the stack as preparation for doing a function call.
+
+SAVE_CONTINUATION_POINTER(IP) {
+    ASSERT(VALID_INSTR(*($IP)));
+    *E = (BeamInstr) ($IP);
+}
+
+// Return to the function whose continuation pointer is stored
+// at the top of the stack and set that word to NIL.
+
+RETURN() {
+    SET_I(cp_val(*E));
+    *E = NIL;
 }
 
 NEXT0() {