In our engineering post last month, I talked about improving Go’s panic() on Windows by hand-writing assembly. In this post, we’ll dive deeper into how to properly write assembly code in a Go application.

In case you missed last month’s article, I wanted to call some C functions from Go. Normally, I would have used CGo for this task, but for various reasons, CGo was unavailable. I thus needed to glue Go and C together using assembly code written by hand.

Instead of showing you the final, production-ready assembly listing, we are going to start with a naïve implementation and fix one bug at a time. Let’s go!

Attempt #1: Prototype (x64)

Let’s focus on 64-bit x86—which Go calls amd64 and which Windows calls x64. This is the most popular architecture for Windows applications.

Our goal is to call a C function. I want to start slow and steady, so let’s pick an easy function: IsDebuggerPresent() from the Windows API:

// This is what the function looks like in C:
typedef int BOOL;
BOOL IsDebuggerPresent() {
  /* ... */
}

// This is what the function looks like in CGo:
/*
#include <Windows.h>
*/
import "C"
func IsDebuggerPresent() C.BOOL {
  /* ... */
}

My task is to call this function without CGo. That means writing assembly code. Here’s the first draft:

// main.go
package main

import (
  "syscall"
)

// (1a)
var func_IsDebuggerPresent uintptr
// (2a)
func cIsDebuggerPresent() int32

func main() {
  // (1b)
  kernel32, _ := syscall.LoadLibrary("kernel32.dll")
  func_IsDebuggerPresent, _ = syscall.GetProcAddress(
    kernel32, "IsDebuggerPresent")

  if cIsDebuggerPresent() == 0 {
    print("Debugger is not present.\n")
  } else {
    print("Debugger is present!\n")
  }
}

// asm_amd64.s

// (2b)
// func cIsDebuggerPresent() int32
TEXT ·cIsDebuggerPresent(SB),0,$0-4
  // (3)
  MOVQ ·func_IsDebuggerPresent(SB), BX
  CALL BX
  // (4)
  MOVQ AX, ret+0(FP)
  RET

Let’s break this down step by step. First, we need to load the IsDebuggerPresent() function ((1a) and (1b)):

// (1a)
var func_IsDebuggerPresent uintptr

func main() {
  // (1b)
  kernel32, _ := syscall.LoadLibrary("kernel32.dll")
  func_IsDebuggerPresent, _ = syscall.GetProcAddress(
    kernel32, "IsDebuggerPresent")
}

We load the DLL that has the code for IsDebuggerPresent() and get a pointer to the IsDebuggerPresent() C function. (kernel32.dll is actually pre-loaded for every Windows program, but we still need to get a handle to it in order to call syscall.GetProcAddress().) This approach was discussed in last month’s article.

Next, we expose our assembly function to the Go code ((2a) and (2b)):

// main.go
// (2a)
func cIsDebuggerPresent() int32

// asm_amd64.s
// (2b)
TEXT ·cIsDebuggerPresent(SB),0,$0-4

What does this line with TEXT mean? Let’s break it down:

Our function does not need to use any stack space, so we write “0” for the frame size. Our function returns an int32, which consumes four bytes, so we write “4” for the caller frame size (params and returns).

Next up is the call to the C function (3):

  // (3)
  MOVQ ·func_IsDebuggerPresent(SB), BX
  CALL BX

This MOVQ instruction loads 8 bytes of memory from the global variable named func_IsDebuggerPresent into the rbx x86 register. Why do we load into rbx specifically? No particular reason; it’s an unused general-purpose register.

rbx is the 64-bit version of the bx register. Go assembly puts the register size in the instruction name (MOVQ, not MOV) and not in the register name (BX, not RBX). In AT&T assembly style (used by GCC’s tools), you might write this instruction as movq func_IsDebuggerPresent(%rip), %rbx. In Intel assembly style (used by Microsoft’s tools), you might write this instruction as mov rbx, qword ptr func_IsDebuggerPresent[rip].

The following CALL BX instruction jumps to the function pointed to by the rbx register. That would be the IsDebuggerPresent() function.

When the call completes, IsDebuggerPresent() will store its return value in the rax register. (This is specified in the Windows x64 calling convention documentation.) Our final step is to return the result back to Go code (4):

  // (4)
  MOVL AX, ret+0(FP)
  RET

This MOVQ instruction copies rax (IsDebuggerPresent()‘s return value) into offset +0 of the caller’s frame (FP). The caller’s frame contains our function’s parameters and return values. Because we only have one return value and no parameters, our return value is at offset +0. ret, written before the offset, is used by tools such as go vet to check that we got the right offset. (See Go’s assembler guide for details.)

The final instruction, RET, returns from our assembly function to the calling Go code. This is like return in Go.

Testing the Prototype

Let’s try out our program! First, let’s test without a debugger:

PS> go build .
PS> .\example.exe
Debugger is not present.

Perfect! No crashes. =]

Let’s test with a debugger. I’m using GDB as the debugger because it lets me easily see the program’s print() output:

PS> gdb ./example.exe
(gdb) run
Starting program: example.exe
Debugger is present!
[Inferior 1 exited normally]

Success!

Attempt #2: The Right Stack

Our call-C-from-Go code is too simple. We’re calling a function—IsDebuggerPresent()—that barely does anything. What happens if we start calling real code?

The first issue I ran into was a stack overflow. In order to reproduce the bug, two things had to happen:

1. Call the C function from a new goroutine (not the main goroutine)
2. The C function had to use a decent amount of stack space (e.g. 64 kilobytes)

Even with these conditions, the stack overflow happened occasionally, not every time. What’s happening?

A stack in the context of a stack overflow is the place where local variables and function returns live. The Go runtime has two kinds of stacks: normal Go stacks and system stacks.

Normal Go stacks are small (at least 2 kilobytes, often less than 8 kilobytes) but they dynamically grow. Dynamic growth happens with calls to runtime.morestack() that are inserted by the Go compiler. But the C compiler (who knows nothing about Go) will never insert calls to Go’s runtime.morestack()!

That’s where system stacks come in. C programs often expect larger stacks to be allocated, such as 512 kilobytes or 4 megabytes. A C function expecting a megabyte of stack won’t run on a tiny Go stack. System stacks are larger stacks designed specifically for such C function. System stacks do not grow dynamically (they are a fixed size) so a call to runtime.morestack() or a similar function is not necessary.

Before CGo calls a C function, it switches the OS thread to a system stack. When the C function is finished, it switches OS thread back to the normal Go stack. We need to do the same when we call our C function.

To switch to the system stack, we have two options:

1. 1. Write a Go function that calls our assembly function. Annotate the Go function with the //go:systemstack directive. This will cause the Go compiler to generate the required stack-switching code for us.
   2. Call the runtime.systemstack() function directly.

It turns out that option #1 won’t work. If we try to use //go:systemstack, we get a compilation error:

main.go:35:3: //go:systemstack only allowed in runtime

Shucks. We are not writing code inside the Go runtime, so we are not allowed to use the magic //go:systemstack.

How do we call runtime.systemstack() direct? Its Go signature is as follows:

func systemstack(f func ())

Unfortunately, because this is a private function, we cannot call it from Go. But we can call it from assembly!

We could take the easy path of writing a function in assembly code whose only job is to call runtime.systemstack(), then we’d write Go code to create the func (). But this is a tutorial on assembly, so let’s do it the hard way: writing everything in assembly.

We are going to write two functions in assembly: One function that calls runtime.systemstack() and deals with parameters and returns, and another function that runs on the system stack and that calls our desired C function.

Switching to the System Stack the Hard Way

In order to call runtime.systemstack(), we need to pass a func (). Under the covers, a func () is a pointer to a closure. A closure is a struct whose first field is a pointer to the function’s code. Only the first field is required; we can add whatever other fields we want after it.

Let’s define our closure struct:

// type cIsDebuggerPresent_Closure struct {
//   // Pointer to cIsDebuggerPresent_Trampoline().
//   // (We'll talk about it later.)
//   // offset 0, size 8
//   f unsafe.Pointer
//   // IsDebuggerPresent()'s return value.
//   // offset 8, size 4
//   result uint32
//   // Size: 8+4 = 12
// }

We aren’t going to use this struct from Go. Instead, we’re going to write assembly code that uses this struct. That’s why I wrote the type definition in a comment, and that’s why I wrote the memory offset and size for each field.

Before we dive into the assembly code, let’s think carefully about memory allocation. We need to call runtime.systemstack() with a pointer to our cIsDebuggerPresent_Closure struct. We need to allocate memory for this closure struct, so let’s put it on the stack. Because runtime.systemstack() expects its argument to be passed via the stack, we also need to allocate memory for the func () (i.e. the pointer to that closure struct) on the stack. This argument to runtime.systemstack() (or any other function we call) must be on the bottom of the stack (i.e. have the lowest address). Here is a visualization of what our stack frame layout should be:

// type StackFrame struct {
//   // Argument to runtime.systemstack().
//   // A pointer to StackFrame.closure.
//   // offset 0, size 8
//   arg *cIsDebuggerPresent_Closure
//
//   // Our closure.
//   // offset 8, size 12
//   closure cIsDebuggerPresent_Closure
//   // Size: 8+12 = 20
// }

Let’s expand this struct to only mention basic data types:

// type StackFrame struct {
//   // Argument to runtime.systemstack().
//   // offset 0, size 8
//   arg unsafe.Pointer
//   // Pointer to cIsDebuggerPresent_Trampoline().
//   // offset 8, size 8
//   closure_f unsafe.Pointer
//   // IsDebuggerPresent()'s return value.
//   // offset 16, size 4
//   closure_result uint32
//   // Size: 8+8+4 = 20
// }

Important: Our previous implementation used no temporary stack space, but now we are consuming 20 bytes for StackTemporaries. Unfortunately, there’s a small problem: if we try to say that our function’s frame is 20 bytes, compilation fails with a cryptic message:

asm: unaligned stack size 28
asm: assembly failed

(Yes, the error says 28 instead of 20!) Go wants us to round our frame size up to the nearest 8-byte boundary. Instead of asking for 20 bytes, we must ask for 24 bytes. Let’s add an unused variable at the end of our struct so it’s the right size:

// type StackFrame struct {
//   // Argument to runtime.systemstack().
//   // offset 0, size 8
//   arg unsafe.Pointer
//   // Pointer to cIsDebuggerPresent_Trampoline().
//   // offset 8, size 8
//   closure_f unsafe.Pointer
//   // IsDebuggerPresent()'s return value.
//   // offset 16, size 4
//   closure_result uint32
//   // Unused.
//   // offset 20, size 4
//   padding uint32
//   // Size: 8+8+4+4 = 24
// }

Hopefully you’re with me so far, because I’m about to invert everything. Go requires us to refer to variables in the stack frame ‘backwards’ relative to SP (the stack pointer). Instead of saying “closure_f is at SP+8 (i.e. 8 bytes inside our stack frame)”, we must subtract our offsets by our stack frame size and say “closure_f is at SP-16 (i.e. 16 bytes before the end of our stack frame)”. And Go’s syntax is funny, so we actually need to write closure_f-16(SP). We can pick any name we want before the -, but there must be a name. If we omit the name, the behavior is totally different. (I’ve made this mistake many times!)

For clarity, let’s re-document our struct using this backwards syntax:

// type StackFrame struct {
//   arg unsafe.Pointer        // arg-24(SP)
//   closure_f unsafe.Pointer  // closure_f-16(SP)
//   closure_result uint32     // closure_result-8(SP)
//   padding uint32            // padding-4(SP)
// }

Now that we understand our stack layout for the cIsDebuggerPresent() function, let’s look at the code. This code will (1) set up the closure on the stack, (2) call runtime.systemstack(), then (3) extract the return value out of the closure:

// asm_amd64.s

// func cIsDebuggerPresent() int32
TEXT ·cIsDebuggerPresent(SB),0,$24-4
  // (1) Set up the closure.
  // var closure cIsDebuggerPresent_Closure
  // closure.f = &cIsDebuggerPresent_Trampoline
  MOVQ $·cIsDebuggerPresent_Trampoline(SB), AX
  MOVQ AX, closure_f-16(SP)
  // We don't need to initialize closure.result.

  // (2) Call runtime.systemstack().
  // temp = &closure.f
  MOVQ $closure_f-16(SP), AX
  // runtime.systemstack(temp)
  MOVQ AX, arg-24(SP)
  CALL runtime·systemstack(SB)

  // (3) Extract the return value out of the closure.
  // temp = closure.result
  MOVL closure_result-8(SP), AX
  // return temp
  MOVL AX, ret+0(FP)
  RET

Note: In this function, we are using rax (AX) as a temporary register. x86 does not support a combined load+store instruction, so we need to load with one instruction then store with another instruction.

Important: In some MOVL/MOVQ instructions, we wrote $, and in others we did not. The $ signals that we want to copy the address. Without $, we would be loading data from the address. You can think of $ as Go’s & operator.

With this code, runtime.systemstack() will eventually call our cIsDebuggerPresent_Trampoline() function. What does that function look like? It’s actually very similar to our original cIsDebuggerPresent() function, but some details have changed in non-obvious ways:

// asm_amd64.s continued

// (1a)
#define NOSPLIT 4

// (1b)
// special func cIsDebuggerPresent_Trampoline(
//   rdx *cIsDebuggerPresent_Closure)
TEXT ·cIsDebuggerPresent_Trampoline(SB),NOSPLIT,$8-0
  // (2) Save the closure pointer.
  MOVQ DX, closurePtr-8(SP)

  // (3) Call the C function as normal.
  // temp = IsDebuggerPresent()
  MOVQ ·func_IsDebuggerPresent(SB), BX
  CALL BX

  // (4) Load the saved closure pointer.
  MOVQ closurePtr-8(SP), DX
  // (5)
  // closure.result = temp
  MOVL AX, 8(DX)
  RET

The first thing that’s odd is NOSPLIT ((1a) and (1b)). Now that our function is running on the system stack, we need to tell the Go compiler to not emit calls to runtime.morestack() in our function.

Wait, the Go compiler? We’re supposed to be writing assembly code! No compilers! Actually, Go’s assembler is “smart”. It does all sorts of things normal assemblers do not do, so I mentally think of Go’s assembler like a compiler. One thing the Go assembler does is generate function preambles (aka prologues) and epilogues for us. Part of the preamble (which is injected before the first instruction of our function) is checking if the stack needs to grow (by calling runtime.morestack()). In our case, we are running on the system stack, not a normal Go stack, so this check is going to cause us problems. The NOSPLIT special flag tells the Go assembler to omit the stack growth code.

For some reason, we need to define the NOSPLIT constant ourselves (1a) as it’s not defined by the Go assembler for us. (We could have written 4 directly (1b), but that would be unidiomatic.) Go’s assembly guide documents flags such as NOSPLIT.

Another thing that’s not obvious is rdx (2). When runtime.systemstack() calls our function, it stores a pointer to our closure (which we created earlier in cIsDebuggerPresent()) in rdx (DX). This isn’t documented; it’s something I figured out by reading the assembly code for runtime.systemstack().

When we call the IsDebuggerPresent() function, there is no guarantee that rdx is untouched. The Windows x86 calling convention states that the rdx register is volatile. If we want to preserve the value of this volatile register, we must save it on the stack (2) and later restore it (4).

The middle of this function should look familiar: a MOVQ instruction to load our pointer to the C function and a CALL instruction to call the function (3).

After loading our closure pointer back from the stack (4), we take the result of the C function call (from the eax register) and store it in the result field of the closure (5). We use MOVL to store 4 bytes (instead of MOVQ which would store 8 bytes). Notice that we’re using a ‘normal’ struct offset here, not the backward offsets needed for Go’s closure_result-8(SP) syntax.

Phew, that was a lot of work to build a closure! Fundamentally, what we are doing is actually quite simple. But we don’t have a high-level language to help us out, so we need to write each line carefully.

Now our C function is correctly running on the system stack. Our C function is in its happy place with plenty of space to allocate for temporaries without needing to invoke Go’s stack growing routines. Wonderful! But more testing reveals more problems…

Attempt #3: The Stack Right

Assembly code is difficult to write. I think the main reason for this difficulty is that you need to keep so many constraints in mind at the same time. We’ve already looked at constraints such as the system stack, stack layout, NOSPLIT, counting bytes, aligning stack frame sizes, and runtime.systemstack()‘s special calling convention (rdx).

In assembly, there’s always some bug lurking that you won’t find until you do more testing. Even production code in a compiler toolchain used for a decade by tens of thousands of developers might have have a bug due to an incorrect assumption in assembly, as I discovered three years ago.

The assembly code we’ve been writing has (at least) two bugs. You won’t notice the bugs by staring at the assembly code, but you will notice them by reading documentation (or by your program exploding in production).

The Windows x64 ABI documents how different pieces of C code talk to each other at the assembly (and binary) level. Go plays by different rules. (Go actually has two different ABIs: ABI0 and internal. We won’t discuss the Go ABIs in detail.)

Our assembly code is not respecting two parts of the Windows x64 ABI: shadow space and stack alignment.

Shadow Space

Shadow space, sometimes called the shadow store, is space reserved on the stack for a function’s arguments. In the Windows x64 calling convention, the first four arguments are passed via registers. (There are plenty of exceptions to this rule I’m ignoring here.) The shadow space is where a function can store those four arguments into memory if it needs to use those registered for other purposes. Importantly, the shadow space is reserved by the caller, not the callee. That means our assembly code (caller) has to keep the shadow space in mind when calling a C function.

What if our function accepts no arguments? Surprisingly, the shadow space still needs to exist. The shadow space always includes space for four 64-bit argument registers even if the function has fewer parameters.

Our assembly code isn’t reserving shadow space at all. If a C function does use the shadow space, our stack will get corrupted!

How do we fix this? When calling IsDebuggerPresent() (or any other C function), we must reserve 32 bytes (64 bits × 4 registers) of space at the bottom of the stack. What does that look like? Let’s modify our cIsDebuggerPresent_Trampoline() code:

// asm_arm64.s

// special func cIsDebuggerPresent_Trampoline(
//   rdx cIsDebuggerPresent_Closure)
// (1)
TEXT ·cIsDebuggerPresent_Trampoline(SB),NOSPLIT,$40-0
  MOVQ DX, closurePtr-8(SP)

  MOVQ ·func_IsDebuggerPresent(SB), BX
  CALL BX

  MOVQ closurePtr-8(SP), DX
  MOVL AX, 8(DX)
  RET

Did you see it?

In the TEXT line, we added 32 bytes to our stack frame size (1): now it’s 40 bytes, up from 8 bytes. The stack space that we were using is already at the top of the stack (closurePtr-8(SP)), so we didn’t need to move it. Easy!

Of course, this is the simplest possible case. In more complicated cases, especially if some arguments to the C function are passed on the stack, we need to be more conscious of the shadow space. But for our IsDebuggerPresent() example, we just need to ask the Go assembler to reserve more stack space.

Stack Alignment

Microsoft’s documentation on x64 stack usage states:

The stack will always be maintained 16-byte aligned, except […]

In our case, this means that the rsp (SP) register must be a multiple of 16 inside our function. This really matters at when we execute a CALL instruction; a C function can assume that rsp is a multiple of 16 and might misbehave if it’s not.

What does misbehaving look like? The most common symptom is an access violation using an __m128 variable or an SSE instruction like such as movdqa:

Go’s own ABI rules guarantee that rsp is a multiple of 8. But not all multiples of 8 are multiples of 16, so rsp in our program is not necessarily a multiple of 16. Our code has a stack pointer alignment bug.

How do we fix rsp? Luckily, fixing it isn’t too hard, but the code is quite crazy and has some caveats:

  MOVQ SP, CX      // (1)
  ANDQ $~15, SP    // (2)
  MOVQ CX, 32(SP)  // (3)

  /* Your code goes here. */

  MOVQ 32(SP), CX
  MOVQ CX, SP      // (4)

Before executing our main code, we very cleverly clear the bottom 4 bits of the rsp register (2). This has the effect of either doing nothing (if rsp is already a multiple of 16) or subtracting 8 from rsp (if rsp is a multiple of 8 but not a multiple of 16). After adjusting rsp, we store the original value of rsp (1) onto the stack (using an address computed using the new value of rsp) (3). Trippy! After executing our main code, we need to undo the changes to rsp (4).

I didn’t figure out this stack aligment trick myself. I’m not that smart. I learned it from assembly code in the Go runtime.

Important: Do not forget to reserve 8 more bytes in your function’s stack frame (in case rsp was adjusted) by editing the frame size in the TEXT directive.

Important: Our rsp-modifying tricks confuse the Go assembler. If we use any (SP)-relative and (FP)-relative variables, such as closurePtr-8(SP), the Go assembler will generate the wrong offsets in the final machine code. This tripped me up until I figured out what was happening. Be extra careful!

Full Working Example

We’ve talked about fixes to our assembly code to make it work with the many assumptions made by C code (as mandated by the Windows x64 ABI). For completeness, here is the full program with all fixes:

// main.go
package main

import (
  "syscall"
)

var func_IsDebuggerPresent uintptr
func cIsDebuggerPresent() int32

func main() {
  kernel32, _ := syscall.LoadLibrary("kernel32.dll")
  func_IsDebuggerPresent, _ = syscall.GetProcAddress(
    kernel32, "IsDebuggerPresent")

  if cIsDebuggerPresent() == 0 {
    print("Debugger is not present.\n")
  } else {
    print("Debugger is present!\n")
  }
}

// asm_arm64.s

// type cIsDebuggerPresent_Closure struct {
//   f unsafe.Pointer
//   result uint32
// }

#define NOSPLIT 4

// func cIsDebuggerPresent() int32
TEXT ·cIsDebuggerPresent(SB),0,$24-4
  MOVQ $·cIsDebuggerPresent_Trampoline(SB), AX
  MOVQ AX, closure_f-16(SP)

  MOVQ $closure_f-16(SP), AX
  MOVQ AX, arg-24(SP)
  CALL runtime·systemstack(SB)

  MOVQ closure_result-8(SP), AX
  MOVQ AX, ret+0(FP)
  RET

// special func cIsDebuggerPresent_Trampoline(
//   rdx *cIsDebuggerPresent_Closure)
TEXT ·cIsDebuggerPresent_Trampoline(SB),NOSPLIT,$48-0
  MOVQ DX, closurePtr-8(SP)

  MOVQ SP, CX
  ANDQ $~15, SP
  MOVQ CX, 32(SP)

  MOVQ ·func_IsDebuggerPresent(SB), BX
  CALL BX

  MOVQ 32(SP), CX
  MOVQ CX, SP

  MOVQ closurePtr-8(SP), DX
  MOVL AX, 8(DX)
  RET

Try it out on your machine. Hopefully I didn’t miss any assumptions!

Portability

In this article, we focused on Windows and x64. There are other architectures supported on Windows, such as ARM64 and 32-bit x86. If we want our code to work on those machines, we need to write asm_arm64.s and asm_386.s assembly respectively. And if we want to support macOS or Linux (or another OS), we need to write new routines to for those platforms too because the calling conventions and ABIs on those platforms are different than Windows’s. That’s a lot of work just to call a C function from Go!

I won’t get into the details for other architectures and platforms. Sorry, but I just don’t have the time.

Tools such as CGo are fantastic for productivity. Instead of hand-writing assembly wrappers for many platforms, we can ask CGo to do everything for us. If you can use CGo, it’s likely a better option than writing everything yourself.

But if you can’t use CGo for whatever reason, or if you like to do things the hard way with assembly, or if you just wanted to learn more about writing Go-style assembly code, I hope this article has been helpful.