Undefined Behavior as Compiler Optimization Fuel

Modern C compilers like GCC and Clang have become increasingly aggressive with optimizations based on undefined behavior. This can lead to surprising code transformations that might not match your intuitive understanding of how the code should work.

Consider this seemingly innocent example:

int foo(int x) {
    return x + 1 > x;
}

What would you expect this function to return? Intuitively, it seems like it should always return 1 (true), since x + 1 should always be greater than x.

But when compiled with optimizations enabled, many compilers will transform this into:

int foo(int x) {
    return 1;
}

This is usually correct, but it overlooks the case of integer overflow, which is undefined behavior in C. If x is INT_MAX, then x + 1 overflows, entering undefined behavior territory.

The compiler reasons: “If x is INT_MAX, then x + 1 would overflow, which is undefined behavior. The program can’t rely on any particular outcome in this case. Therefore, I can assume this case never happens, because the programmer must have ensured it won’t (otherwise they’ve written a program with undefined behavior, which is invalid).”

Another fascinating example:

void process(int *p) {
    *p = 1;
    if (p == NULL) {
        // Handle null pointer case
        *p = 0;  // This never happens, right?
    }
}

A compiler might optimize this to:

void process(int *p) {
    *p = 1;
    // 'if' statement eliminated entirely
}

The reasoning: by the time we reach the if statement, we’ve already dereferenced p. If p were NULL, that dereference would be undefined behavior. Therefore, the compiler assumes p cannot be NULL, making the condition always false.

Understanding these optimizations is crucial for systems programmers. Here are some practical guidelines:

  1. Be extremely wary of code that could trigger undefined behavior conditionally
  2. Use appropriate sanitizers (-fsanitize=undefined) during testing
  3. Consider compiler flags like -fwrapv for specific cases where you want defined semantics for overflow
  4. Remember that newer compiler versions might optimize more aggressively

The relationship between undefined behavior and optimization is a double-edged sword: it enables more efficient code generation but requires a deep understanding of the language specification to avoid surprises.