Previously I showed you an anti-debug technique using the macOS exception ports API to detect the presence of a debugger. Today, I’m going to show you a different exception-port-based anti-debug trick which actually detaches the debugger by stealing the exception ports it is using. Some readers may be familiar with similar tricks done on Windows, however it’s very rare to see something like this done on macOS, though I have seen this kind of thing in the wild so here goes.

Continuing my series on anti-debug technique and how to defeat them, today we have a mach exception ports based trick on macOS.

For those unfamiliar with how the macOS flavor of Unix does things, these ports are used by debuggers to handle exceptions like breakpoints and bad-access. With this knowledge, a piece of software that does not want to be debugged, like a piece of malware trying to prevent analysis, can check if these ports are open and do something else in response, like shutdown.

The code for mach exception handling is fairly complex and poorly documented, but I’ve created a fairly simple example of debugger-detection code so we can dive right in.

Every reverse engineer who handles software for macOS knows about ptrace(PT_DENY_ATTACH, 0, 0, 0), the infamous kernel-enforced anti-tracing DRM feature added to OS X years back (somewhere around Leopard) and most-notably used in iTunes. There are plenty of resources out there on how to bypass the common use of this feature, ranging from using a debugger to loading up a custom kernel-extension, but clever hackers have found new ways to abuse this feature to try to prevent researchers from debugging their malicious code.

I debated publishing this for a while as this information could misused, but since these techniques are being used in malware in the wild, I think it’s important to document how to defeat them.

For a long time Apple has had a debugger-detection function on their developer site. It’s not very hard to defeat with a debugger, but since this technique is sometimes used by malware to try to prevent debugging, I decided to document how to defeat it.