|
DynamoRIO API
|
|
|
DynamoRIO API
|
|
This section gives an overview of how to use DynamoRIO, divided into the following sub-sections:
DynamoRIO exports a rich Application Programming Interface (API) to the user for building a DynamoRIO client. A DynamoRIO client is a library that is coupled with DynamoRIO in order to jointly operate on an input program binary:
To interact with the client, DynamoRIO provides specific events that a client can intercept. Event interception functions, if supplied by a user client, are called by DynamoRIO at appropriate times.
DynamoRIO can alternatively be used as a third-party disassembly library (see IA-32/AMD64/ARM/AArch64 Disassembly Library).
A client's primary interaction with the DynamoRIO system is via a set of event callbacks. These events include the following:
Typically, a client will register for the desired events at initialization in its dr_client_main() routine. DynamoRIO then calls the registered functions at the appropriate times. Each event has a specific registration routine (e.g., dr_register_thread_init_event(): see the names in parentheses in the list above) and an associated unregistration routine. The header file dr_events.h contains the declarations for all registration and unregistration routines.
Note that clients are allowed to register multiple callbacks for the same event. DynamoRIO also supports mutiple clients, each of which can register for the same event. In this case, DynamoRIO sequences event callbacks in reverse order of when they were registered. In other words, the first registered callback receives event notification last. This scheme gives priority to a callback registered earlier, since it can override or modify the actions of clients registered later. Note that DynamoRIO calls each client's dr_client_main() routine according to the client's priority (see Multiple Clients and dr_register_client() in the deployment API).
Systems registering multiple callbacks for a single event should be aware that client modifications are visible in subsequent callbacks. DynamoRIO makes no attempt to mitigate interference among callback functions. It is the responsibility of a client to ensure compatibility among its callback functions and the callback functions of other clients.
Clients can also unregister a callback using the appropriate unregister routine (see dr_events.h). While unusual, it is possible for one callback routine to unregister another. In this case, DynamoRIO still calls routines that were registered before the event. Unregistration takes effect before the next event.
On Linux, an exec (SYS_execve) does NOT result in an exit event, but it WILL result in the client library being reloaded and its dr_client_main() routine being called again. The system call events can be used for notification of SYS_execve.
DynamoRIO provides clients with a powerful library of utilities for custom runtime code transformations. The interface includes explicit support for creating transparent clients. See the section on Client Transparency for a full discussion of the importance of remaining transparent when operating in the same process as the application. DynamoRIO provides common resources clients can use to avoid reliance on shared libraries that may be in use by the application. The client should only use external resources through DynamoRIO's own API, through DynamoRIO Extensions (see DynamoRIO Extensions), through direct system calls, or via an external agent in a separate process that communicates with the client (see Communication). Third-party libraries can be used if they are linked statically or loaded privately and there is no possibility of global resource conflicts (e.g., a third-party library's memory allocation must be wrapped): see Using External Libraries for more details.
DynamoRIO's API provides:
See dr_tools.h and dr_proc.h for specifics of each routine.
Another class of utilities provided by DynamoRIO are structures and routines for decoding, encoding, and manipulating IA-32, AMD64, ARM, and AArch64 instructions. These are described in Instruction Representation.
In addition, on Windows, DynamoRIO provides a number of utility functions that it fowards to a core Windows system library that we believe to be safe for clients to use:
In general, these routines match their standard C library counterparts. However, be warned that some of these may be more limited. In particular, _vsnprintf and _snprintf do not support floating-point values. DynamoRIO provides its own dr_snprintf() that does support floating-point values, but does not support printing wide characters. When printing floating-point values be sure to save the application's floating point state so as to avoid corrupting it.
To simplify reachability in a 64-bit address space, DynamoRIO guarantees that all of its code caches are located within a single 2GB memory region. It also places all client memory allocated through dr_thread_alloc(), dr_global_alloc(), dr_nonheap_alloc(), or dr_custom_alloc() with DR_ALLOC_CACHE_REACHABLE in the same region.
DynamoRIO loads client libraries and Extensions (but not copies of system libraries) within 32-bit reachability of its code caches. Typically, the code cache region is located in the low 4GB of the address space; thus, to avoid relocations at client library load time, it is recommended to set a preferred client library base in the low 4GB. The -reachable_client runtime option can be used to remove this guarantee, though this would normally not be done unless the client is statically linked with the application.
The net result is that any static data or code in a client library, or any data allocated using DynamoRIO's API routines (except dr_raw_mem_alloc() or dr_custom_alloc()), is guaranteed to be directly reachable from code cache code. However, memory allocated through system libraries (including malloc, operator new, and HeapAlloc), as well as DynamoRIO's own internally-used heap memory, is not guaranteed to be reachable: only memory directly allocated via DynamoRIO's API. The -reachable_heap runtime option can be used to guarantee that all memory is reachable, at the risk of running out of memory due to the smaller space of available memory.
To make more space available for the code caches when running larger applications, or for clients that use a lot of heap memory that is not directly referenced from the cache, we recommend that dr_custom_alloc() be called to obtain memory that is not guaranteed to be reachable from the code cache (by not passing DR_ALLOC_CACHE_REACHABLE). This frees up space in the reachable region.
When inserting calls, dr_insert_call() and dr_insert_clean_call() assume that the call is destined for encoding into the code cache-reachable memory region, when determining whether a direct or indirect call is needed. An indirect call will clobber r11. Use dr_insert_clean_call_ex() with DR_CLEANCALL_INDIRECT to ensure reachability when encoding to a location other than DR's regular code region, or when a clean call is not needed, dr_insert_call_ex() takes in a target encode location for more flexible determination of direct versus indirect.
DynamoRIO does not guarantee that any of its memory is allocated in the lower 4GB of the address space. However, it provides several features to make it easier to reference addresses absolutely:
var in a client library, the client can create an operand with the address &var and it will auto-magically turn into a pc-relative addressing mode. OPND_CREATE_ABSMEM() directly creates a pc-relative operand, while opnd_create_abs_addr() will convert to a pc-relative operand when an absolute reference will not encode. An opnd_create_rel_addr() operand will also convert to an absolute reference when that will reach but a pc-relative reference will not.instr_t pointer as an immediate, use the routines instrlist_insert_mov_instr_addr() or instrlist_insert_push_instr_addr() to conveniently insert either one or two instructions depending on whether the resulting instr_t encoded address is in the lower 4GB or not.All strings in the DynamoRIO API, whether input or output parameters, are encoded as UTF-8. DynamoRIO will internally convert to UTF-16 when interacting with the Windows kernel. A client can use dr_snprintf() or dr_snwprintf() with the S format code to convert between UTF-8 and UTF-16 on its own. (The _snprintf() function forwarded to ntdll does not perform that conversion.)
To use the DynamoRIO API, a client should include the main DynamoRIO header file:
The client's target operating system and architecture must be specified by setting pre-processor defines before including the DynamoRIO header files. The appropriate library must then be linked with. The define choices are:
WINDOWS, LINUX, or (coming soon) MACOS X86_32, X86_64, ARM_32, or ARM_64 Currently we provide a private loader for both Windows and Linux. With private loading, clients use a separate copy of each library from any copy used by the application.
If the private loader is deliberately disabled, for transparency reasons (see Client Transparency), clients should be self-contained and should not share libraries with the application. Without the private loader, 64-bit clients must take care to try and load themselves within reachable range of DynamoRIO's code caches by setting a preferred base address, although this may not always be honored by the system loader.
The DynamoRIO release supplies CMake configuration files to facilitate building clients with the proper compiler and linker flags. CMake is a cross-platform build system that generates Makefiles or other development system project files. A DynamoRIOConfig.cmake configuration file, along with supporting files, is distributed in the cmake/ directory.
In its CMakeLists.txt file, a client should first invoke a find_package(DynamoRIO) command. This can optionally take a version parameter. This adds DynamoRIO as an imported target. If found, the client should then invoke the configure_DynamoRIO_client() function in order to configure build settings. Here is an example:
Note that when building a 32-bit client in Linux using gcc, the stack alignment should be 4-byte only. Using the function configure_DynamoRIO_client() will configure the build settings correctly. Otherwise, appropriate options should be passed to the compiler: e.g., -mpreferred-stack-boundary=2.
The samples/CMakeLists.txt file in the release package serves as another example. The top of DynamoRIOConfig.cmake contains detailed instructions as well.
When configuring, the DynamoRIO_DIR CMake variable can be passed in to identify the directory that contains the DynamoRIOConfig.cmake file. For example:
The compiler needs to be configured prior to invoking cmake. If using gcc with a non-default target platform, the CFLAGS and CXXFLAGS environment variables should be set prior to invoking cmake. For example, to configure a 32-bit client when gcc's default is 64-bit:
Note that CXXFLAGS should be set instead for a C++ client, and both should be set when building both types of clients from the same configuration (e.g., samples/CMakeLists.txt).
To improve clean call performance (see Clean Calls and -opt_cleancall), we recommend high levels of optimization when building a client.
If a client is not using CMake, the appropriate compiler and linker flags can be gleaned from DynamoRIOConfig.cmake. One method is to invoke CMake to generate a Makefile and then build with VERBOSE=1. We also summarize here the key flags required for 32-bit clients for gcc:
And for cl:
For a 64-bit client with cl:
For 64-bit Linux clients, setting the preferred base takes several steps. Refer to DynamoRIOConfig.cmake for details.
To make clean call sequences more likely to be optimized, it is recommended to compile the client with optimizations, -O2 for gcc or /O2 for cl.
DynamoRIO supports extending the API presented to clients through separate libraries called DynamoRIO Extensions. Extensions are meant to include features that may be too costly to make available by default or features contributed by third parties whose licensing requires using a separate library. Extensions can be either static libraries linked with clients at build time or dynamic libraries loaded at runtime. A private loader is used to load dynamic Extensions.
Current Extensions provide symbol access and container data structures. Each Extension has its own documentation and has its functions and data structures documented separately from the main API. See the full list of Extensions here: DynamoRIO Extensions.
Be aware that some of the DynamoRIO Extensions have LGPL licenses instead of the BSD license of the rest of DynamoRIO. Such Extensions are built as shared libraries, have their own license.txt files, and clearly identify their license in their documentation. (We also provide static versions of such libraries, but take care in using them that their LGPL licenses match your requirements.)
Clients are free to use external libraries as long as those libraries do not use any global user-mode resources that would interfere with the running application, and as long as no alertable system calls are invoked on Windows (see Avoid Alertable System Calls). While most non-graphical non-alertable Windows API routines are supported, native threading libraries such as libpthread.so on Linux are known to cause problems.
Currently we provide a private loader for both Windows and Linux. Clients must either link statically to all libraries or load them using our private loader, which will happen automatically for shared libraries loaded in a typical manner. With private loading, the client uses a separate copy of each library from any copy used by the application. This helps to prevent re-entrancy problems (see Resource Usage Conflicts). Even with this separation, if these libraries use global resources there can still be conflicts. Our private loader redirects heap allocation in the main process heap to instead use DynamoRIO's internal heap. The loader also attempts to isolate other global resource usage and global callbacks. Please file reports on any transparency problems observed when using the private loader.
By default, all Windows clients link with libc. To instead use the libc subset of routines forwarded from the DynamoRIO library to ntdll.dll (which keeps clients more lightweight and is usually sufficient for most C code), set this variable prior to invoking configure_DynamoRIO_client():
C++ clients and standalone clients link with libc by default.
On Windows, DynamoRIO does not support a client (or a library used by a client) making alertable system calls. These are system calls that can be interrupted for delivery of callbacks or asynchronous procedure calls. At the Windows API layer, they include many graphical routines, any Wait function invoked with alertable=TRUE (e.g., WaitForSingleObjectEx or WaitForMultipleObjectsEx), any Windows message queue function (GetMessage, SendMessage, ReplyMessage), and asynchronous i/o. In general, avoiding graphical, windowing, or asynchronous i/o library or system calls is advisable. DynamoRIO does not guarantee correct execution when a callback arrives during client code execution.
DynamoRIO's loader searches for libraries in approximately the same manner as the system loader. It also has support for automatically locating Extension libraries that are packaged in the usual place in the DynamoRIO file hierarchy.
DynamoRIO supports setting DT_RPATH for ELF clients, via setting the DynamoRIO_RPATH variable to ON prior to invoking configure_DynamoRIO_client(). On Windows, setting that variable will create a "<client_basename>.drpath" text file that contains a list of paths. At runtime, DynamoRIO's loader will parse this file and add each newline-separated path to its list of search paths. This file is honored on Linux as well, though it is not automatically created there. This allows clients a cross-platform mechanism to use third-party libraries in locations of their choosing.
Sometimes, a client wishes to invoke system library routines with the application context, rather than having them redirected and isolated by DynamoRIO. This can be accomplished using dynamic binding rather than static: dynamically looking up each desired library routine via DR's own routines (such as dr_get_proc_address()). (Using GetProcAddress will not work for this purpose as the result will be redirected.)
On Linux, if the private loader is deliberately disabled, ld provides the -wrap option, which allows us to override the C library's memory heap allocation routines with our own. For convenience, DynamoRIO exports __wrap_malloc(), __wrap_realloc(), and __wrap_free() for this purpose. These routines behave like their C library counterparts, but operate on DynamoRIO's global memory pool. Use the -Xlinker flag with gcc to replace the libc routines with DynamoRIO's _wrap routines, e.g.,
The ability to override the memory allocation routines makes it convenient to develop C++ clients that use the new and delete operators (as long as those operators are implemented using malloc and free). In particular, heap allocation is required to use the C++ Standard Template Library containers. When developing a C++ client, we recommend linking statically to the C++ runtime library if not using the provided private loader.
On Linux, this is most easily accomplished by specifying the path to the static version of the library on the gcc command line. gcc's -print-file-name option is useful for discovering this path, e.g.,
A full gcc command line for building a C++ client when disabling the private loader (which is not the default) might look something like this (note that this requires static versions of the standard libraries that were built PIC, which is not the case in modern binary distributions and often requires building from source):
The 3.0 version of DynamoRIO added experimental full support for C++ clients using the STL and other libraries.
On Windows, when using the Microsoft Visual C++ compiler, we recommend using the /MT compiler flag to request a static C library. The client will still use the kernel32.dll library but our private loader will load a separate copy of that library and redirect heap allocation automatically. Our private loader does not yet support locating SxS (side-by-side) libraries, so using /MD will most likely not work unless using a version of the Visual Studio compiler other than 2005 or 2008.
We do not recommend that a client or its libraries invoke their own system calls as this bypasses DynamoRIO's monitoring of changes to the process address space and changes to threads or control flow. Such system calls will also not work properly on Linux when using sysenter on some systems. If you see an assert to that effect in debug build on Linux, try the -sysenter_is_int80 option.
Due to transparency limitations (see Client Transparency), DynamoRIO can only support certain communication channels in and out of the target application process. These include:
DynamoRIO provides a binary annotation mechanism which allows the target application to communicate directly with the DynamoRIO client, or with DynamoRIO itself. A binary annotation is generated from a macro that can be manually inserted into the source code of the target program. When compiled, the resulting sequence of assembly instructions has no effect on native execution (i.e., it is a nop, or resolves to a static default value), but during execution under DynamoRIO, each annotation is detected and transformed into a function call to a set of registered handlers. Currently DynamoRIO provides 2 simple annotations:
An annotation may be declared void, as in DYNAMORIO_ANNOTATE_LOG(), or it may have a return value, as in the boolean DYNAMORIO_ANNOTATE_RUNNING_ON_DYNAMORIO(). The return value can be used in a branch predicate, such that some of the target app's code only executes under DynamoRIO, or it can be used for in-process communication to obtain data from DynamoRIO or its client that is only available during binary translation.
Adding an annotation to a target application is primarily a simple matter of invoking the annotation macro at the desired program location. The macros are declared in the C header file include/annotations/dr_annotations.h, and each macro operates syntactically like a function call. An annotation having a return value can be used as an expression. DynamoRIO provides a module which defines the annotations, and clients may also provide modules containing custom annotations. Each compilation unit that uses annotations must be statically linked with the corresponding annotation module(s). For projects using cmake, a convenience function use_DynamoRIO_annotations(target, srcs) will configure the specified srcs to be linked with the annotation module.
When DynamoRIO encounters an annotation in the target app, it instruments that program location in one of two ways:
Annotation handlers are registered using API functions dr_annotation_register_call() and dr_annotation_register_return(). Note that changes to handler registration will have no effect on annotations that have already been translated by DynamoRIO into the code cache (until the annotated basic blocks are removed from the cache and retranslated). The annotation instrumentation invokes the handlers using a separate clean call for each handler. The return value for an annotation can be set within a handler function using the API function dr_annotation_set_return_value().
DynamoRIO client developers may wish to create new annotations to facilitate client-specific communication with the target application. For example, a client that inspects memory usage may have false positives for variables in the target app that are never referenced after initialization. The create_annotation walks through the process of creating a new annotation that allows target application developers to explicitly mark any variable as defined.
DynamoRIO's behavior can be fine-tuned using runtime parameters. Options are specified via drconfig, drrun, or dr_register_process(). See Deployment.
-no_follow_children: By default, DynamoRIO follows all child processes. When this option is disabled via -no_follow_children, DynamoRIO follows only into child processes for which a configuration file exists (typically created by drconfig; see Deployment). On Linux, forked children are always followed and this option only affects execve.
To follow all children in general but exclude certain children, leave -follow_children on (the default) and create config files that exclude the desired applications by running drconfig with the -norun option.
-persist_dir. See Persisting Code for more details.Options controlling notifications from DynamoRIO:
cmd console on Windows 7 or earlier or on any Windows version when running a graphical application in cmd (even with dr_enable_console_printing(), as that only affects clients calling dr_printf() or dr_fprintf()) but the output can be viewed from cmd by redirecting to a file. For the provided Linux debug builds, -stderr_mask defaults to 0xF; for the Linux release builds, its default is 0xE. The default on Windows is 0.Options aiding in debugging:
Options available only in the debug build of DynamoRIO:
drconfig, or drrun configuration for the target application. The runtime option -logdir can be used to override the default directory. There is one main log file per directory named app.0.TID.html, where TID is the thread identifier of the initial thread. There is also a log file per thread, named log.N.TID.html, where N is the thread's creation ordinal and TID is its thread identifier. The loglevel may be changed during program execution, but if it began at 0 then it cannot be raised later. The -logmask parameter can be used to control which DynamoRIO modules output data to the log files. dr_log() allows the client to write to the above logfiles.When using a complex system like DynamoRIO, problems can be challenging to diagnose. This section contains some debugging tips and shows how to get help.
For questions and discussion, join the DynamoRIO Users group.
For bug reports, use the Issue Tracker. Please include a detailed description of the problem (is it an application crash? a DynamoRIO crash? a hang? a debug build assert?) and how to reproduce it.
If the client library doesn't seem to function for a given process, it is likely that the client library wasn't loaded due to errors.
A process under control of DynamoRIO can be executed within a debugger. For debugging on Windows we recommend using windbg version 6.3.0017 (not the newer versions, as they have problems displaying callstacks involving DynamoRIO code).
Normally, the debugger will not be aware of the DynamoRIO library or the client library. We provide a windbg script that locates the DynamoRIO library, the client library, and any privately-loaded dependent libraries. The script is in bin32/load_syms.txt and bin64/load_syms64.txt. To load it from windbg, execute the following command:
When debugging often, modify the shortcut that launches windbg to include this command as a -c argument. E.g.:
On Windows, the -no_hide option can alternatively be used so the debugger can see the DynamoRIO library, but the debugger will still not be able to see the client library or any of its dependent libraries. We recommend using our script.
To attach to a process on Windows, use the -msgbox_mask option and attach the debugger while the dialog box has paused the application. On Linux, the same option can be used and the debugger attached while the application waits for enter to be pressed. Since this may not work for applications that themselves read from standard input, we also provide the -pause_via_loop runtime option which sits in an infinite loop rather than waiting for a keypress.
To run an application on Linux under a debugger from process start you can launch drrun under gdb as you would normally:
Because the executable changes from drrun to the app, the app cannot be re-run from gdb's prompt.
On Linux, the main drawback of debugging from application start rather than attaching is that breakpoint instructions (int3) inserted by the debugger get copied into the code cache. This includes internal debugger breakpoints automatically placed in the loader, as well as user-defined breakpoints. For example, gdb puts a breakpoint on __nptl_create_event, which is called by pthread_create and related calls. See https://github.com/DynamoRIO/dynamorio/issues/490. The debugger will handle these traps, but the user must tell it to continue, which is an annoyance. For user breakpoints, consider using read watchpoints on the code in question instead.
On Windows, if an application invokes OutputDebugString() while under a debugger, DynamoRIO can end up losing control of the application.
For additional tips, check the DynamoRIO wiki page on debugging: https://github.com/DynamoRIO/dynamorio/wiki/Debugging
DynamoRIO API version 7.1.0 --- Fri Feb 22 2019 14:52:31