Our first rule of transparency is to avoid conflicts in resource usage with the application. A primary example of this is avoiding sharing library usage with the application. One bad scenario is if the application is in the middle of a non-reentrant library routine, and we intercept it and end up calling that same routine, there's going to be trouble. And yes, we did run into this problem multiple times while developing DynamoRIO.
To follow this rule on a typical operating system like Linux, we just have to make sure that our external resources come straight from the OS via system calls, with no user-mode libraries or state in between that may get mixed up with the application. On Windows, however, the problem is exacerbated by the fact that the documented method for application interaction with the OS is through an API layer of user-mode libraries, with the actual system call layer being hidden. There's no way around this -- we have to live at the system call layer and use these undocumented interfaces, which is one of the primary challenges of building a system like this on Windows.
|Copyright © 2004 Derek Bruening|