Vtrace Debugger Framework
Vtrace is a mostly native python debugging framework which can be used to quickly write programatic debuggers and research tools.
I’m not known for writting great docs... but the code should be pretty straight forward...
This has been in use for many years privately, but is nowhere near free of bugs... idiosyncracies abound.
==== Werd =====================================================
Blah blah blah... many more docs to come.
Brought to you by kenshoto. e-mail invisigoth.
Bases: exceptions.Exception
An exception which is raised on bad-touch to memory
Bases: exceptions.Exception
A universal way to represent a failure in the platform layer for this tracer. platformFoo methods should raise this rather than allowing their platform specific exception types (which don’t likely pickle, or are not cross platform)
Bases: envi.memory.IMemory, envi.registers.RegisterContext, envi.resolver.SymbolResolver, object
The main tracer object. A trace instance is dynamically generated using this and many potential mixin classes. However, API users should not worry about the methods that come from the mixins... Everything that is meant to be used from the API is contained and documented here.
Add a breakpoint by address. This will create the Breakpoint object for you and add it to the trace. It returns the newly created breakpoint id.
Optionally, set fastbreak=True to have the breakpoint behave in “fast break” mode which automatically continues execution and does not fire notifiers for the breakpoint.
Example: trace.addBreakByAddr(0x7c770308)
Add a breakpoint by resolving an expression. This will create the Breakpoint object for you and add it to the trace. It returns the newly created breakpoint id.
Optionally, set fastbreak=True to have the breakpoint behave in “fast break” mode which automatically continues execution and does not fire notifiers for the breakpoint.
Example: trace.addBreakByExpr(‘kernel32.CreateFileA + ecx’)
Add a breakpoint/watchpoint to the trace. The “breakpoint” argument is a vtrace Breakpoint/Watchpoint object or something that extends it.
To add a basic breakpoint use trace.addBreakpoint(vtrace.Breakpoint(address)) NOTE: expression breakpoints do not get evaluated in fastbreak mode
This will return the internal ID given to the new breakpoint
By adding an IgnoreSignal you tell the tracer object to supress the notification of a particular type of signal. In POSIX, these are regular signals, in Win32, these are exception codes. This is mostly useful in RunForever mode because you still need the process to begin running again. (these may be viewed/modified by the metadata key “IgnoredSignals”) FIXME: make address do something.
Allocate a chunk of memory inside the target process’ address space. Memory wil be mapped rwx unless otherwise specified with perms=envi.memory.MM_FOO values. Optionally you may suggest an address to the allocator, but there is no guarentee. Returns the mapped memory address.
Build a new/clean trace “like” this one. For platforms where a special trace was handed in, this allows initialization of a new one. For most implementations, this is very simple....
Setup the “stack” and call the target address with the following arguments. If the argument is a string or a buffer, copy that into memory and hand in the argument.
The current state of ALL registers are returned as a dictionary at the end of the call...
Additionally, a “convention” string may be specified that the underlying platform may be able to interpret...
See addIgnoreSignal for a description of signal ignoring. This removes an ignored signal and re-enables it’s delivery.
Put the tracer object in to AutoContinue mode for the specified event. To make all events continue running see RunForever mode in setMode().
Retrieve the list of vtrace notification events that will be auto-continued.
Return a reference to the breakpoint with the requested ID.
NOTE: NEVER set locals or use things like setBreakpointCode() method on return’d breakpoint objects as they may be remote and would then be coppies of the bp objects. (use the trace’s setBreakpointCode() instead).
Return the breakpoint object (or None) for a given virtual address.
Return the python string of user specified code that will run when this breakpoint is hit.
An accessor method for returning if a breakpoint is currently enabled. NOTE: code which wants to be remote-safe should use this
Retrieve the current signal/exception posted to the process. If there are no pending signals/exceptions the API will return None. For POSIX systems, this will be a traditional POSIX signal. For Windows systems it will be a current exception code (if any).
Example: sig = trace.getCurrentSignal()
Get a list of (fd,type,bestname) pairs. This is MOSTLY useful for HUMON consumtion... or giving HUMONs consumption...
If the most receent event is a memory access error, this API will return a tuple of (<addr>,<perm>) on supported platforms. Otherwise, a (None, None) will result.
Example: import envi.memory as e_mem vaddr,vperm = trace.getMemoryFault() if vaddr != None:
print ‘Memory Fault At: 0x%.8x (perm: %d)’ % (vaddr, vperm)
Return a list of the currently mapped memory for the target process. This is acomplished by calling the platform’s platformGetMaps() mixin method. This will also cache the results until CONTINUE. The format is (addr,len,perms,file).
Get some metadata. Metadata is a clean way for arbitrary trace consumers (and notifiers) to present and track additional information in trace objects.
If you specify a default and the key doesn’t exist, not not only will the default be returned, but the key will be set to the default specified.
Get the value for a mode setting allowing for a clean default...
Symbols are stored internally based off of “normalized” library names. This method returns the list of normalized names for the loaded libraries.
(probably only useful for writting symbol browsers...)
Retrieve the envi.registers.RegisterContext object for the specified thread. Use this API to iterate over threads register values without setting the global tracer thread context.
Returns a list of (instruction pointer, stack frame) tuples. If stack tracing results in an error, the error entry will be (-1,-1). Otherwise most platforms end up with 0,0 as the top stack frame
Retrieve a vstruct structure populated with memory from the specified address. Returns a standard vstruct object.
This method returns either the structure names, or the structure namespaces that the target tracer is aware of. If “namespace” is specified, it is structures within that namespace, otherwise it is “known namespaces”
Return an envi Symbol object for an address. Use exact=False to get the nearest previous match.
Return the entire symbol list for the specified normalized library name.
Get a dictionary of <threadid>:<tinfo> pairs where tinfo is platform dependant, but is tyically either the top of the stack for that thread, or the TEB on win32
Check to see if a metadata key exists... Mostly un-necissary as getMeta() with a default will set the key to the default if non-existant.
Much like the python hex routine, except this will automatically pad the value’s string length out to pointer width.
Create a new thread inside the target process. This thread will begin execution on the next process run().
Inject a shared object into the target of the trace. So, on windows this is easy with InjectDll and on *nix... it’s.. fugly...
Return boolean true/false for weather or not this trace is currently attached to a process.
Kill the target process for this trace (will result in process exit and fire appropriate notifiers)
Parse a python expression with many useful helpers mapped into the execution namespace.
Example: trace.parseExpression(“ispoi(ecx+ntdll.RtlAllocateHeap)”)
Change the page protections on the specified region of memory. See envi.memory for perms values.
Read memory from address. Areas that are NOT valid memory will be read back as s (this probably goes in a mixin soon)
Register a notifier who will be called for various events. See NOTIFY_* constants for handler hooks.
Release resources for this tracer. This API should be called once you are done with the trace.
A utility method for other methods to use in order to require being attached
Just a quick method to throw an error if the tracer is already running...
Allow the traced target to continue execution. (Depending on the mode “Blocking” this will either block until an event, or return immediately) Additionally, the argument until may be used to cause execution to continue until the specified address is reached (internally uses and removes a breakpoint).
The runAgain() method may be used from inside a notifier (Notifier, Breakpoint, Watchpoint, etc...) to inform the trace that once event processing is complete, it should continue running the trace.
Search a memory range for the specified sequence of bytes
Search for symbols which match the given regular expression. Specify libname as the “normalized” library name to only search the specified lib.
Example: for sym in trace.searchSymbols(‘.*CreateFile.*’, ‘kernel32’):
Set the “current thread” context to the given thread id. (For example stack traces and register values will depend on the current thread context). By default the thread responsible for an “interesting event” is selected.
Send an asynchronous break signal to the target process. This is only valid if the target is actually running...
Because breakpoints are potentially on the remote debugger and code is not pickleable in python, special access methods which takes strings of python code are necissary for the vdb interface to quick script breakpoint code. Use this method to set the python code for this breakpoint.
An accessor method for setting a breakpoint enabled/disabled.
NOTE: code which wants to be remote-safe should use this
Set the currently pending signal for delivery to the target process on continue. This is intended for use by programs wishing the mask or change the delivery of exceptions on a NOTIFY_SIGNAL event.
Example: trace.setCurrentSignal(None)
Set some metadata. Metadata is a clean way for arbitrary trace consumers (and notifiers) to present and track additional information in trace objects.
Any modules which use this should initialize them on attach (so when they get re-used they’re clean)
Some examples of metadata used: ShouldBreak - We’re expecting a non-signal related break ExitCode - The int() exit code (if exited) PendingSignal - The current signal
Set a mode setting... This is ONLY valid if that mode has been iniitialized with initMode(name, value). Otherwise, it’s an unsupported mode for this platform ;) cute huh? This way, platform sections can cleanly setmodes and such.
Set a named variable in the trace which may be used in subsequent VtraceExpressions.
Example: trace.setVariable(“whereiam”, trace.getProgramCounter())
Single step the target process ONE instruction (and do NOT activate breakpoints for the one step). Also, we don’t deliver pending signals for the single step... Use the mode FastStep to allow/supress notifier callbacks on step
Bases: vtrace.notifiers.Notifier, vtrace.util.TraceManager
Encapsulate several traces, run them, and continue to handle their event notifications.
Add a new tracer to this group the “proc” argument may be either an long() for a pid (which we will attach to) or an already attached (and broken) tracer object.
Similar to vtrace.getTrace(), but also init’s the trace for being managed by a TraceGroup.
Return the the trace for process PID if we’re already attached. Return None if not.
Bases: envi.expression.MemoryExpressionLocals
A class which serves as the namespace dictionary during the evaluation of an expression on a tracer.
Return the address of the saved base pointer for the specified frame.
Usage: frame(<index>)
A shortcut for trace.runAgain() which may be used in breakpoint code (or similar even processors) to begin execution again after event processing...
Return a tracer object appropriate for this platform. This is the function you will use to get a tracer object with the appropriate ancestry for your host. ex. mytrace = vtrace.getTrace()
Some specialized tracers may be constructed by specifying the “plat” name from one of the following list. Additionally, each “specialized” tracer may require additional kwargs (which are listed).
t = vtrace.getTrace() # A tracer for this os
t = vtrace.getTrace(plat=’android’) # The default ADB device
t = vtrace.getTrace(plat=’vmware32’, host=’localhost’, port=31337)
Test for platform functionality (for internal use).
Breakpoint Objects
Breakpoints in Vtrace are platform independant objects that use the underlying trace objects to get things like the program counter and the break instruction. As long as platfforms are completely implemented, all breakpoint objects should be portable.
Actually store off and replace memory for this process. This is caried out by the trace object itself when it begins running or stops. You probably never need to call this (see isEnabled() setEnabled() for boolean enable/disablle)
Repair the process for continued execution. this does NOT make a breakpoint inactive, but removes it’s “0xcc” from mem (see isEnabled() setEnabled() for boolean enable/dissable)
This will return the address for this breakpoint. If the return’d address is None, this is a deferred breakpoint which needs to have resolveAddress() called to attempt to set the address.
Return the current python string that will be run when this break is hit.
A callback to do housekeeping at the time the breakpoint is added to the tracer object. This should be used instead of activate for initialization time infoz to save on time per activate call...
Breakpoints may also extend and implement “notify” which will be called whenever they are hit. If you want to continue the ability for this breakpoint to have bpcode, you must call this method from your override.
Try to resolve the address for this break. If this is a statically addressed break, just return the address. If it has an “expression” use that to resolve the address...
An initialization callback which will be executed when the actual address for this breakpoint has been resolved.
Bases: vtrace.breakpoints.Breakpoint
A special breakpoint which will restore process state (registers in particular) when it gets hit. This is primarily used by the call method inside the trace object to restore original state after a successful “call” method call.
Additionally, the endregs dict will be filled in with the regs at the time it was hit and kept until we get garbage collected...
Bases: vtrace.breakpoints.Breakpoint
This type of breakpoint is exclusivly for marking and code-coverage stuff. It removes itself. (most frequently used with a continued trace)
Bases: vtrace.breakpoints.Breakpoint
A special breakpoint type which will produce vtrace snapshots for the target process when hit. The snapshots will be saved to a default name of <exename>-<timestamp>.vsnap. This is not recommended for use in heavily hit breakpoints as taking a snapshot is processor intensive.
Bases: vtrace.breakpoints.Breakpoint
When hit, take the tracer out of run-forever mode and remove this breakpoint.
Bases: vtrace.breakpoints.Breakpoint
This breakpoint will turn off RunForever mode on the tracer object when hit. it’s a good way to let things run on and on processing exceptions but stop when you get to this one thing.
Bases: vtrace.breakpoints.Breakpoint
A breakpoint which will record how many times it was hit (by the address it was at) as metadata for the tracer.
Some tools that require the envi framework to be installed
Bases: vtrace.Trace, vtrace.platforms.base.TracerBase
Wrap an arbitrary emulator in a Tracer compatible API.
Vtrace notitifers base classes and examples
Vtrace supports the idea of callback notifiers which get called whenever particular events occur in the target process. Notifiers may be registered to recieve a callback on any of the vtrace.NOTIFY_FOO events from vtrace. One notifier may be registered with more than one trace, as the “notify” method is passed a reference to the trace for which an event has occured...
Bases: vtrace.notifiers.Notifier
A notifier which will distributed notifications out to locally registered notifiers so that remote tracer’s notifier callbacks only require once across the wire.
Bases: object
The top level example notifier... Anything which registers itself for trace events or tracegroup events should implement the notify method as shown here.
Bases: vtrace.notifiers.Notifier
QtGui objects which assist in GUIs which use vtrace parts.
Bases: vqt.tree.VQTreeModel
Bases: envi.qt.VQMemoryMapView, vtrace.qt.VQTraceNotifier
A memory map view which is sensitive to the status of a trace object.
Bases: vqt.tree.VQTreeModel
Bases: vqt.tree.VQTreeView
Bases: vqt.tree.VQTreeModel
Bases: vqt.tree.VQTreeView, vtrace.qt.VQTraceNotifier
A pure “list view” object for registers
Bases: PyQt4.QtGui.QWidget
A register view which includes the idea of “sub views” for particular sets of registers per-architecture.
Bases: vqt.tree.VQTreeModel
Bases: vtrace.notifiers.Notifier
A bit of shared mixin code for the handling of vtrace notifier callbacks in various VQTreeViews...
Cobra integration for remote debugging
Bases: cobra.CobraProxy
A “factory” object for creating tracers and wrapping them up in a proxy instance to the local server. This object is shared out via the pyro server for vtrace clients.
If necissary, start a callback daemon. Return the ephemeral port it was bound on.
All the code related to vtrace process snapshots and TraceSnapshot classes.
Bases: vtrace.Trace, vtrace.platforms.base.TracerBase
A tracer snapshot is similar to a traditional “core file” except that you may also have memory only snapshots that are never written to disk.
TraceSnapshots allow you to take a picture of a process from a given point in it’s execution and manipulate/test from there or save it to disk for later analysis...
Extra utilities for vtrace
A trace-manager is a utility class to extend from when you may be dealing with multiple tracer objects. It allows for persistant mode settings and persistent metadata as well as bundling a DistributedNotifier. You may also extend from this to get auto-magic remote stuff for your managed traces.
Deliver a local event to the DistributedNotifier managing the traces. (used to locally bump notifiers)
Watchpoint Objects
Bases: vtrace.watchpoints.Watchpoint
A special “watchpoint” that uses memory permissions to watch for accesses to whole memory maps. This requires OS help and only works on platforms which support: * platformProtectMemory() * signal/exceptions which denote the fault address on SEGV
NOTE: These must be added page aligned
Bases: vtrace.breakpoints.Breakpoint
The basic “break on access” watchpoint. Extended from Breakpoints and handled almost exactly the same way...