gdb and debugging plugins 2003-11-27 - By deane@(protected)
Back
On Mon, Nov 24, 2003 at 11:55:58AM +0100, Andrew Cammarano wrote:
>
> Does anyone know of a good tutorial on using gdb to debug .so's. Better
> yet, does anyone want to outline the specific steps I need to take to step
> through my plugin code using gdb? Not sure what commands to call or if I
> need to attach to the Maya process..etc.
Sure, I do it all the time.
First off, when you compile your plugin, remember to get rid of the '-O'
switch, unless it says '-O0' and add '-g' and '-gstabs+'. That latter is
very important as otherwise gdb will have a lot of trouble finding
non-static methods within classes.
Next, download and install 'ddd' so that you can debug without going crazy.
ddd provides a graphical front-end to gdb which provides a scrollable
display of source code on which you can click to set breakpoints, etc.
Very useful.
When you're ready to run maya, use the following command:
maya -d ddd
After the ddd window comes up, start Maya by using Program -> Run Again.
(If you need to give Maya some command line args, then use Program -> Run,
which will pop up a window where you can fill in the args.)
Once Maya is up, load your plugin: ddd/gdb won't know about your sources
until you do.
Now you can set breakpoints in your source code using Source ->
Breakpoints. The window which comes up will have a little stop-sign icon
near the top. Click on that to get an input field.
If you want to set a breakpoint at the start of a particular method, then
enter:
myClass::myMethod
If you want to set a breakpoint at line 123 of source file 'myFile.cpp',
then enter:
myFile.cpp:123
Alternatively, you can use File -> Open Source to display a specific source
file, then scroll to where you want the breakpoint, right-click in the
margin to the left of the line and select 'Set Breakpoint'.
Now do whatever you need to do in Maya to hit the breakpoint. When you do,
execution will halt and ddd will pop up a palette of buttons to let you do
things like single-step, step into functions, step over functions, continue
normal execution, and so on.
While execution is halted, ddd will also display the page of code in which
it halted, with the exact line indicated with an arrow. You can see the
value of a variable by either hovering the cursor over its name in the
displayed source, or else highlighting its name and clicking the 'Print'
button. The same thing works for more complex expressions: highlight the
expression and click 'Print'. Note, however, that because the Maya API
hides most of its internal data behind opaque types, you will rarely be
able to see anything useful within a Maya API variable. For example, you
won't be able to see the contents of an MString.
You can see the callback stack by using Status -> Backtrace, which is
especially useful for investigating crashes. The backtrace displayed is
for the current thread. Usually that's the one you want, but for some
particularly tricky bugs (e.g. when Mental Ray is involved) you may need to
see the call stack for a different thread. You can do that by using Status
-> Threads to select a different thread, then using the Backtrace function
again to see the new thread's stack.
That covers about 95% of what I do in ddd/gdb. For the rest, cruise
through the menus. Also note that every menu item in ddd displays the
corresponding gdb command in the command-line portion of the ddd window, so
you can use that to learn about gdb commands, if you want.
===========================================================================
- deane Gooroos Software: Plugging you into Maya
Visit http://www.gooroos.com for more information
-- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---
List-help: <mailto:listar@(protected)?Subject=help>
List-archive: <http://www.highend3d.com/maya/devarchive/>
|
|
