Coding for love

If you’re a Maya API developer, you’ve probably wanted to pull your hair out several times. But all in all, the API itself isn’t that bad. The Houdini HDK is a mess. Seems like they designed it that way so you wouldn’t want to use it, unless you have to. I’ve heard from some people that the XSI API isn’t so bad, just a bit limiting. I don’t know anything about the 3DS Max or Blender API.

Developing Maya plug-ins generally means crashing every now and again. Or maybe more often than any reasonable person is willing to admit. Why so many crashes when working with the Maya API? Simple answer: someone poked a hole in the data condom.

What? Okay, on a more serious not, I think that most crashes in Maya usually begin life as something that resembles an out of bounds operation. Access an array out of bounds for write, will probably crash. Sometimes it will surprise you. So knowing this and not knowing at all how Maya stores the data internally…why in the world is there not away to access the raw elements of a MFloatPointArray? I mean, if you’re going to crash by making an silly mistakes why can’t you be allowed to make a really silly mistake?

Again, I don’t know how Maya stores the arrays internally. It could be doing some magic. But you could speculate that for performance reasons, the array might be contiguous…maybe? So assuming it is, wouldn’t it be nice if you could set the length and get the pointer?

So, lets say you’re writing a Maya plug-in (or plugin) in Linux one day and you load it up and it crashes Maya. You’re using GCC 4.1 since that’s what’s recommended and you like following recommendations. Okay, maybe not – but for the sake of argument, lets say you do. Back to the crashing plug-in…you do some debugging, and you find out that it’s in the constructor of one of your classes called Matrix. You comment some code out and you fiddle around, but now it crashes in a different place. You do some more debugging and find out it’s now in the destructor of the same class, but nothing has changed. And you notice it looks some thing like:

...
Reading symbols from /usr/autodesk/maya2009-x64/lib/libOpenMayaUI.so...done.
Loaded symbols for /usr/autodesk/maya2009-x64/lib/libOpenMayaUI.so
Reading symbols from /usr/autodesk/maya2009-x64/lib/libOpenMayaRender.so...done.
Loaded symbols for /usr/autodesk/maya2009-x64/lib/libOpenMayaRender.so
0x00007f2c1bf4cff1 in nanosleep () from /lib/libc.so.6
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f2c27141780 (LWP 4739)]
0x00007f2c1ea508a3 in Matrix::Matrix () from /usr/autodesk/maya2009-x64/lib/libAnimSlice.so
(gdb) bt
#0  0x00007f2c1ea508a3 in Matrix::Matrix () from /usr/autodesk/maya2009-x64/lib/libAnimSlice.so
...

- or -

...
Reading symbols from /usr/autodesk/maya2009-x64/lib/libOpenMayaUI.so...done.
Loaded symbols for /usr/autodesk/maya2009-x64/lib/libOpenMayaUI.so
Reading symbols from /usr/autodesk/maya2009-x64/lib/libOpenMayaRender.so...done.
Loaded symbols for /usr/autodesk/maya2009-x64/lib/libOpenMayaRender.so
0x00007f88621a3ff1 in nanosleep () from /lib/libc.so.6
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f886d398780 (LWP 4385)]
0x00007f8864ca811a in Matrix::~Matrix () from /usr/autodesk/maya2009-x64/lib/libAnimSlice.so
(gdb) bt
#0  0x00007f8864ca811a in Matrix::~Matrix () from /usr/autodesk/maya2009-x64/lib/libAnimSlice.so
...

…why is the Matrix class constructor and destructor being called in libAnimSlice.so?
So you decide to dump the symbols from the respective .so files and take a look:

> nm yourPlugin.so | grep Matrix --color
...
000000000002630e W _ZN6MatrixC1ERKS_
000000000002b664 T _ZN6MatrixC1ERSs
000000000002b81c T _ZN6MatrixC1Ef
000000000002b85c T _ZN6MatrixC1Ev
000000000002b740 T _ZN6MatrixC2ERSs
000000000002b83c T _ZN6MatrixC2Ef
000000000002b88c T _ZN6MatrixC2Ev
000000000002638e W _ZN6MatrixD1Ev
000000000002e49e T _ZN6MatrixaSERKS_
000000000003e476 W _ZN6MatrixclEjj
000000000002e55c T _ZN6MatrixmLERKS_
000000000002e5d0 T _ZN6MatrixmlERKS_
...

> nm /usr/autodesk/maya2009-x64/lib/libAnimSlice.so | grep Matrix --color
...
00000000002198a0 T _ZN6MatrixC1ERKS_
0000000000219a00 T _ZN6MatrixC1Ei
0000000000219c00 T _ZN6MatrixC1EiiPd
0000000000219e20 T _ZN6MatrixC1Eiid
0000000000219890 T _ZN6MatrixC2ERKS_
0000000000219b00 T _ZN6MatrixC2Ei
0000000000219d10 T _ZN6MatrixC2EiiPd
0000000000219f20 T _ZN6MatrixC2Eiid
000000000021a110 T _ZN6MatrixD1Ev
000000000021a010 T _ZN6MatrixD2Ev
000000000021a080 T _ZN6MatrixaSERKS_

I colored the matching symbols in Red and Blue. I haven’t had the time to figure out exactly what the T/W characters mean. The 64bit numbers in front of that are the offset (or something to that effect) in to the library of where the symbols live.

I’m sure there is a gcc or ld flag somewhere that throws an error to prevent things like this. I haven’t had time to research that either. I got around the problem by just renaming my Matrix class to Matrix44. C++ namespaces works just as well. If I had to guess, it probably has something to do with symbol visibility or a specific ld flag.