New Debugger branch - Functional

[HyperHacker]

I've been busy the last few days, but I've almost finished a variable viewer/editor, for convenient monitoring of multiple memory locations. I've hit a bit of a snag with floats though - the format my system wants to use doesn't seem to match the N64's, and I found a magic snippet that does some pointer casting to convert from N64 format, but I have no idea how to convert back, so entering new values doesn't work.

Not a whole lot of work left to do on it, the hard part will be finding free time and figuring out how to do things with Gtk since I've never used it before.

 

[HyperHacker]

Hm, the "next" button on the disassembler is doing some odd things. Occasionally it jumps to a random instruction, usually after switching to another window, as if the game had been running briefly. Also it doesn't seem to follow JR instructions, just skips right on by them.

Anyway here's a WIP patch because I can't figure out why gtk_file_chooser_dialog_new() is hanging, seems to be a GTK bug.

-Removed unused variables from init_memedit().
-Added variable list.
-Increased breakpoint window width a bit to prevent Bs at edge looking like Es
-Added auto-refresh checkbox to memory editor.
-Replaced some printf()s with alert_message()s.

The variable list isn't finished, so there are still some quirks:
-Type column is not a dropdown list. Choose a type by entering a hex number (0-C). Add 0x80 (i.e. prefix with 8) for a pointer.
-Import loads from a hardcoded path, because it crashes when I try to display a file open dialog. Export is not implemented yet.
-Float and Double values display correctly, but attempting to edit them writes the wrong values, because I have no idea how the magic conversion works.
-No checking for invalid values or addresses.

Too long to post now: patch, varlist.c, varlist.h

 

[DarkJezter]

Ok, I've been AWOL for a while here, but I've had the chance to solve some of the technical challenges for the disassembler frontend. I also have some free time tomorrow that I'm going to use to get everything polished and ready. Hopefully this will leave enough time to merge the branch, and also get the opportunity for some other people to try play with it before the 1.4 release.

Richard: you have a timeline in mind for the 1.4 release?

Hyperhacker: sorry btw about the latest patches, I'll be including them in the branch as soon as I have the scroll changes merged into the branch. (read end of Thursday at the latest)

 

[HyperHacker]

Well I can't do much until I figure out what's up with the file chooser dialog anyway. Any ideas?

 

[DarkJezter]

Haven't the foggiest, nothing appears to be obviously wrong with it. I'll play with it after I commit my changes. Knowing this, those patches probably won't make it into the branch until we get that fixed.

If you're have development questions, you may want to consider joining the irc channel, there's a sticky thread with information on it if you didn't already know. May make collaboration such as this a bit easier

 

[HyperHacker]

Well it's still functional, only the import/export feature needs that, so just disabling that button (or changing the hardcoded path to one that will work on other machines) would probably be enough for now. It's a GTK issue as best I can tell (there's been similar problems before), so I filed a bug, no response yet. Unfortunately it's not giving any sort of stack dump or trace so there's not much I can do to tell what's wrong with it.

Knowing my luck, it's probably some memory corruption or something somewhere, but I've narrowed it down as best I can (disabled all other debugger windows, etc) so if that is the case, it's unlikely in any code I've written.

 

[DarkJezter]

Hyper:
I was playing with the patch, and I didn't have any problems with the file load dialog after I commented the static filename code. Not sure if this is a system specific bug, or not, but I'll include it in the branch and we can decide where to go from there... but so far it works great for me.

Oh, and unfortunately, I did drop the code to make the columns of the disassembler resizable... I'm using fixed_height, which requires fixed column widths to be set if I'm not mistaken... however it's supposed to be for performance enhancement... and I haven't noticed if it's made any difference or not... I may revisit this later and try it with resizable columns instead.

 

[HyperHacker]

Well as it is, the columns are too small for longer opcodes like BGEZALL.

The only thing I've heard about that bug is a problem on 64-bit systems, but mine's 32-bit, so I don't know what's going on there.

 

[DarkJezter]

The debugger is nearly ready to be merged back into the trunk. The next phases of work on the debugger will involve patching the recompiler, and will likely need to wait until after the 1.4 release.

The current most pressing issues with the debugger:

1) Does not function as expected with dynamic-recompiler and repairing will require some alterations. In the meantime, disabling single-step execution and breakpoints while using the dynamic-recompiler will suffice. While it will probably not be ready for the 1.4 release, we can expect to see full debugger support for the recompiler in the 1.5 release.

2) The new components have been included from a couple contributors, and have not yet been thoroughly tested. Anyone willing to try them please do.

3) There is still no way to edit register values, this will also likely not be ready for the 1.4 release, but is anticipated to be ready soon after with full support for the recompiler. However, now thanks to HyperHacker there is a working memory editor that is able to edit r/w ram values, as well as monitor variables.

4) The GUI is VERY temporary, and very ugly. Currently over 8 seperate windows are open when using the debugger. The debugger is going to include full support for enabling and disabling these windows individually in time for the merge, through the menus in the rom-browser window. What we're hoping to begin afterward is a completely dockable interface that will also include the rom-browser.

At this point, since there's a push for release, I'm eager to ask anyone to try it and post if you find any serious bugs that need repair for the release.

 

[Tillin9]

Hm... it compiles with a lot of warnings but seems to be working. Also, you should mention that dis-asm.h (in binutils-dev on Debian based distros) is now needed.

I tried the debugger, the double scrollbar on the desassembler (is that the right word?) seems confusing, but more importantly, the font is too large so I couldn't see the registers. This is odd considering that the font was fine on the other windows. I can post an image if desired.

Some documentation would also be very useful, both for using the debugger and on figuring out how to refine the interface. We do seem to need a view menu, so I'll give that some more priority. Implementing one and dockable widgets (via GtkHandleBox) for the toolbar and filter is on my todo. I can get that done fairly rapidly this week-end and help out with some of the gtk stuff.

 

[HyperHacker]

Yeah, documentation is needed. "Desassembler" should be "Disassembler", but it's written like that in the code too.

 

[DarkJezter]

I'm guessing the mispell there is due to the original dev being French


[edit]
Oh and on the scrollbars... one on the right is linear, the one on the left is logarithmic. Try scrolling over a couple lines with each, and you'll see the difference. I think it may be a bit more obvious if we put one on each side of the window, but that's more an issue of aesthetics.

 

[HyperHacker]

So I've been finding some difficulty actually using this, because of what would seem to be a cache or recompiler (even using the interpreter core). Changing instructions in memory has no effect; the disassembler shows the new instruction, but the CPU still executes the old one. That makes it somewhat difficult to hack. I tried changing it as soon as it was in RAM when the game started up (long before it would have executed), but typically the game just re-copies the block, changing it back; putting a breakpoint a few instructions before doesn't work, as it's already too late; changing it here has no effect.

Also, I was considering a way to log all RSP commands executed during one frame, effectively dumping all display lists. Could be useful in picking apart Mario Kart. I suppose that's better done in a graphic plugin, in which case, what's the best way to go about implementing debug features in plugins?

 

[DarkJezter]

I completely agree, especially for working on plugin artifacts. The interpreter does a pre-pass to populate a series of structures for each instruction, interestingly enough they are the same structures used for dynamic recompiliation. There's this and another issue with the recompiler that I want to work on for the disassembler, but once we're able to switch between the recompiler and interpreter on the fly (which is my next goal with the debugger) I'll be investigating both updating the recompiled code as well as the interpreted code. Right now, this is on hold until after 1.4 is in it's final release, saving the mean time to play with compiling on different platforms (I'm waiting on ram to get the win32 system ready, I have osx ready and waiting)

 

[Richard42]

So I've been finding some difficulty actually using this, because of what would seem to be a cache or recompiler (even using the interpreter core). Changing instructions in memory has no effect; the disassembler shows the new instruction, but the CPU still executes the old one.

I merged the debugger branch into the trunk, so to continue with this testing/development you'll have to nuke your local copy of the branch and check out the trunk.

Also, I committed a change in rev 503 which should fix this problem for both the interpreter and recompiler. Please test and let me know if it works or not.

 

[HyperHacker]

Yeah, it seems to be working now, but "make DBG=1 install" fails.

./install.sh /usr/local
Installing Mupen64Plus to /usr/local
/usr/bin/install: cannot stat `mupen64plus': No such file or directory

It looks for mupen64plus instead of mupen64plus_dbg.

Also if you add some breakpoints, close the window, and open it again, they aren't listed; there needs to be an "update_breakpoints();" at the end of init_breakpoints() in breakpoints.c.

As for interface improvements, two things I can think of right away:
1) Remember window positions and sizes.
2) Put the video output in place of the ROM list (keeping the toolbar and status bar), instead of a separate window.

 

[Richard42]

HyperHacker, I have fixed these 2 issues in SVN r532. For the install problem, I got rid of the mupen64plus_dbg executable so we always build just 'mupen64plus' even when the graphical debugger is built. I also added the updated_breakpoints() call. Please update and re-test.

 

[HyperHacker]

Hey, sorry I haven't been around lately. I just tried to compile the latest trunk build:

TexLoad4b.h: In function ‘DWORD Load4bIA(unsigned char*, unsigned char*, int, int, int, int, int)’:
TexLoad4b.h:3059: error: PIC register ‘ebx’ clobbered in ‘asm’
TexLoad4b.h: In function ‘DWORD Load4bI(unsigned char*, unsigned char*, int, int, int, int, int)’:
TexLoad4b.h:3718: error: PIC register ‘ebx’ clobbered in ‘asm’
TexLoad8b.h: In function ‘DWORD Load8bIA(unsigned char*, unsigned char*, int, int, int, int, int)’:
TexLoad8b.h:1090: error: PIC register ‘ebx’ clobbered in ‘asm’
TexLoad8b.h: In function ‘DWORD Load8bI(unsigned char*, unsigned char*, int, int, int, int, int)’:
TexLoad8b.h:1230: error: PIC register ‘ebx’ clobbered in ‘asm’
TexLoad32b.h: In function ‘DWORD Load32bRGBA(unsigned char*, unsigned char*, int, int, int, int, int)’:
TexLoad32b.h:469: error: PIC register ‘ebx’ clobbered in ‘asm’
TexConv.h: In function ‘void TexConv_ARGB1555_ARGB4444(unsigned char*, unsigned char*, int, int)’:
TexConv.h:125: error: PIC register ‘ebx’ clobbered in ‘asm’
TexConv.h: In function ‘void TexConv_AI88_ARGB4444(unsigned char*, unsigned char*, int, int)’:
TexConv.h:193: error: PIC register ‘ebx’ clobbered in ‘asm’
TexConv.h: In function ‘void TexConv_AI44_ARGB4444(unsigned char*, unsigned char*, int, int)’:
TexConv.h:323: error: PIC register ‘ebx’ clobbered in ‘asm’
TexConv.h: In function ‘void TexConv_A8_ARGB4444(unsigned char*, unsigned char*, int, int)’:
TexConv.h:459: error: PIC register ‘ebx’ clobbered in ‘asm’
make[1]: *** [TexCache.o] Error 1

 

[Richard42]

This should be fixed in r838.

 

[HyperHacker]

Well I've been continuing to work on the version I have (needed to to get my hacks done ) so hopefully that gets fixed soon so I can migrate it all to the latest build.

There seems to be a dependency on specific versions of libopcodes and libfd? It won't run on my laptop because it can't find these files, but binutils and binutils-dev are installed.

I finally found what was causing the random stack smashing when interrupting in the debugger. regCop1.c, at the beginning of update_FGR():

char txt[24];

The size of this buffer needs to be increased - "double"-type numbers can reach 308 digits, and using %f will print them all.

It still crashes if I try to remove all breakpoints at once. Haven't looked into that yet.

I also figured out what's up with the "random" jumping around in the disassembler. It doesn't handle branches properly. If you step over a branch it steps to the second instruction after that and then branches, and if you step over a JAL, it doesn't follow it. Similarly if you put a breakpoint on an instruction two after a branch, the breakpoint hits when the branch is executed, even if it would have skipped the actual instruction the breakpoint is on.

I added a logging function to glN64, but how would I go about communicating with the plugin from the GUI code to tell it when to start and stop logging?