Porting Mupen64Plus to Android

Ari64 · Dec 11, 2010

paulscode said:
I'm confused.. I thought that since the assembly code is being built into the core library rather than as a separate library, that I can use the linking options "-Xlinker --section-start -Xlinker .init=0x08000000" set up the memory layout correctly.

If the core is statically linked, and not in a .so, then that will work.

When I run the binary that I built from the original source based on mupen64plus v1.5, /proc/self/maps looks like this:

Code:

00008000-00021000 r--p 00000000 08:01 26198025   /media/sda1/devel/mupen64plus (ld.so symbol table)
07000000-08000000 rwxp 07000000 00:00 0                                        (dynarec target)
08000000-080ea000 r-xp 00020000 08:01 26198025   /media/sda1/devel/mupen64plus (.init, .text)
080f1000-080f6000 rw-p 00109000 08:01 26198025   /media/sda1/devel/mupen64plus (.data)
080f6000-09ed0000 rw-p 080f6000 00:00 0          [heap]                        (.bss)
40000000-4001d000 r-xp 00000000 b3:02 17317      /lib/ld-2.9.so
(...lots more shared libraries...)
450c8000-458c7000 rw-p 450c8000 00:00 0
458c7000-458d8000 r--p 00000000 08:01 24535139   /media/sda1/devel/fonts/font.ttf
458d8000-4592c000 rw-p 458d8000 00:00 0
80000000-80800000 rw-p 80000000 00:00 0                                        (rdram)
bedcc000-bede2000 rw-p befea000 00:00 0          [stack]

Using gdb, I can verify that the relevant offsets are within 32M

Code:

(gdb) print/x &dyna_linker
$1 = 0x80cacc4
(gdb) print 0x80cacc4-0x7000000
$2 = 17607876

paulscode · Dec 11, 2010

Ari64 said:
If the core is statically linked, and not in a .so, then that will work.

Gotcha ... so you statically linked the core in your Pandora port.

That is certainly one option for this project, but not entirely uncomplicated. Firstly, I'd need to come up with a different workaround for the build bug I mentioned. I'd probably have to build the entire thing from the command line, which would be a real pain to debug (I much prefer development in an IDE). Secondly, I'd have to research if an executable could be packaged into an .apk for distribution on the Android market (if not, then this is not a viable option). Finally, I would have to make changes to Pelya's SDL port, since it is set up to link with with Java dynamically, not for compiling into an executable.

I'm looking into some of your other suggestions as well, just so I have some other options if that one doesn't pan out.

First up, it turns out there is not enough room for allocating 16MB in the .bss section. For the full core library, objdump indicates that the size of .bss is 01b07ce4 (over 28MB), which leaves just under 4MB before going over that 32MB limit.

If I compile linkage_arm.s by itself, the .bss section is only 4006b0 (about 4MB), but in my limited understanding of the issue, this is not an option either, since separating linkage_arm from the rest of the core would also separate the text segment and the buffer for the dynamically generated code, and thus the two would likely not be within the 32MB limit either (correct me if I am wrong about this analysis) .

Another suggestion you mentioned was setting up trampolines for jumping to locations that are further away. I understand that this would be a considerably more complicated solution, so let me study up on assembly before I start discussing this option.

Even though there is not enough room for the "16MB in .bss" option, could you still answer a couple more questions for me (for educational purposes)?
1) Would I allocate the 16MB of memory in new_dynarec_init() after pointing BASE_ADDR to the correct location, or would I allocated it in linkage_arm.s?
2) If the later were true for question 1, what are the assembly commands to allocate a particular amount of memory, and how do I point BASE_ADDR to that memory?
3) If the later were true for question 1, I also would remove the "mmap" command from new_dynarec_init(), right? Would there be any other locations in the code that I would likely need to edit as well?

If I do go with non-static libraries, I would really prefer to compile linkage_arm.s by itself into a separate .so from the core if possible (it would save me a butt-load of development time down the road when I'm debugging other parts of the core and upgrading to new versions of Mupen64Plus as they come out, since I wouldn't have to keep working around that compiler bug). What kind of considerations would be necessary for this to work (I assume the "trampoline" option would be the only viable one in this case)

Ari64 · Dec 11, 2010

Your first Android project, and you choose this...

paulscode said:
Gotcha ... so you statically linked the core in your Pandora port.

Mupen64Plus v1.5 was designed that way. The shared library is new in 2.0. This is one of the main reasons why this code hasn't been merged yet.

paulscode said:
First up, it turns out there is not enough room for allocating 16MB in the .bss section. For the full core library, objdump indicates that the size of .bss is 01b07ce4 (over 28MB), which leaves just under 4MB before going over that 32MB limit.

Try rearranging the bss segment by putting other stuff at the end.

paulscode said:
If I compile linkage_arm.s by itself, the .bss section is only 4006b0 (about 4MB), but in my limited understanding of the issue, this is not an option either, since separating linkage_arm from the rest of the core would also separate the text segment and the buffer for the dynamically generated code, and thus the two would likely not be within the 32MB limit either (correct me if I am wrong about this analysis) .

Yes. By the way, those 0x4006b0 bytes are explicitly allocated as such:

Code:

        .space  64+16+16+8+8+8+8+256+8+8+128+128+128+16+8+132+4+256+512+4194304

paulscode said:
Another suggestion you mentioned was setting up trampolines for jumping to locations that are further away. I understand that this would be a considerably more complicated solution, so let me study up on assembly before I start discussing this option.

If you don't want to do that, I'll look into it when I have some time (maybe after the holidays). Otherwise, look at the current memory layout and try to come up with something that will work.

paulscode said:
Even though there is not enough room for the "16MB in .bss" option, could you still answer a couple more questions for me (for educational purposes)?

There probably is enough space, if you're creative about it. If you want to allocate 16MB in .bss, then add 16MB to the space in assem_arm.s, and set up a pointer to it. Besides BASE_ADDR, there's a jump to this location in new_dyna_start. You'll probably need to re-mmap this region to set the exec flag.

paulscode · Dec 12, 2010

Ari64 said:
Your first Android project, and you choose this...

Haha, people have made similar comments before. I tend to tackle projects which deal with topics I have little experience in. I really enjoy being challenged and learning new things from experienced developers.

Ari64 said:
Try rearranging the bss segment by putting other stuff at the end.

Ah, I see -- so the idea here is to compile parts of the code together so that even though .bss goes over 32MB, the necessary components are still within 32MB of each other. How is this done, by choosing the order of the .o files that are linked together into the .so library? If that's how its done, which files need to be grouped together, besides the ones in the new_dynarec and r4300 folders of course?

Ari64 said:
Yes. By the way, those 0x4006b0 bytes are explicitly allocated as such:

Code:

.space 64+16+16+8+8+8+8+256+8+8+128+128+128+16+8+132+4+256+512+4194304

Oh, yes, I saw this line and was wondering if thats what it is for. So the sum of numbers is just for keeping track of what is used by the various globals, correct? Do the following changes look reasonable?

Code:

.global extra_memory

Code:

        .space  64+16+16+8+8+8+8+256+8+8+128+128+128+16+8+132+4+256+512+4194304+16777216

Code:

extra_memory = memory_map + 419430
	.type	extra_memory, %object
	.size	extra_memory, 16777216

...in new_dyna_start (this one I am unsure about):

Code:

mov pc, #extra_memory

And how would I go about pointing BASE_ADDR to this area once I allocate it? I see I can use something like "extern void *extra_memory" in the c code, but I am confused whether extra_memory should be "externed" as a pointer or as a void "blob" (I'll keep tinkering with this, as I'm sure I am close to the correct setup, I just have to get BASE_ADDR working as an integer/pointer for a couple of other places in new_dynarec.c to compile)

Ari64 said:
You'll probably need to re-mmap this region to set the exec flag.

Ok, I'll give it a shot.

Chaoz · Dec 12, 2010

You can't link it into an executable. Android can only execute native code located in .so files.

paulscode · Dec 12, 2010

Chaoz said:
You can't link it into an executable. Android can only execute native code located in .so files.

That's what I thought. I don't want to require the end user to root his/her phone to install the emulator executable plus any missing Linux components needed to run it. I definitely want to go with one of these other options then. I'll see if I can get the "16MB in .bss" concept to work.

paulscode · Dec 12, 2010

Ok, so I think I'm almost there. Everything compiles, but I would like a quick sanity check, since I've made a couple of assumptions (and it seems like this was a bit too easy to be correct). Here are the changes I made:

In linkage_arm.s:

Code:

/* added a new global, at the top where all the other globals are listed */
.global extra_memory

Code:

/* appended 16MB to the end of the allocated space */
        .space  64+16+16+8+8+8+8+256+8+8+128+128+128+16+8+132+4+256+512+4194304+16777216

Code:

/* defined the extra_memory global, where all the other globals are defined*/
extra_memory = memory_map + 419430
	.type	extra_memory, %object
	.size	extra_memory, 16777216

Code:

/* jumped to the new location, in new_dyna_start */
	mov pc, #extra_memory /*mov	pc, #0x7000000*/

In assem_arm.h:

Code:

extern void *extra_memory;  // assuming it can be "externed" as a void*
#define BASE_ADDR ((int)extra_memory)
//#define BASE_ADDR 0x7000000 // Original code generator target address

If these are correct, I'll focus next on how to read /proc/(pid)/maps (or equivalent) on an Android device, so I can start figuring out where things are being loaded in memory.

Ari64 · Dec 12, 2010

paulscode said:
Ah, I see -- so the idea here is to compile parts of the code together so that even though .bss goes over 32MB, the necessary components are still within 32MB of each other. How is this done, by choosing the order of the .o files that are linked together into the .so library? If that's how its done, which files need to be grouped together, besides the ones in the new_dynarec and r4300 folders of course?

Yes. Either rearrange how the .o files are linked, or remove some of the larger declarations from other files and put them in linkage_arm.s in the order that you want.

paulscode said:
So the sum of numbers is just for keeping track of what is used by the various globals, correct? Do the following changes look reasonable?

Yes. You might want to put that 16MB at the beginning though. Just be sure to keep everything from dynarec_local through memory_map intact. There's a 4096-byte limit here, another quirk of the ARM instruction set. Normally it is not necessary to specify the exact offset of every variable, but there are some specific requirements here, so the declaration is a bit unusual.

paulscode said:
...in new_dyna_start (this one I am unsure about):

You'd need something like:

Code:

 ldr pc,pointer
...
pointer:
 .word extra_memory

paulscode said:
And how would I go about pointing BASE_ADDR to this area once I allocate it? I see I can use something like "extern void extra_memory" in the c code, but then I'm not sure what to #define BASE_ADDR as. I am confused whether extra_memory should be "externed" as a pointer or as a void "blob" (I'll keep tinkering with this, as I'm sure I am close to the correct setup, I just have to get BASE_ADDR working as an integer/pointer for a couple of other places in new_dynarec.c to compile)

extern char extra_memory[16777216];
#define BASE_ADDR extra_memory

paulscode said:
Ari64 said:

You'll probably need to re-mmap this region to set the exec flag.

Click to expand...

Ok, I'll give it a shot.

If you're going to call mmap on this region, make sure it's page-aligned. Use .align 12 and make the size a multiple of 4096.

paulscode · Dec 12, 2010

Thanks, I'll work on the suggested changes. In particular, I'm not clear on the assembly (I'll have to look at this a little closer)

paulscode · Dec 12, 2010

Ok, I think I got it (but I need another sanity check, since I'm dealing with a programming language I have no prior experience with).

First up, I moved extra_memory to the beginning instead of the end, and added ".align 12" (and of course 16777216 is already a multiple of 4096). I've made some assumptions here about the proper syntax by studying how you wrote your code:

Code:

	.bss
	.align	12
	.type	extra_memory, %object
	.size	extra_memory, 16777216	
extra_memory:
	.space	16777216+64+16+16+8+8+8+8+256+8+8+128+128+128+16+8+132+4+256+512+4194304
dynarec_local = extra_memory + 16777216
	.align	4	
	.type	dynarec_local, %object
	.size	dynarec_local, 64

Next, I switched things up in new_dyna_start:

Code:

	.align	2
	.global	new_dyna_start
	.type	new_dyna_start, %function
new_dyna_start:
	ldr	r12, .dlptr
	mov	r0, #0xa4000000
	stmia	r12, {r4, r5, r6, r7, r8, r9, sl, fp, lr}
	sub	fp, r12, #28
	add	r0, r0, #0x40
	bl	new_recompile_block
	ldr	r0, [fp, #next_interupt-dynarec_local]
	ldr	r10, [fp, #reg_cop0+36-dynarec_local] /* Count */
	str	r0, [fp, #last_count-dynarec_local]
	sub	r10, r10, r0
	ldr pc, .pointer /*mov	pc, #0x7000000*/
.dlptr:
	.word	dynarec_local+28
.pointer:
	.word	extra_memory
	.size	new_dyna_start, .-new_dyna_start

And finally, I changed BASE_ADDR in assem_arm.h:

Code:

extern char extra_memory[16777216];
#define BASE_ADDR ((int)(&extra_memory))

As you can see, I altered your suggestion a little, because BASE_ADDR is used as an integer representation of a pointer in a couple of places in new_dynarec.c. For example:

Code:

  int end=((((int)out-BASE_ADDR)>>(TARGET_SIZE_2-16))+16384)&65535;

Everything compiles, but I want to make sure there isn't anything obvious wrong.

paulscode · Dec 12, 2010

My next problem is it seems that the proc/(id)/maps is always empty, even if I access it from the program itself using the following code:

Code:

char* execCmd( char* cmdline )
{
	__android_log_print( ANDROID_LOG_VERBOSE, "execCmd()", "Executing command '%s'", cmdline );
	strcpy( outBuffer, "" );
    FILE* pipe = popen( cmdline, "r");
    if( !pipe )
    {
        __android_log_print( ANDROID_LOG_VERBOSE, "execCmd()", "Failed to open a pipe" );
    	return NULL;
    }
    if( !feof( pipe ) )
    {
        if( fgets( outBuffer, MAX_COMMAND_OUTPUT_LENGTH, pipe ) != NULL )
        {
        	pclose( pipe );
            return (char*) &outBuffer;
        }
    }
    pclose(pipe);
    return NULL;
}

char* readProcname( char* pidStr )
{
	strcpy( cmdBuffer, "cat /proc/" );
	strcat( cmdBuffer, pidStr );
	strcat( cmdBuffer, "/cmdline" );
	return execCmd( (char*) &cmdBuffer );
}

int readMemoryMaps()
{
	char* procname = "paulscode.android.mupen64plus";
	char* procdir = "/proc";
	DIR* dip;
	struct dirent* dit;

	char* c;

	if( ( dip = opendir( procdir ) ) == NULL )
	{
	    __android_log_print( ANDROID_LOG_ERROR, "readMemoryMaps()", "Unable to open directory stream for '%s'", procdir );
	    return -1;
	}

	while( ( dit = readdir( dip ) ) != NULL )
	{
		c = readProcname( dit->d_name );
		if( c != NULL && strcmp( c, procname ) == 0 )
		{
			__android_log_print( ANDROID_LOG_VERBOSE, "readMemoryMaps()", "Found process '%s', pid='%s'", procname, dit->d_name );

			strcpy( cmdBuffer, "cat /proc/" );
			strcat( cmdBuffer, dit->d_name );
			strcat( cmdBuffer, "/maps > /sdcard/data/mp64pmapcontents.txt" );
			execCmd( cmdBuffer );

			return atoi( dit->d_name );
		}
	}
	return -1;
}

Output shows it finds the correct process id and map, and "cats" it to a file:

Code:

12-12 14:11:42.123: VERBOSE/execCmd()(7372): Executing command 'cat /proc/7372/cmdline'
12-12 14:11:42.123: VERBOSE/readMemoryMaps()(7372): Found process 'paulscode.android.mupen64plus', pid='7372'
12-12 14:11:42.123: VERBOSE/execCmd()(7372): Executing command 'cat /proc/7372/maps > /sdcard/data/mp64pmapcontents.txt'

It creates the file, but it is empty. Another method I used (more hackish, but it verifies that "maps" is empty) was to stick an infinite loop in the program just before the line that crashes it, then pull up the terminal on the phone and cat the file. This also results in no output.

There are a couple of things I can think of that might be causing this. Perhaps someone here might know (I'm asking around on various Android development forums as well):
1) proc/(id)/maps is only populated for statically-linked programs
2) It takes a long time for this file to be populated
3) The Android OS does not utilize this feature of Linux
4) I do not have the proper permissions to read the contents of this file
5) User error // probably the most likely reason

Are there other ways to determine where components are loaded in memory in relation to each other?

Ari64 · Dec 13, 2010

paulscode said:
My next problem is it seems that the proc/(id)/maps is always empty, even if I access it from the program itself using the following code:

You learn something new every day...

Code:

#include <stdio.h>

void readmemorymaps() {
  FILE *file;
  file=fopen("/proc/self/maps","r");
  if(file) {
    char buffer[65536];
    int size;
    size=fread(buffer,1,65536,file);
    fclose(file);
    if(size>0)   
    {
      file=fopen("/sdcard/data/mp64pmapcontents.txt","w");
      fwrite(buffer,1,size,file); 
      fclose(file);
    }
  }  
  else printf("Error opening /proc/self/maps\n");
}

paulscode said:
Are there other ways to determine where components are loaded in memory in relation to each other?

Use gdb to examine the pointers.

paulscode · Dec 13, 2010

Haha, that looks MUCH nicer than what I came up with.. I'll give it a shot.

paulscode · Dec 13, 2010

Excellent! That worked beautifully! Output, minus a whole bunch of the entries that were clearly unrelated:

Code:

00008000-00009000 r-xp 00000000 b3:35 16         /system/bin/app_process
00009000-0000a000 rwxp 00001000 b3:35 16         /system/bin/app_process
0000a000-002fd000 rwxp 00000000 00:00 0          [heap]
10000000-10001000 ---p 00000000 00:00 0 
10001000-10100000 rwxp 00000000 00:00 0 
4000b000-4000c000 r-xp 00000000 00:00 0 
41e3d000-41e4c000 rwxp 00000000 00:00 0 
41e4c000-41e82000 rwxp 00000000 00:00 0 
41e84000-41e87000 rwxp 00000000 00:00 0 
428dd000-42922000 rwxp 00000000 00:00 0 
4317d000-43212000 rwxp 00000000 00:00 0 
43402000-43413000 rwxp 00000000 00:00 0 
43473000-434a7000 rwxp 00000000 00:00 0 
436c4000-436cd000 r--s 00025000 fe:00 13531      /mnt/asec/paulscode.android.mupen64plus-1/pkg.apk
436cd000-436e0000 r-xp 00000000 b3:38 307275     /data/dalvik-cache/mnt@[email protected]@[email protected]
456af000-456b2000 r--s 00023000 fe:00 13531      /mnt/asec/paulscode.android.mupen64plus-1/pkg.apk
47363000-47364000 ---p 00000000 00:00 0 
47364000-47463000 rwxp 00000000 00:00 0 
47463000-47464000 ---p 00000000 00:00 0 
47464000-47563000 rwxp 00000000 00:00 0 
47679000-4767a000 ---p 00000000 00:00 0 
4767a000-47779000 rwxp 00000000 00:00 0 
47779000-4777a000 ---p 00000000 00:00 0 
4777a000-47879000 rwxp 00000000 00:00 0 
47879000-479a3000 r--s 00000000 fe:00 13531      /mnt/asec/paulscode.android.mupen64plus-1/pkg.apk
479a3000-47acd000 r--s 00000000 fe:00 13531      /mnt/asec/paulscode.android.mupen64plus-1/pkg.apk
47acd000-47b53000 rwxp 00000000 00:00 0 
47cb5000-47cb6000 ---p 00000000 00:00 0 
47cb6000-47db5000 rwxp 00000000 00:00 0 
486ed000-487b6000 rwxp 00000000 00:00 0 
49080000-49881000 rwxp 00000000 00:00 0 
81000000-8103c000 r-xp 00000000 b3:38 89198      /data/data/paulscode.android.mupen64plus/lib/libsdl-1.2.so
8103c000-8103e000 rwxp 0003b000 b3:38 89198      /data/data/paulscode.android.mupen64plus/lib/libsdl-1.2.so
8103e000-81040000 rwxp 00000000 00:00 0 
81100000-81162000 r-xp 00000000 b3:38 89238      /data/data/paulscode.android.mupen64plus/lib/libsdl_mixer.so
81162000-8116d000 rwxp 00061000 b3:38 89238      /data/data/paulscode.android.mupen64plus/lib/libsdl_mixer.so
8116d000-81190000 rwxp 00000000 00:00 0 
81200000-81230000 r-xp 00000000 b3:38 89287      /data/data/paulscode.android.mupen64plus/lib/libsdl_image.so
81230000-81231000 rwxp 0002f000 b3:38 89287      /data/data/paulscode.android.mupen64plus/lib/libsdl_image.so
81231000-81242000 rwxp 00000000 00:00 0 
81b00000-81b09000 r-xp 00000000 b3:38 89083      /data/data/paulscode.android.mupen64plus/lib/libapplication.so
81b09000-81b0e000 rwxp 00008000 b3:38 89083      /data/data/paulscode.android.mupen64plus/lib/libapplication.so
81b0e000-81b0f000 rwxp 00000000 00:00 0 
81c00000-81c03000 r-xp 00000000 b3:38 89086      /data/data/paulscode.android.mupen64plus/lib/libsdl_main.so
81c03000-81c04000 rwxp 00002000 b3:38 89086      /data/data/paulscode.android.mupen64plus/lib/libsdl_main.so
81d00000-81d06000 r-xp 00000000 b3:38 89490      /data/data/paulscode.android.mupen64plus/lib/libaudio-sdl.so
81d06000-81d07000 rwxp 00005000 b3:38 89490      /data/data/paulscode.android.mupen64plus/lib/libaudio-sdl.so
81e00000-81e0b000 r-xp 00000000 b3:38 89077      /data/data/paulscode.android.mupen64plus/lib/librsp-hle.so
81e0b000-81e0c000 rwxp 0000a000 b3:38 89077      /data/data/paulscode.android.mupen64plus/lib/librsp-hle.so
81e0c000-81e1e000 rwxp 00000000 00:00 0 
84000000-840a7000 r-xp 00000000 b3:38 89237      /data/data/paulscode.android.mupen64plus/lib/libcore.so
840a7000-840ad000 rwxp 000a6000 b3:38 89237      /data/data/paulscode.android.mupen64plus/lib/libcore.so
840ad000-86bb6000 rwxp 00000000 00:00 0 
a87de000-a87f0000 rwxp 00000000 00:00 0 
a9552000-a9553000 rwxp 00000000 00:00 0 
ab212000-ab215000 rwxp 00000000 00:00 0 
abb0a000-abb0c000 rwxp 00000000 00:00 0 
abe09000-abe0b000 rwxp 00000000 00:00 0 
acaa1000-acaa3000 rwxp 00000000 00:00 0 
ad383000-ad389000 rwxp 00000000 00:00 0 
ad9df000-ad9e0000 rwxp 00000000 00:00 0 
adeda000-adedc000 rwxp 00000000 00:00 0 
af0a2000-af0a4000 rwxp 00000000 00:00 0 
af90f000-af91e000 rwxp 00000000 00:00 0 
afd44000-afd4f000 rwxp 00000000 00:00 0 
b000d000-b0016000 rwxp 00000000 00:00 0 
befc5000-befd1000 rwxp 00000000 00:00 0          [stack]

paulscode · Dec 13, 2010

Sigh, thanks to that compiler bug I don't think I'll be able to use gdb from within Eclipse and the Sequoya plugin. Guess I'll have to build and debug from the command line...

Ari64 · Dec 13, 2010

paulscode said:

Code:

84000000-840a7000 r-xp 00000000 b3:38 89237      /data/data/paulscode.android.mupen64plus/lib/libcore.so
840a7000-840ad000 rwxp 000a6000 b3:38 89237      /data/data/paulscode.android.mupen64plus/lib/libcore.so
840ad000-86bb6000 rwxp 00000000 00:00 0

Since that's ~43MB, I assume you added the 16MB. Now use objdump -T --section=.bss to find out the relative offset of where that 16MB ended up.

Also I don't see an 8MB allocation for rdram[] so you might need to do something about that.

paulscode · Dec 13, 2010

Ari64 said:
Since that's ~43MB, I assume you added the 16MB.

Yes, the updated sections of linkage_arm.s look like this:

Code:

	.bss
	.align	12
	.type	extra_memory, %object
	.size	extra_memory, 16777216	
extra_memory:
	.space	16777216+64+16+16+8+8+8+8+256+8+8+128+128+128+16+8+132+4+256+512+4194304
dynarec_local = extra_memory + 16777216
	.align	4	
	.type	dynarec_local, %object
	.size	dynarec_local, 64

Code:

	.align	2
	.global	new_dyna_start
	.type	new_dyna_start, %function
new_dyna_start:
	ldr	r12, .dlptr
	mov	r0, #0xa4000000
	stmia	r12, {r4, r5, r6, r7, r8, r9, sl, fp, lr}
	sub	fp, r12, #28
	add	r0, r0, #0x40
	bl	new_recompile_block
	ldr	r0, [fp, #next_interupt-dynarec_local]
	ldr	r10, [fp, #reg_cop0+36-dynarec_local] /* Count */
	str	r0, [fp, #last_count-dynarec_local]
	sub	r10, r10, r0
	ldr pc, .pointer /*mov	pc, #0x7000000*/
.dlptr:
	.word	dynarec_local+28
.pointer:
	.word	extra_memory
	.size	new_dyna_start, .-new_dyna_start

Ari64 said:
Now use objdump -T --section=.bss to find out the relative offset of where that 16MB ended up.

The line for extra_memory (if the first number is the offset, then it is about 4MB in):

Code:

003bb000 g    DO .bss	01000000 extra_memory

The entire output (I am still unclear which of these need to be within 32MB of each other):

Code:

026bf450 g    DO .bss	0000a000 insn
013bb188 g    DO .bss	00000008 lo
017bc000 g    DO .bss	00000001 isGoldeneyeRom
000ad500 g    DO .bss	00000004 rom_size
013bb320 g    DO .bss	00000004 branch_target
000ad49c g    DO .bss	00000004 g_CoreConfig
003baac8 g    DO .bss	00000004 no_compiled_jump
013bb328 g    DO .bss	00000084 fake_pc
026bf420 g    DO .bss	00000004 next_vi
017bc960 g    DO .bss	00040000 readmemd
013bb060 g    DO .bss	00000008 readmem_dword
013bb324 g    DO .bss	00000004 PC
017fc960 g    DO .bss	00040000 readmemh
026bf42c g    DO .bss	00000004 fast_memory
021bec68 g    DO .bss	00000004 jump_target
021bec6c g    DO .bss	00000004 local_rt32
026c9450 g    DO .bss	00004000 jump_in
0183c960 g    DO .bss	00000034 sp_register
021bec70 g    DO .bss	00000008 local_rs
026cd450 g    DO .bss	00004000 ccadj
026d1450 g    DO .bss	00000004 is_delayslot
021bec78 g    DO .bss	00000004 actual
026d1454 g    DO .bss	00000004 linkcount
003bb000 g    DO .bss	01000000 extra_memory
026d1458 g    DO .bss	00004000 instr_addr
02bb5504 g    DO .bss	00000004 previousPC
021bec7c g    DO .bss	00000680 tlb_e
021bf2fc g    DO .bss	00000004 rompause
026d5458 g    DO .bss	00000004 start
0183c994 g    DO .bss	00040000 writememb
026d545c g    DO .bss	00001000 rs2
013bb190 g    DO .bss	00000080 reg_cop0
026d645c g    DO .bss	00001000 rt2
0187c994 g    DO .bss	00000040 PIF_RAM
026d745c g    DO .bss	00001000 bt
026d845c g    DO .bss	00000004 using_tlb
003baaec g    DO .bss	00000004 vi_counter
003bab10 g    DO .bss	00000004 g_NumBreakpoints
017bc004 g    DO .bss	00000004 ROM_HEADER
026d8460 g    DO .bss	00008000 branch_unneeded_reg_upper
026e0460 g    DO .bss	00008000 requires_32bit
000ad4fc g    DO .bss	00000004 rom
026e8460 g    DO .bss	00008000 p32
013bb180 g    DO .bss	00000008 hi
026f0460 g    DO .bss	00001000 likely
026f1460 g    DO .bss	00001000 itype
026f2460 g    DO .bss	00004000 jump_out
026f6460 g    DO .bss	00001000 us2
013bb044 g    DO .bss	00000004 cycle_count
026f7460 g    DO .bss	00001000 opcode2
017bc008 g    DO .bss	00000808 g_romdatabase
026bf430 g    DO .bss	00000004 inst_pointer
013bb074 g    DO .bss	00000002 hword
013bb3b0 g    DO .bss	00000100 mini_ht
021bec38 g    DO .bss	00000030 Controls
013bb05c g    DO .bss	00000004 address
026f8460 g    DO .bss	00001000 lt1
021bf300 g    DO .bss	00000004 llbit
013bb040 g    DO .bss	00000004 next_interupt
021bf304 g    DO .bss	00000004 CIC_Chip
026f9460 g    DO .bss	00060000 stubs
0187c9d4 g    DO .bss	0000003c vi_register
021bf308 g    DO .bss	00000100 reg_cop1_fgr_64
021bf408 g    DO .bss	00000004 local_rs32
0187ca10 g    DO .bss	00000028 ai_register
02759460 g    DO .bss	00004000 ba
0187ca38 g    DO .bss	00000004 rdword
02bb5514 g    DO .bss	00000600 g_Breakpoints
026bf434 g    DO .bss	00000004 return_address
0275d460 g    DO .bss	00008000 branch_unneeded_reg
000ad4ac g    DO .bss	00000004 g_TakeScreenshot
0187ca3c g    DO .bss	00000030 dpc_register
02765460 g    DO .bss	00000004 literalcount
003bab08 g    DO .bss	00000004 tracedebug
02765464 g    DO .bss	00004000 wont_dirty
013bb310 g    DO .bss	00000010 rounding_modes
02769468 g    DO .bss	00068000 constmap
027d1468 g    DO .bss	00008000 unneeded_reg
027d9468 g    DO .bss	00004000 needed_reg
021bf40c g    DO .bss	00400000 blocks
013bb000 g    DO .bss	00000040 dynarec_local
027dd468 g    DO .bss	00001000 us1
013bb050 g    DO .bss	00000004 pcaddr
026bf424 g    DO .bss	00000004 interp_addr
0187ca6c g    DO .bss	00040000 readmemb
027de468 g    DO .bss	00001000 is_ds
027df468 g    DO .bss	00000004 stop_after_jal
018bca6c g    DO .bss	00000028 rdram_register
026bf438 g    DO .bss	00000004 max_code_length
018bca94 g    DO .bss	00000010 si_register
02bb5508 g    DO .bss	00000004 debugger_done_cond
026bf43c g    DO .bss	00000004 dst_block
018bcaa4 g    DO .bss	00040000 writememd
013bb058 g    DO .bss	00000004 invc_ptr
027df46c g    DO .bss	00001000 dep1
013bb6b0 g    DO .bss	00400000 memory_map
003bab0c g    DO .bss	00000004 g_DebuggerActive
003baae8 g    DO .bss	00000004 vi_field
026bf440 g    DO .bss	00000004 code_length
013bb076 g    DO .bss	00000001 byte
000ad4a4 g    DO .bss	00000004 g_MemHasBeenBSwapped
018fcaa4 g    DO .bss	00040000 writememh
026bf444 g    DO .bss	00000004 src
027e046c g    DO .bss	00000004 pagelimit
013bb078 g    DO .bss	00000004 FCR0
025bf40c g    DO .bss	00000004 jump_to_address
02bb550c g    DO .bss	00000004 run
027e0470 g    DO .bss	0000d000 regmap
027ed470 g    DO .bss	0000d000 regmap_entry
02bb5510 g    DO .bss	00000004 mutex
0193caa4 g    DO .bss	00000034 pi_register
013bb068 g    DO .bss	00000008 dword
013bb210 g    DO .bss	00000080 reg_cop1_simple
003bab04 g    DO .bss	00000004 jump_marker
013bb4b0 g    DO .bss	00000200 restore_candidate
027fa470 g    DO .bss	00008000 unneeded_reg_upper
02802470 g    DO .bss	00000004 slen
025bf410 g    DO .bss	00000004 last_addr
000ad4a8 g    DO .bss	00000004 g_EmulatorRunning
003baac4 g    DO .bss	00000004 r4300emu
02802474 g    DO .bss	00004000 will_dirty
003baad0 g    DO .bss	00000004 dyna_interp
0193cad8 g    DO .bss	00002000 SP_DMEM
017bc810 g    DO .bss	00000125 ROM_SETTINGS
02806474 g    DO .bss	00000004 known_reg
003baad8 g    DO .bss	00000008 debug_count
026bf428 g    DO .bss	00000004 op
02806478 g    DO .bss	00001000 opcode
0193ead8 g    DO .bss	00000080 outBuffer
02807478 g    DO .bss	000c0000 branch_regs
028c7478 g    DO .bss	00004000 jump_dirty
028cb478 g    DO .bss	00000068 known_value
013bb07c g    DO .bss	00000004 FCR31
003baacc g    DO .bss	00000004 skip_jump
019bec38 g    DO .bss	00400000 tlb_LUT_r
028cb4e0 g    DO .bss	000c0000 regs
0193eb58 g    DO .bss	00000020 ri_register
013bb290 g    DO .bss	00000080 reg_cop1_double
013bb054 g    DO .bss	00000004 stop
025bf414 g    DO .bss	00000004 delay_slot
0298b4e0 g    DO .bss	00100000 hash_table
026bf448 g    DO .bss	00000004 dst
01dbec38 g    DO .bss	00400000 tlb_LUT_w
0193eb78 g    DO .bss	00040000 readmem
02a8b4e0 g    DO .bss	00000004 cop1_usable
0197eb78 g    DO .bss	00040000 writemem
000ad90c g    DO .bss	00000004 savestates_job
019beb78 g    DO .bss	00000001 cpu_byte
02a8b4e8 g    DO .bss	00008000 pr32
025bf418 g    DO .bss	00000008 local_rt
02a934e8 g    DO .bss	00000004 copy
02a934ec g    DO .bss	00001000 dep2
02a944ec g    DO .bss	00000004 expirep
019beb7c g    DO .bss	00000024 MI_register
013bb04c g    DO .bss	00000004 pending_exception
02a944f0 g    DO .bss	0000c000 link_addr
02aa04f0 g    DO .bss	00004000 imm
02aa44f0 g    DO .bss	00001000 rs1
000ad4a0 g    DO .bss	00000004 g_FrameCallback
017bc938 g    DO .bss	00000024 empty_entry
02aa54f0 g    DO .bss	0000d000 regmap_pre
013bb070 g    DO .bss	00000004 word
019beba0 g    DO .bss	00000008 rsp_register
02ab24f0 g    DO .bss	00000004 stubcount
02ab24f4 g    DO .bss	00000004 source
013bb080 g    DO .bss	00000100 reg
019beba8 g    DO .bss	00000080 cmdBuffer
019bec28 g    DO .bss	00000010 dps_register
013bb048 g    DO .bss	00000004 last_count
025bf420 g    DO .bss	00100000 invalid_code
02ab24f8 g    DO .bss	00001000 rt1
02ab3500 g    DO .bss	00100000 shadow
017bc95c g    DO .bss	00000004 use_flashram
02bb3500 g    DO .bss	00002000 literals
02bb5500 g    DO .bss	00000004 out

Ari64 said:
Also I don't see an 8MB allocation for rdram[] so you might need to do something about that.

Ah, I see that.. This also seems to be assuming a static linkage, as it is just pointing to "0x80000000" rather than to an allocated block of memory. The relevant lines:

Code:

	.global	rdram
rdram = 0x80000000

Code:

// This is defined in linkage_arm.s, but gcc -O3 likes this better
#define rdram ((unsigned int *)0x80000000)

Do I need to allocate 8 additional MB of memory for this in .bss? I assume it must also be within 32MB of extra_memory, etc.

Ari64 · Dec 13, 2010

paulscode said:
The line for extra_memory (if the first number is the offset, then it is about 4MB in):

Code:

003bb000 g DO .bss 01000000 extra_memory

Yep. Looks fine.

paulscode said:
Ah, I see that.. This also seems to be assuming a static linkage, as it is just pointing to "0x80000000" rather than to an allocated block of memory. The relevant lines:

Code:

.global rdram rdram = 0x80000000

Code:

// This is defined in linkage_arm.s, but gcc -O3 likes this better #define rdram ((unsigned int *)0x80000000)

Do I need to allocate 8 additional MB of memory for this in .bss? I assume it must also be within 32MB of extra_memory, etc.

The mmap is in memory/memory.c. If this mapping isn't possible, then you'll need to put it in .bss. It does not need to be within 32MB, but if you are not able to map the ram at its native location of 0x80000000, then all memory references will need to be accessed via a pointer. To do this, set using_tlb=1 in new_dynarec_init(). This will slow down the emulator by around 5%.

See http://pandorawiki.org/Mupen64plus_dynamic_recompiler#Translation_lookaside_buffer_emulation for details.

paulscode · Dec 14, 2010

Cool, I'll take a look at that. Thanks again for all the help!

paulscode · Dec 14, 2010

Some improvement now (the ap crashes later now that rdram is actually allocated..) I still need a sanity check to see if my changes make sense, though.

First, I added 8MB more to .bss:

Code:

	.bss
	.align	12
	.type	extra_memory, %object
	.size	extra_memory, 16777216
extra_memory:
	.space	16777216+8388608+64+16+16+8+8+8+8+256+8+8+128+128+128+16+8+132+4+256+512+4194304
rdram_memory = extra_memory + 16777216
	.align	12
	.type	rdram_memory, %object
	.size	rdram_memory, 8388608
dynarec_local = rdram_memory + 8388608
	.align	4
	.type	dynarec_local, %object
	.size	dynarec_local, 64

Then I pointed rdram to this area in assem_arm.h:

Code:

// This is defined in linkage_arm.s, but gcc -O3 likes this better
//#define rdram ((unsigned int *)0x80000000)
extern char rdram_memory[8388608];
#define rdram ((unsigned int *)rdram_memory)

Next, I mmapped it in new_dynarec_init():

Code:

  u_char *tmp=(u_char*)rdram;
  if (mmap (tmp, 8388608,
            PROT_READ | PROT_WRITE | PROT_EXEC,
            MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS,
            -1, 0) <= 0) {__android_log_print( ANDROID_LOG_ERROR, "new_dynarec", "Error: mmap() of rdram failed!" );}

And munmapped it in new_dynarec_cleanup():

Code:

  if (munmap ((void *)rdram, 8388608) < 0) {__android_log_print( ANDROID_LOG_ERROR, "new_dynarec", "Error: munmap() of rdram failed!" );}

And finally, I set using_tlb to 1 in new_dynarec_init():

Code:

  // TLB
  //using_tlb=0;
  using_tlb=1;

Memory info after the changes:

Code:

003ba000 g    DO .bss	01000000 extra_memory
013ba000 g    DO .bss	00800000 rdram_memory

Code:

81b00000-81b09000 r-xp 00000000 b3:38 89330      /data/data/paulscode.android.mupen64plus/lib/libapplication.so
81b09000-81b0e000 rwxp 00008000 b3:38 89330      /data/data/paulscode.android.mupen64plus/lib/libapplication.so
81b0e000-81b0f000 rwxp 00000000 00:00 0 
81c00000-81c03000 r-xp 00000000 b3:38 89077      /data/data/paulscode.android.mupen64plus/lib/libsdl_main.so
81c03000-81c04000 rwxp 00002000 b3:38 89077      /data/data/paulscode.android.mupen64plus/lib/libsdl_main.so
81d00000-81d06000 r-xp 00000000 b3:38 89083      /data/data/paulscode.android.mupen64plus/lib/libaudio-sdl.so
81d06000-81d07000 rwxp 00005000 b3:38 89083      /data/data/paulscode.android.mupen64plus/lib/libaudio-sdl.so
81e00000-81e0b000 r-xp 00000000 b3:38 89334      /data/data/paulscode.android.mupen64plus/lib/librsp-hle.so
81e0b000-81e0c000 rwxp 0000a000 b3:38 89334      /data/data/paulscode.android.mupen64plus/lib/librsp-hle.so
81e0c000-81e1e000 rwxp 00000000 00:00 0 
84000000-840a6000 r-xp 00000000 b3:38 89086      /data/data/paulscode.android.mupen64plus/lib/libcore.so
840a6000-840ac000 rwxp 000a5000 b3:38 89086      /data/data/paulscode.android.mupen64plus/lib/libcore.so
840ac000-873b5000 rwxp 00000000 00:00 0

Code:

Idx Name          Size      VMA       LMA       File off  Algn
  0 .hash         00004218  000000d4  000000d4  000000d4  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  1 .dynsym       000087f0  000042ec  000042ec  000042ec  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  2 .dynstr       000068b0  0000cadc  0000cadc  0000cadc  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .rel.dyn      00004e40  0001338c  0001338c  0001338c  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .rel.plt      00000368  000181cc  000181cc  000181cc  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  5 .plt          00000530  00018534  00018534  00018534  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  6 .text         00082014  00018a64  00018a64  00018a64  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  7 .rodata       00006a20  0009aa78  0009aa78  0009aa78  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  8 .ARM.extab    000000ac  000a1498  000a1498  000a1498  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  9 .ARM.exidx    00004180  000a1544  000a1544  000a1544  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 10 .data.rel.ro  00001670  000a66c4  000a66c4  000a56c4  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 11 .dynamic      000000f8  000a7d34  000a7d34  000a6d34  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 12 .got          00001050  000a7e2c  000a7e2c  000a6e2c  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 13 .data         00002460  000a8e80  000a8e80  000a7e80  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 14 .bss          03308b14  000ac000  000ac000  000aa2e0  2**12
                  ALLOC
 15 .comment      0000045c  00000000  00000000  000aa2e0  2**0
                  CONTENTS, READONLY
 16 .debug_aranges 00004938  00000000  00000000  000aa740  2**3
                  CONTENTS, READONLY, DEBUGGING
 17 .debug_pubnames 00007ff6  00000000  00000000  000af078  2**0
                  CONTENTS, READONLY, DEBUGGING
 18 .debug_info   00030b28  00000000  00000000  000b706e  2**0
                  CONTENTS, READONLY, DEBUGGING
 19 .debug_abbrev 00006e81  00000000  00000000  000e7b96  2**0
                  CONTENTS, READONLY, DEBUGGING
 20 .debug_line   00021478  00000000  00000000  000eea17  2**0
                  CONTENTS, READONLY, DEBUGGING
 21 .debug_frame  0000f0bc  00000000  00000000  0010fe90  2**2
                  CONTENTS, READONLY, DEBUGGING
 22 .debug_str    0000c8af  00000000  00000000  0011ef4c  2**0
                  CONTENTS, READONLY, DEBUGGING
 23 .debug_loc    00026a6f  00000000  00000000  0012b7fb  2**0
                  CONTENTS, READONLY, DEBUGGING
 24 .debug_ranges 00009700  00000000  00000000  0015226a  2**0
                  CONTENTS, READONLY, DEBUGGING
 25 .ARM.attributes 00000029  00000000  00000000  0015b96a  2**0
                  CONTENTS, READONLY

If everything seems to be in order, my next step is to get gdb working so I can start debugging and comparing pointers to try and find whats causing the ap to crash.

Porting Mupen64Plus to Android

New member

New member

New member

New member

New member

New member

New member

New member

New member

New member

New member

New member

New member

New member

New member

New member

New member

New member

New member

New member