What's new

Porting Mupen64Plus to Android

OP
paulscode

paulscode

New member
I'm thinking my somewhat uneducated attempt to translate the information from the original makefiles into NDK's special Android.mk files may be related. The console output when building linkage_arm.s looks like this:
Code:
Compile thumb  : core <= /home/paul/workspace/mupen64plus/project/jni/../jni/core/src/r4300/new_dynarec/linkage_arm.s
arm-eabi-gcc: unrecognized option '-XLinker'
/home/paul/bin/android-ndk-r4b/build/prebuilt/linux-x86/arm-eabi-4.4.0/bin/arm-eabi-gcc
-I/home/paul/bin/android-ndk-r4b/build/platforms/android-8/arch-arm/usr/include -fpic
-mthumb-interwork -ffunction-sections -funwind-tables -fstack-protector -fno-short-enums 
-D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__  
-Wno-psabi -march=armv5te -mtune=xscale -msoft-float -mthumb -Os -fomit-frame-pointer 
-fno-strict-aliasing -finline-limit=64  -I/home/paul/workspace/mupen64plus/project/jni/../jni/core 
-DANDROID  -I/home/paul/workspace/mupen64plus/project/jni/../jni/core/../stlport/stlport  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/src  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/src/debugger  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/src/memory  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/src/main  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/src/main/zip  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/src/r4300  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/src/r4300/x86  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/src/r4300/new_dynarec  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/src/api  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/src/plugin  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/src/osal 
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/../sdl-1.2/include  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/../sdl_mixer/include  
-I/home/paul/workspace/mupen64plus/project/jni/../jni/core/../sdl_image/include 
-DNEW_DYNAREC -DDYNAREC -D__arm__ -O3 -Wa,--noexecstack -O2 -DNDEBUG -g -c -MMD -MP -MF 
/home/paul/workspace/mupen64plus/project/obj/local/armeabi/objs/core/src/r4300/new_dynarec/linkage_arm.s.d 
/home/paul/workspace/mupen64plus/project/jni/../jni/core/src/r4300/new_dynarec/linkage_arm.s 
-o /home/paul/workspace/mupen64plus/project/obj/local/armeabi/objs/core/src/r4300/new_dynarec/linkage_arm.s
arm-eabi-gcc: unrecognized option '-XLinker'
SharedLibrary  : libcore.so
/home/paul/bin/android-ndk-r4b/build/prebuilt/linux-x86/arm-eabi-4.4.0/bin/arm-eabi-gcc 
-nostdlib -Wl,-soname,libcore.so -Wl,-shared,-Bsymbolic  
/home/paul/workspace/mupen64plus/project/obj/local/armeabi/objs/core/src/api/callbacks.o 

...EDITED...
/home/paul/workspace/mupen64plus/project/obj/local/armeabi/libsdl_image.so 
/home/paul/bin/android-ndk-r4b/build/platforms/android-8/arch-arm/usr/lib/libc.so 
/home/paul/bin/android-ndk-r4b/build/platforms/android-8/arch-arm/usr/lib/libstdc++.so 
/home/paul/bin/android-ndk-r4b/build/platforms/android-8/arch-arm/usr/lib/libm.so  
-Lobj/local/armeabi  -XLinker --section-start -Xlinker .init=0x08000000 --arm_linux --gnu  
-Lobj/local/armeabi  -XLinker --section-start -Xlinker .init=0x08000000 --arm_linux --gnu  -Wl,
--no-undefined -Wl,-z,noexecstack 
-L/home/paul/bin/android-ndk-r4b/build/platforms/android-8/arch-arm/usr/lib -lGLESv1_CM -ldl -llog 
-lz -Wl,-rpath-link=/home/paul/bin/android-ndk-r4b/build/platforms/android-8/arch-arm/usr/lib -o 
/home/paul/workspace/mupen64plus/project/obj/local/armeabi/libcore.so
/home/paul/workspace/mupen64plus/project/obj/local/armeabi/objs/core/src/r4300/new_dynarec/linkage_arm.s: 
Assembler messages:
...
In particular, the interesting message here is "arm-eabi-gcc: unrecognized option '-XLinker'". The LDFLAGS options from my Android.mk file are:
Code:
LOCAL_LDLIBS := -lGLESv1_CM -ldl -llog -lz

LOCAL_LDFLAGS := -Lobj/local/armeabi

LOCAL_LDFLAGS += $(APPLICATION_ADDITIONAL_LDFLAGS)

LOCAL_LDFLAGS += -XLinker --section-start -Xlinker .init=0x08000000
LOCAL_LDFLAGS += --arm_linux --gnu
I don't actually know what some of these are for. The XLinker stuff was from my attempt to translate information from Ari64's makefile:
Code:
  ifeq ($(CPU), ARM)
    ifeq ($(ARCH), 32BITS)
      LDFLAGS += -Xlinker --section-start -Xlinker .init=0x08000000
    endif
  endif
The --arm_linus --gnu option was a suggestion from that webpage Richard42 posted (but it didn't help, unfortunately). The rest of the options were in the file autogenerated by Pelya's SDL port. Perhaps someone with more NDK experience like Chaoz can see a problem here (let me know if you need more information).

For reference, here the entire Android.mk file I'm using for the core library (its pretty messy at the moment):
Code:
LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)

LOCAL_MODULE := core
APPDIR := $(shell readlink $(LOCAL_PATH)/src)
SRCDIR := $(shell readlink $(LOCAL_PATH)/src)src

APP_SUBDIRS := $(patsubst $(LOCAL_PATH)/%, %, $(shell find $(LOCAL_PATH)/$(APPDIR) -path '*/.svn' -prune -o -type d -print))
ifneq ($(APPLICATION_SUBDIRS_BUILD),)
APPLICATION_SUBDIRS_BUILD_NONRECURSIVE := $(addprefix $(APPDIR)/, $(filter-out %/*, $(APPLICATION_SUBDIRS_BUILD)))
APPLICATION_SUBDIRS_BUILD_RECURSIVE := $(patsubst %/*, %, $(filter %/*,$(APPLICATION_SUBDIRS_BUILD)))
APPLICATION_SUBDIRS_BUILD_RECURSIVE := $(foreach FINDDIR, $(APPLICATION_SUBDIRS_BUILD_RECURSIVE), ...EDITED...
     $(shell find $(LOCAL_PATH)/$(APPDIR)/$(FINDDIR) -path '*/.svn' -prune -o -type d -print))
APPLICATION_SUBDIRS_BUILD_RECURSIVE := $(patsubst $(LOCAL_PATH)/%, %, $(APPLICATION_SUBDIRS_BUILD_RECURSIVE) )
APP_SUBDIRS := $(APPLICATION_SUBDIRS_BUILD_NONRECURSIVE) $(APPLICATION_SUBDIRS_BUILD_RECURSIVE)
endif

LOCAL_CFLAGS :=

ifeq ($(CRYSTAX_TOOLCHAIN),)
# Paths should be on newline so launchConfigure.sh will work properly
LOCAL_CFLAGS += \
				-I$(LOCAL_PATH)/../stlport/stlport
endif

# Paths should be on newline so launchConfigure.sh will work properly
LOCAL_CFLAGS += \
				$(foreach D, $(APP_SUBDIRS), -I$(LOCAL_PATH)/$(D)) \
				-I$(LOCAL_PATH)/../sdl-$(SDL_VERSION)/include \
				$(foreach L, $(COMPILED_LIBRARIES), -I$(LOCAL_PATH)/../$(L)/include)

#LOCAL_CFLAGS += -DNO_ASM
LOCAL_CFLAGS += -DNEW_DYNAREC
LOCAL_CFLAGS += -DDYNAREC
LOCAL_CFLAGS += -D__arm__

LOCAL_CFLAGS += $(APPLICATION_ADDITIONAL_CFLAGS)

#Change C++ file extension as appropriate
LOCAL_CPP_EXTENSION := .cpp

#LOCAL_SRC_FILES := $(foreach F, $(APP_SUBDIRS), $(addprefix $(F)/,$(notdir $(wildcard $(LOCAL_PATH)/$(F)/*.cpp))))
#LOCAL_SRC_FILES += $(foreach F, $(APP_SUBDIRS), $(addprefix $(F)/,$(notdir $(wildcard $(LOCAL_PATH)/$(F)/*.c))))
LOCAL_SRC_FILES := \
	$(SRCDIR)/api/callbacks.c \
	$(SRCDIR)/api/common.c \
	$(SRCDIR)/api/config.c \
	$(SRCDIR)/api/debugger.c \
	$(SRCDIR)/api/frontend.c \
	$(SRCDIR)/api/vidext.c \
	$(SRCDIR)/main/main.c \
	$(SRCDIR)/main/util.c \
	$(SRCDIR)/main/cheat.c \
	$(SRCDIR)/main/eventloop.c \
	$(SRCDIR)/main/adler32.c \
	$(SRCDIR)/main/md5.c \
	$(SRCDIR)/main/rom.c \
	$(SRCDIR)/main/ini_reader.c \
	$(SRCDIR)/main/savestates.c \
	$(SRCDIR)/main/zip/ioapi.c \
	$(SRCDIR)/main/zip/zip.c \
	$(SRCDIR)/main/zip/unzip.c \
	$(SRCDIR)/memory/dma.c \
	$(SRCDIR)/memory/flashram.c \
	$(SRCDIR)/memory/memory.c \
	$(SRCDIR)/memory/pif.c \
	$(SRCDIR)/memory/tlb.c \
	$(SRCDIR)/osal/dynamiclib_unix.c \
	$(SRCDIR)/osal/files_unix.c \
	$(SRCDIR)/plugin/plugin.c \
	$(SRCDIR)/plugin/dummy_video.c \
	$(SRCDIR)/plugin/dummy_audio.c \
	$(SRCDIR)/plugin/dummy_input.c \
	$(SRCDIR)/plugin/dummy_rsp.c \
	$(SRCDIR)/r4300/r4300.c \
	$(SRCDIR)/r4300/bc.c \
	$(SRCDIR)/r4300/cop0.c \
	$(SRCDIR)/r4300/cop1.c \
	$(SRCDIR)/r4300/cop1_d.c \
	$(SRCDIR)/r4300/cop1_l.c \
	$(SRCDIR)/r4300/cop1_s.c \
	$(SRCDIR)/r4300/cop1_w.c \
	$(SRCDIR)/r4300/exception.c \
	$(SRCDIR)/r4300/interupt.c \
	$(SRCDIR)/r4300/profile.c \
	$(SRCDIR)/r4300/pure_interp.c \
	$(SRCDIR)/r4300/recomp.c \
	$(SRCDIR)/r4300/special.c \
	$(SRCDIR)/r4300/regimm.c \
	$(SRCDIR)/r4300/tlb.c \
	$(SRCDIR)/r4300/empty_dynarec.c \
	$(SRCDIR)/r4300/new_dynarec/new_dynarec.c \
	$(SRCDIR)/r4300/new_dynarec/fpu.c \
	$(SRCDIR)/r4300/new_dynarec/linkage_arm.s \
	$(SRCDIR)/r4300/new_dynarec/assem_arm.c \
	$(SRCDIR)/debugger/debugger.c \
	$(SRCDIR)/debugger/dbg_decoder.c \
	$(SRCDIR)/debugger/dbg_memory.c \
	$(SRCDIR)/debugger/dbg_breakpoints.c
	
	
	

ifneq ($(APPLICATION_CUSTOM_BUILD_SCRIPT),)
LOCAL_SRC_FILES := dummy.c
endif

LOCAL_SHARED_LIBRARIES := sdl-$(SDL_VERSION) $(filter-out $(APP_AVAILABLE_STATIC_LIBS), $(COMPILED_LIBRARIES))

LOCAL_STATIC_LIBRARIES := stlport $(filter $(APP_AVAILABLE_STATIC_LIBS), $(COMPILED_LIBRARIES))

LOCAL_LDLIBS := -lGLESv1_CM -ldl -llog -lz

LOCAL_LDFLAGS := -Lobj/local/armeabi

LOCAL_LDFLAGS += $(APPLICATION_ADDITIONAL_LDFLAGS)

LOCAL_LDFLAGS += -XLinker --section-start -Xlinker .init=0x08000000
LOCAL_LDFLAGS += --arm_linux --gnu 

LIBS_WITH_LONG_SYMBOLS := $(strip $(shell \
	for f in $(LOCAL_PATH)/../../libs/armeabi/*.so ; do \
		if echo $$f | grep "libcore[.]so" > /dev/null ; then \
			continue ; \
		fi ; \
		if [ -e "$$f" ] ; then \
			if nm -g $$f | cut -c 12- | egrep '.{128}' > /dev/null ; then \
				echo $$f | grep -o 'lib[^/]*[.]so' ; \
			fi ; \
		fi ; \
	done \
) )

ifneq "$(LIBS_WITH_LONG_SYMBOLS)" ""
$(foreach F, $(LIBS_WITH_LONG_SYMBOLS), \
$(info Library $(F): abusing symbol names are: \
$(shell nm -g $(LOCAL_PATH)/../../libs/armeabi/$(F) | cut -c 12- | egrep '.{128}' ) ) \
$(info Library $(F) contains symbol names longer than 128 bytes, \
YOUR CODE WILL DEADLOCK WITHOUT ANY WARNING when you'll access such function - \
please make this library static to avoid problems. ) )
$(error Detected libraries with too long symbol names. Remove all files under project/libs/armeabi, make these libs static,  ...EDITED...
    and recompile)
endif

APP_LIB_DEPENDS := $(foreach LIB, $(LOCAL_SHARED_LIBRARIES), $(abspath $(LOCAL_PATH)/../../obj/local/armeabi/lib$(LIB).so)) 
APP_LIB_DEPENDS += $(foreach LIB, $(LOCAL_STATIC_LIBRARIES), $(abspath $(LOCAL_PATH)/../../obj/local/armeabi/lib$(LIB).a))

include $(BUILD_SHARED_LIBRARY)

ifneq ($(APPLICATION_CUSTOM_BUILD_SCRIPT),)

LOCAL_PATH_SDL_APPLICATION := $(LOCAL_PATH)

.NOTPARALLEL: $(realpath $(LOCAL_PATH)/../../obj/local/armeabi/libcore.so) $(LOCAL_PATH)/src/libcore.so

$(shell rm $(LOCAL_PATH)/src/libcore.so) # Enforce rebuilding
# $(shell rm $(LOCAL_PATH)/../../obj/local/armeabi/*.so) # libcore.so may try to link with wrong libraries, prevent that

$(LOCAL_PATH)/src/libcore.so: $(LOCAL_PATH)/src/AndroidBuild.sh $(LOCAL_PATH)/src/AndroidAppSettings.cfg $(APP_LIB_DEPENDS)
	echo Launching script $(LOCAL_PATH_SDL_APPLICATION)/AndroidBuild.sh
	cd $(LOCAL_PATH_SDL_APPLICATION)/src && ./AndroidBuild.sh

$(realpath $(LOCAL_PATH)/../../obj/local/armeabi/libcore.so): $(LOCAL_PATH)/src/libcore.so OVERRIDE_CUSTOM_LIB
	cp -f $< $@

.PHONY: OVERRIDE_CUSTOM_LIB

OVERRIDE_CUSTOM_LIB:

endif
 
Last edited:

Ari64

New member
Your problem is that gcc options are case-sensitive and you capitalized it incorrectly.

As for what this does... Unlike x86, branch instructions on ARM are limited to a 32MB range. All your code has to fit within 32MB, otherwise you have to call things through pointers or use other nasty hacks. These linker options rearrange the memory map of the process so that the (static) text segment, and the buffer for the dynamically generated code both fit within 32MB.

Since you're building a shared library instead of a single binary, you're probably going to have to come up with a different memory layout than what I used. One possibility would be to allocate 16MB in the .bss section, define BASE_ADDR to be this location, and hope everything fits.
 
OP
paulscode

paulscode

New member
Your problem is that gcc options are case-sensitive and you capitalized it incorrectly.
D'oh! Careless me..

I'm hoping there's a solution that won't require compiling the thing separately, but I probably won't be so lucky.
 
Last edited:
OP
paulscode

paulscode

New member
Ari64, I've decided to give your suggestion a try (assuming I figure out how to get the s file to compile into a library that can be utilized by the c code). However, I am unclear on the terminology, and have a few questions. Perhaps you can help me out a bit (and sorry for being a total newbie when it comes to assembly, I really do need to study up on the subject).

Firstly, what do I add to the .bss section to allocate 16MB? I see where the section starts in the code:
Code:
	.bss
	.align	4
	.type	dynarec_local, %object
	.size	dynarec_local, 64
Also, I assume that the first 16MB is being allocated somewhere now (is this done entirely by those linker options, or would something else need to be edited to not allocate this memory?)

Secondly, I see that BASE_ADDR is defined in assem_arm.h (and referenced numerous times in new_dynarec.c), but how does one use this to define the 16MB of space that will be allocated in the .bss section? Is the change needed in only this one location, or would there be other required changes related to this as well?

Thirdly, how would the linker options/LDFLAGS need to change in a setup like this? Would there be anything specific required when building the library initially? Anything specific in the subsequent project that links it with the c code?

Fourthly, what is the significance of the number 0x08000000? I see it referenced in assem_arm.c (unrelated, perhaps), and of course it is also referenced in the linker options. I understand why the data needs to be arranged as you described, but could you elaborate on how "-Xlinker --section-start -Xlinker .init=0x08000000" accomplishes this? Makefiles and linker options are another area I need to study up on as well.

And finally, what did you mean by "hoping everything fits"? Would this method further limit the amount of memory lower than the 32MB that it has now?

I actually have more questions about the c code part of your new dynarec, but I'll get to those after I solve the assembly problem (they may get answered along the way).

Thanks again for all your help!
 
Last edited:
OP
paulscode

paulscode

New member
I've determined that the NDK can compile this file. I've only gotten it to work from the command line, but at least that means the separate shared library is an option (and won't cost me anything). I'm sure there has got to be a way for this to compile in the project as well (output from the builder shows that it is using the same executable I used from the command line). I really think I just need to get the compiler options right in my Android.mk file. I'll keep tinkering around with this.

For reference, here are the commands for building it into a shared library from the terminal:
Code:
/home/paul/bin/android-ndk-r4b/build/prebuilt/linux-x86/arm-eabi-4.4.0/bin/arm-eabi-gcc
-fpic -c linkage_arm.s -o linkage_arm.o

/home/paul/bin/android-ndk-r4b/build/prebuilt/linux-x86/arm-eabi-4.4.0/bin/arm-eabi-gcc
-shared -Wl,-soname,liblinkage_arm.so.1 -nostdlib -o liblinkage_arm.so linkage_arm.o
 
Last edited:

Ari64

New member
Basically the dynarec allocates 16MB of memory and generates code there. Due to the limitations of branch instructions in the ARM architecture this has to be less than 32MB away from the rest of the code.

The 16MB is currently allocated in new_dynarec_init:
Code:
  if (mmap (out, 1<<TARGET_SIZE_2,
            PROT_READ | PROT_WRITE | PROT_EXEC,
            MAP_FIXED | MAP_PRIVATE | MAP_ANONYMOUS,
            -1, 0) <= 0) {printf("mmap() failed\n");}
You could put 16MB of space before dynarec_local, as you suggest. The problem is that the bss segment is already quite large, so it might end up being over 32MB if you do this. Use objdump -h to see how big it actually is.

This also assumes that the Android kernel doesn't enforce w^x permissions on the bss segment.

The significance of 0x08000000 is that the code generation buffer is at 0x07000000, so this is within 32MB. This whole idea of putting the core in a shared library makes things more complicated since the shared library isn't guaranteed to be loaded at any specific address.

I suppose you could statically link everything and not build a shared library. That somewhat defeats the point of the new mupen64plus architecture though.
 
Last edited:

Richard42

Emulator Developer
I suppose you could statically link everything and not build a shared library. That somewhat defeats the point of the new mupen64plus architecture though.

This may be a desirable thing to do for mobile device ports of mupen64plus. You would still have the advantage of a documented and well-defined API; the only potential downside is that your application would have to be GPL-licensed.
 

Ari64

New member
There's a similar problem on x86-64 where the branch offset is limited to 32 bits, but the shared libraries can be loaded above 2GB. It seems that 1.99.4 is able to avoid this on Linux, but not with some other operating systems. A possible solution, for both ARM and x86-64, would be to set up trampolines to jump to locations that are further away.

The way I got my dynarec to work on x86-64 was to map the rom and rdram into the lower 2GB so these could be accessed with 32-bit pointers. gcc spits out hundreds of warnings about 32-bit pointer conversions when you compile this, but it works.
 

Richard42

Emulator Developer
Yeah this behavior of loading shared libraries into wacky high address spaces is most unfortunate. The way I got around this in my 64-bit dynarec is by keeping R15 as a pointer into the data segment of the core shared library, and making all accesses to the emulated rom/ram/CPU-state register-relative to R15. It should work on any OS. The only thing keeping a 64-bit build from working on Windows is the 32-bit 'long' type in VC and the AT&T style inline ASM.
 
OP
paulscode

paulscode

New member
Ari64, I have an unrelated question about the c code part of your new dynarec. When I plugged in the new_dynarec files as they were in your project, the compiler threw a couple hundred "undeclared, first use" errors for assem_arm.c. A bunch of these were for globals that were "#defined" in new_dynarec.c. Most of the rest were for variables that were also declared in new_dynarec.c. A couple were for definitions, variables, and functions from other parts of the core and standard c libraries. In order to get assem_arm.c to compile, I added a bunch of "#includes", "#defines", and "externs" at the top of the file. I'm guessing this is also related to how I set up my Android.mk file, and therefore its building things differently than in your project (otherwise I don't see how this file would have ever compiled).

For reference, the changes included:
Code:
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <sys/types.h>
#include "new_dynarec.h"
#include "../r4300.h"
#include "../recomp.h"
#include "../macros.h"
#include "../../memory/memory.h"
#define MAXBLOCK 4096
#define MAX_OUTPUT_BLOCK_SIZE 262144
#define CLOCK_DIVIDER 2
extern u_char byte;
extern u_int start;
extern u_int *source;
extern u_int pagelimit;
extern char insn[MAXBLOCK][10];
extern u_char itype[MAXBLOCK];
extern u_char opcode[MAXBLOCK];
extern u_char opcode2[MAXBLOCK];
extern u_char bt[MAXBLOCK];
extern u_char rs1[MAXBLOCK];
extern u_char rs2[MAXBLOCK];
extern u_char rt1[MAXBLOCK];
extern u_char rt2[MAXBLOCK];
extern u_char us1[MAXBLOCK];
extern u_char us2[MAXBLOCK];
extern u_char dep1[MAXBLOCK];
extern u_char dep2[MAXBLOCK];
extern u_char lt1[MAXBLOCK];
extern int imm[MAXBLOCK];
extern u_int ba[MAXBLOCK];
extern char likely[MAXBLOCK];
extern char is_ds[MAXBLOCK];
extern uint64_t unneeded_reg[MAXBLOCK];
extern uint64_t unneeded_reg_upper[MAXBLOCK];
extern uint64_t branch_unneeded_reg[MAXBLOCK];
extern uint64_t branch_unneeded_reg_upper[MAXBLOCK];
extern uint64_t p32[MAXBLOCK];
extern uint64_t pr32[MAXBLOCK];
extern signed char regmap_pre[MAXBLOCK][HOST_REGS];
extern signed char regmap[MAXBLOCK][HOST_REGS];
extern signed char regmap_entry[MAXBLOCK][HOST_REGS];
extern uint64_t constmap[MAXBLOCK][HOST_REGS];
extern uint64_t known_value[HOST_REGS];
extern u_int known_reg;
extern struct regstat regs[MAXBLOCK];
extern struct regstat branch_regs[MAXBLOCK];
extern u_int needed_reg[MAXBLOCK];
extern uint64_t requires_32bit[MAXBLOCK];
extern u_int wont_dirty[MAXBLOCK];
extern u_int will_dirty[MAXBLOCK];
extern int ccadj[MAXBLOCK];
extern int slen;
extern u_int instr_addr[MAXBLOCK];
extern u_int link_addr[MAXBLOCK][3];
extern int linkcount;
extern u_int stubs[MAXBLOCK*3][8];
extern int stubcount;
extern u_int literals[1024][2];
extern int literalcount;
extern int is_delayslot;
extern int cop1_usable;
extern u_char *out;
extern struct ll_entry *jump_in[4096];
extern struct ll_entry *jump_out[4096];
extern struct ll_entry *jump_dirty[4096];
extern u_int hash_table[65536][4]  __attribute__((aligned(16)));
extern char shadow[1048576]  __attribute__((aligned(16)));
extern void *copy;
extern int expirep;
extern u_int using_tlb;
extern u_int stop_after_jal;
extern u_char restore_candidate[512];
extern int cycle_count;
extern void div64(int64_t dividend,int64_t divisor);
extern void divu64(uint64_t dividend,uint64_t divisor);
extern void mult64(uint64_t m1,uint64_t m2);
extern void multu64(uint64_t m1,uint64_t m2);
extern void MFC0(void);
extern void MTC0(void);
extern void TLBP(void);
extern void TLBR(void);

/* registers that may be allocated */
/* 1-31 gpr */
#define HIREG 32 // hi
#define LOREG 33 // lo
#define FSREG 34 // FPU status (FCSR)
#define CSREG 35 // Coprocessor status
#define CCREG 36 // Cycle count
#define INVCP 37 // Pointer to invalid_code
#define TEMPREG 38
#define FTEMP 38 // FPU temporary register
#define PTEMP 39 // Prefetch temporary register
#define TLREG 40 // TLB mapping offset
#define RHASH 41 // Return address hash
#define RHTBL 42 // Return address hash table address
#define RTEMP 43 // JR/JALR address register
#define MAXREG 43
#define AGEN1 44 // Address generation temporary register
#define AGEN2 45 // Address generation temporary register
#define MGEN1 46 // Maptable address generation temporary register
#define MGEN2 47 // Maptable address generation temporary register
#define BTREG 48 // Branch target temporary register

/* instruction types */
#define NOP 0     // No operation
#define LOAD 1    // Load
#define STORE 2   // Store
#define LOADLR 3  // Unaligned load
#define STORELR 4 // Unaligned store
#define MOV 5     // Move
#define ALU 6     // Arithmetic/logic
#define MULTDIV 7 // Multiply/divide
#define SHIFT 8   // Shift by register
#define SHIFTIMM 9// Shift by immediate
#define IMM16 10  // 16-bit immediate
#define RJUMP 11  // Unconditional jump to register
#define UJUMP 12  // Unconditional jump
#define CJUMP 13  // Conditional branch (BEQ/BNE/BGTZ/BLEZ)
#define SJUMP 14  // Conditional branch (regimm format)
#define COP0 15   // Coprocessor 0
#define COP1 16   // Coprocessor 1
#define C1LS 17   // Coprocessor 1 load/store
#define FJUMP 18  // Conditional branch (floating point)
#define FLOAT 19  // Floating point unit
#define FCONV 20  // Convert integer to float
#define FCOMP 21  // Floating point compare (sets FSREG)
#define SYSCALL 22// SYSCALL
#define OTHER 23  // Other
#define SPAN 24   // Branch/delay slot spans 2 pages
#define NI 25     // Not implemented

/* stubs */
#define CC_STUB 1
#define FP_STUB 2
#define LOADB_STUB 3
#define LOADH_STUB 4
#define LOADW_STUB 5
#define LOADD_STUB 6
#define LOADBU_STUB 7
#define LOADHU_STUB 8
#define STOREB_STUB 9
#define STOREH_STUB 10
#define STOREW_STUB 11
#define STORED_STUB 12
#define STORELR_STUB 13
#define INVCODE_STUB 14

/* branch codes */
#define TAKEN 1
#define NOTTAKEN 2
#define NULLDS 3

I just want to verify that this isn't likely to break anything. I wouldn't think so, unless there was some particular reason for not having any includes or externs at the top of this file. It should just be telling the compiler that all these things exist somewhere outside the current c file I'll probably clean the code up at some point by moving everything into a header file (the #defines in particular so I won't have to potentially changed them in two places when you post future updates to your dynarec).
 
OP
paulscode

paulscode

New member
I've put the .s file into a project of its own to try and isolate what is causing the problem. There are actually a number of compiler options that it is sticking into there:
Code:
-fpic -mthumb-interwork -ffunction-sections -funwind-tables -fstack-protector -fno-short-enums -D__ARM_ARCH_5__
-D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__  -Wno-psabi -march=armv5te -mtune=xscale
-msoft-float -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64  -I/home/paul/workspace/HelloAndroid/jni
-DANDROID  -Wa,--noexecstack -O2 -DNDEBUG -g -c -MMD -MP -MF

In particular, the definitions of __ARM_ARCH_5*__ are interesting. I'll try using a few of these options to compile the file from the command line and see if any of them cause it to break.
 

Ari64

New member
Normally assem_arm.c is #included in new_dynarec.c and these are compiled together. You can split it up and compile them separately if you want, but you will have to duplicate all the declarations and function prototypes. As these two files have quite a lot of inter-dependencies, I found it much easier to compile it in one shot.

I've put the .s file into a project of its own to try and isolate what is causing the problem. There are actually a number of compiler options that it is sticking into there:

Most of these options are irrelevant when compiling assembly code. The code is ARMv5, so that's not a problem. I'm guessing your problem is -mthumb. This is not thumb code, nor does the recompiler generate thumb code.
 
OP
paulscode

paulscode

New member
Normally assem_arm.c is #included in new_dynarec.c and these are compiled together. You can split it up and compile them separately if you want, but you will have to duplicate all the declarations and function prototypes. As these two files have quite a lot of inter-dependencies, I found it much easier to compile it in one shot.

--EDIT-- Oh! I see you already have it that way in your code. I totally missed that when I was figuring out how it links together. All I have to do is remove assem_arm.c from the list of source files in Android.mk. Sorry for another pointless question!

I'm guessing your problem is -mthumb. This is not thumb code, nor does the recompiler generate thumb code.
Thanks, I'll play around in the settings menus when I get home this evening to see if there is a way to remove that option.
 
Last edited:
OP
paulscode

paulscode

New member
I haven't gotten it to work yet, but I feel like I am getting closer. In my stripped-down project that has only the .s file, I've been playing around with the settings in Android.mk. Currently, it looks like this:
Code:
LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)

LOCAL_MODULE	:= linkage_arm
LOCAL_ARM_MODE	:= arm
APP_ABI			:= armeabi-v7a
TARGET_CPU_ABI	:= armeabi-v7a

### Add all source file names to be included in lib separated by a whitespace
LOCAL_SRC_FILES := linkage_arm.s

TARGET_CFLAGS	:= -march=armv7-a -mtune=cortex-a8

LOCAL_LDFLAGS	:= -Xlinker --section-start -Xlinker .init=0x08000000

include $(BUILD_SHARED_LIBRARY)

The first build command looks much nicer with these settings
Code:
/home/paul/bin/android-ndk-r4b/build/prebuilt/linux-x86/arm-eabi-4.4.0/bin/arm-eabi-gcc
-march=armv7-a -mtune=cortex-a8 -O2 -fomit-frame-pointer -fstrict-aliasing -funswitch-loops -finline-limit=300
-I/home/paul/workspace/HelloAndroid/jni -DANDROID  -Wa,--noexecstack -O2 -DNDEBUG -g -c -MMD -MP -MF
/home/paul/workspace/HelloAndroid/obj/local/armeabi/objs/linkage_arm/linkage_arm.s.d
/home/paul/workspace/HelloAndroid/jni/linkage_arm.s
-o /home/paul/workspace/HelloAndroid/obj/local/armeabi/objs/linkage_arm/linkage_arm.s

However, the build still fails on the second command (with the usual "bad instruction" messages). The second build command comes out like this:
Code:
/home/paul/bin/android-ndk-r4b/build/prebuilt/linux-x86/arm-eabi-4.4.0/bin/arm-eabi-gcc
-nostdlib -Wl,-soname,liblinkage_arm.so -Wl,-shared,-Bsymbolic
 /home/paul/workspace/HelloAndroid/obj/local/armeabi/objs/linkage_arm/linkage_arm.s
-Wl,--whole-archive  -Wl,--no-whole-archive
/home/paul/bin/android-ndk-r4b/build/prebuilt/linux-x86/arm-eabi-4.4.0/bin/../lib/gcc/arm-eabi/4.4.0/libgcc.a
/home/paul/bin/android-ndk-r4b/build/platforms/android-8/arch-arm/usr/lib/libc.so
/home/paul/bin/android-ndk-r4b/build/platforms/android-8/arch-arm/usr/lib/libstdc++.so
/home/paul/bin/android-ndk-r4b/build/platforms/android-8/arch-arm/usr/lib/libm.so
-Xlinker --section-start -Xlinker .init=0x08000000 -Wl,--no-undefined -Wl,-z,noexecstack
-Wl,-rpath-link=/home/paul/bin/android-ndk-r4b/build/platforms/android-8/arch-arm/usr/lib
-o /home/paul/workspace/HelloAndroid/obj/local/armeabi/liblinkage_arm.so

I still don't know what the problem is, but at least I can go to the terminal and focus on that second command to try and figure out what is causing it to fail.
 
OP
paulscode

paulscode

New member
Wow, so I found the problem. As you can see in my previous post, for some reason the builder decides to give the output from the first command an extension of ".s" instead of ".o". This then confuses the builder when it executes the second command (apparently it sees that ".s" extension and expects a text file with assembly commands, which is why it spits out a hundred "bad instruction" errors).

What a stupid bug! Although it is so simple, I honestly have no idea how to get around it short of compiling from the command line -- I have no clue how to debug the Eclipse plug-in so it names the output (name).o instead of (name).s. I'll probably have to build the entire core from the command line, and just plug the resulting library into my project, rather than having it built inside the project along with the other libraries. The other way would be to compile linkage_arm.s separately like we've been discussing, but I'd really rather not tinker with Ari64's code..
 
OP
paulscode

paulscode

New member
I figured out an acceptable workaround. I've created two mirror projects, one with the new dynarec as part of the core, the other with the cached interpreter. I run the builder for the first project and wait for the errors. Then I select copy the last build command onto the clipboard, and pull up a terminal. I rename that output file to linkage_arm.o, paste the command, change the word linkage_arm.s to linkage_arm.o, and run it. This creates the good core library. Then I load up the second project and build it completely. I swap out the old core library with the good one, and can then deploy the .apk to the phone or emulator. This method is a pain in the butt, but at least I am back on track.

I now have some debugging to do, because the program crashes when it tries to start the dynamic recompiler, but right now I am just thrilled that I finally got the thing out of Eclipse onto the phone.
 

Ari64

New member
I now have some debugging to do, because the program crashes when it tries to start the dynamic recompiler, but right now I am just thrilled that I finally got the thing out of Eclipse onto the phone.
Of course it's not going to work until you fix the memory layout. After launching the process, cat /proc/(pid)/maps to see what the current layout looks like.

Some possible approaches are:

1) Try to fit the buffer into the bss segment.

2) Statically link the core library, and use the original linker options to relocate the text segment.

3) Create a jump table at runtime so that the dynamically generated code can call functions that are more than 32M away.
 
OP
paulscode

paulscode

New member
Of course it's not going to work until you fix the memory layout.

I'm confused.. I thought that since the assembly code is being built into the core library rather than as a separate library, that I can use the linking options "-Xlinker --section-start -Xlinker .init=0x08000000" set up the memory layout correctly. Sorry to keep bring up this same simple concept, but apparently I haven't grasped it yet..

I'll take a look at /proc/(pid)/maps and see whats there (assuming the required required components for this are actually on the Android - the OS is only loosely based on Linux). Although I'll have to access it quickly, because the program doesn't stay running for very long..

--EDIT-- There is not enough time to load the terminal emulator before the ap crashes, so I'm researching how to determine the process id programatically (from the native code). I'll use that to read the contents of /proc/(pid)/maps and output it to logcat from within the core library itself (that way when the ap crashes, I'll still have the output to review at my leisure). I'll post the results once I get it working.

--EDIT2-- Hmm.. it appears that the contents of proc/(pid)/maps is empty. That seems like a problem..
 
Last edited:

Top