Hey You! Pikachu - Possible HLE Implementation

[Falkoner]

I always enjoyed Hey You! Pikachu, since the game seemed to have to many secrets and things left to discover. It's always made me a bit sad that it hasn't been emulatable due to the microphone, however, I thought through it a bit, and it seems like with a fairly small amount of effort, a usable HLE implementation could be made, that could end up even better than the original as far as Pikachu's ability to understand what you just said.

Looking at the last and only attempt to emulate the N64 Mic(forum.pj64-emu.com/showthread.php?t=3583), I decided that it's not worth trying to do an LLE implementation, mostly because I'm a complete novice in that area.

However, considering we have the list of phrases available in the game(pastebin.com/aG3rugAm), someone should be able to rapidly pull the matching digital responses for each phrase in the game(this may even already be listed in the ROM with the corresponding string, I couldn't manage to view the strings though). From there, a simple GUI could be made with buttons for key phrases allowing you to at least play the game.

Once that's done, you'd use one of the open-source speech-recognition softwares (en.wikipedia.org/wiki/List_of_speech_recognition_software#Open_source_acoustic_models_and_speech_corpus) out there to pick up on the words and automatically send the correct digital command.

If that were in place, I think Pikachu may become more intelligent, it actually wouldn't be hard to add additional synonymous phrases that correspond to existing phrases to Pikachu's vocabulary, and you could even do translations to any language rapidly!

Any takers?

Also, for those interested, I put together a prototype of using a PC mic to detect the game's commands:

Hey You, Pikachu! Mic Test

So I went ahead and built a dictionary for Hey You! Pikachu using PocketSphinx, it works pretty well (about as well as the original game did). Needs some tweaking for commonly used words and Pokemon names (Pikachu is hit and miss), but otherwise it's actually quite good.

Run the batch file and say any of the in-game commands into your mic to see if it properly detects it.

I just need to look into tweaking the dictionary to be more precise for certain words and stop it from attempting to find more than one phrase used at once. Of course, still needing someone to find the VRU output for each phrase to pair it with this setup.

 

[Falkoner]

So I'd like to know if anyone knows how to extract the string data from Hey You! Pikachu, I attempted with a few of the available tools, but I assume it's compressed in some way, and I'm a serious newb in this area.

Basically, I'd like to see if we can get the list of recognized phrases in the game, paired with the codes the game associates with them when recieved from the mic adapter.

 

[AIO]

So I'd like to know if anyone knows how to extract the string data from Hey You! Pikachu, I attempted with a few of the available tools, but I assume it's compressed in some way, and I'm a serious newb in this area.

Basically, I'd like to see if we can get the list of recognized phrases in the game, paired with the codes the game associates with them when recieved from the mic adapter.

You could try searching for text in RAM, if you think it's compressed.

 

[Falkoner]

You could try searching for text in RAM, if you think it's compressed.

I created a savestate in Project64 (By the way, from what I can tell, most current graphics plugins can't seem to handle the game), and then I extracted the .zip and tried searching it with a Hex Editor. I can see some strings, such as "Hey You Pikachu" backwards, but I can't seem to find the list of actionable phrases.

Looks like the whole output is 32-bit byte-swapped, anyone know how to reverse that in a hex editor? EDIT: Nevermind, just figured it out. Now the file is much more readable.

However, doing some searches doesn't appear to be turning up any results. I don't know if the verb list is stored in RAM, but apparently someone online was able to dump it, I'd just like a full dump (with the pokemon names as well), along with the codes associated with each phrases that would be recieved from the mic.

 

[Fanatic 64]

Wait, do you really think the voice commands are stored as plain text on the ROM?

Heck, I think I once read that the voice is interpreted inside the Voice Pak, which just sends standard inputs to the console.

 

[Falkoner]

Wait, do you really think the voice commands are stored as plain text on the ROM?

Heck, I think I once read that the voice is interpreted inside the Voice Pak, which just sends standard inputs to the console.

Well, based on the reasoning given at the top and bottom of this list of voice commands, it appears it must be available, unless somehow the dude had the original source code.

Even if it isn't, one could still play through the game using Adaptoid and record the codes returned to the system for the various keywords throughout the game. However, I don't believe this is necessary due to the existence of that list.

 

[Fanatic 64]

Isn't it possible that he just played trough the game and wrote down all the possible commands?

 

[Falkoner]

Isn't it possible that he just played trough the game and wrote down all the possible commands?

Not sure if you actually looked at the list:

Here is the full list of voice commands in Hey You Pikachu!, as they were entered into the coding, only taking slight liberties in punctuation for ease of reading. There are many repeats, and it's rather disorganized, but that is only the fault of the programmers themselves.

(Hey You Pikachu is property of Nintendo and Ambrella, and I am not affiliated with either company. This list was transcribed from the coding of the game by Leo G., also known as "ViolentGirlfriend" in January of 2013)

Also, further evidence:

pikachu
pikachu2
pikachu3
pikachu4
pikachu5
pikachu6
pikachu7
pikachu8

If he were just writing it down, there wouldn't be any reason to put down Pikachu 8 times, but obviously for the code there were various ways people said Pikachu.

Besides, even if he did, it means we'd at least have a complete list of what would need to be tested for a functional game, even if it weren't complete. Just test each phrase and grab what code the VRU returns through the Adaptoid. However, it's exactly THAT that I want to avoid, my hope is that the list was listed in something like a 2D array with the codes for each phrase listed along with the phrases, or maybe just two matching lists. Maybe a helpful first step would be for someone to record what the returned data looks like for a few of them when said into the VRU, so we can search for that in the ROM.

 

[Falkoner]

Looking into it further, assuming we can get the system working to the point that it can pass in the correct commands programmatically, we'd generate a custom Sphinx dictionary for exactly the words recognized by Pikachu.

 

[zoinkity]

Text strings aren't sent to the VRU to initialize it. They send an encoded format that hasn't been completely decrypted. In fact, I think the people working on it were working with the Japanese unit first since it's better documented.

They have to send an encoded format to deal with combined sounds. In Japanese that would be vowel replacement, and in English that's vowel combinations (oo, ae, ie, a*e, etc.) and consonant combinations (th, ch, etc.). The unit does not have the logic necessary to do grammatical interpretation.

A first step is to determine where they ripped the strings from and their correspondance to data. Most likely these are internal notes, either for a debugging or just comments compiled in. They aren't a complete list of commands either. Where's "Sega"?

 

[Falkoner]

Text strings aren't sent to the VRU to initialize it. They send an encoded format that hasn't been completely decrypted. In fact, I think the people working on it were working with the Japanese unit first since it's better documented.

They have to send an encoded format to deal with combined sounds. In Japanese that would be vowel replacement, and in English that's vowel combinations (oo, ae, ie, a*e, etc.) and consonant combinations (th, ch, etc.). The unit does not have the logic necessary to do grammatical interpretation.

A first step is to determine where they ripped the strings from and their correspondance to data. Most likely these are internal notes, either for a debugging or just comments compiled in. They aren't a complete list of commands either. Where's "Sega"?

Exactly, that's the idea, rather than try to decrypt the format, simply hardcode a list of words and their corresponding response that the VRU sends back to the N64.

Does anyone know how to get the uncompressed contents of the ROM?

 

[dazey238]

No idea I'm afraid. Although I was thinking that maybe a different avenue might be to construct something like this -

http://www.pieter-jan.com/node/10

And attach a VRU to it to get the signals???

 

[zoinkity]

Haven't written a decompressor for it yet. Busy with another project at the moment. Everything else is on hold.

You're barking up the wrong tree if you're looking for a textual word list.
If you want to avoid device initialization outright, just record the output packet on the SI from a real device when you talk into it. PC side, you'd only have to have a VRU detect a word and match the sequence against a dictionary of indicies.

 

[Falkoner]

Text strings aren't sent to the VRU to initialize it. They send an encoded format that hasn't been completely decrypted. In fact, I think the people working on it were working with the Japanese unit first since it's better documented.

They have to send an encoded format to deal with combined sounds. In Japanese that would be vowel replacement, and in English that's vowel combinations (oo, ae, ie, a*e, etc.) and consonant combinations (th, ch, etc.). The unit does not have the logic necessary to do grammatical interpretation.

A first step is to determine where they ripped the strings from and their correspondance to data. Most likely these are internal notes, either for a debugging or just comments compiled in. They aren't a complete list of commands either. Where's "Sega"?

It seems very likely to me that the list is complete:

It was heavily rumored that, by saying things like "Sony", "Playstation" or even "Sega" into the microphone, you could get Pikachu to become mad. However, it appears that this is in fact an urban legend, as there is no indication in the data file that these words were supposed to be registered. The few registered cases of an Angry Pikachu were probably just caused by the game not understanding what was said, which in turn led it to just pick a random word similar to what was heard.

Haven't written a decompressor for it yet. Busy with another project at the moment. Everything else is on hold.

You're barking up the wrong tree if you're looking for a textual word list.
If you want to avoid device initialization outright, just record the output packet on the SI from a real device when you talk into it. PC side, you'd only have to have a VRU detect a word and match the sequence against a dictionary of indicies.

The existence of that other list and citations from other sources talking about reviews of the data files leads me to believe that a wordlist does indeed exist. At the very least, words in-game that are recognized are always highlighted in purple. The real question is if this wordlist also contains its corresponding VRU response code, or if that'll have to be compiled manually, as you stated.

No idea I'm afraid. Although I was thinking that maybe a different avenue might be to construct something like this -

http://www.pieter-jan.com/node/10

And attach a VRU to it to get the signals???

You can just use an Adaptoid, no need to get the soldering gun out

I'm assigned to a massive project currently, but I should be finished around the end of February, around there I'll likely get started on this project, even if it means manually finding the codes. I already have the VRU, I'll just need to buy an Adaptoid and get coding.

 

[tony971]

Bump?

 

[Falkoner]

Bump?

Well, unfortunately getting my hands on a real Adaptoid is a lot harder than I previously imagined, so I've been pretty majorly setback by that

I would still love to see this, maybe I actually will consider soldering my own Adaptoid together That being said, if anyone on here has one and the Hey You! Pikachu VLU, they could definitely help out.

 

[zoinkity]

Didn't mention before, but the file compression is a headerless lzh, format 5.

 

[Falkoner]

Didn't mention before, but the file compression is a headerless lzh, format 5.

Oh man, that is a major help! I unfortunately am terrible at reverse-engineering, and since there's no -lh5- header to find when I hex edit the file, I'm not sure how to go about decompressing the data. It appears to be byte-swapped, but that's not getting me anywhere, is there any way you could give me a few pointers to getting an uncompressed result so I can search the results for a string table?

I've got your Midway N64 Decompressor installed, but I'm not sure exactly how to use it on my .n64 file. I think making a file list is outside my skillset.

Interestingly enough, simply byteswapping it allowed me to find the original list of phrases that link had, they begin at the 000b0010 offset. My new list also includes the Pokemon names that were excluded from the previous one, so in total there are 640 commands in Hey You, Pikachu!, however, many of them are repeats for different pronunciations. Decompressing it, I'd like to see if there are any other near 640-sized arrays that could be the matching codes that can be received from the VRU.

Anyone who is interested in supporting this project, I'd gladly send them a VRU assuming they can prove that they will be able to move the project forward with it in some way.

 

[Falkoner]

Just went through and made version of the Hey You, Pikachu! command list with duplicates merged, so it reduced the number of actual in-game commands down to 459. Just trying to get it ready for potentially having to brute-force the codes, and 459 isn't too many to deal with.

Still need to find someone with an Adaptoid to help me out though

 

[weinerschnitzel]

Well, they are pretty pricey on ebay, and I'm sure the auctions are still difficult to win. Maybe find someone on the BST forums of a place like assemblergames or nintendoage?