Figuring out possible symbol table in a PS2 game - DDRMAX2 [JP] - Please help.

[travelsonic]

EDIT: May be in the wrong subforum, admin or moderator, if it is, feel free to move to the right place.

Well, technically, this could be applicable for DDRMAX US/JP, DDRMAX2 US/JP, DDR EXTREME US/JP, DDR PARTY COLLECTION, DancingStage mixes from that time period, and possibly DDR EXTREME 2, as I have observed what I'm sharing in all those games.

I was looking through the ELF executable for DDRMAX2 JP - and noticed at the end something like this:

(click for full size)
and
(click for full size)

Maybe I am off, but it looks to me to be some sort of symbol table. I heard that these tables, though not often, are left in tact at the end of PS2 executables.

Funny thing is, if I disassemble this in IDA PRO, these symbols are not used... which, hyper-analytical me, I start doubting myself.

Question 1: Am I off the mark on this, or are these symbols, am I looking at a symbol table of some sort?
Question 2: If I am correct in my analysis, is it possible to incorporate this information into a disassembly of the game? If so, what is the best means of figuring this out? I mean, I honestly do not know if SONY deviated at all from the ELF standards - whether in structure of everything else, or of the symbol tables, which is really making this a multi-million-dollar question... but if this data is what it is, and is structured well in a usable way, it would make figuring out, hacking this game apart a LOT easier.

Any answers, and help is appreciated, welcome. I am in uncharted territory so far as hacking this series goes, so if I seem a bit lost, well, I am.

[47iscool]

Try this

[travelsonic]

Try this

Tried it, doesn't utilize this data by default, gonna learn more about it to see if there is another way to try to get it to use it.

It has to be symbol table data, or at least variable names [wonder what the extra data between each piece of text would be, though, if not the information you'd find in a symbol table]

[dnawrkshp]

If you are looking at the executable then the issue might be that the elf has been packed. Certain data will then appear to not be used even though it is. I'd suggest getting a dump of the game and opening that in IDA or PS2Dis. The address that you found those strings at in the elf will be different than the ones in the dump.

As for associating assembly with strings in PS2Dis you have to run the analyzer (Analyzer -> Invoke Analyzer). Then you can browse labels or go directly to the place in the memory, select it with space, and hit F3 to find a reference.

[travelsonic]

If you are looking at the executable then the issue might be that the elf has been packed. Certain data will then appear to not be used even though it is. I'd suggest getting a dump of the game and opening that in IDA or PS2Dis. The address that you found those strings at in the elf will be different than the ones in the dump.

As for associating assembly with strings in PS2Dis you have to run the analyzer (Analyzer -> Invoke Analyzer). Then you can browse labels or go directly to the place in the memory, select it with space, and hit F3 to find a reference.

VERY useful information. Thanks a lot.

Now to find a utility to unpack PS2 ELF executables, or get ahold of an old version of PCSX2 that still has debugging capabilities [including dumping memory].

[dnawrkshp]

VERY useful information. Thanks a lot.

Now to find a utility to unpack PS2 ELF executables, or get ahold of an old version of PCSX2 that still has debugging capabilities [including dumping memory].

Unpacking most likely won't work because the ones available are probably not compatible with the one used to pack that elf.

As for dumping you can use Cheat Engine's Memory Viewer. From the viewer go to File -> Save memory region, set the start to 20080000 and the end to 22000000 and click add. Check "Don't include cheat engine header in file" and then save it. This way allow you to use whatever version of PCSX2 you want as long as it uses the same memory region for storing the game's memory.

Oh, when you load the dump in PS2Dis (which I'd suggest using over IDA as it handles dumps better IMO), be sure to set the starting address to 00080000 and not 00000000 (default).

[travelsonic]

Unpacking most likely won't work because the ones available are probably not compatible with the one used to pack that elf.

As for dumping you can use Cheat Engine's Memory Viewer. From the viewer go to File -> Save memory region, set the start to 20080000 and the end to 22000000 and click add. Check "Don't include cheat engine header in file" and then save it. This way allow you to use whatever version of PCSX2 you want as long as it uses the same memory region for storing the game's memory.

Oh, when you load the dump in PS2Dis (which I'd suggest using over IDA as it handles dumps better IMO), be sure to set the starting address to 00080000 and not 00000000 (default).

Alas, I set the region to those values, checked "Don't include cheat engine header in file" getting the dump from PCSX2 [running 1.0.whatever] and then ran the dump through PS2DIS setting the starting address as you specified and running the analyzer, and ... it found more stuff, but not the aforementioned labels I found.

A pity really, as it appeared that data was what it was. I'm not giving up yet though.

[travelsonic]

I've come to the conclusion that these symbols may not be salvagable, but ever since versions of PCSX2 with working debuggers - both the latest version, and a custom build called PCSX2DIS - have come out, I've been able to manually figure out where some of these symbols went, were used. Needless to say, I think parts of the BEMANI community will have a field day once I've finally compiled my notes on the DDRMAX - DDR EXTREME engine.

[Cronotrigga]

I've come to the conclusion that these symbols may not be salvagable, but ever since versions of PCSX2 with working debuggers - both the latest version, and a custom build called PCSX2DIS - have come out, I've been able to manually figure out where some of these symbols went, were used. Needless to say, I think parts of the BEMANI community will have a field day once I've finally compiled my notes on the DDRMAX - DDR EXTREME engine.

Pcsx2dis has a memory dumper. Open this up in the ps2dis, since in my opinion pcsx2dis for mapping things isn't worth the effort. These strings will probably show up more then once in the memory dump depending on the time you dump the game. I would go to the string and hit space bar and then hit F3 to find referrals. If no referrals are found copy the address of the beginning of the string
and hit shift + 3, paste the address you copied and press ok. This should find you ether a pointer that will point to that string which you may be able to find a function loading or you may be directed to a function loading this string. I have also seen in some games such as socom ca that a unique 32bit value may be used to locate the string. You will notice that a string will have weird data above or below and most strings in the game will follow this pattern. If this is the case try hex searching for half the value or the full value. Hex searching can be accomplished by flipping the bytes of the data.
example
24080008 will become 08 00 08 24

Note: if a pointer is found, be sure to try all the steps again on that pointer address.

[root670]

I've come to the conclusion that these symbols may not be salvagable, but ever since versions of PCSX2 with working debuggers - both the latest version, and a custom build called PCSX2DIS - have come out, I've been able to manually figure out where some of these symbols went, were used. Needless to say, I think parts of the BEMANI community will have a field day once I've finally compiled my notes on the DDRMAX - DDR EXTREME engine.

Looking forward to reading your notes . I've poked around Extreme a bit and mapped out most of the FILEDATA.BIN file, but was never able to find any sort of index in the main ELF with offsets. Did you find anything related to that? I did manage to make an LZSS de/compressor and figured out the graphics format to fool around with the game's graphics, but it was really tedious to identify each graphic's offset/size in the file by hand. Menu graphics, song banners, song backgrounds, song title cards, step information (SSQ format), MPEG2 videos, 3D characters, and the beginner helper graphics are all in there.

[travelsonic]

Looking forward to reading your notes . I've poked around Extreme a bit and mapped out most of the FILEDATA.BIN file, but was never able to find any sort of index in the main ELF with offsets. Did you find anything related to that? I did manage to make an LZSS de/compressor and figured out the graphics format to fool around with the game's graphics, but it was really tedious to identify each graphic's offset/size in the file by hand. Menu graphics, song banners, song backgrounds, song title cards, step information (SSQ format), MPEG2 videos, 3D characters, and the beginner helper graphics are all in there.

Dude, please share this info with me, maybe I can do something with it. I'm especially interested in the step information format, where that data is located, etc as well as graphics info.

So far, I've figured out a lot of littler things, some specific to one game (specific addresses used for what), and things more general to the game engine (how it does one thing or another:

• Filled in a few gaps about the song data definitions - the information that appears on the song wheel (BPM, difficulties, voltage, chaos, etc) for each song. A lot of info on this exists already on the forums for the site Aaron in Japan, but is incomplete, and full of guesses. I've also, I think, found variable names in the debugging information tacked on at the end of the ELF file that directly correspond to each piece in these definitions, which I say because the variable names follow the order of the known information PERFECTLY, and those I didn't recognize were between knwon information just like the bytes whose purpose we don't know are between the bytes we already know.... I THINK that made sense. .
  
• The game engine seems to index all the announcer clips, sound effects, and preview music, and - whenever a sound needs to be used - gets that index. I've done an incomplete mapping of each byte value, and the corresponding sound, for MAX2 JP. An interesting thing I've noticed about this is that the sounds aren't entirely grouped by what they are (announcer, SFX, song preview, etc) - and there are chunks of announcer sounds in between song previews and other sfx, for example.
  
• Yesterday I think I discovered where the game engine loads, and uses the actual BPM info (which would most likely, based off of other observations made by users on AIJ, be in frames, like how timing windows appear to be in frames). Still experimenting on this, and will keep my notes on this updated for sure.
  
• I've found out a bit about the song wheel itself, spacing between slots, how much of a "curve" it has - and even manipulated it into being a vertical listing (instead of a wheel), a-la SOLO BASS MIX and SOLO 2000 MIX.
  
• I've also figured out quite a few memory addresses, and what is stored in them at any given time:

• Where indexes for song data information [the information the player sees - bpm, difficulties, etc] are stored
• The number of slots in the song wheel. This I think is also used for the nonstop course selection
• The actual order of the song wheel, a series of words, each halfword being an index for a specific song. If you increment the number of slots in the wheel, you can append the list of values for each song wheel in the memory directly following when this one ends, and have it show up just fine. I think this area is also used for the order of the nonstop courses on the wheel as well.
  • That the Roulette option is the VERY LAST slot to be added, AFTER all the songs have been loaded

  
  
• - Scoring data during a song - perfects, greats, goods, boos, misses, max combo, and score. I've discovered 2 memory addresses for the score, one being the "actual score," what gets displayed, the other being the score that is being appended to.
• Where step data in edit mode is stored, and a bit more about what its like in this uncompressed format - though the arrow combos that each byte translates to might differ from game to game, might have to look into it more though. Interesting though how you can have, in the edit mode, quads, and triple-steps (or whatever you call hitting 3 arrows at once), but the quads don't show up at all in the game, and only some of the triples show up in game.

• I've actually copied over music from one game to another. Replaced Burning Heat! (3 Option Mix)'s music in MAX2JP with MAX. (Period) from EXJP successfully.
  
• I've even ALMOST managed to *SORTA* put in custom music, using a tool called ps2adpcm to convert the music I wanted to put in from wav to ADPCM, problem is, it's choppy as hell, once that gets fixed though, well, custom music FTW

I hope to take everything I discover, and everything else that has been discovered, attributing everybody's work with proper credit of course, and make a wiki for hacking DDR games... starting with the PS2 MAX - EXTREME series, and eventually expanding to include PSX and arcade mixes, as well as post-EXTREME PS2 mixes, and other console versions. IT would consolidate ALL the known information that quite a few people have compiled to assure preservation.

Another goal is to get the MAX-EXTREME DDR series games disassembled much like Sonic the Hedgehog games are now, where you have neatly organized, documented source code, and data (song data, BGMs, step data, songwheel song info definitions, game graphics, etc), and anyone wanting to make a custom version can edit the data, edit the assembly code, and assemble a game into a working ISO image.