PalMod: ROM Exploration.

This guide is intended for people that want to find or even modify regions of a game ROM that PalMod does not already know about. There are other arguably better ways to do this: this is just one way you can do it using PalMod.

Tools You Need For This

PalMod
PalMod itself: a tool used to modify palettes in game ROMs or files. When exploring ROMs, it can be useful to have a second instance of PalMod open to a "known" game so that you can check what a given palette looks like. You can even paste the palette colors from your ROM exploration onto a sprite we have to see if it looks correct.
The ROM for the game you're interested in.
If you want your life to be simple, you're going to start with one of the game ROMs that PalMod already supports. If you're feeling frisky, you can load other ROMs into PalMod.
An Extras file.
This is a text-file extension to PalMod. This is a powerful extension that lets you use PalMod to read regions of the ROM that PalMod doesn't already know about. To do this you place an Extras file in the same directory that you launch PalMod from. If you are certain you know what you're doing and that what you're looking at is a palette, you can even update the ROM at those locations. Bad Extras files may crash PalMod: : to fix that just fix or remove that bogus Extras file.

Concepts

Game textures
In order to efficiently allow a game to recolor a sprite, the sprite is usually stored as a texture. A texture is raw image data that is functionally a paint-by-number sheet. When people create a RAW file, that is the equivalent of a game texture.
Palettes / Color Tables
Palettes, or color tables, are the colors used to fill in that paint-by-number texture. One important factor is that there will always need to be a color for the background of the sprite: that is usually the "color" at position 0 in the palette. Some games actually stuff extra data in that position so it's important to never change that. The position 0 color is called the transparency color (since you want to set the sprite inactive area to be clear) or the counter color (since some games stuff counter data or other information in this 'color').
Hexadecimal
You tell PalMod what locations to read using hexadecimal (hex for short). That is a base 16 system that is usually represented as "0xNUMBER". (Decimal is represented by "0nNUMBER", but PalMod forcibly treats all numbers as hex.) Hex numbers count as 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b, c, d, e, f, 10, 11, etc. PalMod will let you write a hex number as either "a9a9a9" (raw) or "0xa9a9a9" (has the indicative '0x' prefix). I prefer the "0xa9a9a9" style.
Palette storage
Games will typically store their palettes in one region of one game file. Palettes for most games only ever start on even numbered locations (ending in 0, 2, 4, 6, 8, a, c, or e). If you get lucky, the palettes will align / start on 0x...0 locations. If the palettes have their transparency color at a different location, please make sure you update your palette file to have that transparency color location set as the first position in that Extra. If you don't do this and try to have PalMod write to the game, we might stomp on the transparency color which can cause weird effects.
Palette length
Classic fighting games typically use 16 colors per palette, then there is another palette (lead by another transparency color), and repeat until the end of palette storage. When games started to get fancy around the SF3 era, they started using more colors: palettes can be a hard maximum of 256 colors before you would expect there to be another transparency color (or the end of that palette). If a sprite wants to use more than 256 colors, that would typically require multiple textures referencing multiple palettes.

Getting Started With Extras Files

Your Extra file can contain a very large number of Extra definitions, but it's easiest to start small. Extra definitions are a simple way for you to tell PalMod to try to treat arbitrary regions of the game ROM as if that was color data. It might not be: figuring out where the color data is for the game takes some laborious trial and error. The Extras file names you can use for the various games are covered in the ReadMe included in PalMod. You don't need to know these names: you can use PalMod's Tools : Open Extra File menu option for any games that would support Extras files and we will open it up for you. We will also give you some useful options.

Each Extra definition is comprised of three lines. You specify one line with the title for a given palette, the next line being the starting position in the ROM for that palette, and the third line being the ending position in the ROM for that palette. For example:

Ryu's Pet Hamster 0x2400e 0x2402e This would create a PalMod entry for the "Ryu's Pet Hamster" palette that reads the 0x20 (0n32) bytes (representing 16 colors at two bytes per color) between 0x2400e to 0x2402e. You can specify a lot of Extras entries if you need to: at one point Jojo's support was entirely being done via SFIII3's Extras file support.

All Extras loaded this way will be available at the bottom option of the top listbox as "Extra Palettes". When that is selected your specific Extras will show up in the bottom listbox. Any palette larger than 128 colors will be shown across multiple pages. If you specify a gigantic palette we may show you a warning to be careful. We don't show any preview sprites for Extra Palettes.

Please note that any ROM locations/palettes present in both PalMod and in the Extras file is shenanigans: we will end up creating two references to that palette and you would need to modify *both* to change it. Please don't duplicate palettes. PalMod will make a good faith effort to check for duplicates when it loads Extra files and will try to warn you if it sees any dupes. For the purposes of ROM exploration, you might want/need to duplicate palettes: just ignore the warnings in that case for your personal use. For integration with PalMod, your Extras file should not contain any errors or dupes.

If you want to try to experiment with a single new ROM via Extras file hacking, use PalMod's "Unknown Game ROM" mode at the end of the Load ROM by Game file filter list for a single file ROM. That lets you load any single file you want. If you want to try to experiment with an interleaved set of files via Extras file hacking, use the Developer Mode option under Load Directory.
You can then specify the color format you think it might be under Settings :: Color Format. Set how often you expect the transparency color under Settings :: Color Format :: Alpha. Then reload to use the new settings, and use the Extras file to explore the ROM and see if you can find the palettes. This is difficult and laborious, but it's much easier this way than it is most other ways. If PalMod supports another game in that game's series, you can open that other game in PalMod and try to see if your new unsupported ROM uses the same color mode that PalMod uses for the game it does support. As long as you are not changing color size (16/24/32), PalMod will update to use any new color format once you display any palette other than what is currently being shown.

ROM Exploration with PalMod

Create an Extras file identifying the ROM regions you want to explore. When I'm getting started I might read ridiculously long ROM regions. For example for COTA the first part of my exploratory COTAE.txt Extras file looked like this:

Early ranges 0xe 0x2011e Slightly later 0x2011e 0x3011e That of course is a huge range, and PalMod helpfully shows me a warning about that madness. PalMod disallows any native palette larger than 256 colors (the maximum size of a standard color table), but Extras file palettes can be gigantic and unwieldy. I then reloaded the ROM and got some more (expected) warnings about dupes and reading unexpectedly low areas of the ROM.

Now we change the top listbox drop-down to be "Extra Palettes" if it isn't already. We should see PalMod's interpretation of the ROM data as if everything was a color. (You can probably close the Preview window at this point, too: you can start loading in textures, if you have them, once you're getting to late-stage refinement of the ROM locations. )
Viewing a ROM in 8 color per line mode.  You may prefer 16 colors per line if you're looking for stages.

Now for the boring part. Just keep walking ROM pages. You're looking for patterns in the data and chunks of colors that look related. Character palettes are usually 16 colors each. Everything in the COTA ROM is probably going to look like a jumbled mess until about when you hit 0x2611e. Things start getting interesting at this point.
Starting to look like object palettes

Part of finding patterns is knowing what to look for. Helpfully, the CPS2/Vs series games reuse a lot of sprites, so we can try to check for palettes from this ROM that match palettes from another game in the series. And on "still early (103/256) 0x2671e" we hit our first bingo: we find an exact match for the main MvC2 Psylocke palette. You can copy the palette from 0x2673e over to MvC2's Psylocke and you'll see that it matches exactly. Additionally, most of the CPS2-era characters have their P1 main palette followed by three P1 "extra" palettes followed by their P2 palette palette (and three P2 extra palettes). And if you look at 0x267be - yep, that's Psylocke's P2 color. And the P1/P2 extra palettes generally match their MvC2 versions which makes things simpler.
Psylocke's region in COTA: note it matches exactly with the MvC2 palette

So offhand if we're in a character region we would expect the next character palette set at (start of character P1 palette + (0x20 * 4 palettes (1 main, 3 extra)) * 2 (P1 and P2 colors)). And indeed at 0x2683e we see a markedly different palette. If you try to match that against palettes for COTA characters present in MvC2, you'll realize that that's Colossus' palette. And to jump ahead: you'll realize the the COTA character sort order is Psylocke (0x2673e), Colossus (0x2683e), Cyclops (0x2693e), Storm (0x26a3e), Iceman (0x26b3e), Spiral (0x26c3e), Omega Red (0x26d3e), Sentinel (0x26e3e), Juggernaut (0x26f3e), Magneto (0x2703e), Wolverine (0x2713e), Silver Samurai (0x2727e), and finally ST Akuma/Gouki (0x2737e). (The sort order in a game is usually adhered to, but can differ for things like character select icons.)

So now as an experiment I would update my Extra files to only contain the color palettes/ranges I wanted to change. Then I would reload the game (to reload the Extras file), modify those palettes, save to ROM, and then test it in WinKawaks. We don't need fancy palmods for this: you can blast them redder or greener or alternating solid color pixels: just something that will indicate that these are the right palettes for what you care about. And indeed if you modify those you'll see that those characters change. Typically when I am just getting started I will make one entire PalMod page (256 colors) red, the next one green, the next one blue, and so forth: that lets me quickly set (number of colors) * 256 pages worth of palette data at once. I note what sprites that I care about are changed through playing and experimenting with the game, and then restore the ROM from backup so I can try the next bunch of pages. Once I have done that to my satisfaction, I go back to individual pages and do the same thing line by line for each color page that I care about.

We can check MvC2's palette sets to figure out which palettes we might be dealing with/looking for. But if we're just lost or want to experiment: blast pages of the ROM to specific colors. You'll want to keep track of this, but as you slowly modify a page of colors at a time to be all red or green or yellow or etc, you can figure out if those colors are used in the places in the game that you care about or not. That lets you start honing in on the exact regions and eventually individual palettes that you care about. For example, I now know that the Master Mold stage colors are stored around 0x36cfe: if I ever want to start working on stages, that's where I'd start.

An important thing to keep in mind is that you only want to use PalMod to write to palette/color regions. PalMod can write colors well, but cannot write non-color data. Please ensure that the Extras ranges you modify solely contain color information before you try to write to the ROM. If you modify anything other than palettes/colors with PalMod, you will quite likely corrupt your ROM (since converting raw game data to color data is bad) and need to restore it from backup.

Remember to keep updating your Extras file as you go along. If a section of a ROM isn't of interest, comment it out with a ";" in front of each line, or add comments or update the description string so you know what it is. I slowly keep slicing ROM sections in my Extras file until I know clearly what everything I care about is, and then I fold it into PalMod.

Good luck. :)