by mh » Thu Sep 10, 2009 2:23 pm
Storing in the BSP would require a new format because the lightmap s/t calculation is hardcoded as a >> 4 in the engine, so at the very least some kind of flag to say "hey! shift by 3 instead!" would be required. You'd also need to increase the size of the blocklights array, as the current 18 x 18 is strictly based on a max surface extents that assumes left-shift by 4. Finally, the lightmap offset for each surface would also need to be changed or multiplied to reflect the bigger format. Something has to signal to the engine that these need to be done.
I like the idea of doing it in the LIT file, possibly extending the version number header integer to take a combination of flags. This way the old lighting can still be retained in the BSP (retaining compatibility with any engines that don't support the larger size), and the LIT file itself can signal to the engine that things are different.
Be aware however that doubling the lightmap resolution in either direction would have a 4 x overhead on animated and dynamic light calculations and uploads.