Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[etabeta]

Hi Roman,

since you stated you were going to think a bit more about additional support for MESS software lists, let me move here the 'discussion' so that the MESS 0.141 thread at mameworld can rest in peace
Feel free to ignore this post until you are ready for look into the issue, I just wanted to collect here the info you might eventually need


First, the current batchrun option is very valuable and of course should be kept as is, given that it is the only way to scan/rebuild the software lists that have not yet been fully connected to MESS. We have indeed a few lists in the hash/ directory that cannot still be used to load software in MESS but that are important to document known dumps (e.g. c64_cart, a7800, a800, etc.)!

However, for the lists that have been fully hooked up to the emulator, it would be nice to have a slightly more complete support. Let me sum up a bit the ways in which you can currently retrieve xml lists from MESS.exe, so that you can get a better picture:

1. software lists appear in the -lx output: if a system has a list 'connected', its entry will look something like the gbcolor one:

Code: [Select]

        <machine name="gbcolor" sourcefile="gb.c" cloneof="gameboy" romof="gameboy">
                <description>Game Boy Color</description>
                <year>1998</year>
                <manufacturer>Nintendo</manufacturer>
                <rom ...>
                <chip ...>
                <softwarelist name="gbcolor" status="original" />
                <softwarelist name="gameboy" status="compatible" />
         </machine>Notice that
- there are 'original' lists and 'compatible' lists (the latter are always 'original' for some other systems, gameboy in the example above), to allow MESS to support backward compatibility of some systems; you can safely skip the "compatible" ones for basic support
- clones of a system will often have the same software lists as the parent (but not always, like the gbcolor example shows ), therefore lists are repeated many times inside the xml output
- some systems have multiple 'original' lists (e.g. fm7 has both fm7_cass for tapes and fm7_flop for disks)
- some systems have no lists
As a result, you can use -lx to obtain a full list of the 'connected' software lists (e.g. by listing each <softwarelist> entry and removing duplicates)

2. you can obtain a single xml file with all the software lists by using the "mess -listsoftware" command (I have also added a short version -lsoft this morning, so there will be also the short version in next version). This produces a (large) xml file containing all the lists

Code: [Select]

<softwarelists>
    <softwarelist name="a5200" description="Atari 5200 cartridges">
        <software name="xxx">
            ...etc...
        </software>
        ...etc...
    </softwarelist>
    ...etc...
    <softwarelist name="rx78" description="Gundam RX-78 cartridges">
        <software name="basic">
            ...etc...
        </software>
    </softwarelist>
</softwarelists>and here each list appears only once.

3. you can use -listsoftware with any particular system to retrieve its own list (say "mess nes -listsoftware" to obtain the nes.xml content), if there is one; but I think this might only be suitable for you if you ever get to advanced option support...


With all this in mind, a first possibility (but it is just a proof of concept, and it might possibly not fit clrmame, I dunno) could be
a. you run -lx and create a list of the available software lists (removing duplicates)
b. you prompt the user with a check list with the available software lists, so that he can select the ones he wants
c. you run -listsoftware generating the full set of lists in a file, and only parse the file to extract the selected lists

or, simplifying it further, you can only let the user to scan/rebuild all lists at once (skipping point b. and only parsing the whole -listsoftware output)

notice that the officially supported folder structure is only
- rompath/
- rompath/listname1/
- rompath/listname2/
- rompath/listname3/
- ....

where listnameX is the name of each list (in the example above gameboy.xml -> rompath/gameboy/, gbcolor.xml  -> rompath/gbcolor etc.), so that you can even only ask the user for the rompath he wants and then rebuild/scan like with the current batchrun


That's all, let me know if there is any point which is not clear, and thanks for the attention

[Roman]

Option 1)

Since there is currently no assignment of rompaths to special filter options possible (currently only per-bios / standard / mechanical is), the whole multi-softwarelist stuff needs to get split up into single profiles somehow. It would mean that if the parser finds several <softwarelist> elements in one dat/xml it needs to split up the dat into separated ones...and from there onwards it's like using profiles as usual...

Option 2)

jeeeeesus.....some kind of per-rompath assignment for multiple softwarelists (or generally per root-element)....sounds like too much work for now....maybe one day if I'm on holiday...alone...

[etabeta]

first off, thanks for the reply.

my idea was definitely along the lines of option 1, but keeping the part "it needs to split up the dat into separated ones...and from there onwards it's like using profiles as usual..." invisible to the user as if the whole stuff was a single profile at his eyes: users should setup a unique rompath "roms/" and then clrmame can load the different lists treating them as different profiles with fixed rompaths "roms/list1/", "roms/list2/" etc.

option 2 sounds indeed appealing but I can barely imagine how all the possible options might fit together... therefore, I would go definitely for option 1, unless you have *loooong* holidays

[Roman]

well....your 1) ideas of automatically creating rompaths within a given root can be already done by the batcher adding this thing again just invisible for the user is just a waste of time....but time will tell...currently I simply have nearly no time...and other things are on my agenda for years...e.g. chd-rebuild (which is actually not too far away).

[etabeta]

adding this thing again just invisible for the user is just a waste of time....

I agree, but a lot of people (Haze and Twisty among them) keep complaining that it would be *OH SO* better if the list rebuild process could be closer to a 1-click operation. Hence, I had at least to ask

I have anyway to admit that making it invisible would slightly reduce the time needed for the whole thing, and it might really improve the feeling towards lists if you could make the final report to sum up all missing files across the lists (something that you cannot do with a batchrun)

If you really think it is worthless, however, no problem: I can live with it

[Roman]

I should have a patent on "one-click-do-all"

Well...as mentioned...it's on my list....and if I find time I will have a look at it again...currently it's more collect ideas....and when I got 2 days off add them.....or a subset of them...

[etabeta]

I understand perfectly. Thanks for the discussion.

[Pugsy]

I must admit I'm not a great lover of the batch facility, I've tried it the past to do a full TOSEC ROM rebuild and it would have taken for ever. In the end for TOSEC, I made a small program that joined the hundreds of dats into about 4 combined dats with "rebuildto" fields - strewth was that fast. There were a few problems with scanning and duplicate names but they were fairly easy to overcome.

Anyway, this thread got me thinking why not do the same thing for the MESS SoftwareLists and take some of the pain out of rebuilding/scanning process.

I took my original TOSEC joining program and altered it to parse the MESS XMLs and create a single simple clrmame.dat file (I ignored clone relationships and chds and used the .dat format because I had already done the code for that before).

It actually works fairly well, coupled with some slight changes to clrmame it could work really well for MESS and TOSEC.

Anyway, I've documented the current process and have some questions/suggestions here:-

http://cheat.retrogames.com/MESS_XMLS_TO_DAT/

[Roman]

Well, rebuilding speed (no matter if in batch mode or not) depends on the settings you're using and of course the sets themselves....even a full MAME rebuild can take rather long (with recompress and sha1 checks) or can be rather quick (with no recompress etc)...

rebuildTo flags got their charme...yes...but only for rebuilding...the scanner doesn't know them. Setsubfolders (rom name="apple/pacman/bla.bin") can be used, too.

There is currently no assignment of rompaths to a special condition (excep bios / standard /mechanical).

So a) multiple rootelements "<softwarelist>" need to get parsed into one profile....and you need some kind of set of rompaths/samplepaths for each of these...actually this sounds quiet easy during typing this text...addpaths etc can be handled 'for all' I guess....so all I need is some kind of scan/merge/rebuild loop for all each of such list...

so actually from a std:vector<Sets> to a std:vector<std:vector<Sets>> .... :O)

[Pugsy]

Thanks, that sounds hopeful... I've wanted multi-folder rebuilding/scanning since I first touched TOSEC. That and the capability of having a dat which will recreate a folder structure that contains both normal files and archives (ala GB64).

I found the batch mode slow in the past (I last used it years ago) because I generally do a full rebuild so I give CLRMAME a folder with a hundred gigs or more of ROMs in it and let it do it's stuff. The problem I had with multiple dats was it would parse this source over and over and over again. Join DATs together and this extra parsing of the source folder was reduced dramatically.

[Roman]

well, there's a rebuilder advance option which skips a sourcefolder after x skipped files....guess that could decrease the time of looking in ypur folders over and over again in a batch run...however you need to set that up per profile....
well...lets see what 2011 brings...