EMULAB Forum
clrmamepro [English] => clrmame Discussion => Topic started by: etabeta on 28 April 2010, 09:00
-
Hi Roman.
In the past few weeks, we have added in MESS support for software lists (in xml format). In other words, MESS is now including software info (in xml format) like MAME does, in addition to bios info.
Software lists currently work as follows:
* -listxml output now has a <softwarelist> item for systems which supports xml lists (as if it's a device for the system)
* a new command, -listsoftware allows to either export all the supported xml lists (if launched as mess.exe -listsoftware) or a single xml list for a given system (if launched as e.g. mess.exe pokemini -listsoftware)
Notice that the xml structure is basically a simplified version of the MAME one (see http://git.redump.net/cgit.cgi/mess/tree/hash/softwarelist.dtd )
Now, let me move to the point of this post: is there any hope to see clrmame directly support our lists [1]? and if so, is there anything we can do to help you with your side of the work? e.g. would it be eaier if all lists appear in the main listxml output? or would you prefer the current approach where we leave lists as a separate output?
A possible setup might be:
- clrmame runs mess -lx
- then, it parses the output looking for <softwarelist>
- finally, it offers to the user the choice of using one of the lists to rebuild or scan a directory (and the actual xml to be used can be called by running again mess with "system -listsoftware" parameter)
Alternatively, clrmame might create profiles for each of the software lists, when it parses the listxml output... or even ask the user if he wants to import the software list...
Not knowing the inner structure of clrmame, I have no idea if any of the proposals fits clrmame architecture, but I would be interested to know if there is any hope for this.
Regards
[1] I think it's already possible (but I haven't tried) to use our xml to produce a clrmame dat for the specific systems, by using logiqx tools... but it's a procedure which is a bit too complicate, compared to how easy is to scan MESS bioses and MAME romsets
-
You got a mess binary handy to play with? One which supports the additional commands?
....and of course you can always use XSLT to transform the software list into something readable ;)
ah...got one...hmm...I will have a look at it...but don't count on it...
-
hmm...ok...got a latest SVN build but using the commands only print out an empty list....I guess the real software list data is kept externally and will be included there if available....do you have some test data to play with?
-
that's because bobz prebuilt exe do not includes the hash/ directory. sorry, I hadn't thought to that.
you can take the current lists from
http://git.redump.net/cgit.cgi/mess/tree/hash/
notice that you have to take the xml files, not the hsi, and put them in the hash/ directory. not all the lists are currently supported (even if I'm working on that, and probably tomorrow most of them will be supported). I suggest you to take scv.xml, pv1000.xml and ibmpcjr_cart.xml to test.
thanks for looking into this
-
hmm...still no luck with it...all I get is (besides of the !ELEMENT stuff)
<softwarelists>
<softwarelist name="ibmpcjr_cart">
</softwarelist>
</softwarelists>
-
mmm... here, if
1. I take the exe from http://bobz38.free.fr/mess_autobuild/index.php
2. download ibmpcjr_cart.xml from the git repository
3. put ibmpcjr_cart.xml in hash/ (after creating the directory)
4. launch mess -listsoftware or mess ibmpcjr -listsoftware
it works... going to investigate some more...
EDIT: I confirm that the above procedure should work. the only things I can think of to cause your problems are: you downloaded by mistake ibmpcjr.hsi in place of ibmpcjr_cart.xml, or you have some different hashpath set in your mess.ini. I suggest to try in a new folder to exclude the latter possibility
-
no go....
D:\temp\mess>mess ibmpcjr -listsoftware
<?xml version="1.0"?>
<!ELEMENT softwarelists (softwarelist*)
<!ELEMENT softwarelist (software+)>
<!ATTLIST softwarelist name CDATA #REQUIRED>
<!ATTLIST softwarelist description CDATA #IMPLIED>
<!ELEMENT software (description, year?, publisher, part*)>
<!ATTLIST software name CDATA #REQUIRED>
<!ATTLIST software cloneof CDATA #IMPLIED>
<!ATTLIST software supported (yes|partial|no) "yes">
<!ELEMENT description (#PCDATA)>
<!ELEMENT year (#PCDATA)>
<!ELEMENT publisher (#PCDATA)>
<!ELEMENT part (dataarea*)>
<!ATTLIST part name CDATA #REQUIRED>
<!ATTLIST part interface CDATA #REQUIRED>
<!ATTLIST part feature CDATA #IMPLIED>
<!ELEMENT dataarea (rom*)>
<!ATTLIST dataarea name CDATA #REQUIRED>
<!ATTLIST dataarea size CDATA #REQUIRED>
<!ATTLIST dataarea databits (8|16|32|64)
"8">
<!ATTLIST dataarea endian (big|little) "
little">
<!ELEMENT rom EMPTY>
<!ATTLIST rom name CDATA #IMPLIE
D>
<!ATTLIST rom size CDATA #REQUIR
ED>
<!ATTLIST rom crc CDATA #IMPLIED
>
<!ATTLIST rom md5 CDATA #IMPLIED
>
<!ATTLIST rom sha1 CDATA #IMPLIE
D>
<!ATTLIST rom offset CDATA #IMPL
IED>
<!ATTLIST rom status (baddump|no
dump|good) "good">
<!ATTLIST rom loadflag (load16_b
yte|load16_word|load16_word_swap|load32_byte|load32_word|load32_word_swap|load32
_dword|load64_word|load64_word_swap|reload) #IMPLIED>
<softwarelists>
<softwarelist name="ibmpcjr_cart">
</softwarelist>
</softwarelists>
-
does your mess.ini contain the correct hashpath? that kind of result usually points towards a missing xml list, but then it should not even show the name="ibmpcjr_cart"... it's so weird...
does the ibmpcjr_cart.xml file contain all these pieces:
http://git.redump.net/cgit.cgi/mess/plain/hash/ibmpcjr_cart.xml ?
at my end, it works like a charm with the procedure above (I tried again from scratch on my setup, winxp 32bit, and it also works on macosx 10.6 32&64 bit with self compiled executable). I really have no idea what the source of your problems could be...
-
ok..got it working now
-
great! :)
-
sorry if I reply to my own post, but I wanted to point out that now all the xml lists (but 32x) are hooked up to the correspondent drivers. in other words, if you need more testcases, you can choose any available xml file, put it in the hash directory you created and it will be recognized :)
-
hmm...well...currently I only see a chance to support them as a xml file, i.e. you need to create the xml for the softwarelist yourself and then you can load it in the profiler similar to other dats...
<softwarelists>
<softwarelist name= ..../>
</....>
hmm....I guess I will also only support one softwarelist in softwarelists.....
-
for sure it would be a good start and it would make my work way easier (whenever I want to rebuild one of the lists, I currently have to convert lists to use MAME-like fields and then to convert it through datutil and finally load the dat in clrmame ;) )
notice that we currently
1. do not define a specific DOCTYPE
2. use cloneof only without romsof (and this is probably not going to change)
is anything of the above a problem for clrmame?
also, would you need any specific change to the xml format (or to the dtd) to make easier the support? or is there anything else which could allow (or help) further integration between our lists and clrmame (except hiring you to do this as a full time job ;) )?
in any case, thanks to have looked into the thing :)
-
you got a software list handy with cloneof statements? The url http://git.redump.net/cgit.cgi/mess/plain/hash seems to be down at the moment
-
here you can find the whole bunch of softlists (~1MB)
http://mamedev.emulab.it/etabeta/fast/files/roman_xml.zip
plenty of examples both in small lists and in larger ones. notice, though, that most of the parent/clone relations in current lists do not imply shared roms (due to most cart dumps consisting of a single file); things will change with NES which will support separate prg & chr files and which will feature many clones using the chr file of the parent with a clone-specific prg file like parent-clones in MAME (even if MAME xml also adds romof in this case, while we don't, to differentiate the presence of a bios parent). I can provide a sample of this (not yet ready) NES list, if needed.
-
ok...hashfile works in profiler, -listsoftware output works also... however I found something weird looking in ibmpcjr_cart:
<rom name="basic.bin" size="32768" crc="8fc2a724" sha1="216915e954d393e42a30d1eb8666371439dcbd38" offset="0"/>
<rom size="32768" offset="8000" loadflag="reload" />
the second rom tag doesn't hold any name......any comment on this? Should this mean to load the same rom again at a different offset?
-
apparently yes, even if I hadn't noticed that such a case could occur: that code is supposed to reload the same rom in the new offset (at loading time, softwarelists "dataarea" are transformed into MAME rom regions, and various MAME flags might apply).
I think that, even if more flags could be added later, you can safely assume that only <rom> with a name corresponds to separate rom entries to be built in a set...
-
ok...but cmpro will show a warning for the rom without name then.....
-
fine with me, as long as you can say "yes to all" when the first warning appears ;)
-
hehe...no...it will show that for each rom...but actually I can switch that off for softwarelists...
-
btw, it just hit me the fact I haven't mentioned one software list feature which could make possible a different clrmame approach. mess looks for softlist romsets only inside the following directory structure:
roms/systemname/
where roms/ is the rompath (where we usually store system BIOS) and systemname is either the name of the system (say gamepock, gmaster, ibmpcjr, etc.) or of its parent (e.g. snespal also looks for roms in roms/snes/). in other words, you can start e.g. gamepock software with "mess gamepock -cart reversi" only if reversi.zip is in roms/gamepock/
hence, in addition to the current approach of separate support for each gamelist (which is perfect from the point of view of a dev working on a particular software list), I was wondering if the following more user-friendly approach is somehow feasible: clrmame could be pointed to a main rom directory and, after parsing the xml to check which gamelists are supported, it could automatically generate the correspondent subfolders inside the main directory. e.g. if I point the "mess softlist profile" to use a folder called 'myroms/', clrmame could create the various subfolders there
myroms/snes/
myroms/gba/
myroms/jaguar/
etc. and point scanning and rebuilding of each list to the corresponding subfolder.
probably you will immediately find some flaws in my proposal, but I think it was worth a mention :)
-
well, the flaw is that I only added reading of the software list xmls ;) They are handled as any other datfile and you as an user have to care about how to setup rompaths.
If you now drop a snes.xml, it will show you a datfile in the profiler
name="snes" description="Nintendo SNES cartridges"
and it holds the single sets, 2020bb, 2020bbj, 96zenko etc....in this case these are 1-rom-sets.
I guess any further support beyond that is something for later releases....if at all..
-
perfectly fine with me :)
thanks for the time you spent on this!
-
just one more thing: is clrmame going to handle the parent/clone relationship with no changes to our xml format, isn't it?
I'd like next MESS full version (0.138) to be capable of creating xml lists which are ready-to-be-dropped in clrmame :)
-
yes, cloneof is being parsed and the setname which is listed there is the parent......works fine...at least with snes...which I've tested....
-
Great! I will write a small guide to the usage of MESS software lists before 0.138 comes out, with the instruction to handle them in clrmame as well.
Thanks for the support.
At a later stage, I hope we will be able to add some more handling features, but as a first implementation I think this might already work great, especially considering that the whole software list support in MESS still needs a lot of work on the emulator side :)
-
maybe I should send you a current build this evening before I release the next cmpro version....then you can recheck it a bit...
let me know your email address somehow...
-
has my PM arrived?
-
guess so....I will send you one when I'm home...
-
just wanted to be sure it worked... :)
-
sent...
-
Sorry to jump in the middle of your discussion.
Let me see if I understand all of this correctly:
1) next version of MESS will have a list of games/software supported by each of the various systems that MESS emulates
2) So basically MESS will include a 2 layers list: a "top" one which is the current list of system supported by MESS, and a "lower" one which is a ramification of the top one, where under every system there is the list of supported software for that system
3) CLRMAME will natively support such a 2 layers structure, automatically checking firstly for the systems, and secondly for the software in each and every system.
If it's the case, you guys ARE AWESOME!
I'll finally get rid of all those good, non-good, tosec, no-intro, etc.etc. lists; and only keep what is really and officially "approved" by the MESS team.
Thank you!
-
not fullly correct...at least for the cmpro part.
I've only added support to read MESS software list xml files or mess <system> -listsoftware output xmls like common datfiles.
So you can drag'ndrop/add/use such xmls in the profiler as you can do with any other dat...
...it won't auto-get all software lists from a mess binary....you will have one profile for your mess bios stuff and other profile(s) for software list(s)....
-
yeah, at the moment clrmame support is limited, but given how recent the whole feature is (and the amount of work still to be done on the MESS side [1]), I already see this as a big achievement. I plan to post more instructions and details on my blog around 0.138 release.
[1] e.g. so far only carts can use software lists, while floppies and tapes will require more work