EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: ROMs dumped from clones with matching hashes to BIOS files  (Read 10786 times)

Chad

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 19
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 66.0.3359.139 Chrome 66.0.3359.139
    • View Profile

I was wondering if you could take a look at the example below. When I run this DAT through ClrMamePro (Non-Merged Sets) it discards the ROM's in thegladpcb that have an equal hash but a different name from the BIOS ROM files. Their is no merge information so it seems like it should work. I am running 4.034 64-bit. I have a few ROMs from the current FBA (lr-fbalpha) that are doing the same thing.

When running 4.023 32-bit it does keep the equal hash files in Non-Merged mode. I don't know if I set something differently in that version or if it just worked differently back then. I spent quite a while trying to figure out if there's an option I was missing to get the current version working the same.

```
<game isbios="yes" name="pgm">
      <description>PGM (Polygame Master) System BIOS [BIOS only]</description>
      <year>1997</year>
      <manufacturer>IGS</manufacturer>
      <rom name="pgm_t01s.rom" size="2097152" crc="1a7123a0"/>
      <rom name="pgm_m01s.rom" size="2097152" crc="45ae7159"/>
      <rom name="pgm_p01s.u20" size="131072" crc="e42b166e"/>
      <rom name="pgm_p02s.u20" size="131072" crc="78c15fa2"/>
      <rom name="ddp3_bios.u37" size="524288" crc="b3cc5c8f"/>
      <rom name="bios.u42" size="131072" crc="517cf7a2"/>
   </game>
   <game name="theglad" romof="pgm">
      <description>The Gladiator - Road Of The Sword / Shen Jian (V101) [Incomplete Dump]</description>
      <year>2003</year>
      <manufacturer>IGS</manufacturer>
      <rom name="v101.u6" size="524288" crc="f799e866"/>
      <rom name="t04601.u33" size="8388608" crc="e5dab371"/>
      <rom name="a04601.u2" size="8388608" crc="d9b2e004"/>
      <rom name="a04602.u4" size="8388608" crc="14f22308"/>
      <rom name="a04603.u6" size="8388608" crc="8f621e17"/>
      <rom name="b04601.u11" size="8388608" crc="ee72bccf"/>
      <rom name="b04602.u12" size="4194304" crc="7dba9c38"/>
      <rom name="w04601.u1" size="8388608" crc="5f15ddb3"/>
      <rom name="theglad_igs027a_execute_only_area" size="392" status="nodump"/>
      <rom name="theglad_igs027a_v100_overseas.bin" size="15992" crc="02fe6f52"/>
      <rom name="v107.u26" size="2097152" crc="f7c61357"/>
      <rom name="pgm_t01s.rom" merge="pgm_t01s.rom" size="2097152" crc="1a7123a0"/>
      <rom name="pgm_m01s.rom" merge="pgm_m01s.rom" size="2097152" crc="45ae7159"/>
      <rom name="pgm_p01s.u20" merge="pgm_p01s.u20" size="131072" crc="e42b166e"/>
      <rom name="pgm_p02s.u20" merge="pgm_p02s.u20" size="131072" crc="78c15fa2"/>
      <rom name="ddp3_bios.u37" merge="ddp3_bios.u37" size="524288" crc="b3cc5c8f"/>
      <rom name="bios.u42" merge="bios.u42" size="131072" crc="517cf7a2"/>
   </game>
   <game name="thegladpcb" cloneof="theglad" romof="theglad">
      <rom name="glad_v100.43" size="524288" crc="bcf3b172"/>
      <rom name="t04601.u71" size="8388608" crc="e5dab371"/>
      <rom name="a04601.u30" size="8388608" crc="d9b2e004"/>
      <rom name="a04602.u31" size="8388608" crc="14f22308"/>
      <rom name="a04603.u32" size="8388608" crc="8f621e17"/>
      <rom name="b04601.u40" size="8388608" crc="ee72bccf"/>
      <rom name="b04602.u41" size="4194304" crc="7dba9c38"/>
      <rom name="w04601.u8" size="8388608" crc="5f15ddb3"/>
      <rom name="igs29.bin" size="2097152" crc="51acb395"/>
      <rom name="thegladpcb_igs027a_execute_only_area" size="392" status="nodump"/>
      <rom name="thegladpcb_igs027a_v100_japan.bin" size="15992" crc="d7f06e2d"/>
      <rom name="igs_v100.62" size="2097152" crc="0f3f511e"/>
      <rom name="pgm_t01s.u72" size="2097152" crc="1a7123a0"/>
      <rom name="pgm_m01s.u4" size="2097152" crc="45ae7159"/>
      <rom name="bios.42" size="131072" crc="517cf7a2"/>
   </game>
```
« Last Edit: 06 July 2018, 16:04 by Chad »
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3423
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 67.0.3396.99 Chrome 67.0.3396.99
    • View Profile

well, yes, it detects

<rom name="pgm_t01s.u72" size="2097152" crc="1a7123a0"/>
<rom name="pgm_m01s.u4" size="2097152" crc="45ae7159"/>

as bios roms (which in fact they are ;))
and the other roms as being clone roms, so in the end nothing really unique is left for the clone

MAME for example would load the clone fine since it will also check the parent.....question is what would be the "right" interpretation.

Now you can say, there is no merge information and the name differs, so don't mark it being a bios rom within the clone....
....or you say "only if there is a merge element it's a clone rom", so you would end with a copy of parent and clone since they are identical....
...etc...

when I find a little time over the weekend I will do some tests....if you got some more examples (I don't think there aren't any in MAME, are there?) let me know....
Logged

Chad

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 19
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 66.0.3359.139 Chrome 66.0.3359.139
    • View Profile

Hi Roman and thank you for the reply. Yes I see your points and they make perfect sense in how they may be interpreted in more than one way and also end up with duplicate files possibly. You're right they do run fine. I spent a few hours yesterday going through the DAT and testing it to understand what was happening.

I've seen people giving out wrong information on what's happening on other forums. These "errors" (as they call them) have been in several versions of the lr-fbalpha DAT now so the posts are far and wide. They didn't bother to view the DAT and see what's happening and just tell people to manually add the "missing" files. Maybe there's no answer here but just an awareness.

I completely became aware when I had upgraded from a really old version and extra files where being removed between the two versions. I went back to the old version for testing and it does indeed leave them in the archive in non-merged mode and removes them in current ClrMamePro in non-merged mode. Just to clarify has the processing changed between versions and it's not a setting I am missing correct?

There's 3 files are listed below.

BIOS (pgm.zip):
<rom name="pgm_t01s.rom" size="2097152" crc="1a7123a0"/>
<rom name="pgm_m01s.rom" size="2097152" crc="45ae7159"/>
<rom name="bios.u42" size="131072" crc="517cf7a2"/>

CLONE (thegladpcb.zip):
<rom name="pgm_t01s.u72" size="2097152" crc="1a7123a0"/>
<rom name="pgm_m01s.u4" size="2097152" crc="45ae7159"/>
<rom name="bios.42" size="131072" crc="517cf7a2"/>

I do have another example but it's also from the same parent/BIOS set just a different clone.

BIOS (pgm.zip):
<rom name="pgm_p02s.u20" size="131072" crc="78c15fa2"/>

CLONE (dmnfrntpcb.zip):
<rom name="pgm_p02s.u42" size="131072" crc="78c15fa2"/>

   <game isbios="yes" name="pgm">
      <description>PGM (Polygame Master) System BIOS [BIOS only]</description>
      <year>1997</year>
      <manufacturer>IGS</manufacturer>
      <rom name="pgm_t01s.rom" size="2097152" crc="1a7123a0"/>
      <rom name="pgm_m01s.rom" size="2097152" crc="45ae7159"/>
      <rom name="pgm_p01s.u20" size="131072" crc="e42b166e"/>
      <rom name="pgm_p02s.u20" size="131072" crc="78c15fa2"/>
      <rom name="ddp3_bios.u37" size="524288" crc="b3cc5c8f"/>
      <rom name="bios.u42" size="131072" crc="517cf7a2"/>
   </game>
   <game name="dmnfrnt" romof="pgm">
      <description>Demon Front (V105)</description>
      <year>2002</year>
      <manufacturer>IGS</manufacturer>
      <rom name="v105_16m.u5" size="2097152" crc="bda083bd"/>
      <rom name="t04501.u29" size="8388608" crc="900eaaac"/>
      <rom name="a04501.u3" size="8388608" crc="9741bea6"/>
      <rom name="a04502.u4" size="8388608" crc="e104f405"/>
      <rom name="a04503.u6" size="8388608" crc="bfd5cfe3"/>
      <rom name="b04501.u9" size="8388608" crc="29320b7d"/>
      <rom name="b04502.u11" size="2097152" crc="578c00e9"/>
      <rom name="w04501.u5" size="8388608" crc="3ab58137"/>
      <rom name="dmnfrnt_igs027a.bin" size="16384" status="nodump"/>
      <rom name="v105_32m.u26" size="4194304" crc="c798c2ef"/>
      <rom name="pgm_t01s.rom" merge="pgm_t01s.rom" size="2097152" crc="1a7123a0"/>
      <rom name="pgm_m01s.rom" merge="pgm_m01s.rom" size="2097152" crc="45ae7159"/>
      <rom name="pgm_p01s.u20" merge="pgm_p01s.u20" size="131072" crc="e42b166e"/>
      <rom name="pgm_p02s.u20" merge="pgm_p02s.u20" size="131072" crc="78c15fa2"/>
      <rom name="ddp3_bios.u37" merge="ddp3_bios.u37" size="524288" crc="b3cc5c8f"/>
      <rom name="bios.u42" merge="bios.u42" size="131072" crc="517cf7a2"/>
   </game>
   <game name="dmnfrntpcb" cloneof="dmnfrnt" romof="dmnfrnt">
      <rom name="demonfront_v107-u43.bin" size="2097152" crc="671d8a31"/>
      <rom name="t04501.u29" merge="t04501.u29" size="8388608" crc="900eaaac"/>
      <rom name="a04501.u3" merge="a04501.u3" size="8388608" crc="9741bea6"/>
      <rom name="a04502.u4" merge="a04502.u4" size="8388608" crc="e104f405"/>
      <rom name="a04503.u6" merge="a04503.u6" size="8388608" crc="bfd5cfe3"/>
      <rom name="b04501.u9" merge="b04501.u9" size="8388608" crc="29320b7d"/>
      <rom name="b04502.u11" merge="b04502.u11" size="2097152" crc="578c00e9"/>
      <rom name="w04501.u5" merge="w04501.u5" size="8388608" crc="3ab58137"/>
      <rom name="dmnfrnt_igs027a.bin" merge="dmnfrnt_igs027a.bin" size="16384" status="nodump"/>
      <rom name="demonfront_v107-u62.bin" size="4194304" crc="cb94772e"/>
      <rom name="pgm_t01s.rom" merge="pgm_t01s.rom" size="2097152" crc="1a7123a0"/>
      <rom name="pgm_m01s.rom" merge="pgm_m01s.rom" size="2097152" crc="45ae7159"/>
      <rom name="pgm_p02s.u42" size="131072" crc="78c15fa2"/>
   </game>

I don't know if you heard of RetroPie (using a Raspberry Pi) and the emulator is called lr-fbalpha (Libretro core) using Retroarch as the frontend. Here's the [Github page][https://github.com/libretro/fbalpha/tree/master/dats] with the DATs I used based off FBA 0.2.97.43.

I don't know of any MAME examples. I did look at 0.198 of the same games and none of these examples are valid and it looks much cleaner in there now. :)

Kind off topic but I was wondering is it possible to tell which merge files belong to the parent and which ones to the BIOS or do you need to check each "ROM" entry to really know? I always look at the Clone/Parent/BIOS unless there is none. I tried building some ROM sets out of zips with DatUtil and it works great for "flat" sets but I was trying to figure out a way to use it to build a DAT with merge information but had no luck. I suppose the program needs a way to know the parent/clone/bios relationships. Can you pull a parent and all clones (possibly the BIOS) in ClrMamePro. I know I build fix DATs of missing ROMs frequently and that works great.

Are files flagged as "no dump" or "bad dump" processed the same as regular ROMs at default settings?

I know there's a reason here I am sure but when you disable separate BIOS sets it still complains they are missing when you actually check it as an option, how come?

Anyway thank you for all you do and I often sing your praises on other forums. Oh and I always think of the Sentinel now when I visit! ;) Have a great weekend!



Logged

coccola

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 56
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Firefox 61.0 Firefox 61.0
    • View Profile

Excuse me. May I try to help?  :|

Additionally to the dat generated by FB Alpha itself, it's necessary to follow the instructions below to run the affected games in the FB Alpha emulator:

Quote
Even if FB Alpha v0.2.97.43 (ClrMame Pro XML).dat says the following files are not needed for dmnfrntpcb.zip, svgpcb.zip and thegladpcb.zip, you need to add pgm_m01s.rom and pgm_t01s.rom manually to dmnfrntpcb.zip and svgpcb.zip plus bios.u42 to thegladpcb.zip to make these three sets as available and to make them run. You can find all the needed files to be added manually in the archive "pgm.zip".

It looks like this workaround is related to the problem described by Chad.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3423
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 67.0.3396.99 Chrome 67.0.3396.99
    • View Profile

Weekend time...and not much time to read through / answer everything now..

so... yes, I will rethink the "merge" element rules since at least in this example there is a break in logic regarding the bios ones. Normally cmpro detects clone and bios files by name + checksum compare. Meaning that different names but equal hashes within a parent/clone relationships mean 'different files, do not merge'. Then you got the profiler->parse rom/disk merge tags', when enabled, cmpro uses the given merge name for the name + checksum check (so it's a bit less strict then)

Keeping this in mind, the 2 bios files in your example should actually not be removed from the clone since they don't have a merge element and the names differ. Guess it is merged because of some very old routines for bios files which don't care about names but only look for identical hashes..

...as I said, I will double check this...maybe I think about a stricter merge element mode which would mean that only files with a merge element are considered being a clone......time will tell...

That somehow answers your question which files of a cloneset belong to the parent....the ones which don't belong to the bios (and the ones which don't belong to device sets) and the ones which are identical (hash/size) to the ones listed in the parent

Logged

Chad

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 19
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 66.0.3359.139 Chrome 66.0.3359.139
    • View Profile

@coccola - Thank you for the information I appreciate it. I do find that information on different forums frequently. That's what I was hoping to avoid, needing to manually process files and let ClrMamePro do it's thing. Technically, depending on the emulator, they wouldn't need to be added as stated in the paragraph to work even. If the PGM BIOS is located in the ROMs folder or "search path" it will work. Basically it will find the ROMs in the parent or BIOS. I think Roman's last post was more the direction I was hoping for, basically looking at how the DAT/ROMs are processed.

@Roman - I am glad there's others that take the weekend off too. When I was young taking time away from a computer was unheard of and now it's the opposite. Being older I actually spend my weekends with family/friends, doing "chores", honey-do-lists, etc. Basically spend time away from a PC. Well at least most weekends unless it's old school gaming in a group setting always seems to be a hit at social/family gatherings! ;)
Logged

coccola

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 56
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Firefox 61.0 Firefox 61.0
    • View Profile

My intention was not to tell you a way of running the games. This forum maintained by Roman, the author of clrmamepro, so this is of course the place to report bugs and ask technical questions. I mainly posted that so that Roman has extra information on the problem and workarounds, and to add that svgpcb.zip is affected too.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3423
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 67.0.3396.99 Chrome 67.0.3396.99
    • View Profile

ok....I've made a change....and thegladpcb would contain 14 (+1 nodump) files afterwards.....similar for the other.

The new rule is something like:
if the datfile/exeimport uses merge tags....and parse merge tag usage is enabled in profile....then a rom is only a bios or a parent rom if it has a merge tag (same for disks)

For current MAME it won't change anything.
For software lists it won't change anything, since they don't use merge tags at all, so the standard cmpro hash compare will try to match parent relationships....now you can start a discussion if this is useful or not...this is done in cmpro since 1997 (or let's say 1998 when MAME added merging)... ;-) and that's looong ago before MAME actually added the merge elements...

Will do some more tests before a new nightly will be out...

Logged

Chad

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 19
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 66.0.3359.139 Chrome 66.0.3359.139
    • View Profile

@coccola - Sorry it might be miscommunication here. The games work fine and nothing is broken. I think Roman knew that from the beginning, as he pointed out, that they should run fine and he was spot on.

Really the whole paragraph is incorrect. The games do not need the files to be present in the clone to work as stated. They will look to the parent/BIOS and use hash matching. Basically you don't even need them to be named correctly and name them anything you want and it will still work. I guess I was just pointing on how they are parsed and it's not really a bug, well maybe, depending on how you interpret the DAT information.

Yes sir I definitely know who Roman is and I do sing his praises and love his responses. Cool, calm and collect. Maybe he has to pistol whip the radical purveyor occasionally. I've been at this desk for 22 years and I believe he has been developing ClrMamePro for 20 or 21 years, I think something like that. I read a lot more than I post here. Anyway I have no doubts I couldn't even remotely match his institutional knowledge of MAME nor dictate his autonomy over ClrMamePro. I happened upon the issue by mere chance and decided to share it as I always have found Roman to be a reasonable well written man. And...viola, look at us now with an awesome update on the horizon to quiet the masses about these few, but nonetheless monumental, games! ;)

Anyway, joking aside, I appreciate it and I am definitely not trying to rub you the wrong way by any means so I'll let that go now.

@Roman - Thank you for the update. I think it sounds perfect and look forward to the release. Is there a difference between a merge tag and element?

Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3423
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 67.0.3396.99 Chrome 67.0.3396.99
    • View Profile

no difference....actually it's an xml attribute "merge"....no idea why I call it merge element/tag....

So from now on: MERGE attribute.....at the rom/disk element
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3423
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 67.0.3396.99 Chrome 67.0.3396.99
    • View Profile
Re: ROMs dumped from clones with matching hashes to BIOS files
« Reply #10 on: 11 July 2018, 19:11 »

new nightly out...
Logged

Chad

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 19
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 66.0.3359.139 Chrome 66.0.3359.139
    • View Profile
Re: ROMs dumped from clones with matching hashes to BIOS files
« Reply #11 on: 11 July 2018, 22:55 »

Thank you Roman for the update and answering the questions! I appreciate it and have a wonderful evening. :)
Logged
Pages: [1]   Go Up
 

Page created in 0.12 seconds with 21 queries.