EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: [BUG] clrmamepro-generated 1G1R ROM sets missing ROMs or selecting wrong region  (Read 12451 times)

andrebrait

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 4
  • Operating System:
  • Linux Linux
  • Browser:
  • Chrome 79.0.3945.136 Chrome 79.0.3945.136
    • View Profile

Hi there,

I have tried to generate a 1G1R set for the Nintendo DS using DAT-o-matic's Parent/Clone XML (with default parent data) and I observed a few issues with it.

ROMs like "Mario Kart DS" would be simply missing, despite being available for all regions I selected (USA and EUR, in that order), among others. I see the full data for them in the DAT, including region data, but the ROM file is simply never picked up for selection.

For GBA, also, a number of Animal Crossing titles were missing, as well as a few other ones here and there.

Again, I checked the DAT and all the Parent/Clone data was there for those games.

I'll generate a better bug report with more detailed information, as all that happened a few weeks ago.

I created a small Python application that generates what the ROM selection is supposed to be like, from a Parent/Clone DAT, as a means of validating the selection clrmamepro makes (as well as creating my own tool for that because I wanted the sets immediately back then, instead of reporting the bug and letting someone fix it, wait, etc )

You can find it here: https://github.com/andrebrait/1g1r-romset-generator
« Last Edit: 26 January 2020, 10:00 by andrebrait »
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3443
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 79.0.3945.130 Chrome 79.0.3945.130
    • View Profile

The 1G1R algorithm hasn't changed since 2009.

If you face issues with the selection it's usually simply based on the used datfile or a misunderstanding what 1G1R actually does.
So if you're sure that Mario Kart DS should be selected, send the datfile and your settings and I'm pretty sure I can tell you what is wrong on your side or in the dat.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3443
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 79.0.3945.130 Chrome 79.0.3945.130
    • View Profile

So I made a quick test....downloaded the Nintendo - Nintendo DS (Decrypted) (Parent-Clone) (20200126-032320).dat with that DAT-o-matic, loaded it in cmpro, enabled the 1G1R option, enabled EUR as region and I get Mario Kart DS (Europe) (En,Fr,De,Es,It) listed (see screenshot)
When I select region USA I get the mario usa set with checksum eb26155d...

So everything works fine here and I don't see any bug.

As mentioned before, Logiqx' 1G1R algorithm (which uses a scoring algorithm) is unchanged for over a decade and the results highly depend on a) your enabled regions/languages, plus b) the order of the enabled regions/languages, c) the original datfile and its "release" elements....and sometimes users simply have a totally different understanding what 1G1R is all about (how the setnaming and set selection works).

In your Python code I don't see Logiqx' algorithm implemented but surely you can point me to it in your code....
I guess part of Logiqx' code can be found on old forum posts like https://forum.no-intro.org/viewtopic.php?f=2&t=544
« Last Edit: 26 January 2020, 20:56 by Roman »
Logged

andrebrait

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 4
  • Operating System:
  • Linux Linux
  • Browser:
  • Chrome 79.0.3945.130 Chrome 79.0.3945.130
    • View Profile

So I made a quick test....downloaded the Nintendo - Nintendo DS (Decrypted) (Parent-Clone) (20200126-032320).dat with that DAT-o-matic, loaded it in cmpro, enabled the 1G1R option, enabled EUR as region and I get Mario Kart DS (Europe) (En,Fr,De,Es,It) listed (see screenshot)
When I select region USA I get the mario usa set with checksum eb26155d...

So everything works fine here and I don't see any bug.

As mentioned before, Logiqx' 1G1R algorithm (which uses a scoring algorithm) is unchanged for over a decade and the results highly depend on a) your enabled regions/languages, plus b) the order of the enabled regions/languages, c) the original datfile and its "release" elements....and sometimes users simply have a totally different understanding what 1G1R is all about (how the setnaming and set selection works).

In your Python code I don't see Logiqx' algorithm implemented but surely you can point me to it in your code....
I guess part of Logiqx' code can be found on old forum posts like https://forum.no-intro.org/viewtopic.php?f=2&t=544

I'll dedicate some more time to this, but meanwhile, what I did was to follow the procedures from here:
https://forums.launchbox-app.com/topic/40303-rom-filtering-guide-for-clrmamepro-filter-all-usa-games-plus-all-exclusives-from-all-regions/

And I wasn't the only user who reported it excluding some games. Well, maybe the problem is in the steps in this link, then? I tried a number of other things but I failed to make it produce a set that included Mario Kart DS at all.

And I didn't implement his algorithm. I made my own (basically, it's also a scoring system and sorting a list of candidates based on it).
I'll check his algorithm and see where they differ, too.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3443
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 79.0.3945.130 Chrome 79.0.3945.130
    • View Profile

I've checked that 'tutorial' and I don't see a problem with it. I used their description on the Nintendo datfile (used their region ordering USA/CAN/AUS/EUR....) and I still get Mario Kart listed. With the USA regio as highest prio, the algorithm picks Mario Kart DS (USA) (Demo) (Kiosk).nds / eb26155d as 'the chosen' set for that parent/clone relationship.

If people say "may game xxx is not appearing at all" then normally 3 cases might happen:
1) the game belongs to a region which is not enabled and so the set is filtered out
2) due to the scoring mechanism and the arrangement in the dat, a different clone/parent is picked as you might expect (see above Mario Kart DS example, I personally would have picked the "Mario Kart DS (USA)" set and not the "Mario Kart DS (USA) (Demo) (Kiosk)"....but of course scoring algorithms aren't perfect and depending on the datfile it picks what scores best and not what you think is the best guess ;-)...in such a case you might better filter out demo/kiosk versions before.
3) your xxx clone is actually used but gets a different naming due to the definitions in the datfile

So again....the scoring algorithm was designed by Logiqx and is used in several rom managing tools and runs for a decade now....every now and then users come up and are not satisfied with what the algorithm picks...but that's the way it was designed. Surely you can write your own routine...but you cannot compare it to what is used in various rom managing tools.
« Last Edit: 27 January 2020, 07:08 by Roman »
Logged

andrebrait

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 4
  • Operating System:
  • Linux Linux
  • Browser:
  • Chrome 79.0.3945.136 Chrome 79.0.3945.136
    • View Profile

Got it. Funny, it probably shouldn't be picking a demo ROM in this case (as there's a retail one available). I'll look into the algorithm. I also think DAT-o-matic's DATs might be to blame here, as they omit a lot of information in the XML (like language, for example) and maybe they're also to blame for the algorithm picking up a demo ROM. That could explain why it's missing from my output set: I don't have Demos, Samples, Prototypes, etc.

I'll look closer into the scoring system to find out why it's picking up that demo instead of a retail ROM.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3443
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 79.0.3945.130 Chrome 79.0.3945.130
    • View Profile

I haven't looked into the actual score but I could imagine that both USA sets (the demo and the normal one) might get the same score value and then maybe the last one is prefered for no real reason. I guess the algorithm was more for a scenario where you only got distinct regions within one parent/clone relationship.
Logged

andrebrait

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 4
  • Operating System:
  • Linux Linux
  • Browser:
  • Chrome 80.0.3987.132 Chrome 80.0.3987.132
    • View Profile

I haven't looked into the actual score but I could imagine that both USA sets (the demo and the normal one) might get the same score value and then maybe the last one is prefered for no real reason. I guess the algorithm was more for a scenario where you only got distinct regions within one parent/clone relationship.

Well, I think my own algorithm has reached a point where I can say it's able to achieve a pretty good 1G1R set, as long as the No-Intro naming convention is used.

I have refined it with the help of some users and I think it's safe to say it works.

At the moment, it's only present as the Python script I linked in the original post for this thread.
Here's some tutorial of usage, if you're interested: https://www.reddit.com/r/RetroPie/comments/f5b2gb/how_to_make_1g1r_rom_sets_using_nointro_dats/
(Yes, the title is a bit provocative on purpose  ;D)

Of course, generating 1G1R sets is all it does, but it may be worth trying it and seeing if you think something like this could eventually be implemented in ClrMamePro itself.
« Last Edit: 08 March 2020, 21:12 by andrebrait »
Logged

joaoluiz

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 8
  • Operating System:
  • Linux (Ubuntu) Linux (Ubuntu)
  • Browser:
  • Chrome 80.0.3987.87 Chrome 80.0.3987.87
    • View Profile

I have a related question. I have generated no-intro p/c dats and used them to filter my sets to 1G1R. However, I did not check the "Default parent if there is no P/C data" option when I generated those dat files. My understanding is that, if the p/c info is missing for a rom, then the dat file will not say anything about it being a parent or a clone. So what does ClrMamePro's 1G1R mode do if the p/c clone info is missing for a rom? Does it treat that rom as a parent?
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3443
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 80.0.3987.132 Chrome 80.0.3987.132
    • View Profile

A set is always a parent unless it's a clone ;-)
« Last Edit: 15 March 2020, 17:27 by Roman »
Logged

joaoluiz

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 8
  • Operating System:
  • Linux (Ubuntu) Linux (Ubuntu)
  • Browser:
  • Chrome 80.0.3987.87 Chrome 80.0.3987.87
    • View Profile

Perfect! And, phew!!!  ;D

Thank you!
Logged
Pages: [1]   Go Up
 

Page created in 0.062 seconds with 21 queries.