EMULAB Forum

clrmamepro [English] => clrmame Discussion => Topic started by: Balteck on 05 June 2015, 19:56

Title: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 05 June 2015, 19:56
I found this the bug while scanning with CLRMAME the Mame 0.162 sets stored in 2 paths (./roms for arcade and arcade CHDs, ./softlist with a lot of subfolders in it)
The DAT is created within CMP using MAME64.exe 0.162 with softlist option.
I'm using a full merged set (Option:"Hash Collision Name" and "forbid merge different systems").

The problem is that CMP tries to merge some Arcade CHDs with a softlist CHDs or a softlist CHD with another softlist CHD. For instance:

It wants to have Area51 CHDs all in the saturn softlist, but in the same time it gives me error because the sha1 doesn't match

or

It wants to merge shinsets PSX game with Shinsets pcecd game together with Hidsouls saturn game that is the parent of the shinsets saturn game

Another problem is that the scan window result gives me as missing sets any clone of softlist CHD games, but doesn't give me missing CHD (because I have them in the right parent folder)

And the last bug is about Tower of Cabin, that the rebuilder creates in the right way (a 7zip with the parent name and inside the 2 clones), but the scan Window result gives me error like if I miss them

I hope I explained in a good way and I attach the scan log to be more clear

https://mega.co.nz/#!qEVQQDYA!SpTmNdizD2WObmOpD0ybaRXK-NzU09BuHUwyNKszyrk


Thank you
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 06 June 2015, 08:13
I'm on vacation....guess you have to wait at least a week for a detailed answer....

when you use software lists directly in combination with mame, you need to setup system default paths (within the systems dialog)....and the folders need to be unique....so a merge between different lists should actually not happen....of course there could be a bug in system assignment somewhere but for thiis I need to be back home....

it would be intersting to see your system dialog set and rompaths setup....there should be one rompath per list plus the one for mame..
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 06 June 2015, 10:59
I've Always used CMP with MESS and I hadn't any issue till now.
The system default path is roms folder for standard, mechanical, device and all BIOS and one unique folder for each softlist (psx for psx, saturn, for saturn, pcecd for PC-Engine CD, ecc), otherwise CMP gives me a warning that the same folder is used on more softlists.
In the PATH settings I have \roms and every softlist\folder (ex: softlist\32x, softlist\a800 and so on).

I can wait, no problem. I'm only hoping that CMP didn't merge roms and CHD in wrong way, moving a lot of files around the folders; but I've always activated the backup option and infact I found a lot of CHDs in backup and I think it will be easy to restore them in the right way.

Have a good vacancy!

Thank you very much
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 06 June 2015, 11:59
well...good to hear that the setup seems to be ok...as mentioned,,,there could be a problem with software list assignment...some lists even share identical files...but for deeper checking I need to do some debugging,..pretty hard when the sourcecode is some hundred kilometers away.... :-)
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 16 June 2015, 18:39
well...yes..there is an issue with chd detection when software lists and non-software lists share the same setname and got chds in them...(e.g. area51 known from MAME and area51 from saturn software list)....
Fixing this will take a while...I got it on my list...
In the meantime, you should use separate profiles to scan them...then you don't run into any trouble....
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 23 July 2015, 16:37
Hello!. I've tried the new version and all issues are disappeared!

Well done!
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 23 July 2015, 16:44
thanks for retesting!
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: AMMalena on 13 September 2015, 02:18
Would someone mind sharing with me how they're using MAME now with the MAME/MESS merger?

I'd hate to have to keep two copies of MAME (two directories) if it can co-exist with MESS ROM BIOSs and Software... but honestly, I can't even find one reliable source of what the MESS folder structure is supposed to be now that it's in MAME.  I have one Italian site that is somewhat helpful.

Thanks.
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: f205v on 13 September 2015, 08:45
Nothing has changed in respect to MAME.
Let's assume that the MAME.EXE executable is in D:\MAME
You will need to put your ROMs (former MAME roms + former MESS bios) into D:\MAME\ROMS
Your samples (former MAME samples + former MESS samples) into D:\MAME\SAMPLES
Your CHDs can stay in the ROMS subdir, each of them in its own subdir; or alternatively you can group all of them into D:\MAME\CHDS, and add this path to mame.ini
Regarding software: put them into D:\MAME\SOFTWARE each system in its own subdir.

This should be enough to make it run correctly.
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 13 September 2015, 19:40
BIOS-wise it's easy..well..more or less...the MESS BIOS will simply come with the MAME binary output....the only problem is you can't really separate them (e.g. like neogeo or others). There is no good indicator within the MAME listxml output which can identify a former MESS one...

Another story are software lists...actually...you may want to use them as separate profiles.
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: AMMalena on 13 September 2015, 22:16
So apparently there will be no conflicts then.... pretty awesome.
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 17 September 2016, 18:33
Hi Roman, now the same bug re-appears.

I did a rom scan on last mame and I found Area51 CHDs deleted from ROM folder and softlist folders.
Again CLRMame Pro wants to merge Arcade + Softlist like a version of last year.

May you correct it again?

Thanks
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 18 September 2016, 11:25
Hmm....interesting. I know that they recently renamed the area51 software list chd since it causes conflicts..
http://git.redump.net/mame/commit/?id=cd2e853167cd2325a588543cb265eaca6c8d385b (http://git.redump.net/mame/commit/?id=cd2e853167cd2325a588543cb265eaca6c8d385b)

so...which MAME version are you using at the moment? Official .177?
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: oddi on 18 September 2016, 12:12
@Baltec which chd it's removed, post sha1 here.
I not have not problem with this chds, have problem when "re-tag" clone and cmpro try "move" chds to right folder ( if cmpro move chd to right place not removed empty chd folder )
I used full merge mode ,  my structure it's:

Mame CHDS:

area51\
area51.chd
area51t.chd


area51mx\
area51mx.chd

------------------------

Saturn SL sets:

area51e\
area 51 (t-18613g).chd
area51e.chd
sat-0721-area_51-usa.chd


That is
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 18 September 2016, 19:14
Ah...I think I might have broken it with a recent fix....

try this...

http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160918.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160918.rar)

and let me know if it works for you.
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 19 September 2016, 08:40
Hmm....interesting. I know that they recently renamed the area51 software list chd since it causes conflicts..
http://git.redump.net/mame/commit/?id=cd2e853167cd2325a588543cb265eaca6c8d385b (http://git.redump.net/mame/commit/?id=cd2e853167cd2325a588543cb265eaca6c8d385b)

so...which MAME version are you using at the moment? Official .177?

I'm using official .177.
My folders are roms\area51 and softlist\saturn\area51 (not e because I'm running clrmame pro on previous set)

clrmame wants to "merge" arcade with saturn instead of rename saturn set as area51e

Now I will try your testing version (I hope is for me, not for @oddi)
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 19 September 2016, 08:47
yes..it's for you..please test it...

I'm pretty sure that the recent 4.030 fix "unneeded chds are not showing up if nothing of the set exists" broke it.
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: oddi on 19 September 2016, 12:16
Quote
clrmame wants to "merge" arcade with saturn instead of rename saturn set as area51e

hmmm, strange behavior. Hope new cmp forum build fix that.

@Baltek: What mode u used - Split, full Merge mode ?
and your cmp settings
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 19 September 2016, 12:25
Oddi....

pretty simple to repeat....
full merged sets, combined MAME with softwarelists (psx, saturn) import, sets: area51*.
As soon as it finds the first area51 folder it can get confused and moans about a standard area51 chd that it should be moved to saturn software list assigned folder....(or similar...depending on what it finds first)

But yes, the latest build should fix it.
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: oddi on 19 September 2016, 13:05
Ahhhh, if used import MAME+SL - have this problem, understand.
I used separated xml from github.
Tnx for explanation.
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 19 September 2016, 13:08
I tried it and I don't have anymore error on AREA51.

But:

1) clrmame doesn't rename area51 saturn set in area51e, so I have arcade CHD in roms\area51 and saturn CHD in softlist\saturn\area51. Mame starts correctly the 3 versions of saturn chds (jap, USA and Euro)

2) I had parent/clone relationship on PSX softlist:
      I put "marvel super heroes vs. street fighter - ex edition (japan) [slps-01915].chd" in root path and with scanner
      at first-pass clrmame moves it in the right folder (mshvsf).
      With scanner second-pass clrmame wants to rename mshvsf in mshvsfj, creating a split set
      then it deletes mshvsfj because it is and unknown set (???). The same problem is with kof95/kof95j, kof99/kof99j
      and many other that I don't have the parent, but only the clone.

A similar problem is present also in genesis set (arcade):
The parent doesn't exist, but it exists only the clone (TSS Chip).
Clrmame tells that I'm missing the rom, but I have the zip with inside a folder "genesis_tmss" and clone rom. I'm using Full Merge Mode: Hash Collision Name

Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 19 September 2016, 13:36
Good to hear the problem is gone...


I will look at the other issues later at home
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 19 September 2016, 20:49
1) well...why should it rename it? The name change comes with the next MAME version
2) yeah...tricky thing...fixed with the next nightly build ( http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160919.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160919.rar) )
3) genesis...well...if you use full merged sets, you need all clone files merged into the parent. So you don't need "genesis_tmss" but its files in "genesis".
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 20 September 2016, 07:14
1) well...why should it rename it? The name change comes with the next MAME version

Oddi's answer confused me. I've understood that Area51e was already in mame 0.177. So cmpro20160918 did well

Quote
2) yeah...tricky thing...fixed with the next nightly build ( http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160919.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160919.rar) )

I'll try it

Quote
3) genesis...well...if you use full merged sets, you need all clone files merged into the parent. So you don't need "genesis_tmss" but its files in "genesis".

Genesis have two set: a parent without any rom and a clone with a rom "tmss_usa.bin". cmpro builds a zip in this way:

<ziproot>\
<ziproot>\genesis_tmss\
<ziproot>\genesis_tmss\tmss_usa.bin

For Mame is ok. But for cmpro scanner the parent set and the clone set is labeled missing
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 20 September 2016, 07:30
1) Don't listen to Oddi ;-) He's always using nightly MAME builds...

2) yes please

3) Erm...looks ok to me. Genesis is the parent set (which got no roms...fine...not very common but possible), "genesis_tmsss" is the clone. Again, if you use full merged sets, all files (parent and clones) are stored in the folder/archive named after the parent (even if that parent doesn't have a rom).
So they belong to "genesis" and since you're using "Full Merge Mode: Hash Collision Name", cmpro will store clones in subfolders within the parent.

So

<ziproot>\
<ziproot>\genesis_tmss\
<ziproot>\genesis_tmss\tmss_usa.bin

looks fine to me....cmpro should not moan about it (but I will double check this tonight)
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: oddi on 20 September 2016, 09:28
1) Don't listen to Oddi ;-) He's always using nightly MAME builds...

absolutly right !

cover o.'s my head with ashes
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 20 September 2016, 11:41
I've tested the new build and I can confirm that clone/relationship issues are gone:
 no more genesis alert and no more move from mshvsf to mshvsfj and viceversa.

Now, after the good news, there is a bad news. I discovered a strange behavior on 3 scenarios:

1) Gakuen King CHD is present is softlist\pc98_cd\gakuking and softlist\fmtowns_cd\gakuking. cmpro shows
    wrong SysDefPath and move CHD from pc98_cd to fmtowns_cd appending a number al last of the file.
    So I have two indentical CHDs (one with right name, one with collision name) in fmtowns_cd.
    After that cmpro wants to delete the one with collision name. I did it
    Then cmpro wants to move the right name CHD in softlist\pc98_cd\gakuking and the wheel restarts...
    Both softlists share the same CD, because Gakue King is a Hybrid FM Towns / PC98 title.

2) I have softlist\genius\englisha.7z and softlist\gl2000\englisha.7z. They are different sets with same name, but
    different rom inside; cmpro shows wrong SysDefPath and wants to move softlist\genius\englisha.7z to
    softlist\gl2000\englisha.7z.
    I answered yes, but cmpro did nothing. Every time I did a scan it wants to move in right place.
    But it is wrong: this two sets are in right place with the right roms inside.
2a) Same behavior with softlist\tandy6k\xenix30d.7z and softlist\trs80m2\xenix30d.7z.
      This time is the same named set and same named roms inside (so they are shared like Gakuen King)
2b) Same behaviour with softlist\lisa\xenixos.7z and softlist\lisa2\xenixos.7z.
      This time they are two different sets with same name and inside roms this same name, but different crcs.
 
3) I have some sets that are removed or renamed from the previous version of Mame. For instance:
   
    -  softlist\msx1_cart\happyfrt.7z is unneeded, but cmpro shows wrong SysDefPath and wants to move in
       softlist\msx1_flop. I've already in softlist\msx1_flop the right happyfrt.7z, so after telling yes to move in correct
       path, cmpro did nothing (no move, no delete, no add rom in destination archive).
       I was expecting to mark it unneeded and move it on backup folder.
    -  softlist\apple2\alice.7z is renamed in alcwndrl.7z, but cmpro shows wrong SysDefPath and wants to move in
       softlist\cdi. I've in softlist\cdi a folder alice with a CHD, so after telling yes to move in correct
       path, cmpro moved alice.7z in softlist\cdi. A second pass with scanner cmpro deleted this file, because there are
       apple2 roms inside and not a CDI set.
       I was expecting to rename it and not to move in another place and then move on backup folder. Finally a rebuild
       from backup folder it create the right named set.
    -  softlist\psx\burningr\burning road (japan) [slps-00518].chd is renamed in softlist\psx\broad\burning road (japan) [slps-00518].chd.
       cmpro shows wrong SysDefPath and wants to move in softlist\saturn. I've in softlist\saturn a folder burningr with
       parent and clone CHDs of Saturn Burning Road, so after telling yes to move in correct path, cmpro moved
       burning road (japan) [slps-00518].chd in softlist\saturn\burningr. A second pass with scanner cmpro deleted
       this file (obviously) from the softlist\saturn\burningr folder.
       I was expecting to rename it and not to move in another place and then move on backup folder.
       This time rebuilder cannot did the magic from backup folder and if I don't know that this CHD is legit, I lost it
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 20 September 2016, 12:03
to sum it up.....so all 1 to 3 are all about wrong "wrong SysDefPath" messages?
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 20 September 2016, 12:14
2) I have softlist\genius\englisha.7z and softlist\gl2000\englisha.7z. They are different sets with same name, but
    different rom inside; cmpro shows wrong SysDefPath and wants to move softlist\genius\englisha.7z to
    softlist\gl2000\englisha.7z.
    I answered yes, but cmpro did nothing. Every time I did a scan it wants to move in right place.
    But it is wrong: this two sets are in right place with the right roms inside.

I've tried to delete englisha from the both path and start a scan.
Incredibly cmpro shows that only gl2000\englisha.7z is missing.
So I started a rebuild and it creates only englisha.7z in gl2000 (removing source file), but not englisha.7z in genius (leaving the file in backup).
I've started a new scan and for cmpro is all ok. But I've not englisha.7z in Genius, so my softlist is no complete even if cmpro tells me no missing roms...
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 20 September 2016, 12:15
to sum it up.....so all 1 to 3 are all about wrong "wrong SysDefPath" messages?

yes. I think it is the same problem also if different scenarios.
But I did a big mistake (maybe I'm becoming too old... :-[):

cmpro don't load all the xml present in the hash dir of mame, but it leaves somes (lisa2, tandy6k, genius, advantage, amigaaga, amigaocs, amigaecs, pippin_flop and maybe others) that were in my settings rom path.
So cmpro saw these folders, but they are not associated to any system in scanner or rebuilder.

Removing these folder containg softlist sets (by hash folder), but not softlist sets (by mame) the point 2. issue disappears.

Remain the point 1 and 3 and I think that it is strange that cmpro doesn't showed me all files in these folders as unneeded...
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 20 September 2016, 12:20
well if it did not generate or scan genius software lists, the set or the software list might be disabled.....

but let me first check the wrong "wrong sysdefpath" thingie...
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 20 September 2016, 13:33
cmpro doesn't load the hash files at all ;-)

it uses what MAME prints out when you run it with -listsoftware (or whatever the switch was called). And that output does not necessarily match what you get when you use the hash xmls directly....(ok..do some fingerpointing to MAMEDEVs here ;-)) but to be fair...haven't double checked that for years.

"cmpro doesn't showed me all files in these folders as unneeded" .... another thing (beside the wrong sysdefpath thingie) I will look into....


Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 20 September 2016, 13:45
Mame uses xmls in hash folder.

If you empty the hash folder, the output of -listsoftware is 0
With hash folder full of xmls, the output of -listsoftware is a big dat contains all xmls data

But, probably, in Mame executable there is a flag that loads only some xmls from the hash folder, not all with the option -listsoftware.

I've always used this option to manage softlist, but I had a big HDD crash and I found a softlist collection made directly from xmls and not with -listsoftware. That's why I have these (for now) unneeded folders

Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 20 September 2016, 13:51
MAME's standard -listxml output only contains placeholders for software lists...from this list cmpro builds the selectable list when you want to add software lists.
If you single-select some there, cmpro will call MAME with single requests for the lists...which can even return empty lists for some (at least that was the case in the past). If you select all it will import all by using -listsoftware and not by sequentially calling single ones)...which leads to something else....

As I said...haven't tried that and compared the output for ages...but back in the past it was a mess ;-)
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 20 September 2016, 21:00
here we go again:
( http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160920.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160920.rar) )

this should fix the bad wrong sysdefpath thing.....which again was a bit tricky...since it appeared for 100% identical sets which are shared around various software lists.....so cmpro got a bit confused when it found a fully valid Gakuen King CHD for software list A in an assigned folder for software list B....tsk tsk tsk...fixed now.

Regarding the "I got additional rompaths which are not assigned to a software list" issue...well...as long as such paths contain sets which appear in the currently loaded database, they won't be listed as unneeded, no matter if they are assigned to an active software list/etc or not. Any file which doesn't represent a set should be listed as unneeded though.
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 21 September 2016, 09:01
here we go again:
( http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160920.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160920.rar) )

this should fix the bad wrong sysdefpath thing.....which again was a bit tricky...since it appeared for 100% identical sets which are shared around various software lists.....so cmpro got a bit confused when it found a fully valid Gakuen King CHD for software list A in an assigned folder for software list B....tsk tsk tsk...fixed now.

I confirm: no more Gakuen King CHD issue if both chd are present.
If one of two is present (not important if it is fmtowns_cd or pc98_cd) cmpro shows no missing (but it is wrong)
If both are not present, cmpro shows that I'm missing both CHD (that's is correct).
It seems that is one set is good, because it shares the same file, cmpro doesn't check if the file is present also in the other path.

Quote
Regarding the "I got additional rompaths which are not assigned to a software list" issue...well...as long as such paths contain sets which appear in the currently loaded database, they won't be listed as unneeded, no matter if they are assigned to an active software list/etc or not. Any file which doesn't represent a set should be listed as unneeded though.

I confirm that cmpro works as expected:
it shows unneeded the unknown file name and shows same set in different path if the file name has the same name of a right set in any softlist folder.
It was my fault to have added a wrong search path.

Last, but not least the point 3 issue remains. I've attached two roms to better explain.
The first happytr.7z, that contains a .rom file, is located in softlist\msx1_cart and it needs to be deleted, because it was present in a previous mame version, but not in .177
the second happytr.7z, that contains two .cas files, is located in softlist\msx1_cass and it is in the right place.
cmpro, during scan, when it reachs the first file, wants to move in correct path (from softlist\msx1_cart to softlist\msx1_cass) and if i press yes, it does nothing (no move, no delete, no add .rom file in the archive with .cas files)


Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 21 September 2016, 09:29
you keep me busy...but thanks for testing...that's highly appreciated.

"If one of two is present (not important if it is fmtowns_cd or pc98_cd) cmpro shows no missing (but it is wrong)"  ...hmmm....that should not happen....grrr..have to look at this again...I got an idea what might caused this (an old annoying optimization which hits me hard again and again and again...)

...and issue 3)....I will check that...
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 21 September 2016, 20:14
ok....here we go....since the main page is currently down, I've attached the latest build.

issue 3 part 1: fixed. Another bad "wrong sysdefpath" issue...sigh... also fixed the Gakuen King problem that it does not see the missing set when you only deleted 1 out of 2 identical ones...so 2 fixes for tonight...

issue 3 part 2: you wonder why happyfrt cart set is not listed as unneeded. Well, the reason for this is the following:
- during the unneeded sets check it only checks the setname if it appears in the setlist or not. happyfrt is a valid name, so it's not listed as an unneeded set. Let's assume you fake a set (use a valid setname and as roms you got some random files), then the rom unneeded check will identify such files and lists them as unneeded. So you will see them.
In your case however, the rom check won't even touch the set since in reality it does not exist for that software list....
So yes....in this weird case such sets won't appear as unneeded.
I will double check if I will update the set-unnneded check in such cases a bit...but for now it's a normal (even if weird) behaviour. (but I guess I will update that little point as well...)
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 22 September 2016, 10:28
I tried the new version.

Now if I have both gakuen king.chd cmpro shows no errors.
If I delete the CHD of fmtowns_cd, cmpro shows missing set (but not missing CHD in set) for fmtowns_cd
If I delete both CHDs, cmpro shows missing set and missing CHD for both sets

For the issue of renamed/unneed sets with a name of a new set, cmpro shows nothing. No error, no missing, no wrong path, etc. Like if happytr.7z on msx1_cart was deleted (but it is always there in msx1_cart folder). So it was better before, that I could check any wrong path by hand and I could choose what to do and not having dead files around.

As I understood the logic of cmpro, I will ask if it possible to change the behavior in this way:

Now cmpro scans for files, it finds happytr.7z (so it assumes that is the set "happytr"), looks for this set and finds that it is in msx1_cass (not msx1_cart where it found it). Tries to move in right place (in this specific case, it fails because it finds a file named happytr.7z in right place, instead of merging two archives together having an happytr.7z contains 2. cas and 1.rom files, like cmpro does with CHDs, it skips to move the 7z file) and in a second scan discovers that the 7z contains a rom unneeded for this set. It deletes it and if it is a rom of another set (because mamedev simply renamed the set and used the same name for another set in another soflist), cmpro rebuilder does the magic.

My idea is that scans for files, it finds happytr.7z (so it assumes that is the set "happytr"), looks for this set and finds that it should be in msx1_cass (not msx1_cart where it found it). Instead to move it in right place, scans inside this set and discovers that:
1) is a rom of another set of same softlist (ie: softlist\apple2\alice.7z that was renamed in softlist\apple2\alcwndrl.7z)
    and it renames the 7z or add the rom to existent alcwndrl.7z file because it a old-parent rom become clone rom
2) is a rom not present in any set of any softlist and it marks it unneeded
3) is a rom that really misplaced and it moves it in the right place.

In this way I think that it will be solved the same issue with CHD. For example burning fight (japan) [slps-00518].chd, in previous mame it was a parent rom of set softlist\psx\burningr. Now is a clone rom of set softlist\psx\broad (the parent rom is the USA version of CHD), but there is a parent set named burningr in softlist\saturn and cmpro wants to move the psx CHD in saturn set, deleting it in second pass scan because is not in the right set, but ignoring that is a CHD for the softlist\psx\broad set.
 
What do you think?
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 22 September 2016, 20:35
since the page is going up and down at the moment due to some ISP things....I've attached the latest build which should list the missing chd in the missing set and should list the unneeded msx1_cart set.
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 23 September 2016, 15:02
Here I am with test result.

Very goog good news. The last cmpro did wonderful scan.
It found many wrong named/unneeded/double roms that before. With order:

1) in "softlist\alice32" cmpro found "coloric.7z" that is a clone of "colormac.7z". It asks for rename to "colormac.7z". Because "colormac.7z" already exists, it moves "coloric.7z" in backup folder instead of renaming it or adding it in "colomarc.7z". With rebuilder function on backup folder as source, cmpro added coloric rom inside "colormac.7z" archive.
In this way cmpro found all orphaned files I have. Some files (like coloric) were a clone 7z that's not present in parent set, some other files were a clone 7z that's also present in parent 7z archive.

2) in "softlist\psx", cmpro found "hatsukoi" folder that contains "hatsukoi barentain (japan) [slps-00831].chd" and renamed the folder in "hatsuval".
In this way it found other PSX and saturn CHDs in wrong (old named set for previous mame) folder and renamed them accordingly to .177 named set.

3) Gakuen King CHD is reported correctly in any way (miss one, both misses, both presents)

4) happyfrt and many others removed in last mame are correctly found as uneeded and moved in backup folder.

But stangely, cmpro didn't renamed "softlist\psx\burningr\burning road (japan) [slps-00518].chd" in "softlist\psx\broad\burning road (japan) [slps-00518].chd", but it detected wrong path and moved the CHD in "softlist\saturn\burningr". A second pass with scanner deletes "burning road (japan) [slps-00518].chd" from "softlist\saturn\burningr". So I moved "burning road (japan) [slps-00518].chd" in generic roms folder from backup folder and in third pass with scanner cmpro moved it in the right folder (softlist\psx\broad).

So I'm very happy that my collection is a clean full merge set and I have no more orphaned files around or false missing CHD because they stay in unknown folders

Good work (as usual)
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 23 September 2016, 15:35
Wooaah....so much names, so much text.. ;-) So 1-3 is all good and what you expect?
Regarding the "but strangely" part in 4), well a rename can fail (e.g. destination already exist, 7z solid archives etc etc..blabalbalba). I did a test here here with burnding road in psx and the move to saturn worked fine.

So....I guess I can make a new release on the weekend now that you're happy
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 23 September 2016, 16:52
I did a test here here with burnding road in psx and the move to saturn worked fine.

What I found strange is:

during scan, cmpro found in softlist\psx the folder hatsukoi tagging it as wrong name, so it renamed it as hatsuval. (right choice)
then cmpro found in softlist\saturn the folder bakushou tagging it as wrong name, so it renamed it as yoquizdx. (right choice)
then cmpro found in softlist\psx the folder burningr tagging it as wrong path, so it moved it in softlist\saturn\burningr (wrong choice, infact in second scan it deleted the moved CHD)

3 wrong named set, but 2 different treatment.
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Roman on 23 September 2016, 18:43
well, I know the reason for this...but I'm too tired to explain it here.
I will see if I can find a solution over the weekend.
Title: Re: wrong merging logic with new MAME 0.162 (MESS/MAME together) - another bug
Post by: Balteck on 23 September 2016, 18:49
Thank you very much. Don't be in hurry only for me.

When you want (if you wish) send me a PM to explain the reason.
I'm very curious and I like to be a good beta-tester

Thank you again for the very quick fixes