Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Kolano]

I'm seeing an issue with v4.016, where multiple runs of "Scan" on the same folder will report different sets of ROMs as being missing, even though all ROMs seem to be in place. Typically this has been against the most recently release TOSEC dats. For instance, multiple scans over "Sharp X6800 - Games (TOSEC-v20014-05-13)" dat gave the following results...

Initial "New Scan":
Sets: 0/3245
ROMs: 1173/3245

Initial "Scan"
Sets: 0/3245
ROMs: 158/3245

Secondary "Scan"
Sets: 0/3245
ROMs: 0/3245

Secondary "New Scan"
Sets: 0/3245
ROMs: 1148/3245

The folder contents were created with the Rebuilder, and no change to the folder or it's contents occurred between runs. I tend to see issues with the Rebuilder on these same sets where it will indicate there is an error adding files to a set on a few files. Usually these will be ROMs that appear to already be in the set.

I'm currently using 7z compression on the ROMs, unclear if the compression method makes a difference.

[Roman]

well...actually I wouldn't care so much about the stats...the only real important information is "is the scanner tree window empty or not".

But anyway...the initial new scan, initial scan and secondary scan results can be pretty ok....first you miss stuff, fix missing/rebuild added something for the initial scan...and the secondary one fixed some more (which can happen from time to time...you you had a rebuild in between)..
the only weird one is the "secondary new scan"...Question would be what is shown in this case.

Well, if the rebuilder complains about adding fails (due to already existing files), it's only a rebuilder problem which should not affect scan results...


So..you may want to provide the dat, the belonging settings (.cmp) file and maybe the scan results tree output.....so I can have a look what's going on there...but again...the interesting part is the scan tree results window....

[Kolano]

The scanner tree window is not empty, until the scan that eventually reports none missing but that can take 2-3 rescans to reach.

I think you may be confused regarding what I'm reporting though, there is no rebuilding or fixing occurring over these scans (i.e. all Fix options are disabled, and the scans are run back to back without rebuild runs in between or other changes to the files). The initial rebuild seems to create a complete set. The scans then make no changes to the files, but report fewer missing ROMs with each repeated scan until there are none. Repeating a new scan typically then shows missing ROMs again, but a different number than reported previously.

For example, I just reran "New Scan" on the set four times (again with all Fix options off, and no changes occurring to the files between runs). First run reported 466 missing ROMs, the second reported 613, third 550, and the fourth 452.

I've attached the requested items. I couldn't find an option to export the Scan Results window content, but included screenshots of the end of run 3 and 4's contents. Related ROM files can be pulled from: https://archive.org/details/Sharp_X68000_TOSEC_2012_04_23

Regarding the rebuilder, it will normally error fewer times than the scans do. On this set only ~50-70 failed adds vs the hundreds of missings reported by scan, similar to the scan the specific files and their number seem to change with each run. I'd guess the issues may be related though, in that both are failing to pick up on ROMs that are in place.

[Roman]

Well...I will do some investigation...but generally..there are some cases where it's pretty normal that two scan runs will do some fixing...First run may move a wrong placed file to backup and depending on parent/clone relationships and set ordering it may be picked up from there (to be placed elsewhere) in a second scan....

But yes...that should take 2 scans at all...not 3...

[Roman]

hmm..well.... ok..grabbed the set from your provided link, moved the zipfiles in one new rompath and did a new scan....

Missing
·Sets                 0/3245
·ROMs                 0/3245
·CHDs                 -/0
·Samples              -/0
·Bytes                0/4gb

first scan did remove several unneeded files...but anyway....doing some more scans/new scans all lead to this result...with 0 missing...

So next step, I removed all sets starting with Z....new scan...shows some missing...but always the same number....
So I enabled fix-missing (I moved the Z ones to backup before) ...and all files were picked up...and the stats show 0 miss again..

so...what exactly did you do?
I need something like a step by step procedure so I can repeat/build up your scenario

[Kolano]

I'm still unclear regarding what's happening, but it only seems to be effecting the files located in a particular directory. Copies of the files to new locations, or even the same named location (via copy+folder renames) allows for successful scans. A WinMerge compare of the problematic folder with ones that scan without issue doesn't indicate any differences in their contents.

The files are located on a NAS on a set of RAID6 drives. I'd think the RAID6 parity would ensure that even a bad disk wouldn't result in problems, but will be performing a more thorough drive scan shortly to see if that turns up anything.

[Roman]

hmm....on a NAS......hmmm....I rememeber a problem a long long long time ago where users reported rebuilding problems when the files were stored on a MAC/network drive/Samba blabla...There you had effects where file access sometimes skipped files....so 1st rebuilder run took let's say 90 out of 100 files...and skipped 10...then a second took the rest...

Your problem sound a bit a like that...first scan failed on accessing some sets....second found the skipped ones...

I don't remember what the solution for the SAMBA problem was...or if it was driver/network/timing related...but maybe I find something in old forum posts (actually on old not emulab forums) or old emails...

As you already mentioned, if you don't use the NAS it works...

On the other hand, cmpro simply uses Win api calls to open/read/write files...nothing really special...


ok..when searching this board you find some comments already...but unfortunately no real solution....
"It has been reported several times and all users came back with something like "My NAS was crap, got a different brand and it's ok now" or "I changed access rights and it's ok now"."  or comments about files being simply locked in the moment they got scanned....

[Kolano]

Thanks for the feedback Roman.

The disk scan came back as healthy, and as indicated previously it's a RAID6 array with 2 separate parity disks so it should be resilient to disk problems should they be present. For now I've just purged the copy that was acting oddly. Sorry that I'm not able to provide details to help pin down what may be going on in these situations. You can chalk up at least one more report that something odd sometimes happens when using network disks over Samba.

[Roman]

well...I will check if I can setup a NAS or google a bit to see if there are known Win API issues....but don't expect anything soon at the moment...