EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: Leftovers after full rebuild  (Read 7418 times)

haynor666

  • retro maniac
  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 65
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 71.0.3578.98 Chrome 71.0.3578.98
    • View Profile
Leftovers after full rebuild
« on: 31 December 2018, 20:06 »

Today I've rebuild whole mame set again from uncompressed directories into zip archives. Again I also noticed that some SEGA firmware files are not removed from rebuild directory. Before rebuild I've scanned source folder and everything was fine so why segadimm, seganetw and segasp directories are not removed after rebuild?
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3423
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 71.0.3578.98 Chrome 71.0.3578.98
    • View Profile
Re: Leftovers after full rebuild
« Reply #1 on: 01 January 2019, 11:31 »

could be anything... ;-)
Is there a file in the folder? Or just an empty folder? If there is a file, the file most likely did not match anything valid in the database and was not rebuilt/removed....and so the folder is not empty and won't be removed, either. Then the question would be why a previous scan did not report the left over file as unneeded.....
Of course there is always the operating system which may lock a file / folder and when cmpro calls a remove file/folder function it simply fails due to locking (e.g when a windows backgroundtask accesses it..(virusscanners etc..).
When using network drives it can also be a (temporary) network problem......there are known SAMBA issues for example where files are simply skipped (due to outdated drivers etc..).
So...'can be anything' ;-)

So....what you can try is to simply rebuild the left overs again to see if they get cleaned up. Check if you got the rebuilder advanced option enabled which is reposible to remove empty folders...

I will do some tests here, too....later....Happy New Year





*update * ok...I can repeat the behaviour.....interesting...very interesting.....
« Last Edit: 01 January 2019, 13:00 by Roman »
Logged

haynor666

  • retro maniac
  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 65
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 71.0.3578.98 Chrome 71.0.3578.98
    • View Profile
Re: Leftovers after full rebuild
« Reply #2 on: 01 January 2019, 16:53 »

Files are:

segadimm\203_203.bin
segadimm\213_203.bin
segadimm\217_203.bin
segadimm\317_312.bin
segadimm\401_203.bin
segadimm\fpr23718.ic36

seganetw\ver1305.bin

segasp\fpr-24208a.ic72
segasp\fpr-24407.ic72

Second rebuild removes all these files from source folder:

Analyzed 9
Created 0
Matched 108
Skipped 9

Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3423
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 71.0.3578.98 Chrome 71.0.3578.98
    • View Profile
Re: Leftovers after full rebuild
« Reply #3 on: 01 January 2019, 17:15 »

I already found a clue what the reason for this is....the CRC32 values are identical but the SHA1 values differ....and I guess the rebuilder marks a file as "was rebuilt already" by crc32+size match...so it finds the first e.g. a738ea1c value, rebuilds all instances (there it does check the sha1 value) and marks all a738ea1c values as "been there, done that, bought a tshirt" and later on the files get skipped....and of course they are not removed....etc..etc...something like that...
will look into it later on....

for the second run nothing is marked yet and then they get removed....doh ;)


shoud be fixed in one of the upcoming nightly builds
« Last Edit: 01 January 2019, 17:16 by Roman »
Logged

haynor666

  • retro maniac
  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 65
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 71.0.3578.98 Chrome 71.0.3578.98
    • View Profile
Re: Leftovers after full rebuild
« Reply #4 on: 01 January 2019, 17:28 »

So there is a problem with listxml data ?
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3423
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 71.0.3578.98 Chrome 71.0.3578.98
    • View Profile
Re: Leftovers after full rebuild
« Reply #5 on: 01 January 2019, 19:07 »

no, not at all....crc32 hashes are simply not distinct enough to separate such roms...the sha1 values are. The rebuilder does rebuild by crc32, size and sha1 check, however cmpro remembers already rebuilt files...and I guess there it's lazy and remembers by crc32 only...and so the first checksum match is rebuilt but the others get falsely marked as 'already rebuilt' and get skipped (and so they are not deleted either)...as I said....I will check the actual problem in the cmpro source later on....

so...nothing to do from your or MAME side...it's my job
Logged

haynor666

  • retro maniac
  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 65
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 71.0.3578.98 Chrome 71.0.3578.98
    • View Profile
Re: Leftovers after full rebuild
« Reply #6 on: 01 January 2019, 20:19 »

Ok, thanks for checking this Roman :)
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3423
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 71.0.3578.98 Chrome 71.0.3578.98
    • View Profile
Re: Leftovers after full rebuild
« Reply #7 on: 03 January 2019, 22:36 »

should be fixed in latest nightly build
Logged

haynor666

  • retro maniac
  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 65
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 71.0.3578.98 Chrome 71.0.3578.98
    • View Profile
Re: Leftovers after full rebuild
« Reply #8 on: 04 January 2019, 16:48 »

Downloaded. I try it in next 3 days. Thanks for fix :)
Logged
Pages: [1]   Go Up
 

Page created in 0.099 seconds with 21 queries.