Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[B2K24]

Very rarely this happens, but it has happened to me rebuilding Full MAME ROM sets and some FBA zips.
I'm very curious to know how and why does this happen?

using latest version CMP X64 version on Win 7 X64 Ultimate

To make matters worse Torrentzip doesn't error out when tzipping a file with dupes inside.

[EDIT] I forgot to elaborate that new scan + all fixes does not detect the dupe/unneeded files
even emptying out the profile and starting fresh doesn't catch what's going on here.

A sample file can be look at here  >>> http://www.sendspace.com/file/qogxyd

[Roman]

I will check this

[Roman]

hmm...wasn't able to reproduce it here....(however I'm aware of such archives)
just a sidenote, your 'h' zip folder doesn't have any dupes, 'g' does though.

Next cmpro got an option which will warn you about such dupes...

But I really want to find the source for this issue.
How did you rebuild? With or without recompression? From unzipped files to compressed ones? I need some more details to reproduce this here...

no matter if you use a non-recompression or recompress run, if a file is added to a zip, the routines first check if the file exists and removes it, then the file is added. In case of the rebuilder it checks if the file exists, if so (and crc32 matches), the archive won't be touched at all.
So...currently I can only imagine that something rarely goes wrong when the file is removed and then added...but looking at the routines everything is fine...so I need a robust way to reproduce it....

[B2K24]

I think it's not possible to reproduce this issue if you rebuild into an empty Destination Directory.
Now my drive G:\ I always keep a Full Complete MAME romset that I update with every u cycle. My drive H:\ I only keep MAME full releases, so in this case it's 144 then 145 and skipping the u cycles.

I use only the default settings to clrmame and don't change anything. I first preform the scanner and execute new scan  with all fix options checked. Then when some misses exist I jump to rebuilder and on the rebuilder I have checked the first 3 options.

compress files Zip
Recompress files
show statistics

In the rebuilder the source Directory also contains zip archives  I'm always rebuilding from zipped files into a destination that also always contains zipped archives.

So far I have encountered this updating MAME ROMs and working with FBA sets. The only things in common I can think of are the destination directories have previous complete sets in them and source directory always have zip archives.

I have no idea right now if my MESS softlists are affected, but I enjoy using batch mode working with those 

Thanks for everything and I wish you the best of luck at finding out how this occurs. It's really rare, but does happen and I'm completely stumped at the moment.

[oxyandy]

Any chance the new 4.03a
Detects these sort of zips yet ?
Thanks

[Roman]

Yes, 4.03a got an option to detect them. Check Settings->Compressor->General

[B2K24]

Yes, 4.03a got an option to detect them. Check Settings->Compressor->General

Thank you very much Roman, for adding this feature.
Maybe investigating these zips from MAME ROMs might be something in common to give you idea how this happens.
If I come across anymore I will list them.

G:\MAME\roms\m1cik.zip got identical named files
G:\MAME\roms\m5tempt.zip got identical named files
G:\MAME\roms\sc2scshxstar.zip got identical named files

[oxyandy]

B2K24
In the world you live,
there is a program which re-zips roms to a standard..
I feel that this program is flawed, in that it does not notice these files.
If I was to post about this problem in that 'world'
I would surely be ignored, but maybe from some-one like yourself,
a similar post would be noticed..

I had pm'ed a user (due to sensitivity of attaching zips in that forum)
with samples 'dupe internal' files, explained the issues,
and asked if they could 'mod' the code to detect and 'fix' these zips.
I got a reply, but it went no-where
I leave it for you.

Extra,
I had contacted Roman previously directly
and provided him with some sample 'problem' zips.
And as always, thanks Roman.

[B2K24]

It's actively being worked on and it's a work in progress.
I would be happy if it maxed my CPU and all cores/threads to 100% the same as chdman does ;-)

It's best if you post your ideas in the forum because more than one person works on that code.
A post from me would be the same as a post from you (rank doesn't have any effect on this)
It's always easier said than actually done and I don't have the skill to do it and get it done.

[Roman]

seems like I nailed it down

seems to be a rare issue when removing files from archives. If you got double filename entries in the provided list of files (which can happen) which should get removed, the lib call causes an assertion and it seems that the file is not removed...the rest are follow up errors.....

so this should be history now....

[Roman]

oh actually that was only one half of it...just discovered the other

[B2K24]

This is wonderful news. I'm happy you have discovered how this happens because it's such a rare occurrence. I didn't know if it would ever be discovered.

As always, Thank you and very impressive work.

[Roman]

actually is pretty easy to repeat...

assume a zipfile with files: a and b and a task to rename a to b...et voila...you got 2 b's....

[oxyandy]

Excellent !
So glad you nailed it !
Looking forward to the future release