EMULAB Forum

clrmamepro [English] => clrmame Discussion => Topic started by: geekbrain on 03 November 2020, 01:39

Title: 4.038a - Extremely long scan time (titles.zip resource)
Post by: geekbrain on 03 November 2020, 01:39
I updated to version 4.038a and was able to run successfully with MAME 0.226. I tried running my Titles zip resource (from http://www.progettosnaps.net/snapshots/) and it ran for over 12 hours before I stopped thinking was hung.  Granted my machine is not the latest vintage but it never took more than a few minutes. I tried again and had the same result.  Thinking my titles.zip file was corrupt, I downloaded the base set and updates.  I cannot seem to get through a scan (with 'fix') on the base set with 0.225 dat file.  Don't know if this is a corrupt dat file, corrupt titles.zip file, or issue with ClrMAMEPro update.  I was able to scan my logos.zip file is reasonable time without issue.

Anyone have similar issue?
Title: Re: 4.038a - Extremely long scan time (titles.zip resource)
Post by: Roman on 03 November 2020, 05:38
Turn off fix missing and belonging advanced fixed missing options.

The reason is that progetto sets consist of 1 set with thousands of files and often it holds a pretty large amount of pngs placeholders (identical 'default' pngs). So if you miss them, cmpro looks into the set (and addpath, backup), finds it somewhere and adds it (recompress), then the next one...same procedure again...and again...when using compressed sets it's evil....when using solid 7z archives it's hell.

For progetto snaps it's better to add files with the rebuilder instead of using fixmissing...and consider simply using not compressed files instead of archived ones.
Title: Re: 4.038a - Extremely long scan time (titles.zip resource)
Post by: coccola on 03 November 2020, 17:25
Quote
[...]progetto sets consist of 1 set with thousands of files and often it holds a pretty large amount of pngs placeholders (identical 'default' pngs). So if you miss them, cmpro looks into the set (and addpath, backup), finds it somewhere and adds it (recompress), then the next one...same procedure again...and again...

Couldn't clrmamepro notice that the same PNG placeholder file is used multiple times and simply add them all at once?
Title: Re: 4.038a - Extremely long scan time (titles.zip resource)
Post by: Roman on 03 November 2020, 18:20
If I add it yes, for now: no
For such 1 sets-100000 files sets it's currently better to follow my advice from the start of this post
Title: Re: 4.038a - Extremely long scan time (titles.zip resource)
Post by: geekbrain on 04 November 2020, 01:16
Roman-
Thanks for the prompt reply.

I am hesitant to uncheck the fix missing because in the past it seemed like a few 'fell through the cracks' - the update file was missing something that need to be fixed from the prior release.

I tried upzipping the png's and placing in the titles directory but ClrMAMEPro re-zipped them into a titles.7z zip file. Do the png's have to be zipped?  Is there any way to leave them in their native png state in the titles (logo, etc.) directory and still use the fix missing option?

-Thanks-
Title: Re: 4.038a - Extremely long scan time (titles.zip resource)
Post by: Roman on 04 November 2020, 06:03
Titles/Snaps/Bosses/etc etc don't have to be zipped/7z'ed/rar'ed at least not for checking. For the actual use it depends on the frontend.
Regarding fix progetto snaps, I usually do the following:
- run a scan with all fix options but fix missing
- use the rebuilder to add the update packs and if something is still missing you may rebuild the backup folder to readd stuff which was put there. Most of the time you end up with 100% if not you may want to take a manual look at it
Title: Re: 4.038a - Extremely long scan time (titles.zip resource)
Post by: geekbrain on 05 November 2020, 01:23
Turned off fix missing option but not belonging advanced fixed missing option.  Still is taking forever and was never this long.

You indicated the secondary resources don't need to be zipped/7z'ed/rar'ed.  Why did ClrMAMEPro re-zip them into the 7z archive?  Is there a setting for this?
Title: Re: 4.038a - Extremely long scan time (titles.zip resource)
Post by: geekbrain on 05 November 2020, 03:27
Tried canceling the operation over an hour and a half ago - no response.
Title: Re: 4.038a - Extremely long scan time (titles.zip resource)
Post by: Roman on 05 November 2020, 11:21
"Why did ClrMAMEPro re-zip them into the 7z archive?"

If it does that on your machine then you're using 7z archives. Simply as that.

cmpro autodetects how you stored your sets (either not compressed or rar/zip/7z). So you store your snaps in titles.7z it of course adds found missing files into that 7z. "Adding" can mean different operations. Adding from one zip archive to another takes no time since it copies the compressed data. Adding from a decompressed file to an archive takes compression time for that file. Adding to a solid 7z archive means repacking the full archive.

Again, for progretto snaps I'd go with decompressed sets (at least for checking).