Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Roman]

https://mamedev.emulab.it/clrmamepro/binaries/clrmame_008_015_1.zip
https://mamedev.emulab.it/clrmamepro/binaries/readme.html

UI:
misc: restore default window positions when positions get out of range
misc: increase display time of tooltips
misc: slightly regrouped ui elements
added: scanner, context menu option to restore old scan results on startup and selection

Core:
misc: updated spdlog to 1.15.0
misc: updated to 7zip 24.09
misc: don't reload xml and build up structures when they are already in memory and need no refresh
misc: align and share rebuilder/scanner common routines
added: scanner, samples, supporting flac
fixed: scanner, typo in wine script
fixed: scanner, internal sha1 check for files with identical crc32 but different sha1 values isn't run
fixed: scanner, samples, wrong named files (case check only) aren't fixed
fixed: scanner, samples, wrong named files (case check only) also appear as unneeded
fixed: scanner, samples, decompressed files are marked as unneeded
fixed: scanner, samples, machines which reference themselves via sampleOf are marked as unneeded in full mode
fixed: scanner, confusing 'can't remove/backup' messages in case of circular renames between different sets
fixed: scanner, confusing 'can't remove/backup' messages in case a wrong file with a right name is replaced with a fill in

[john iv]

Thanks Roman, nice update.

I am seeing an oddity though.
1. Run new scan w/ samples path set and fix issues chosen.
2. Get warnings about those self referential samples.  See that they are attempted to be fixed.
3. Run scan.
4. No more warnings.
5. Quit app.
6. Relaunch and do a new scan.
7. Warnings return.
8. So either the sample only set wasn't added or it's forgetting it or thinking it should be fixed?

[Roman]

The warnings appear if you have sample paths setup and *load* MAME's xml output. They are the known-for-a-decade warnings about sample only sets etc...
If you don't quit and run another scan, the xml isn't loaded again since it's still in memory (see first info log message in your second screenshot). That feature is coming with yesterdays version.....so all is fine


By the way....come on...you only post here to show your 1 second full scan result :-)

[john iv]

Guess the warning confused me with its verbiage of 'sample-set only was added'. I took that to be some sort of fixing that would prevent the errors in the future, and when I re-ran the scan in memory and didn't see the errors it got my hopes up.

Oh, 1 second? I hadn't noticed that result, heh. The new caching speed is very nice for the usual multiple checks and runs.

[oddi]

Hi Roman
With Progetto-Snaps arts sets this tool work very very slowly, few hours.
Play with pS_AllProject_20241020_270_(cm).dat - scan finished after 5 hours, and when enable "fix issues" - more 3 hours.
Only for info. Thank you

[Roman]

I just scanned the snaps dat (the biggest single one) on a decompressed snaps set (so it needs to calculate the hashes) and it took < 1 min in a not cached scan and 4 seconds in a second, cached scan....

Keep in mind that the "all project" datfile uses a totally different folder structure than the single progetto datfiles. The all datfile uses set subfolders (like artpreview\artpreview\005.png, where the 1st artpreview is the setname and "artpreview\005.png" is the romname. While the single ones don't use setsubfolders (besides of SoftwareList dats).

So if you don't already have the files stored in the all-project way, there will be >300.000 files to be reorganized. Same would happen if you specify the rompath wrong which is -when it comes to progretto snaps- a rather common problem.
People need to remember the storing method which needs to be used rompath\setname\file1...file n for decompressed sets, rompath\setname.zip (.7z) for compressed sets. So for example if you're using the "bosses" progretto snaps datfile it contains 1 set called "bosses" where all pngs are stored in that set. So if you have a rompath called "c:\test\progretto\bosses", you either have a bosses.zip file in that folder holding the pngs or you got another subfolder in there "c:\test\progretto\bosses\bosses"  which holds the singe pngs decompressed.

Progetto snaps dats usually have 1 (SL maybe a handful more) sets with thousands of pngs inside it. Having them compressed can cause slow downs when fixing (especially when using solid 7z sets).

Another point when using decompressed files is of course: each file needs to be read and the crc32 (at least) is calculated...a full progetto snaps collection is ~300000 files...now you can calculate yourself how long it can take for decompressed sets....that can be hours

let's take for example 1/100 second for reading the file plus calculating the hash.... that will be 100 files per second....for 300k files it's 3000 seconds.....50 minutes 0.83 hours....if it's 1/10 second per file...you already end up with 8 hours....and so on ;-)

....and another point....is that the less set specifications you have (and in the progetto case you usually only have 1), the less speed you will get from threading, since threading works on checking sets in parallel, not files....


But to give this thread an end....I will look what can be optimized when working with huge 1 set dats where the files are decompressed. ....actually I found something which can be improved already...so will see what I can do.....

[Dullaron]

Tested again using the /home/mitchell/Programs/Prefixes/tools/drive_c/ path.

Working without an rebuiding error now.

Only works on Split. Full and Standard still on Split. Using a mame.xml.

I'm trying the plain wine64 next. WINEPREFIX=~/wine64/ WINEARCH=win64 wineboot

Heroic and Lutris probably putting out bad readings when create it own wine files. I don't like this behavior.

[Roman]

"Only works on Split. Full and Standard still on Split"

? ? ?