Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

221

clrmame Discussion / Re: Snan ROMs without subfolders

« on: 08 June 2023, 04:47 »

No. The storing method is
Rompath/setname/file1…file n for deompressed sets or
Rompath/setname.zip (.7z/.rar) (where the archive holds file 1…file n) for compressed sets. Where file 1…file n corresponds to rom files of that one set.

You have to follow that.

If you want single-file sets on the same folder level you need to edit the datfile to have one set only holding all sets in one set.

222

clrmame Discussion / Re: App freezes when scanning specific DAT

« on: 02 June 2023, 06:00 »

Scanner->Advanced ->Fix Missing Options -> turn off "Deeper check for fixable missing files" and turn off "additionally test sha1/md5 matches.

Progetto Snaps collections use a lot of placeholder pngs where you usually have at least one copy available. So as soon as you miss one of such files, fix misssing (and the upper mentioned ones) look through all your rompaths/backup files...and that takes really long. And when the next missing file is detected it does the same over and over again.

So it's not freezing but it is busy like hell. For progetto snap collections, you should turn off all the fix-missing related stuff. The rebuilder should be used to add the missing parts instead.

223

clrmame Discussion / Re: App freezes when scanning specific DAT

« on: 02 June 2023, 04:49 »

You‘ve got fix-missing enabled? If yes, turn it off.

224

clrmame Discussion / Re: Rebuilder 0.06 released

« on: 07 May 2023, 18:27 »

 

225

clrmame Discussion / Re: Rebuilder 0.06 released

« on: 07 May 2023, 17:57 »

Not really a realistic scenario, since I don't know anyone who does a) a full rebuild and having b) compress: rezip and sha1:both enabled. Such options are timekillers (beside the amout of a rebuild).
As an example, if you don't use rezip, sha1 none and hardlinks, your full rebuild is most likely done within an hour (depending a bit from where to where you operate)
I know the argument "but I want to be 100% sure that the archives are fine"...but actually a post run commandline based archive test is faster. Same can apply for "but I want identically compressed archives".
The typical use case for the rebuilder is to update an existing collection with a bunch of sets.

But of course, if you like to spend 4.25 hours to always redo everything, ok...it's of course doable ;-)

226

clrmame Discussion / Re: Rebuilder 0.06 released

« on: 05 May 2023, 04:41 »

….and now compare it to old cmpro :-)

You have to be careful when comparing results...the diskcache can fool you immensly. So you need to compare either both with or without diskcache...
So the best ist (besides using the same source/destination) a reboot of the machine in between the tests and doing 1 scan.
Or you run multiple ones in a row for one version and take an average.

I'd expect something about 1 to 30% speed gain compared to 0.05...but yes, it also depends on your machine/number of cores

227

clrmame Discussion / Rebuilder 0.06 released

« on: 04 May 2023, 18:35 »

Rebuilder 0.06 released

https://mamedev.emulab.it/clrmamepro/binaries/rebuilder_v006.zip
https://mamedev.emulab.it/clrmamepro/binaries/readme.html

Doing source and destination matches now in multiple threads...gives a nice speed boost.

228

clrmame Discussion / Re: How to rebuild romset without added clones?

« on: 24 April 2023, 05:59 »

The set merge modes (standalone, new rebuilder tool), full (merged), split (merged), un/not-merged don't play a role here. They only define how the sets are stored on your system. If you want to "cherry pick" several sets, you need to disable the ones you don't want. In old clrmamepro, you can do this in Scanner->Scan Results Tree window, bottom left 'set information' button. In Set Information you have various ways to enable/disable sets either by single-ticking, regular expressions, filter lists and so on. Only enabled ones will be rebuilt.
In the new rebuilder tool you can set a regular expression via the -f --filter parameter.

229

clrmame Discussion / Re: -listxml VS hash/*.xml

« on: 22 April 2023, 20:14 »

Ask the devs 😁

230

clrmame Discussion / Re: -listxml VS hash/*.xml

« on: 22 April 2023, 18:59 »

Especially the rompath setup is really really annoying which I have to admit....but as I said, this might change drastically.
Actually I don't know the reason why -listsoftware does not include all software lists of the hash folder. Well, at least in the past, several were not included in the extract. And since normally only a handful of lists change from MAME version to version, it makes more sense to have a profiler per software lists. Rompath generation and assignement can be here done via the batcher...

231

clrmame Discussion / Re: rebuilder 0.05 released

« on: 22 April 2023, 18:41 »

There is a difference of parallel accessing the disk or doing parallel calculate threads….it needs a bit of tuning and checking to get a good result. If you force too many threads your cpu needs time for swapping etc… so as mentioned I will play around a bit. Dont expect too much….you also produce overhead with starting threads

232

clrmame Discussion / Re: -listxml VS hash/*.xml

« on: 22 April 2023, 05:40 »

When you import your data from a mame.exe you're asked after the import (unless the belonging option is unticked) if you want to import software lists. Then cmpro does load this -listsoftware data and you got all software lists in one profile. But....first of all, -listsoftware does not include all software lists from mame's hashfolder and secondly, it's a real pain to set this mode up correctly. You'd need unique distinct rompaths for each software list and scanning takes long, more memory usage etc..etc...This mode is not recommended.
Hopefully such problems will be history with a new scanner. You see that the new rebuilder already handles -listsoftware flawlessly and efficient.

233

clrmame Discussion / Re: rebuilder 0.05 released

« on: 21 April 2023, 06:32 »

As you (the audience) might have noticed, "fool-for-a-day, troll-for-a-life-time" Jessica Jones, rolled over the boards again and was removed by admins....nothing unusual...this happens once a year...

But the really funny part is:

The discussion gave me an idea where threading can give some extra speed up....It's not related to the troll comments at all (CUDA for SHA1 etc...)

But a first small test was already pretty positive which gave a extra ~25% speed boost when sha1 input scanning was enabled....
For non sha1 runs it's not that big since you simply have pure lookups without decompression/sha1 and the actual gain by multiple threads is eaten up by the overhead to create the threads...

I will check how this scales for larger data and for number of threads....

So..Thank you Jessica Jones...you made my day. See you next year.

234

clrmame Discussion / Re: rebuilder 0.05 released

« on: 17 April 2023, 11:24 »

Hash values of an archive is of course obsolete.....order of added files and compression rate of course alters it....there is no guranteed add order, no sort, no nothing

235

clrmame Discussion / Re: rebuilder 0.05 released

« on: 17 April 2023, 11:02 »

Nice to hear...that old cmpro is slower ;-) I was a bit sceptical since you turned off all of the major speed advantages....but sounds good to me....

236

clrmame Discussion / Re: rebuilder 0.05 released

« on: 16 April 2023, 18:59 »

Restart your machine first, otherwise you will cheat using the disk cache ;-)
...and of course you need to enable "recompress" in cmpro's rebuilder... (which corresponds to 'rezip')
Would be interesting to see....I mean with turning on "rezip" you of course disabled the major speed optimization....since it doesn't copy full archives in that case

237

clrmame Discussion / Re: rebuilder 0.05 released

« on: 16 April 2023, 17:12 »

ok...looks like other people also give some feedback:

a complete 253 (roms + chds) split sets to "mode: full" rebuild, "compress: zip", "sha: none", on one and the same HD without deletion of source files: 3441,69 seconds....which is 57 minutes...

...but "use links: hard" was used....so...a little bit of cheating :-) but still...remarkable

238

clrmame Discussion / Re: rebuilder 0.05 released

« on: 16 April 2023, 16:52 »

I pretty sure  a recursive commandline unzip -t on all archives afterwards would be faster ;-) but ok...

239

clrmame Discussion / Re: rebuilder 0.05 released

« on: 16 April 2023, 16:28 »

rezip...yikes...that of course unzips each file from each archive to temp and recompresses the files....why do you do that? If you want zip files in your destination, simply use "zip"....if something decompressed, something in a rar or 7z matches in the source it will be (unpacked) and repacked to zip....everything which is already zipped in the source can be copied without unpack/repacking....you're wasting time with rezip ;-)

240

clrmame Discussion / Re: rebuilder 0.05 released

« on: 16 April 2023, 15:38 »

hmm...all mechanial ones (3 chds, ~15k sets), old cmpro (without chds): 210 seconds, new rebuilder (with chds): 109 seconds...
(split, no sha1, zip2zip, hd to (different) hd)