EMULAB Forum

clrmamepro [English] => clrmame Discussion => Topic started by: Roman on 22 November 2023, 19:02

Title: Some new scanner wip
Post by: Roman on 22 November 2023, 19:02
While wasting a little bit of time with delta chd support in old cmpro, I recently found a little time to work on the new scanner again..

So what's the status?...well, it scans nicely:

- mame -listxml output, mame hash files, mame -listsoftware output
- full/split/standalone merged
- multi rom/sample paths suppport
- pattern scanning supported (if you keep your sets split by bios etc)
- roms, disks and samples, compressed (7z/zip) or uncompressed
- writing of fix dats
- writing of report output
- remembering old scan information for a faster scan
- commandline available (compareable to new rebuilder)
- preliminary gui available (compareable to new rebuilder)

What's on the to do list:

Well, the actual fixing part, but that be rather simple since it simple needs to run through a list of tasks like
'rename this to that', 'delete this file here', 'add file from there to there'...

Got a small wishlist already some some UI related features here and there...but they don't have any priority...

and of course.....after that, a profiler...

Ever wanted to scan a full software lists collection? Remember the pain in cmpro to set it up? Well, now it's just adding a rompath and hitting start...a little screenshot shown below should give you an idea. Generally it should be way easier to set it up...no need to define 'system default paths', no need to setup thousands of distinct rompaths just for softwarelists.....
Title: Re: Some new scanner wip
Post by: nullz3 on 23 November 2023, 03:53
Hi Roman,

Thank you for sharing your updates.  I'm loving the new UI.  It's so clear and pleasing to the eyes.  Forgive me for bringing this up again, but I notice a box for filters, will that mean I would still be able to filter MAME sets by driver status in a future release?

Nullz
Title: Re: Some new scanner wip
Post by: Roman on 23 November 2023, 05:49
for now it only accepts a regular expression to filter machines by name....but yes, a filter is part of my menioned nice-2-have things which will most likely be able to filter with xpath expressions....and then you can filter on anything inside the xml....
Title: Re: Some new scanner wip
Post by: Roman on 23 November 2023, 19:10
Since people asked for some "benchmarks"...(e.g. .260, no softwarelists, but roms/disks/samples)

generally, you need to differ between 2 things: no diskcache (that's a full scan after a just started PC)
and diskcache (that's a full scan after a scan already took place where your PC hopefully uses the cached data)

old scanner: 8 min 45 seconds (no diskcache)
old scanner: 1 min  5 seconds (diskcache)
new scanner: 5 min 49 seconds (no diskcache)
new scanner: 0 min  5 seconds (diskcache)

Scanning a not existing MAME 260 collection (so everything is missing)
old scanner: 1 min 31 seconds (no diskcache)
old scanner: 1 min 28 seconds (diskcache)
new scanner: 0 min  1 second  (no diskcache) however it takes 26 seconds to render the tree :-)
new scanner: 0 min  0 seconds (diskcache)    however it takes 26 seconds to render the tree :-)

Or some progretto snaps (Snapshots with software list datfile, unpacked pngs on the disk)
old scanner: 7 min 21 seconds (no diskcache)
old scanner: 1 min 59 seconds (diskcache)
new scanner: 4 min  7 seconds (no diskcache)
new scanner: 0 min  6 seconds (diskcache)


The tester's system: i7-8700K, 16GB RAM, Seagate IronWolf HD
Title: Re: Some new scanner wip
Post by: ntt3 on 24 November 2023, 22:52
Hi Roman
thank you, great news!!! Can't wait to try it in practice   :D
Title: Re: Some new scanner wip
Post by: CoreyEMTP on 14 December 2023, 00:42
Since people asked for some "benchmarks"...(e.g. .260, no softwarelists, but roms/disks/samples)

generally, you need to differ between 2 things: no diskcache (that's a full scan after a just started PC)
and diskcache (that's a full scan after a scan already took place where your PC hopefully uses the cached data)

old scanner: 8 min 45 seconds (no diskcache)
old scanner: 1 min  5 seconds (diskcache)
new scanner: 5 min 49 seconds (no diskcache)
new scanner: 0 min  5 seconds (diskcache)

Scanning a not existing MAME 260 collection (so everything is missing)
old scanner: 1 min 31 seconds (no diskcache)
old scanner: 1 min 28 seconds (diskcache)
new scanner: 0 min  1 second  (no diskcache) however it takes 26 seconds to render the tree :-)
new scanner: 0 min  0 seconds (diskcache)    however it takes 26 seconds to render the tree :-)

Or some progretto snaps (Snapshots with software list datfile, unpacked pngs on the disk)
old scanner: 7 min 21 seconds (no diskcache)
old scanner: 1 min 59 seconds (diskcache)
new scanner: 4 min  7 seconds (no diskcache)
new scanner: 0 min  6 seconds (diskcache)


The tester's system: i7-8700K, 16GB RAM, Seagate IronWolf HD

Those are some impressive gains, and the new GUI is a much-appreciated update. Just want to say I'm glad you've found a renewed interest in these tools, as it looked like you were understandably experiencing a bit of burnout. I think I can safely speak for others when I say we all appreciate it more than you know!
Title: Re: Some new scanner wip
Post by: eesbritish on 02 January 2024, 06:46
Those are remarkable increases, and the new GUI is welcome. It seemed like you were burnt out, so I'm delighted you've rediscovered these tools. I speak for others when I say we appreciate it more than you realize!
Title: Re: Some new scanner wip
Post by: Dirstac on 06 January 2024, 02:22
Hopefully language files can be supported
Title: Re: Some new scanner wip
Post by: haynor666 on 12 January 2024, 19:23
Looks very promising  :)