EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: Some new scanner wip  (Read 7029 times)

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 113
  • Offline Offline
  • Posts: 3292
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 119.0.0.0 Chrome 119.0.0.0
    • View Profile
Some new scanner wip
« on: 22 November 2023, 19:02 »

While wasting a little bit of time with delta chd support in old cmpro, I recently found a little time to work on the new scanner again..

So what's the status?...well, it scans nicely:

- mame -listxml output, mame hash files, mame -listsoftware output
- full/split/standalone merged
- multi rom/sample paths suppport
- pattern scanning supported (if you keep your sets split by bios etc)
- roms, disks and samples, compressed (7z/zip) or uncompressed
- writing of fix dats
- writing of report output
- remembering old scan information for a faster scan
- commandline available (compareable to new rebuilder)
- preliminary gui available (compareable to new rebuilder)

What's on the to do list:

Well, the actual fixing part, but that be rather simple since it simple needs to run through a list of tasks like
'rename this to that', 'delete this file here', 'add file from there to there'...

Got a small wishlist already some some UI related features here and there...but they don't have any priority...

and of course.....after that, a profiler...

Ever wanted to scan a full software lists collection? Remember the pain in cmpro to set it up? Well, now it's just adding a rompath and hitting start...a little screenshot shown below should give you an idea. Generally it should be way easier to set it up...no need to define 'system default paths', no need to setup thousands of distinct rompaths just for softwarelists.....
« Last Edit: 22 November 2023, 19:03 by Roman »
Logged


nullz3

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 6
  • Operating System:
  • Linux Linux
  • Browser:
  • Firefox 115.0 Firefox 115.0
    • View Profile
Re: Some new scanner wip
« Reply #1 on: 23 November 2023, 03:53 »

Hi Roman,

Thank you for sharing your updates.  I'm loving the new UI.  It's so clear and pleasing to the eyes.  Forgive me for bringing this up again, but I notice a box for filters, will that mean I would still be able to filter MAME sets by driver status in a future release?

Nullz
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 113
  • Offline Offline
  • Posts: 3292
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 119.0.0.0 Chrome 119.0.0.0
    • View Profile
Re: Some new scanner wip
« Reply #2 on: 23 November 2023, 05:49 »

for now it only accepts a regular expression to filter machines by name....but yes, a filter is part of my menioned nice-2-have things which will most likely be able to filter with xpath expressions....and then you can filter on anything inside the xml....
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 113
  • Offline Offline
  • Posts: 3292
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 119.0.0.0 Chrome 119.0.0.0
    • View Profile
Re: Some new scanner wip
« Reply #3 on: 23 November 2023, 19:10 »

Since people asked for some "benchmarks"...(e.g. .260, no softwarelists, but roms/disks/samples)

generally, you need to differ between 2 things: no diskcache (that's a full scan after a just started PC)
and diskcache (that's a full scan after a scan already took place where your PC hopefully uses the cached data)

old scanner: 8 min 45 seconds (no diskcache)
old scanner: 1 min  5 seconds (diskcache)
new scanner: 5 min 49 seconds (no diskcache)
new scanner: 0 min  5 seconds (diskcache)

Scanning a not existing MAME 260 collection (so everything is missing)
old scanner: 1 min 31 seconds (no diskcache)
old scanner: 1 min 28 seconds (diskcache)
new scanner: 0 min  1 second  (no diskcache) however it takes 26 seconds to render the tree :-)
new scanner: 0 min  0 seconds (diskcache)    however it takes 26 seconds to render the tree :-)

Or some progretto snaps (Snapshots with software list datfile, unpacked pngs on the disk)
old scanner: 7 min 21 seconds (no diskcache)
old scanner: 1 min 59 seconds (diskcache)
new scanner: 4 min  7 seconds (no diskcache)
new scanner: 0 min  6 seconds (diskcache)


The tester's system: i7-8700K, 16GB RAM, Seagate IronWolf HD
« Last Edit: 23 November 2023, 19:24 by Roman »
Logged

ntt3

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 26
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 119.0.0.0 Chrome 119.0.0.0
    • View Profile
Re: Some new scanner wip
« Reply #4 on: 24 November 2023, 22:52 »

Hi Roman
thank you, great news!!! Can't wait to try it in practice   :D
Logged

CoreyEMTP

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 15
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 120.0.0.0 Chrome 120.0.0.0
    • View Profile
Re: Some new scanner wip
« Reply #5 on: 14 December 2023, 00:42 »

Since people asked for some "benchmarks"...(e.g. .260, no softwarelists, but roms/disks/samples)

generally, you need to differ between 2 things: no diskcache (that's a full scan after a just started PC)
and diskcache (that's a full scan after a scan already took place where your PC hopefully uses the cached data)

old scanner: 8 min 45 seconds (no diskcache)
old scanner: 1 min  5 seconds (diskcache)
new scanner: 5 min 49 seconds (no diskcache)
new scanner: 0 min  5 seconds (diskcache)

Scanning a not existing MAME 260 collection (so everything is missing)
old scanner: 1 min 31 seconds (no diskcache)
old scanner: 1 min 28 seconds (diskcache)
new scanner: 0 min  1 second  (no diskcache) however it takes 26 seconds to render the tree :-)
new scanner: 0 min  0 seconds (diskcache)    however it takes 26 seconds to render the tree :-)

Or some progretto snaps (Snapshots with software list datfile, unpacked pngs on the disk)
old scanner: 7 min 21 seconds (no diskcache)
old scanner: 1 min 59 seconds (diskcache)
new scanner: 4 min  7 seconds (no diskcache)
new scanner: 0 min  6 seconds (diskcache)


The tester's system: i7-8700K, 16GB RAM, Seagate IronWolf HD

Those are some impressive gains, and the new GUI is a much-appreciated update. Just want to say I'm glad you've found a renewed interest in these tools, as it looked like you were understandably experiencing a bit of burnout. I think I can safely speak for others when I say we all appreciate it more than you know!
Logged

eesbritish

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 1
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 120.0.0.0 Chrome 120.0.0.0
    • View Profile
Re: Some new scanner wip
« Reply #6 on: 02 January 2024, 06:46 »

Those are remarkable increases, and the new GUI is welcome. It seemed like you were burnt out, so I'm delighted you've rediscovered these tools. I speak for others when I say we appreciate it more than you realize!

Dirstac

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 8
  • Operating System:
  • Linux Linux
  • Browser:
  • Chrome 100.0.4896.58 Chrome 100.0.4896.58
    • View Profile
Re: Some new scanner wip
« Reply #7 on: 06 January 2024, 02:22 »

Hopefully language files can be supported
Logged

haynor666

  • retro maniac
  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 64
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 120.0.0.0 Chrome 120.0.0.0
    • View Profile
Re: Some new scanner wip
« Reply #8 on: 12 January 2024, 19:23 »

Looks very promising  :)
Logged
Pages: [1]   Go Up
 

Page created in 0.227 seconds with 20 queries.

anything