EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: New scanner, filtering  (Read 2590 times)

john iv

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 72
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 127.0.0.0 Chrome 127.0.0.0
    • View Profile
New scanner, filtering
« on: 23 August 2024, 18:38 »

So I set up the scanner to take the a2600.xml from MAME.
I left the rom paths c:\mame\roms and c:\mame\software (the a2600 directory exists in c:\mame\software).
I then ran the scanner with the attached settings and it tried to remove as uneeded all the non-a2600 content and eventually crashed/closed as my temp directory on a ramdrive filled up completely.

With these settings I feel I should probably using filtering or pattern of some kind but I haven't done that before.. or is the preferred way to just change the rom path to the a2600 directory.
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3414
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 127.0.0.0 Chrome 127.0.0.0
    • View Profile
Re: New scanner, filtering
« Reply #1 on: 24 August 2024, 07:03 »

Interesting that it crashes when temp fails due to ramdrive is filled up....Will check that, however I wonder why so much files go to temp?

So you're scanning a MAME collection (which most likely has no or nearly no a2600 files) and a full software list collection (same here...a2600 files only in the a2600 subfolder...maybe some in related 2600 atari machines)....so it should find millions of unneeded.

First question: why do you scan a a2600 hashfile over all paths? if you scan a 2600.xml, use the softwarelist a2600 path for it :-)

You may better use the other paths as addpath, so they act as possible fill ins for missing files.


Filtering: well...filtering works on the loaded xml, so you can define which machines in the loaded xml should be active or not. The problem in your setup is that you have countless other folders (standard mame plus all the other software lists).
So with filtering you define subsets of the loaded data, either via regexpression, xpath or file. So e.g. in a softwarelist export you only take the Commodore and Atari software lists.

Patterns play a role when you have weird setups, like split up romcollections by bios (similar as system default paths in old cmpro). When using a softwarelist export you automatically have a not visible #SOFTWARELIST# pattern, so it uses a storing mechanism rompath\softwarelistname\setname....

So...if you load a a2600 standalone hash file you should have 1 rompath pointing to the a2600 folder.
If you load a softwarelist export and only want to check the a2600 files, point to your software list (parent) folder and use an xpath filter which only filters on a2600.
Logged

john iv

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 72
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 128.0.0.0 Chrome 128.0.0.0
    • View Profile
Re: New scanner, filtering
« Reply #2 on: 24 August 2024, 18:50 »

Thank you Roman, yes it was just me excited about running your new apps quickly and then afterwards RTFM'ing. :) You're right of course, just use the \software\a2600 directory as rompath and all is fine.  The scanner didn't produce a crash message in a classical sense when the temp directory on the ram drive ran out of space it simply quit out.  Maybe it logged something somewhere though.

So tell me if I am doing this in the least steps for my favorite \software items, a2600 for example.
1. Add the \hash\a2600.xml to source.
2. Add the \software\a2600 directory to rom path.
run with appropriate settings.
Rinse repeat for other softlist items.

I thought I had read in one of your updates that you could speed this process by doing step 1, but in step 2 have the top level \software directory listed instead and the scanner would be smart enough to know to put the a2600.xml content into \software\a2600?  Actually, that's probably what you're describing with the xpath filter it looks like.

I see that you retain like 10 of the most recently used xml sources in a drop down (and their associated rompaths when chosen), is there a way to make more of them available?  Course in cmpro they're all there in the profiler. :)

Oh, this was the text in question from your post about the differences:
Simply us 1 rompath and use either one of the patterns (e.g. #BIOSSPLIT# which is identical to system default paths) or in case of scanning a -listsoftware collection, you can even keep that blank since it internally uses #SOFTLIST#. You only need to store your software lists in rompath subfolders named after the list though.
For example you use: rompath: c:\mame\softwarelist and inside there are folders like a2600, a2600_cass, c64_flop_misc, amiga_hardware and so on which hold the files for the belonging lists.

« Last Edit: 24 August 2024, 19:00 by john iv »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3414
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 127.0.0.0 Chrome 127.0.0.0
    • View Profile
Re: New scanner, filtering
« Reply #3 on: 25 August 2024, 19:16 »

With the new scanner I'd suggest something I wouldn't suggest for cmpro when using software lists.

use a -listsoftware input and add 1 rompath pointing to your software list folder (where a2600, a7200 etc are subfolders) and scan. That's it....if you don't have all software lists you want to limit the output to not get flooded with hundreds of other missing software lists. Limit can be set with the filter option. For example with xpath expressions like "xp://softwarelist[contains(@description, 'Atari') or contains(@description, 'Commodore')]/software" to limit on Atari and Commodore :)

So no real need to use standalone hash files. On the other hand: MAME's -listsoftware output does not contain all data which is available in the hash folder....never understood the reason for it....but hardcore collectors will most likely know and have the other files available as well...

Currently switching the XML combo box (and it allows up to 10 settings) will restore the last used setting for the xml/exe. So you can somehow have 10 different profiles (e.g. 1 for MAME (-listxml) and 1 for MAME (-listsoftware) .... plus a handful for the rest....)...and yes...this will be changed as soon as a profiler is written....
Logged

Loraineaker

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 1
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 128.0.0.0 Chrome 128.0.0.0
    • View Profile
    • slope game
Re: New scanner, filtering
« Reply #4 on: 11 September 2024, 15:13 »

Changing the ROM path to directly point to c:\mame\software\a2600 is likely the quickest and easiest fix. However, if you need to manage multiple ROM directories or want more control, using filters or patterns can be very effective.
Logged

john iv

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 72
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 128.0.0.0 Chrome 128.0.0.0
    • View Profile
Re: New scanner, filtering
« Reply #5 on: 11 September 2024, 18:39 »

FWIW, after chatting w/ Roman in this thread and others I've started using the -listsoftware export as the most efficient workflow path for the ~32 machines I care about in the \software directory. This method brought forth some additional gotchas like shared roms prompting missing/partial messages so I had to get those too to quiet the scan.  Crafting a filter for that many varied directories/machines may not be viable.  But these are the machines I use if you see an easy way to filter. :)

xegs
a800
a800_cass
a800_flop
a2600
a2600_cass
a5200
a7800
astrocde
coleco
coleco_homebrew
gameboy
gamegear
gba
gbcolor
intv
intvecs
jaguar
lynx
megadriv
megatech
microvision
n64
neogeo
nes
ngp
ngpc
pce
sms
snes
tg16
vboy
vectrex
videopac
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3414
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 128.0.0.0 Chrome 128.0.0.0
    • View Profile
Re: New scanner, filtering
« Reply #6 on: 11 September 2024, 18:59 »

I still prefer xpath filters by companies...

xp://softwarelist[contains(@description, 'Commodore') or contains(@description, 'Atari')]/software

Guess you can add Nintendo and some more :-)
Logged

john iv

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 72
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 128.0.0.0 Chrome 128.0.0.0
    • View Profile
Re: New scanner, filtering
« Reply #7 on: 11 September 2024, 19:16 »

How long can the filter line be? Seems like it could get unwieldy in this case.

Hmm, is there an xpath argument for finding the machine name like 'a2600'?  Could you string together an xpath with the ~30 machine names in the list?
« Last Edit: 11 September 2024, 19:31 by john iv »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3414
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 128.0.0.0 Chrome 128.0.0.0
    • View Profile
Re: New scanner, filtering
« Reply #8 on: 11 September 2024, 19:50 »

xp://softwarelist[
    starts-with(@name, 'a2600')
or starts-with(@name, 'a800')
or @name='xegs'
...
...
...
]/software

yes, gets rather long...but so what :)
That's why I said, go by manufacturers....but yes, Commodore is more than C64 and Amiga....
« Last Edit: 11 September 2024, 19:50 by Roman »
Logged

john iv

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 72
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 128.0.0.0 Chrome 128.0.0.0
    • View Profile
Re: New scanner, filtering
« Reply #9 on: 11 September 2024, 20:11 »

Ok, laugh, it appears this xp gets me what I want just checking the existing \software directories and it doesn't bother with the shared found roms because of the explicit filter. :)

xp://softwarelist[@name='xegs' or @name='a800' or @name='a800_cass' or @name='a800_flop' or @name='a2600' or @name='a2600_cass' or @name='a5200' or @name='a7800' or @name='astrocde' or @name='coleco' or @name='coleco_homebrew' or @name='gameboy' or @name='gamegear' or @name='gba' or @name='gbcolor' or @name='intv' or @name='intvecs' or @name='jaguar' or @name='lynx' or @name='megadriv' or @name='microvision' or @name='n64' or @name='neogeo' or @name='nes' or @name='ngp' or @name='ngpc' or @name='pce' or @name='sms' or @name='snes' or @name='tg16' or @name='vboy' or @name='vectrex' or @name='videopac']/software
Logged
Pages: [1]   Go Up
 

Page created in 0.07 seconds with 21 queries.