EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Roman

Pages: 1 2 3 4 5 6 [7] 8 9 10 11 12 ... 174
121
ok that's interesting because I thought it shows something like "missing but fixable rom" and not just a "missing" one...will check that...your xml is just a "mame.exe -listsoftware" I guess?
Hiding...well...only by using filter to limit the output to software lists you got....xpath expressions are powerful ... depending on how many SL you have it's either better to create a not expression or one which lists the ones you have....

122
Ah...nice finding. -listsoftware and the hashfile for e.g. neogeo do differ in regards to dataarea entries:


<dataarea name="sprites" size="0x800000">
            <rom loadflag="load16_byte" name="043-c1.c1" offset="0x000000" size="0x100000" crc="bad2d67f" sha1="04928e50ca75b7fbc52b64e816ec5701901f5893" />    <!-- TC5316200 -->
            <rom size="0x100000" offset="0x400000" loadflag="continue" />
            <rom loadflag="load16_byte" name="043-c2.c2" offset="0x000001" size="0x100000" crc="a7fbda95" sha1="9da3c5faf22592a7eaf8df9fa6454f48c2a927ae" />    <!-- TC5316200 -->
            <rom size="0x100000" offset="0x400001" loadflag="continue" />
            <rom loadflag="load16_byte" name="043-c3.c3" offset="0x200000" size="0x100000" crc="f00be011" sha1="2721cdba37a511a966a2a53b9bd6240f181d920c" />    <!-- TC5316200 -->
            <rom size="0x100000" offset="0x600000" loadflag="continue" />
            <rom loadflag="load16_byte" name="043-c4.c4" offset="0x200001" size="0x100000" crc="1887e5c0" sha1="9b915359add7c10c78d8b281b4084eceea8f0499" />    <!-- TC5316200 -->
            <rom size="0x100000" offset="0x600001" loadflag="continue" />
         </dataarea>


versus

            <dataarea name="sprites" size="8388608">
               <rom name="043-c1.c1" size="2097152" crc="bad2d67f" sha1="04928e50ca75b7fbc52b64e816ec5701901f5893" loadflag="load16_byte"/>
               <rom name="043-c2.c2" size="2097152" crc="a7fbda95" sha1="9da3c5faf22592a7eaf8df9fa6454f48c2a927ae" loadflag="load16_byte"/>
               <rom name="043-c3.c3" size="2097152" crc="f00be011" sha1="2721cdba37a511a966a2a53b9bd6240f181d920c" loadflag="load16_byte"/>
               <rom name="043-c4.c4" size="2097152" crc="1887e5c0" sha1="9b915359add7c10c78d8b281b4084eceea8f0499" loadflag="load16_byte"/>
            </dataarea>


looks like there is a bug in the new tools when parsing the "loadflag=continue" part


should be an easy quick fix....but surely I will first check if megadrive is the same problem....

i7000_card and msx1_card....well...click the
  • in front of the list and you will see what's wrong...I assume that you have a possible missing but fixable files in there. Some software lists share files....e.g. Atari ST and Amiga have some identical ones...i7000_card got some in common with coleco (sha1 5e2b96c19c4f5c63a5afa2de504d29fe64a4c908) and so on....so yes, you might already have fill-ins for sl lists which you don't to collect (yet).

123
I will have a look at it tomorrow.....

124
See my previous post, yes, you can scan them (the SL ones) in one go...and no, there are not all roms in the -listsoftware export which are available as hash files...but I think it's more about complete hashfiles being not included, not single roms in it....ask MAME Devs...I haven't checked that for years...

If you're only interested in some SL, you can use the filter option or you can use the contextmenu to hide fully missing software lists etc...

125
clrmame Discussion / Re: New scanner, filtering
« on: 25 August 2024, 19:16 »
With the new scanner I'd suggest something I wouldn't suggest for cmpro when using software lists.

use a -listsoftware input and add 1 rompath pointing to your software list folder (where a2600, a7200 etc are subfolders) and scan. That's it....if you don't have all software lists you want to limit the output to not get flooded with hundreds of other missing software lists. Limit can be set with the filter option. For example with xpath expressions like "xp://softwarelist[contains(@description, 'Atari') or contains(@description, 'Commodore')]/software" to limit on Atari and Commodore :)

So no real need to use standalone hash files. On the other hand: MAME's -listsoftware output does not contain all data which is available in the hash folder....never understood the reason for it....but hardcore collectors will most likely know and have the other files available as well...

Currently switching the XML combo box (and it allows up to 10 settings) will restore the last used setting for the xml/exe. So you can somehow have 10 different profiles (e.g. 1 for MAME (-listxml) and 1 for MAME (-listsoftware) .... plus a handful for the rest....)...and yes...this will be changed as soon as a profiler is written....

126
clrmame Discussion / Re: New scanner, filtering
« on: 24 August 2024, 07:03 »
Interesting that it crashes when temp fails due to ramdrive is filled up....Will check that, however I wonder why so much files go to temp?

So you're scanning a MAME collection (which most likely has no or nearly no a2600 files) and a full software list collection (same here...a2600 files only in the a2600 subfolder...maybe some in related 2600 atari machines)....so it should find millions of unneeded.

First question: why do you scan a a2600 hashfile over all paths? if you scan a 2600.xml, use the softwarelist a2600 path for it :-)

You may better use the other paths as addpath, so they act as possible fill ins for missing files.


Filtering: well...filtering works on the loaded xml, so you can define which machines in the loaded xml should be active or not. The problem in your setup is that you have countless other folders (standard mame plus all the other software lists).
So with filtering you define subsets of the loaded data, either via regexpression, xpath or file. So e.g. in a softwarelist export you only take the Commodore and Atari software lists.

Patterns play a role when you have weird setups, like split up romcollections by bios (similar as system default paths in old cmpro). When using a softwarelist export you automatically have a not visible #SOFTWARELIST# pattern, so it uses a storing mechanism rompath\softwarelistname\setname....

So...if you load a a2600 standalone hash file you should have 1 rompath pointing to the a2600 folder.
If you load a softwarelist export and only want to check the a2600 files, point to your software list (parent) folder and use an xpath filter which only filters on a2600.

127
Hi John!

1) currently there is no purge. It's on the to do list and I was already thinking about something like "older than a month" or something....but surely it will come

2) I know what you're talking about....same here...I'd like to cleanup backup/addpaths too, currently it's doing a dummy rebuild job afterwards....yes...you're requests are heard (actually they are already on the list)

128
https://mamedev.emulab.it/clrmamepro/binaries/readme.html

https://mamedev.emulab.it/clrmamepro/binaries/scanner003_rebuilder010.zip

Core:

- added: long path/filename support (32k)

- misc:  update to bit7z 4.0.8

- misc:  scanner: propagate changed file timestamp up to rom/samplepath level
- misc:  scanner: improved preferedPackMethod a bit
- misc:  scanner: whenever possible, move (instead of copy/delete) missing but fixable chds or decompressed roms

- fixed: erroneously allow \ as an machine name character

- fixed: scanner, stopping the scan during fix operations doesn't have an effect
- fixed: scanner, freeze when fixing unneeded decompressed files (or chds) (0.02.1)
- fixed: scanner, wrong named files which are also fill-ins for missing files can trigger an obsolete file removal operation (sdiamond MAME.268)
- fixed: scanner, unneeded files/folders contain multiple identical entries (aa3020 MAME.268)
- fixed: scanner, fixing a wrong named folder failed when additional pattern folders were involved (copy error)
- fixed: scanner, fixing a wrong named folder failed when folder can have multiple new names (aa3020/a3010/aa5000 MAME.268)  (move error)
- fixed: scanner, prefered path lookup for software list collections never returned additional pattern folders (copy error)
- fixed: scanner, prefered path lookup for wrong named machines which only consist of chds failed (freeze during backup)
- fixed: scanner, removal of unneeded files can result in an empty folder which isn't removed

UI:

- added: scanner, always have software list collection sublevels (before a complete sl list only showed the complete top level)
- added: scanner, context menu option to auto expand scan results tree
- added: scanner, context menu list options are now also available on software list level

- fixed: active paths are not always remembered correctly when switching xmls or on initial load

- fixed: scanner, empty sofware lists show complete instead of empty icon
- fixed: scanner, scan results tree/log split isn't refreshed after minimizing/maximizing window




and in case you're interested in old cmpro:

4.049

fixed: falsely remove backup's _unknown folder in case of a failed backup which might remove already backuped and removed unneeded files. Only if _unknown folder isn't present before.
fixed: skip a non software list rompath if it is also assigned to a non active software list which would list all sets in that path as fully missing (combined mode only)
fixed: missing disk merging information within softwarelists, so that some identical chds stay in the clone set (combined mode only)
added: verify delta chds (via chdman)

129
Sneak peek....(new version most likely next week...)

Core:
- added: long path/filename support (32k)

- misc:  scanner: propagate changed file timestamp up to rom/samplepath level

- fixed: erroneously allow \ as an machine name character
- fixed: scanner, freeze when fixing unneeded decompressed files (or chds) (0.02.1)
- fixed: scanner, wrong named files which are also fill-ins for missing files can trigger an obsolete file removal operation (sdiamond MAME.268)
- fixed: scanner, unneeded files/folders contain multiple identical entries (aa3020 MAME.268)
- fixed: scanner, fixing a wrong named folder failed when additional pattern folders were involved (copy error)
- fixed: scanner, fixing a wrong named folder failed when folder can have multiple new names (aa3020/a3010/aa5000 MAME.268)  (move error)
- fixed: scanner, prefered path lookup for software list collections never returned additional pattern folders (copy error)
- fixed: scanner, prefered path lookup for wrong named machines which only consist of chds failed (freeze during backup)
- fixed: scanner, removal of unneeded files can result in an empty folder which isn't removed

UI:
- added: scanner, always have software list collection sublevels (before a complete sl list only showed the complete top level)
- added: scanner, context menu option to auto expand scan results tree
- added: scanner, context menu list options are now also available on software list level

- fixed: active paths are not always remembered correctly when switching xmls or on initial load
- fixed: scanner, empty sofware lists show complete instead of empty icon
- fixed: scanner, scan results tree/log split isn't refreshed after minimizing/maximizing window

130
Thanks for the screenshots, I will definetly look into the folder and not-showing-empty issue.
Regarding -listsoftware vs MAME hashfolder issue: Yes, MAME's -listsoftware output does not include all hashfiles. The reason: Well, guess you need to ask a MAMEDev about it. Maybe some flag specifying that a hashfile should not yet be included in the full list or something. I don't have an answer for that, but it's known since the beginning of -listsoftware ;)

Update #1: Well, the output of the fmtowns empty folders is correct though. It finds a zip file and the empty folder for a machine where the machine doesn't have any chds on its own. So it says it founds a dupe (multiple machine) and does not remove any of the two. The actual problem is when it moved/removed the chd before, there it forgets to remove the folder.

Update #2: Showing (complete and/or empty) machines on completed sl level is now in
Update #3: Auto expand tree feature is now in

Update #4: the removal of the empty folders is now fixed... (besides other stuff) ... now I only got 3 minor nice-2-have things from my side to add before I put out a new version....still no combined rebuilder/scanner though.

131
Sure you got "complete" enabled?
Because I only see the fmtowns output and since there is no filtering, what about all the complete software lists?

If a software list is complete, you'd only see a filled green circle for e.g. "Commodore 128 catridges", and no sub machines in there even no empty ones.
This is most likely hiding empty machines.

I can create a list of the 114 "empty" ones if you like but I guess they simply belong to software lists which aren't listed (and are complete if I'm not mistaken). Maybe I add an option also show complete machines on software list level, however depening on what you got, the tree might be relatively big which takes time to render (unless I make it dynamic with a lazy-draw-on-expand thing)...

132
I will have a look at the fmtowns_cd issue with keeping the empty chd folder. A confirmation prompt for fixes .... I put it on the wish list, not planned yet ;-)
The 114 empty ones and they don't appear...interesting....no idea at the moment...I will have a look ;)

Thanks for testing / reporting

133
ah, so it's the same problem for gta and idanthology....Had another look at the filenames again...yeah...so...for now, trust the new scanner and not the combined old cmpro mode ;-)
Thanks for testing and reporting.

134
clrmame Discussion / Re: CHD scanning time with cmpro
« on: 08 August 2024, 07:58 »
Yep, it's all about speed ;-)
Actually it's not hard to code at all (we're not talking about solid archives here), it's just some modifying header entries and moving data blocks....Before using the zipArchive class routines I had an own routine back in the 90s ;-)
I still hope that bit7z (currently used for 7z and rar) might include it at least for zip so I'd be able to only have one 3rd party lib for archives....

135
ok...two problems here:

1) id Anthology and gran turismo 2, well, currently I don't have an idea why they popped up as unneeded now for you. The information hasn't changed in the 287-288 cycle....Can you verify the chd if it really matches (e.g. a chdman info / verify...or a rebuild with the new rebuilder ;-)). If something failed while reading the header information of these files, it might lead to the unneeded message...but as I said...currently no idea.

2) the general issue with the clone chds. Old cmpro's full-combined mode is wrong here (so most likely the old -listsoftware list parser). It does not apply merging information on the disks. If the cdi hash file is loaded as a single profile it's ok.See attached screenshot which shows the difference of the combined and single mode (at lease you can spot the different texts colors indicating a clone file or not). I guess I can fix that in old cmpro. So well, the new scanner is right, old cmpro combined mode is wrong.



Update: I've already fixed 2) (collecting other stuff before a release happens though)

136
Hi, first of all, thanks for the donation.

Long thread, so I hope I can answer everything here...

> "I like the idea of a full merged ROMset, but ClrMAMEPro 4.048d can't do it the way I want, with clone subfolders"

You can do this. It automatically does it when you got merge collisions (same filename, different hashes inside a parent/clone relationship). If you want subfolders everywhere, you need to enable settings->full merge mode -> hash collision mode

>"replacing commas by periods in some machines' refresh rates"

well, yes and no. the tools do some changes to things which would lead to a bad filename (e.g. "." can't be at the end of a filename in Windows or |,*,? can't be within a filename, that's why they get replaced for machine/rom/disk/sample names). The refresh rates , . replacements are however done by pugixml which does some localized transformation of decimal numbers automatically. No big deal though.

> "After MAME finished verifying both collections, I had 47,016 txt files on each drive. 5,323 of them had 0 bytes"

Erm...does MAME creates them? Because cmpro / new tools don't create dummy sets

> "kizuna4p is a clone of kizuna, but kizuna4p.zip isn't created by ClrMAMEPro"

old cmpro has an option for "fake or dummy clones". Profiler->Options->Parsing Options->Don't create dummy clones.
new tools would handle them as sets on their own


>"Side note: I am puzzled as to why some machines which don't have ROMs receive OK"

Well, guess you need to ask MAME devs why MAME's audit does so. Yes, one is a "non runnable" device. While the other is a machine also representing a softwarelist holder.

>"Continuing, the new rebuilder inserted 13 files marked as NO GOOD DUMP KNOWN inside 11 zipfiles"

I can't repeat this. Can you tell me your rebuilder settings? Can you repeat it? If yes, can you send me one of the input folder archives which lead to this?

>"It warned me about misplaced and wrongly named CHD disks".

Would be interesting to know which ones. If the rebuilder created them, they should be in the correct place. Can you give me information about which files are affected and what the message is? Maybe run a non-fix scan and send me the scan result (either from "scans" folder or a copy to clipboard via context menu)




137
clrmame Discussion / Re: CHD scanning time with cmpro
« on: 06 August 2024, 19:36 »
Well, I mainly use ZipArchive because I'm used to it and it supports copying data (single/multi files) from one archive to another without recompression ;-)

138
clrmame Discussion / Re: CHD scanning time with cmpro
« on: 06 August 2024, 18:27 »
The UI is currently more or less a throw-away ui...which was quickly done in (yuck) MFC.
The separated core is stl /c++ 20, plus 3rd party libs which are portable.

Yes, one of the next steps is to combine scanner and rebuilder and add some profiler. Currently I'm doing some smaller things first, one minor thing which was found during MAME's .268 cylce and 32k path length support. After that I try to combine the 2 modules and after that the profiler will come....

139
clrmame Discussion / Re: CHD scanning time with cmpro
« on: 06 August 2024, 08:08 »
hmm...it doesn't work in WINE? Why so? Any error message?

140
clrmame Discussion / Re: CHD scanning time with cmpro
« on: 05 August 2024, 18:23 »
Thanks for the files but testing leads to the same result.
It's fast. I can't and won't test it under Wine though,plain Windows 11 and a network folder as rompath (\\fileserver\bla...).

As I said: When reading a chd file only 128 byte are read to get the header and the stored sha1 values from it and if you say it worked way better before, maybe something else (e.g. Wine) has changed. Sorry, but I don't have an idea and can't repeat the problem.

Give the new scanner a try.

Pages: 1 2 3 4 5 6 [7] 8 9 10 11 12 ... 174

Page created in 0.094 seconds with 20 queries.