1
clrmame Discussion / Re: My clrmame pro is in hospice. I think I killed it.
« on: 20 March 2023, 07:11 »
Thanks for the feedback....sounds weird for a new drive...hmm..maybe it has an encryption process enabled (bitlocker etc)?
- 23 March 2023, 18:16
- Welcome, Guest
News:
The new forum is online, hope you enjoy it!
Show Posts
This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.
2
clrmame Discussion / Re: My clrmame pro is in hospice. I think I killed it.
« on: 19 March 2023, 19:09 »
It lists found files, not roms, archives most of the time. This is done by a standard tree walk, so nothing special. It does not print out any set but maybe each 10th or 100th, can't remember.
Again a number for:...that "looking for unknown sets" at the start takes about 12-15 seconds (depends a bit how many rompaths you have).
During a scan I see HD activity at about 50% active time and between 5 and 20MB/s read speed (most of the time 5-9 with some small hits to 20).
That's on Windows 11, I7-8700K, 16GB and a Seagate IronWolf 4 TB HD
0.1MB/s sounds way way too slow...
No idea what's going on on your system. Try a different drive, a different machine, just to see how it performs there...(if possible)
Again a number for:...that "looking for unknown sets" at the start takes about 12-15 seconds (depends a bit how many rompaths you have).
During a scan I see HD activity at about 50% active time and between 5 and 20MB/s read speed (most of the time 5-9 with some small hits to 20).
That's on Windows 11, I7-8700K, 16GB and a Seagate IronWolf 4 TB HD
0.1MB/s sounds way way too slow...
No idea what's going on on your system. Try a different drive, a different machine, just to see how it performs there...(if possible)
3
clrmame Discussion / Re: My clrmame pro is in hospice. I think I killed it.
« on: 19 March 2023, 17:05 »
Operations are file not set based, so e.g. the rebuilder will - when it finds a hash match - transfer the one single file to the destination. As long as you're using zip files, this is done without recompressing the files (unless you turned on recompressing). For rar/7z files it's different, there, data will be temporarily unpacked and added. Solid 7z files are a pain, since it will require a recompression of the whole archive over and over again. So yes, zip (or decompressed sets) are fast, 7z/rar is slow, solid 7z archives are very very slow.
As I said, a normal scan with fix operations disabled should be fast since no archive operations are involved. If this already takes hours for you, you have a different problem. To give you a number....a first full MAME scan on a 6 year old i7 machine is approx. 5 minutes...a second one where most of the disk access is cached, 1 minute, using a HD (not SSD).
The normal (non-fix) scan tree output should already give you an impression of the state of your collection. If nearly each and any set is listed with issues, you may have chosen a wrong merge mode or you're scanning a wrong path which would result in thousands of archive changes....so double check before running fixes.
General speed tips:
- using the rebuilder to add new files is better than using fix-missing, since fix-missing looks up several sources and is slower in the end
- fix missing can be a time killer (e.g. in progetto snaps sets)
- use zip files (when using 7z or even solid 7z, a post operation may be better which then recompresses touched files in one go)
- use split sets
- turn off sha1 checking
- know from where to where you do rebuild operations, same disk can crawl (in this context, the cmpro temp folder is also a thing to take into consideration)
- for rebuilding, use the new standalone rebuilder ;-)
As I said, a normal scan with fix operations disabled should be fast since no archive operations are involved. If this already takes hours for you, you have a different problem. To give you a number....a first full MAME scan on a 6 year old i7 machine is approx. 5 minutes...a second one where most of the disk access is cached, 1 minute, using a HD (not SSD).
The normal (non-fix) scan tree output should already give you an impression of the state of your collection. If nearly each and any set is listed with issues, you may have chosen a wrong merge mode or you're scanning a wrong path which would result in thousands of archive changes....so double check before running fixes.
General speed tips:
- using the rebuilder to add new files is better than using fix-missing, since fix-missing looks up several sources and is slower in the end
- fix missing can be a time killer (e.g. in progetto snaps sets)
- use zip files (when using 7z or even solid 7z, a post operation may be better which then recompresses touched files in one go)
- use split sets
- turn off sha1 checking
- know from where to where you do rebuild operations, same disk can crawl (in this context, the cmpro temp folder is also a thing to take into consideration)
- for rebuilding, use the new standalone rebuilder ;-)
4
clrmame Discussion / Re: My clrmame pro is in hospice. I think I killed it.
« on: 19 March 2023, 14:40 »
A normal scan on a (more or less) modern drive usually takes (depens a bit on the sha1 settings) just a couple of minutes, so yes, something is wrong...
I'd go with a clean install (so all default options are set) and then simply do a scan without any fix options enabled. That should also be pretty fast and you already get an impression what is wrong with your collection. If too much appears, you may even have setup wrong rompaths...or a wrong merge mode (and fixing tries to recreate all the sets...). So checking the first non-fix scan would be a good idea to see if something is totally wrong configuration wise.
After that you should add missing stuff with the rebuilder and not via fix-missing (so keep that disabled).
Solid 7z archives may become a problem as soon as write operations are done on them since it would require a complete recompression of the files.
3rd party tools like virusscanner can interfear. If you're using a network drive, bad configurations can cause slow downs...even bad drivers can do.
A soon dying harddisk can also be a slow down factor....but don't worry...pretty rare case...
....and sure I recommend to use an official MAME import, no softwarelists, split sets and zipfile usage
I'd go with a clean install (so all default options are set) and then simply do a scan without any fix options enabled. That should also be pretty fast and you already get an impression what is wrong with your collection. If too much appears, you may even have setup wrong rompaths...or a wrong merge mode (and fixing tries to recreate all the sets...). So checking the first non-fix scan would be a good idea to see if something is totally wrong configuration wise.
After that you should add missing stuff with the rebuilder and not via fix-missing (so keep that disabled).
Solid 7z archives may become a problem as soon as write operations are done on them since it would require a complete recompression of the files.
3rd party tools like virusscanner can interfear. If you're using a network drive, bad configurations can cause slow downs...even bad drivers can do.
A soon dying harddisk can also be a slow down factor....but don't worry...pretty rare case...
....and sure I recommend to use an official MAME import, no softwarelists, split sets and zipfile usage
5
clrmame Discussion / Rebuilder 0.04 released
« on: 12 March 2023, 17:58 »
2023-03-12 V0.04 released
- support reading of (split)rar/(not split)7z and writing of 7z files
- detection of zip, 7z, rar, chd files by byte signature (instead of extensions, but
not within archives)
- selectable tempfolder in settings.xml
- minor speed up due to upfront matching size check
- updated various 3rd party libs, added 7z.dll
- ctrl-c will stop the rebuilding and cleans up temporary files/folders
- various internal cleanup
https://mamedev.emulab.it/clrmamepro/binaries/rebuilder_v004.zip
https://mamedev.emulab.it/clrmamepro/binaries/readme.html
- support reading of (split)rar/(not split)7z and writing of 7z files
- detection of zip, 7z, rar, chd files by byte signature (instead of extensions, but
not within archives)
- selectable tempfolder in settings.xml
- minor speed up due to upfront matching size check
- updated various 3rd party libs, added 7z.dll
- ctrl-c will stop the rebuilding and cleans up temporary files/folders
- various internal cleanup
https://mamedev.emulab.it/clrmamepro/binaries/rebuilder_v004.zip
https://mamedev.emulab.it/clrmamepro/binaries/readme.html
6
clrmame Discussion / Re: Diff Scan not usable with incomplete sets
« on: 12 March 2023, 10:57 »
Diff scan internally builds up a database which only holds the differences of a previous and a current dat....so you misinterpret the functionality, of course old missing stuff isn't part of it.
It can be used when you old datfile collection is complete.
You need to do a fullscan.
It can be used when you old datfile collection is complete.
You need to do a fullscan.
7
clrmame Discussion / Re: CMPro64 4.029b (2016-06-19) doesn't move chd from root to folder
« on: 11 March 2023, 09:03 »
Cool ;-)
By the way, since the rebuilder also understands a full -listsoftware mame xml file (mame.exe -listsoftware >252soft.xml) which includes all software lists (*) and automatically adds the software list name as a subfolder to the output path, you might even get rid of the for loop.
So you may end up with just:
rebuilder -x 252soft.xml -i o:\ROMs\backup -o o:\ROMs\MAMESL -m full -r -d
(*) not fully true, I guess -listsoftware still excludes some of the hash files in MAME's hash folder.....but haven't checked for years)
By the way, since the rebuilder also understands a full -listsoftware mame xml file (mame.exe -listsoftware >252soft.xml) which includes all software lists (*) and automatically adds the software list name as a subfolder to the output path, you might even get rid of the for loop.
So you may end up with just:
rebuilder -x 252soft.xml -i o:\ROMs\backup -o o:\ROMs\MAMESL -m full -r -d
(*) not fully true, I guess -listsoftware still excludes some of the hash files in MAME's hash folder.....but haven't checked for years)
8
clrmame Discussion / Re: Quick questions
« on: 27 February 2023, 08:05 »
I don't think I will touch clrmamepro anymore unless there are some real big issues with upcoming MAME releases.
The new rebuilder
(https://mamedev.emulab.it/clrmamepro/binaries/readme.html, https://mamedev.emulab.it/clrmamepro/binaries/rebuilder_v003.zip)
has a modern core and is way way way easier to maintain and of course it's just a start, a scanner should follow of course.
The new rebuilder
(https://mamedev.emulab.it/clrmamepro/binaries/readme.html, https://mamedev.emulab.it/clrmamepro/binaries/rebuilder_v003.zip)
has a modern core and is way way way easier to maintain and of course it's just a start, a scanner should follow of course.
9
clrmame Discussion / Re: Quick questions
« on: 26 February 2023, 09:43 »
There is no log or report if you turn off the parsing messages....in the past decade I don't think there was anything really bad though ;-)
Regarding window to front...oops sorry...didn't see that it was already part of an earlier post....unfortunately I don't think I will look into that soon. Real life takes a lot of my time at the moment and when I've got time I only look into the new rebuilder tool....which doesn't have windows to pop up ;-)
Regarding window to front...oops sorry...didn't see that it was already part of an earlier post....unfortunately I don't think I will look into that soon. Real life takes a lot of my time at the moment and when I've got time I only look into the new rebuilder tool....which doesn't have windows to pop up ;-)
10
clrmame Discussion / Re: Quick questions
« on: 25 February 2023, 12:23 »
Misc_DisplayLIErrors = 0 disables nearly all warnings during reading in a datfile, I guess fatal problems will be still shown.
When "Can't merge" messages are not shown, such sets will be automatically transformed to not-parent-clone-sets, so actually you won't see any warning later on and they are handled as stand alone sets.
in cmpro.ini there's an option "Adv_WindowToFront = on" which may solve your problem when you set it to off before starting cmpro....but actually I can't remember...that stuff is too old ;-)
When "Can't merge" messages are not shown, such sets will be automatically transformed to not-parent-clone-sets, so actually you won't see any warning later on and they are handled as stand alone sets.
in cmpro.ini there's an option "Adv_WindowToFront = on" which may solve your problem when you set it to off before starting cmpro....but actually I can't remember...that stuff is too old ;-)
11
clrmame Discussion / Re: CMPro64 4.029b (2016-06-19) doesn't move chd from root to folder
« on: 20 February 2023, 19:51 »
What is mamesl?
Sounds like some unofficial build which may produce a different output as the official one. Or it is a datfile which handles chd files differently.
There shouldn't be a problem with chds in official MAME .251 with or without software lists.
In general, if you place chds in a rompath root folder, the scanner (with all check/fix options enabled) detects wrong placed chds and moves them to their correct belonging (with subpath) path (don't allow chds on rompath root).
On the other hand, the all in one software list+mame mode isn't the best way to go since it's a pain to setup and usually takes way too long. Better use a single profiler per software list.
But of course, you can use the new rebuilder tool which is able to rebuild (or in that case move) chd files pretty quickly and supports MAME, MAME multi softwarelist and single software list mode out of the box.
Don‘t forget to name a concrete example which file was moved and how it was stored before. Best is to send over cmpro.ini, the settings cmp file for the used profile and the datfile (if no direct exe import) which was used. 4.047 had no relevant changes for the scanner at all.
Sounds like some unofficial build which may produce a different output as the official one. Or it is a datfile which handles chd files differently.
There shouldn't be a problem with chds in official MAME .251 with or without software lists.
In general, if you place chds in a rompath root folder, the scanner (with all check/fix options enabled) detects wrong placed chds and moves them to their correct belonging (with subpath) path (don't allow chds on rompath root).
On the other hand, the all in one software list+mame mode isn't the best way to go since it's a pain to setup and usually takes way too long. Better use a single profiler per software list.
But of course, you can use the new rebuilder tool which is able to rebuild (or in that case move) chd files pretty quickly and supports MAME, MAME multi softwarelist and single software list mode out of the box.
Don‘t forget to name a concrete example which file was moved and how it was stored before. Best is to send over cmpro.ini, the settings cmp file for the used profile and the datfile (if no direct exe import) which was used. 4.047 had no relevant changes for the scanner at all.
12
clrmame Discussion / clrmamepro 4.047 released
« on: 20 December 2022, 12:03 »
4.047
misc: updated ziparchive (4.6.9) and unrar (6.20.3)
misc: show warning when trimming removed a whitespace rom names with subfolders
misc: updated ziparchive (4.6.9) and unrar (6.20.3)
misc: show warning when trimming removed a whitespace rom names with subfolders
13
clrmame Discussion / Re: Trimming of a leading space in a filename
« on: 18 December 2022, 15:15 »
Actually I saw that a new zip archive lib is out.....so I got a reason to update cmpro....and quickly added your request
14
clrmame Discussion / Re: Trimming of a leading space in a filename
« on: 17 December 2022, 12:00 »
Possible yes, however pretty unlikely since there is no real development going on these days.....I put it on the list though
15
clrmame Discussion / Re: Trimming of a leading space in a filename
« on: 16 December 2022, 09:31 »
Ok, I had a closer look.
If you have a datfile with "bla\ blabla", the space gets already trimmed when the datfile is loaded/parsed.
If you have a datfile with "bla\ blabla", the space gets already trimmed when the datfile is loaded/parsed.
16
clrmame Discussion / Re: Trimming of a leading space in a filename
« on: 15 December 2022, 13:54 »
Well, most likely the archive operation (where maybe a file unpack to disk is involved) kills the space since I assume that 7z / zip unpack will create the file on disk without the leading space....and then the file is repacked to the destination. Maybe even the OS itself kills the space, too.
So...what are your rebuilder settings, if 7z is involved, the files will definetly be unpacked first and recompressed via 7z.exe from your disk....
So...what are your rebuilder settings, if 7z is involved, the files will definetly be unpacked first and recompressed via 7z.exe from your disk....
17
clrmame Discussion / Re: Trimming of a leading space in a filename
« on: 15 December 2022, 10:36 »
Erm..no, cmpro does not change anything!
If the dat holds the "\ ", then it's taken over. So again: how was the dat being generated?
If you've exported it with cmpro, then the original dat (before merge) holds the " " or you have a space in your naming patter (profiler settings: %f\%1 -> %f\ %1)
If the dat holds the "\ ", then it's taken over. So again: how was the dat being generated?
If you've exported it with cmpro, then the original dat (before merge) holds the " " or you have a space in your naming patter (profiler settings: %f\%1 -> %f\ %1)
18
clrmame Discussion / Re: Trimming of a leading space in a filename
« on: 15 December 2022, 10:20 »
But wouldn't it then be a job for the generator to trim?
19
clrmame Discussion / Re: Trimming of a leading space in a filename
« on: 15 December 2022, 09:33 »
Erm...since when does MAME use subfolders in its xmls?
Anyhow, cmpro does support them but for cmpro the rom name is the full (relative) path, including "dogfights\"...so a space in the middle somewhere is not checked at all since normally it's valid. I think I do trim the complete entry though (left/right)...but definetly not checking anything inside.
I take a note, but don't expect anything anytime soon.
Anyhow, cmpro does support them but for cmpro the rom name is the full (relative) path, including "dogfights\"...so a space in the middle somewhere is not checked at all since normally it's valid. I think I do trim the complete entry though (left/right)...but definetly not checking anything inside.
I take a note, but don't expect anything anytime soon.
20
clrmame Discussion / Re: Homepage not avaible!?
« on: 15 December 2022, 07:08 »
Actually I wonder if this also fixes the auto update problems which some users reported over the last years....