Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

61

clrmame Discussion / Re: Scan deleted my softwarelist.xml

« on: 25 January 2025, 19:45 »

Well it simply assumed that the xml file comes from a mame.exe -listsoftware and so it removed the xml file….under the assumption that it recreates it…..but your case is a xml file which was created outside of clrmame

62

clrmame Discussion / Re: Scan deleted my softwarelist.xml

« on: 25 January 2025, 11:07 »

https://mamedev.emulab.it/clrmamepro/binaries/clrmameUI_20250125.zip

that should do it

63

clrmame Discussion / Re: Scan deleted my softwarelist.xml

« on: 24 January 2025, 19:05 »

I will have a look at it over the weekend or early next week....real life takes over at the moment....

64

clrmame Discussion / Re: Scan deleted my softwarelist.xml

« on: 24 January 2025, 18:11 »

Your log says:

2025-01-23 13:10:02   info   hash folder change detected, reloading data

That's happening when it detects a change in the hashfolder.
For detecting a change it builds up a hash from the *.xml files in the specified hashfolder (or MAME's exe hashfolder but I know you're using the settings.xml setting if I'm not mistaken) and it's last written timestamp. If that differs, old export file is removed and gets reimported.

So.....the XML (export) file and MAME's hash files (and your specified hash folder) shouldn't be the same

65

clrmame Discussion / Re: Scan deleted my softwarelist.xml

« on: 24 January 2025, 13:38 »

Did you touch/add/remove files from the hash folder? If so, such changes are being detected and it regets data

66

clrmame Discussion / clrmame v0.2 released

« on: 20 January 2025, 19:59 »

https://mamedev.emulab.it/clrmamepro/binaries/clrmame_v02.zip
https://mamedev.emulab.it/clrmamepro/binaries/readme.html

UI:
- added: #thread selector
- added: scanner, Clipboard->Copy Machine Names option
- fixed: rebuilder log Copy To Clipboard misses linefeed

Core:
- added: you can define a thread pool size which is used for parallel scanning rebuilding/etc. Either you keep auto (default value) or pick the number of threads used for the pool yourself. "auto" picks a high value when you work with compressed files and a low value when using decompressed ones. A low value is better when you run into file seek overhead issues. If you're not happy with auto, you can try to tweak it yourself.
- fixed: scanner and rebuilder ignore output path, mergemode, pattern changes when xml data hasn't changed
- misc: more effective reimplementation of thread pools and thread queueing
- misc: prefetching sha1s on an archive if needed (speed increase, espically for archives with lots of files)
- misc: rebuilder: major update / reworked internal logic, less complex, way smaller memory footprint especially for dats out of hell (dats with thousands of dupe files), faster.
- misc: scanner, smaller memory footprint and speed update for "dats out of hell"
- fixed: scanner, accidently getting a crc32 on sub folders showing an error
- fixed: scanner, detection of unnneeded empty sub folders failed
- fixed: scanner, using old archive name when trying to remove some unneeded files causing can't remove files/can't access messages
- misc:  updated bit7z to 4.0.9
- misc:  updated pugixml to 1.15
- misc:  use "-" instead of "_" for replacing illegal file chars (cmpro align)
- misc:  monitoring hash folder for -listsoftware exports. In case of a change, cached data is ignored and a new export is generated. This is more for users which update their hash folders during the development phase of MAME on a regular basis and don't want to recompile the binary.
- misc: use one version number, fixed a tooltip typo ;-)

67

clrmame Discussion / Re: Using CMPro but Addpath doesn't appear to be working

« on: 12 January 2025, 19:23 »

Yep no crc32 or sha1, no add/rebuild/etc

68

clrmame Discussion / Re: Using CMPro but Addpath doesn't appear to be working

« on: 12 January 2025, 18:54 »

How does the datfile looks like? Samples llisted as samples (sounds weird eh? :-)) don‘t have a hashvalue listed and so they can‘t be detected or rebuilt.

69

clrmame Discussion / Happy New Year...

« on: 12 January 2025, 14:52 »

Well well well....first of all...Happy New Year...2025.....a little bit late but anyhow, I'm alive and working...

A new version is somewhere around the corner.....I've spent the last days on doing some major rework on the rebuilder. Why? Well, it really performed lousy for "dats out of hell"....
For example, the progetto icons datfile which has 34000 icons where roughly 5000 are dupes....but running such a colleciton through the rebuilder means that it finds ~5000 matches for each of the ~5000 files....the rebuilder in clrmame wasn't well designed for that....and the memory footprint went up very high....and of course the rebuilding time.

The good thing is: The new implementation is way better. Plus it's also faster than the old one.
Just to give an impression: old cmpro rebuilder on the icons: 1m57s
clrmame rebuilder (while being fine on other stuff it totally failed here): not worth mentioning....I quit after an hour ;-)
reworked clrmame rebuilder for the next version: 1m42s

I have some minor stuff on my list before doing a release...but here's already a little sneak peak:



UI:
- added #thread selector

Core:
- misc: major rebuilder update / reworked internal logic, less complex, way  smaller memory footprint especially for dats out of hell (dats with thousands of dupes), faster.
 
- misc: reimplementation of thread pools
 
- added: you can define a thread pool size which is used for parallel scanning rebuilding/etc. Either you keep auto (default value) or pick the number of threads used for the pool yourself. "auto" picks a high value when you work with compressed files and a low value when using decompressed ones. A low value is better when you run into file seek overhead issues. If you're not happy with auto, you can try to tweak it yourself.

- fixed: scanner and rebuilder ignore output path, mergemode, pattern changes when xml data hasn't changed
- fixed: scanner, accidently getting a crc32 on sub folders showing an error
- fixed: scanner, detection of unnneeded empty sub folders failed
- fixed: scanner, using old archive name when trying to remove some unneeded files causing can't remove files/can't access messages
- misc:  updated bit7z to 4.0.9
- misc:  use "-" instead of "_" for replacing illegal file chars (cmpro align)
- misc:  monitoring hash folder for -listsoftware exports. In case of a change, cached data is ignored and a new export is generated. This is more for users which update their hash folders during the development phase of MAME on a regular basis and don't want to recompile the binary.
- misc: use one version number ;-)

70

clrmame Discussion / Re: Why i can post to this forum !!?!!?

« on: 05 January 2025, 13:05 »

Well, the forum got some spam protection which is annoying, yes. Usually you can easily 'fix' it by logging out and relogging in.
And if it says "session time out while posting" you can simply hit the post button again.
I can't change it since I'm not the owner of the board.

I've added your requests to my list.....

71

clrmame Discussion / Re: Trouble with clrmame pro in "directory_iterator"

« on: 28 December 2024, 21:21 »

Sounds weird. I've just setup a mapped network drive to a server far far away with a lousy connection and VPN involved and it works fine.
Maybe ensure that the drive/files are accessible by opening them in an explorer window right before a scan.

Do you have more information about the network? Is SAMBA involved (since it always caused timing problems in the last decades.....guess you can even search the forum for that....), drivers up to date, etc etc...
Try to setup 1 folder with just one set and test that one....maybe something is in your folder which causes the problem and the exception avoids continuing with the other files.....

72

clrmame Discussion / Re: Trouble with clrmame pro in "directory_iterator"

« on: 28 December 2024, 20:29 »

Searching the internet gets you:

Possible Causes:
The "Invalid Signature" error could occur when the system or the library encounters an unexpected or malformed response from the network drive during directory iteration.
You may not have the appropriate permissions to access certain directories or files on the network drive, especially if network authentication or certain security settings are involved.
Even if the network drive is mapped correctly, certain directories might require elevated privileges or additional handling that your program doesn't support.
Mapped network drives can sometimes be unreliable, especially if the connection is slow or unstable. std::recursive_directory_iterator may attempt to access files or directories that are temporarily unavailable due to network latency or disconnects.

Solutions
Check Network Drive Availability:
Before using std::recursive_directory_iterator, ensure that the network drive is fully accessible and stable. You can check if the network share is mounted and if you can access the drive manually (e.g., through a file explorer).
Verify Permissions:
Ensure that your program has sufficient permissions to access the directories and files on the network drive. You may need to run your program with elevated privileges or adjust security settings on the network share.
Timeouts and Network Configuration:
Ensure that your network connection is stable and that your program is not timing out or facing issues with slow network drives. Depending on your network settings, it might be useful to implement retries or set timeouts in case the network drive is temporarily unreachable.

73

clrmame Discussion / Re: Trouble with clrmame pro in "directory_iterator"

« on: 28 December 2024, 09:14 »

Hmmmm….maybe double check access rights of the files in the folders?

74

clrmame Discussion / Re: clrmame (scanner 0.08.1, rebuilder 0.15.1) released

« on: 17 December 2024, 10:51 »

"Only works on Split. Full and Standard still on Split"

? ? ?

75

clrmame Discussion / Re: Confirmation of odd query RE: cmpro.ini

« on: 15 December 2024, 11:43 »

yep, cmpro.ini is saved on exit

76

clrmame Discussion / Re: Confirmation of odd query RE: cmpro.ini

« on: 14 December 2024, 11:52 »

Sorry, but I don't understand the problem. Where exactly is the problem of not having a title bar....
Either you want to close the app, then ALT-F4 for closing should do it when not having a close button or you don't want to quit the app ;-)

77

clrmame Discussion / Re: to thread or not to thread that is the question

« on: 11 December 2024, 21:03 »

ok..some quick test runs are done....on a HD

Scenario 1, typical MAME style, lots of sets with some files per set, compressed
Scenario 2, typical Progetto style, small amout of sets with huge number of files per set, decompressed

Using let's say 28 versus 1 thread for Scenario 1, the more threads, the faster it gets.......good...end of story

For Scenario 2, you run into a real speed decrease as soon as you use more than 1 thread (in a concrete example of ~40000 files spread over 12 sets we talk about 5 (1 thread) versus 22 minutes (28 threads)

Now you'd say: ok, when you have decompressed files, use 1 thread...but sorry...it's not that easy. It depends on the number of files per set, the file sizes and how they are stored on the hd. For scenario 1 but in a decompressed form, you can easily get better results with more threads....*sigh*

The threads work on sets, so for Scenario 2, it tries to scan all 28 (ok there are only 12 available) sets (each with lots of files in it) in parallel leading to massive seeking overhead.
For Scenario 1, compressed, it scans 28 sets which (means just 28 files)  in parallel which seems to work pretty fine on a hd.

I think a general "use max threads when running on compressed data, "use 1 thread when running on decompressed data" can be a good start....maybe with some manual overwrite by the user if wanted....will see.....

78

clrmame Discussion / to thread or not to thread that is the question

« on: 11 December 2024, 09:36 »

Sorry, no profiler news at the moment.
Oddi's remark about slowlyness when scanning progetto snaps keeps me a bit busy these days. Not only that I already found some hash lookups which can be optimized a bit, I wonder how threading can influence the scanning speed.
Now John IV would say: I'm getting 1 second for a full scan, so threading rocks. yes, it does...when you're on a sdd...when you're on a hdd, too many threads may lead to too many seek operations which may slow the process down.
In case of an unpacked progetto collection (we talk about > 300000 files), each file's crc32 needs to be calculated....which is a lot seeking, reading and calculating (even if the actual calc is quickly done). Using too many threads may slow down the process....so currently I will do some benchmarking what the best value is.....I keep you updated with the output. Even if I see that too many threads slow it down, I somehow need a way to determine a good value (ok, I can show an input box where you simply define it as last possible solution ;-))

79

clrmame Discussion / Re: a nightly cmpro...

« on: 11 December 2024, 09:29 »

I think the reason for having a parent/clone relationship hacked in cmpro was that there were some device romof which actually shared identical files....can't really rememeber.
If I find a little time I will double check it. For now, I'd go with the new solution....no "cloneof" -> no parent/clone relationship. "romof" however plays a role for dependencies (e.g. in standalone mode). BIOSes are included that way for example. Open for comments ;-)

80

clrmame Discussion / Re: a nightly cmpro...

« on: 09 December 2024, 07:02 »

Ah....devices with romof relations but not cloneof....

hd44780
hd44780u romof hd44780
sed1278   romof hd44780
....

clrmamepro hacked in a parent/clone relationship based on the "romof" statement. But actually hd44780 is not the parent of sed1278 /etc.


the new tools know about linked dependencies (following the romof chain) but for "split or full merge" mode, the cloneof attribute is used.

That's why old cmpro complains about the files (since it wants them in the fake parent), while the new scanner keeps them separated...