Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

141

clrmame Discussion / Re: Why i can post to this forum !!?!!?

« on: 05 January 2025, 13:05 »

Well, the forum got some spam protection which is annoying, yes. Usually you can easily 'fix' it by logging out and relogging in.
And if it says "session time out while posting" you can simply hit the post button again.
I can't change it since I'm not the owner of the board.

I've added your requests to my list.....

142

clrmame Discussion / Re: Trouble with clrmame pro in "directory_iterator"

« on: 28 December 2024, 21:21 »

Sounds weird. I've just setup a mapped network drive to a server far far away with a lousy connection and VPN involved and it works fine.
Maybe ensure that the drive/files are accessible by opening them in an explorer window right before a scan.

Do you have more information about the network? Is SAMBA involved (since it always caused timing problems in the last decades.....guess you can even search the forum for that....), drivers up to date, etc etc...
Try to setup 1 folder with just one set and test that one....maybe something is in your folder which causes the problem and the exception avoids continuing with the other files.....

143

clrmame Discussion / Re: Trouble with clrmame pro in "directory_iterator"

« on: 28 December 2024, 20:29 »

Searching the internet gets you:

Possible Causes:
The "Invalid Signature" error could occur when the system or the library encounters an unexpected or malformed response from the network drive during directory iteration.
You may not have the appropriate permissions to access certain directories or files on the network drive, especially if network authentication or certain security settings are involved.
Even if the network drive is mapped correctly, certain directories might require elevated privileges or additional handling that your program doesn't support.
Mapped network drives can sometimes be unreliable, especially if the connection is slow or unstable. std::recursive_directory_iterator may attempt to access files or directories that are temporarily unavailable due to network latency or disconnects.

Solutions
Check Network Drive Availability:
Before using std::recursive_directory_iterator, ensure that the network drive is fully accessible and stable. You can check if the network share is mounted and if you can access the drive manually (e.g., through a file explorer).
Verify Permissions:
Ensure that your program has sufficient permissions to access the directories and files on the network drive. You may need to run your program with elevated privileges or adjust security settings on the network share.
Timeouts and Network Configuration:
Ensure that your network connection is stable and that your program is not timing out or facing issues with slow network drives. Depending on your network settings, it might be useful to implement retries or set timeouts in case the network drive is temporarily unreachable.

144

clrmame Discussion / Re: Trouble with clrmame pro in "directory_iterator"

« on: 28 December 2024, 09:14 »

Hmmmm….maybe double check access rights of the files in the folders?

145

clrmame Discussion / Re: clrmame (scanner 0.08.1, rebuilder 0.15.1) released

« on: 17 December 2024, 10:51 »

"Only works on Split. Full and Standard still on Split"

? ? ?

146

clrmame Discussion / Re: Confirmation of odd query RE: cmpro.ini

« on: 15 December 2024, 11:43 »

yep, cmpro.ini is saved on exit

147

clrmame Discussion / Re: Confirmation of odd query RE: cmpro.ini

« on: 14 December 2024, 11:52 »

Sorry, but I don't understand the problem. Where exactly is the problem of not having a title bar....
Either you want to close the app, then ALT-F4 for closing should do it when not having a close button or you don't want to quit the app ;-)

148

clrmame Discussion / Re: to thread or not to thread that is the question

« on: 11 December 2024, 21:03 »

ok..some quick test runs are done....on a HD

Scenario 1, typical MAME style, lots of sets with some files per set, compressed
Scenario 2, typical Progetto style, small amout of sets with huge number of files per set, decompressed

Using let's say 28 versus 1 thread for Scenario 1, the more threads, the faster it gets.......good...end of story

For Scenario 2, you run into a real speed decrease as soon as you use more than 1 thread (in a concrete example of ~40000 files spread over 12 sets we talk about 5 (1 thread) versus 22 minutes (28 threads)

Now you'd say: ok, when you have decompressed files, use 1 thread...but sorry...it's not that easy. It depends on the number of files per set, the file sizes and how they are stored on the hd. For scenario 1 but in a decompressed form, you can easily get better results with more threads....*sigh*

The threads work on sets, so for Scenario 2, it tries to scan all 28 (ok there are only 12 available) sets (each with lots of files in it) in parallel leading to massive seeking overhead.
For Scenario 1, compressed, it scans 28 sets which (means just 28 files)  in parallel which seems to work pretty fine on a hd.

I think a general "use max threads when running on compressed data, "use 1 thread when running on decompressed data" can be a good start....maybe with some manual overwrite by the user if wanted....will see.....

149

clrmame Discussion / to thread or not to thread that is the question

« on: 11 December 2024, 09:36 »

Sorry, no profiler news at the moment.
Oddi's remark about slowlyness when scanning progetto snaps keeps me a bit busy these days. Not only that I already found some hash lookups which can be optimized a bit, I wonder how threading can influence the scanning speed.
Now John IV would say: I'm getting 1 second for a full scan, so threading rocks. yes, it does...when you're on a sdd...when you're on a hdd, too many threads may lead to too many seek operations which may slow the process down.
In case of an unpacked progetto collection (we talk about > 300000 files), each file's crc32 needs to be calculated....which is a lot seeking, reading and calculating (even if the actual calc is quickly done). Using too many threads may slow down the process....so currently I will do some benchmarking what the best value is.....I keep you updated with the output. Even if I see that too many threads slow it down, I somehow need a way to determine a good value (ok, I can show an input box where you simply define it as last possible solution ;-))

150

clrmame Discussion / Re: a nightly cmpro...

« on: 11 December 2024, 09:29 »

I think the reason for having a parent/clone relationship hacked in cmpro was that there were some device romof which actually shared identical files....can't really rememeber.
If I find a little time I will double check it. For now, I'd go with the new solution....no "cloneof" -> no parent/clone relationship. "romof" however plays a role for dependencies (e.g. in standalone mode). BIOSes are included that way for example. Open for comments ;-)

151

clrmame Discussion / Re: a nightly cmpro...

« on: 09 December 2024, 07:02 »

Ah....devices with romof relations but not cloneof....

hd44780
hd44780u romof hd44780
sed1278   romof hd44780
....

clrmamepro hacked in a parent/clone relationship based on the "romof" statement. But actually hd44780 is not the parent of sed1278 /etc.


the new tools know about linked dependencies (following the romof chain) but for "split or full merge" mode, the cloneof attribute is used.

That's why old cmpro complains about the files (since it wants them in the fake parent), while the new scanner keeps them separated...

152

clrmame Discussion / Re: clrmame (scanner 0.08.1, rebuilder 0.15.1) released

« on: 09 December 2024, 06:55 »

I just scanned the snaps dat (the biggest single one) on a decompressed snaps set (so it needs to calculate the hashes) and it took < 1 min in a not cached scan and 4 seconds in a second, cached scan....

Keep in mind that the "all project" datfile uses a totally different folder structure than the single progetto datfiles. The all datfile uses set subfolders (like artpreview\artpreview\005.png, where the 1st artpreview is the setname and "artpreview\005.png" is the romname. While the single ones don't use setsubfolders (besides of SoftwareList dats).

So if you don't already have the files stored in the all-project way, there will be >300.000 files to be reorganized. Same would happen if you specify the rompath wrong which is -when it comes to progretto snaps- a rather common problem.
People need to remember the storing method which needs to be used rompath\setname\file1...file n for decompressed sets, rompath\setname.zip (.7z) for compressed sets. So for example if you're using the "bosses" progretto snaps datfile it contains 1 set called "bosses" where all pngs are stored in that set. So if you have a rompath called "c:\test\progretto\bosses", you either have a bosses.zip file in that folder holding the pngs or you got another subfolder in there "c:\test\progretto\bosses\bosses"  which holds the singe pngs decompressed.

Progetto snaps dats usually have 1 (SL maybe a handful more) sets with thousands of pngs inside it. Having them compressed can cause slow downs when fixing (especially when using solid 7z sets).

Another point when using decompressed files is of course: each file needs to be read and the crc32 (at least) is calculated...a full progetto snaps collection is ~300000 files...now you can calculate yourself how long it can take for decompressed sets....that can be hours

let's take for example 1/100 second for reading the file plus calculating the hash.... that will be 100 files per second....for 300k files it's 3000 seconds.....50 minutes 0.83 hours....if it's 1/10 second per file...you already end up with 8 hours....and so on ;-)

....and another point....is that the less set specifications you have (and in the progetto case you usually only have 1), the less speed you will get from threading, since threading works on checking sets in parallel, not files....


But to give this thread an end....I will look what can be optimized when working with huge 1 set dats where the files are decompressed. ....actually I found something which can be improved already...so will see what I can do.....

153

clrmame Discussion / Re: a nightly cmpro...

« on: 08 December 2024, 07:19 »

I will look into it.........tomorrow...

154

clrmame Discussion / a nightly cmpro...

« on: 07 December 2024, 20:06 »

if anyone still use the old tool....

https://mamedev.emulab.it/clrmamepro/binaries/cmpro64_20241207.zip
(64bit exe only)

it only fixes a problem with not detecting unneeded files in a sample parent set..... The bug was in there since decades I think

Definetly time to use the new tools ;-)

155

clrmame Discussion / Re: clrmame (scanner 0.08.1, rebuilder 0.15.1) released

« on: 04 December 2024, 07:08 »

The warnings appear if you have sample paths setup and *load* MAME's xml output. They are the known-for-a-decade warnings about sample only sets etc...
If you don't quit and run another scan, the xml isn't loaded again since it's still in memory (see first info log message in your second screenshot). That feature is coming with yesterdays version.....so all is fine


By the way....come on...you only post here to show your 1 second full scan result :-)

156

clrmame Discussion / clrmame (scanner 0.08.1, rebuilder 0.15.1) released

« on: 03 December 2024, 19:04 »

https://mamedev.emulab.it/clrmamepro/binaries/clrmame_008_015_1.zip
https://mamedev.emulab.it/clrmamepro/binaries/readme.html

UI:
misc: restore default window positions when positions get out of range
misc: increase display time of tooltips
misc: slightly regrouped ui elements
added: scanner, context menu option to restore old scan results on startup and selection

Core:
misc: updated spdlog to 1.15.0
misc: updated to 7zip 24.09
misc: don't reload xml and build up structures when they are already in memory and need no refresh
misc: align and share rebuilder/scanner common routines
added: scanner, samples, supporting flac
fixed: scanner, typo in wine script
fixed: scanner, internal sha1 check for files with identical crc32 but different sha1 values isn't run
fixed: scanner, samples, wrong named files (case check only) aren't fixed
fixed: scanner, samples, wrong named files (case check only) also appear as unneeded
fixed: scanner, samples, decompressed files are marked as unneeded
fixed: scanner, samples, machines which reference themselves via sampleOf are marked as unneeded in full mode
fixed: scanner, confusing 'can't remove/backup' messages in case of circular renames between different sets
fixed: scanner, confusing 'can't remove/backup' messages in case a wrong file with a right name is replaced with a fill in

157

clrmame Discussion / Re: clrmame (scanner 0.07, rebuilder 0.14) released

« on: 01 December 2024, 11:08 »

A "can't excess" log entry does not have anything to do with using it under Windows or Linux/Wine. It simply means that the mentioned file was at a particularly point in time not accessible. That can be caused by all kind of things.
Simply showing "can't acces" doesn't help unfortunately. Without having the files here or a more detailed information what clrmame tried, I can't help. You could for example scan without the fix option enabled and let me know what it complains about the ad5hir file (scan results tree). Or you can test the archive for corruptness...it also may be locked by other processes etc....

158

clrmame Discussion / Re: FAIL to use newest Scanner and Rebuilder after current release 0.272. Help req..

« on: 30 November 2024, 16:56 »

you could use a "file:" filter which points to a textfile holding your current machine names (1 machine name per row), like

pacman
outrun
194x

and so on....but you'd need to create that list on your own...e.g by redirecting a dir command with a little bit of textediting....
For the future I will think about adding a filter feature for that....(currently it supports regular expressions, xpaths and files)

159

clrmame Discussion / Re: FAIL to use newest Scanner and Rebuilder after current release 0.272. Help req..

« on: 30 November 2024, 14:42 »

The rebuilder got a backup path on its own which needs to be specified.

by the way, there is not really a need to use the rebuilder. If a missing file is listed as fixable, scanner's fix option will automatically pick the file it found somewhere and adds it to your rompath. If you really have missing files and already have a folder somewhere which holds newly colltected update files, you can specify an add path.

160

clrmame Discussion / Re: Sneak....

« on: 28 November 2024, 08:39 »

yep, such sample errors will be gone (refers to - fixed: scanner, samples, machines which reference themselves via sampleOf are marked as unneeded in full mode)

..speedwise...yes....your second full MAME scan will be done then in 3-4 seconds instead of 6 :-)