EMULAB Forum
clrmamepro [English] => clrmame Discussion => Topic started by: stecy on 24 February 2013, 18:29
-
Hi,
I've been trying to get clrmamepro to recognize the files in the cabinets folder using a DatFile but the program always telling me that all the files are unneeded. That is not so!!!! They are needed.
Can anyone tell me how to configure clrmamepro to properly scan the cabinets folder. As an aside, other folders are behaving the same.
Here how the datfile I have is configured:
<datafile>
<header>
...
</header>
<game name="cabinets">
<description>cabinets</description>
<rom name="005.png" size="241910" crc="88effcb8" sha1="a8a14256a58a226d3425bc9222c552762c62ebd3"/>
....
</game>
</datafile>
My cabinets files are located in I:\Mame\Cabinets
I've tried setting the ROMPATH for clrmamepro to I:\Mame\Cabinets
I've tried disabling SETS from the scanner
Thank you for any help.
-
There are 2 supported storing methods:
rompath\setname\file 1.... file n (for decompressed sets)
rompath\setname.zip (.rar/.7z) (for compressed sets, where the archive then holds file 1 ... file n)
Looking at the datfile you see that the setname is <game name="cabinets"> "cabinets".
"My cabinets files are located in I:\Mame\Cabinets" ....so your rompath needs to be setup to "I:\Mame" if you follow the upper rule.
Now it sounds a bit like I:\MAME is used for other stuff (which the cmpro scanner most likely will mark as unneeded), you may think about having another subpath where you put your cabinets...
-
Well, I have the following folder structure:
I:\Mame
I:\Mame\Cabinets
I:\Mame\Roms
I:\Mame\Samples
etc...
I could have a "cabinets" inside the I:\Mame\Cabinets but frankly that would seem odd.
However, I managed to do it by unchecking the "You want to scan sets" in the scanner.
Apart from having a message that it is bad having this disabled, everything succeeded.
Is there a better way?
May I suggest having the program determines that such a setup is fine?
Thank you and have a nice week.
-
Do NOT disable sets.
You need to change your storing method. Again, you need to use the storing methods:
rompath\setname\file 1.... file n (for decompressed sets)
rompath\setname.zip (.rar/.7z) (for compressed sets, where the archive then holds file 1 ... file n)
Yes, you may need to add another folder level for your files then, e.g. I:\Mame\ArtFiles\Cabinets
Depending on other profiles you may even think of another level or specify exclude folders (settings, path selector).
There won't be support for any other storing method.
By the way, if you're using decompressed sets you should disable scanner/advanced/deeper check for fixable missing files....this is usually a slow-down option when handling decompressed sets.
-
Can you explain why disabling sets is bad? Why is the option there in the first place then? ;D
I've scanned all my art files and everything checks out ok.
However, I am willing to give it a try but I want to be sure on how to setup properly.
So here's what I am going to start from scratch and re-run the mame extraction so that I end up with:
I:\Mame
+------artwork
+------ctrlr <-- contains not compressed files
+------docs
+------hash
+------hlsl
+------roms <-- contains compressed files, one per game
+------samples <-- contains compressed files, one per game
[files... like mame.exe, mame.ini]
I'll then create the following structure inside the artwork folder:
+------artwork
+-------artwork <-- bezels and layouts, one zip per game
+-------bkground <-- contains not compressed files
+-------cabinets <-- contains not compressed files
+-------cpanel <-- contains not compressed files
+-------flyers <-- contains not compressed files
+-------icons <-- contains not compressed files
+-------manuals <-- contains not compressed files
+-------marquees <-- contains not compressed files
+-------pcb <-- contains not compressed files
+-------snap <-- contains not compressed files
+-------titles <-- contains not compressed files
Will I need to amend the mame.ini configuration file to add all those subfolders or is Mame smart enough to figure it out?
Also, for now the files inside the subfolders are uncompressed and although I could compress them using 7z and a solid archive (with a small solid size) I am not sure of the impact it would have on Mame upon loading a game. It would certainly make an impact on the storage size though... ;)
-
You turn off all set specific check options (set = collection of roms and/or samples and/or chds.....so you turn off all checks which work on the folder/archive itself. The rom checks work on the files within the folder/archive). When turned off you will e.g. not notice if the set was renamed in a future dat (I admit this is only very rarely happening for artwork based dats, but still it already happened with them).
So turning this off is in my opinion not a good workaround for a wrong storing method.
Why is there an option, well, because cmpro got an option for everything :) From 1997 onwards it had the ability to turn on/off any possible check/fix since some advanced users required this. The normal use case is to use the default settings (which includes enabled sets/roms/samples/chds checks).
The storing method originally comes from MAME. Well, artwork dats are - compared to MAME sets - different, since they put all files into 1 set. But still the 2 storing methods should be followed to audit them. Keep in mind that everything found in a rompath which does not belong to a set is unneeded, that's why you should separate your profile's rompaths.
If you keep them decompressed or compressed is fully a personal taste.
Last but not least you need to keep in mind that when it comes to filenames and paths generation (e.g. when fixing or reading files), cmpro will generate rompath\setname\ or rompath\setname.zip(.rar/.7z). So if you don't follow the storing methods you will end in chaos sooner or later...
-
Sorry, maybe this sounds a bit noob, but I need some guidance of you guys.
Attached a datfile I want to use, never got it to work properly with uncompressed files. As an example the first file of the dat and my folder structure for it:
C:\007 - Agent im Kreuzfeuer (Germany)\007 - Agent im Kreuzfeuer (Germany).nkit.gcz
Clrmame (latest build) don't recognize them as a valid set, same file zipped works fine.
Any hint what I'm doing wrong?
TIA for any advice!
-
You need to follow the correct storing method:
rompath\setname\file 1...file n for decompressed sets
rompath\setname.zip (.rar/.7z) for compressed sets (where the archive holds file 1...file n for that one set)
So assume you got it stored like this:
E:\temp\007 - Agent im Kreuzfeuer (Germany)\007 - Agent im Kreuzfeuer (Germany).nkit.gcz
then e:\temp is the rompath you have to setup....
in your example C:\ would be the rompath (but I guess you don't want that, since everything in C:\ which is not part of the dat will be marekd as unneeded).
So, I suggest to add a folder level, e.g. C:\GameCube ...then this is the rompath ...and you store it like
E:\GameCube\007 - Agent im Kreuzfeuer (Germany)\007 - Agent im Kreuzfeuer (Germany).nkit.gcz
-
C:\ was only for example. My rompath is "\\SYNOLOGY_DS414J\public\Emulation\ISOz\NINTENDO GameCube (Redump.org) (NKit)".
So the complete path for the mentioned set is "\\SYNOLOGY_DS414J\public\Emulation\ISOz\NINTENDO GameCube (Redump.org) (NKit)\007 - Agent im Kreuzfeuer (Germany)\007 - Agent im Kreuzfeuer (Germany).nkit.gcz". But this isn't working at all, still getting no sets in place and Clrmame is asking to remove them. But as soon as I put them in a zip container it's fine.
-
with such a long path, you may hit the pathlimit.....either use the latest nightly build or use \\? prefix (see https://www.emulab.it/forum/index.php?topic=22.msg17279#msg17279 (https://www.emulab.it/forum/index.php?topic=22.msg17279#msg17279))
I've tried it with a standard (relatively small rompath) and it worked fine
or map a drive letter to \\SYNOLOGY_DS414J\public\Emulation\ISOz\NINTENDO GameCube (Redump.org) (NKit) and use that one then...
-
I'm already using the latest nightly build.
Also mapping the rompath as a network drive isn't working either and \\? don't do it. :( Why should it change a thing while zipped files with the same rompath are working fine?
-
It works fine for me with a 'normal', drive letter based rompath, like e:\temp\gamecube
but you're right, even "\\SYNOLOGY_DS414J\public\Emulation\ISOz\NINTENDO GameCube (Redump.org) (NKit)\007 - Agent im Kreuzfeuer (Germany)\007 - Agent im Kreuzfeuer (Germany).nkit.gcz" shouldn't hit the path limit.
"Clrmame (latest build) don't recognize them as a valid set" ...so what exactly does this mean in detail? what does it say?
For a test you may want to copy it on a non network drive....if it's working there, there might be an issue with the network...
...or you give me access to the zip file somehow...so I can do further tests
-
Tested on a local drive and it works. Still the question why it works on the netwwork path as zipped files but not as uncompressed. Weird...
They sets are marked as unneeded, that's what I meant with none valid.
I don't think that my network is the issue as all of my collection is located there and there's no problem at all. Only as soon as they're uncompressed...
-
Can be anything...access rights, 3rd party tools (virusscanners) interfearing...etc...
There are for example known SAMBA issues where large files cause problems (bad network drivers)...
The are a lot of differences while working with zip files versus files. For example when calculating the hash for the files. In zip/7z/rar mode it simply takes the hash from the stored crc32 information (unless you don't check sha1 as well, then the file is decompressed to memory) while for a decompressed file it reads the file and calculates the checksum (on network drives this means all data is transfered).
It can also be Window specific library calls which may have a problem with the network when accessing/reading files...an we're talking here about larger files.
I try to rebuild the scenario with a bigger file in a longer path on a network drive...
What you can do...well....you may want to disable virusscanners for that folder....by the way...is it the only file which is listed as unneeded or all of that datfile?
-
All of the datfile is listed as unneeded, verifies fine as zipped. There're also far bigger and more files I audit on this NAS with Clrmame (all in rar or zip, btw.), so I don't think that's the problem. Virusscanner isn't the problem either, Windows specific, maybe...
The only reason I don't want to zip in the case is, that you can use this format (gcz) directly with the emulator Dolphin and it's already compressed. Only 2 options for me left: zipping only as stored or no auditing. Not really sadified with both. :(
-
Yeah, see your point...any chance you can provide
- your cmpro.ini
- the used datfile (guess I got that) and the belonging *.cmp file from cmpro's settings folder
- one of the failing files in the rompath
-
I will send you a link with those files.
Edit: Done!
-
thanks...
-
Hi Roman, hope you are well ! 8)
Long time no posts from me..
I have always wished your wonderful program - with so many uses beyond emulation,
could handle unarchived files in a single folder - IE. dir2dat pointed to a folder at root level
contents indexed and able to be recreated / scanned from a dat.
Cheers
-
You need to follow the correct storing method:
rompath\setname\file 1...file n for decompressed sets
rompath\setname.zip (.rar/.7z) for compressed sets (where the archive holds file 1...file n for that one set)
Like an option to simply 'ignore setname' this way an existing dat such as:
NINTENDO GameCube (Redump.org)
..\007 - Agent im Kreuzfeuer (Germany)\007 - Agent im Kreuzfeuer (Germany).nkit.gcz
Could be used to store / scan all files at root level 'minus' the subfolders.
-
simply create a dat with one set and all files in it :-)
-
and now back to the original problem....Yay...I can repeat it on a network drive ;-)
Since I used a network drive on the other side of the world it was way too quick that it could have calculated the checksum that fast....interesting....after some coffee and cake I will investigate this later tonight.
but on the other hand I guess it is related to the length of the path...
\\something here\Emulation\ISOz\NINTENDO GameCube (Redump.org) (NKit)\2 Games in 1 - Bob L'eponge - Le Film + Tak 2 - Le Sceptre des Reves (France) (Disc 1) (Bob L'eponge - Le Film)\2 Games in 1 - Bob L'eponge - Le Film + Tak 2 - Le Sceptre des Reves (France) (Disc 1) (Bob L'eponge - Le Film).nkit.gcz
is beyond the path limit while
\\something here\Emulation\ISOz\NINTENDO GameCube (Redump.org) (NKit)\2 Games in 1 - Bob L'eponge - Le Film + Tak 2 - Le Sceptre des Reves (France) (Disc 1) (Bob L'eponge - Le Film).zip
is not
I know of at least one Window library call which has a problem with such long names (some directory tree walk call).....will test later if it's related to that one....if so I need to find a walkaround (sigh) (see comment on the nightly build post)
-
Thanks Roman for investigating this, I appriciate it and of course your hard work for this nice program! :)
Btw., I also tried a shorter path on my NAS and it hadn't worked either with uncompressed files.
-
You need to reach something below 260 chars includong the filename
-
Confirmed, testing with "\\SYNOLOGY_DS414J\public\GC\4x4 Evo 2 (USA)\4x4 Evo 2 (USA).nkit.gcz" works fine. But with the set in same place "\\SYNOLOGY_DS414J\public\GC\007 - Agent im Kreuzfeuer (Germany)\007 - Agent im Kreuzfeuer (Germany).nkit.gcz" it doesn't work.
-
Actually I found the cause....
\\?\c:\dir\file.txt UNC Drive path 32k bytes
\\?\UNC\server\share\dir\file.txt UNC Server path 32k bytes
means that you (actually cmpro (*)) needs to specify a different prefix for a network path...
When doing so it will at least start to calculate the hash...but there seem to be other places where such a long network path fails...so I need to check some other stuff as well.
(*) the nightly build of cmpro sees if a path is too long and automatically adds \\?\ in front of it (as long it's not a network path or a relative path). But for network paths it needs to add \\?\UNC\ ..
-
I tried to put \\?\UNC\ in settings by using F2, but it seems like it doesn't change a thing. After hitting return it's back to \\SYNOLOGY_DS414J\public\Emulation\ISOz\NINTENDO GameCube (Redump.org) (NKit)\ .
Maybe I did something wrong? Should the path be \\?\UNC\\\SYNOLOGY_DS414J\public\Emulation\ISOz\NINTENDO GameCube (Redump.org) (NKit)\ ?
-
\\?\UNC\SYNOLOGY_DS414J\public\.....
(just one (1) backslash \ after UNC) ...but since cmpro's nightly build auto-adds/removes it you may run into trouble....gimme some time to fix it (in the meantime I found the reason for the other problem, too...there was a not mapped file operation which did not run through the prefix adder...)
try this:
http://mamedev.emulab.it/clrmamepro/binaries/cmpro20191006.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro20191006.rar)
-
Works fine now! Many thanks Roman!! :)
About oxyandys request:
Like an option to simply 'ignore setname' this way an existing dat such as:
NINTENDO GameCube (Redump.org)
..\007 - Agent im Kreuzfeuer (Germany)\007 - Agent im Kreuzfeuer (Germany).nkit.gcz
Could be used to store / scan all files at root level 'minus' the subfolders.
That would be nice not forced to put every file in a folder... :)
-
Thanks for reporting and helping...
Well, about that "I want them all uncompressed in a folder" support.....you need to see how MAME originally works. A set usually contains more than one file (gfx roms, sound roms, dumps holding the code etc), so they got a container, a folder named after the set, holding these files. Using an archive (7z/rar/zip) is pretty the same - a container (again named after the set).
Scanning/Merging/Rebuilding is built around this.
I see your point when it comes to 1-file-per-set collections. Keeping them decompressed leads to the a bit obsolete storage method with the extra folder.
What you can do is to change the datfile definition to only define one set where all files are stored. This would allow to have all files in one folder (named after the setname defined in the dat). Cons: you loose additional information like a detailed description per set (however as you can see from these big filenames, they simply put all in there...) and you get the scan results after finisihing this one giant set.
Theoretically I can add an option which changes a loaded datfile into an one-set-billion-roms dat......time will tell
-
This isn't anything of rush, Clrmame is already my favorite rom auditing tool. Thanks for your hard work!! :)
-
Here is a basic response just to say thanks Roman for even considering this.
When I have time in the next few days I will point out a few issues you may not even be aware of.
As you know I like to include detail in my posts so will not just make a 'rushed post',
I'll take my time and draft a really decent response, include screenshots and real examples to remove any ambiguity.
Cheers 👍
Oxyandy
-
better I run for cover..... ;-)
-
Hi Roman,
other than RAM and HDD what could cause hash computations to be an incorrect result please (wrong CRC calculated)
Thanks
EDIT: silly question I know 🤣
Failing: motherboard, processor, i/o controller, inadequate or failing power supply.
Time to dump this hardware (well most)
Scanner Output: "wrong crc32: Some_file.bin [wrong: 537b1a1f] [right: f38a3b0f]"
Yet, independently - Some_file.bin is actually f38a3b0f
-
theoretically any involved driver or hardware...or even software (a virusscanner might block the access...but that should end in a missing or not accessible file rather than a bad hash)