EMULAB Forum
clrmamepro [English] => clrmame Discussion => Topic started by: Cassiel on 27 August 2014, 10:34
-
What I'm trying to do (I have reasons!):
I have a DAT with 1 rom per 1 set. I want to keep the roms uncompressed, and I want to keep them all in the root of the rom path, and have CMP able to scan them (rebuilder doesn't matter), i.e:
not like this:
rom_path\set1\rom1
rom_path\set2\rom2
but like this
rom_path\rom1
rom_path\rom2
This is akin to how you can choose to have CMP deal with CHDs, though on a DAT level rather than global.
Is this possible?
-
nope..cmpro follows the storing method of MAME which is
(sample/rom)path/setname/file 1...file n for decompressed sets (and chds)
(sample/rom)path/setname.zip/.rar/.7z for archived sets
There is an exception switch to keep chds on rompath rootlevel...but that was really annoying thing to add...and I don't recommend to use it....
The only way is to put all your roms into one game definition...but then you can only compress them in one archive....(if 7z/zip/rar is wanted)
-
Hmmm… I suspected as much. Is this something that you would consider adding for ROMs too, given it exists for CHD already?
Maybe if I explain WHY, rather than you thinking I don't know what the hell I'm doing…
I've started keeping a WIP mirror of the whole TOSEC (main) collection, in the same hierarchy but uncompressed. The reasons for this are many, but it's mainly so I can make high level changes, such as renaming/amending (then dir2dat) orphaned DATs, searching for duplicate ROMs, searching and amending specific publishers/flags…. all this kind of stuff.
Whilst there are some multi rom TOSEC DATs, in general it is overwhelmingly 1 rom per 1 set (single file) DATs. Therefore I have them organised as above, i.e. rom_path\rom1, rom2, rom3 unless they are multi rom per set were as obviously I have them arranged in the traditional MAME style. I've done this because: 1) I don't have to (manually) rename both a folder AND a file, 2) avoid over long path issues, 3) it makes identifying wrong extensions easier.
Now… this is all fine and 'setup', BUT once a new release is out I have to update all the DATs/ROMs which have submitted to me outside of the changes I have made to my mirror. MASSIVE pain in the ass doing this manually, AND easy to cock up if not paying attention.
I do appreciate that this is rather a specific issue to me, but if it is easy to implement it would be a huge help.
-
Sorry if I did not make it clear enough....It does NOT really exist for chds.
As mentioned the chd lookup on rompath root has nothing, really nothing, to do with checking sets in general.
Consider this as a very ugly hack.
I do see the points in having 1-rom-per-game collections though...but currently I don't have any plans to support this somehow....
-
OK, I understand. Maybe something for the future...
Thanks for coming back to me so promptly anyway.
-
if I find some time I might think about it...maybe it's as easy as having 32k pathlength support (which I got a pretty easy idea for...however I don't know if it'll work out...)....most likely that idea fails for rar/7z/zip directly...since the sdks don't support it or something like this....hehehe
-
Thanks, that would be appreciated.
RE 32k, I know both WinRAR and 7-Zip can handle >260 paths no problem. Don't know about SDK though, I know you've said before how poor the 7-Zip SDK is in general. ;)
32k path support would be great.
-
well....the basic idea is pretty simple....
the only part where you need 32k path support is:
- the rompaths / samplepaths
- the rebuilder source / destination paths (which do not necessarily be rompaths) (plus drag'n drop)
- merger destination
- addpaths (scanner fix missing)
Now such paths should actually be absolute paths...e.g. "D:\temp\test\my mame collection\blablabla"
(have to double check this since I remember that I somehow support network paths, too...)
so internally I simply have to change the getRompath(x) (or whatever) function to
something like 'return "\\?\" + rompath(x)'
actually this somehow sounds way to easy... ;-)
...and I think a check on startup should be done if this is supported by the OS...
-
Hahaha... there you go then, put the snotty tissues in the bin and get find&replacing. ;)
-
bin is already full......and I'm back at work already...still not 100% fit though
-
Well....the "\\?\" idea was easy...and would work for cmpro...but not for the archive libs....there are too many places where they play around with MAX_PATH constants
...so for a full support I dropped the idea...
BUT:
I've just tested the rebuilder...and acutally if you enter a destination like:
\\?\E:\temp\this is a long folder name right or wrong blabalbalblablablablabalabl\
and you want to create a file
ABM (1980)(Muse) & Invasion Force (19xx)(Gordon Lurie) & Missile Defense (1981)(On-Line Systems) & Norad (19xx)(-) & Planet Protector (19xx)(-)[cr] & Rocket Command (1980)(Bozo NYC).zip
it will work ;-) so you only have to add the \\?\ prefix
(without it won't work)
however this only works if the folder already exists...since currently cmpro also uses MAX_PATH constants when it comes to folder creation...
so maybe I work on this (getting rid of MAX_PATH) and you yourself have to add \\?\ if needed....
time will tell...
-
ok...fiddled a bit with MAX_PATH ....and it works...eheheheheh....
(the file is actually a dummy one, name taken from this forum...)
if you want to play with it:
http://mamedev.emulab.it/clrmamepro/binaries/cmp20140828.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmp20140828.rar)
seems to work for rar/zip/7z at the moment....
so all you have to do is to add \\?\ in front of your absolute (rom/source/destination/whatever) path:
\\?\d:\tosec\my_very_long_blalballablalbalblabla
-
It would be easy to write a DAT parser, which re-writes DAT files: (in Bulk)
"to put all your roms into one game definition"
The re-written DATs could be used for Scanning & Rebuilding..
The parser could even add tag "Force Packing = unzip"
It can also be done (manually) with TextFX plugin for Notepad very quickly per dat, but that's the problem, it would need to be done 'per dat'
For a whole TOSEC dat set, one by one -- ahhhh
-
would be an easy ant job with xmltask and a simple xslt transformation...but wait ....they still use the deprecated format eheheheh...
but back to the important stuff.... 32k path length support....seems to work for me
-
iTunes have the long name issue. I wish they fix their end. I had to removed the date of the movies because some of those movies the date ether doesn't show or get cut. I guest there is a limited. :)
Anyway back to the thread. Sorry about that.
-
OK, came up with a new idea…
Since I already have a 100% complete TOSEC collection in RAR format, which is organised as:
X:\TOSEC\Acorn\Applications
X:\TOSEC\Acorn\Demos
X:\TOSEC\Acorn\Games
blah blah
I thought I'd Dat2Dir the whole lot to make 1x custom DAT by pointing Dat2Dir at X:\TOSEC\ and enabling 'Set-Subfolder Mode' and disabling 'Keep Archives As Files'.
BUT can't seem to get it to play ball, it doesn't retain the paths in the DAT.
Any ideas of the magic toggling of options that I might be able to achieve this?
-
have to check at home...there is a way...some weird combination... ;-)
-
Maybe this helps:
assume you got a folder structure like this:
E:\test\NeoGeo\3countb.zip
E:\test\NeoGeo\mslug\mslug.zip
E:\test\NeoGeo\mslug\mslug2.zip
E:\test\NeoGeo\mslug\mslug3.zip
and you use E:\test as dir2dat source path and got set-subfolder and keep-archives options enabled...you end up with:
<game name="neogeo">
<description>neogeo</description>
<rom name="3countb.zip" size="..." crc="..."/>
<rom name="mslug\mslug.zip" size="..." crc="..."/>
<rom name="mslug\mslug2.zip" size="..." crc="..."/>
<rom name="mslug\mslug3.zip" size="..." crc="..."/>
</game>
so the first subfolder in the source path is used as name/description and files within it will be roms on root level...subfolders will then be subfolders of that set...
-
Nah... no good. Think I'm going to have to give up on this plan. :'(
I was thinking on the way home last night that I don't think I will be able to get it to work anyway due to the mix of single ROM sets and multi ROM sets. Frustrating.
Thanks for checking though.
-
Testing 32k (with offical binary atm since all existing paths) and seems to work perfectly with uncompressed sets.
-
You definetly need to use the test build posted here....
As soon as a path does not exists and cmpro tries to create a folder, the official binary will fail...the test build should work.
The "\\?\" prefix stuff should already work in the official binary...the only necessary stuff was to increase MAX_PATH constants which are used when creating or changing folders...so the fix and so the support was pretty easy to add...
-
OK, let me swap them out now and continue...
-
Hmmm... not all peaches and cream unfortunately.
It seems to rebuild OK (both Source and Destination I added \\?\) but when I go to scan, it throws up this 'fix' which it (unsurprisingly) cannot complete.
(http://redsplinter.com/tosec/1.png)
(http://redsplinter.com/tosec/2.png)
I've added \\?\ to the 1st ROM path.
-
7z? rar? sorry but I cannot speak for 3rd party libs to support this right. maybe you can zip up roms and dat for me to test
-
Uncompressed... no packer.
-
http://redsplinter.com/tosec/%5bST%5d.rar (http://redsplinter.com/tosec/%5bST%5d.rar)
Here is the Atari ST DAT I was using + some files that are short and overlong.
I'm rebuilding/scanning everything uncompressed, no packer involved at all.
-
thanks...a *sigh*...MS Lib call CFileFind does not handle 32k paths...need to use some w32 api calls....
-
today's test version should include a fix for your issue, too...By the way...it was of course no wrong-case scan error...it failed due to the long name..
Still have to find out which core calls do not handle 32k paths natively...FindFirstFile was one....wonder if all dir tree walks handle it....
so please keep testing...and I will check which things may go wrong, too....
-
Thanks... I will test this morning and feedback.