EMULAB Forum

clrmamepro [English] => clrmame Discussion => Topic started by: fleggett on 23 July 2010, 08:34

Title: Download folder - what's it do?
Post by: fleggett on 23 July 2010, 08:34
The more I use this program, the less I understand how it works.

First problem - I keep getting this message about cannot retrieve version information when switching between options.  But that's relatively minor.

The big thing I don't understand is what the download folder is suppose to do and how it's actually utilized.  I <ahem> download roms to a specified folder and have that folder defined in clrmamepro.  I thought that by running the scanner the program would take the updated roms from my download folder and somehow "merge/update" them into my main MAME roms folder.

However, that doesn't seem to be happening.  Or, if it is, I can't tell it's happening.

What do I have to do in order to get the updated roms into my MAME roms folder?  Will clrmamepro do it and I'm just not using the program correctly or is there some additional manual intervention I have to do at my end?

Oh, I don't know if this affects anything, but I've got my roms in my MAME roms directory separated out by systems.  Thanks in advance.
Title: Re: Download folder - what's it do?
Post by: Roman on 23 July 2010, 10:58
what message about switching options do you mean?? screenshot please.

the download folder is for cmpro's auto download options not for your downloaded files. easiest way to add files is via rebuilder or dragndrop in the scanner.

cmpro's download options let you download files from the reported issue list from a given url. they will be stored in the downloadfolder.
Title: Re: Download folder - what's it do?
Post by: fleggett on 24 July 2010, 07:28
Here's the error message I keep running across:

(http://www.awfulbooks.com/Capture.JPG)

I still don't understand how I'm suppose to move the roms from my download directory to the main MAME rom folder.  As I wrote, I've got all my roms separated by machine, like so:

 7/23/2010   3:53         <DIR>    .
 7/23/2010   3:53         <DIR>    ..
 7/11/2010   4:43         <DIR>    acpsx
 7/23/2010   4:19         <DIR>    airbios
 7/09/2010   1:51         <DIR>    aleck64
 7/23/2010   9:56         <DIR>    alg_bios
 7/23/2010   4:23         <DIR>    ar_bios
 7/23/2010   8:09         <DIR>    atarisy1
 7/09/2010   1:51         <DIR>    atluspsx
 7/09/2010   1:51         <DIR>    atpsx
.
.
.etc.

Should I create an "unsorted" or "uncategorized" directory, dump all the downloaded roms in there, and let clrmamepro "do its thing" when the scanner feature is run?  If I do that, am I going to have to add this extra directory to ROM-Paths under Settings?  If that's not the right way, how am I suppose to add files via the rebuilder or drag 'n' drop the downloaded roms in the scanner?  There doesn't appear to be an interface for this purpose.

Also, am I correct in deducing that if I'm using MAME's internal rom database (contained in the EXE) that I don't have to use any external DAT files?

Like I said, the more I use clrmamepro, the dumber I seem to get.  And I've read the various docs.  I hope you don't mind holding my hand while I struggle through these issues.
Title: Re: Download folder - what's it do?
Post by: Roman on 24 July 2010, 15:44
well, I'm on holiday, so here's the short answer:

the "version" prompt....guess this appears if the auto-update check does not find the needed version information on the clrmame homepage....which is a bit weird unless you don't have an internet connection ready. I can check this when I'm back. If you don't want to see it anymore, change the update check (cmpro's about window) to don't check daily/weekly/etc...

Again, the "download folder" is for cmpro's own download functionality which you can use from the scanner popupmenu. It lets you download missing listed items from a given URL...and these downloaded files will be stored in the download folder. Nothing more, nothing less.

The common way to update sets is if the new data is loaded in in the profiler, go to the scanner, enable all check/fix options and do a full scan. this should fix all fixable stuff and you most likely end with some missing sets (the ones which are new or were replaced in the new version). Now use the rebuilder to add the missing stuff. Do a scan again...and repeat the described way until nothing is listed anymore.

Assuming you've downloaded new sets to a folder c:\temp, let c:\temp be your rebuilder source folder and the rebuilder destintion should point to your cleaned up collection. The rebuilder adds all valid files. Even easier if once configures, you can simply drag'n drop files into the scan results window...this will internally use the rebuilder to add the files.

If you want to split sets by bios sets as you wrote, you need to use "system default paths". You can set them up in "Systems". Also set them up as rompaths. Since cmpro's rebuilder and scanner's fixmissing option needs to know if it should use system default paths, you need to enable the belonging options in the rebuilder or scanner-advanced.

Yes, you don't need external datfiles for MAME, MESS, RAINE, PINMAME. clrmamepro gets the data directly from the executable...and if you replace the exe file, it regets the data.

Just in case you're using Windows7 or Vista, please store cmpro in a NON UAC protected folder (e.g. NOT in c:\programs). I use for example d:\cmpro.

Don't worry if the advanced options make your hair gray at the moment...start the simple way:

1) profiler: create a profile based on your mame binary
2) settings: setup rompaths (and sample paths if you're using samples, too)
3) scanner: select your prefered merge mode, enable all check options (that's the default) and hit new scan...

3.1) scanner: enable all fix options and hit scan
3.2.)scanner: drag'n drop new stuff (after configurating the rebuilder) into the window

repeat 3.1/3.2 until you're done. There are also options which do a scan after a drag'n drop automatically....but again...start simple.
Title: Re: Download folder - what's it do?
Post by: fleggett on 26 July 2010, 03:30
Roman, thanks for taking time off from your vacation to answer my post.  However, I want to table those issues for a bit, as I've run into what I think is a fairly serious problem with the scanning feature.  I have all the "Check" and "Fix" buttons checkmarked.  However, when I do a scan, this happens:

(http://www.awfulbooks.com/clrmamepro/cmpro1.jpg)

I answer "Yes To All" and the scanning feature seems to complete normally.  However, just for a lark, I ran the scan feature again and once again got the same message:

(http://www.awfulbooks.com/clrmamepro/cmpro1.jpg)

I dropped to a DOS CLI to see what was going on and found this:

(http://www.awfulbooks.com/clrmamepro/cmpro2.jpg)

I would say the file was just copied over with the correct name and not renamed per-se, but the difference in filesizes between the two CHDs nixes that theory.

I then looked at the log entry and it reported this:

(http://www.awfulbooks.com/clrmamepro/cmpro3.jpg)

Notice the "[not fixed]" at the end of the entry.

I can run the scan feature all day long and encounter the same result.  This can't be normal, can it?  Is there an option I haven't set somewhere?  This is running the latest version, BTW (3.134b 64-bit).  Oh, and clrmamepro has full privileges to the MAME ROM folder in question.
Title: Re: Download folder - what's it do?
Post by: fleggett on 26 July 2010, 05:04
Okay, I'm getting back to this part of my question.  However, I'm still having issues.

the "version" prompt....guess this appears if the auto-update check does not find the needed version information on the clrmame homepage....which is a bit weird unless you don't have an internet connection ready. I can check this when I'm back. If you don't want to see it anymore, change the update check (cmpro's about window) to don't check daily/weekly/etc...
I just go rid of it by not checking it at all.  It's not a biggie, but I would be interested to hear what you find out when you get back.

Quote
Again, the "download folder" is for cmpro's own download functionality which you can use from the scanner popupmenu. It lets you download missing listed items from a given URL...and these downloaded files will be stored in the download folder. Nothing more, nothing less.
So, it sounds like it's pretty useless for my purposes, then.  I download roms via usenet (the horror!), not from a specific site.

Quote
The common way to update sets is if the new data is loaded in in the profiler, go to the scanner, enable all check/fix options and do a full scan. this should fix all fixable stuff and you most likely end with some missing sets (the ones which are new or were replaced in the new version). Now use the rebuilder to add the missing stuff. Do a scan again...and repeat the described way until nothing is listed anymore.
Well, no matter how many times I use the rebuilder, there are still a LOT of flagged files which are reported by the scanner.  However, see my previous message about some scanner weirdness.  Could the two be related?

Quote
Assuming you've downloaded new sets to a folder c:\temp, let c:\temp be your rebuilder source folder and the rebuilder destintion should point to your cleaned up collection. The rebuilder adds all valid files. Even easier if once configures, you can simply drag'n drop files into the scan results window...this will internally use the rebuilder to add the files.
Okay, this added a critical part to my configuration.  I had no idea that I was suppose to define my rom download path here.

Now, regarding drag 'n' dropping - it doesn't work (for me).  I can highlight my download directory or all the contents in that directory and I can see where the pointer changes to that little plus sign underneath the pointer, but nothing seems to happen.  The same with the scan results window.  I don't know if this is particularly critical, but it is worrisome.

I did run the rebuilder and it picked up on my rom download directory and "did its thing".  However, as I wrote, I'm still getting a LOT of flagged items in the scanner.  I really don't understand what I'm doing wrong and am beginning to feel like the program is starting to defeat me.

Quote
If you want to split sets by bios sets as you wrote, you need to use "system default paths". You can set them up in "Systems". Also set them up as rompaths. Since cmpro's rebuilder and scanner's fixmissing option needs to know if it should use system default paths, you need to enable the belonging options in the rebuilder or scanner-advanced.
I'm pretty sure I've done this.  The only options NOT checked in the Advanced Scanner Options are "Allow not separated BIOS ROMs", "Allow CHDs in ROMPath root", and "Show 'baddump' ROM & inverted CRCs".  All the other options are checked.

Quote
Yes, you don't need external datfiles for MAME, MESS, RAINE, PINMAME. clrmamepro gets the data directly from the executable...and if you replace the exe file, it regets the data.
That takes a load off.  I keep seeing these websites with all these DAT files available for download and my heart sinks.  That clrmamepro handles DAT processing via the executable itself is a Godsend.

Quote
Just in case you're using Windows7 or Vista, please store cmpro in a NON UAC protected folder (e.g. NOT in c:\programs). I use for example d:\cmpro.
Well, I am using Win7 x64 and clrmamepro is installed in C:\Program Files\clrmamepro, but I have Properties -> Compatibility set to run the program as an admin.  And it does ask me that UAC question every time I invoke the program.  Do you think this could still cause problems?
Title: Re: Download folder - what's it do?
Post by: Roman on 26 July 2010, 14:30
Quote
I just go rid of it by not checking it at all.  It's not a biggie, but I would be interested to hear what you find out when you get back.
Well, I will do some testing....however I somehow doubt I will be able to do this before next week....

Quote
So, it sounds like it's pretty useless for my purposes, then.  I download roms via usenet (the horror!), not from a specific site.
Download only supports HTTP:// and FTP:// ...however both can be configured freely so that maybe usenet via http: would work...anyway..ignore it for now.

Quote
Well, no matter how many times I use the rebuilder, there are still a LOT of flagged files which are reported by the scanner.  However, see my previous message about some scanner weirdness.  Could the two be related?
In your screenshot I see that a wrong named chd should be renamed to a correct name. The wrong named one already got a "_random number" extension which cmpro adds if a file is being moved to backup and an equally named file already exists. A kind of safe-move without overwriting something. So check if the "new name" file already exist and maybe you need to to a little bit of manual work here.
What is a bit scary is the fact that normally such extensions are only used inside the backup folder....that should be a separate folder, no rompath...maybe you should double-check this.
The rebuilder only ADDs files (however it uses the correct rom and name, no matter how the source file is named).  It does only work on ROMs (at the moment). So wrong names can only be fixed from within the scanner....if the rename operation fails, you should always check if the "new name" already exists.


Quote
Now, regarding drag 'n' dropping - it doesn't work (for me).

Welcome to the world of MS Vista/Windows 7. You can't do drag'n drop from one folder to another if there are different access rights involved. If cmpro is stored within an UAC protected folder and run as admin, you will run into issues when you drag files from other places with other rights...That's windows. Live with it....or even better: store cmpro like I described before...not in c:\programs :)=

Quote
Just in case you're using Windows7 or Vista, please store cmpro in a NON UAC protected folder (e.g. NOT in c:\programs). I use for example d:\cmpro.



edit: it would be interesting to know how the chd issue is based....can you give me some details about the stored files in the mentioned folder drmn2mpu (or whatever it was).....and maybe send me: your cmpro.ini file plus the *.cmp file belonging to your profile (usually cmpro's settings folder). In case of UAC folders, these may be mirrored in the windows roaming (C:\Users\....\AppData\Roaming\clrmame ?!?) folder... as mentioned..UAC sucks ;)
Title: Re: Download folder - what's it do?
Post by: fleggett on 28 July 2010, 04:31
In your screenshot I see that a wrong named chd should be renamed to a correct name. The wrong named one already got a "_random number" extension which cmpro adds if a file is being moved to backup and an equally named file already exists. A kind of safe-move without overwriting something. So check if the "new name" file already exist and maybe you need to to a little bit of manual work here.
You kinda lost me there.  Which file is suppose to be the right one?  And why are the two filesizes so dissimilar?  Should I just manually delete one?  If so, how do I know which one to erase?

Quote
What is a bit scary is the fact that normally such extensions are only used inside the backup folder....that should be a separate folder, no rompath...maybe you should double-check this.
If you're referring to what's contained in ..\clrmamepro\backup\MAME, the file I screenshotted doesn't exist in there (but 107 other files do).  Is there a setting I'm missing within clrmamepro?

Quote
The rebuilder only ADDs files (however it uses the correct rom and name, no matter how the source file is named).  It does only work on ROMs (at the moment). So wrong names can only be fixed from within the scanner....if the rename operation fails, you should always check if the "new name" already exists.
Hrrrr.  Once again, I seem to've dropped a few IQ points.  How do I know which is the "right" file after the scanner/rename operation has run?  And how many of these could potentially be affected?

Quote
Welcome to the world of MS Vista/Windows 7. You can't do drag'n drop from one folder to another if there are different access rights involved. If cmpro is stored within an UAC protected folder and run as admin, you will run into issues when you drag files from other places with other rights...That's windows. Live with it....or even better: store cmpro like I described before...not in c:\programs :)=
<groan> That's just great.  Well, when I move the program over to D:, is there any way I can restore my current configuration?  It took me probably two hours to get everything defined initially.

Maybe I should just install it in G:, which is where my MAME application is installed.

Quote
edit: it would be interesting to know how the chd issue is based....can you give me some details about the stored files in the mentioned folder drmn2mpu (or whatever it was).....and maybe send me: your cmpro.ini file plus the *.cmp file belonging to your profile (usually cmpro's settings folder). In case of UAC folders, these may be mirrored in the windows roaming (C:\Users\....\AppData\Roaming\clrmame ?!?) folder... as mentioned..UAC sucks ;)
<sigh> Yes, it certainly does.

I can PM you with the file attachments (I think), but I'm not sure what you need to know regarding the files in the drmn2mpu folder.  Can you be very specific?  Like I said, with this program, while I can understand the reason behind its existence, the nitty-gritty details seem to completely elude me.

Oh, before I forget, I meant to ask - once the rebuilder is run based on the files found in my usenet download directory, can I then delete those downloaded files?  Or should I keep them around for future rebuilds/rescans?
Title: Re: Download folder - what's it do?
Post by: Roman on 28 July 2010, 07:56
Quote
You kinda lost me there.  Which file is suppose to be the right one?  And why are the two filesizes so dissimilar?  Should I just manually delete one?  If so, how do I know which one to erase?
The right one is the new name of course. The one without the _18524 extension. What filesizes are you talking about. i don't see any filesizes.


Quote
If you're referring to what's contained in ..\clrmamepro\backup\MAME, the file I screenshotted doesn't exist in there (but 107 other files do).  Is there a setting I'm missing within clrmamepro?
No, I was refering to the fact that files only get a _randomnumber extension in cmpro's backup folder....the screenshot shows that your chd file got such an extension in your rompath...which is a bit spooky and I wonder how this was created.


Quote
Hrrrr.  Once again, I seem to've dropped a few IQ points.  How do I know which is the "right" file after the scanner/rename operation has run?  And how many of these could potentially be affected?
Normally you don't have to care, since normally wrong named files are renamed flawlessly. And I guess you scanner's advanced option "move not renamed files to backup" option enabled, too....so again, normally you shouldn't run into anything manual....
If cmpro says you should rename the chd, let cmpro do so.


Quote
<groan> That's just great.  Well, when I move the program over to D:, is there any way I can restore my current configuration?  It took me probably two hours to get everything defined initially.
two hours? oh please...unzip the cmpro package and if you don't have 1000 defined already, that's it. Maybe create a new MAME profile. If you got 1000 datfiles, you can copy cmpro's datfile folder and let the batchrun create profiles for you...ok..too much advanced stuff already.
But generally unzipping the program, creating a MAME profile, setting up rompaths is a 2 minutes job.


Quote
I can PM you with the file attachments (I think), but I'm not sure what you need to know regarding the files in the drmn2mpu folder.  Can you be very specific?  Like I said, with this program, while I can understand the reason behind its existence, the nitty-gritty details seem to completely elude me.

Simply let me know which files are present in the4 drmn2mpu. If it's only one chd file which got a _randomnumber extension, then simply let cmpro rename it.

Quote
Oh, before I forget, I meant to ask - once the rebuilder is run based on the files found in my usenet download directory, can I then delete those downloaded files?  Or should I keep them around for future rebuilds/rescans?
The rebuilder rebuilds/adds anything valid from the source to the destination. There is an option which removes rebuilt files from the source. These includes zips and folder if they are empty after the rebuilt file was removed. So it cleans itself automatically...Usually you got text files or other garbarge...they won't be removed.
So check what is still in the source after you've used the remove source file after rebuilt.....and save that for later.
Title: Re: Download folder - what's it do?
Post by: Roman on 04 August 2010, 20:28
at least I found the reason for the "version" prompt issue....fixed for next release
Title: Re: Download folder - what's it do?
Post by: fleggett on 10 August 2010, 04:40
The right one is the new name of course. The one without the _18524 extension. What filesizes are you talking about. i don't see any filesizes.
The one I listed earlier with the DOS CLI screenshot.  One file is 99,344 bytes and the other is 151,743.  Obviously, one is very different than the other.  Are you not seeing that screenshot?

Quote
two hours? oh please...unzip the cmpro package and if you don't have 1000 defined already, that's it. Maybe create a new MAME profile. If you got 1000 datfiles, you can copy cmpro's datfile folder and let the batchrun create profiles for you...ok..too much advanced stuff already.
But generally unzipping the program, creating a MAME profile, setting up rompaths is a 2 minutes job.
C'mon, Roman, don't get snarky.  I honestly don't know how to use this program, what with its numerous, rather arcane-sounding options.  Defining system paths in and of itself is VERY clunky and requires a LOT of mouse-scrolling and clicking.  I just spent the last half-hour redefining those system paths after you (strongly) encouraged me to move the cmpro installation folder.  If there was an easier way, such as saving a config file, I'd love to hear it.  And I have NO idea what you mean by letting "batchrun create profiles for you" (though since I'm not using external datfiles, I assume that's not relevant).  When I uninstalled cmpro via its own uninstallation utility, I did see where several files were left over, but I had no idea what to do with them in preparation for a new installation placement.

Quote
Simply let me know which files are present in the4 drmn2mpu. If it's only one chd file which got a _randomnumber extension, then simply let cmpro rename it.
Well, again, that's detailed in the DOS CLI screenshot.  There're only the two CHD files, one with the _randomnumber extension.  And the entire problem is that cmpro does NOT want to rename it (see log entry screenshot).  What other information do you need to know?

Quote
The rebuilder rebuilds/adds anything valid from the source to the destination. There is an option which removes rebuilt files from the source. These includes zips and folder if they are empty after the rebuilt file was removed. So it cleans itself automatically...Usually you got text files or other garbarge...they won't be removed.
So check what is still in the source after you've used the remove source file after rebuilt.....and save that for later.
Hmmm, I'll have to play around with that option.  If you've enabled the option to remove rebuilt files from the download source, why would you want to save any leftover files for later?
Title: Re: Download folder - what's it do?
Post by: Roman on 10 August 2010, 07:33
Quote
The one I listed earlier with the DOS CLI screenshot.  One file is 99,344 bytes and the other is 151,743.  Obviously, one is very different than the other.  Are you not seeing that screenshot?

Ah there...well...what you can do is, simply rename both chds to something random. clrmamepro will detect them as being wrong placed/named and will move/rename them to their correct place/name.

Quote
C'mon, Roman, don't get snarky.  I honestly don't know how to use this program, what with its numerous, rather arcane-sounding options.  Defining system paths in and of itself is VERY clunky and requires a LOT of mouse-scrolling and clicking. 

Actually not. First of all, system default paths is already something the more experienced users use and secondly, Hit the "auto detect sysdefpaths" button and it will assign them for you.

Quote
Well, again, that's detailed in the DOS CLI screenshot.  There're only the two CHD files, one with the _randomnumber extension.  And the entire problem is that cmpro does NOT want to rename it (see log entry screenshot).  What other information do you need to know?
Of course it does not rename it because the new name file already exists as you can see. So do what's stated above, rename them to something random, cmpro will do the rest for you.

Quote
Hmmm, I'll have to play around with that option.  If you've enabled the option to remove rebuilt files from the download source, why would you want to save any leftover files for later?

Because you might need them? Becuase they may be files which get added to MAME later on? Because you collect .txt files in zips ;)...Why should cmpro remove not rebuilt files?
Title: Re: Download folder - what's it do?
Post by: fleggett on 10 August 2010, 22:11
Ah there...well...what you can do is, simply rename both chds to something random. clrmamepro will detect them as being wrong placed/named and will move/rename them to their correct place/name.
Okay, I'll try that and will get back to you.  Right now, I'm running a merging operation that's going to take awhile, as I've got about 200 gigs of files for cmpro to process.

Quote
Actually not. First of all, system default paths is already something the more experienced users use and secondly, Hit the "auto detect sysdefpaths" button and it will assign them for you.
Well, I thought that was probably an "advanced feature", but figured it was better to bite the bullet now rather than tackle it later.  Also, from what I recall, the ADS button didn't assign the subdirectories properly, as I did try the option.  However, when this merging operation is finished, I'll try it again and see what happens.

Actually, regarding the discrete system paths, I've got a problem.  The "move sets" doesn't seem to work anymore.   I did a rebuilder operation last night (which ran just about all night) and it just dumped everything into my root roms\ directory.  And the manual button doesn't seem to do anything.  I'll be able to experiment a bit more after this merging operation is complete.

Quote
Because you might need them? Becuase they may be files which get added to MAME later on? Because you collect .txt files in zips ;)...Why should cmpro remove not rebuilt files?
Hey, that's why I was asking.  And I meant in regards to successfully rebuilt files, not unsuccessful ones.  I'm a big fan of keeping a "pruned" system, so if a file has been processed and isn't needed anymore, I'd like it deleted.  What sort of .txt files would one collect in a zip file?  What sort of files should one definitely keep around for "just in case" purposes?  Both of those are honest questions (IOW, I'm not trying to be sarcastic).
Title: Re: Download folder - what's it do?
Post by: fleggett on 10 August 2010, 22:14
at least I found the reason for the "version" prompt issue....fixed for next release
That's good news.  At least all my I'm sure tedious questions haven't been entirely unproductive. ;)
Title: Re: Download folder - what's it do?
Post by: Roman on 11 August 2010, 07:40
Quote
Okay, I'll try that and will get back to you.  Right now, I'm running a merging operation that's going to take awhile, as I've got about 200 gigs of files for cmpro to process.
Don't tell me you're using the Merger for this....Scanner and rebuilder are way faster and more reliable. The Merger is an old time left over...


Quote
The "move sets" doesn't seem to work anymore.   I did a rebuilder operation last night (which ran just about all night) and it just dumped everything into my root roms\ directory.  And the manual button doesn't seem to do anything.  I'll be able to experiment a bit more after this merging operation is complete.
Rebuilding a whole night? Either you got a very very slow CPU/HD or you didn't use the "no recompress" option.
What is the manual button? What "move set" operation? If you want to rebuild to systemdefaultpaths, you need to tell the rebuilder to do so.

Quote
if a file has been processed and isn't needed anymore, I'd like it deleted.
That's excatly what the remove-rebuilt-sourcefile option in the rebuilder does.
Title: Re: Download folder - what's it do?
Post by: fleggett on 12 August 2010, 03:13
Quote
Okay, I'll try that and will get back to you.  Right now, I'm running a merging operation that's going to take awhile, as I've got about 200 gigs of files for cmpro to process.
Don't tell me you're using the Merger for this....Scanner and rebuilder are way faster and more reliable. The Merger is an old time left over...
Well, yes.  The program and documentation makes it very clear that cmpro MUST know what sort of files its dealing with when doing its various operations, so I figured I'd go ahead and make sure I had the sets in the right "format" (split-sets).  The Merger seemed like the only way to ensure this, so I hit the button and just let her rip.

Quote
Rebuilding a whole night? Either you got a very very slow CPU/HD or you didn't use the "no recompress" option.
No, it's a pretty fast system (Core 2 Quad 9550 [non-OC'd]) with four gigs of PC2-8000 running Win7 Ultimate x64.  And you're right, I had the "Recompress Files" option checked (I don't remember checking it, though).  I take it this is an unnecessary option to enable?  Since it is an option, when is it necessary?

Quote
What is the manual button? What "move set" operation? If you want to rebuild to systemdefaultpaths, you need to tell the rebuilder to do so.
What I referred to the "manual button" is (in reality) the Move Sets button under systems.  Sorry, it was late and I wasn't being clear.

I just tried the Move Sets button again and very little (if any) seems to've gotten moved.  I've still got over 8,700 files in my main roms\ directory.  If they don't belong to any particular system, shouldn't cmpro be moving them to at least the roms\standard\ directory?  I have the "Standard" system directory defined as "G:\M.A.M.E.\MameUI64\roms\standard\".

Here are a smattering of files that are sitting in my root roms\ directory.  They look orphaned, which is why I'm concerned.  I don't know if this will help (maybe you'll recognize some of them):

005.zip
act2000.zip
aliens.zip
dassault.zip
imago.zip
maxforce.zip
parodious.zip
quantum.zip
skywolf.zip
xevious.zip
zoar.zip

Is there any way to determine if these files (and others) are part of a system?

On the bright side, I THINK I'm finally beginning to understand this application a bit better.  Not a lot, but at least a little.  It is definitely not for the faint-of-heart, though.
Title: Re: Download folder - what's it do?
Post by: fleggett on 12 August 2010, 04:10
Oh, on a (positive) note, I did rename both those CHDs that cmpro was stumbling over.  I named the smaller file "small.chd" and the larger file "large.chd".  cmpro did, indeed, pinpoint which file was correct, named it appropriately, and deleted the "bad" file.  I actually had to do that to about ten files.  So that problem has at least been solved.
Title: Re: Download folder - what's it do?
Post by: Roman on 12 August 2010, 07:58
Quote
Well, yes.  The program and documentation makes it very clear that cmpro MUST know what sort of files its dealing with when doing its various operations, so I figured I'd go ahead and make sure I had the sets in the right "format" (split-sets).  The Merger seemed like the only way to ensure this, so I hit the button and just let her rip.
Not really....Scanner shows "you prefer". If you scan a fully merged collection as split merged....well, cmpro will split merge it for you during fixing..of course this takes longer ;)
Merger is obsolete...since years.

Quote
No, it's a pretty fast system (Core 2 Quad 9550 [non-OC'd]) with four gigs of PC2-8000 running Win7 Ultimate x64.  And you're right, I had the "Recompress Files" option checked (I don't remember checking it, though).  I take it this is an unnecessary option to enable?  Since it is an option, when is it necessary?
When you care about perfectly compressed sets...If you're sure that all of your zips are already compressed with a fine zip compression setting, you don't need to enable it.
Or you recompress them later on as a batch (or with Zipmax).

Quote
What I referred to the "manual button" is (in reality) the Move Sets button under systems.  Sorry, it was late and I wasn't being clear.
I just tried the Move Sets button again and very little (if any) seems to've gotten moved.  I've still got over 8,700 files in my main roms\ directory.  If they don't belong to any particular system, shouldn't cmpro be moving them to at least the roms\standard\ directory?  I have the "Standard" system directory defined as "G:\M.A.M.E.\MameUI64\roms\standard\".
Ah...ooook...I will have a look at that functionality. Can you send me your cmpro.ini file plus the .cmp file belonging to your profile (in cmpro's settings folder). Currently I assume you got a rompath "G:\M.A.M.E.\MameUI64\roms" and defined the "standard" sysdeffolder as G:\M.A.M.E.\MameUI64\roms\standard"....
by the way, having capsuled rompaths ain't a good thing to use...

Quote
Is there any way to determine if these files (and others) are part of a system?
Either look at the datfile (if you used an exe to import the data you can export the datfile in set-information window) or check the set-information window and click on the names....you find the "rom-of" information at the top of the screen.
Title: Re: Download folder - what's it do?
Post by: fleggett on 13 August 2010, 01:50
Not really....Scanner shows "you prefer". If you scan a fully merged collection as split merged....well, cmpro will split merge it for you during fixing..of course this takes longer ;)
Merger is obsolete...since years.
Oh, well, errr, okay.  I take it this means it really shouldn't ever be used?  Is that section of the program going to be eliminated in a future release?

Quote
When you care about perfectly compressed sets...If you're sure that all of your zips are already compressed with a fine zip compression setting, you don't need to enable it.
Or you recompress them later on as a batch (or with Zipmax).
Well, I wasn't sure they were already compressed, with a fine zip compression setting or a coarse one, hence why I used this option.

Quote
Ah...ooook...I will have a look at that functionality. Can you send me your cmpro.ini file plus the .cmp file belonging to your profile (in cmpro's settings folder). Currently I assume you got a rompath "G:\M.A.M.E.\MameUI64\roms" and defined the "standard" sysdeffolder as G:\M.A.M.E.\MameUI64\roms\standard"....
That's correct.  It seemed the most logical way to do things in order to fully separate rom files from their respective systems.  Is this not the usual way it's done?

I sent you those two files over regular email.  Please let me know if you receive them.

Quote
by the way, having capsuled rompaths ain't a good thing to use...
Uh oh.  I'm almost afraid to ask what this means.  You mean it isn't a good idea to use the whole "Systems" feature and split out the roms accordingly?

Quote
Either look at the datfile (if you used an exe to import the data you can export the datfile in set-information window) or check the set-information window and click on the names....you find the "rom-of" information at the top of the screen.
Ah, okay.  I'll definitely give that a whirl.
Title: Re: Download folder - what's it do?
Post by: fleggett on 13 August 2010, 02:45
BTW, I have no idea if this'll help in any way, but here's my "System & assigned System-Default Paths"

System Paths (http://www.awfulbooks.com/clrmamepro/system%20paths.JPG)

(Click on "System Paths" to bring up the graphic.)
Title: Re: Download folder - what's it do?
Post by: donnyj on 13 August 2010, 03:24
what is the advantage to having folders laid out that way?
Title: Re: Download folder - what's it do?
Post by: Roman on 13 August 2010, 05:01
Quote
Uh oh.  I'm almost afraid to ask what this means.  You mean it isn't a good idea to use the whole "Systems" feature and split out the roms accordingly?

It means rompaths shouldn't have other rompaths as subfolders.
Title: Re: Download folder - what's it do?
Post by: Roman on 13 August 2010, 05:02
BTW, I have no idea if this'll help in any way, but here's my "System & assigned System-Default Paths"

System Paths (http://www.awfulbooks.com/clrmamepro/system%20paths.JPG)

(Click on "System Paths" to bring up the graphic.)

looks perfectly fine...I only wondered if you got another rompath above that....as somehow mentioned in the "MOVE" issue...
Title: Re: Download folder - what's it do?
Post by: Roman on 13 August 2010, 05:03
what is the advantage to having folders laid out that way?

You got your romsets split by systems....some people prefer that....including myself.
Title: Re: Download folder - what's it do?
Post by: Roman on 13 August 2010, 16:51
Ok, I had a look at the "MOVE SETS" operation in the systems window.
As the prompt (which appears when you click it) says: It scans all system default paths and moves not correctly placed sets to the correct one.

Works fine here...

For you and your setup (iirc, you had unsorted ones in G:\M.A.M.E.\MameUI64\roms and want them moved to G:\M.A.M.E.\MameUI64\roms\standard), you need G:\M.A.M.E.\MameUI64\roms setup as one sysdefpaths, otherwise the MOVE operation won't work for you....
Title: Re: Download folder - what's it do?
Post by: fleggett on 13 August 2010, 19:17
For you and your setup (iirc, you had unsorted ones in G:\M.A.M.E.\MameUI64\roms and want them moved to G:\M.A.M.E.\MameUI64\roms\standard), you need G:\M.A.M.E.\MameUI64\roms setup as one sysdefpaths, otherwise the MOVE operation won't work for you....
But how do you "add" a definition/directory to SysDefPaths?  It looks like all the slots are hardcoded.  I did add "G:\M.A.M.E.\MameUI64\roms\" to the ROM-Paths section of cmpro, but that didn't seem to do anything, so I'm presuming it's all tied into the SysDefPaths screen.
Title: Re: Download folder - what's it do?
Post by: Roman on 13 August 2010, 19:42
You can't add additional ones but you can assign a folder to a bios by double-clicking the "assigned sysdefpath" column.

So..."Move Sets" runs through all sysdefpaths and 'moves' incorrectly placed sets in there to their assigned folder (based on the bios assignment).

In your case the problem is that G:\M.A.M.E.\MameUI64\roms is not a sysdefpath.

You could e.g. move all your sets to your G:\M.A.M.E.\MameUI64\roms\standard folder and then click on move to get them sorted to the others...(well, of course only the ones are moved which don't belong to 'Standard').
Title: Re: Download folder - what's it do?
Post by: fleggett on 14 August 2010, 04:07
You can't add additional ones but you can assign a folder to a bios by double-clicking the "assigned sysdefpath" column.
Oh, believe me, I know.  That's what I was doing before using that Auto button (boy, was that a mistake....).

Quote
You could e.g. move all your sets to your G:\M.A.M.E.\MameUI64\roms\standard folder and then click on move to get them sorted to the others...(well, of course only the ones are moved which don't belong to 'Standard').
Well, I think I may've utterly FUBAR'd my a great chunk of my whole setup tonite.  I don't know what happened, but here's what I did:

1.  From G:\M.A.M.E\MameUI64\roms I CD'd to G:\M.A.M.E\MameUI64\roms\standard.

2.  I then did a "move /o ..\* ." (I'm using JP Soft's TCC) because I didn't want to overwrite anything.

3.  I noticed a LOT of files left over in the "roms\" directory.  This got me to thinking that, perhaps, I had a TON of dupes between "roms\" and "roms\standard\".  I would've thought this would've been caught by the scanner, but maybe not.

4.  I made some visual comparisons between G:\M.A.M.E\MameUI64\roms and G:\M.A.M.E\MameUI64\roms\standard and they all looked the same.  I then went ahead and deleted all the files (not the subdirectories) in G:\M.A.M.E\MameUI64\roms, as they appeared to simply be dupes.

5.  I then ran a new scan.  This took me completely by surprise, as it warned me of MANY instances of what you referenced earlier about capsuled rompaths (I forget the exact wording the cmpro log used).  The Scan function was also complaining that a LOT of roms were in the wrong SysDefPath, but cmpro made no attempt to move them to the right path.

6.  I knew something had gone south at that point, so I panicked (I guess) and ran over to Systems from the Scanner dialog.  I then hit that "SetAutoDefPaths" button and, while it kept some paths correct, the vast majority reset to G:\M.A.M.E\MameUI64\roms\standard.  Remember, before I hit that button, ALL the systems and paths were perfect.  I have NO idea why the majority of paths were (re)set to the G:\M.A.M.E\MameUI64\roms\standard directory.

7.  New scans revealed nothing new.  The multiple capsuled path warning was gone, which was a good sign (I'm guessing), but there was still a lot of roms being reported as being in the wrong place.  No amount of scans resolved this.

8.  Not knowing anything else to do, I manually reset the SysDefPaths (a lot of mouse clicking).  I then tried to automatically move the roms using that auto-move function and it didn't work.  So, I was effectively back at square one.

9.  Not knowing anything else to do part 2, I started rebuilding the sets.  This time, I unclicked the "recompress" option, so it's running at LOT faster (up to 40% by the time of this part of this message).  I expect it'll be done in an hour or so.

I really, REALLY hope I haven't totally corrupted things to the point of no return.  I've obviously got a lot of work to do tonite with cmpro, so I'll do what I can and report back.  If you've got any suggestions in the meantime, though, I'm ALL ears.
Title: Re: Download folder - what's it do?
Post by: Roman on 14 August 2010, 06:48
Calm down.

Cmpro moves replaced and "unneeded" files into its backup folder.
So if you're missing something simply rebuild from that folder to your collection.
Title: Re: Download folder - what's it do?
Post by: fleggett on 14 August 2010, 07:30
Cmpro moves replaced and "unneeded" files into its backup folder.
So if you're missing something simply rebuild from that folder to your collection.
Well, there's almost nothing in the backup folder (14 files), so I can't imagine it helping much.

I don't think I'm "missing" anything.  I just think cmpro isn't placing what I do have in the correct locations.

I guess my frustration stems from exactly what I thought cmpro is suppose to accomplish versus what it actually does accomplish.  For example, these are the first three lines from my scan log (I sent you the complete log over email):

Set exists in various rompaths: G:\M.A.M.E\MameUI64\roms\neogeo\zupapa.zip; G:\M.A.M.E\MameUI64\roms\standard\zupapa.zip
Set exists in various rompaths: G:\M.A.M.E\MameUI64\roms\standard\zortonbr.zip; G:\M.A.M.E\MameUI64\roms\alg_bios\zortonbr.zip
Set exists in various rompaths: G:\M.A.M.E\MameUI64\roms\naomi\zombrvn.zip; G:\M.A.M.E\MameUI64\roms\standard\zombrvn.zip

It says this every time I run a scan.

Why doesn't cmpro look into these files, determine which systems they belong to, and automatically move them to the appropriate locations?  I've got each and every system directory defined, so that shouldn't be the problem.  There must be 200+ entries in my log that say the same thing, so fixing things manually would take a distressingly long time (and leave me with very little hair), as it would also require me to determine which system each rom belongs to.  And the Move Sets button again does nothing.

I've either stumbled across a pretty serious bug with this iteration of cmpro (which is doubtful, else everyone would be complaining), or there's something incredibly wrong with my particular installation, order of actions, or collection of settings (or all three).  Unless someone has a brainstorm, I really don't know what else to do other than starting over (and even then, I have some doubts).
Title: Re: Download folder - what's it do?
Post by: Roman on 14 August 2010, 08:18
Well, basically your setup should not include rompaths in rompaths...that's all you have to care about.
Now that you got multiple instances of files in your rompaths, your cleanup steps will be a bit more ...hmm..manual.
Take one of the multiple instances and move (copy+delete) it to a new folder (not a rompath folder). Then use the rebuilder to readd it to the correct place in your collection.
Title: Re: Download folder - what's it do?
Post by: fleggett on 14 August 2010, 08:37
Well, basically your setup should not include rompaths in rompaths...that's all you have to care about.
I keep hearing this, but don't understand what it means.  So, what should my directory structure look like?  Surely I've got to at least have a root roms\ directory with every system path beneath it...?  Anything else would seem like complete chaos.

Quote
Take one of the multiple instances and move (copy+delete) it to a new folder (not a rompath folder). Then use the rebuilder to readd it to the correct place in your collection.
Well, I'll do you one better.  I'll MOVE everything out of the standard\ directory (even the subdirectories) into my usenet download directory.  Then let rebuilder take it from there.  HOPEFULLY, that will resolve this mess I've gotten myself into.  I just don't have the patience to do this one-at-a-time.
Title: Re: Download folder - what's it do?
Post by: Roman on 14 August 2010, 08:40
Quote
I keep hearing this, but don't understand what it means.  So, what should my directory structure look like?  Surely I've got to at least have a root roms\ directory with every system path beneath it...?  Anything else would seem like complete chaos.

It means one of your rompaths should not be a subfolder of another one.

G:\M.A.M.E\MameUI64\roms\atpsx\
G:\M.A.M.E\MameUI64\roms\awbios\

that's ok

G:\M.A.M.E\MameUI64\roms\atpsx\
G:\M.A.M.E\MameUI64\roms\awbios\
G:\M.A.M.E\MameUI64\roms\

That's bad since the 1st & 2nd rompaths are a subfolder of the 3rd one.


Regarding some automatic clean up of dupes...I will think about it for the next version....
Title: Re: Download folder - what's it do?
Post by: fleggett on 16 August 2010, 03:48
Here's an update...and it's not a particularly good one.

I "removed" the single G:\M.A.M.E\MameUI64\roms\ directory.  As such, my directory structure looked like so:

G:\M.A.M.E\MameUI64\roms\.
G:\M.A.M.E\MameUI64\roms\..
G:\M.A.M.E\MameUI64\roms\acpsx
G:\M.A.M.E\MameUI64\roms\airbios
G:\M.A.M.E\MameUI64\roms\aleck64

...

G:\M.A.M.E\MameUI64\roms\triforce
G:\M.A.M.E\MameUI64\roms\vspsx
G:\M.A.M.E\MameUI64\roms\dir.txt

I'm ASSuming that that lone "dir.txt" file shouldn't cause a problem, as it's something created by MAME during its default installation.  If it is, let me know and I'll get rid of it.

Moved everything out of "G:\M.A.M.E\MameUI64\roms\standard" into my temporary usenet download directory, but left the standard\ directory itself (as empty).

Then let the Rebuilder do its thing.  It should be noted that these are the two temp directories I used (and have them defined as Add-Paths):

E:\download\alt.binaries.emulators.mame
E:\download\alt.binaries.emulators.mame.chd

(If anyone wants to know which newsgroups to use to grab ROMs and CHDs - well, the secret is out.  BTW, does it matter if these are checked or not?)

In Rebuilder, I made sure the following were checked:

Use Add-Paths
Scan Subfolders (this was for the CHDs)
Use System Default Paths
Split-Sets
Compress Files -> zip (this was probably unnecessary)
Show Statistics
Remove Matched Sourcefiles

I then hit Rebuild.

Took it about an hour, complained about five or six files being corrupt (I just deleted those)...and dumped everything back into "G:\M.A.M.E\MameUI64\roms\standard".  The Rebuilder didn't even touch the CHDs.  If I recall, it recognized that they were there, but other that that, nada.  I had to copy those back over manually (which, of course, I had to make the assumption they belonged in standard\ and not somewhere else).

I just can't believe that out of a total of 8981 ROM files that I have (give or take), 7645 of those belong in standard\ and shouldn't go in other system directories.  And the Move Sets button still doesn't do anything.  Maybe I'm wrong.  Maybe only 15% of the MAME ROMs in the wild belong in other system directories, but if that's the case, then there's little reason not to dump everything into just one big roms\ directory (IMO).

Dunno what else to do at this point.  Feel like I've exhausted just about every option.  Should I just wait for the next version and start over?
Title: Re: Download folder - what's it do?
Post by: Roman on 16 August 2010, 07:07
Rebuilder works on ROMs only. CHDs may be included in a future version.
No idea how many sets belong to standard at the moment...have to check that, but compared to the other ones it's the one with several thousand sets.

There is no need to wait...simply scan the collection, if something is wrongly placed in systemdefault paths, the scanner will inform you.
Title: Re: Download folder - what's it do?
Post by: Roman on 16 August 2010, 18:15
~7752 is correct for standard 139u1. Believe it or not ;)
This number can vary on the used settings (like merge mode, fake clones etc). But it's that high....