EMULAB Forum
clrmamepro [English] => clrmame Discussion => Topic started by: Carbon on 28 September 2016, 15:20
-
Until last month CMP worked fine for years, but today when I tried to check my ROMs for the new release, it hangs at 0% at "Scanning Files" (everything else seems to work).
Maybe it's due to the fact that I updated last week macOS to version 10.12 (I'm using CMP using Wine 1.8 under macOS), or the new MAME release is to blame.
Anybody else experiencing any problems?
-
maybe a prompt is shown and that prompt is hidden behind other windows....
-
I already checked that. No such prompt.
-
I'm starting to think that the problem is due to macOS or WINE (or a combination thereof) as I just found out that I can't click on the "Cancel" button either. CMP hangs and I have to force restart it to regain control.
-
did you try the cmpro.ini entry
[CMPRO SETTINGS]
Adv_HideWindow = off
(quit cmpro, edit the file, search the section, set it from on to off, restart cmpro).
not that it's anyhow connected but maybe worth a try....
same for a clean install of cmpro...
....and you can disable all fix options and turn off "ask before fixing"...then you lower the chance that prompts are shown.
-
Setting Adv_HideWindow to off had no effect, as did disabling all fix options and turning off asking before fixing.
-
So your unchanged cmpro installation worked before the OS update..hmm......any update for wine available?
-
Sadly no. I'm stuck with a rather old 1.8 version on macOS, although 1.9 is already available for other platforms.
I will either patiently wait for an update of Wine, or try CrossOver in the meantime (but that one's commercial).
Thanks for your help as usual, Roman.
-
can you make a screenshot from the cmpro windows....so I might get an idea where it hangs
-
Here you go:
(http://img15.hostingpics.net/pics/902659screen.png)
-
hmm...ok...so you don't see anything like the typical
"Creating hash tables"
"Testing unneeded"
which appear after pressing "new scan" but before the single sets are actually scanned?
and the setinformation button is not shown at all (should be at bottom of the scan results tree window)...weeird
-
In fact, I can see "Creating Hash Tables" (runs fine from 0 to 100 %), but then it stops.
So no "Testing unneeded" and nothing else either, of course.
-
Hi, just thought I'd chime in here, as I'm having the same problem as Carbon.
If it helps at all, the Rebuilder appears to work (It updated my collection using a 0.177 to 0.178 update set), but afterwords, switching back to the scanner yielded the same result ('Creating Hash Tables' had already occurred during the use of the Rebuilder, so then pressing 'New Scan' caused an immediate lockup - The scan will not begin, and the animation of the chips and magnifying glass does not move.)
-
So the same version of cmpro worked before? And you updated your OS?
Can you test MAME .177 again?
just in case...MAME .178 added some new systems.
Go to "Systems" and check if there are unassigned paths. If all are not assigned, fine. If some are and some are not, then please assign the missing ones.
Cmpro will show a prompt in case it finds missing entries and that prompt *could* be the reason why it hangs.
-
Hi MAMEBase :)
Interesting, so it is definitely not due to something funky in my setup, and as I noticed that you are on 10.12, too, it seems it is somehow due to an incompatibility between Sierra and Wine.
-
Interesting would be:
- does the system -> assign paths (either all or none) work for you
- does the old MAME version work for you
-
I did not have any undefined paths, but just to make sure I removed them all and re-added them. Made no difference.
There's one thing different for me compared to MAMEBase, though: my "chips and magnifying glass" icon is animated.
-
That may be due to one variable I neglected to mention...
I'm running Wine 1.6.1 (which has worked fine for me up until today)
-
"up until today" sounds like "I haven't changed my system BUT I've updated MAME to .178".
So....any chance to step back to .177 and check if scan starts (you can turn off fix options and stop scanning if it runs).
-
I restored MAME 0.177 from backup and tried with that one: still no luck.
Besides updating to MAME 0.178, both MAMEBase and I (if I can believe the sidebar below a user's name) have updated macOS from 10.11 (El Capitan) to 10.12 (Sierra).
So that could contribute also to the problem we are experiencing.
-
MAMEBase, did you update your OS and/or WINE, too?
-
Again, apologies, but I had updated my OS to 10.12...
I can see if I can get ahold of 0.177, and try that... In the meantime, and thoughts as to whether I should also update my Wine to 1.8.x? (I was waiting for the stable 1.9...)
I'm compiling a fresh 0.177, but I see Carbon beat me to it...
Carbon, I'm wondering if the Rebuilder works for you as well?
-
Sorry, I've never used the Rebuilder, so I don't really know how to use it.
-
The funny part is that between "building up hashes" (which does appear) and the "looking for unneeded" (which does not appear is not that much code...or code which changed lately.
What is in between is the check for system default paths...and if cmpro finds something unusual there it shows a messagebox (which blocks other stuff)...
maybe such a box is shown (or better not shown for you...or not visible) and waits for confirmation....
within the next days I might be able to build you some debug builds which allow some more testing
-
So my next question (albeit one that stems from my own limited experience with CMP) would be: Is there any sort of hidden log file that might help determine at what point the process(es) have stopped?
-
that's what I can enable with some private builds....
-
I see... :)
Thank you for your consideration, and (amazingly) quick attention in this matter! :D
-
well...the latest (non-debug/logging build) is here
http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160928.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160928.rar)
(since I've updated rar/7z dlls recently, the latest ones are included, too).
I doubt that it will fix your problem...but please give it a try.
-
Sorry to say, same result... :(
-
;-) I guessed so...
so I try to set something up in the next days...
-
No worries... In the meantime, if there are other variables that I may have neglected to mention, please let me know, and I'll try to respond promptly! ;)
-
well, you could disable all check and fix options besides "missing"...and/or enable only "sets" + "missing"...
but generally..that's just a trial and error method...pure guessing
-
Same result... Hash Tables are created, and then... Nothing... :(
I'm considering updating WINE, just on general principals... :-\
-
that build was broken anyway ;-)
just for completeness, the current build is here:
http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160928a.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160928a.rar)
-
Well, that's odd... The 'Current Build' you linked doesn't seem to want to open at all... :-\
-
the files from http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160928a.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro20160928a.rar) work fine on my windows machine
and I quickly made this: http://mamedev.emulab.it/clrmamepro/binaries/cmpro_log.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro_log.rar) which will write cmpro.log when you hit scan....would be interesting to see what it holds afterwards (just very high level for now)
-
Well, the cmpro20160928 file opens, and runs (up to the scanning problem), but the cmpro20160928a file for reasons unknown fails to open at all under wine (now 1.8rc-4)
Shall I try cmpro_log with the working cmpro20160928? And to do so, do I simply place the file in the same folder as cmpro20160928?
-
the log file version has a different filename and can be started from the same folder...
it's weird that the 'a' version doesnt work for you since it only differs by one if statement.
if it doesnt open, you might want to double check the 7z and rar dlls....you should use the one from the 'a' version....
enough for today.....need sleep
-
Fair enough, I'll try the 7z and rar dlls, and let you know...
Do I need to do anything other than place cmpro_log.exe in the folder along with the application?
Sleep well, and thank you again for looking into this!
-
no. simply run the log exe instead the normal one (it's a 32build by the way if that matters)
-
This all there is in the log file:
Before BuildHashes
After BuildHashes
-
wow..that's erm....not much ;-)
actually the next log entry would follow 2 lines later....so it gives me a first idea where it hangs (for whatever reason)... thanks...stay tuned for more
-
and again please...
http://mamedev.emulab.it/clrmamepro/binaries/cmpro_log_new.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro_log_new.rar)
-
Too big to quote inline now, so here's the complete log file:
-
thanks....so it hangs during a directory tree walk after file sc4milja.zip (I'm pretty sure you got also sets from sc4* to z* in that folder....but it doesn't reach that for whatever reason.
I will try to rewrite that routine a bit later today
-
For what it's worth, I just tried the cmpro_log_new.exe, and my result was very similar (I got as far as sc4himig.zip).
The only difference I could see is that I keep all my ROMs, LDs and CHDs in a single directory, if that makes any difference.
-
first of all, thanks to both of you....
well... the routine runs through all rompaths and more-or-less lists the files in there (to get an idea what your fav packer type is)...so there's nothing in that treewalk look but
- get the filename, - get the file extension, - count the type
...the one with the highest count wins (e.g. .zip)....
For both of you it seems to freeze somewhere near 26000 files...no typical number for something like a type overflow or something.
As I said...I will try to change that one routine to see if it makes any difference...but it's still weird
-
In the meantime, if there are any other paces we can put cmpro_log_new.exe through that might shed any additional light on this problem, pease let us know... :D
-
http://mamedev.emulab.it/clrmamepro/binaries/cmpro_log_type.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro_log_type.rar)
ok..changed the routine a bit with some more type safetiness...
-
New result: my LD (laser disks) and CHDs folder are now scanned, but hang occurs at same file.
-
Oh, and I just checked, sc4milja is the 25'963th file in my ROMs folder. Along with 671 files in the CHD folder and 8 files in the LD folder, this makes it the 26'642th file examined (this is without counting the subfolders inside the CHD and LD folder). So nothing obvious there.
-
and now with a 1000 files limit counter...
http://mamedev.emulab.it/clrmamepro/binaries/cmpro_log_limit.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmpro_log_limit.rar)
(beware that the exe files are named differently....so don't run an old exe by mistake ;-))
-
I think there is progress: CMP still stops at sc4milja, but now I see all the file names in "Testing Unneeded" being tested (before there was no output at all) and there are additional lines at the end of the log file.
-
well....the unneeded scan makes a directory walk, too....
so if it freezes now there we're back where we started...a problem when a directory walk reaches a special limit.
Nothing really has changed in such routines for years....so I somehow really think it's a bug in WINE.
-
FWIW, I just tried CrossOver (commercial version of WINE), that's much more recent as the current release of WINE (dates from September 20th), and it exhibits the same behaviour. :(
-
just wondering...where are thte rom sets stored? on a (samba) network drive?
Maybe something special with the file it hit? Something locking the access?
It's still weird. MAMEBase mentioned that the rebuilder worked for him...and the rebuilder also uses a common directory tree walk.
Since it's not WINE...maybe it's the OS? What I can do is to setup a small Ubuntu setup with WINE to see if behaves the same...sorry no Mac OS available here.
Another idea would be to split up the rompaths...what happens if you split the one rompath with >20000 sets in 2 with ~10000 sets?
-
(Just back from work)
So I tried the latest cmpro_log_limit.exe, and told it to scan only sets, and check only for missing sets.
Like Carbon, I got further than before, but still locks during checking for unknown sets,
Log file enclosed.
I'm considering making a temporary ROM Directory with about 500 sets included, and see if that makes a difference.
-
Okay, so here's some progress (I think... :-\ )
I tried making a new directory (ROMS Temp) that started off with the first 1,500 sets...
I was able to run a scan to the end (Having checked only the 'Sets' and 'Missing')
Upon completion, the expected results widow comes up, but is empty, and when attempting to redo the scan, all checkboxes are extremely slow to respond (I thought it had locked up again, but it appears that all processes are taking a minute or so to respond to mouse clicks)
I was eventually able to start a second scan (with all options enabled). The statistics window still comes up blank upon completion, but at least the scan went to 100%.
I'll have to do more testing tomorrow, once I get home from work, adding more files to see if there's some 'magic number' that causes the lockup, but in the meantime, I'd count this as progress! :D
-
ok...let's sum up the things.
- all was good before updating your Mac OS
- Wine and Crossover show the following problems now:
- a directory tree walk seems to freeze when a folder holds something around 25000 files
- if using rompaths with less files, the scan runs but the stats window is empty? (screenshot please) And some background process seems to be still running.
Well, currently from what I read I'd blame the OS update :-(
Intersting to know would be: Where are the files stored (network drive, etc..etc..) and where is cmpro stored. Do such folders need additional access rights, etc etc...
-
Here are some additional observations:
- all my files and CMP are on local disks (in fact on the internal disk drive), no special permissions or anything applied
- I'm currently running CMP under full virtualisation (VMware): no problem here, it is at the stage of checking the ROMs (although it is extremely slow, after 10 minutes still at the letter 'a', hence why I prefer to use Wine, which is much faster)
I think we can conclude with some confidence that a change in macOS 10.12 triggered something in Wine.
-
All righty then...
Based on what we know so far, the problem appears to be related to the update to Mac OS X 10.12. What remains unknown, is whether the problem lies in the OS, or WINE (I think we've effectively eliminated CMP, and/or changes in MAME as the culprit).
The remaining question then becomes; Where do we go from here?
Roman - I cannot begin to thank you enough for taking the time to look into this matter... If you think additional testing would be beneficial, I'm on board! (Incidentally, you mentioned that you don't have a Mac OS environment to test in... Were that to change, would that be of any use to you? I'm not making any promises, but I'm looking into options... ;) )
Carbon - For me, it's looking like my next step will be to set up a Boot Camp partition, and run CMP under actual Windows (Seems a bit of overkill for the ability to run a single Windows-Only App, but as long as I can point to the ROMs directory on my Mac HD, and have CMP work its magic, it's a livable, albeit (hopefully) temporary solution...
We can't be the only ones affected by this...
Which brings me back to the previous question; Is the next logical step to contact the author(s) of WINE, and see if they can pinpoint the fault? It would be a simple matter to supply them with a copy of CMP, but they would then need all 36,000+ Rom Sets to fully test (not an impossibility, but the logistics leave much to be desired... :-\ )
-
Currently I'm knocked out by a damn cold. I come back on this topic around tuesday I think.
-
ok..fully back tomorrow I guess....
Regarding "Upon completion, the expected results widow comes up, but is empty, and when attempting to redo the scan, all checkboxes are extremely slow to respond (I thought it had locked up again, but it appears that all processes are taking a minute or so to respond to mouse clicks)"
What is the "results window", the scanner tree with the expandable results or the statistics window?
Do you auto-save fix datfiles? Scanner Tree window context menu Save...
If so, disable it for a test.
Regarding the tree-walk problem, I will try to create a mini stand alone program which simply prints out the files for a path....similar what cmpro does with it. If this fails on MacOS already..tough luck...but maybe useful for Wine developers then. Maybe it's simply a problem with the compiler optimization levels....
well..again...recovering at the moment....will check later this week
-
ok...a did a very simple test program
http://mamedev.emulab.it/clrmamepro/binaries/DirWalk.rar (http://mamedev.emulab.it/clrmamepro/binaries/DirWalk.rar)
The rar file holds 3 exe files
one with default visual studio 2015 community compiler settings
one with extra compile set to support older OS (like XP)
one with the settings used in cmpro
A window, a scan button (where you should select your rompath with > 25000 files). After selecting it, the box below the button gets filled with the single filenames (no directory recursion)...top right shows the file counter.
Successful run: All your romfiles should appear
Failure: well, as you know it from cmpro, it holds somewhere...most likely somewhere near 25000 files starting with s*
would be interesting if that works for you...if not I'm happy to upload the source code so that WINE developers can have a look at it....
-
Output from DirWalk_default.exe:
Found Files: 25'958 (there are in fact 33'552 fils in the folder) and nothing appears in the text box below
Output from DirWalk_Opti.exe:
Found Files: 25'716 (and nothing in the text box below, either)
Output from DirWalk_XP.exe:
Found Files: 25'923 (still nothing appears in the text box)
Running DirWalk_default.exe a second time, now there were 25'943 files found.
Subsequent running of the 3 programs always yielded in slightly different results.
-
so it has the same problem no matter what the compile and link options are...very interesting.
I will upload the source code tomorrow...
and I guess some other path with less files works for you...
thanks for testing.
-
Yep, selecting for example my Art folder containing 1389 files, all are displayed correctly in the text box and additionnally I can quit the program by clicking on the close box.
In the previous case, I had to quit them from Wine's Task Manager.
-
I hope I find some time today to:
- run a quick test with Wine/Ubuntu (pretty stupid I know...)
- modify the test program to see if a non-MFC version would work
- release the source
Well, the basic code is something like this: https://msdn.microsoft.com/en-us/library/f33e1618.aspx#Anchor_2
Question is: maybe WINE's interpreation of the MFC CFileFind calls opens but does not close files within this loop and you somehow run out of filehandles...or something...pure guessing here.
-
http://mamedev.emulab.it/clrmamepro/binaries/DirWalk2.rar (http://mamedev.emulab.it/clrmamepro/binaries/DirWalk2.rar)
with sourcecode....
I've added a second scan button. One uses the MFC function calls, one plain Win32 api...
and I've removed filtering of directories and system files...maybe *that* got a problem (I've read some old article on directory checks causing an assertion from time to time...so let's test this).
So...If both buttons fail for your...you got the sourcecode to go to WINE/Crossover developers to check what's going on.
-
And here are my results:
- Scan MFC button: stops at 25'782 files, no output in text box
- Scan button: stops at 25'781 files, no output in text box
-
I took your advice and went to the WINE site to explain the bug. There I noticed they had an up-to-date build for macOS (I'd always used WineBottler, a nicely packaged version with additional features like easy environment set-up, but alas a few versions behind).
So I tried WINE 1.9.19 anyway, and there it worked! So I will use that version until WineBottler is updated.
Sorry for bothering you Roman and, once again, big thank you for your help and of course CMP Pro :)
-
hip hip hurray....now what to do with the preordered mac? ;-)
-
Well, I guess this is what happens when I leave town for a couple of days... :-\
I'm thrilled that it's looking like an update to WINE will solve this issue (Now I just have to figure out how to do that myself... ;) )
Roman - Again, thank you for the amazingly quick responses, and willingness to help with this issue (and, of course for CMP... ;) )
Carbon - If you have a spare moment or two, if you wouldn't mind PMing me, and let me know what steps I might have to do, to get my own WINE environment situated (I believe they're up to 1.9.20... :) )
-
just out of interest....what were the options for the macOS you were talking about ;-)
-
Well, to be honest, I was starting to look over eBay (among other places) for a reasonably priced Mac Mini that I would then offer to donate to the cause... ;D
Sure, you'd still have to supply the monitor, keyboard and mouse, but it would provide you with a Mac OS environment to play with/develop in... :)
Who knows? It might even inspire you to develop a Mac/Linux version of CMP (I kid... I'm not sure if that's even a possibility)
I'm not a rich individual, by any stretch of the imagination, but I do like to contribute what I can, when I can to the community (I got my start with MAME around version 0.2x... Back when the ROM Set total was less than 100... I used to maintain a small, external database of ROMs, cheats, and other, miscellaneous files needed to make MAME work... Hence my Screen name). MAME has grown exponentially over the years, and I struggle to keep up with it all... I'm relatively new to CMP, and all it can do (up until a couple of years ago, I would attempt to manage my ROM Sets by hand, using a variety of Mac utilities/frontends), and as a retired Apple Service Tech, I was initially reluctant to make use of CMP, as it was a windows-only utility. WINE has made it possible for me to make use of CMP, and I can't imagine not using it going forword... ;)
If supplying you with a Mac OS environment would prove at all beneficial, I'll continue my search... If not (and my feelings will not be injured in the slightest, should that be the case), please let me know, and I'll focus my attention elsewhere... :D
-
wow...what a generous offer...but use your time with your family and friends...and spend your money on them ;-)
I think postage and custom fees on top of it will still make it way too expensive as a cmpro donation.
If you're Tim Cook and you need a new one and got one old over and before you put it in the bin, I'd take it of course.
I'm glad the problem is resolved...
A native cmpro version is not planned...the core is old...clrmame is old...actually I've started 1997. Actually I always wanted to start over..with something better portable and way of new ideas ...but with family and real life the time is really limited...and when I started, users came back with a bug in the current core...or MAME devs decided to change something.....but I still like the idea of starting over..hehehehe...