EMULAB Forum
clrmamepro [English] => clrmame Discussion => Topic started by: SpaceAgeHero on 21 February 2016, 06:18
-
Hi Roman,
wouldn't it be awesome to have clrmamepro running on Linux natively? Isn't it time to move away from Windows?
How about opening source on Github? It'll be fun.
-
Time to move away from Windows...damn...and I thought Desktop Linux failed....
Enough kidding...no, the current core won't run on Linux kernels natively nor will it be easy to update the code so that it runs there.....for now it remains closed source.
-
Hello,
Moving to .NET could be a first step to the universal app, Linux and MacOS X users will be able to run cmpro using Mono instead of Wine which is painful as soon as an app is using lot of disks I/O
btw, as I talk about I/O : cmpro seems to be incredibly slow on empty subdirs when "looking for unknown sets", that strange behavior is far more visible thru samba shares, at a point that seconds become minutes
regards
-
you mean an empty folder? with no files or a folder with no files belonging to the currently loaded database?
wine or native windows?
-
I mean all setnamed folders that should contain chd files inside rom path, except that almost chd were removed, and so those folders are currently empty
This happens on clrmamepro thru wine with direct access on a mounted ntfs drive *and* also on windows native when accessing thru samba share to that same mounted ntfs drive (using ntfs-3g fuse filesystem), the cpu goes above 80% on mount.ntfs-3g process during that phase (and it's significantly faster on checking unknown zip files than those empty directories)
Please note that I'm currently converting that drive from ntfs to ext4
I tried to scan the same rom dir with RomVault (using Mono and also native windows thru samba share), and got no general problem of performance, but I don't know what is exactly done while scanning with that rom manager...
Please also note that the rom path on NTFS drive is a SYMLINKD on another ROM path on that same drive
-
so you got for example "your_rompath/area51/" (for the chd) but nothing ( no files at all) in that subpath and area51.zip with the roms elsewhere...
I will have a look.. ..
-
Yes, that’s right
It’s faster with ext4, but still present, it should be also visible on windows with slow drives (let’s say on usb2 keys for example)
-
hmm...I've created a new rompath and batch created something around 700 empty folders in there for the chd sets in mame and started a scan with that one as rompath...and it flies right through it...no delay, no lag....
that's with native windows 10
-
That may not be noticeable if your hardware is too fast, I started to see a problem when I was working on slow hardware, but even under wine with ext4 on sata drive or usb3 drive, that won't be visible, but if you use usb2 port, and/or an old drive, and/or a shared network, or whatever which slow down I/O you should start to notice that "looking for unknown sets" phase will be slow on empty directories, and will accelerate on zip files...
I tried ntfs and ext4 (but not fat), wine and native, local and shared network... the problem is present everywhere at different degrees when using slow hardware
Another odd problem on slow hardware (in my case it's an Intel Celeron J800 with 2GB DDR3@1333) especially with wine, when I click on "Scan" or "New Scan" with a profile that use a full mame software list, I have to wait lot of minutes (around 30mn) before the progress bar start to show with "Creating HashTables"... During this time the only thing I see is cmpro's memory usage growing slowly from 270MB up to ~1GB , no disk io, and its process at 100%, I suppose you are making many small struct allocation in memory for each rom/set... is it possible to look at that part to see if it's possible to optimize? Please note that this 30mn wait does happen with a freshly started cmpro, it does not happen if I start a scan again, or if I reload the same profile
Thank you
-
Well, yes, I'm on a rather fast PC (I7-8700K, 16GB DDR4@2666) but still, an empty folder shouldn't have any impact at all, since well, cmpro will check the files in there during the unneeded check....no files, no check...
Sure, during the Creating HashTables process a hashtable containing any rom plus hash is generated, but still, since the final size is known it preallocates it first and then fills it up.
When I find a little time I can try to optimize it....
-
So if I understand well the situation, on my J1800 with wine (debian buster), the preallocation takes 30mn, then the progress bar show up, then the hash table is filled up and take less than 1mn
Many thanks to look at it when possible, an optimisation would be greatly appreciable on this preallocation part
BTW, on my Core-I5 3570 with native windows 10, this is almost immediate, there is a big gap! I would be tempted to try with virtualbox on my I5 just to see if it's a wine problem or a hardware problem...
-
no no no...the preallocation should be no time at all...since this is just a one line allocate memory......I thought the filling (during progress bar display) takes long....very strange...I will have a look what happens before the actual progress bar stuff.....(still weird...since nothing really happens there...)
-
yes it's a very slow memory allocation problem, I'm currently installing a fresh debian on virtualbox on my windows computer to see if the problem is linux/wine or hardware
-
Same problem under virtualbox on my Windows10 PC with Intel I5, it took 25mn for memory allocation, so it's not an hardware problem
I'll now try 2 other things :
- Wine32/CMPro32
- Debian Stretch (stable) instead of Debian Buster (testing)
-
I highly doubt it's based on manual allocation....
you shouldn't waste time finding something on your end until I looked at the source to see what's happening there...by the way what's mn stand for? don't say minutes....In the meantime I've tested in on an Ubuntu Virtual box with wine...and I don't see any enormous delay.....
-
ok I don't say minutes then (but I think hard enough)
-
One more thing... this memory alloc slow down is happening only with software lists included (with all configured systems/roms paths), it's immediate (<1s) with no software list
-
now *that* is interesting and actually you should use softwarelists in singe profiles, not in the combined mode, especially not when you're on an old machine.
-
Indeed, Intel J1800 is a slow processor, so yes, I could use batch mode with single profiles for software lists, *but* with my I5-3570 it's anything but slow, combined mode is very fast under windows, and even with virtualbox+wine overhead, that's not normal that the allocation phase for hash tables goes from <1s on windows to >1500s on VB+Linux+Wine with the same computer...
Anyway, I'll check on a Dual Xeon E5-2640v3 server with 256GB DDR4@2400, SSD only, running Proxmox
-
use native Windows then. cmpro is not intended to be used under WINE ;-)
-
Hello
Ok, so I started to write from scratch another Rom manager in Java...
Regards
-
Good luck
-
Thanks...
https://github.com/optyfr/JRomManager
;)