Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[ReadOnly]

Hello Roman,

Last week I have rebuilt some of my sets to merge them with torrents. I launched the rebuilder before going to work at to my great disappointment it stopped due to an error by the time I was back, a bad zip or whatever. I asked it to ignore and go on, on the morning it stopped again, and so on for days. It was incredibly annoying as you can imagine. So even tho it's too late for me please add an option "always ignore errors" at the first warning. May be helpful for others or might be helpful for me in the future.

On a not so related note, I noticed decompressing and t7z re-compressing manually with a batch file file is about 20 times faster than asking clrmame to do it. Can you look into this please? I don't get why clrmame would be that butt slow, it basically does the same stuff as the batch, minus the checksum verification.

As always keep up the good work.  Thanks.

[Roman]

Well, usually you get an archive error only if something critical happened during writing to an archive (i.e. reading a bad zip will list something in the warningswindow but does not prompt). So the prompt makes sense normally....I will think about an ignore option though.

Regarding rebuilding 7z vs standalone 7z....pretty easy: The rebuilder is file based. If the crc32 hash matches something in the database it will additionally check sha1/md5 (optionally, if available)...this needs decompressing to memory (which -due to an old 7z sdk- is done at once and not done blockwise). Then it adds that one file to the destination archive....as mentioned file by file...that takes long. If you do it in a batch you 7z up a folder most likely and 7z handles it at once...

[ReadOnly]

What do you mean by very critical, that's frightening me. Indeed that wasn't just a bad zip. It seemed to block only on files with very long filenames, over  a hundred characters, a issue with NTSF file system maybe? Of course I am really not sure about it as I haven't performed extensive tests, so can be wrong. What I can say is it was always blocking on the same files.

But whatever. Thanks you Roman.

[Roman]

critical = the created archive is corrupt for whatever reason, disk is full or the filename is illegal (e.g. too long for the filesystem)

[ReadOnly]

currently rebuilding whole 0.148 set and I see this problem still here, cmpro keeps randomly complaining about issues and I have no choice other than resuming manually whenever it happens, I wanted to let the rebuilder work over night but no doubt it's gonna fail at some point, wonder how many days it will take once again to complete the process and resuming whenever I'm back from work or wake up =/

[Roman]

"cmpro keeps randomly complaining about issues", please be more specific. I need more details.

[ReadOnly]

yes should be more specific sorry
still the same problem as OP


happening when rebuilding whole 0.148 mame set with torrent7z in merged mode

verified the source set "avengrgs.zip" was not corrupt
verified the target set avengrgs.7z, not corrupt either and does contain mcg-04.3j cmpro reported, that file is also still there in cmpro temp folder

I've had tons of these errors and if only we could have an "always ignore" option to tick at first error, would help big time, now going to sleep, it will fail and stop, tomorrow morning I'll resume go to work and when back it will be stopped again and so on for days in all likelihood =/

[ReadOnly]

checked computer after waking up, and it failed again after creating 500 files, so if it stops every 500 files and I have to resume manually two times a day (after sleep and after work) then it's going to rebuild 1000 files per day, MAME 0.148 has 214451 files so it's going to take 200 days to rebuild mame set? Whyyyyyyyyyyyyyyyyyyyyyyy?

[Roman]

Well, if 7z creates an error during adding a file, tough luck...could be everything. The prompt is shown when an add operation failed, i.e. the commandline 7z returned an error.
As I said...with 7z it could be everything ...Out of memory (7z is a memory eating snake with 12312312 heads...) and latest alpha and betas of 7z are pretty unreliable.

We can discuss about showing the prompt of simply skip it....that's on my list for ages...but did not have time yet to implement it.

[ReadOnly]

We can discuss about showing the prompt of simply skip it....that's on my list for ages...but did not have time yet to implement it.

You should consider it within the next 200 days then because it's the time it will take to rebuild 0.148 set if you do nothing ;P

BTW I'm using torrent7z so it's a stable version, not some beta, I suspect a memory issue too, it's not an anti virus conflict for sure since the whole cmp folder and the destination folder are both excluded from surveillance

[Roman]

I can rebuild a full mame set so quickly with zip and no-recompression. I wonder if I can even get a cup of coffee in between.
Besides of that...there is no need for a full rebuild anyway. Maybe if you're a starter who only got a mess on your hd...then you do it once. The rest is handled by the scanner and when a new MAME version comes out you add the few files with the rebuilder to your existing collection.

As you might already noticed: I don't really care that much about 7z (or even torrent7z)....such things don't have any priority. 7zip's SDK is let's say it polite: not the best...and currently there are no real good alternatives (well documented dlls (not COM based) or sdks) out there...I check the lib7zip project from time to time...but for current cmpro core, the external 7z exe will be used and I don't see a chance for a change soon.

[ReadOnly]

the problem isn't that much about 7zip being slow than rebuilder failing and ever prompting me, if not for that prompt it would have ended over night

fact: 7zip is now fully supported by MAME, but nobody can force you to add full 7z support to cmpro, it's up to you

the reason why I rebuild the whole set is because of 7z support added to MAME, last time I updated 7zip was not supported, so it's time to recompress my whole set and use all the power of 7z-parent-clone-merging, my estimations based on a sample of file is the set will be reduced to 64% of its current zipped size which is more that enough to convince me and all torrent users

[Roman]

normally such a zip to 7z conversion is done with a commandline script....most likely faster than anything else....
..and actually 7z creates memory holes when used in MAME...haven't checked if that was fixed by now....

..as I said, the rebuilder prompt yes, that's something which should not stay....It should log the errors instead...but as I said...that's on my list already.

[oxyandy]

ReadOnly
It sounds like you are specifying t7z.exe in the Compressor setting of CMP..
Simple, never do this..

Rebuild to Zip, then use t7z to convert Zip to 7z

Plus the other errors you report, (when the process stops), maybe caused by a lack of hard drive space, either the drive t7z lives or CMP temp / program folder ??

[Roman]

Yes...that's what he's doing...
and again...torrentzip is NOT supported

[ReadOnly]

isn't t7z basically a strict cross platform version of 7zip, so therefore less prone to OS specific bugs. mame is compatible with strict cross platform 7zip too, and some 7z files made by windows 7z binaries may not work, so doesn't make much sense to support win 7zip and not t7z

just my 0.02

[Roman]

Your assumption is wrong.

You cannot simply replace t7z with 7z....here is what your sent in details will create:

t7z.exe a -y -r -ms=off -mx9 E:\temp\dest\hustler.7z b

torrent7z_0.9.1beta/Thu Jul 23 03:08:33 2009
using 7-Zip (A) 4.65  Copyright (c) 1999-2009 Igor Pavlov  2009-02-03

warning: unsupported switch ignored: -r
warning: unsupported switch ignored: -ms=off
warning: unsupported switch ignored: -mx9
error: invalid argument: b

7z does not have a problem with this....

[ReadOnly]

the ignored switches shouldn't be a problem, the switches were there when I was using windows 7zip and just forgot to remove them when changing to t7z, but they are just ignored, not the real issue there

the real issue appears to be it believes "b" is a parameter while it's really a filename, all other single letter filenames failed too, not sure if there is a way to go around this, may indeed be considered as a t7z bug

but although that single letter filename scenario is the only one I could reproduce, it's not the only one where rebuilder failure happens, it also happen very often in other scenarios which I was unable to reproduce, these errors seemed to be completely random and for some reason t7z didn't make any log as it's supposed to when failing, why? no idea

[Roman]

maybe you can fool torrent7z by specifying .\%2 in the cmpro commandline option....

[ReadOnly]

That worked fine, thank you Roman, still as helpful as ever, even with (seemingly) "ungrateful bastards" such as myself.

I may sound rude at times, but I am really grateful for clrmame pro, saved me a lot of time. clrmame will remain my main rom manager for all zipped or raw sets, but for 7zip I will start looking at the competition. I believe the root of all of this is in clrmame being a file based manager instead of a set based manager, that can not work fine with 7zip unfortunately precisely because it's not as file based as zip is, that's as much its flaw some will say as it is its power others will say. Fact if you compress a file duplicated 100 times, the resulting zip archive will be 100 times bigger, and it's not true for 7zip, so while it is nowhere as fast as zip, that "checking ahead" is part of what is making 7zip the most powerful archiver in term of ratio. 7zip is being particularly efficient when combined with parent-clone merging so all the more reason to use it with all sets supporting parent-clone relationship.