Topic: Strange 7z LZMA2 warnings (Read 25222 times)

haynor666 · « **on:** 21 June 2013, 19:53 »

clrmamepro some files in mame set packed with 7z 9.22 using LZMA2 mode give warnings. But files are fine for WinRAR or 7zip itself, even total commander 7z plugin sees them as correct. I'm using following options:

a -y -r -ms=off -mx9 -m0=LZMA2 -mfb=256 -md=256m %1 %2

clrmamepro can't move/delete those files or even decompress.

File are:
bingor2.7z problematic file inside archive is bingo roll vip2 v26.02.02_l.bin
kodja.7z problematic file inside archive is kd_20.9c
mladyhtr.7z problematic file inside archive is mlht_20.bin
sc5dndbbf.7z problematic file inside archive is p3209p11.lo

Roman · « **Reply #1 on:** 22 June 2013, 19:31 »

well, usually the errors come from the 7z sdk itself....which is..hmm..let's say...problematic here or there...not to talk about the current alpha beta whatever status of 7z....

you can try to send me the archives and I will have a closer look.

haynor666 · « **Reply #2 on:** 23 June 2013, 20:48 »

Ok, I've put link for these files in PM.

Roman · « **Reply #3 on:** 23 June 2013, 21:32 »

Actually, the error comes from cmpro's full archive test (when you enabled this). It compares the found crc32 from the archive header with the recalculated one by decompressing the data into memory. I will check your sent files later tomorrow (if I find a little time)...

Roman · « **Reply #4 on:** 24 June 2013, 19:04 »

Well tough luck, the sdk call SzArEx_Extract simply fails for decompressing (and so it fails for recalculating sha1/md5/crc32 and comparing them with the header).

...in other words...you have to wait till the 7z sdk gets updated....if this ever happens

haynor666 · « **Reply #5 on:** 24 June 2013, 20:35 »

You are using 9.22 SDK or 9.20 SDK ?

Roman · « **Reply #6 on:** 25 June 2013, 06:52 »

haynor666, sdk 9.22 is used... I debugged it a bit inside of the SzArEx_Extract function but it fails within decoding this LZMA2:17 (whatever type is this...since LZMA2 is no problem) data stream....maybe I will have a deeper look where inside the 7z code it fails...but my time is currently very limited.....

to be more concrete...it fails in 7zDec.c where the status is not LZMA_STATUS_FINISHED_WITH_MARK...."status" at that point is LZMA_STATUS_NEEDS_MORE_INPUT

if (state.decoder.dicBufSize != outSize || lookahead != 0 ||
(status != LZMA_STATUS_FINISHED_WITH_MARK))
res = SZ_ERROR_DATA;

hmm....guess I need more time to debug.....

haynor666 · « **Reply #7 on:** 25 June 2013, 19:26 »

SDK will be probably created when next beta will show up. Just nobody knows when it be.

Roman · « **Reply #8 on:** 25 June 2013, 19:41 »

well...generally the C part of the sdk is limited (esp. when it comes to solid archives). CPP part however uses some annoying COM interface stuff.....if I got lots of time (doubful in the near future) I may look into this again....

Roman · « **Reply #9 on:** 04 July 2013, 21:30 »

some good news....I quickly looked at the CPP 7z sdk part and its way to extract/test/compress files....and hacked in a quick decompress using that...your files are decompressed correctly...

I may have some time end of this month...so most likely I switch to CPP based routines for extraction (maybe later for compression, too) of 7z (currently it uses the C part of the SDK)....blockwise decompression (less memory usage) might be a nice sideeffect....

oxyandy · « **Reply #10 on:** 05 July 2013, 03:40 »

Quote

blockwise decompression (less memory usage) might be a nice sideeffect....

That will make 32 bit users happy !

<----

Roman · « **Reply #11 on:** 05 July 2013, 06:59 »

oxyandy · « **Reply #12 on:** 05 July 2013, 07:06 »

You forget already ?
http://www.emulab.it/forum/index.php?topic=3474.msg13387#msg13387

Bye
Great work Roman !!

Roman · « **Reply #13 on:** 05 July 2013, 07:46 »

Ah...ok...right...totally forgot that there are users which don't have at least 16GB RAM

Actually I forgot that there are people out there which use PCs....I rarely touch it these days...

Well, I can't promise that it will be better memorywise but at least I started to have a look at the CPP SDK code....and I managed to do a quick direct extraction to disk of the problematic 7z in question....
Cleanup and adding callbacks for blockwise checksum calculation etc is next.....but again...I doubt I have time before end of this month...

haynor666 · « **Reply #14 on:** 02 August 2013, 19:51 »

Thanks for fix this problem

I understand this is not implemented in most recent version of clrmame, right?

BTW. I hope some day solid compression will be fully supported in clrmame. Right repacking torrent7z files are big pain.

Roman · « **Reply #15 on:** 02 August 2013, 21:19 »

not yet in since I had less time as expected

haynor666 · « **Reply #16 on:** 19 December 2013, 18:49 »

Problem is not fixed in recent version.

Roman · « **Reply #17 on:** 19 December 2013, 19:07 »

No it's not in...since I actually did not look at it anymore....
7z is very very low priority within the very very low amount of time I got....

get more RAM instead :-)

haynor666 · « **Reply #18 on:** 19 December 2013, 23:10 »

I have 6 Gb

Roman · « **Reply #19 on:** 20 December 2013, 07:37 »

I've got 16 ;-)

I think there was a still a problem with some callback functions (e.g. to calculate sha1 on a buffer etc...)....but as I said...I haven't looked at it anymore...maybe in 2014..

News:

Author Topic: Strange 7z LZMA2 warnings (Read 25222 times)