EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: 1 2 [3]   Go Down

Author Topic: 7z blockwise memory decompress  (Read 34471 times)

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 269
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Firefox 25.0 Firefox 25.0
    • View Profile
    • .
Is the bug is specific to 7z only
« Reply #40 on: 14 March 2014, 09:11 »

Is the bug is specific to 7z only ?
OK, let's answer that question.
First, let's explain ORIGINAL archive has 3 files.


2 at Root level and one in a subfolder (file is same CRC as one of the root files)
Dat should remove the subfolder & it's contents.
Instead a rename is taking place, creating a dupe entry.

3 TESTS, ZIP, RAR, 7z
ZIP
No Bug, after "Scanner" is complete, archive is perfect 2 files inside.


RAR
Yes, after "Scanner" is complete, archive has 3 files inside. (inc one dupe)
Another "Scanner" "New Scan" test, does NOT produce a warning message.
Can I consider no warning of dupes a bug too ? ZIP & 7z report this error, I checked.


7z
Yes, after "Scanner" is complete, archive has 3 files inside. (inc one dupe)
Another "Scanner" "New Scan" test, does produce a warning message.

« Last Edit: 14 March 2014, 11:20 by oxyandy »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 116
  • Offline Offline
  • Posts: 3305
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 33.0.1750.149 Chrome 33.0.1750.149
    • View Profile
Re: 7z blockwise memory decompress
« Reply #41 on: 14 March 2014, 09:52 »

well...it's what the external packers do if they readd a file which already exist. zip/rar removes the one instance and 7z seems to simply add it...so you got 2...

will check if there is a cmdline option to force a replace....or I need to manually remove it first...will check that later
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 269
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Firefox 25.0 Firefox 25.0
    • View Profile
    • .
Re: 7z blockwise memory decompress
« Reply #42 on: 14 March 2014, 10:13 »

I will leave it in your very capable hands.
Here are 3 perfect original archives, one of each type - zip, rar, 7z to do the tests with.
And the required DAT.
"Scanner" will break the Rar & 7z archive (see above) - zip will be fine.
« Last Edit: 14 March 2014, 16:41 by oxyandy »
Logged

Starshadow

  • Member
  • *
  • Karma: 1
  • Offline Offline
  • Posts: 42
  • Operating System:
  • Linux Linux
  • Browser:
  • Firefox 27.0 Firefox 27.0
    • View Profile
Re: 7z blockwise memory decompress
« Reply #43 on: 14 March 2014, 17:17 »

Starshadow please,
Some feedback if you will, for the massive over 4Gb archives, even with the 32Bit CMP if possible.

>4GB 7z archives have no unique problems in cmp20140313 both 32 and 64 bit. Testing, hash checking, and extracting for rebuild all seem to be working fine for them. The hash checking could be faster for large files if it used multi-threading and multiple buffering, but that's a discussion for another day :-X. Thanks for your hard work Roman!
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 116
  • Offline Offline
  • Posts: 3305
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 33.0.1750.149 Chrome 33.0.1750.149
    • View Profile
Re: 7z blockwise memory decompress
« Reply #44 on: 14 March 2014, 21:53 »

http://mamedev.emulab.it/clrmamepro/binaries/cmp20140314.rar


btw...I don't have any issues with 7-Zip [64] 9.32 alpha  (2013-12-01) (64bit)
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 269
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Firefox 25.0 Firefox 25.0
    • View Profile
    • .
Re: 7z blockwise memory decompress
« Reply #45 on: 15 March 2014, 01:02 »

Very cool  8)

Fix confirmed !
Now oxy rushes off to work !
Logged

oddi

  • Member
  • *
  • Karma: 2
  • Offline Offline
  • Posts: 193
  • Operating System:
  • Windows NT 6.3 Windows NT 6.3
  • Browser:
  • Firefox 27.0 Firefox 27.0
    • View Profile
Re: 7z blockwise memory decompress
« Reply #46 on: 15 March 2014, 08:46 »

Many tnx guys ! :)
Roman write here your 7z settings in cmp. Now me repalce with new cmp forum build and reinstall 7zip 9.32 x64, but not have time for test. Later test it .tnx :)
« Last Edit: 15 March 2014, 08:47 by oddi »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 116
  • Offline Offline
  • Posts: 3305
  • Operating System:
  • Mac OS X Mac OS X
  • Browser:
  • Safari 7.0 Safari 7.0
    • View Profile
Re: 7z blockwise memory decompress
« Reply #47 on: 15 March 2014, 12:26 »

I'm using the default ones
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 269
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Firefox 25.0 Firefox 25.0
    • View Profile
    • .
Re: 7z blockwise memory decompress
« Reply #48 on: 15 March 2014, 14:48 »

oddi,
Use the same 7z cmd line settings as you used previously.
The 'Default' setting with modern 7z.exe will create LZMA2 archives, LZMA2 can not be processed by t7z..
Setting t7z.exe directly in CMP's settings is just crazy.
So post processing t7z.exe with after CMP "Scanning & Fixing" is unavoidable.
Hence the reason I set -mx0
Plus the way CMP works 'as current' by adding 'file by file' on a Rebuilder run, so using higher 7z compression is a waste of time & resources..
I would never suggest anyone try Rebuild to 7z (a whole set)
This is why I call my 7z cmd settings 'Maintenance Settings" for using CMP with an already 7z & merged set.
Someone wishing to convert a MAME split & zip set to merged 7z, should use ZIP and "Scanner" to merge & Fix FIRST.
Then send the 'ROM Root' to t7z.exe (or Drag, Drop folder onto t7z.exe) to convert to t7z..
After complete, then configure CMP to work with 7z.
(I will re-t7z the 'ROM Root' occasionally, definitely not with every dat update)
Maybe if Roman makes some core changes 'one day', well that advice might change  ;D

Roman,
Monday I will continue to test everything I can think of - starting over again with today's build.
Thanks for the effort you have put in, it's really appreciated.
I know I have worked with you fine-tuning CMP to be the best it can be, and you haven't rejected anything.
The new CMP opens up 7z usage, on a 32bit OS which couldn't be achieved previous to the latest updates.
Champion you are !

Starshadow,
+1 for your effort on checking the >4Gb 7z, both 32 & 64
« Last Edit: 15 March 2014, 17:37 by oxyandy »
Logged

oddi

  • Member
  • *
  • Karma: 2
  • Offline Offline
  • Posts: 193
  • Operating System:
  • Windows NT 6.3 Windows NT 6.3
  • Browser:
  • Firefox 27.0 Firefox 27.0
    • View Profile
Re: 7z blockwise memory decompress
« Reply #49 on: 15 March 2014, 17:41 »

Oxy tnx for the info, me use old "best" cmp 7z settings :). Now start activity use cmp x64 + 7z 9.32 x64, if found any bug report here. Tnx for all :)
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 269
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Firefox 25.0 Firefox 25.0
    • View Profile
    • .
Re: 7z blockwise memory decompress
« Reply #50 on: 15 March 2014, 18:12 »

You can use the 7z.exe (cmd line program) from 9.32 x64...
Just copy & paste the 7z.exe & put it in CMP's program folder..
(Path in settings 'default' 7z.exe)

Because, if you don't want the warning about "Information past the end of file" that t7z files cause,
then use an 'Older Version' of the 7z UI, sorry not sure which one, you will need to try yourself..

I haven't tested 7z.exe 9.32 much, it seems to work OK..
On Monday I will use it exclusively with CMP and report any issue.
I haven't used it enough to be confident there is any difference to using 9.28..
When I did the testing with 9.28, the 'latest alpha' at that time was 9.29 and that did not work as well as 9.28.
That's about all I remember, haha.
Now latest alpha is 9.32..
I am not sure there is any reason to update from 9.28 cause for me, 9.28 works very reliably.
Remember we only use the cmd 7z.exe in CMP only with 7z archives.
So if you don't use LZMA2, and one day will t7z the set, then 7z.exe is just a 'middle man'
Still I will test 9.32 thoroughly.
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 269
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Firefox 25.0 Firefox 25.0
    • View Profile
    • .
Re: 7z blockwise memory decompress
« Reply #51 on: 15 March 2014, 18:33 »

I just tried 9.32 7z UI, seems fine.
Oh, this warning ?



That is nothing ! You only see that when you click "Information" button.
I thought maybe 7z UI was reporting error OR (Annoying pop-up box) (Click OK) when testing archives or something.
Again Nothing, 9.32 seems fine, use it then.
2.30AM (again) zzzzzzzzz
« Last Edit: 15 March 2014, 18:37 by oxyandy »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 116
  • Offline Offline
  • Posts: 3305
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 33.0.1750.154 Chrome 33.0.1750.154
    • View Profile
Re: 7z blockwise memory decompress
« Reply #52 on: 15 March 2014, 18:47 »

Thanks for testing everybody....highly appreciated...looks like you get something before Easter Holiday :)
...and I'm happy that after all these years I finally got the c++ sdk part running (not so bad after all :))

One of the bigger things on the list done....next thing..hmmm..new rebuilder core....(ehe...I'd say don't expect anything before 2015....)
Logged

oddi

  • Member
  • *
  • Karma: 2
  • Offline Offline
  • Posts: 193
  • Operating System:
  • Windows NT 6.3 Windows NT 6.3
  • Browser:
  • Firefox 27.0 Firefox 27.0
    • View Profile
Re: 7z blockwise memory decompress
« Reply #53 on: 15 March 2014, 18:49 »

Oxy, about saga with 7z, me used Total Commander with two internal plugins for 7z, yesterday upgrade manually one plugin to 9.32 x64. Separated have instaled version of 9.32 x64 in win8.1 x64. Little circus :P :P :P
Btw - about my mame t7z sets, see pic: ( date column ). That is not problem, I dont know where is problem with incorrect date TC or 7z, but is not fatality.


Roman , if u not remember - don't touch my engine.cfg  :P :P :P

Found one problem with t7z LZMA - cmp with 7z 9.32 not remove files from t7z archive. If decompress and build new 7z archive with 9.32 LZMA2 - cmp no problem with remove files inside archive.
« Last Edit: 16 March 2014, 05:39 by oddi »
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 269
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Firefox 25.0 Firefox 25.0
    • View Profile
    • .
Re: 7z blockwise memory decompress
« Reply #54 on: 16 March 2014, 09:33 »

Hi oddi
Quote
"Found one problem with t7z LZMA - cmp with 7z 9.32 not remove files from t7z archive."
Well, see that's important, I will confirm same issue.
Maybe have time this evening (another 3-5 hours)
Oh are your delete strings correct ?
I use
Code: [Select]
d -y -ms=off -m1=LZMA -mx0 %1 %2
___________________________________________________________________________________

Yes confirmed,
9.32 is not removing files from t7z archives..



Swapped back to 9.28 alpha - worked perfect. ;)
« Last Edit: 16 March 2014, 09:44 by oxyandy »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 116
  • Offline Offline
  • Posts: 3305
  • Operating System:
  • Mac OS X Mac OS X
  • Browser:
  • Safari 7.0 Safari 7.0
    • View Profile
Re: 7z blockwise memory decompress
« Reply #55 on: 16 March 2014, 14:03 »

well...removal is done via calling the 7z exe with the parameters stated in the 7z tab....maybe they are not supported anymore in 7.32....
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 116
  • Offline Offline
  • Posts: 3305
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 33.0.1750.154 Chrome 33.0.1750.154
    • View Profile
Re: 7z blockwise memory decompress
« Reply #56 on: 16 March 2014, 14:26 »

delete works fine for me with

"7-Zip [64] 9.32 alpha  Copyright (c) 1999-2013 Igor Pavlov  2013-12-01"

..and delete settings:

d -y -ms=off -mx9 %1 %2


...maybe you're using solid archives...
« Last Edit: 16 March 2014, 14:26 by Roman »
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 269
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Firefox 25.0 Firefox 25.0
    • View Profile
    • .
Sorry
« Reply #57 on: 18 March 2014, 10:20 »

I am guilty of not even opening CMP since Sunday evening.
When time permits, I will do some more checking.
Logged
Pages: 1 2 [3]   Go Up
 

Page created in 0.163 seconds with 19 queries.

anything