EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: Problems with files ≥4 GB  (Read 18969 times)

amarok

  • Karma: 0
  • Offline Offline
  • Posts: 6
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Problems with files ≥4 GB
« on: 09 May 2009, 23:09 »

Hi,
I'm having some problems with files bigger than 4 GB. The files are compressed using 7-zip 4.65. clrmamepro's log window shows the following (just an excerpt, it does happen with all files ≥4 GB):
Code: [Select]
Corrupt Archive File:C:\Emulation\Dumps\Sony Playstation 2\Champions_E.7z | Reason: NO ENTRIES
That file is actually perfectly valid; I tried uncompressing it and checked the CRC manually - it matched. I'm using Vista64, the 64 bit versions of clrmamepro and 7-zip and I'm on an NTFS partition, so I can pretty much rule out any 32 bit file size limitations.
Any suggestions? :D

TIA, amarok
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #1 on: 10 May 2009, 08:35 »

hmm...7z...interesting...if it would be zip I tell you that I only support standard zip which is limited to 4gb. The 7z sdk is lousy but I will have a look and check if I find something suspicious.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #2 on: 10 May 2009, 18:48 »

actually I'm looking into a newer 7z sdk at the moment...and there might be the chance that the known issues might get solved soon...so stay tuned ;)
Logged

amarok

  • Karma: 0
  • Offline Offline
  • Posts: 6
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #3 on: 10 May 2009, 19:04 »

Wow, that would be so awesome :D Please keep me updated :)
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #4 on: 11 May 2009, 18:59 »

well....the new sdk fixes the >=4GB access issue, however unzip/test operations are still not blockwise.....at least I didn't find out yet how to write my own blockwise routine.... :-\
Logged

amarok

  • Karma: 0
  • Offline Offline
  • Posts: 6
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #5 on: 11 May 2009, 21:18 »

That's great news! Well, at least the 4 GB part, because I don't really understand what "blockwise" means in this context :) can we expect to see a 3.126b soon or will I have to wait until 3.127 or later for the new SDK to be implemented?
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #6 on: 11 May 2009, 21:38 »

Not really good news since the 4GB issue is only fixed for the table-of-contents reader....

With blockwise I mean a method to decompress/test 7zip data in chunks of some reasonable memory (e.g. 1 MB).
Currently it tries to allocate the unpacked size of the file for this....which of course causes problems for huge files.

So if anyone writes some block-wise C based "SzAr_Extract" function (see 7z sdk 7zExtract.c), please let me know.


No idea when a new version is released. My private life takes nearly all my time at the moment.
Trying to find a new and much bigger appartment plus full time job...
Logged

amarok

  • Karma: 0
  • Offline Offline
  • Posts: 6
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #7 on: 11 May 2009, 22:10 »

I see, that's too bad. Personally, I'm not a programmer at all :( Maybe someone else will eventually run into problems with these huge files too and write that function.
Oh, and good luck with that apartment and job search :)
Logged

Starshadow

  • Member
  • *
  • Karma: 1
  • Offline Offline
  • Posts: 42
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #8 on: 12 May 2009, 02:12 »

I've been doing a little research into this. The LZMA SDK seems to be focused on providing an implementation of the LZMA codec, not necessarily 7z archives. 7z archive capability looks like it was added as an afterthought a few versions ago. Calling functions from 7z.dll seems to be the better way to work with 7z archives. I've got little Windows development experience, so I can't determine how much work changing to this architecture would take, but I think it's something to look into.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #9 on: 12 May 2009, 06:00 »

Find me a 32 and 64 bit 7z.dll and a good documentation of the function calls then ;)
This is not a request but more a call for help since I already tried to find anything useful on that for years and in the past it was a louse com based mess. Compiling the dll out of the SDK also fails easily since some source files are simply missing or were not updated to their correct name.....as mentioned before....the SKD s***s
« Last Edit: 12 May 2009, 06:08 by Roman »
Logged

Starshadow

  • Member
  • *
  • Karma: 1
  • Offline Offline
  • Posts: 42
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #10 on: 12 May 2009, 17:30 »

You can pull 7z.dlls directly from a 7-Zip install. I tried to post mine here, but they are too big. As far as documentation for using them though, he points to the example in "CPP\7zip\UI\Client7z" in the 7-Zip source code, and 7-Zip application source itself. Other applications that implement 7z seem to do it this way, or are using a custom 7z.dll that exports more functions. If you are able to get 7z.dll integrated, you should be able to make clrmamepro read any archive that 7-zip supports, as well as write 7-Zip archives without calling 7z.exe. If you wanted to go that far, that is :). Speaking of 7z.exe, would it be possible to call that for the testing and extracting instead of doing it internally and forget this whole problem?
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #11 on: 12 May 2009, 18:12 »

you didn't see what I tried to point out.....find me a 64bit dll ;)
Logged

Starshadow

  • Member
  • *
  • Karma: 1
  • Offline Offline
  • Posts: 42
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #12 on: 12 May 2009, 18:29 »

The 7z.dll in a x64 7-Zip install should be 64bit. I'll email mine to you just in case.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #13 on: 12 May 2009, 18:36 »

The 7z.dll in a x64 7-Zip install should be 64bit. I'll email mine to you just in case.


oopsi..yeah...forgot that they provide 64bit releases lately
« Last Edit: 12 May 2009, 18:44 by Roman »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #14 on: 12 May 2009, 19:49 »

well...I had a look at the sample file and actually compiling some parts of it into cmpro is possible...however it's a pain in the ass, i.e. it takes too much time to adopt my current functions to use the dll functions. Look at the com and callback hell in there and it includes half of the SDK files....*sigh*...on the other hand look at the easy structured unrar.dll.

So for now: I don't waste time with 7z. ;D
Logged

amarok

  • Karma: 0
  • Offline Offline
  • Posts: 6
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #15 on: 12 May 2009, 20:05 »

Thanks for having a look at it anyway :) I guess I'll rar my files for now (at least the ones ≥4 GB) until there's a better solution.
Logged

Starshadow

  • Member
  • *
  • Karma: 1
  • Offline Offline
  • Posts: 42
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #16 on: 12 May 2009, 23:07 »

So for now: I don't waste time with 7z. ;D

Can't blame you. I had no idea the 7-Zip code was so convoluted. I did find one other possibility though, to look at in your free time.
http://akky.cjb.net/download/7-zip32.html
Outdated, but English documentation
It seems to be a 7zip dll with an actual API :D. The source is included so you should be able to compile a 64 bit version.
Logged

amarok

  • Karma: 0
  • Offline Offline
  • Posts: 6
  • Operating System:
  • Windows Vista Windows Vista
  • Browser:
  • Firefox 3.0.10 Firefox 3.0.10
    • View Profile
Re: Problems with files ≥4 GB
« Reply #17 on: 22 May 2009, 10:21 »

Thank you so much for 3.126b :D Even though it might not be an ultimate solution for the 7z problem it makes my files get scanned (and renamed) correctly at last :) Great work, Roman :D
Logged
Pages: [1]   Go Up
 

Page created in 0.178 seconds with 19 queries.