EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: Roman please look at this rebuilder issue  (Read 3450 times)

B2K24

  • Member
  • *
  • Karma: 2
  • Offline Offline
  • Posts: 29
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Firefox 42.0 Firefox 42.0
    • View Profile
Roman please look at this rebuilder issue
« on: 30 November 2015, 16:40 »

Roman, It has been reported when using the rebuilder with 7zip sets (Hash collision all clones in relationship)  - (Full Merge Mode - Hash collision name)

 I see while rebuilding is taking place It constantly packs and unpacks the 7zip file for every single change that it makes. In other words It opens (unzips) and closes (compresses) the 7z file for each change to each containing set instead of opening it once, make all the changes and then close it.

Is there anyway to make this operation to be more efficient as in open the 7z once and close it when all changes have completed?

Please see this in action for yourself using HBMAME and set kof2001
http://www.1emulation.com/forums/files/go/0111e5f9898b7b9cd04b1485cc3176ad/hbmame

Here is a setup to reproduce the issue. Have destination folder completely empty and you'll see what I mean.

http://i.imgur.com/AbCU0AK.png

http://i.imgur.com/HqWx3nl.png

http://i.imgur.com/iCj2vVy.png

kof2001 7zip file for source path rebuild
https://mega.nz/#!aos0BKgR!xh15byqMficDlQG7mgBx1HaHq-ptUe4d-WPSJtTVrrU

Thank You for any attention you can give to this matter :)
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 113
  • Offline Offline
  • Posts: 3292
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 46.0.2490.86 Chrome 46.0.2490.86
    • View Profile
Re: Roman please look at this rebuilder issue
« Reply #1 on: 30 November 2015, 19:16 »

The rebuilder is file based not archive or set based. This means it takes one file from the source (archive), matches it against the database and if it does match each instance of the file is added (with the correct name) in the destination. So files get added one-by-one to the destination. No big deal if you work with zipfiles.

Solid 7z files are different (and your file IS solid). For solid files this means you end with repacking ALL files in the destination archive...because they are stored solid the 7zip program needs to redo its work over and over again.

Never understood why people use 7z or even solid 7z archives in times where multi-terabyte harddisks come for less than 100 EUR....just to gain some free bytes....

...and I don't start to moan about torrent7z....
« Last Edit: 01 December 2015, 08:34 by Roman »
Logged
Pages: [1]   Go Up
 

Page created in 0.233 seconds with 20 queries.

anything