EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: suggestion for an optimization for the rebuilder  (Read 7318 times)

ReadOnly

  • Karma: 0
  • Offline Offline
  • Posts: 27
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Firefox 11.0 Firefox 11.0
    • View Profile
suggestion for an optimization for the rebuilder
« on: 11 April 2012, 13:18 »

Currently in merged mode, the rebuilder remake an archive whenever it finds a rom that is included in the set. that's not a problem with common stuff, but if you try rebuilding say a doctor layton 7zipped set which includes about 7 or 8 2-gig roms, then it becomes very nasty as you can imagine. Wouldn't it be smarter to look for everything that can be included in the set, then rebuild the archive.
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 113
  • Offline Offline
  • Posts: 3293
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Chrome 18.0.1025.152 Chrome 18.0.1025.152
    • View Profile
Re: suggestion for an optimization for the rebuilder
« Reply #1 on: 11 April 2012, 13:29 »

The rebuilder is file based, not set based. It scans one file or one file within an archive and creates (adds) all instances of that file in case of a checksum match.
There might be some minor room for some small "look ahead" optimization....but don't expect anything anytime soon.
Logged

ReadOnly

  • Karma: 0
  • Offline Offline
  • Posts: 27
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Firefox 11.0 Firefox 11.0
    • View Profile
Re: suggestion for an optimization for the rebuilder
« Reply #2 on: 11 April 2012, 14:01 »

That's fine with me. Keep up the good work, thanks Roman.
Logged

ReadOnly

  • Karma: 0
  • Offline Offline
  • Posts: 27
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Firefox 18.0 Firefox 18.0
    • View Profile
Re: suggestion for an optimization for the rebuilder
« Reply #3 on: 13 January 2013, 19:47 »

Any progress? With all due respect maybe it's time to entirely rewrite cmpro?

When your rom manager writes 50 gigs of data on your HDD to make a 1 gig archive, you know there is a problem somewhere.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 113
  • Offline Offline
  • Posts: 3293
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 24.0.1312.52 Chrome 24.0.1312.52
    • View Profile
Re: suggestion for an optimization for the rebuilder
« Reply #4 on: 13 January 2013, 21:53 »

The problem is more in the hands of 7z and solid archives and no good sdk out there...
Using zip with no recompression will be way faster ...and in times of cheap multi terabyte hds I really wonder why people still care about some 7z over zip space gain....

but no, no progress since real life takes all my time. I got some nice ideas but I doubt it will be added into the old core at all. And the new core does not exist yet at all :) But the ideas are on my list
Logged
Pages: [1]   Go Up
 

Page created in 0.175 seconds with 20 queries.

anything