EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: Smart Size Filter  (Read 5096 times)

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 269
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Firefox 29.0 Firefox 29.0
    • View Profile
    • .
Smart Size Filter
« on: 25 June 2014, 02:02 »

Hi Roman,
I make a wish !
Currently I am using a header skipper plug-in with CMP..

My wish ?
CMP quickly scans through the DAT - determines largest file size, then when rebuilding, ignores all files over 'that' size :)
Explained clear enough ?

Thanks
Oxyandy
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 113
  • Offline Offline
  • Posts: 3292
  • Operating System:
  • Mac OS X Mac OS X
  • Browser:
  • Safari 7.0 Safari 7.0
    • View Profile
Re: Smart Size Filter
« Reply #1 on: 25 June 2014, 19:20 »

actually the rebuilder only checks files which have a size which is available in the currently loaded dat....at least I've added such a check years ....can double check it in 2 weeks....
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 113
  • Offline Offline
  • Posts: 3292
  • Operating System:
  • Mac OS X Mac OS X
  • Browser:
  • Safari 7.0 Safari 7.0
    • View Profile
Re: Smart Size Filter
« Reply #2 on: 25 June 2014, 19:27 »

hmmm....could be that I combibed size and hash calc .... so it would cause useless hash calc time....wonder if I was that dumb....as I said I can check that in 2 weeks earliest....since Im now enjoying sun and beach.....
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 269
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Firefox 29.0 Firefox 29.0
    • View Profile
    • .
Re: Smart Size Filter
« Reply #3 on: 28 June 2014, 03:37 »

 ;D
Hi Roman,
Thanks for taking a look :)

Enjoy your holiday !
"If" I am lucky I will get a holiday sometime in the next month..
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 113
  • Offline Offline
  • Posts: 3292
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 35.0.1916.153 Chrome 35.0.1916.153
    • View Profile
Re: Smart Size Filter
« Reply #4 on: 06 July 2014, 20:09 »

ok, I've checked the code...and there is a "smart size filter" already in....(tested with a MAME dat and some large iso file I had floating around...it gets skipped directly...)

So....what makes you think that bigger files are checked? Or is it an archive inside an archive?
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 269
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Firefox 30.0 Firefox 30.0
    • View Profile
    • .
Re: Smart Size Filter
« Reply #5 on: 07 July 2014, 12:34 »

Hi Roman,
When using a NES DAT with a header skipper,
Rebuilding files which had a couple of movies in the folders & CMP would always 'choke' on them.
Or large archives which I block CMP from 'looking into' by changing the extension :P
Thanks
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 113
  • Offline Offline
  • Posts: 3292
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 35.0.1916.153 Chrome 35.0.1916.153
    • View Profile
Re: Smart Size Filter
« Reply #6 on: 07 July 2014, 13:04 »

ah header skipper....well yes...then it takes long...that disables the check....(there are some other cases...e.g. if you work with dats which only provide hashes not size information...)....I will check if the header check can also be so smart to do some skipping...
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 113
  • Offline Offline
  • Posts: 3292
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 35.0.1916.153 Chrome 35.0.1916.153
    • View Profile
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 269
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Firefox 29.0 Firefox 29.0
    • View Profile
    • .
Re: Smart Size Filter
« Reply #8 on: 08 July 2014, 04:45 »

I placed 2489 "wanted files" with about 15Gb worth of 'other random stuff'  ::) in the Source folders of Rebuilder.
Everything was not already archived (RAW)

Took the new build for a test drive:
7 minutes to complete

As a comparison, same "Source Folders/Files" with Public Release 4.014b
27 minutes to complete

Thanks Roman, that did the trick ;)
Logged
Pages: [1]   Go Up
 

Page created in 0.189 seconds with 20 queries.

anything