EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: 7zip alpha 'rn' command renaming inefficiency  (Read 9746 times)

Starshadow

  • Member
  • *
  • Karma: 1
  • Offline Offline
  • Posts: 42
  • Operating System:
  • Linux Linux
  • Browser:
  • Firefox 18.0 Firefox 18.0
    • View Profile
7zip alpha 'rn' command renaming inefficiency
« on: 20 January 2013, 16:33 »

I very much appreciate all the recent improvements that have been made in regards to cmpro's handling of 7zip archives. It's far and away the best it's ever been. But there's one thing that's bothering me about the use of the 7zip alpha 'rn' command.

If I'm correct, before calling 7z.exe 'rn' to rename a file inside an archive, it looks like cmpro calls 7z.exe 'd' to delete any file that already has the new name. Now, this is as much 7zip's fault as cmpro's, but instead of recognizing when no file exists with the new name to delete and quitting, 7z.exe processes the entire archive anyway. This results in the entire archive being processed once for a usually fruitless delete, and again for the actual rename, making any 7zip file rename operation take exactly 2x longer than it should. When renaming sets of many large archives, this time adds up quickly.

If possible, could you please add a check to see if a delete is actually required before calling 7z.exe 'd'?

Thanks.
« Last Edit: 20 January 2013, 16:38 by Starshadow »
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 24.0.1312.52 Chrome 24.0.1312.52
    • View Profile
Re: 7zip alpha 'rn' command renaming inefficiency
« Reply #1 on: 21 January 2013, 10:33 »

ok I will have a look
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 24.0.1312.52 Chrome 24.0.1312.52
    • View Profile
Re: 7zip alpha 'rn' command renaming inefficiency
« Reply #2 on: 21 January 2013, 21:11 »

should be easy to add a test if the file is actually in the archive (however it needs to read the table of contents of the archive again)...and I guess I can
a) filter out real existing files
b) (since the "remove" operation can handle multiple files at ones) maybe use @filelists options...

I will play around with it....
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 24.0.1312.52 Chrome 24.0.1312.52
    • View Profile
Re: 7zip alpha 'rn' command renaming inefficiency
« Reply #3 on: 21 January 2013, 21:24 »

ok...first little improvement(?)...
remove operation for external packers (rar/7z) test if the file in question really exists in the archive....

http://mamedev.emulab.it/clrmamepro/binaries/cmp20130121.rar

let me know if it really speedup things for you....
Logged

Starshadow

  • Member
  • *
  • Karma: 1
  • Offline Offline
  • Posts: 42
  • Operating System:
  • Linux Linux
  • Browser:
  • Firefox 18.0 Firefox 18.0
    • View Profile
Re: 7zip alpha 'rn' command renaming inefficiency
« Reply #4 on: 21 January 2013, 23:46 »

That did the trick. My archive was only processed once to rename a file inside it. Thanks a lot.

You mentioned removing multiple files at once. If you could rename multiple files at once too, that would eliminate even more extra processing. The 7z 9.30 documentation claims it supports multiple renames on the command line, but it doesn't seem to work for me. I'm not sure about rar. Maybe a thought for the future.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Mac OS X Mac OS X
  • Browser:
  • Safari 6.0 Safari 6.0
    • View Profile
Re: 7zip alpha 'rn' command renaming inefficiency
« Reply #5 on: 22 January 2013, 05:55 »

commandline will be always limited by its length. so it may work for a couple of files but not for lets say 30 or more...unless there is an option for listfiles where a new file holds the filenames. I will check this
Logged

Starshadow

  • Member
  • *
  • Karma: 1
  • Offline Offline
  • Posts: 42
  • Operating System:
  • Linux Linux
  • Browser:
  • Firefox 19.0 Firefox 19.0
    • View Profile
Re: 7zip alpha 'rn' command renaming inefficiency
« Reply #6 on: 11 March 2013, 17:11 »

I think I have encountered a bug that resulted from the changes to the way files are deleted from archives. I have some roms in 7z archives that have been removed from their dat. The scanner detects the files as unneeded and copies them to the backup directory, but then the rom is never actually deleted. These particular roms all have backslashes in their names, which results in the rom file being inside a series of subfolders in the archive. I suspect this might be causing the check for file existence to fail, and the deletion command is not issued. The same roms when stored in the root of the archive without the subfolders are successfully deleted.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 25.0.1364.160 Chrome 25.0.1364.160
    • View Profile
Re: 7zip alpha 'rn' command renaming inefficiency
« Reply #7 on: 11 March 2013, 18:27 »

as always, send some files :) dat/roms/settings/ini
Logged

Starshadow

  • Member
  • *
  • Karma: 1
  • Offline Offline
  • Posts: 42
  • Operating System:
  • Linux Linux
  • Browser:
  • Firefox 19.0 Firefox 19.0
    • View Profile
Re: 7zip alpha 'rn' command renaming inefficiency
« Reply #8 on: 12 March 2013, 16:58 »

Here you are. I created a small test dat and duplicated the bug on a fresh install of clrmamepro 4.010 32-bit with minimal config changes.

File 1.7z contains a good rom in a subfolder with no unneeded roms.
File 2.7z contains a good rom and an unneeded rom not in a subfolder.
File 3.7z contains a good rom and an unneeded rom in a subfolder.

If you run a scan with 'fix unneeded' enabled, you will see the unneeded rom in File 2.7z deleted, but the unneeded rom in File 3.7z not deleted.
« Last Edit: 12 March 2013, 17:04 by Starshadow »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 25.0.1364.172 Chrome 25.0.1364.172
    • View Profile
Logged

Starshadow

  • Member
  • *
  • Karma: 1
  • Offline Offline
  • Posts: 42
  • Operating System:
  • Linux Linux
  • Browser:
  • Firefox 19.0 Firefox 19.0
    • View Profile
Re: 7zip alpha 'rn' command renaming inefficiency
« Reply #10 on: 12 March 2013, 19:34 »

Looks good! Thanks as always. :)
Logged
Pages: [1]   Go Up
 

Page created in 0.114 seconds with 20 queries.

anything