EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: [BUG?] Rebuilder removes junctions in source  (Read 6102 times)

RainyShadow

  • Karma: 0
  • Offline Offline
  • Posts: 6
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Opera 12.12 Opera 12.12
    • View Profile
[BUG?] Rebuilder removes junctions in source
« on: 27 January 2013, 06:18 »

I have several HDDs in my system, so i make heavy use of junction points to organize my files.
Also, i usually dump various roms in my backup (junk) folder and use it as source for rebuilding.
What i noticed is that, when i create a junction to a roms folder  in my backup folder, it gets deleted after a rebuild as if it was an empty folder.
I think the proper handling would be to delete the junction only when its target folder is really empty.

CMP v4.09a
WinXP SP3
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 24.0.1312.56 Chrome 24.0.1312.56
    • View Profile
Re: [BUG?] Rebuilder removes junctions in source
« Reply #1 on: 27 January 2013, 10:30 »

Junctions???? What do you mean exactly?
The rebuilder reads files (or files within an archive) from a given source, checks the hash values against the database and creates any found match of it in the destination and uses the correct naming. If you got "remove rebuilt files" enabled, it deletes the file from the source (archive) and if the archive or folder is empty after this it gets removed, too (however there is a rebuilder advanced option which allows you to remove empty folders or not).
Logged

RainyShadow

  • Karma: 0
  • Offline Offline
  • Posts: 6
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Opera 12.12 Opera 12.12
    • View Profile
Re: [BUG?] Rebuilder removes junctions in source
« Reply #2 on: 27 January 2013, 16:48 »

Junctions???? What do you mean exactly?
Junctions are the NTFS equivalent of symbolic links. With these you can have i.e. a folder containing all roms/movies/etc. from all your hard drives in a single place; or you can replace a non-movable folder (Quicktime cache, browser cache, etc.) with a junction to a folder on another drive... You can find some information on http://en.wikipedia.org/wiki/NTFS_junction_point
I use KillCopy to create them as WinXP comes with only a command line utility (fsutil.exe) to manipulate such links. There are some other alternative programs available as well - NTFSLink, Junction Link Magic and others.
The rebuilder reads files (or files within an archive) from a given source, checks the hash values against the database and creates any found match of it in the destination and uses the correct naming. If you got "remove rebuilt files" enabled, it deletes the file from the source (archive) and if the archive or folder is empty after this it gets removed, too (however there is a rebuilder advanced option which allows you to remove empty folders or not).
I am aware of the "Remove Matched Sourcefiles" and "Remove empty sourcefolders" options, and i use them when appropriate. The issue surfaces if there are junctions in the source - they get removed regardless if they point to an empty folder or to a folder full of roms. So, if i go through several CMP profiles and rebuild from that source, only the first one would receive the matched roms from the junction folder.
I noticed something which may be related - the properties of a folder containing a junction don't count the junction target in size calculation. Here is an example folder structure:
C:\ROMs
C:\ROMS\FBA_roms   < a junction to D:\FBA\ROMs
Now, if you right-click C:\ROMs and select "Properties", it would show 0 in the "Size" field (Windows Explorer bug?).
If you do the same for C:\ROMS\FBA_roms it would properly show the size of the D:\FBA\ROMs folder.
Maybe Clrmamepro is affected by the same bug and considers the junction as an empty folder, so it is removed when the above two options are enabled.

HTH
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 24.0.1312.56 Chrome 24.0.1312.56
    • View Profile
Re: [BUG?] Rebuilder removes junctions in source
« Reply #3 on: 27 January 2013, 18:32 »

Cmpro simply calls Windows api functions to remove single files / folders... If there are junctions affected, the OS should handle it correctly. I currently don't see why cmpro needs to care about it.
« Last Edit: 27 January 2013, 18:33 by Roman »
Logged

Zandro

  • Member
  • *
  • Karma: 1
  • Offline Offline
  • Posts: 40
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Firefox 18.0 Firefox 18.0
    • View Profile
Re: [BUG?] Rebuilder removes junctions in source
« Reply #4 on: 28 January 2013, 00:22 »

I had the same issue when setting up a backup system using rsync.  The linked folders were being removed from my custom root and full copies filled up in its place.  I did not find a solution for this, sad to say.

Reading this NGP to stir memory, use of an --exclude argument is used to skip a tree, which got me thinking, is there a no-delete list available somewhere in cmpro a directory can be added to??
« Last Edit: 28 January 2013, 00:31 by Zandro »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 24.0.1312.56 Chrome 24.0.1312.56
    • View Profile
Re: [BUG?] Rebuilder removes junctions in source
« Reply #5 on: 28 January 2013, 08:22 »

well, as I said....cmpro does pretty basic file / folder operations by calling Windows API functions. A file removal or a folder removal. It does not know about existing junctions. If a folder is removed from the source which is in fact a junction, the OS should actually complain when calling the removedirectory function. I can check if that function got special instructions to let it fail for junctions (so they are not removed) when I find some little time....pretty busy with real life/job at the moment.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 24.0.1312.56 Chrome 24.0.1312.56
    • View Profile
Re: [BUG?] Rebuilder removes junctions in source
« Reply #6 on: 28 January 2013, 18:56 »

From the windows api doc:

RemoveDirectory removes a directory junction, even if the contents of the target are not empty; the function removes directory junctions regardless of the state of the target object.

Guess I have to explicity test if the folder is empty before using RemoveDirectory then...

you might want to test this:
http://mamedev.emulab.it/clrmamepro/binaries/cmp20130128.rar
« Last Edit: 28 January 2013, 19:28 by Roman »
Logged

RainyShadow

  • Karma: 0
  • Offline Offline
  • Posts: 6
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Opera 12.13 Opera 12.13
    • View Profile
Re: [BUG?] Rebuilder removes junctions in source
« Reply #7 on: 28 January 2013, 20:50 »

This build handles junctions fine according to my quick test.
I created two junctions - one pointing to an empty folder and one pointing to a non-empty folder. Then i used their parent as a rebuilder source and tried all combinations of the above mentioned two options.
The result - only the junction pointing to the empty folder was deleted, and it required both options enabled. The other junction remained after all three rebuilds.
Success!

Thank You!

P.S. i could test only the 32bit binary as my system doesn't support x64.
« Last Edit: 28 January 2013, 20:51 by RainyShadow »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 24.0.1312.56 Chrome 24.0.1312.56
    • View Profile
Re: [BUG?] Rebuilder removes junctions in source
« Reply #8 on: 28 January 2013, 20:54 »

hurray...
Logged
Pages: [1]   Go Up
 

Page created in 0.124 seconds with 20 queries.

anything