Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

1

clrmame Discussion / Re: BUG: 7z and dat changes: loss of files even with "backup" turned on

« on: 14 October 2013, 06:05 »

Ahh. Yes it was x64 version of 7z "7-Zip 9.20 (x64 edition)", but now I am on "7-Zip 9.30 (x64 edition) alpha" to see how it goes. It is also the 64bit version of cmpro (exe named cmpro64.exe). But my RAM is very low on that system (3GB); hopefully thats not causing the problem.

As you say it might be because cmpro uses zip internally to backup files, and, thats what is having problems with the large iso files. I do notice sometimes cmpro backups valid files (for example when I update mame) and I usually just drag-n-drop the backup folder back on the scan window afterwards to rebuild them; so that could be happening here, but, it errors out with zipping during backup?

"Decompressed files" isn't really an option for me, and I don't have the "Decompress Rom & check crc32" option checked, maybe one of the other options causes the same behaviour though. I'll look into the new "7zip rn" option (i seen it on the cmpro compress window, but, new features sometimes scare me with cmpro hehe). If I run into the problem again on 7.30 then i'll try turning on the "7z rn" option.

2

clrmame Discussion / BUG: 7z and dat changes: loss of files even with "backup" turned on

« on: 13 October 2013, 22:31 »

This has been a problem for a while but thought I'd report. Information.

Old Set: My File - Great.7z
New Set: My File (Great).7z
clrmamepro: Version 4.011a (same with older ones though)
7zip version: 9.20

"My File - Great.7z" contains a single "My File - Great.iso"

After updating any of my sets dats, filename changes (maybe ~25%) become fatal to me, and cmpro permanently deletes my files. It has come to the point where I usually have to say "no to all", manually backup any files, then let cmrpo corrupt most renames, and then drag-n-drop the manually backed up files for rebuild; as you can guess tedious. This is what I usually observe:

1. During first scan it says "rename request: yes, yes to all, no, no to all". Say yes.
2. Warning log says: "7z: ERROR SZ_ERROR_MEM -> E:\set\My File - Great.7z\My File - Great.iso".
3. cmpro creates a 22byte "backup" which contains no files, and not a valid zip: "My File - Great.zip".
4. cmpro deletes "E:\set\My File - Great.7z" (which at this point is a valid .7z).
5. New scan.
6. Warning log says: "Corrupt Archive File: E:\cmp\backup\set-name\My File (Great).zip | Reason: NO ENTRIES".


Note: I recently updated to 9.30 alpha in hopes it isn't a cmpro problem, but, I think it is, and is probably related to how cmpro handles double quotes as I reported something like this before and cmpro fixed the issues with how it was handling double quotes for 7z, but, ideally these two would be implemented as well even if a cmpro fix is done:

My Suggestions for cmpro: 1: If backup is turned on, validate the backup file before deleting the set version.
My Suggestions for cmpro: 2: Stop progress if an error cmpro cannot handle occurs. Something like "Unexpected/Unhandled error with X occured. Proceed or Stop?". This gives the user time to manually look at the state of the system and decide "ok I need to backup that file because cmpro is probably going to delete it now and that backup it created is corrupted data". This should be good even if backup is turned off.

Let me know if you need more information.
EDIT: My cmpro backup folder, and cmpro set are on different drives if this information helps.
EDIT:
cmpro 7z: Executable: C:\ProgramFiles\7-Zip\7z.exe
cmpro 7z: Compress: a -y -r -ms=off -mx0 %1 %2
cmpro 7z: Delete: d -y -ms=off -mx0 %1 %2
cmpro 7z: Rename (unchecked)

3

clrmame Discussion / Re: Suggestion: Better Uncompressed Set Support

« on: 10 September 2010, 19:25 »

Sounds good.  If possible though I'd still suggest benchmarking the zlib crc32 func ya using to a faster assembly version.  If what you say is true and the 40sec "new scan" is calculating crc32 multiple times then maybe it is okay *with caching* but if you find out your zlib version is even 25% slower than a custom made crc32, that is 25% faster cmpro .

Zlibs crc32 only advantage is it stability, but, crc32 is simple *easy to make stable*; using a custom fast assembly version *or at least giving people an option to use an assembly version in say the settings window* could bring massive speedups to cmpro, especially to uncompressed sets.


Also you mentioned other options like 'name checks' need to calculate crc32? (I assume calculates crc32 and checks to make sure name matches the entry in the dat with that crc32).  As an additional suggestion, the other checks * theoretically* shouldn't need crc32; aka if the current "set\filename" is not in the dat it failed the name check; should be able to scan dat for "set" and then "name" as easily as it scans for "crc".  Unneeded file (with checksum button unchecked) would simply check "filesize"&"set\name" (items always instantly available) to determine if it passed or not.  I aggree it wont be as accurate, and cmpro may move valid items to the backup folder that were say only renamed (etc), but, if someone manually unchecks "checksums" on the scan window they are already assuming no hashs are being calculated/used during the scan.


Being able to do a new scan and still check/fix all the scan options (without calculating a single hash) would be very ideal for keeping uncompressed sets up-to-date as crc32 isn't readily available like in compressed sets.

4

clrmame Discussion / Suggestion: Better Uncompressed Set Support

« on: 09 September 2010, 23:18 »

I find uncompressed sets pretty tedious to work with in cmp.  Can I make the follow suggestions.

1. Optimize the CRC32 calculation.
   Rational: If I have a single 4GB file set and want to just verify with a "New Scan" (SHA/MD5 is turned off) then it takes about 40 seconds for cmp to finish the "New Scan".  If I calculate the CRC with the following public program: http://www.codeproject.com/KB/recipes/crc32.aspx  (using dynamic assembly) the crc32 generation of the 4GB file takes 15 seconds.  cmpro seems to take between 2x-3x the time.
   Possible Bug: Even if I uncheck the "Checksums" checkbox on the scan window, the "New Scan" button seems to still check checksums.  Is this intended?

2. If possible use a "Move" operation when rebuilding for uncompressed sets instead of "Copy".
   Rational: When rebuilding the set mentioned above it takes ~40 seconds to crc32 then about 60 seconds to complete the copy operation.  This is extrmely long.  If possible you should be able to check if this is "a) uncompressed, & b) This rom isn't used at another place in the set, & c) Removed matched source files is checked" then it should be safe to issue a move operation instead of a copy, else, do what it normally does.


With just the above two suggestions rebuilding uncompressed sets could be 4x-5x faster, and "new scan" uncomperssed sets should be at least 2x faster.  Also users should be able to "new scan" a folder with the "Checksums" checkbox unchecked and still be able to check for incorrect sizes/names/etc (if the bug above is intended)?

3. Port the "No SetFolder for decompressed sets" from the rebuilder window to the scan window.
   Rational: If cmpro supports building this file-structure, cmpro should support scanning this file structure as well.

However, with "1" and "2" both implemented "3" may not be as tedious to do manually with a full rebuild whenever you want to clean it.  But being able to scan this structure would be the most ideal solution.

Thanks.  Any other improvments you can think of for uncompressed sets would be welcome too as some of us do use it