EMULAB Forum
clrmamepro [English] => clrmame Discussion => Topic started by: oxyandy on 17 July 2014, 13:34
-
Hi Roman,
I am suddenly in dire need of a logging feature when Rebuilding.
1. The name of the Source archive.
2. The internal file-name. (Which matched the dat)
3. What they were Rebuilt to.
I have never played with CMP logging previously, so does something like this exist already ?
Many thanks
Oxyandy
-
Yep...exists already...
Rebuilder->Advanced->Create logfile
....
creates something like:
C:\Users\Roman\Desktop\circus.zip\circus.2a
-> springbd.zip\93448.2a
-> circus.zip\9005.2a
-
Ok,
Many thanks for the quick response..
Where do I find log ? Guess cmp program folder root.
With each run, is log appended or starts new ?
I guess I'll find out soon enough..
I'll try now
Thanks again.
EDIT: Ok all answered, I give it a run
-
as you might know by now...you can specify a full path to the file...and right beside that input field you got a checkbox for appending/overwriting the file.....
-
O my god! Me continuous learn CMP :)
-
Yes, I saw that in Rebuilder Options as soon as I setup "Logging"
I see is a per 'dat' log & not global, but that's ok..
I just finished writing a program to reformat CMP's log output to the 'exact' format I require.
Has all the info I need, just perfect
Thanks again
Oxy
-
Oxy what is that your program?, give me any private beta for test :)))
-
if you got any requirements for the layout of the file...let me know.....
-
Hi Roman,
Really I am very happy already, no need for you to change anything.
Already making use of logging to delete "Source Files" while keeping a reference list of the matching "File-names"
Output from my 'CMP Log Parser' is like this:
New: "File-name matching dat.ext" Old: "File-name found when Rebuilding.ext"
This lines is kept if (New: & Old:) name differs greatly, otherwise is filtered from the "New_Log.log" I make.
It also gets alpha sorted & merged to a Master.log
Thanks for the kind offer. ;D
-
D:\My Documents\Downloads\小霸王SB-A8 NES解密版\小霸王 SB-A8-NES\WXNDecrypt\三侠五义.wxn [SKIPPED Reason: no hash match]
D:\My Documents\Downloads\小霸王SB-A8 NES解密版\小霸王 SB-A8-NES\WXNDecrypt\三侠五义.wxn.nes
I am not sure why lines like these have no [REASON ?
They no longer exist (oops probably should have done a trial run first)
They create a quirk in my log, which I can re-write code to handle..
Sadly right now I can not repeat it, but I assume is related to previous file-name ?
EDIT: (previous file-name ?) No, I guess not.
EXAMPLE: A single archive with 521 internal files = 8 files skipped with-out [Reason.
Seems almost random.. I need to get more familiar with what's happening exactly how logging works. :-\
Anyway Roman, no problem, I just have more learning to do..
I need to learn what causes output lines like these.
My program starts with a 10Mb CMP_Log.txt after parsing by my exe, it is split into 3 separate logs
1. 178Kb ROM log
in this format
New: "100m Hurdles (Waixing).nes" Old: "110米栏.wxn.nes"
New: "1943 - The Battle of Midway (U) [!].nes" Old: "1943 (U).nes"
New: "2011 Tank by Lei Fan (Battle City Hack).nes" Old: "2011坦克_leiyin.nes"2. 178Kb Source Archive/Folder log.
(needed cause something internal names are poor, compared to archive name)
3. 9.7Mb Remaining files, which I need to write a filter for (extensions I want to be alerted about)
-
I will have a look why (or when) no reason is shown....
-
I am trying hard to make a simple repeatable scenario, but having trouble with that:
For example I scan the whole D: drive.
After I look in the log.
D:\Folder A\Archive_A.zip is causing 'missing reasons'
So if I drag & drop Archive_A.zip onto Rebuilder, everything is OK.
If I set Source D:\Folder A\ in Rebuilder, again everything is OK.
trying hard here....
-
Hang on, an obvious check, I will disable my AV and try over whole drive again.
That made no difference so it seems, it still looks like it's happening... sigh
-
D:\Desktop\nesroms\不能运行的[vt].zip\高速赛车.wxn.rar [SKIPPED Reason: no hash match]
D:\cmp4014__32\temp\高速赛车.wxn_11449.rar\高速赛车.wxn.nes [EXISTS: Highway Racing I (Ch) (Wxn)\Highway Racing I (Ch) (Wxn).pas]
D:\Desktop\NES_TOOL_TEST_ROMS\New Folder (5)\不能运行的[vt].zip\高速赛车.wxn.rar [SKIPPED Reason: no hash match]
D:\cmp4014__32\temp\高速赛车.wxn_9813.rar\高速赛车.wxn.nes
Hard to repeat, seems random.. will keep searching for "Repeatable Event"
EDIT:不能运行的[vt].zip seems to be quite repeatable so I copied it to several locations.
-
Roman,
The ONLY clue I seem to have is that the [Reason - only seems to be missing from files which actually match the dat.
These should be [EXISTS:
D:\My Documents\Downloads\南晶科技.rar\南晶科技\[NJ057]魔界塔士.zip [SKIPPED Reason: no hash match]
D:\cmp4014__32\temp\南晶科技\[NJ057]魔界塔士_29345.zip\card_cover.jpg [SKIPPED Reason: no hash match]
D:\cmp4014__32\temp\南晶科技\[NJ057]魔界塔士_29345.zip\Mo Jie Ta Shi (C).NES
D:\cmp4014__32\temp\南晶科技\[NJ057]魔界塔士_29345.zip\box_cover.jpg [SKIPPED Reason: no hash match]
-
I got it !
I seem to have a "Repeatable Scenario" now.
I will upload a folder & a dat.
See if you can repeat also..
I will PM link to files - ATM Dat is private. :)
-
well..yes...it happens when a file was already rebuild....so if A is rebuild to a1,a2,a3...and then A comes again it skippes the reason...(it should show EXISTS then....)...fixed...
http://mamedev.emulab.it/clrmamepro/binaries/cmp20140718.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmp20140718.rar)
-
Thanks very much Roman !
Now I can make some very quick use of this.
I have LOTS of files to deal with ;)
I was just going to point out, the exact thing you said..
After a file has been 'match once already' - it happens
D:\Desktop\New Folder (19)\0_2D Escape (Ch) (Wxn).zip\2D Escape (Ch) (Wxn).nes [EXISTS: 2D Escape (Ch) (Wxn)\2D Escape (Ch) (Wxn).nes]
D:\Desktop\New Folder (19)\1_2D Escape (Ch) (Wxn).zip\2D Escape (Ch) (Wxn).nes
D:\Desktop\New Folder (19)\2_2D Escape (Ch) (Wxn).zip\2D Escape (Ch) (Wxn).nes
-
Confirmed Roman,
Working perfectly :D
D:\Desktop\New Folder (19)\0_2D Escape (Ch) (Wxn).zip\2D Escape (Ch) (Wxn).nes [EXISTS: 2D Escape (Ch) (Wxn)\2D Escape (Ch) (Wxn).nes]
D:\Desktop\New Folder (19)\1_2D Escape (Ch) (Wxn).zip\2D Escape (Ch) (Wxn).nes [EXISTS: 2D Escape (Ch) (Wxn)\2D Escape (Ch) (Wxn).nes]
D:\Desktop\New Folder (19)\2_2D Escape (Ch) (Wxn).zip\2D Escape (Ch) (Wxn).nes [EXISTS: 2D Escape (Ch) (Wxn)\2D Escape (Ch) (Wxn).nes]
-
Hi again Roman,
Logging is working great,
I have a problem with "Header Skipper XML's"
<?xml version="1.0"?>
<detector>
<name>No-Intro NES Dat iNES Header Skipper</name>
<author>Yakushi~Kabuto</author>
<version>20070321</version>
<rule start_offset="10">
<data offset="0" value="4E4553"/>
</rule>
</detector>This works great for 'Making a DAT' but fails with scanning because of a few files which have
NESM = 4E 45 53 4D in bytes 16-19
4E 45 53 1A 02 01 01 00 00 00 00 00 00 00 00 00
4E 45 53 4D 1A 01 01 01 80 80 80 80 84 80 53 6FSo I tried using yours:
<?xml version="1.0"?>
<detector>
<name>Roman's NES Header Skipper</name>
<author>Roman Scherzer</author>
<version>1.0</version>
<rule start_offset="10" end_offset="EOF" operation="none">
<data offset="0" value="4E45531A" result="true"/>
<data offset="8" value="0000000000000000" result="true"/>
</rule>
</detector>When using your "Header Skipper" to make a dat, I quickly discovered that 85 files I had were
datted as the 'whole file' = no bytes were skipped in the HASH calculation.
Some had 16 Byte - iNES 2.0 headers and the rest had 'dirty headers' with 'odd' bytes between 8-16
Examples
4E 45 53 1A 04 04 C0 30 00 01 00 00 00 00 00 00
4E 45 53 1A 04 04 80 C0 00 01 00 00 00 00 00 00
4E 45 53 1A 20 40 52 00 04 00 00 00 00 00 00 00
Anyway, I guess the easiest fix is to add the 1A
<?xml version="1.0"?>
<detector>
<name>No-Intro NES Dat iNES Header Skipper</name>
<rule start_offset="10">
<data offset="0" value="4E45531A"/>
</rule>
</detector>
This is new to me - so tell me is this line in your rule expecting bytes 8-16 to all be Zero ?
<data offset="8" value="0000000000000000" result="true"/>
So if a header is polluted with DiskDude! your skipper will 'not ignore' the header when hashing ?
Oxy
-
The nes header file is like 8 years old...erm...erm...all I remember is that a clean NES header is 16 bytes long starts with 4E45531A don't ask me where the zero bytes come from...could be based on some old discussion with Cowering....as I said...8 years....
if you google the iNes header format you find:
0-3: Constant $4E $45 $53 $1A ("NES" followed by MS-DOS end-of-file)
4: Size of PRG ROM in 16 KB units
5: Size of CHR ROM in 8 KB units (Value 0 means the board uses CHR RAM)
6: Flags 6
7: Flags 7
8: Size of PRG RAM in 8 KB units (Value 0 infers 8 KB for compatibility; see PRG RAM circuit)
9: Flags 9
10: Flags 10 (unofficial)
11-15: Zero filled
so actually only byte 11 to 15 should be tested against 0
-
Hi Roman,
Yes thanks, I had a play I understand it much better now.
This rule works fine for creating a dat (Source files headered) & scanning a set stored (without any headers)
<?xml version="1.0"?>
<detector>
<name>iNES Header Skipper</name>
<rule start_offset="10">
<data offset="0" value="4E45531A"/>
</rule>
</detector>I found for my needs this works perfectly :)
As you pointed out iNES 2.0 Headers do use bytes in range 08 to 0C
And because some header polluted files do exist (such as DiskDude!) which uses bytes all the way to 0F
Then this check can never exist.
<data offset="8" value="0000000000000000" result="true"/>
If source files being datted contained polluted headers (such as DiskDude!) then the resulting dat would be a mix of files some with 'whole file' hashed and the rest as they should be (hash = whole file minus first 16 bytes)
I get it now, sorry to disturb you :)
Thanks for the response
-
feel free to provide a new better xml.....
-
feel free to provide a new better xml.....
Hmm, well in all honesty I do see people get confused (often) with the
No-Intro NES dat and it's usage with CMP.
However, as I have seen the No-Intro provided Header Skipper is flawed.
So if CMP was to come with a NES header skipper XML
Then it would have the name.
no-intro_NES.xml
<?xml version="1.0"?>
<detector>
<name>iNES Header Skipper</name>
<rule start_offset="10">
<data offset="0" value="4E45531A"/>
</rule>
</detector>
And you would dump the one you currently provide ;)
EDIT: From specs you provided above
"9: Flags 9"
I confirmed Nestopia uses "09" bits 01 as PAL & 00 as NTSC
-
Hi 'again' Roman,
(For anyone reading this should be done with CMP closed)
In CMP_Program_Folder\settings\
1. I opened all settings_files with Notepad++
Batch replaced
Rebuilder_CreateLogFile = off
with
Rebuilder_CreateLogFile = on
Rebuilder_AppendLog = off
with
Rebuilder_AppendLog = on
And
Rebuilder_Logfile = \n
Rebuilder_Logfile = Rebuilder_Logfile.log\n
(Logging in CMP_Program_Folder\ is just fine for me, but I really don't want all DATs using same log file.)
= quick & dirty :)
SO, what I ask is there some kind of string I could use so logs are created per Dat_Name
as in Rebuilder_Logfile = %Dat_Name%.log ??
I know this is a quick way to enable 'Global Logging' - is there some other method ?
2. I messed up - I either forgot, had 2 instances of CMP open - when I closed (one or the other) it over-wrote the settings etc.
But anyway I did a Rebuilder run, thinking logging was enabled and well, it wasn't, oops, hehe.
So is it possible default "Global Logging" logs written by "Dat Name.log" ?
Thanks
Oxy
EDIT or maybe even
as in Rebuilder_Logfile = %timestamp%_%Dat_Name%.log
or (Specifically to deal with 2 instances of CMP being open simulataeously using same DAT)
Rebuilder_Logfile = %sequential_numbering%_%Dat_Name%.log
001_Dat_name.log
002_Dat_name.log
So if exist (002_Dat_name.log)
log_name = 003_Dat_name.log
would do fine too;)
-
hmm...global logging...hmmm....I will think about it
-
Memory refresh: http://forums.no-intro.org/viewtopic.php?f=2&t=1560&start=0#p5778 (http://forums.no-intro.org/viewtopic.php?f=2&t=1560&start=0#p5778) ;)
-
hmm...global logging...hmmm....I will think about it
Thanks Roman,
Have been away on holidays myself ;)
-
thinking about a good way for it. not yet sure if it it should be a batcher option or just some variable logfile naming based on the dat name or so.
....or do you want one big file for all.... nah...
ideas are welcome
-
logfile naming based on the dat name
Would be great.
However, if I have 2 instances of CMP open using the same dat file, then using same log_file is a problem.
So sequential numbering ?
001_Dat_name.log
So if exist (001_Dat_name.log)
log_name = 002_Dat_name.log
etc
I was thinking %Date_Time_Stamp%_Dat_Name.log
However - really not needed - and they often don't sort well. So nah.
Yes as a Batch option (like "Always Compress Files" works now) would work, too
However, I see that as just a distraction for the 'Regular User'
So nah, again.
So I conclude a line in cmpro.ini - which would need to be manually edited.
Pro_GlobalLogging = OnLogging by %Dat_File_name%
Then when I load a 'New Dat'....
No need to remember to enable logging (or set a log file name)
And ... if we can do what is currently not possible, allow for 2 instances by sequential number.
I know this would also mean a new log would be made for each time that DAT is loaded and that may very well be useful too :)
001_Dat_File_Name.log
002_Dat... etc
-
ah..multiple instances of cmpro...users shouldn't use that ;-)
well....I will do some more thinking...guess I have time next week for this...
the current wip whatsnew list is:
misc: corrected handling of sets with only bios roms and sample clone (MAME 154 gp110 sets)
misc: dir2dat not always writes cr/lf as line delimiter (deprecated format only)
misc: optimized general hash calculation / file read routine
misc: removed crc=-1 from suspicious checksum check
misc: rebuilder log show EXISTS entry in case of an already rebuilt file
misc: changed nes header file to be a bit less strict
misc: updated to latest ziparchive class lib, unrar dll
....and I wanted to do some work on
better uncompressed sets support (some hash caching)
h/r/s attribute warnings saved in log
..and you log thingie....
-
misc: optimized general hash calculation / file read routine
Nice tweak, thanks
Speaking of 'uncompressed sets' CMP has a bug..
I have known for a long time..
When Rebuilding from Drive A: to Drive B: & the destination file already exists. [uncompressed]
CMP throws an error.
Want to Stop ?
Error while rebuilding:
A:\Some_File.ext
to
B:\Some_File.ext
Do you want to stop?
It's like CMP does not know the file is already present at the Destination.
Understand ?
I could make an example Dat & Files (If needed)
;D
Edit: Drive A: to Drive B:
Drive letters just example of course..
-
Speaking of 'bugs..'
I have also had CMP crash (quite a bit), while "Redrawing last Scan Result"
I will keep on eye on this, it hasn't been happening with previous builds, just one of the most recent ones.
It has only been with the one dat which I used for the recent MAME update (I think)
BUT, I haven't used the newest build much as I have been away on holiday.
So, again I will persevere and let you know.
cmpro_32.exe [crc32 4CF91535] from cmp20140718.rar
(http://ndsxdelta.no-ip.org/Capture_07312014_200038.png)
:-X
-
Ok Roman,
Great, I found a scenario where this crash is 100% repeatable.
If I open CMP. Load the DAT and go to Scanner.
CMP loads fine, without a crash. "Redrawing last Scan Result" works. (I have a single unneeded file)
Now I open CMP load the DAT, go to Rebuilder, when I use the "Jump to Scanner" button
(Bottom Right Hand Corner of Rebuilder Window)
It crashes every single time while "Redrawing last Scan Result"
Correction: (Nearly every single time)
I guess I should try reduce down the CMP Program Folder, and send you the whole thing 'as is'
EDIT: Ok done that, CMP program folder reduced,
crash still occurs while switching between "Scanner + Rebuilder"
Going to Boot over to Win7 X64 and try same CMP folder there.
EDIT_2:It did not happen there :'(
I will just ignore it, see how things work out on next 'Official Release' - doesn't stop me using CMP.
-
hmm...don't happen here (win7 64bit)...guess I have to run a virtual xp machine....
...no luck in 32bit and virutal xp either....
-
Roman,
Please ignore Reply #32 & #33 - don't waste any time on my XP *only* crash (for the moment)
Maybe it will go away with next build ? Maybe not.. Is not a big issue to me.
(Plus I have a CMP program folder, which *repeats* the crash - I can upload this)
Reply #31 does mention a bug with Uncompressed sets though..
Time would be better spent looking into that, and your existing list ;)
Thanks anyway.
-
nah a crash comes first :-)
most likely a not initialized variable or something. something which randomly messes up the memory. I will have a look
-
nah a crash comes first :-)
OK then, in another attempt to make your life just that 'little bit' easier..
Toggle between Rebuilder <> Scanner with this dat loaded... (Sometimes it takes upwards to 5 toggles Rebuilder <> Scanner <> Rebuilder)
http://speedy.sh/GYxzM/CMP-CRASH-TEST.7z (http://speedy.sh/GYxzM/CMP-CRASH-TEST.7z)
You may need to manually edit the data before you open CMP so 'ROM path exists'.
Currently it expects Q:\MAME - ROMs (v0.154)\
Sure an empty folder will work..
While Re-drawing Last Scanner Result <CRASH>
-
hmmm..no luck yet....switched like 40 times....
guess I will look at the source instead ;-)
-
...and regarding your other remark about unpacked sets...can't repeat that either...
Simply used a decompressed MAME set (have the same decompressed on in source and destination rebuilder folder)...ran rebuild....and all files were skipped...no prompt...everything was fine....
-
OK, here we go..
Sorry, I really don't work with Uncompressed files often.
<SNIP>
Now this is confirmed: Both folders can be on same drive
This archive contains 2 Folders - 1 called SOURCE - 1 called DESTINATION & a dat.
So extract them both to "Somewhere"
Load the DAT supplied - Jump to Rebuilder
Set Source to Source folder - Destination to Destination folder
Now tick 'removed matched files' & Click Rebuild
Yes, I think you are going to say this relates to the file type..
I checked other file types, it doesn't happen ;)
-
ok I can repeat it with your scenario but only if the remove option is ticked...
will check later at home, too
-
ok...that 'issue' is fixed
http://mamedev.emulab.it/clrmamepro/binaries/cmp20140804.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmp20140804.rar)
-
Great, now I checked that 'other' issue :P
At this point in time, with toggling [Rebuilder <> Scanner] using
(http://ndsxdelta.no-ip.org/butt.png)
The magic number is 3 toggles ::)
(http://ndsxdelta.no-ip.org/number_3.png)
-
I had like a hundred toggles here and no issue :-)
-
Yeah, yeah, I know.. I am blessed..
What I will do - is install another virgin XP SP3 32 and take folder over there.
I know I couldn't repeat it on win 7 x64 Ultimate.. No matter how many times I tried, so that's a good thing I guess
-
nah I bet it's a not initialized structure or variable...and depending on the OS it may be zero filled or not...
hard to spot though....
ah...I can repeat it on my virtual machine... ;-)
-
found and fixed...finally....short version: uninitialized HTREEITEM which was used for expanding a tree item.....under some other rare circumstances...
http://mamedev.emulab.it/clrmamepro/binaries/cmp20140805.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmp20140805.rar)
-
AH Roman you are the best.. !
Great work, as usual..
Is this something from a recent change ?
Something that maybe has been there a long time ?
I replaced EXE in my CMP_CRASH_TEST folder..
Toggled maybe 100 times, of course, no crash !
-
must be in for years....decades...ages...
it does not harm Win7 though....and it only happens if you got a "not fixed" item on root level (like your unneeded file thing)
-
ok...that 'issue' is fixed
http://mamedev.emulab.it/clrmamepro/binaries/cmp20140804.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmp20140804.rar)
I usually confirm your fixes as soon as I download the new build, this time I didn't, sorry..
We have a problem..
I just tried a Rebuild to test this fix.
Now here I am Rebuilding from Back-Up folder.
Problem 1. Targets already exist, so should not be Rebuilt at all.(7 Matched | 7 Created)
Problem 2. After the File from Back-Up folder is Rebuilt (Wrongly), Source file is not being Removed..
(http://ndsxdelta.no-ip.org/rebuild_uh_uh.png)
:o
Note: This was tested on the Crash_Fix build (Redrawing Scanner) that came after 'this' fix.
Not tested on previous build.
LATE EDIT:
To reinforce what's happening, after the Rebuild, every matched files claims to have been Rebuilt, no source file was removed.
So if I click Rebuild again, I see an exact repeat, 7 Matches 7 Rebuilt, over & over.
I could probably make you a dat & ROMs to replicate (if you need just holler)
-
well...yes...you got me...I cheated....when not building to an archive, it overwrites the existing file (makes a backup though optionally)...so it does rebuilt them....It's on my list to change that...
regarding 2)..hmm...have to check that....
-
:D
Really ?
No problem, on "To Do List" is good enough for me, I live with CMP and it's numerous bugs.
Still the best ROM Manager in the world :)
-
do you have that 7z file again where we tested the rebuilder issue when unpacked sets are rebuilt? Can you upload it again please?
-
@5.00AM my local time !
Sure why not
http://speedy.sh/VUaqW/DESTINATION.7z (http://speedy.sh/VUaqW/DESTINATION.7z)
-
Problem:
C:\SOURCE
C:\DESTINATION\
Yes all files are REMOVED from C:\SOURCE with 'Remove Matched Ticked', but they all end up in the BackUp folder
Try Rebuild from BackUp folder = 'Post #50 Above'
Got it ?
-
On the other thread:
You said:
when I resolved your rebuilder thingie....
maybe...not much time this week....will see....
However using this build posted on this thread (your cheat fix) I am having a couple of problems Rebuilding
I am looking for an updated EXE I can drop into my current CMP folder..
See if the issue is gone.
If so, I will leave you alone to quietly apply these 'other' updates/fixes to CMP..
Otherwise, if problem persists I will supply CMP folder & DAT to replicate :)
Sorry
-
I said cheat fix build, but I meant the one after (Crash fix toggling Scan <> Rebuild) cmp20140805.rar
-
well...let me know which rebuilder issues you got (with current official cmpro) and I put them on my list)
-
Yeah OK, I could roll back to 'Official' build and see if is gone.
This is what I see "Now"
1. Using Scanner: (I get asked to fix: Looks like it's working: But another Scan, shows those files not fixed)
(http://ndsxdelta.no-ip.org/scan_f.png)
2. Using Rebuilder (Same Msg. for every file)
(http://ndsxdelta.no-ip.org/rebuid_f.png)
Now using what I think might be official (EXE dated 17th April 2014)
And to my surprise it is still exact same on 3 Sets (yeah archives seem fine)
Scanning archive is 7z
Rebuilding source is ZIP with my missings, destination 7z
-
As you can probably guess, right at this moment, I am attempting to make a Reduced CMP Folder..
Dat & ROM set to reproduce bug.
That way I can send you whole thing..
You extract Set folder:
C:\Bug\Set
C:\Bug\Fix_Files
Open CMP and you shall see, I'll need time to get it just right..
-
Ok, I take a huge step backwards
:o
This is something I have never seen before.
Is only 4 7z archives it is happening with.
Archive 1: 3super8.7z
Test with WinRAR 5.10
(http://ndsxdelta.no-ip.org/what_the.png)
Archive extracts fine with WinRAR 5.10 too.
I even checked CRC32 of extracted files, yes perfect..
(I did this with all 4 7z archives = same results)
BUT, when I open (any of the 4 7z archives) with 7z File Manager 9.28a
I get
(http://ndsxdelta.no-ip.org/7z_gone_mad.png)
I sent this archive to torrent7zip and it extracted & rebuilt just fine.
Crazy huh ?
Same with other 'problem' sets...
So here's a sample for you.
What makes this 'type' of 7z so unique ?
Tell me what you find ?
<Attachment Snip>
-
So anyway, I 'fixed' my set.. Ignore my stupid bug report :-[
I simply removed these '4 unique' 7z archives from my set...
Played with them a bit:
All 4 test fine with WinRAR.
All 4 extract fine with WinRAR.
All extracted files proved to match original CRCs (no corruption upon extraction)
All 4 can not be opened by 7z File Manager 9.28a, only version I have tried, wont be using others.
Latest versions of 7z File Manager complain t7z files have data past end of file.
(All same error - "Can not open as archive")
All 4 can be extracted and converted to t7z format, by t7z.exe
CMP can open & Rebuild all 4 as Source files. (No files are skipped)
However, CMP can not add to them if they exist in Rebuilder's Destination Path.
Quirky stuff ;D
EDIT: Oh and these 4 quirky 7z archives must have been made through CMP:
I don't have the patience to manually go adding files/archives to a set.
That's what a ROM Manager and DAT is for.
-
hmm...well...as you know cmpro calls 7z exe to add files to an archive....so...it was your 7z which created the files... ;-) maybe you used some buggy alpha in between....
...I had no problems with your attachment (7z 9.22)
-
...I had no problems with your attachment (7z 9.22)
Yeah, Oddi said also no problem opening this sample 7z with (7z 9.34a) either.
I'll remove it now ;)
So I checked the CMD 32bit version of 7z.exe 9.34, same exact behaviour as 7.28a 7z CMD line exe..
(Wont add to these 4 7z archives either)
Restored CMP back to 9.28a 7z cmd exe, cause some of us have seen quirks with newer betas.
I give up, I am just happy there is nothing for you to actually do here.
I know you 'have enough on your plate' for the moment with CMP, I really didn't want to add to it.
haha
-
so actually we only have your unpacked rebuilder thingie left over....
-
ah...I found out why your source / destination setup is reuilt again and again and again, even if the files already exist...I did not cheat ;-)
It's because you use chd files as roms....and there is one place in the rebuilder code where this makes a difference and so your files are not checked for existance in the destination.....and so they get rebuilt....again...and again...
hmm...will see if I can change it....
http://mamedev.emulab.it/clrmamepro/binaries/cmp20140814.rar (http://mamedev.emulab.it/clrmamepro/binaries/cmp20140814.rar)
(be warned...this build also includes the changes regarding possible full merge conflicts and use of subfolders)
-
(be warned...this build also includes the changes regarding possible full merge conflicts and use of subfolders)
Great, I'll try the sub-folder additions out right away
Then in the morning, I'll see if I can clean up miss matched CHD versions using a dat of current 'actual' files.
Knowing this time, if I have any duplicates they should be deleted (rather than backed up)
Thanks
Any news on the Model 110 thingy ?
-
Ok, I'll try the CHD thingy...
Cause I didn't get far, sub-folder test
(http://ndsxdelta.no-ip.org/goodnight-roman.png)
CHD rebuild, works very fast & perfectly... nice fix
EDIT: I first got message about 18w clone and rename with sub-folder, saw file was going to be moved to parent.
I allowed in CMP looks like it was doing something... no file dates changed on archives.
-
Model 110 should be gone since some test versions...
your rebuilder problem gone, too nice...
crash...well...as I said....not yet fully tested ;-) ...and I removed the package...good to hear the rebuild thing is gone
I know what the crash is all about (index error...most likely based on the change of name and resort....)...but I need some more time next week to find the (most likely simple) root issue...
so...delete the build and use the official cmpro ;-)
-
Well, this DAT & Files doesn't repeat crash.
But it does show archives remain the same after a run of "Scanner".
Set everything merged 7z
Untick Samples. Tick parse merge tags..
Use these 7z's as your ROM set
Then Scan away..
EDIT: I will keep using this CMP as it seems for everything else is OK
Plus a couple of recent changes I am now dependant on for speed.
-
as I said..do not use yesterdays build :-)
See it as a fixed version for your rebuilding thing..nothing more...it has known other issues
-
Good to know that you are working on the fixes and changes. :)
-
Well, this DAT & Files doesn't repeat crash.
But it does show archives remain the same after a run of "Scanner".
Set everything merged 7z
Untick Samples. Tick parse merge tags..
Use these 7z's as your ROM set
Then Scan away..
EDIT: I will keep using this CMP as it seems for everything else is OK
Plus a couple of recent changes I am now dependant on for speed.
I blame 7z... ;-) Works fine with zip.... your 7z files are solid....modifying them won't work....since 7z will simply fail on deleting single entries (to reorder them)....
-
I wrote a 7z Front_End.exe captures all Inputs/Outputs of CMP & 7z processing.
Some message boxes & text logs.
It also confirms that 7z cmd line 9.34a is a complete flop when using with CMP.
##########################################
7-Zip 9.34 alpha Copyright (c) 1999-2014 Igor Pavlov 2014-06-22
Scanning
Creating archive D:\cmp4014__32\temp.7z
Compressing qwerty
Everything is Ok
Kernel Time = 0.000 = 0%
User Time = 0.031 = 200%
Process Time = 0.031 = 200% Virtual Memory = 4 MB
Global Time = 0.015 = 100% Physical Memory = 5 MB
7-Zip 9.34 alpha Copyright (c) 1999-2014 Igor Pavlov 2014-06-22
Updating archive D:\cmp4014__32\temp.7z
Everything is Ok
Kernel Time = 0.031 = 100%
User Time = 0.015 = 50%
Process Time = 0.046 = 150% Virtual Memory = 2 MB
Global Time = 0.031 = 100% Physical Memory = 4 MB
##########################################
While it passes CMP's internal initial test..
In the real world it fails with many archives..
So Roman if you do any tests, please use 9.28a of 7z.exe >:(
##########################################
7-Zip 9.34 alpha Copyright (c) 1999-2014 Igor Pavlov 2014-06-22
Error:
There is some data block after the end of the archive
Not implemented
Kernel Time = 0.031 = 66%
User Time = 0.015 = 33%
Process Time = 0.046 = 100% Virtual Memory = 3 MB
Global Time = 0.046 = 100% Physical Memory = 4 MB
System error:
Not implemented
##########################################
I learnt today that this 9.34a 7z.exe could not process the files in my earlier post...(Attached @ post #70 in this thread.)
9.28a works perfectly ::) I had forgotten I still had 9.34a in there after attempting to prevent the crash while scanning the merged MAME set.
-
so I switched to the last official (9.20) one from www.7-zip.org (http://www.7-zip.org) (I know there are other releases on sourceforge) and it works perfectly fine with your #70 post files.....most likely because the "rn" rename command cannot be used with this version...
gonna search a "rn" support version now and will check what the reason for failing could be...
ok...used 9.28...with enabled rn support...also no issues with #70 files....so I guess I can put this problem on the list "weird things with new 7z versions"
-
I been using the 9.28a very long time with the clrmamepro. More stable than the current.
-
Hi Dullaron,
yes I have been a huge advocate of 9.28a since I did lots of testing with 7z & CMP (quite a while ago)
From the output of 9.34a we see:
7-Zip 9.34 alpha Copyright (c) 1999-2014 Igor Pavlov 2014-06-22
Error:
There is some data block after the end of the archive
Not implemented
This means 9.34a will simply refuse to do 'anything' with a 7z created with t7z..
Not ideal for many of us :P
7-Zip 9.34 alpha = no, no, no !
There maybe other 7z versions (9.25a?) which are useful, but 9.28a was the one I settled on after exhaustive testing
Users will also find Newer Installations of 7z UI will give constant annoying messages about 'this type' of 7z archive too.
???