Topic: Detector rule: size (Read 14073 times)

Zandro · « **on:** 26 January 2013, 07:07 »

I was attempting to write my first header skipper containing offset rules for files less than a specific filesize, and became frustrated that it was seemingly ignoring them, when it dawned on me that the skipper must be depending on the dat for size info rather than the physical files. The dat does not contain size info, so I must find another method to determine if a file is under a tested size. Is this possible?

There is no particular series of bytes marking the end of 'real data' in these files, and EOF can slide 100s of bytes between files. I considered forcing an illegal seek to default to EOF, but this also resets start_offset to 0.

I'll be the first to admit that I am less than spry with logic, so I hope you can show me the light with this. Or else confirm there is a limitation that I needn't feel stupid for anymore.

Zandro · « **Reply #1 on:** 26 January 2013, 14:34 »

I am also experiencing crashes involving an infinite loop (max cpu) when the rebuilder attempts to scan a smaller file, if no test is given for a header type guaranteeing at least the minimum filesize applicable for the defined range to hash check.

Roman · « **Reply #2 on:** 26 January 2013, 18:50 »

send me the header xml plus dat plus example files so I can have a look at that infinite loop...

regarding your original header idea...what exactly do you want to achieve?

Zandro · « **Reply #3 on:** 27 January 2013, 03:52 »

PM sent. I was interested in accounting for these several insufficiently sized files in addition to header skipping normally, and because I found a seemingly insurmountable bug in this situation, think it is better to have those files fixed by the collector to permit equal treatment of all.

Roman · « **Reply #4 on:** 27 January 2013, 12:55 »

I will have a look as soon as I find some time....

Roman · « **Reply #5 on:** 28 January 2013, 20:48 »

well...ok..the infinite loop was caused by a file where the EOF was reached already but not detected....
http://mamedev.emulab.it/clrmamepro/binaries/cmp20130138_header.rar

this should be resolved in that build...however I did not have time yet to think about your original task

Zandro · « **Reply #6 on:** 28 January 2013, 21:27 »

Using rules:
<rule start_offset="100" end_offset="10200">
<file size="10200" result="false" operator="less"/>
</rule>

<rule start_offset="100" end_offset="EOF">
</rule>

...Rebuilder now creates undersized test file with no more crashing, but also recreates on each run with statistics repeating "Created Files: 1" and archive timestamp updating. Scan results statistics is now blank, and scanner wants to remove rebuilt rom for wrong hashes. I'm just yutzing around at this point, I shouldn't need to use this method in the future. Re-focusing, moving on.. I appreciate your checking however.

Roman · « **Reply #7 on:** 28 January 2013, 21:36 »

I will do further checks (scanner/rebuilder) later...maybe next weekend earliest due to some business stuff to do first.....but at least the EOF loop is fixed

Zandro · « **Reply #8 on:** 28 January 2013, 21:37 »

This is Odd.
Wrong crc32: rom [wrong: a7ee68fd] [right: ea87ce34]
-- Hash is ~~f80c4a76.~~ ea87ce34 after header skip.
wrong sha1: rom [wrong: 7db857036ac9fba8fa9140d6b13079a86ba75684] [right: ad5e4a4486392aa52220a5bf8dfe64b553c9114b]
-- hash is ~~21b28e4c0b19908d4948088aea9fcbaef6860d36.~~ ad5e4a4486392aa52220a5bf8dfe64b553c9114b after header skip.

Roman · « **Reply #9 on:** 28 January 2013, 21:50 »

yeah yeah...I will look at it next week....

oxyandy · « **Reply #10 on:** 29 January 2013, 06:34 »

Hi Roman & Zandro,
A header related thing... so not starting a new thread, when I can hi-jack yours Zandro

This file is a NES header file 16 bytes long..
http://savage-blades.cjb.net/NES_HEADER_BLANK.hdr
Now, if I say try to rebuild a No-Intro NES set (which uses the header skipper),
and this file exists in the Rebuilder source folder,
well, CMP will freeze and never continue past it..

Do you think one day when you have some time, you could look into this ?
Thanks Roman !

Roman · « **Reply #11 on:** 29 January 2013, 08:12 »

you used the upper mentioned build already?

oxyandy · « **Reply #12 on:** 29 January 2013, 10:31 »

Quote

you used the upper mentioned build already?

No I have not..
Does the build you posted in this thread not have this problem ?
Ok, I will try it later..
Will see.. ??

Roman · « **Reply #13 on:** 29 January 2013, 10:48 »

well, http://mamedev.emulab.it/clrmamepro/binaries/cmp20130138_header.rar fixes an infinite loop.....

oxyandy · « **Reply #14 on:** 29 January 2013, 11:08 »

OMG, yes, I just used the version posted in this thread..
You have fixed it !
This is great, the 'infinite loop' getting stuck on 16 byte headers, has long bugged me, have meant to bring it to your attention dozens of times.

Thank-you Roman !

Roman · « **Reply #15 on:** 01 February 2013, 21:11 »

Zandro,

http://mamedev.emulab.it/clrmamepro/binaries/cmp20130201.rar

this should fix the remaining "undersized" rebuild/scan issues

Zandro · « **Reply #16 on:** 02 February 2013, 02:46 »

Yes, it does

However, stats output is still empty.

With Save LogFile enabled, this is written at the end:

Quote

----------
No statistics are available yet. Please run at least one scan.

I did do New and fast scans...

Maybe some debugging flag was left enabled?

Oxy, do you still get stats output?

Roman · « **Reply #17 on:** 02 February 2013, 18:34 »

Stats work fine here

oxyandy · « **Reply #18 on:** 04 February 2013, 06:28 »

Quote

Oxy, do you still get stats output?

Ah, I did not see your edit..
Yes I do..
(Roman's Beta builds, must always be added to a fresh download of current 'release' to ensure all files are present)
If you do not have stats.ini present in the folder "Statistics" window will be blank.
see here
http://www.emulab.it/forum/index.php?topic=3622.new#new

oxyandy · « **Reply #19 on:** 04 February 2013, 07:21 »

Zandro,
Can you repeat 'bug' from archive I posted ?
Extract, run CMP, set path to folder apple2gs

New Scan = Merge problem not fixed & Blank "Statistics" output

EDIT: Blank "Statistics" output (caused by missing stats.ini from program folder)

News:

Author Topic: Detector rule: size (Read 14073 times)