EMULAB Forum

clrmamepro [English] => clrmame Discussion => Topic started by: yescabernetnointernet on 14 April 2020, 09:25

Title: [SOLVED] chdman v4 > v5 changes SHA1, clrmamepro reports wrong chd sha1sum
Post by: yescabernetnointernet on 14 April 2020, 09:25
Hey Roman (https://www.emulab.it/forum/index.php?action=profile;u=62), congrats for you huge work with that nice piece of software which is cmamep... It's super!  8)
And also fits really nice in Linux via WINE!  ;)

As in title, I'd like to have a clarification by you, please be so kind!

I'm using Latest Nightly Build (https://www.emulab.it/forum/index.php?topic=4176.0).
So... I found some CHDs have mismatch version for what I say in cmamep to check against to.
Some of them are v4 and some others are v5.
I tried and done some experiments... 'Cause I'd like to have all CHDs converted to v5!  ;D
If I change the version to v4 to v5 the SHA1 also changes, and cmamep found a wrong CHD entry (before conversion, that CHD was was recognized as correct).

Here's to you what happens:
Code: [Select]
ls *.chd | parallel chdman info -i {}               
chdman - MAME Compressed Hunks of Data (CHD) manager 0.218 (unknown)
Input file:   gdl-0012.chd
File Version: 4
Logical size: 1,344,333,888 bytes
Hunk Size:    9,792 bytes
Total Hunks:  137,289
Unit Size:    2,448 bytes
Total Units:  549,156
Compression:  zlib (Deflate)
CHD size:     110,731,452 bytes
Ratio:        8.2%
SHA1:         aae4c1321fdee37d5405c6cbe648e0596624ed13
Data SHA1:    6326b21cfdc04490eb0ba82bc15d2fbc1ac8fac8
[...]

chdman - MAME Compressed Hunks of Data (CHD) manager 0.218 (unknown)
Input file:   gdl-0012v50174.chd
File Version: 5
Logical size: 1,344,333,888 bytes
Hunk Size:    9,792 bytes
Total Hunks:  137,289
Unit Size:    2,448 bytes
Total Units:  549,156
Compression:  cdlz (CD LZMA), cdzl (CD Deflate), cdfl (CD FLAC)
CHD size:     66,551,507 bytes
Ratio:        5.0%
SHA1:         d2097461b5c6cffca5214a4fc7c685e5b620c156
Data SHA1:    6326b21cfdc04490eb0ba82bc15d2fbc1ac8fac8
[...]

chdman - MAME Compressed Hunks of Data (CHD) manager 0.218 (unknown)
Input file:   gdl-0012v50218.chd
File Version: 5
Logical size: 1,344,333,888 bytes
Hunk Size:    9,792 bytes
Total Hunks:  137,289
Unit Size:    2,448 bytes
Total Units:  549,156
Compression:  cdlz (CD LZMA), cdzl (CD Deflate), cdfl (CD FLAC)
CHD size:     66,551,507 bytes
Ratio:        5.0%
SHA1:         d2097461b5c6cffca5214a4fc7c685e5b620c156
Data SHA1:    6326b21cfdc04490eb0ba82bc15d2fbc1ac8fac8
[...]

Is this a normal behaviour? I mean, MAME uses only SHA1 in its xml list to check CHD's datas' correctness?
Or cmamep can also verify internal SHA1 (Data SHA1)?

I did a test with some settings in Scanner, with no luck  :'(
(https://i.ibb.co/XLT9Gs1/clrmamepro-scanner-advanced.jpg) (https://ibb.co/XLT9Gs1) (https://i.ibb.co/NLb1TZr/clrmamepro-scanner-hashandchd.jpg) (https://ibb.co/NLb1TZr)


Thanks, bye!!
Title: Re: chdman v4 > v5 changes SHA1 (NO Data), and so clrmamepro reports wrong chd
Post by: Roman on 14 April 2020, 10:49
MAME uses only SHA1 in its xml list to check CHD's datas' correctness?
Yes, the XML lists a sha1 for the decompressed data of a chd. This sha1 is also stored in the chd header and a standard check (by cmpro and most likely when MAME loads a chd) is reading the chd header and comparing the stored information there with the one in the XML (same as e.g. crc32 values and zip file entries).

Enabling the "decompress CHD & check sha1/md5" option will verify the chd by calling chdman internally (I'm not sure if running an exe works in a WINE environment, it surely does under Windows...)


And yes, the sha1 value changes by using a newer CHD format since the newer one included some metadata and so the sha1 hash is calculated over additional data.
Title: Re: chdman v4 > v5 changes SHA1 (NO Data), and so clrmamepro reports wrong chd
Post by: yescabernetnointernet on 14 April 2020, 11:08
calling chdman internally (I'm not sure if running an exe works in a WINE environment, it surely does under Windows...)
I can confrim it works! 100% sure  ;)
You first need to define the correct path to chdman.exe in settings (or via cmpro.ini)... Then when the Scanner calls chdman internally the WINE environment brings up chdman.exe correctly  8)
Code: [Select]
[CMPRO SETTINGS]
[...]
Packer_ChdMan_Exe = C:\Program Files\mame0174\chdman.exe
[...]

Btw, Roman (https://www.emulab.it/forum/index.php?action=profile;u=62), is there a way to add a feature in cmamep to do the doublecheck between SHA1 and Data SHA1?
Are you intereted in adding it?  ;)
Title: Re: chdman v4 > v5 changes SHA1 (NO Data), and so clrmamepro reports wrong chd
Post by: Roman on 14 April 2020, 11:16
There is only one sha1 entry in MAME's -listxml output, so that should give you the answer ;-)
Title: Re: chdman v4 > v5 changes SHA1 (NO Data), and so clrmamepro reports wrong chd
Post by: yescabernetnointernet on 14 April 2020, 11:31
Yup, you're right... I know, but...
See if I'm right with this example:

You think it's worth adding in cmamep?


Bye!
Title: Re: chdman v4 > v5 changes SHA1 (NO Data), and so clrmamepro reports wrong chd
Post by: Roman on 14 April 2020, 11:50
No it's not worth it.
The chd format changes maybe once in a decade and cmpro only works on the data provided by the datfile xml....

There is also no need in general. The  current sha1 value check against the stored header information is enough unless you got corrupt hardware where bits get flipped in the image itself or you got incomplete downloads.
Title: Re: chdman v4 > v5 changes SHA1 (NO Data), and so clrmamepro reports wrong chd
Post by: yescabernetnointernet on 14 April 2020, 12:09
And then I'll uncheck "List CHD version mismatches during scan" (Settings>Compressor>CHDMAN)!  ;)


Thanks Roman (https://www.emulab.it/forum/index.php?action=profile;u=62)!
Title: Re: [SOLVED] chdman v4 > v5 changes SHA1, clrmamepro reports wrong chd sha1sum
Post by: Roman on 14 April 2020, 12:37
Actually, I don't think there still are CHDMAN V4 images in MAME, aren't there? So you can keep that option enabled....
Title: Re: [SOLVED] chdman v4 > v5 changes SHA1, clrmamepro reports wrong chd sha1sum
Post by: yescabernetnointernet on 14 April 2020, 13:54
After years of my goodbye to MAME (around 2005 ~ 2006 I guess) I'm rediscovering all my old backups and returning to this dark side of (MAME) emulation... Eh eh eh :D
Just to pass the time while remainig home due to COVID-19...

A lot of things changed in MAME world, btw!