Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[SiriusVI]

Hey everyone,

I usually convert my disc based roms, such as PSX or Dreamcast, to CHD in order to save space and also because a single .chd file is much easier to deal with than .cue with multiple .bin files.

Now I want to find out which roms I'm missing, however, I cannot get any rom manager working with my .chd collections. Clrmamepro says I don't have a valid set, which makes sense.

My question is whether it is at all possible to scan my collection for missing roms ect. even though I converted my sets to .chd.

Thanks for your help! =)

[Roman]

CHD stands for  "C"ompressed "H"unks of "Data", so it's a container (like a zipfile) for files, in this case hd/dvd/ld/floppy/etc../etc dumps.
Using chdman to convert your files to  a CHD is one thing, auditing is a different. First of all, you need to specify the chd in the datfile. Take a look at for example MAME's -listinfo output. CHD images are encoded in disk xml elements listing their decompressed sha1 values (which is also stored in the chd header).

For example:
<disk name="a51site4-2_01" sha1="48496666d1613700ae9274f9a5361ea5bbaebea0" region="ide:0:hdd:image" index="0" writable="yes"/>

So you need a custom datfile to check your sets since you need to tell cmpro what to look for and what the correct hash values are.

Secondly, you need to follow the general storing method, which is coming originally from MAME:
rompath\setname\file 1... file n for decompressed sets
rompath\setname.zip (.rar/.7z) for compressed sets
A set is a collection of roms and/or samples and/or chds. Since it doesn't make sense to put chds in another archive (7z/zip/rar) you store them in the first described way. For example:F:\MAME\ROMs\Standard\a51site4a\a51site4-2_0.chd where F:\MAME\ROMs\Standard is the rompath, a51site4a is the setname (defined in the datfile) and a51site4-2_0.chd is the actual chd file. Its base name also comes from the datfile.


So coming back to your question "My question is whether it is at all possible to scan my collection for missing roms ect. even though I converted my sets to .chd", the answer is no since the custom datfile will only contain information about the chd container and not the single files inside the container. You can scan for missing chds and sets of course but everything the rommanager knows about is what you (or whoever) defined in the used datfile.

[SiriusVI]

Thank you for you reply. I already expected the answer to be 'no'. I thought that maybe there was a hidden feature that lets me scan for existing rom files by file name. I'm sure the file names are still the same as in the .dat file, even after conversion to CHD.

[Roman]

I don't really understand your scenario...since either you have a datfile for dreamcast/etc and you are able to scan your not chd-converted images (and you know what is missing), or you need to setup a datfile on your own...which will take your existing chds as basis.....or you find a datfile which uses chds instead of raw images.

well....I hope you understand the main point. The main factor is your datfile.
You need to understand what it means if you the datfile lists the file as chd or as rom.
You can of course list the chd file as a rom, then the rom crc32/sha1 are the hash values of the chd file. If the datfile lists them as disks, the given sha1 is the hash value of the decompressed file.

So...if you on your own decide to compress images as chd files, you need to ensure that you have a datfile handy which supports your task.

So maybe I should rephrase my answer.
Of course you are able to scan for missing files (whatever they represent), as long as you use a correct datfile for your task.

[Roman]

Maybe the following helps to understand better:

Let's assume you currently use cmpro to scan your dreamcast/etc files in cue/bin format.
For this, you need a dafile which lists the sets with the individual files. They are listed there with size, name, hash information. It holds all sets which the datfile author added to the datfile. A set is collection of roms and/or samples and/or chds. A datfile is a collection of sets.

Now if you decide that you compress your files, you actually change information which is listed in the datfile. While you can compare a CHD with a zip container in terms of compression, you can't compare them in terms of auditing. You would be able to compress your sets to 7z/rar/zip and you would still be able to scan/audit them. But CHD is different, a chd is part of a set and not a compressed representation of a set.

So you can of course use 7z to compress your sets (cue + img + whatever other files are listed) to an archive and are still able to use the datfile.

If you really want to use the chd way and still want to scan your sets you can edit the used datfile and change the sets you converted or ask the original datfile author if he/she wants to provide a chd version

[SiriusVI]

Thanks for this answer. I understand that the dat file is the issue here. I just wish there was a way to let a rom manager scan for file names listed in the dat, and not the extra information, such as crc32/sha1, etc.

My scenario is the following:

1. I have an (incomplete) rom set.
2. I convert the rom set to CHD to save space and for easier handling.
3. I delete the original rom files
4. I want to be able to check for missing roms in the future, when the set is updated, so don't have to acquire the complete rom set again.

The reasons why I cannot use custom dat files for this are:
1. There are no custom dat files for self converted CHD sets.
2. Even if there were, the would be different for every user, since different users might use different levels of chd compression
3. I am not able to creat such dat files myself.

Thanks again for your answer =)

[Roman]

Thanks for this answer. I understand that the dat file is the issue here. I just wish there was a way to let a rom manager scan for file names listed in the dat, and not the extra information, such as crc32/sha1, etc.

Erm....clrmampro can do that. Simply turn off the checksum check.
But of course some other tests still need hashes to work correctly. For example if a file got a wrong name a match with the hashvalue will find the correct one.

[SiriusVI]

Really? I tried that, but mame just told me that I had an "invalid set" and after the scan all roms were missing. Don't know what I did wrong...

[Roman]

MAME told you?
If clrmamepro told you that no valid sets were found, you most likely did not follow the official storing schema which I already menioned in this thread (rompath/setname/file 1.. file n, etc..etc..)

But of course by converting the files to chd, you change the names (*.cue/.img to .chd) and since you did not update the datfile cmpro tries to use the hash to get the correct name of the new file...which will fail since you did not update the dat ;-) And unneeded files get removed (if you got unneeded check enabled and fix options on).

So...maybe you should reduce the scan options to "missing" only.

And again...what you try to achieve is a bit away what auditing is all about. You change formats, filenames and number of files and you don't update the used database (datfile) and expect that it will work out of the box

[SiriusVI]

I meant clrmampro told me, sorry. Ah yes the file extentions might be a problem here. Well, I guess that I'll just have to live with the fact that what I want is not possible right now.

[yescabernetnointernet]

Hey SiriusVI,
not later than yesterday I too provided for the conversion from zipped roms into CHDs, and I also confirm that there won't be any kind of audit you can issue against yours CHDs.
Like Roman said, you'll need a conformant DAT file to check against your CHDs; but, precisely, there are not DAT files which contains CHDs infos about your converted sets, out there... Sadly! 

Let me clarify the concept once more...
For example, let's take in consideration a SEGA Dreamcast's TOSEC DAT file:

• Grab TOSEC - DAT Pack - Complete (3076) (TOSEC-v2020-10-31).zip
• Now you'll fire up in clrmamepro the desired DAT, let's say Sega Dreamcast - Games - US (TOSEC-v2020-01-24_CM).dat
• clrMAMEpro will check your sets: compressed (*.zip, or *.7z) or not (folders), it doesn't matter (in this case we're working with GD-ROM images of the games)
• This audit procedure will verify datas declared inside Sega Dreamcast - Games - US (TOSEC-v2020-01-24_CM).dat against the files in your HDD
• OK... Now let's take a look at the structure of that DAT file, for example the Sonic Adventure v1.005 (1999)(Sega)(US)(M5)[!][24S] entry:

Code: [Select]

<game name="Sonic Adventure v1.005 (1999)(Sega)(US)(M5)[!][24S]">
<description>Sonic Adventure v1.005 (1999)(Sega)(US)(M5)[!][24S]</description>
<rom name="Sonic Adventure v1.005 (1999)(Sega)(US)(M5)[!][24S].gdi" size="89" crc="c2916cc9" sha1="2ee28c51c196d77dfba4f50de634b5c1e2ca08eb" md5="f43214efaec1e06a3af933cf5c49553d"/>
<rom name="track01.bin" size="34254528" crc="ab4c7e18" sha1="ccfcd5a46924c66454451353b916b96360ab5539" md5="fe340f253871df58bcfa3de35f1db257"/>
<rom name="track02.raw" size="2147376" crc="fddb5a17" sha1="4c5981df13be09e15d38faa124b0d9e38b892ea9" md5="bce8000330b663c25f5b91f9d2508ea1"/>
<rom name="track03.bin" size="1185760800" crc="00c55860" sha1="965dc5def50f6b8c6e9776c292c1a13a0947c403" md5="4d0ea0267b5e472812be4e525d14b804"/>
</game>

• See? clrMAMEpro will only look for that precise list of files in combination with their size+crc+sha1+md5! Nothing more, nothing less

[/lis
The only-and-one thing you could ever do, after converting your sets (aka games, in this case) into CHD sets, will be to issue an audit only checking for names!
So the audit procedure will be made against the <game name=""> or <description></description> values.

it would be really great if there ever existing a kind software to handle automagically double checking against a CHD, uncompress it (chdman extractcd -i somegame.chd -o somegame.*) , check again the extracted files if they're conformant to the original DAT and create a new DAT writing into it the new infos!!! 


In conclusion, the only meager consolation is that you could create your own custom DATs against your converted-into-CHD sets by using the Dir2Dat function.

P. S.: the latest chdman (0.227) now will handle correctly converting ReDump (.cue/.bin) games!! 

• 7422: chdman: Added support for importing redump.org GD-ROM extended bin/cue format (GitHub #6466). [nhand42]