21
clrmame Discussion / Re: Updated to 4.048 - seems to crash when looking for unknown sets/unneeded?
« on: 29 October 2023, 11:01 »
Found it....
- 28 November 2023, 22:21
- Welcome, Guest
News:
The new forum is online, hope you enjoy it!
Show Posts
This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.
22
clrmame Discussion / Re: delta chds, cmpro test version
« on: 29 October 2023, 07:46 »
wow..lots of information...I try to work on them as quick as possible...
regarding the convert fails on some chds, maybe it's related to version differences....there are some parent v4 chds in MAME collections which are flagged as baddump where the clones are v5 chds....
Checking the rest hopefully soon....
regarding the convert fails on some chds, maybe it's related to version differences....there are some parent v4 chds in MAME collections which are flagged as baddump where the clones are v5 chds....
Checking the rest hopefully soon....
23
clrmame Discussion / Re: Updated to 4.048 - seems to crash when looking for unknown sets/unneeded?
« on: 29 October 2023, 07:42 »
hmm...doesn't happen for me....I will do further checks.
25
clrmame Discussion / clrmamepro 4.048 released
« on: 28 October 2023, 18:49 »
4.048
added: delta chd support, show warnings, support convert/revert of chds
fixed: MAME 260 "samples" issue (is shown as unneeded and not found)
For delta chd support, see here:
https://www.emulab.it/forum/index.php?topic=9157.msg25906#msg25906
added: delta chd support, show warnings, support convert/revert of chds
fixed: MAME 260 "samples" issue (is shown as unneeded and not found)
For delta chd support, see here:
https://www.emulab.it/forum/index.php?topic=9157.msg25906#msg25906
26
clrmame Discussion / Re: New samples in MAME 0.260
« on: 28 October 2023, 17:53 »
yep, thanks...as I said...I already know the problem
27
clrmame Discussion / Re: New samples in MAME 0.260
« on: 28 October 2023, 17:32 »
ah...I already know that the problem is....
will work on a fix
will work on a fix
28
clrmame Discussion / delta chds, cmpro test version
« on: 26 October 2023, 18:05 »
Ok, here we go....a test version for some delta chd features.
Disclaimer: This is not an official release, it's just a 64bit cmpro exe file which can replace yours. Do it on your own risk. Any chd convert/revert operation, you do it on your own risk.
What are the delta chd features (screenshots here: https://www.emulab.it/forum/index.php?topic=9148.msg25902#msg25902) ?
In Settings->Compressor...->CHDMan, you find a new section "Delta CHDs". It also holds two new command line boxes for converting and reverting. They are filled with the default copy operations, don't change them.
The section holds options where you can decide if cmpro
- should warn you about chds which can be converted to deltas
- chds which are deltas
- or you don't want to see any warnings at all (default).
There is also a checkbox (ticked by default) which enables backup when a convert/revert operation actually takes place.
After running a (new) scan and having a warning option enabled, you might see warnings in the warnings window below the scanner. It can list something like:
- Non-delta chd: some.chd Convertable via someother.chd ( showed warning about non-delta chds )
- Delta chd: some.chd Revertable via someother.chd (showed warning about delta chds)
- Bad parent delta chd: some.chd Revertable via someother.chd (error: parents can't be deltas)
- Bad clone delta chd: some.chd Revertable via someother.chd (error: the delta clone uses a different chd than the current parent)
- Can't convert/revert some.chd. Can't find a non delta CHD with SHA1: somesha1 (error: parent is not available)
If one or more of the messages are shown, you can use the popup contextmenu on one of the lines (you can also select multiple) and select one of the delta chd options in the menu. You can convert or revert all/selected. Depending on the selection you see either the one or the other or both options...
When selecting convert or revert, chdman is called and run in the background. If something went wrong or your used ctrl-c to exit chdman and you got multiple files in the queue, you will be asked if you want to continue.
If you ticked the backup option (you should), delta and absolute chds are copied to your backup folder. For this, subfolders are created in there _chd_abs_ and chd_delta_.
Known things:
- delta chds are always smaller? No, there are cases, especially when it comes to examples where the parent chd is a v4 baddump and the clone is a v5 chd, the delta version is slightly bigger
- when a chd is converted and reverted, I get the original file back? No, not necessarily. There are chds which were created with an older version of chdman and converting/reverting them with the current one ends up with a slightly smaller file. This is most likely based on the fact that the newer chdman uses a newer version of lzma which is responsible for the size loss. However, the raw data and its sha1 will be the same.
- when thinking about reverting, cmpro's unneeded/set check helps to find parent chds which might be located elsewhere or named incorrectly
- Can I convert all clone chds? If there is a belonging parent chd, yes (when CHD version is >= 3). Belonging? Well, there are sets which have more than one chd. To identify a belonging chd, the chd's "region" (and for software lists, the "part") information is used for a match. If a chd is not listed with a region/part, cmpro won't use it for any delta operations.
Well..happy testing...and don't forget: it's your fault when something goes wrong ;-)
Don't know if it's worth all the trouble....in times of multi terabyte HDs.....but the warnings might be useful.
Disclaimer: This is not an official release, it's just a 64bit cmpro exe file which can replace yours. Do it on your own risk. Any chd convert/revert operation, you do it on your own risk.
What are the delta chd features (screenshots here: https://www.emulab.it/forum/index.php?topic=9148.msg25902#msg25902) ?
In Settings->Compressor...->CHDMan, you find a new section "Delta CHDs". It also holds two new command line boxes for converting and reverting. They are filled with the default copy operations, don't change them.
The section holds options where you can decide if cmpro
- should warn you about chds which can be converted to deltas
- chds which are deltas
- or you don't want to see any warnings at all (default).
There is also a checkbox (ticked by default) which enables backup when a convert/revert operation actually takes place.
After running a (new) scan and having a warning option enabled, you might see warnings in the warnings window below the scanner. It can list something like:
- Non-delta chd: some.chd Convertable via someother.chd ( showed warning about non-delta chds )
- Delta chd: some.chd Revertable via someother.chd (showed warning about delta chds)
- Bad parent delta chd: some.chd Revertable via someother.chd (error: parents can't be deltas)
- Bad clone delta chd: some.chd Revertable via someother.chd (error: the delta clone uses a different chd than the current parent)
- Can't convert/revert some.chd. Can't find a non delta CHD with SHA1: somesha1 (error: parent is not available)
If one or more of the messages are shown, you can use the popup contextmenu on one of the lines (you can also select multiple) and select one of the delta chd options in the menu. You can convert or revert all/selected. Depending on the selection you see either the one or the other or both options...
When selecting convert or revert, chdman is called and run in the background. If something went wrong or your used ctrl-c to exit chdman and you got multiple files in the queue, you will be asked if you want to continue.
If you ticked the backup option (you should), delta and absolute chds are copied to your backup folder. For this, subfolders are created in there _chd_abs_ and chd_delta_.
Known things:
- delta chds are always smaller? No, there are cases, especially when it comes to examples where the parent chd is a v4 baddump and the clone is a v5 chd, the delta version is slightly bigger
- when a chd is converted and reverted, I get the original file back? No, not necessarily. There are chds which were created with an older version of chdman and converting/reverting them with the current one ends up with a slightly smaller file. This is most likely based on the fact that the newer chdman uses a newer version of lzma which is responsible for the size loss. However, the raw data and its sha1 will be the same.
- when thinking about reverting, cmpro's unneeded/set check helps to find parent chds which might be located elsewhere or named incorrectly
- Can I convert all clone chds? If there is a belonging parent chd, yes (when CHD version is >= 3). Belonging? Well, there are sets which have more than one chd. To identify a belonging chd, the chd's "region" (and for software lists, the "part") information is used for a match. If a chd is not listed with a region/part, cmpro won't use it for any delta operations.
Well..happy testing...and don't forget: it's your fault when something goes wrong ;-)
Don't know if it's worth all the trouble....in times of multi terabyte HDs.....but the warnings might be useful.
29
clrmame Discussion / Re: Delta CHDs new feature.
« on: 26 October 2023, 08:12 »
If something fails during chdman conversion or you ctrl-c break out and are currently running multiple ones, you are prompted if you want to continue or not.
Generally, if something fails, the files are not touched...
So it's like
convert .chd -> .delta, if something fails, stop (or prompt in case of a being in a loop)
rename .chd -> .abs (if something fails, remove the .delta)
copy .abs to backup (if something fails, remove the .delta, rename .abs back to .chd)
rename .delta -> .chd (....you know what...)
if we came here, delete .abs
similar for reverting
hopefully I wrote that down without looking at the code ;-)
Generally, if something fails, the files are not touched...
So it's like
convert .chd -> .delta, if something fails, stop (or prompt in case of a being in a loop)
rename .chd -> .abs (if something fails, remove the .delta)
copy .abs to backup (if something fails, remove the .delta, rename .abs back to .chd)
rename .delta -> .chd (....you know what...)
if we came here, delete .abs
similar for reverting
hopefully I wrote that down without looking at the code ;-)
30
clrmame Discussion / Re: Delta CHDs new feature.
« on: 25 October 2023, 18:14 »
Coming closer:
- you can warn about non-deltas or deltas...or you don't care at all about deltas..
- convert or revert found single or multi (delta) chds after a scan (warnings appear in the warnings windows, popup menu gives you options for revert/convert)
- backup old abs/delta chds so nothing gets lost...(backup folder gets _chd_abs_ and _chd_delta_ subfolders)
- detects bad parent chds which are stored as delta (this is not allowed...you need to revert the file if possible)
- detects bad clone chds where the parent is 'lost', i.e. it was created with a different chd than the current parent (you need to revert the file if possible)
- revert possibility when 'parent' chd is found somewhere in your collection
- software lists included via part names
Generally, for converting the parent and clone file have to exist and they share the same "region"/"part name" in the datfile. For reverting, the chd which was used to generate the delta needs to be in your collection. The warnings appear after a scan in the warnings window.
doing some more testing and thinking about a good way when you only do a scan (not new scan) where not all chds are scanned….
by the way: delta converting a file and reverting it may result in a slightly different file….most likely based on the fact that the original chd file was created with a different chdman in the past with an older lzma or zlib version….the sha1 do match though…
- you can warn about non-deltas or deltas...or you don't care at all about deltas..
- convert or revert found single or multi (delta) chds after a scan (warnings appear in the warnings windows, popup menu gives you options for revert/convert)
- backup old abs/delta chds so nothing gets lost...(backup folder gets _chd_abs_ and _chd_delta_ subfolders)
- detects bad parent chds which are stored as delta (this is not allowed...you need to revert the file if possible)
- detects bad clone chds where the parent is 'lost', i.e. it was created with a different chd than the current parent (you need to revert the file if possible)
- revert possibility when 'parent' chd is found somewhere in your collection
- software lists included via part names
Generally, for converting the parent and clone file have to exist and they share the same "region"/"part name" in the datfile. For reverting, the chd which was used to generate the delta needs to be in your collection. The warnings appear after a scan in the warnings window.
doing some more testing and thinking about a good way when you only do a scan (not new scan) where not all chds are scanned….
by the way: delta converting a file and reverting it may result in a slightly different file….most likely based on the fact that the original chd file was created with a different chdman in the past with an older lzma or zlib version….the sha1 do match though…
31
clrmame Discussion / Re: Delta CHDs new feature.
« on: 24 October 2023, 04:33 »
Chdman takes full paths and I provide them whereever the clone and parent were found :-)
So….no need for manual copying/searching.
So….no need for manual copying/searching.
32
clrmame Discussion / Re: Delta CHDs new feature.
« on: 23 October 2023, 20:22 »
Yeah I think I will go with the popup context menu options….more tomorrow
33
clrmame Discussion / Re: Delta CHDs new feature.
« on: 23 October 2023, 19:46 »
yes….need to check how to revert a chd and I still think about the way a conversion makes sense….just having that option enabled can cause a long long scan but currently I dont have a good idea about a different way….I mean if users enable the option they know what is coming up…..
or maybe just showing a warning as mentioned above and you can click it and select ‚convert this‘ or ‚convert all‘….via a popup context menu….need to take a nap 😀
or maybe just showing a warning as mentioned above and you can click it and select ‚convert this‘ or ‚convert all‘….via a popup context menu….need to take a nap 😀
34
clrmame Discussion / Re: Delta CHDs new feature.
« on: 23 October 2023, 18:07 »
ok..let's see what I got....and what needs to be done...
Already added the ability to:
- show an error entry (scanner, tree view) if a parent chd is a delta chd (parent chds can't be deltas, this can happen when MAME changed parent/clone relationships)
- show an error entry (scanner, tree view) if a clone chd is a delta chd but the parent sha1 stored in it doesn't match the current parent chd (as above, maybe some parent/clone relationships changed)
- optionally show a warning about a clone chd which can be a delta but isn't (warning window below scanner) (default: false)
- optionally auto converting a clone to a delta via chdman call (similar to do deep chd checks), see attached screenshot (default: false)
- optionally doing a backup of the original clone chd before it gets converted to a delta (default: false, gets auto-enabled with enabling converting option, but you can disable it afterwards)
To do:
- if the upper mentioned errors appear, you would need to revert the delta to the original image. For that, the right chd needs to be found first (could be outside of the current parent/clone relationship)
- software list chds are not yet supported. Currently a possible delta-conversion is detected if the parent/clone disks are available AND they share the same "region" information. In software lists you have "parts"...and I need to do some more investigation on them...if it's safe to do a match by using this information.
Generally, converting the chds takes a lot of time...plus the checks in cmpro are purely experimental for now.....Keep you updated.
Already added the ability to:
- show an error entry (scanner, tree view) if a parent chd is a delta chd (parent chds can't be deltas, this can happen when MAME changed parent/clone relationships)
- show an error entry (scanner, tree view) if a clone chd is a delta chd but the parent sha1 stored in it doesn't match the current parent chd (as above, maybe some parent/clone relationships changed)
- optionally show a warning about a clone chd which can be a delta but isn't (warning window below scanner) (default: false)
- optionally auto converting a clone to a delta via chdman call (similar to do deep chd checks), see attached screenshot (default: false)
- optionally doing a backup of the original clone chd before it gets converted to a delta (default: false, gets auto-enabled with enabling converting option, but you can disable it afterwards)
To do:
- if the upper mentioned errors appear, you would need to revert the delta to the original image. For that, the right chd needs to be found first (could be outside of the current parent/clone relationship)
- software list chds are not yet supported. Currently a possible delta-conversion is detected if the parent/clone disks are available AND they share the same "region" information. In software lists you have "parts"...and I need to do some more investigation on them...if it's safe to do a match by using this information.
Generally, converting the chds takes a lot of time...plus the checks in cmpro are purely experimental for now.....Keep you updated.
35
clrmame Discussion / clrmamepro 4.047b released
« on: 22 October 2023, 16:58 »
4.047b
fixed: don't mark delta chds as missing
misc: updated unrar (6.24.0)
I will think about additional options (like optionally marking found not-delta chds and maybe a conversion tool/update opption) but for this I need more time.
fixed: don't mark delta chds as missing
misc: updated unrar (6.24.0)
I will think about additional options (like optionally marking found not-delta chds and maybe a conversion tool/update opption) but for this I need more time.
36
clrmame Discussion / Re: Delta CHDs new feature.
« on: 22 October 2023, 12:26 »
ok...attached a quick version so that cmpro works again....can anyone give it a try for delta chds?
Thinking about
a) an optional warning if found clone disks could get replaced by a delta
b) maybe a standalone tool for converting....(guess that can be achieved quicker than adding some conversion thing into old cmpro..)
but don't count on a) and b) in the very near future....
Thinking about
a) an optional warning if found clone disks could get replaced by a delta
b) maybe a standalone tool for converting....(guess that can be achieved quicker than adding some conversion thing into old cmpro..)
but don't count on a) and b) in the very near future....
37
clrmame Discussion / Re: Delta CHDs new feature.
« on: 22 October 2023, 07:22 »
Yeah yeah keeping an old man busy....
Thanks for the testing and information....First thing I will do is of course checking why a delta is not recognized when it got the correct name in the correct place...most likely because the chd header changed a bit...
"fixing" or better "auto-create a delta" out of a found non-delta clone....hmm....will see if this is easy to add (well it is...but I have very limited time).
Question is if (or if it makes sense that) each and every clone chd can be converted to a delta compared to its parent. Haven't checked yet but I'm sure that there are sets with multiple chds and maybe a clone of it also has multiple ones...question is how to find the correct parent chd to the clone chd in such a multi-case. ....most likely if the region entries match...like "region="pci:05.0:ide:0:hdd"" ...
to sum it up: I try my best to update cmpro as quick as possible....
Thanks for the testing and information....First thing I will do is of course checking why a delta is not recognized when it got the correct name in the correct place...most likely because the chd header changed a bit...
"fixing" or better "auto-create a delta" out of a found non-delta clone....hmm....will see if this is easy to add (well it is...but I have very limited time).
Question is if (or if it makes sense that) each and every clone chd can be converted to a delta compared to its parent. Haven't checked yet but I'm sure that there are sets with multiple chds and maybe a clone of it also has multiple ones...question is how to find the correct parent chd to the clone chd in such a multi-case. ....most likely if the region entries match...like "region="pci:05.0:ide:0:hdd"" ...
to sum it up: I try my best to update cmpro as quick as possible....
38
clrmame Discussion / Re: Delta CHDs new feature.
« on: 21 October 2023, 20:36 »
Well, I think regarding auditing nothing much will change. I haven't looked at the changes yet, but I assume that
a) the hash / name values of the disk entries in -listxml output doesn't change
b) the chd header will get some additional flag that shows that it is a delta chd
cmpro reads only the chd header and compares the hashes against the hashes in the xml output ...so that will still work....a deep check is done via chdman anywhow...so I think that will also still work since they probably update chdman's test routine...
....so in the end it might be only something like "couldn't cmpro show me that I am able to use a delta chd instead of a full one" request....
but hey...I might be wrong...time will tell....thanks for the headup
a) the hash / name values of the disk entries in -listxml output doesn't change
b) the chd header will get some additional flag that shows that it is a delta chd
cmpro reads only the chd header and compares the hashes against the hashes in the xml output ...so that will still work....a deep check is done via chdman anywhow...so I think that will also still work since they probably update chdman's test routine...
....so in the end it might be only something like "couldn't cmpro show me that I am able to use a delta chd instead of a full one" request....
but hey...I might be wrong...time will tell....thanks for the headup
39
clrmame Discussion / little hint for new rebuilder users using -p #BIOSSPLIT# or #TYPE#
« on: 17 October 2023, 06:47 »
While testing rebuilder/scanner the following drove me nuts...updating mame.ini with the generated paths from the upper mentioned methods didn't work. "MAME -rompath f:\mame\roms\#default pacman" did work, but adding "f:\mame\roms\#default" to mame.ini failed....so I wondered why...and oh man...feeling stupid......having a "#" as some prefix in a rompath is not a good idea for usage in an ini file. It simply takes the rest of the line as comment :-)
So your rebuilder settings.xml should use either your own values or the following new defaults (without the #)
<TypeNames>
<Bios>bios</Bios>
<Mechanical>mechanical</Mechanical>
<Default>default</Default>
<Device>device</Device>
<BiosSplit>bios_</BiosSplit>
</TypeNames>
Will be updated for next release.....
So your rebuilder settings.xml should use either your own values or the following new defaults (without the #)
<TypeNames>
<Bios>bios</Bios>
<Mechanical>mechanical</Mechanical>
<Default>default</Default>
<Device>device</Device>
<BiosSplit>bios_</BiosSplit>
</TypeNames>
Will be updated for next release.....
40
clrmame Discussion / Re: support empty directories
« on: 12 October 2023, 06:52 »
So not really MAME related ;-)
<rom name="2D/" size="0" crc="00000000" md5="d41d8cd98f00b204e9800998ecf8427e" sha1="da39a3ee5e6b4b0d3255bfef95601890afd80709" />
<rom name="A/" size="0" crc="00000000" md5="d41d8cd98f00b204e9800998ecf8427e" sha1="da39a3ee5e6b4b0d3255bfef95601890afd80709" />
<rom name="audience/" size="0" crc="00000000" md5="d41d8cd98f00b204e9800998ecf8427e" sha1="da39a3ee5e6b4b0d3255bfef95601890afd80709" />
hmm...wonder where md5/sha1 comes from ;-)
Well, easiest would be to auto-add an empty file when it finds a name ending on "/" when it loads the dat
...but then of course you'd have to add the files (can't get rebuilt due to missing hashes)...
will think about it....but don't count on any solution any time soon ;-)
<rom name="2D/" size="0" crc="00000000" md5="d41d8cd98f00b204e9800998ecf8427e" sha1="da39a3ee5e6b4b0d3255bfef95601890afd80709" />
<rom name="A/" size="0" crc="00000000" md5="d41d8cd98f00b204e9800998ecf8427e" sha1="da39a3ee5e6b4b0d3255bfef95601890afd80709" />
<rom name="audience/" size="0" crc="00000000" md5="d41d8cd98f00b204e9800998ecf8427e" sha1="da39a3ee5e6b4b0d3255bfef95601890afd80709" />
hmm...wonder where md5/sha1 comes from ;-)
Well, easiest would be to auto-add an empty file when it finds a name ending on "/" when it loads the dat
...but then of course you'd have to add the files (can't get rebuilt due to missing hashes)...
will think about it....but don't count on any solution any time soon ;-)