EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1] 2 3 4   Go Down

Author Topic: Status  (Read 16747 times)

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 92
  • Offline Offline
  • Posts: 2848
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 38.0.2125.111 Chrome 38.0.2125.111
    • View Profile
Status
« on: 07 November 2014, 19:54 »

....cough cough....shouldn't there be an update already?

erm...well..yes...but the last weeks were exhausting...a lot of real-life work project work kept me extremely busy...

So what's the status on the hash collision thing..(subfolder idea..)...actually...I'm pretty much annoyed of it. You run into issues here and there if you have subfolders and switch modes (from full to split or vice versa). To sum it up...I'm not really happy with the solution at the moment...

I may release a test version with it enabled sooner or later or change the subfolder idea to something else...or roll that change back and release the small other updates...

In the meantime (most likely because I was annoyed looking at the old code) I played a bit (just some minutes actually...) with pugixml (pugixml.org) and was surprised how fast it is....and had the idea of keeping the DOM as internal storage basis and have some xpath layer upon it to work with it...
For a second I was in the mood to write a MAME-only scanner/rebuilder from scratch with all the new ideas..and a totally different way of how things work. Not endless lists of options, not supporting this and that, not cleaning the xml which MAME provides...not exporting here..no batch jobs there....scan/fix/rebuild...done...but again...I'd need more time...

So...please be patient...I guess the current main version works fine...maybe some little fixes soon....
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 92
  • Offline Offline
  • Posts: 2848
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 39.0.2171.65 Chrome 39.0.2171.65
    • View Profile
Re: Status
« Reply #1 on: 25 November 2014, 09:27 »

Well well well... as you might notice...still no new version...

and you know already the answer...the bloody author is busy...yep..I am...and currently I'm so much more interested in doing a rewrite...and watching https://www.kickstarter.com/projects/thimbleweedpark/thimbleweed-park-a-new-classic-point-and-click-adv


So actually...I'm closer to a new version since I'm sick of having the code hanging on my harddrive...and so I decided to change the subfolder idea a bit....but stick to the idea of handling hash collisions differently (no splitting of relationships in split merge mode and 'handling' of it in full merge mode).
So I guess I make it the following...for full merged mode, hash collisions get a pre or postfix setname plus delimiter for the problematic files... like pacman2_1b.bin (for clone pacman2 where 1b.bin got a hash collision). The delimiter (in this case "_") will be stored in the cmpro ini file and can be changed...and actually you could change it to "/" and you get subfolders...(as mentioned before, this mode will have some hickups here and there, use it on your own risk).

Now what do you think....prefix or postfix and what should be the default delimiter?

Edit: or maybe I add the entry to the cmpro.ini where you can put together variables like %s_%r, %s/%r or %r___%s or %r_%c (for %s etname, %r omname, %c crc32) or something like this....
« Last Edit: 25 November 2014, 10:59 by Roman »
Logged

f205v

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 95
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Opera 12.17 Opera 12.17
    • View Profile
    • EMMA dumping team
Re: Status
« Reply #2 on: 25 November 2014, 11:18 »

The idea of using variables in .ini is wonderfull.
Maybe it could be that you split the .ini in 2 files: gui.ini and engine.ini
In this way, people could share engine.ini and be sure to create the exact same set of roms, leaving them free to have different gui settings on their respective PCs.
Logged
-------------
ciao
f205v

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 92
  • Offline Offline
  • Posts: 2848
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 39.0.2171.65 Chrome 39.0.2171.65
    • View Profile
Re: Status
« Reply #3 on: 25 November 2014, 12:38 »

hehe...well...we'll see....guess for a start it can be in the cmpro.ini...if there are hardcore users out there, they will find it anyway...

for now I'm sick of it (the source code)....playing around with boost/pugixml seems to be more fun..but also no real time for it...
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 92
  • Offline Offline
  • Posts: 2848
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 39.0.2171.65 Chrome 39.0.2171.65
    • View Profile
Re: Status
« Reply #4 on: 25 November 2014, 21:27 »

seems to work... (attached screenshot of some collisions with a hashcollide naming string "%1 [%x]"...where %1 is always the romname and you can use %x for hash value lowercase (crc32 for roms, sha1 for chds)....and lots of other variables which you may know from rebuilder advanced destination prestring...)

and another one with "%f_%1_%X" ...with prefix setname and postfix uppercase crc

and another with with a static prefix

and a last one with subfolders and year ;-)
« Last Edit: 25 November 2014, 21:33 by Roman »
Logged

f205v

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 95
  • Operating System:
  • Windows XP Windows XP
  • Browser:
  • Opera 12.17 Opera 12.17
    • View Profile
    • EMMA dumping team
Re: Status
« Reply #5 on: 25 November 2014, 22:28 »

It looks GREAT!
Logged
-------------
ciao
f205v

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 92
  • Offline Offline
  • Posts: 2848
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 39.0.2171.65 Chrome 39.0.2171.65
    • View Profile
Re: Status
« Reply #6 on: 26 November 2014, 08:27 »

well...as mentioned...as long as you don't use "/" it should run perfectly fine...."/" may cause some weirdness if you switch from split to fully merged or vice versa....but time will tell..

I guess I add an input box somewhere so you don't have to manually edit cmpro.ini for this setting....still thinking about a good default/initial one...my personal favourite is "%1 [%x]" at the moment (the first screenshot)....

guess I can put out a test version soon...maybe tonight....
Logged

oddi

  • Member
  • *
  • Karma: 2
  • Offline Offline
  • Posts: 178
  • Operating System:
  • Windows NT 6.3 Windows NT 6.3
  • Browser:
  • Firefox 33.0 Firefox 33.0
    • View Profile
Re: Status
« Reply #7 on: 26 November 2014, 11:20 »

Roman, agree with u, i watching screens - first or second screen is good .
First - u safe original dump name .
second - my old idea with prefix :) :) :)  - name of collisions_rom dump name , checksum is your idea :)
..and wait new test build, i wanna destroyed again my mame/mess sets.
Have a nice day :)
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 92
  • Offline Offline
  • Posts: 2848
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 39.0.2171.71 Chrome 39.0.2171.71
    • View Profile
Re: Status
« Reply #8 on: 26 November 2014, 12:56 »

well...the removal of subfolders where there still were some files in it was fixed some months ago already...
and it's up to you to change the hash collision naming as you want....
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 92
  • Offline Offline
  • Posts: 2848
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 39.0.2171.71 Chrome 39.0.2171.71
    • View Profile
Re: Status
« Reply #9 on: 26 November 2014, 20:58 »

as promised..

http://mamedev.emulab.it/clrmamepro/binaries/cmp20141126.rar


...and I used %1 [%f] as default now...
« Last Edit: 26 November 2014, 20:58 by Roman »
Logged

oddi

  • Member
  • *
  • Karma: 2
  • Offline Offline
  • Posts: 178
  • Operating System:
  • Windows NT 6.3 Windows NT 6.3
  • Browser:
  • Firefox 34.0 Firefox 34.0
    • View Profile
Re: Status
« Reply #10 on: 27 November 2014, 05:19 »

Roman now test over my mame set 0.156 full /t7z merged, see result after first scan :
 
Previous versions of cmp not correct check chds ?
 see first chd, absolutly same ,Post code only from this chd

Code: [Select]
INFO: "<<< g:\MAME\WIP_SVN\CHDs\g:\MAME\WIP_SVN\CHDs\863hdda01.chd >>>"
----------------------------------------------------------------------------------
chdman - MAME Compressed Hunks of Data (CHD) manager 0.156 (Nov 26 2014)
Input file:   g:\MAME\WIP_SVN\CHDs\863hdda01.chd
File Version: 5
Logical size: 3,228,696,576 bytes
Hunk Size:    4,096 bytes
Total Hunks:  788,256
Unit Size:    512 bytes
Total Units:  6,306,048
Compression:  lzma (LZMA), zlib (Deflate), huff (Huffman), flac (FLAC)
CHD size:     595,749,009 bytes
Ratio:        18.5%
SHA1:         0b8dbf1c9caf4abf965dbc6e1a8e6329d48b1c90
Data SHA1:    293b5430bc4946dcf506a935957064549c28540c
Metadata:     Tag='GDDD'  Index=0  Length=35 bytes
              CYLS:6256,HEADS:16,SECS:63,BPS:512.
----------------------------------------------------------------------------------
INFO: "<<< g:\MAME\WIP_SVN\CHDs\g:\MAME\WIP_SVN\CHDs\863hdda01_1.chd >>>"
----------------------------------------------------------------------------------
chdman - MAME Compressed Hunks of Data (CHD) manager 0.156 (Nov 26 2014)
Input file:   g:\MAME\WIP_SVN\CHDs\863hdda01_1.chd
File Version: 5
Logical size: 3,228,696,576 bytes
Hunk Size:    4,096 bytes
Total Hunks:  788,256
Unit Size:    512 bytes
Total Units:  6,306,048
Compression:  lzma (LZMA), zlib (Deflate), huff (Huffman), flac (FLAC)
CHD size:     595,749,009 bytes
Ratio:        18.5%
SHA1:         0b8dbf1c9caf4abf965dbc6e1a8e6329d48b1c90
Data SHA1:    293b5430bc4946dcf506a935957064549c28540c
Metadata:     Tag='GDDD'  Index=0  Length=35 bytes
              CYLS:6256,HEADS:16,SECS:63,BPS:512.

btw - this idea with %1 [%f] is good

and not rename chds, maybe need manually , dont know

 

OPSS, me stop here, have any problem with chds , with every start of scanning CMP wanna move chds:



if click yes, next start scaning same window !

[PS] About second shot: afetr second scan this 2 chds is missing :
cap-jjk-2.chd and cap-jjk000.chd , not inside folders not in cmp backup folder
chds is deleted !!!

« Last Edit: 27 November 2014, 05:52 by oddi »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 92
  • Offline Offline
  • Posts: 2848
  • Operating System:
  • Mac OS X Mac OS X
  • Browser:
  • Safari 8.0 Safari 8.0
    • View Profile
Re: Status
« Reply #11 on: 27 November 2014, 06:05 »

well the chd is not bad but wronlgy placed. so cmpro wants to move it...could be that there is a little hickup with fully merged sets under some circumstances...will check it in the evening....
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 266
  • Operating System:
  • Windows Server Home/Server 2003 Windows Server Home/Server 2003
  • Browser:
  • Firefox 32.0 Firefox 32.0
    • View Profile
    • .
Re: Status
« Reply #12 on: 27 November 2014, 06:57 »

Test:
Source .155 ZIP merged set
Load .156 DAT - Configure Full Merged Settings, then "New Scan"



I allowed that to continue to completion.
Now rebuilding from Back-up (took a few tries to pick up all available files in back-up folder), then Rebuilding Updates in.

Assuming the Updates were packed & prepared properly..
I should have a full set, but not so..
Miss list is interesting.. I would say I had all these in 0.155 (Example below)
Code: [Select]
18 Wheeler (set 2) [folder: 18w2 - parent: 18w - size: 8kb]
missing rom: 18w_b1.rom1 [18w2] [size: 2048] [CRC32: cbc0fb2c] [SHA1: 66b14f0d76baebbd64e8ed107e536ad811d55273]

Donkey Kong 3 (bootleg with 2xAY8910) [folder: dkong3abl - parent: dkong3 - size: 61kb]
missing rom: dk3ba-1.3l [size: 8192] [CRC32: d4a88e04] [SHA1: 4f797c25d26c1022dcf026021979ef0fbab48baf]
missing rom: dk3ba-2.3m [size: 8192] [CRC32: f71185ee] [SHA1: 6652cf958d7afa8bb8dcfded997bb418a75223d8]
missing rom: dk3ba-3.4l [size: 4096] [CRC32: 67ac65d4] [SHA1: d28bdb99310370513597ca80185ac6c56a11f63c]
missing rom: dk3ba-4.4n [size: 4096] [CRC32: 84b319d6] [SHA1: eaf160948f8cd4fecfdd909876de7cd16340885c]
missing rom: dk3ba-5.7f [size: 4096] [CRC32: 07d3fd88] [SHA1: 721f401d077e3e051672513f9df5614eeb0f6466]
missing rom: dk3ba-6.7g [size: 16384] [CRC32: 31b8401d] [SHA1: 0e3dfea0c7fe99d48c5d984c47fa746caf0879f3]
missing rom: dk3ba-7.7i [size: 16384] [CRC32: a9263275] [SHA1: c3867f6b0d379b70669b3b954e582533406db203]

Frogger (bootleg on Amigo? hardware) [folder: froggeram - parent: frogger - size: 22kb]
missing rom: 1.d2 [size: 2048] [CRC32: b680e622] [SHA1: 233dbefa2aae6e85cb61acd60c49480bd4a3388d]
missing rom: 2.e2 [size: 2048] [CRC32: 32c56a50] [SHA1: 4d215fff6ff002e23aa889292c9c5eb242975f5d]
missing rom: 3.f2 [size: 2048] [CRC32: 4223a053] [SHA1: c19555d2fee4172dff99d7cf65ebb44d1336c06e]
missing rom: 4.h2 [size: 2048] [CRC32: bcd02aa7] [SHA1: 987c35bf9af8bb1083ccbf4d9f912be8d74b3d1f]
missing rom: 5.j2 [size: 2048] [CRC32: b11b36f7] [SHA1: d4e9342be7fa23f30565d7b75fa0fb8c6c82669d]
missing rom: 6.l2 [size: 2048] [CRC32: a239048a] [SHA1: a8dcc0b4bdb51f6e391832d69ba3a8727be59ae7]
CONTINUED>>>>>

Yes, it seems those .155 ROMs were delicious, CMP & 'The Void' has consumed them  :o

Oh boy, I rebuilt some of those Missings into the set,
Scanner Results shows improvement..
But then, exporting a fix_dat ...
Ummm looks like whole .156 DAT was saved.
« Last Edit: 27 November 2014, 08:22 by oxyandy »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 92
  • Offline Offline
  • Posts: 2848
  • Operating System:
  • Mac OS X Mac OS X
  • Browser:
  • Safari 8.0 Safari 8.0
    • View Profile
Re: Status
« Reply #13 on: 27 November 2014, 07:27 »

eh?....

nothing really has changed when it comes to standard scanning or unpacking....weird

you are using full merged sets?
Logged

oddi

  • Member
  • *
  • Karma: 2
  • Offline Offline
  • Posts: 178
  • Operating System:
  • Windows NT 6.3 Windows NT 6.3
  • Browser:
  • Firefox 34.0 Firefox 34.0
    • View Profile
Re: Status
« Reply #14 on: 27 November 2014, 07:40 »

Only for info, now reverse to previous CMP , after full scan results:

This chds is forever removed, not found cmp backup folders or any other folders, scan full hhd - nothing
No problem about that - i have full backups of mame/mess sets.
Tnx :)

oddi "the braveheart" vs "monster" CMP
first round:
0 vs 1
« Last Edit: 27 November 2014, 08:07 by oddi »
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 266
  • Operating System:
  • Windows Server Home/Server 2003 Windows Server Home/Server 2003
  • Browser:
  • Firefox 32.0 Firefox 32.0
    • View Profile
    • .
Re: Status
« Reply #15 on: 27 November 2014, 07:52 »

Yes Roman,
full merged ZIP
I just made my last edit to my previous post, so I wont touch that anymore.
Maybe I will edit this one a few times more.

Instead of fix.dat whole .156 dat is saved..
Sorry to be the bearer of bad news.

.156 dat states..
      <rom name="7408.13b" size="524288" crc="1a947f3b" sha1="ad8d52de54c5a507dd759604613e1d85e13db5fd"/>
      <rom name="7408.13b_9efe4c60" merge="7408.13b" size="1048576" crc="9efe4c60" sha1="6462dca2af38517639bd2f182e68b7b1fc98a312"/>

New test CMP Build never solves this problem, it rebuilds the match (7408.13b) in..
A new scan - wants to take it out - wrong size

I could make mini dat - rom combinations for repeating bug & fixing.

« Last Edit: 27 November 2014, 08:08 by oxyandy »
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 92
  • Offline Offline
  • Posts: 2848
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 39.0.2171.71 Chrome 39.0.2171.71
    • View Profile
Re: Status
« Reply #16 on: 27 November 2014, 08:03 »

I'm at the office now...so I can't do anything before tonight...

So let's summarize:

- some weird wrong placed chd warning with removal of them (well...I have an idea about that since I've added a last minute change for wrong place chds...so that should be an easy fixing..)

- unpack errors with some removal of them (ok..that's weird)

- saving of fix dat exports all....(even weirder since that wasn't touched since ages)...


As my brave little testers know...it's a test version, use it on your own risk and only if you know how to get deleted files back ;-)

Ok..now real time work....see you in 12 hours or so....
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 266
  • Operating System:
  • Windows Server Home/Server 2003 Windows Server Home/Server 2003
  • Browser:
  • Firefox 32.0 Firefox 32.0
    • View Profile
    • .
Re: Status
« Reply #17 on: 27 November 2014, 08:50 »

CMP Stable (Merged)
\Funky CMP - MAME v0.156
funkyfig.zip
funkyfiga.zip

CMP Funky Beta (Merged)
\Funky CMP - MAME v0.156
funkyfig.zip







"New Scan", "New Scan" = Loop Forever, DAT attached
Required ROMs
funkyfig.zip
funkyfiga.zip
« Last Edit: 27 November 2014, 08:57 by oxyandy »
Logged

oxyandy

  • Member
  • *
  • Karma: 5
  • Offline Offline
  • Posts: 266
  • Operating System:
  • Windows Server Home/Server 2003 Windows Server Home/Server 2003
  • Browser:
  • Firefox 32.0 Firefox 32.0
    • View Profile
    • .
Re: Status
« Reply #18 on: 27 November 2014, 09:06 »

Hey Roman,
You knows me makes easy 4 u to find/repeat bug..
Enjoy DAT, fix this bug first, other not reported bug vanish... haha
Dont u luvs me Engrish.
Later
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 92
  • Offline Offline
  • Posts: 2848
  • Operating System:
  • Windows 7/Server 2008 R2 Windows 7/Server 2008 R2
  • Browser:
  • Chrome 39.0.2171.71 Chrome 39.0.2171.71
    • View Profile
Re: Status
« Reply #19 on: 27 November 2014, 09:10 »

you're killing me
Logged
Pages: [1] 2 3 4   Go Up
 

Page created in 0.101 seconds with 22 queries.

anything
anything