Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[haynor666]

Most recent update fixes those lzma warnings. Thanks Roman

[Roman]

you are welcome....see sometimes low prio things can come more quickl as expected

[haynor666]

Since so much has been changed in recent versions time to test rebuild, scanner etc.

[GeoMan]

I have a perfect 152 set (7z-merged) checked with a previous ClrMame using the external 7zip.exe but when i replaced the mame64.exe with the 153 version and run ClrMame 4.014a i noticed delays and many errors of the type "romxxx.7z can't be read".

Actually the roms reported by ClrMame can't be opened now. As i store the roms on a Raid 1, i checked the raid and everything reports fine.

Is it possible a bug in the 7z implementation in latest ClrMame to cause rom corruption?

[oxyandy]

latest ClrMame to cause rom corruption

Nope no way, there would hundreds of other reports, not just you.
I assume you were using CMP to "Scan & Fix" this 152 set to 153 ?
(That's when you noticed the errors)

In "Compressor" settings CMP has a "Test archive(decompress to memory)(Scanner Only)" check box.
If you didn't actually ever run this test (not that I am saying people should ever use this all the time)
for the whole set, it may well be there were some corrupt archives already.
Is this a set you actually made completely using a previous CMP version.
Or just a downloaded 'as is' set ?

How many corrupt archives are we talking about.?
Plus there is corrupt (archive can not even display contents)
and corrupt (archive can display contents, but reports error when you try extract/modify contents)
Which type of corruption are you seeing ?

[Roman]

as oxyandy points out....can you repeat this with lets say one file, if yes send me the file plus your cmpro config (cmpro.ini and .cmp file of your used profile)...

[Roman]

and on a sidenote, yes of course things have changed....now the 7z dll is required and it does some calls via com system (create objects etc)....and if such a call fails it usually returns a "cannot open" error. I don't remember seeing or adding a "can't be read" error message though, so I expect it comes from 7z (dll/exe) itself....

would be interesting to see such a file.....

[haynor666]

I've made rebuild two times using MAME 153 as a base dat. Both pass fine with word 256 and dictionary 256m using LZMA2. One pass took about 6,5 h on i5 4670k

Also fast scans (just under 5 mins) and full scans with chd, 7zip integrity, CRC32, MD5 (about 45 mins) are fine.

[Roman]

yeah....I guess it's not a general 7z failure....sounds a bit more 3rd party 7z ( no I don't say the name here) may have a problem.....unless I got a file handy I can't say...or do tests...

[GeoMan]

I had setup 152 with a previous version of ClrMame configured to use Torrent7Z (t7z.exe). When i updated to latest ClrMame i forgot to change it to the included 7z.exe.

Maybe that caused problems with 153? I changed to the included 7z.exe, fixed the damaged roms and i am now checking them with full integrity check. So far no problems (it runs for 13 hours on my i7 [email protected] )...

[Roman]

so I was right with " no I don't say the name here)"

[haynor666]

t7z is ok but only for recompressing for "specific" task. Using it as main compress does not make sense. Better compress with zip then recompress with t7z.

[oxyandy]

GeoMan you may sense something going here.

The times you claim seem overly extended.
As I pointed out earlier.

"Test archive(decompress to memory)(Scanner Only)"

I know exactly what this does & how it works.
Think I have used it maybe 'once' across a whole MAME set, because I had a reason to.

Now the taboo word 't7z.exe'
There is nothing wrong with keeping sets in this format, but it really does require the correct 7z command line version & the correct strings.
(Never set t7z.exe as the program in CMP)

Doing a complete Rebuild from ZIP to 7z, regardless of cmd strings is dumb.
Regardless of the fact that some well know file sharing sites have advertised this way is the BEST.
They are wrong, well for MAME anyway, I aint going to write about all I understand here.

Keeping a set 'maintained' once in t7z with CMP is easy.
But again requires the correct 'technique' searching my posts around here, will find you 'some info'

Question time, your cmd line version of 7z & strings ?

[haynor666]

For better space savings 7z is the best but you get slower rom loading in mame for example c1k or neogeo  bigger roms. Also rebuild,fix, scan processes are longer. I have 3 TB disk so I have plenty of free space but this set is also used in my cab where I have only 250 gb disk so I prefer to keep set in 7z.

Please note that clrmamepro does not build solid archives. After t7z archives even smaller.

From my experience build clean set using zip then using t7z for "special" purpurose is faster than rebuild 7z and then again t7z.

[oxyandy]

build clean set using zip then using t7z for "special" purpose is faster than rebuild 7z and then again t7z.

Yes, of course this is what I was pointing out..
Here's a bit of why.
When CMP Rebuilder finds a source ZIP archive has 10 internal files, even if all those 10 files are a match to the dat.
When Rebuilding to 7z - CMP will do this, 'file by file' 10 times for each file found matching the DAT.
1. Make a 7z with 1st found in the ZIP = result a 7z archive with 1 file.
Done
2. Make new 7z with 2nd file found in the ZIP
To do this CMP adds an extra step, extract all the contents from the first 7z archive it made,
takes that 2nd 'found' file in source ZIP, then make 'new' 7z archive with 2 files.
Done
3. Make new 7z with 3rd file found in the ZIP
CMP extracts contents of 2nd 7z archive made, takes the 3rd 'found' file in source ZIP = make 'new' 7z archive with 3 files.
Done.
And well, that's the pattern, 10 freaking times.

Rebuilder - Source archive ZIP - Destination ZIP
Extract 1st from ZIP, make 'new' ZIP with 1 file.
Extract 2nd from ZIP, add to existing ZIP now 2 files, and so on.
CMP can simply ADD to the destination ZIP file, look at all the extra work that gets cut out.

One day, if lucky enough Roman may just be kind enough to add a read ahead optimisation to CMP.
When Rebuilding - CMP will do this, 'per set' once.
Source file ZIP, 11 internal files - Destination 7z 10 files.
Read source ZIP.
File 1,2,3,4,5,6,7,8,9,10 match dat & 11 no match, end of archive ?
OK, time to build archive = results logged.
Now CMP will build a 7z once with all 10 files.
Remove Source Ticked ? OK, delete 10 from source ZIP = remainder 1

Right enough Roman ?


EDIT: Oh and as you probably know extracting a 'single file' from a solid 7z is a load of extra work too !
So a bit of 'read ahead' would help when source is 7z too

[oxyandy]

Actually I want to edit but think new post is more relevant: I want to make a correction.

When Rebuilding - CMP will do this, 'per set' once.

Now this is a bit tricky in true definition of 'per set'
To do that CMP would have to read & log entire contents of source folder before it built a single archive.
(not impossible, but definitely a bit more tricky)
So a better statement would be instead of 'per set' would be 'per source archive'
That would still give results way better than we have now.
And yeah Roman has a 'life' & we have a suitable work around already.
But we can dream right  haha

[Roman]

Yeah....I got a life, wife, 2 sons and a real life job....which take all/most of my time...actually these days I'm thinking about switching to Windows 8.1 (yikes...do I really have to buy that thing....I wanted to skip it after Windows 7), Visual Studio 2013 Express and start all over with a C++/stl/boost lib based, no features, mame only version....keeping the DOM data in memory using xpath to access them....and doing rebuilder/scanner completely different....way more faster (due to not supporting each and any special thing or old legacy stuff)

...but maybe this is just Friday morning dreaming.....