EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: Hash collision (Bug?)  (Read 7292 times)

yeahimdukenukem

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 20
  • Operating System:
  • Mac OS X 10.13 Mac OS X 10.13
  • Browser:
  • Firefox 61.0 Firefox 61.0
    • View Profile
Hash collision (Bug?)
« on: 07 August 2018, 07:39 »

Hi Roman,

I don't know if this is a bug or not, maybe I don't understand correctly.

Let me try to explain:
I have a problem on the newest HBMAME 0.200 dat.

In full merge mode, when detecting a hash collision, I assume the correct behaviour is to move the collided roms into subfolders with the name of the corresponding clone set. Is that right?

A hash collision is detected when there are two or more roms with the same name, but different CRCs, is this correct?

In this dat now I have two roms with the same name, same size AND same CRCs, but only different SHA-1 values. In this case it seems Clrmamepro does not detect the collision.
Am I understanding something wrong? Please correct me.

I can provide the roms for testing purposes if you need them, these are:
Code: [Select]
kov2p204s08/v204-32m-p08.rom
kov2p204s50/v204-32m-p08.rom
and
Code: [Select]
sfiii3nrs02/sfiii3s03-simm1.0
sfiii3nrs02/sfiii3s03-simm1.1
sfiii3nrs02/sfiii3s03-simm1.3
sfiii3nrs03/sfiii3s03-simm1.0
sfiii3nrs03/sfiii3s03-simm1.1
sfiii3nrs03/sfiii3s03-simm1.3

Thanks and best regards,
Duke
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3414
  • Operating System:
  • Mac OS X Mac OS X
  • Browser:
  • Safari 11.0 Safari 11.0
    • View Profile
Re: Hash collision (Bug?)
« Reply #1 on: 07 August 2018, 07:52 »

Your assumptions are all correct. Equal size and equal crc33 and different sha1 are rare but possible. Could be that the hash collision check only looks for crc33. Since Im on holiday I cant check it before mid of August. Please be patient till then. i will definetly have a look at it. Files could be useful. i will come back to you.
Logged

yeahimdukenukem

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 20
  • Operating System:
  • Mac OS X 10.13 Mac OS X 10.13
  • Browser:
  • Firefox 61.0 Firefox 61.0
    • View Profile
Re: Hash collision (Bug?)
« Reply #2 on: 08 August 2018, 05:56 »

First enjoy your holidays and have fun :-D
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3414
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 68.0.3440.106 Chrome 68.0.3440.106
    • View Profile
Re: Hash collision (Bug?)
« Reply #3 on: 18 August 2018, 15:50 »

grab the latest nightly and retest please (you need to press profiler->clear cache before loading the profile though)
Logged

yeahimdukenukem

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 20
  • Operating System:
  • Mac OS X 10.13 Mac OS X 10.13
  • Browser:
  • Firefox 61.0 Firefox 61.0
    • View Profile
Re: Hash collision (Bug?)
« Reply #4 on: 20 August 2018, 11:46 »

It took very long to retest this, but I can confirm that it seems to work now.

Thank you man, you are the best. =)
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3414
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 68.0.3440.106 Chrome 68.0.3440.106
    • View Profile
Re: Hash collision (Bug?)
« Reply #5 on: 20 August 2018, 11:54 »

long in terms of "finding a way to test it" or long in terms of "something is slower than before"

Actually you can quickly check it by opening the setinformation dialog, go the the set in question and check if the filename contains the setname now ;-)
Logged

yeahimdukenukem

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 20
  • Operating System:
  • Mac OS X 10.13 Mac OS X 10.13
  • Browser:
  • Firefox 61.0 Firefox 61.0
    • View Profile
Re: Hash collision (Bug?)
« Reply #6 on: 21 August 2018, 14:27 »

It just took a bit longer to run through the whole scan. It's not slower than before.
I just tried to verify that it works, without scanning the whole HBMAME dat again, because in the current version 0.200 it took more than 24 hours to scan ;)

But anyways thanks for the great hint, I haven't been aware of that dialog. I have been looking for a tool to explore sets for so long and it was there all the time :-D
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 120
  • Offline Offline
  • Posts: 3414
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 68.0.3440.106 Chrome 68.0.3440.106
    • View Profile
Re: Hash collision (Bug?)
« Reply #7 on: 21 August 2018, 14:32 »

24 hours ? I mean HBMAME is not "that" big and even scanning a full mame collection with sha1 checks enabled etc is done within minutes on a modern system........

The change is actually only called during importing the data, so running speed should not change at all....
« Last Edit: 21 August 2018, 14:32 by Roman »
Logged

yeahimdukenukem

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 20
  • Operating System:
  • Mac OS X 10.13 Mac OS X 10.13
  • Browser:
  • Firefox 61.0 Firefox 61.0
    • View Profile
Re: Hash collision (Bug?)
« Reply #8 on: 23 August 2018, 09:09 »

The running speed has not changed with your changes in my opinion.
HBMAME really has grown A LOT in the last few releases and scanning it as merged set really takes 5 times as long as before (or even longer). But I don't think this is related to Clrmamepro.
Logged
Pages: [1]   Go Up
 

Page created in 0.09 seconds with 21 queries.