EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Pages: [1]   Go Down

Author Topic: Dates don't match  (Read 3983 times)

coccola

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 54
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Firefox 84.0 Firefox 84.0
    • View Profile
Dates don't match
« on: 09 January 2021, 02:30 »

Mess with dates. Used clrmamepro version 4.040 with all fix options enabled. Zip attached.

Original dat: 2010-04-06
Rebuilt zip: 1980-01-01
Fix-dat: 1970-01-01
Logged


Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 87.0.4280.141 Chrome 87.0.4280.141
    • View Profile
Re: Dates don't match
« Reply #1 on: 09 January 2021, 08:42 »

If you create uncompressed files you see on commandline level that the date is correctly set.

Archives (rar/zip etc) store dates in standard MS-DOS format which only allows dates from 1980 onwards....

Bits 00-04: day
Bits 05-08: month
Bits 09-15: years from 1980


So...everything is fine
Logged

coccola

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 54
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Firefox 84.0 Firefox 84.0
    • View Profile
Re: Dates don't match
« Reply #2 on: 09 January 2021, 18:05 »

OK, then. But what about the fix-dat? Why the dates become 1970-01-01?
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 87.0.4280.141 Chrome 87.0.4280.141
    • View Profile
Re: Dates don't match
« Reply #3 on: 09 January 2021, 18:22 »

Erm....you're mixing 2 things.....file definitions and actual filesystem dates.

Your datfile defines the files as 1970-01-01 00:00:00. When scanning/fixing decompressed files on an NTFS filesystem, cmpro will fix the last modified date of the files. Windows won't show it correctly though but on commandline level it is shown as 1970-01-01.

Then you got archivers like rar/zip. The rar/zip format uses MSDOS date format which means the earliest date is 01/01/1980.

And if you create a fix datfile cmpro simply dumps the datfile again but limited to your missing files.
« Last Edit: 09 January 2021, 18:22 by Roman »
Logged

coccola

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 54
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Firefox 84.0 Firefox 84.0
    • View Profile
Re: Dates don't match
« Reply #4 on: 09 January 2021, 18:40 »

So the files end up with date 1980-01-01 because that is the closest to 1970-01-01 that can be achieved. I understand that.

But the dates should actually be 2010-04-06, not 1980-01-01 or 1970-01-01.

Also, fix-dats should only contain files that I don't have in my hard disk according to the original dat. It has nothing to do with the properties of any files on my hard disk. So, if my original dat has files with date 2010-04-06 and I'm missing them, the date on the fix-dat should be 2010-04-06. I don't have these files, so how would the missing files have a different date, on this case 1970-01-01? Where does 1970-01-01 comes from? It doesn't make any sense.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 87.0.4280.141 Chrome 87.0.4280.141
    • View Profile
Re: Dates don't match
« Reply #5 on: 09 January 2021, 19:54 »

Ah sorry...... I falsely looked the fix dat which does list 1970 and for 1970 all my comments are correct. Your original dat indeed lists 2010-04-06 which is not parsed correctly and it is reset to 1970 since it's not in correct format. It's being reset since the datetimestamp is missing. The date attribute should hold a date time string... YYYY-MM-DD HH:MM:SS.... or YYYYMMDDTHHMMSS

But I guess I can support a YYYY-MM-DD, too


Indeed I can.....
https://mamedev.emulab.it/clrmamepro/binaries/cmpro20210109.7z
« Last Edit: 09 January 2021, 20:31 by Roman »
Logged

coccola

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 54
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Firefox 84.0 Firefox 84.0
    • View Profile
Re: Dates don't match
« Reply #6 on: 09 January 2021, 20:43 »

Thank you very much!
Logged

coccola

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 54
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Firefox 84.0 Firefox 84.0
    • View Profile
Re: Dates don't match
« Reply #7 on: 10 January 2021, 19:39 »

I'm using the latest nightly build.

When I rebuild and then scan, the scanner fixes the files I've just rebuilt. When I re-scan everything is OK.

It looks like there is some inconsistency between rebuilder and scanner.
Logged

coccola

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 54
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Firefox 84.0 Firefox 84.0
    • View Profile
Re: Dates don't match
« Reply #8 on: 10 January 2021, 19:52 »

Apparently the rebuilder is not adding files with the right date, which ends up being fixed by the scanner.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 87.0.4280.141 Chrome 87.0.4280.141
    • View Profile
Re: Dates don't match
« Reply #9 on: 10 January 2021, 19:55 »

It does, the problem is your filesystem which applies local timezones...so e.g. for me setting 00:00:00 will end up with 02:00:00 due to the local offset.
I'm working on it...
Logged

coccola

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 54
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Firefox 84.0 Firefox 84.0
    • View Profile
Re: Dates don't match
« Reply #10 on: 10 January 2021, 20:03 »

Thank you for the information.
Logged

Roman

  • Global Moderator
  • Member
  • ***
  • Karma: 112
  • Offline Offline
  • Posts: 3287
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Chrome 87.0.4280.141 Chrome 87.0.4280.141
    • View Profile
Re: Dates don't match
« Reply #11 on: 11 January 2021, 08:34 »

Actually the latest nightly works like this:

- Rebuilding to zip with recompress enable, creates the right date from the dat
- Rebuilding to a decompressed file on NTFS, rar and 7z creates a file where the offset to UTC is included..

- Scanning/fixing a zipfile works fine
- Scanning/fixing decompressed/rar/7z will create a fixing loop due to the UTC offset

I love it...*sigh*...working on the decompressed/7z/rar time handling.....till then..use zip :)


ok...tried to handle decompressed files now as UTC based (which also is the reason for 7z/rar)

https://mamedev.emulab.it/clrmamepro/binaries/cmpro20210111.7z


This should fix the upper stuff...again I find it hard to say what is right if we talk about preservation...and especially if you look at it....you see the dat with 00:00:00 and you look at the archive or the file and you see an offset based value......something to think about when I got more time...but at least you shouldn't run into scan time fixing loops.

« Last Edit: 11 January 2021, 19:35 by Roman »
Logged

coccola

  • Member
  • *
  • Karma: 0
  • Offline Offline
  • Posts: 54
  • Operating System:
  • Windows NT 10.0 Windows NT 10.0
  • Browser:
  • Firefox 84.0 Firefox 84.0
    • View Profile
Re: Dates don't match
« Reply #12 on: 12 January 2021, 05:29 »

Thank you very much for your time and patience!
Logged
Pages: [1]   Go Up
 

Page created in 0.159 seconds with 20 queries.

anything
anything