Who's Afraid Of 2038?

Slideshow Transcript

1. Slide 1: Who's Afraid Of 2038? Michael G Schwern schwern@pobox.com
2. Slide 2: or
3. Slide 3: Saving The Future From The Past
4. Slide 4: Hi I'm Michael Schwern
5. Slide 5: #!/usr/bin/perl -w use Test::More tests => 2; is( $testing => \"easy\" ); Wrote Test::More
6. Slide 6: perl Makefile.PL make make test make install Maintain MakeMaker
7. Slide 7: package Films; use base qw(Class::DBI); Films->table('movies'); Wrote Class::DBI
8. Slide 8: Ask. Please ask questions. It's a long talk.
9. Slide 9: schwern@pobox.com
10. Slide 10: Y2K? Remember y2k?
11. Slide 11: When the whole world was going to explode? And we'd be living in our y2k shelters
12. Slide 12: Dogs & cats, living together. Mass hysteria! y2k came and it didn't happen.
13. Slide 13: 2008 So here it is in 2008 and we're home free, right?
14. Slide 14: If I can't have my flying car, I should at least be able to have a computer that knows what day it is.
15. Slide 15: J.R. Stockton's \"Critical and Significant Dates\" http://www.merlyn.demon.co.uk/critdate.htm Interesting read. Which is not about...
16. Slide 16: ...your dating history, but time and date values in the past and future, from...
17. Slide 17: 13,700,000,000 BC The beginning of the Universe, generally considered to be a bad idea
18. Slide 18: 13,700,000,000 BC (around teatime) The beginning of the Universe, generally considered to be a bad idea
19. Slide 19: 1970-01-01 Thu 00:00:00 GMT To the beginning of the Unix epoch Unix counts the number of seconds since 1970.
20. Slide 20: 1970-01-01 Thu 00:00:00 GMT (Unix epoch) To the beginning of the Unix epoch Unix counts the number of seconds since 1970.
21. Slide 21: 600,000,000,000 AD Text To the end of the world And all the crazy exceptions in between.
22. Slide 22: 600,000,000,000 AD (Tower of Hanoi completed) Text To the end of the world And all the crazy exceptions in between.
23. Slide 23: 600,000,000,000 AD (Tower of Hanoi completed) Text (just after Perl 6.0.0) To the end of the world And all the crazy exceptions in between.
24. Slide 24: So what's the problem? It turns out that in dating...
25. Slide 25: Size matters
26. Slide 26: $time = 2**31-1; print scalar gmtime($time);
27. Slide 27: $time = 2**31-1; print scalar gmtime($time); Tue Jan 19 03:14:07 2038
28. Slide 28: $time += 1; print scalar gmtime($time); Uh oh We've gone from...
29. Slide 29: $time += 1; print scalar gmtime($time); Fri Dec 13 20:45:52 1901 Uh oh We've gone from...
30. Slide 31: No car Normally Perl lets us forget that we're running software on actual silicon But hardware has limits. And this is because of...
31. Slide 32: ...or whatever the C folks say. Perl is supposed to shield us from all that. But sometimes it doesn't.
32. Slide 33: 32 bit signed double floating point static registers ...or whatever the C folks say. Perl is supposed to shield us from all that. But sometimes it doesn't.
33. Slide 34: 32 bit signed integer 31 bits for the integer 1 bit for the sign
34. Slide 35: 2**31 == 2,147,483,648 -2**31 == -2,147,483,648
35. Slide 36: (2**31)-1 == 2,147,483,647 -2**31 == -2,147,483,648
36. Slide 37: 2038-01-19 Tue 03:14:07 2**31 seconds in 2038, 32 bit signed Unix time fails
37. Slide 38: 292,277,026,596-12-04 Sun 15:30:08 2**63 seconds Y 292 billion 64 bit signed Unix time fails
38. Slide 39: 5,391,559,471,918,239,497,011,222,876,596-04-18 Mon 16:02:08 2**127 seconds 128 bit fails at the year 5 non-illions
39. Slide 40: 1,834,652,618,499,343,590,337,415,746,119,712,509,834,124,421,548,072,260,582,352,567,003,896-01-25 Sat 17:06:08 2**255 seconds 256 bit at the year 1 million vig-int-illion
40. Slide 41: 2**1e80 years There are only 10 to the 80 particles in the universe So in year 2 to the 10 to the 80th It is now impossible to write the year.
41. Slide 42: Oddly enough, there was an argument about what to do with 2038 on p5p. (Imagine that) The C programmers on p5p said that...
42. Slide 43: gmtime() uses time_t (internally)
43. Slide 44: time_t is a signed 32 bit integer! (on a 32 bit machine)
44. Slide 45: signed 32 bit numbers overflow at 2**31!
45. Slide 46: 2147483647 + 1 Text Of course. Therefore...
46. Slide 47: 2147483647 + 1 Text == Of course. Therefore...
47. Slide 48: 2147483647 + 1 Text == -2147483648 Of course. Therefore...
48. Slide 49: $ perl -wle 'print scalar gmtime(2**31)' Fri Dec 13 20:45:52 1901 ...you got exactly what you asked for. And BTW, you should be using a 64 bit clean machine.
49. Slide 50: But this is Perl. And Perl is supposed to...
50. Slide 51: DWIM And I don't care about...
51. Slide 52: More importantly, my clients don't care. When they get ...
52. Slide 53: 32 bit signed double floating point static registers More importantly, my clients don't care. When they get ...
53. Slide 54: ...1901 I can't say \"Oh, that's because of...\"
54. Slide 55: ...and expect them to nod knowingly. No, they'll say...
55. Slide 56: 32 bit signed double floating point static registers ...and expect them to nod knowingly. No, they'll say...
56. Slide 57: ...you're FIRED!
57. Slide 58: A number is a number. (Perl is a closet Ayn Rand fan) I don't care how it's represented internally Perl gives no indication how it's represented internally. There's no reason a Perl programmer can suspect there's any limits. So we have to fix this.
58. Slide 59: Why do you care?
59. Slide 60: 29 years There's less than 30 years to go! In 30 years, I'll be eligible for the...
60. Slide 61: ...I can only hope I look that good in 30 years. But...
61. Slide 62: Stuff happens in the future This might seem obvious, but people seem to pretend it doesn't.
62. Slide 63: How many people have a 30 year mortgage? Perhaps you want to construct..
63. Slide 64: ...some sort of death clock. I plan on living past 2038.
64. Slide 65: Stuff happens in the past. Like... photographs!
65. Slide 66: Pittsburgh, 1941
66. Slide 67: September 29, 1927
67. Slide 68: 1938
68. Slide 69: 1941, arsenal of democracy Apparently Picassa doesn't do anything before 1970!
69. Slide 70: April 20th, 1975 167,220,000 seconds The most important date in my life. I was born. Fortunately after the Unix epoch. Some people were born before 1970. Some systems can't handle negative times. Windows can't.
70. Slide 71: 5.6.1 2001-Apr-08 How many people here are still using (or know someone using) 5.6? Seven years from now, people will still be using 5.10. They will still be using 32 bit machines. Or 64 bit machines with 32 bit time_t.
71. Slide 72: 5.6.1 2001-Apr-08 (Seven Years Ago) How many people here are still using (or know someone using) 5.6? Seven years from now, people will still be using 5.10. They will still be using 32 bit machines. Or 64 bit machines with 32 bit time_t.
72. Slide 73: 29 - 7 == Let's do the math... In 2015, with 22 years to go, people will likely still be using 5.10.0 How many people have a 20 year mortgage? So we'd better fix this NOW
73. Slide 74: 29 - 7 == 22 Let's do the math... In 2015, with 22 years to go, people will likely still be using 5.10.0 How many people have a 20 year mortgage? So we'd better fix this NOW
74. Slide 75: 29 - 7 == 22 (2015) Let's do the math... In 2015, with 22 years to go, people will likely still be using 5.10.0 How many people have a 20 year mortgage? So we'd better fix this NOW
75. Slide 76: So that's the problem.
76. Slide 77: Solution? Before we look at the solution
77. Slide 78: !Solution Let's look at what's not a solution.
78. Slide 79: Blame the user! No, only C programmers think this is a feature.
79. Slide 80: We'll all upgrade to 64 bit! And if you use Linux, and use a 64 bit distribution, that's fine. How many Macs out there? New Macs use 64 bit processors. They do not use a 64 bit time_t. Why?
80. Slide 81: 00000001 1 7FFFFFFF 2147483647 80000000 -2147483648 32bits x 3 == 96 bits Computers store numbers in a fixed size. Storing 3 32 bit numbers is 96 bits
81. Slide 82: 0000000000000001 1 000000007FFFFFFF 2147483647 7FFFFFFFFFFFFFFF 9223372036854775807 64bits x 3 == 192 bits (the embarassing part is I had to calculate that in Ruby) But 3 64 bit integers is 192 bits This means the same binary program can't run on a 32 and a 64 bit machine. Linux users are used to recompiling and multiple distributions Apple and Windows users aren't. Apple has clever ways to deal with this Did it for Motorola -> PPPC, PPC -> Intel but they didn't think it's worth the effort Why?
82. Slide 83: Windows tried it. Most PCs these days use a 64 bit processor. Nobody uses 64 bit Windows People are really bad about keeping their types straight Assume that an integer is 32 bit and such.
83. Slide 84: And, of course, not everyone is going to upgrade their hardware. Really what \"let's all upgrade to 64 bit\" is saying is...
84. Slide 87: Don't use the built-ins, use DateTime! Yes, DateTime is fantastic, but a lot of things depend on the built-ins. And a lot of other languages and projects depend on localtime and gmtime C, Python, Perl and Ruby all make use of it. And leaving the built-ins broken leaves Perl with no built in safe date handling. We could suck in DateTime, but there's a problem with that I'll get to in a moment. This is special case of the...
85. Slide 88: Just use <insert library here> libtai is one It's fast It's reliable It avoids the 2038 problem... It doesn't do time zones.
86. Slide 89: Time zones are important Time zones are insane Time zones change Oh, and there's daylight savings time All this information is in your operating system's time zone database. It gets updated along with the operating system. There is no portable API to the OS' time zone database. Things like DateTime ship their own time zone database. But that's a lot of work. And now you have two time zone databases to keep up to date. (That you probably don't know exist)
87. Slide 90: # From Paul Eggert (2001-03-06): # Daylight Saving Time was first # suggested as a joke by Benjamin Franklin # in his whimsical essay ``An Economical # Project for Diminishing the Cost # of Light'' published in the Journal # de Paris (1784-04-26).
88. Slide 91: ENERGY POLICY ACT OF 2005
89. Slide 92: (b) Effective Date.--Subsection (a) shall take effect 1 year after the date of enactment of this Act or March 1, 2007, whichever is later. So we had about a year to adjust all the computers to match. Probably cost far more money and energy than it saved.
90. Slide 93: SEC. 110. DAYLIGHT SAVINGS. (a) Amendment.--Section 3(a) of the Uniform Time Act of 1966 (15 U.S.C. 260a(a)) is amended-- (1) by striking ``first Sunday of April'' and inserting ``second Sunday of March''; and (2) by striking ``last Sunday of October'' and inserting ``first Sunday of November''.
91. Slide 94: (c) Report to Congress.--Not later than 9 months after the effective date stated in subsection (b), the Secretary shall report to Congress on the impact of this section on energy consumption in the United States.
92. Slide 95: (d) Right to Revert.--Congress retains the right to revert the Daylight Saving Time back to the 2005 time schedules once the Department study is complete. So we might have to put it right back!
93. Slide 96: A) Handle time zones b) Use the system tz database 3) Be portable (ANSI C 89) iv) Compatible license w/Perl É) localtime/gmtime compatible Nothing exists.
94. Slide 97: A) Handle time zones b) Use the system tz database GIVEN No tz database API This seems impossible. Well, I lied. There is, sort of.
95. Slide 98: date = localtime(time); time = mktime(date); ANSI C 89 has two functions which talk to the time zone database. This is the wedge we'll use mktime() is the inverse of localtime()
96. Slide 99: How to do it?
97. Slide 100: Step 1: Write 64 bit gmtime() And that's relatively easy. It's just a bunch of math. Perl already detects a native 64 bit integer type (Quad_t). \"Easy\" because writing ANSI C 89 is hard. But if strings aren't involved it's fine.
98. Slide 101: Step 2: ... Before we get into step 2 Let's talk about calendars.
99. Slide 102: The calendar we all think we use is the Julian Calendar, introduced in the west in 46 BC by Julius Caesar It estimates that a year is 365 1/4 days. Thus a leap year every 4 years. This is slightly wrong, but we'll get to that. Looking at on a calendar, there are two important attributes.
100. Slide 103: 1. What week day is Jan 1st?
101. Slide 104: 365 % 7 == 1 2002 Tuesday +1 2003 Wednesday +1 2004 Thursday
102. Slide 105: 366 % 7 == 2 2004 Thursday +2 2005 Saturday +1 2006 Sunday
103. Slide 106: 2. Is it a leap year?
104. Slide 107: 7 possible starting days Leap year every 4 years 7 x 4 = 28 28 year Julian cycle
105. Slide 108: | 0 | 2 | 3 | 4 | | 5 | 0 | 1 | 2 | | 3 | 5 | 6 | 0 | | 1 | 3 | 4 | 5 | | 6 | 1 | 2 | 3 | | 4 | 6 | 0 | 1 | | 2 | 4 | 5 | 6 | Given any year on the Julian calendar We can add or subtract 28 years to get the same calendar year Ah ha!
106. Slide 109: 2038 -> 2010 We can map future dates which localtime() can't handle back to earlier ones
107. Slide 110: 1901 -> 1985 And we can map past dates to ones inside localtime's safe range. 1971 - 2037 is safe
108. Slide 111: But, of course, we don't use the Julian Calendar Else we'd be 2 weeks ahead of the sun.
109. Slide 112: We use the Gregorian calendar This has more complex leap year rules. It changes the rules for centuries. 400 year cycle.
110. Slide 113: 2038 - 1970 < 400 years localtime() doesn't have 400 years to do the mapping. But we can fake it!
111. Slide 114: | 0 | 2 | 3 | 4 | | 5 | 0 | 1 | 2 | | 3 | 5 | 6 | 0 | | 1 | 3 | 4 | 5 | | 6 | 1 | 2 | 3 | | 4 | 6 | 0 | 1 | | 2 | 4 | 5 | 6 | Taking another look at the Julian cycle
112. Slide 115: 2099 | 0 | 2 | 3 | 4 | | 5 | 0 | 1 | 2 | | 3 | 5 | 6 | 0 | | 1 | 3 | 4 | 5 | | 6 | 1 | 2 | 3 | | 4 | 6 | 0 | 1 | | 2 | 4 | 5 | 6 | Here's 2099
113. Slide 116: 2100 | 0 | 2 | 3 | 4 | | 5 | 0 | 1 | 2 | | 3 | 5 | 6 | 0 | | 1 | 3 | 4 | 5 | | 6 | 1 | 2 | 3 | | 4 | 6 | 0 | 1 | | 2 | 4 | 5 | 6 | 2100 (not a leap year)
114. Slide 117: 2101? | 0 | 2 | 3 | 4 | | 5 | 0 | 1 | 2 | | 3 | 5 | 6 | 0 | | 1 | 3 | 4 | 5 | | 6 | 1 | 2 | 3 | | 4 | 6 | 0 | 1 | | 2 | 4 | 5 | 6 | Where does 2101 go? Non-leap years only add 1. 5+1=6
115. Slide 118: 2101? | 0 | 2 | 3 | 4 | | 5 | 0 | 1 | 2 | | 3 | 5 | 6 | 0 | | 1 | 3 | 4 | 5 | | 6 | 1 | 2 | 3 | | 4 | 6 | 0 | 1 | | 2 | 4 | 5 | 6 | There it is.
116. Slide 119: | 0 | 2 | 3 | 4 | | 5 | 0 | 1 | 2 | | 3 | 5 | 6 | 0 | | 1 | 3 | 4 | 5 | | 6 | 1 | 2 | 3 | | 4 | 6 | 0 | 1 | | 2 | 4 | 5 | 6 | It turns out you just add 16 to the year to adjust for exceptional centuries There's a little more numerology, but that's the hack.
117. Slide 120: # Get date in GMT $date = gmtime64($time); my $real_year = $date->{year}; # Map GMT year to 1970-2038 $date->{year} = safe_year($real_year); # Get the epoch time for that my $safe_time = timegm($date); # Then the localized date $safe_date = localtime($safe_time); # Map the date forward again $safe_date->{year} = $real_year; That's the basic algorithm. There's some issues around the new year. gmtime Jan 1st might be Dec 31 in the local zone
118. Slide 121: How accurate is this? Calculating future local times is always dicey, the rules might change. Past local times will become less accurate, but at a certain point the time zone simply no longer exists. What's does Eastern Standard Time mean in 1589? The real danger is mapping near future dates to a point just before the rules changed. As we get closer to 2038 the problem will become more acute. So at worst you'll be off by an hour or two.
119. Slide 122: $ perl -wle 'print scalar gmtime 2**31-1' Tue Jan 19 03:14:07 2038 $ perl -wle 'print scalar gmtime 2**31' Fri Dec 13 20:45:52 1901 This is better than being off by 138 years.
120. Slide 123: #include \"time64.h\" Rather than just patch it in Perl I've decided to write a portable 64 bit clean POSIX time.h
121. Slide 124: http://y2038.googlecode.com/ You can get it here. By making it generic, now we can fix...
122. Slide 125: C
123. Slide 126: Perl localtime() gmtime() (Time::Local) timelocal() timegm() This is being patched into bleadperl You can get the patch from y2038
124. Slide 127: Python time module date.timestamp
125. Slide 128: Ruby Time.at Time.utc Time.local
126. Slide 129: bleadperl? But I want it NOW!
127. Slide 130: Time::Local::Extended Older module Did a similar hack with localtime, but without the careful year mapping (just subtracted 60 years). Hopefully the patched version will be released soon. There's a patch on y2038. You can use that to make your Perl code y2038 safe.
128. Slide 131: use Time::Local::Extended; print scalar gmtime(2**31); print scalar localtime(2**31); # Tue Jan 19 03:14:08 2038 # Mon Jan 18 22:14:08 2038 Does timelocal() and timegm() too. How far does it go? For practical purposes, it can do about 142 million years accurately.
129. Slide 132: use Time::Local::Extended; print scalar localtime 2**52; print scalar localtime -2**52; # Fri Dec 5 22:48:16 142715360 # Mon Jan 25 15:11:44 -142711421 This is because Perl has no portable 64 bit integer scalar type. (I hope to fix that) So it has to use double floats which are accurate to 2**52. After that it loses accuracy. How far can it go?
130. Slide 133: $ ./perl -wle 'print scalar gmtime 2**63-513' Sun Dec 4 15:13:04 292277026596 That -513 is because of floating point error.
131. Slide 134: 292,277,026,596 292 billion
132. Slide 135: 13.7 billion years (age of the universe) So it gives us some room to work.
133. Slide 136: Once you break the 2038 barrier you start to run into new problems.
134. Slide 137: date.tm_year // integer The struct in which C stores the year is defined to be an integer. This leads to the...
135. Slide 138: y2,147,483,648 bug y2 billion bug Have to be careful to store the year as a 64 bit int. Many 64 operating systems miss this. y2038 defines its own Year type to help with that. So time64.h has the option of using a y2 billion safe time struct. bleadperl uses that.
136. Slide 139: y3001 bug Windows' C functions have a y3001 bug, I have no idea why. y2038 gets around that
137. Slide 140: y10k HP/UX has a y10k bug y2038 gets around that
138. Slide 141: In the opposite direction we have the
139. Slide 142: Year Zero Problem Year Zero problem What does 0 mean?
140. Slide 143: If you ask a historian They'll say \"there is no year zero\"
141. Slide 144: 1 BC + 1 year = 1 AD And historians are fine with this Because they still do their work on scrolls And it's not an exact science And being a year off isn't too important This is also why 2001 is the \"real millennium\". For math and computers, the discontinuity is difficult. This sucks. So what does the Gregorian calendar say?
142. Slide 145: Nope, no year 0
143. Slide 146: ISO 8601? How about the International date standard? Nope.
144. Slide 147: Values in the range [0000] through [1582] shall only be used by mutual agreement of the partners in information interchange. Which is ISO-speak for \"we couldn't decide\" But they do have a year 0 and they have the best cop out.
145. Slide 148: −0002-04-12 What is year -2 in ISO?
146. Slide 149: ...the second year before the year [0000] So what's year zero? It's the year before year one. So simple!
147. Slide 150: Turns out, this is how astronomers do it. They're the only people who need to worry about really long dates And do a lot of math where being one year off matters.
148. Slide 151: 1 = 1 AD 0 = 1 BC -1 = 2 BC
149. Slide 152: Step 3: Profit!
150. Slide 153: Fortunately TPF funded my project to do all this. (and thus you all, through your donations)
151. Slide 154: Merijn Brand is helped hooking it into the Perl guts
152. Slide 155: http://y2038.googlecode.com/ The project needs help. I'm the only developer. I'm not a C programmer. Only a handful of time.h functions have been implemented. Python needs a lot more. If you're a C programmer, or want to learn, please sign up. There's a user's list. I'm liberal with handing out commit bits. Be bold.
153. Slide 156: Thanks... Text
154. Slide 157: ...for your money.
155. Slide 158: ...for your money. ;)