WEBVTT 00:00:00.000 --> 00:00:04.080 You with like an affordable amount of spend on tokens, 00:00:04.360 --> 00:00:07.099 you can do what a hundred person 00:00:07.099 --> 00:00:11.419 incredibly great engineering team would do as a startup. 00:00:11.539 --> 00:00:12.599 And that was just totally impossible. 00:00:12.800 --> 00:00:16.570 That was not in the set of options for a 00:00:16.570 --> 00:00:17.750 startup and now it is. 00:00:17.890 --> 00:00:19.870 So I think what you can take on, uh, 00:00:20.030 --> 00:00:21.390 the level of ambition you can have, 00:00:21.449 --> 00:00:22.510 the speed at which you can move, 00:00:22.649 --> 00:00:24.269 The amount stuff you could do at once 00:00:24.809 --> 00:00:26.329 Uh... It's just totally different 00:00:26.329 --> 00:00:27.649 And, um... 00:00:27.649 --> 00:00:30.269 Does that change the shape of problems you feel like 00:00:30.269 --> 00:00:33.450 You'd assign at end-of class for people to attack? 00:00:33.850 --> 00:00:35.990 Y'know. At the End Of That Quarter if you 00:00:35.990 --> 00:00:38.340 were teaching it again I don't think assigning 00:00:39.299 --> 00:00:40.780 Problems To Attack ever works because 00:00:40.780 --> 00:00:43.549 If i can think of a problem 00:00:43.549 --> 00:00:45.250 If I could think about a really great startup idea 00:00:45.810 --> 00:00:47.390 It's obvious enough to me 00:00:48.049 --> 00:00:50.329 Then Its probably obvious too 00:00:50.329 --> 00:00:52.149 A lot of people When we started OpenAI 00:00:52.149 --> 00:00:53.670 We were Like The Uh 00:00:54.229 --> 00:00:56.310 One maybe Generously speaking 00:00:56.310 --> 00:00:58.390 For 4 AGI efforts in the world 00:00:58.390 --> 00:01:00.130 Right And you want to find something like that 00:01:00.130 --> 00:01:01.850 and I'm sure That there exists 00:01:01.850 --> 00:01:04.310 Something today That just wasn't possible at all 00:01:04.310 --> 00:01:06.950 Pre Like automated coding era 00:01:07.469 --> 00:01:09.370 Uh It is totally not obvious 00:01:09.370 --> 00:01:10.430 There will be You know 00:01:10.430 --> 00:01:13.090 uh Multi-trillion dollar market soon 00:01:13.090 --> 00:01:15.549 Ah Only four companies are working on it right now 00:01:15.549 --> 00:01:16.650 But i don't know what that is 00:01:16.650 --> 00:01:17.890 Its much more likely Y'all 00:01:17.890 --> 00:01:18.950 Know What That Is Than I know 00:01:18.950 --> 00:01:21.849 What Thats Just My brain has taken over by OpenAI 00:01:22.349 --> 00:01:25.709 Um The kind of idea someone can assign 00:01:25.709 --> 00:01:27.980 To work On Is Probably not what you want 00:01:27.980 --> 00:01:30.659 Yep Um Okay So that 00:01:30.659 --> 00:01:31.719 That's fair Uh 00:01:31.719 --> 00:01:33.079 But I think it would be helpful 00:01:33.079 --> 00:01:34.439 Since this is a systems class 00:01:34.439 --> 00:01:35.680 To maybe uh 00:01:36.260 --> 00:01:37.599 Reason about A particular problem 00:01:37.599 --> 00:01:39.280 That You have to reason through 00:01:39.280 --> 00:01:41.159 so That They can then apply 00:01:41.159 --> 00:01:42.959 The shape of the techniques used 00:01:42.959 --> 00:01:44.359 To break down from Systems perspective 00:01:44.359 --> 00:01:45.739 That Problem Into solutions 00:01:45.739 --> 00:01:46.879 To their own problems 00:01:46.879 --> 00:01:48.019 Yeah um And 00:01:48.019 --> 00:01:49.400 A concept That Ah 00:01:49.939 --> 00:01:51.019 You had started At TEAS 00:01:51.019 --> 00:01:52.680 In Class Y'know Back in 2014 00:01:52.680 --> 00:01:54.519 and Then Clearly We've 00:01:54.519 --> 00:01:55.540 Talked About Publicly Over 00:01:55.540 --> 00:01:57.079 Years Is Um Scale 00:01:57.939 --> 00:01:59.359 Right Scale is Its own 00:01:59.359 --> 00:02:00.519 Beast It's it's 00:02:00.519 --> 00:02:01.579 its You know Quantity Is 00:02:01.579 --> 00:02:02.780 Its Own Quality We What 00:02:02.780 --> 00:02:04.640 I Would Scale As A concept 00:02:04.640 --> 00:02:06.140 Has Been Something It Seems 00:02:06.140 --> 00:02:08.580 Like you've um Empirically 00:02:08.580 --> 00:02:09.599 Investigated In 00:02:09.599 --> 00:02:10.659 All Kinds Of ways Over 00:02:10.659 --> 00:02:12.000 The last 10 Years So Okay 00:02:12.000 --> 00:02:13.159 Um Could You Help 00:02:13.159 --> 00:02:14.400 On help Us First Unpack 00:02:14.400 --> 00:02:15.419 like what You mean By 00:02:15.419 --> 00:02:16.479 scale Now Ten years 00:02:16.479 --> 00:02:17.800 Later How would You Deconstruct 00:02:17.800 --> 00:02:20.500 That as A So 00:02:25.620 --> 00:02:28.099 I Don't 00:02:28.379 --> 00:02:29.680 Know Why The 00:02:29.680 --> 00:02:31.259 Following Observation Is 00:02:31.259 --> 00:02:32.360 True i Offer 00:02:32.360 --> 00:02:35.610 No Theory That Find 00:02:35.610 --> 00:02:36.789 Satisfying To 00:02:36.789 --> 00:02:37.870 Explain It And that Makes 00:02:37.870 --> 00:02:39.490 Me a Little Bit Nervous 00:02:39.490 --> 00:02:41.439 to Suggest You 00:02:41.439 --> 00:02:42.659 Follow it But Im Going Anyway 00:02:42.919 --> 00:02:44.120 Because Empirically It 00:02:44.120 --> 00:02:45.599 Does Seem Be true Which 00:02:45.599 --> 00:02:47.240 is All Of the Most 00:02:47.240 --> 00:02:48.439 Interesting Things Have 00:02:48.439 --> 00:02:50.060 observed In My Career 00:02:50.060 --> 00:02:51.139 And Watching Other 00:02:51.860 --> 00:02:53.099 Things Happen All Of 00:02:53.099 --> 00:02:54.180 The Most Interesting Ones 00:02:54.180 --> 00:02:55.300 Have 00:02:55.300 --> 00:02:56.419 Had 00:02:56.840 --> 00:02:58.240 Something To Do With 00:02:59.000 --> 00:03:00.199 Immersion Properties That Scale 00:03:00.580 --> 00:03:01.879 Or Scale Continuing 00:03:01.879 --> 00:03:03.180 to Provide Returns 00:03:03.180 --> 00:03:04.620 Far Beyond What the 00:03:04.620 --> 00:03:05.819 Consensus Think Will Work 00:03:06.180 --> 00:03:07.439 and This Obviously 00:03:07.439 --> 00:03:08.800 Happens with Like Scaling 00:03:08.800 --> 00:03:10.699 Loss For AI Models But 00:03:11.300 --> 00:03:12.400 this Happens With 00:03:12.400 --> 00:03:13.740 You 00:03:13.740 --> 00:03:14.780 Know Getting 00:03:14.780 --> 00:03:15.879 More Smart People Together 00:03:15.879 --> 00:03:17.000 To Think About One Problem 00:03:17.000 --> 00:03:18.080 This Has in A 00:03:18.080 --> 00:03:19.680 Research Setting This 00:03:19.680 --> 00:03:22.439 Happens With Companies 00:03:22.439 --> 00:03:23.639 And The Sort Of Economy 00:03:23.639 --> 00:03:24.939 Of Scale You Can Get In All 00:03:24.939 --> 00:03:26.539 These Different Ways I 00:03:26.539 --> 00:03:27.960 Really Learned This At Y Combinator 00:03:27.960 --> 00:03:29.240 When It 00:03:29.240 --> 00:03:30.360 Became Clear 00:03:30.360 --> 00:03:32.060 To Me That Everybody Was 00:03:32.060 --> 00:03:33.259 Saying Oh Why Commenters Gotten 00:03:33.259 --> 00:03:34.379 Too Big It Should Shrink We 00:03:34.379 --> 00:03:35.599 Should Found Less Companies Per Batch 00:03:35.599 --> 00:03:36.719 You Know The Best Times 00:03:36.719 --> 00:03:37.800 A White Common Day What When 00:03:37.800 --> 00:03:38.879 It Is Like 10 Companies Per 00:03:38.879 --> 00:03:40.199 Batch And A 00:03:40.199 --> 00:03:41.259 Lot Of Like Very Smart 00:03:41.259 --> 00:03:43.099 People Were Saying This And 00:03:43.099 --> 00:03:46.000 It Was Like Tempting 00:03:46.000 --> 00:03:47.259 Because It Would Have Been Much Less Work 00:03:47.259 --> 00:03:48.560 The Theory Is That You 00:03:48.560 --> 00:03:50.300 Know Best Companies Are 00:03:50.300 --> 00:03:51.319 Always Kind Of Obvious Then 00:03:51.319 --> 00:03:52.419 You Fund The Rest Not As 00:03:52.419 --> 00:03:53.520 Helpful But 00:03:53.520 --> 00:03:54.759 A Huge Part In Magic 00:03:54.759 --> 00:03:56.039 Of What Made YC 00:03:56.039 --> 00:03:58.240 Work Where Was 00:03:58.240 --> 00:03:59.520 Sort Of Network Effects 00:03:59.520 --> 00:04:00.659 Inside Batch 00:04:01.080 --> 00:04:02.379 And That Was An Emergent Property 00:04:02.379 --> 00:04:11.449 At Scale That Never Happened 00:04:11.449 --> 00:04:12.629 Upon This Observation Of 00:04:12.629 --> 00:04:15.340 When You Do That There's 00:04:16.649 --> 00:04:17.689 Something Important 00:04:17.689 --> 00:04:19.050 That Happens That Just Didn't 00:04:19.050 --> 00:04:20.550 Exist At All At The 110th 00:04:20.550 --> 00:04:23.240 Or 1100 Th Of Scale There 00:04:23.240 --> 00:04:24.379 Is A Bunch Of Other Examples 00:04:24.379 --> 00:04:26.259 Like This I 00:04:26.259 --> 00:04:30.329 Will Skip 00:04:30.329 --> 00:04:31.790 Them In The Interest Of Time But 00:04:31.790 --> 00:04:32.949 I Would 00:04:32.949 --> 00:04:34.689 Say Again Offer 00:04:34.689 --> 00:04:35.949 No Explanation For Why But 00:04:35.949 --> 00:04:37.350 Empirically Speaking When 00:04:37.350 --> 00:04:38.449 You Find A Time 00:04:38.449 --> 00:04:39.689 That Can Push On 00:04:44.600 --> 00:04:46.680 The Smaller Scale 00:04:46.680 --> 00:04:47.860 More Often Than 00:04:47.860 --> 00:04:49.000 Not That Seems To 00:04:49.000 --> 00:04:50.060 Be A Good Idea And 00:04:50.060 --> 00:04:51.100 It Also Seems Something 00:04:51.639 --> 00:04:54.879 Most People Don't 00:04:54.879 --> 00:04:56.899 Do Enough I Don't 00:04:56.899 --> 00:04:58.100 Offer An Explanation For 00:04:58.100 --> 00:04:59.339 This Either But Like In 00:04:59.800 --> 00:05:00.959 You Know When We Were Really 00:05:00.959 --> 00:05:01.980 Going To Scale AR 00:05:01.980 --> 00:05:04.199 Models All Of 00:05:04.199 --> 00:05:05.439 The Geniuses In The Field 00:05:05.439 --> 00:05:06.620 Most Of Them Where Oh This 00:05:06.620 --> 00:05:07.660 Isn'T Working You Know 00:05:07.660 --> 00:05:08.800 That's Barely A Scientific 00:05:08.800 --> 00:05:10.019 Result Its Not Interesting 00:05:16.279 --> 00:05:18.720 To Scale 00:05:19.639 --> 00:05:24.660 This I'm 00:05:24.660 --> 00:05:26.079 A Little Worried About It For Non 00:05:26.079 --> 00:05:27.519 Specific Reasons And 00:05:27.519 --> 00:05:28.560 Again Looking Back At Like A 00:05:28.560 --> 00:05:30.569 Huge Data Set 00:05:30.569 --> 00:05:31.930 Of People That Have Scaled 00:05:32.959 --> 00:05:34.040 Their Companies In All These Different 00:05:34.040 --> 00:05:35.100 Ways There's Almost Always 00:05:35.100 --> 00:05:36.120 Interesting Stuff There So 00:05:36.120 --> 00:05:37.519 I Think Directionally 00:05:37.519 --> 00:05:39.040 That'S Like An Interesting Thing 00:05:39.040 --> 00:05:40.870 To Push On And 00:05:42.629 --> 00:05:43.829 Severely Underexplored 00:05:45.399 --> 00:05:46.420 On The Systems Design 00:05:46.420 --> 00:05:50.199 Part Of That Uh I 00:05:50.199 --> 00:05:51.879 Don't Think One Reason People Don't 00:05:51.879 --> 00:05:53.079 Do It As Much 00:05:53.600 --> 00:05:55.959 Is Stuff Breaks 00:05:55.959 --> 00:05:57.000 Uh At An 00:05:57.000 --> 00:05:58.100 Accelerating Rate And 00:05:58.100 --> 00:05:59.399 In Unpredictable Way As 00:05:59.399 --> 00:06:01.199 You Scale It And If 00:06:01.199 --> 00:06:02.259 You Are Going To Really Scale 00:06:02.259 --> 00:06:05.819 Something Um Its 00:06:05.819 --> 00:06:06.879 Always Like A Little Bit 00:06:06.879 --> 00:06:08.680 Broken There Are Always Very 00:06:08.680 --> 00:06:09.720 Smart People Who 00:06:09.720 --> 00:06:10.779 Say Why You Shouldn T O This 00:06:10.779 --> 00:06:12.120 Y Know Dont Get Too Ambitious 00:06:12.120 --> 00:06:13.240 Dont Get Too Big Lets Try 00:06:13.240 --> 00:06:14.579 The Smaller And 00:06:14.579 --> 00:06:15.759 So Breaking 00:06:15.759 --> 00:06:16.839 That Down Is A Systems 00:06:16.839 --> 00:06:18.220 Problem I Use The Thing 00:06:18.220 --> 00:06:19.420 Of When We Were Like Scaling Up 00:06:19.420 --> 00:06:20.439 AI Models There Was 00:06:21.079 --> 00:06:22.240 Technically Can Do This 00:06:22.240 --> 00:06:23.660 At All Seems Crazy No 00:06:23.660 --> 00:06:24.759 One Ever Thought About Trying 00:06:24.759 --> 00:06:26.040 To Run Across 10,000 00:06:26.040 --> 00:06:27.139 Or 100 Thousand GPUs 00:06:27.139 --> 00:06:28.500 And That Was Going Require Stacks 00:06:28.500 --> 00:06:29.819 Of Engineering Talent Um 00:06:29.819 --> 00:06:30.839 There Was The 00:06:30.839 --> 00:06:32.139 Capital Requirement 00:06:32.139 --> 00:06:33.339 What It Was Going To Take 00:06:33.339 --> 00:06:44.199 To Do This How Learn Something 00:06:44.199 --> 00:06:45.279 Why Not Divide It Up Among 00:06:45.279 --> 00:06:47.240 All These Projects And 00:06:47.240 --> 00:06:48.620 This Also Happens 00:06:48.620 --> 00:06:49.879 In Kind Of Every Area I've 00:06:49.879 --> 00:06:50.980 Looked At Almost 00:06:50.980 --> 00:06:52.439 Every Area For Scale And 00:06:52.439 --> 00:06:53.699 Breaking Down Into 00:06:53.699 --> 00:06:55.589 The Sort Each 00:06:57.019 --> 00:06:58.319 Difficult Are Or Each 00:06:58.319 --> 00:06:59.439 Reason Not To Do It Trying 00:06:59.439 --> 00:07:00.459 To Address Them One At 00:07:00.459 --> 00:07:01.620 A Time That's Been Really 00:07:01.620 --> 00:07:02.819 Important Um 00:07:04.420 --> 00:07:05.480 I'm Going Push On That 00:07:05.480 --> 00:07:07.680 Little Bit Because There 00:07:07.680 --> 00:07:08.779 Is Very Few People Who've Been Able To 00:07:08.779 --> 00:07:11.019 So Repeatedly Scale 00:07:11.019 --> 00:07:12.439 New Products And Systems 00:07:12.439 --> 00:07:13.500 The Way Uh 00:07:13.500 --> 00:07:14.699 The Open Ai Team Has 00:07:14.699 --> 00:07:15.860 Over The Years But 00:07:15.860 --> 00:07:16.939 It Seems Like One Of The 00:07:16.939 --> 00:07:18.699 Issues Is There 00:07:18.699 --> 00:07:19.839 Are All These Prior 00:07:19.839 --> 00:07:21.660 Conditioning Sort 00:07:21.660 --> 00:07:22.980 Of Mental Models And Expectations 00:07:22.980 --> 00:07:24.399 Humans Have And You 00:07:24.399 --> 00:07:25.980 Said Things Break And One 00:07:25.980 --> 00:07:27.980 Of The Things It Seems Often 00:07:27.980 --> 00:07:29.819 Breaks That's Hardest 00:07:29.819 --> 00:07:30.980 Re Factor 00:07:30.980 --> 00:07:32.379 Is Human The 00:07:32.379 --> 00:07:33.660 Human Side Of The 00:07:33.660 --> 00:07:34.759 Systems Design 00:07:35.240 --> 00:07:36.800 Right Wherever There's Human Implementers 00:07:36.800 --> 00:07:37.819 Or There'S Uh 00:07:37.819 --> 00:07:39.000 Human Participants 00:07:39.000 --> 00:07:40.180 In That And So What Have You Learned 00:07:40.180 --> 00:07:41.279 About Humans At Scale 00:07:41.279 --> 00:07:42.339 Like Organizing Humans 00:07:42.339 --> 00:07:43.500 At Scale To Participate 00:07:43.500 --> 00:07:44.759 In A System That May Not Be 00:07:44.759 --> 00:07:45.779 Like 00:07:46.339 --> 00:07:47.439 Just A Redo 00:07:47.439 --> 00:07:48.500 Of Some Past System That 00:07:48.500 --> 00:07:49.620 They Get Naively 00:07:49.620 --> 00:07:51.199 On At A 00:07:51.199 --> 00:07:52.300 Priority On First Blush 00:07:52.300 --> 00:07:55.740 I Think 00:07:55.740 --> 00:07:57.480 Like Clear A 00:07:57.480 --> 00:07:59.579 Clear Goal A Clear 00:07:59.579 --> 00:08:01.220 Plan To Get There And 00:08:01.759 --> 00:08:06.860 Like A Clear Answer 00:08:06.860 --> 00:08:07.939 To The Way That You're 00:08:07.939 --> 00:08:08.959 Going To Get There In Kind Of How 00:08:08.959 --> 00:08:10.079 Your Gonna Make Decisions Along The 00:08:10.079 --> 00:08:11.339 Way That's Very Important 00:08:11.339 --> 00:08:14.350 So Um If We 00:08:14.350 --> 00:08:15.449 Go Back To The Example When We 00:08:15.449 --> 00:08:16.589 Decided To Scale Up Models 00:08:17.009 --> 00:08:18.149 They Were A Lot People Who 00:08:18.149 --> 00:08:19.329 Are Like Uh This Isn't Really 00:08:19.329 --> 00:08:20.350 Gonna Work It'S Gonna Have These 00:08:20.350 --> 00:08:21.449 Problems Its Also Not 00:08:21.449 --> 00:08:33.789 You Know Can Be Like If We Get There 00:08:34.190 --> 00:08:35.830 Uh That'S Very 00:08:35.830 --> 00:08:37.559 Powerful And 00:08:37.559 --> 00:08:40.639 Then For Whatever Reason 00:08:40.639 --> 00:08:41.679 Um We 00:08:41.679 --> 00:08:43.080 Did Not Evolve 00:08:43.080 --> 00:08:45.269 To Be Good At Thinking 00:08:45.269 --> 00:08:47.320 About Exponentials People 00:08:47.320 --> 00:08:48.340 Have A 00:08:48.340 --> 00:08:49.500 Hard Time Imagining That 00:08:50.610 --> 00:08:51.690 Scaling Laws Are Going To Continue 00:08:51.690 --> 00:08:52.809 Exponentially The Revenue 00:08:52.809 --> 00:08:53.970 Will Grow Exponentally That 00:08:53.970 --> 00:08:55.370 An Organization Can 00:08:55.370 --> 00:08:56.909 Take On Exponential Complexity 00:08:57.490 --> 00:08:58.750 And In 00:08:58.750 --> 00:08:59.970 My Experience It Takes 00:08:59.970 --> 00:09:01.710 A Lot Of Time To 00:09:01.710 --> 00:09:03.149 Really Reason Through First Principles 00:09:03.149 --> 00:09:04.309 With People About Why 00:09:04.309 --> 00:09:05.909 That Can Happen Then 00:09:05.909 --> 00:09:07.629 We Take Two Examples To 00:09:07.629 --> 00:09:09.370 Walk Through The First Being 00:09:09.370 --> 00:09:11.009 Chat GPT And Second Codex 00:09:11.009 --> 00:09:12.110 You Know Both Of 00:09:12.110 --> 00:09:13.490 These Have Transformed 00:09:14.039 --> 00:09:15.190 Can Everyone Hear I'm 00:09:15.190 --> 00:09:16.330 Going Try Project It Yeah 00:09:16.330 --> 00:09:18.759 Okay So Let 00:09:18.759 --> 00:09:20.240 Me Put In A Frame 00:09:20.240 --> 00:09:21.320 And Challenge But The 00:09:21.320 --> 00:09:22.419 Assumption Is Then We Can 00:09:22.419 --> 00:09:23.879 Hopefully Reason To Example 00:09:23.879 --> 00:09:25.039 What Happened In 00:09:25.039 --> 00:09:26.759 The Case For Long Time 00:09:26.759 --> 00:09:27.860 Scaling Models 00:09:27.860 --> 00:09:29.419 A Big Mental Block 00:09:29.419 --> 00:09:31.419 That Seem To Be Prevalent 00:09:31.419 --> 00:09:32.799 In The Space Is What 00:09:32.799 --> 00:09:34.039 Are These Things Going To Useful 00:09:34.039 --> 00:09:35.740 For This You Know It's A Research 00:09:35.740 --> 00:09:37.059 Sort Of 00:09:37.059 --> 00:09:38.139 Solution Solution 00:09:38.139 --> 00:09:39.259 Chasing A Problem Research 00:09:39.860 --> 00:09:40.899 First Approach Is 00:09:40.899 --> 00:09:42.299 Not A Product And 00:09:42.759 --> 00:09:44.179 Then Chat GPT 00:09:44.179 --> 00:09:45.379 Came Out And Improved To The World 00:09:45.379 --> 00:09:46.539 That Chat 00:09:47.320 --> 00:09:48.379 Experience Was A Killer 00:09:48.379 --> 00:09:50.179 App For General Models 00:09:50.620 --> 00:09:52.159 At Scale For 00:09:52.159 --> 00:09:53.759 Consumers And Then A 00:09:53.759 --> 00:09:55.159 Couple Of Years Later Yes 00:09:55.159 --> 00:09:56.539 It's Clear That Coding 00:09:56.539 --> 00:09:57.580 Has Been The Killer 00:09:57.580 --> 00:09:59.480 Enterprise App So How 00:09:59.480 --> 00:10:00.860 Would You Compare And Contrast 00:10:00.860 --> 00:10:02.080 The Systems You Guys Used 00:10:02.080 --> 00:10:03.220 To Discover Those Use 00:10:03.220 --> 00:10:05.100 Cases Ship Them Scale 00:10:05.100 --> 00:10:06.419 Them Monetize Them Any 00:10:06.419 --> 00:10:07.919 Salient Learnings 00:10:07.919 --> 00:10:09.100 From Those Two Systems Yes 00:10:11.720 --> 00:10:12.879 So We Had Made 00:10:12.879 --> 00:10:16.379 GPT-3 And We 00:10:16.379 --> 00:10:17.519 Needed To Make Money Because 00:10:17.519 --> 00:10:18.559 We Want On Go Scale Up 00:10:18.559 --> 00:10:26.399 To You Know Billion 00:10:26.399 --> 00:10:27.600 And We Had Been Thinking 00:10:27.600 --> 00:10:28.639 Thinking We Just Couldn't Do 00:10:28.639 --> 00:10:29.759 It We Tried A Few Things They 00:10:29.759 --> 00:10:31.279 Hadn't Worked So 00:10:31.720 --> 00:10:32.799 When You The Models Bring Get 00:10:32.799 --> 00:10:33.840 Better But We Also 00:10:33.840 --> 00:10:35.639 Want To Like Start Revenue 00:10:35.639 --> 00:10:37.059 Engine Sooner And 00:10:37.639 --> 00:10:38.700 We Say Well Since We 00:10:38.700 --> 00:10:39.759 Can Figure Out What Product 00:10:39.759 --> 00:10:41.320 Build Which Is Going Put This Into 00:10:41.320 --> 00:10:42.960 An API And We Are Gonna 00:10:42.960 --> 00:10:44.059 Hope That Somebody Else In 00:10:44.059 --> 00:10:45.539 Fear Our Product Of Build And 00:10:45.539 --> 00:10:46.740 So We Launched In Like 00:10:47.220 --> 00:10:48.279 I Don't Know Something Of The 00:10:48.279 --> 00:10:50.139 Summer Of 2020 Did 00:10:50.139 --> 00:10:51.960 Gpd3 Api And 00:10:52.860 --> 00:10:54.019 Initially It Kind 00:10:54.019 --> 00:10:55.600 Got No Traction 00:10:55.600 --> 00:10:57.559 At All And Then 00:10:57.559 --> 00:10:58.580 About A Month Later 00:10:59.659 --> 00:11:00.700 Randomly As Far As We 00:11:00.700 --> 00:11:02.259 Can Tell It Went Viral 00:11:02.259 --> 00:11:03.419 On Twitter On The Same 00:11:03.419 --> 00:11:04.639 Day Uh Few 00:11:04.639 --> 00:11:05.759 Different Developers Kinda 00:11:05.759 --> 00:11:06.840 Found Gotta Do Something 00:11:06.840 --> 00:11:08.039 Cool Post That Other People 00:11:08.039 --> 00:11:09.659 Started Trying And 00:11:09.659 --> 00:11:10.740 And Then Like 00:11:11.159 --> 00:11:12.299 A Lot Of People Started 00:11:12.299 --> 00:11:13.559 Trying The Api Um 00:11:14.320 --> 00:11:15.620 But It Was Shockingly 00:11:15.620 --> 00:11:16.740 Bad If You Go Back And 00:11:16.740 --> 00:11:17.940 Use Gpd3 00:11:17.940 --> 00:11:19.480 Or 3.5 Um 00:11:20.610 --> 00:11:21.789 you Will Be Astonished 00:11:21.789 --> 00:11:23.529 At How Bad The Models 00:11:23.529 --> 00:11:24.710 Were Then Relative 00:11:24.710 --> 00:11:25.909 To The Amount Of Excitement 00:11:25.909 --> 00:11:27.759 They Generated At Time Uh 00:11:27.759 --> 00:11:28.779 So People Tried 00:11:28.779 --> 00:11:29.940 All These Things And Really 00:11:29.940 --> 00:11:30.960 the Only Business 00:11:30.960 --> 00:11:31.980 That People Got To Work 00:11:32.830 --> 00:11:33.909 In A Significant Way 00:11:33.909 --> 00:11:42.409 With GPT Three A Better Model 00:11:44.360 --> 00:11:45.580 Although That Was 00:11:45.580 --> 00:11:46.779 The Only Business That Was Working 00:11:47.559 --> 00:11:48.600 Developers Had Figured 00:11:48.600 --> 00:11:49.620 Out How To Like Put In A 00:11:49.620 --> 00:11:50.840 Prompt And Get And 00:11:50.840 --> 00:11:52.580 Be Able Chat With It And 00:11:52.580 --> 00:11:54.159 We Saw This Alot 00:11:54.159 --> 00:11:55.639 Like More People Were 00:11:55.639 --> 00:11:57.409 Using They 00:11:57.409 --> 00:11:58.509 Couldn Get The Api 00:11:58.509 --> 00:11:59.750 Work For Their Business But 00:11:59.750 --> 00:12:00.789 They Were Using There API 00:12:00.789 --> 00:12:01.990 Key Just Chat I Said 00:12:01.990 --> 00:12:03.149 Well We Can Build A Good Chap 00:12:03.149 --> 00:12:04.169 Buck People Clearly Want 00:12:04.169 --> 00:12:06.450 That And We Had 00:12:06.450 --> 00:12:07.470 A New Model We Actually Had You 00:12:07.470 --> 00:12:08.529 Before Done But We 00:12:08.529 --> 00:12:09.549 Had An New Model We Were Ready 00:12:09.549 --> 00:12:11.289 To Release In Between Called 3.5 00:12:11.289 --> 00:12:12.350 And We Could 00:12:12.350 --> 00:12:13.370 Figure Out A New Kind 00:12:13.370 --> 00:12:14.529 Of Post Training Where 00:12:14.529 --> 00:12:15.629 We Can Get The Models To Do 00:12:15.629 --> 00:12:16.850 Like Good Job With Instruction 00:12:16.850 --> 00:12:18.090 Following So It Make It Easier 00:12:18.090 --> 00:12:20.350 To Chat With And We Said 00:12:20.350 --> 00:12:22.419 Well You Know The 00:12:22.419 --> 00:12:23.980 Api Is Not 00:12:23.980 --> 00:12:35.440 Working Well We'll Build A Chat Bot 00:12:35.440 --> 00:12:36.779 Around It And We 00:12:36.779 --> 00:12:37.980 Put That Out And We 00:12:37.980 --> 00:12:39.139 Still Didn't Think Was Going To Do That Well 00:12:39.139 --> 00:12:40.419 There Was Really 00:12:40.419 --> 00:12:41.580 Meant As Like A Research 00:12:41.580 --> 00:12:43.100 Demo To 00:12:43.100 --> 00:12:44.299 Convince Other People 00:12:44.299 --> 00:12:45.539 They Should Build Chat Light 00:12:45.539 --> 00:12:46.899 Products In Past For The Api 00:12:46.899 --> 00:12:48.799 But That Went 00:12:48.799 --> 00:12:49.820 Like Crazy Viral 00:12:50.409 --> 00:12:51.419 Another Thing I Had 00:12:51.419 --> 00:12:52.720 Learned From YC Is When 00:12:52.720 --> 00:12:54.279 Something Starts Growing 00:12:54.279 --> 00:12:55.580 Not Very Good 00:12:56.070 --> 00:12:57.139 You Have Like A Guaranteed 00:12:57.139 --> 00:12:58.580 Hit On Your Hands And So 00:12:58.580 --> 00:12:59.600 We Had Like Five Days 00:12:59.940 --> 00:13:01.019 Where The Traffic 00:13:01.019 --> 00:13:02.919 Would Shoot Up Fall Off 00:13:02.919 --> 00:13:04.039 In Everybody Be Like Well That Was 00:13:04.039 --> 00:13:05.259 Just A Hype Cycle But Then Next Day 00:13:05.259 --> 00:13:06.840 It Would Get To Higher Peak Fall 00:13:06.840 --> 00:13:08.039 Off Again Later People Say 00:13:08.039 --> 00:13:09.059 That's A High Hype Cycle 00:13:09.710 --> 00:13:10.750 By The Fourth Or 00:13:10.750 --> 00:13:11.909 Fifth They I Was Like I Know 00:13:11.909 --> 00:13:13.049 How This Works I Know 00:13:13.049 --> 00:13:14.330 What's Going To Happen Like We Have 00:13:14.330 --> 00:13:15.389 The Potential Here 00:13:15.389 --> 00:13:16.549 At A Killer 00:13:16.549 --> 00:13:18.169 Product And 00:13:19.149 --> 00:13:20.269 We Knew We Could Make 00:13:20.269 --> 00:13:21.730 It Much Better When You We 00:13:21.730 --> 00:13:22.870 Could We Know He Had Gpt For 00:13:22.870 --> 00:13:24.250 We Knew We Could Keep Scaling 00:13:24.909 --> 00:13:27.860 But By That 00:13:27.860 --> 00:13:29.600 Fifth Day We Got Everybody 00:13:29.600 --> 00:13:30.620 Together And Said 00:13:31.080 --> 00:13:32.139 This Is An Emergency This 00:13:32.139 --> 00:13:33.700 Isn't Good Kind Of Emergency But 00:13:33.700 --> 00:13:34.740 We Have Build A Company 00:13:34.740 --> 00:13:36.419 And A Product All 00:13:36.419 --> 00:13:38.519 Once We Then 00:13:38.519 --> 00:13:39.639 Had Like Two 00:13:39.639 --> 00:13:41.059 Months Of Crazy Scaling 00:13:41.519 --> 00:13:42.740 And Then He Said You 00:13:42.740 --> 00:13:43.820 Know We Have To Figure 00:13:43.820 --> 00:13:45.179 Out A Business Model Later For 00:13:45.179 --> 00:13:46.399 Now Were Just Going To Charge People 00:13:46.399 --> 00:13:47.519 So That We Don't Run 00:13:47.519 --> 00:13:48.559 Our Compute 00:13:48.559 --> 00:13:49.779 Bills But That's Obviously 00:13:49.779 --> 00:13:51.179 Not The Long-Term Answer That 00:13:51.179 --> 00:13:53.120 Also Turned On Just Work Um 00:13:53.120 --> 00:13:54.259 And There Was 00:13:54.259 --> 00:13:55.419 A Story Of Tracking T And 00:13:55.419 --> 00:13:56.679 Then It Was So Much Utility 00:13:56.679 --> 00:14:07.299 That Before Chats 00:14:07.299 --> 00:14:08.460 Was That We Were Going To Go All 00:14:08.460 --> 00:14:10.440 In On Code Um We 00:14:10.440 --> 00:14:11.500 Knew These Models Could Write 00:14:11.500 --> 00:14:12.759 Code Uh We 00:14:12.759 --> 00:14:14.480 Knew They Be Really 00:14:14.480 --> 00:14:15.519 And We 00:14:15.519 --> 00:14:16.799 Knew It Would Like A Valuable Area 00:14:16.799 --> 00:14:18.120 But Then Had This Incredibly 00:14:18.120 --> 00:14:19.220 Exciting Thing Happen 00:14:19.220 --> 00:14:20.340 Um But Our 00:14:20.340 --> 00:14:21.379 Kind Of Internal Belief At The 00:14:21.379 --> 00:14:22.480 Time Was How 00:14:24.679 --> 00:14:25.919 These Models Would Control 00:14:25.919 --> 00:14:27.220 Things On Computers 00:14:27.220 --> 00:14:28.259 And Robots 00:14:28.259 --> 00:14:37.559 Where How These Models Were Robots 00:14:37.559 --> 00:14:38.860 And Then We 00:14:39.740 --> 00:14:40.919 Knew Intelligent To 00:14:40.919 --> 00:14:42.039 Do Stuff In The World 00:14:42.039 --> 00:14:43.080 So Uh 00:14:43.080 --> 00:14:44.139 It Took Us A While To 00:14:44.139 --> 00:14:46.429 Get There I Think 00:14:46.429 --> 00:14:47.490 Codex Got Really Good 00:14:47.490 --> 00:14:48.629 By Early 00:14:48.629 --> 00:14:49.710 This Year But 00:14:49.710 --> 00:14:50.830 With 5.5 Is 00:14:50.830 --> 00:14:51.990 When We Saw This Real 00:14:51.990 --> 00:14:53.149 Inflection Point Where People Are 00:14:53.149 --> 00:14:54.860 Now Like Doing 00:14:54.860 --> 00:14:56.179 Just Incredible Things 00:14:56.179 --> 00:14:57.240 With It Um 00:14:57.899 --> 00:14:59.600 You Know That Earlier 00:14:59.600 --> 00:15:00.799 In Class Talked 00:15:00.799 --> 00:15:01.919 About How The Capabilities 00:15:01.919 --> 00:15:03.419 Pipeline Is 00:15:03.419 --> 00:15:05.179 Starting To Look It 00:15:05.179 --> 00:15:06.200 Is Trying Become Somewhat More 00:15:06.200 --> 00:15:07.299 Legibly Standard 00:15:07.299 --> 00:15:08.620 Across Different Research 00:15:08.620 --> 00:15:09.759 Groups You Got 00:15:09.759 --> 00:15:11.519 Your Pre-Training Mid 00:15:11.519 --> 00:15:12.580 Training Post Training Then 00:15:12.580 --> 00:15:13.659 You Got The Rl And Supervised 00:15:13.659 --> 00:15:14.740 Feedback Loop Do 00:15:14.740 --> 00:15:16.120 You Think That's Roughly 00:15:16.120 --> 00:15:17.320 Like The Shape Of The 00:15:17.899 --> 00:15:19.080 Pipelines That Allowed Codex 00:15:19.080 --> 00:15:20.159 To Go Through A 00:15:20.159 --> 00:15:21.340 Capability Jump And That Will Basically 00:15:21.340 --> 00:15:22.399 Stay Stable Now In 00:15:22.399 --> 00:15:23.460 Consistent Or Are We Going To 00:15:23.460 --> 00:15:24.639 Go Through A Major Rewrite Of That 00:15:25.139 --> 00:15:26.200 Pipeline I Think It Is 00:15:26.200 --> 00:15:27.639 Definitely The Current Pipeline 00:15:27.639 --> 00:15:28.740 I Expect Will Go 00:15:28.740 --> 00:15:29.919 Through A Major Reright And 00:15:29.919 --> 00:15:31.179 Know When It Will Happen Or Exactly 00:15:31.179 --> 00:15:34.519 How But It 00:15:34.519 --> 00:15:35.539 Is Little Odd To Me 00:15:35.539 --> 00:15:38.419 That So Happens As 00:15:38.419 --> 00:15:39.700 A Pipeline Doesn't 00:15:39.700 --> 00:15:41.320 Quite Feel Like The 00:15:41.320 --> 00:15:42.639 Optimal Solution 00:15:43.259 --> 00:15:44.799 What Would Be An Optimal 00:15:44.799 --> 00:15:46.279 Solution In Your Head 00:15:46.279 --> 00:15:47.620 I Think That's A Research Problem 00:15:47.620 --> 00:15:48.700 For The AI's To Figure Out 00:15:48.700 --> 00:15:49.860 I Think We 00:15:49.860 --> 00:15:51.120 Are At A Point Where And 00:15:51.120 --> 00:15:52.220 We Set This Goal 00:15:52.220 --> 00:15:53.440 That By September Of This Year 00:15:53.440 --> 00:15:54.460 Will Use 00:15:54.460 --> 00:15:55.759 500,000 00:15:55.759 --> 00:15:57.000 800 Equivalent Gps 00:15:57.000 --> 00:15:58.120 Like A Lot Of Computing Power 00:15:58.120 --> 00:15:59.340 As An Ai Research 00:15:59.340 --> 00:16:00.480 Intern And By 00:16:00.480 --> 00:16:01.620 March Of 20 28 That 00:16:01.620 --> 00:16:03.019 We Have A Full 00:16:03.019 --> 00:16:04.480 End-To End Very 00:16:04.480 --> 00:16:05.700 Talented Researcher Like 00:16:05.700 --> 00:16:07.159 Figuring Out Complete Architectures 00:16:08.440 --> 00:16:09.740 So I 00:16:09.740 --> 00:16:11.360 Think We Are Going To Get Like With 00:16:11.360 --> 00:16:12.379 The Current Pipeline In 00:16:12.379 --> 00:16:13.700 The Current Architectures I 00:16:13.700 --> 00:16:14.860 Think Were Gonna Over The Line 00:16:14.860 --> 00:16:15.899 Of When Ai's 00:16:15.899 --> 00:16:17.179 Can Do Incredible Work 00:16:18.740 --> 00:16:19.820 You Know One 00:16:19.820 --> 00:16:21.960 Of Things That 00:16:21.960 --> 00:16:22.980 You Just Described 00:16:22.980 --> 00:16:25.700 There Is Your 00:16:25.700 --> 00:16:27.480 Way Even Talking 00:16:27.480 --> 00:16:29.299 A Lot About Systems Frameworks 00:16:29.299 --> 00:16:31.059 And Analogies To Make Concepts 00:16:31.059 --> 00:16:32.100 From One Domain Legible To 00:16:32.100 --> 00:16:33.139 Other People Who May Not Have All 00:16:33.139 --> 00:16:34.379 Context In 00:16:34.379 --> 00:16:36.620 Another And But Sometimes Because 00:16:36.620 --> 00:16:37.659 Of The Translation 00:16:37.659 --> 00:16:39.200 Problem You Know 00:16:39.200 --> 00:16:40.220 Reasoning By Analogy 00:16:40.220 --> 00:16:41.299 Is Not Helpful Because 00:16:41.299 --> 00:16:42.440 Then Errors Compound Yeah 00:16:42.440 --> 00:16:43.779 Right There 00:16:43.779 --> 00:16:45.240 You Said Our Goal To 00:16:45.240 --> 00:16:46.259 Use It As An Ai 00:16:46.259 --> 00:16:47.820 Intern Which Obviously A 00:16:47.820 --> 00:16:48.899 Very Useful Metaphor Within 00:16:48.899 --> 00:16:50.840 The Context Of Silicon 00:16:50.840 --> 00:16:52.600 Valley Class That Understands 00:16:52.600 --> 00:16:53.720 How These Pipelines Work And So On 00:16:53.720 --> 00:16:54.960 And Then As You 00:16:54.960 --> 00:16:56.100 Scale Actually That Matter For 00:16:56.100 --> 00:16:57.220 Globally People Who 00:16:57.220 --> 00:16:58.620 Might Have All That Context 00:16:58.620 --> 00:16:59.779 Go Start Analogizing 00:16:59.779 --> 00:17:01.000 These Models In Ways They Shouldn't 00:17:01.000 --> 00:17:02.240 Be Like How Should 00:17:02.240 --> 00:17:03.519 We Think About The Limits 00:17:03.519 --> 00:17:05.660 Of Of That What 00:17:05.660 --> 00:17:06.839 Are The Limit To Scale 00:17:07.319 --> 00:17:09.059 Of Or One Of Product 00:17:09.059 --> 00:17:10.619 Analogy Is Research 00:17:10.619 --> 00:17:11.779 An Algae You Find Most 00:17:11.779 --> 00:17:12.960 Useful Within 00:17:12.960 --> 00:17:14.119 Valley And Which One 00:17:14.119 --> 00:17:15.259 Have Found 00:17:15.259 --> 00:17:16.460 About Found About 00:17:16.460 --> 00:17:17.680 The Limits Of Those Analgy 00:17:17.680 --> 00:17:19.460 Scaling Now How 00:17:19.460 --> 00:17:20.640 Do Navigate Between Two 00:17:21.400 --> 00:17:23.519 Problems I've Been 00:17:29.390 --> 00:17:30.710 In Of 00:17:30.710 --> 00:17:31.750 Creating A New Utility 00:17:31.750 --> 00:17:32.769 This Doesn't Happen 00:17:32.769 --> 00:17:34.069 Very Often You Know Electricity 00:17:34.069 --> 00:17:35.210 Is Utilities Internet So Util 00:17:35.210 --> 00:17:36.990 It There Water I Guess 00:17:36.990 --> 00:17:38.509 There's Not A Lot Of These And 00:17:38.509 --> 00:17:39.569 So They Are Not A Lot Of 00:17:39.569 --> 00:17:40.849 Examples That We Can Study 00:17:41.309 --> 00:17:43.130 For Good Metaphors 00:17:43.130 --> 00:17:44.269 Or Learnings About How To 00:17:44.910 --> 00:17:46.109 Explain Us To The World 00:17:46.109 --> 00:17:47.250 But I 00:17:47.250 --> 00:17:49.049 Was Recently Looking 00:17:49.049 --> 00:17:50.210 At What Happened 00:17:50.210 --> 00:17:51.609 When Electric Became 00:17:51.609 --> 00:18:09.140 A Problem In The World Before 00:18:09.140 --> 00:18:10.500 And You Know It 00:18:10.500 --> 00:18:11.599 Feels Sort Of 00:18:11.599 --> 00:18:12.779 Like Very Different Than The World 00:18:12.779 --> 00:18:16.079 Before Maybe They 00:18:16.079 --> 00:18:17.480 Tried To Sell Electricity 00:18:17.480 --> 00:18:18.500 Or Market Electric At First 00:18:18.500 --> 00:18:19.640 I Don't Know But If Any 00:18:19.640 --> 00:18:20.920 Case That Didn't Work And 00:18:20.920 --> 00:18:23.170 Then What They Started Marketing 00:18:23.170 --> 00:18:24.309 Selling People Was Light 00:18:24.309 --> 00:18:26.109 At Night We Are 00:18:26.109 --> 00:18:27.829 Going To Get From 00:18:27.829 --> 00:18:28.950 Us Is Not Electric 00:18:28.950 --> 00:18:30.130 City Its Light At Night 00:18:30.390 --> 00:18:31.470 By The Way You Can Use Same 00:18:31.470 --> 00:18:32.650 Thing That Lets You Get Light 00:18:32.650 --> 00:18:33.769 For All These Other Things 00:18:33.769 --> 00:18:34.950 But People Are Like Well Why 00:18:34.950 --> 00:18:36.029 Would I Want That Now Like Well 00:18:36.029 --> 00:18:37.250 You Know It Will Wash 00:18:37.250 --> 00:18:38.390 Your Clothes For You Someday 00:18:38.390 --> 00:18:39.450 And No No One I 00:18:39.450 --> 00:18:40.470 Can't That's Too Far 00:18:40.470 --> 00:18:41.869 Of A Jump From Me Right Um 00:18:41.869 --> 00:18:43.890 So I Don't 00:18:43.890 --> 00:18:45.049 Know What Our Anal Alogy 00:18:45.920 --> 00:18:47.079 For This Should Be 00:18:47.500 --> 00:18:49.339 But I Suspect 00:18:49.339 --> 00:18:51.589 That Even If We Are 00:18:51.589 --> 00:18:52.650 Totally Right And 00:18:52.650 --> 00:18:53.769 Intelligence Is Going 00:18:53.769 --> 00:18:54.849 To Become This New Utility 00:18:55.230 --> 00:18:56.329 That Every Company 00:18:56.329 --> 00:18:57.930 Every Customer Every 00:18:57.930 --> 00:18:59.910 Government Just Needs 00:18:59.910 --> 00:19:01.869 Access Too In All Sorts 00:19:01.869 --> 00:19:02.910 Of Incredible Ways You 00:19:02.910 --> 00:19:03.990 Will Have Like A 00:19:03.990 --> 00:19:05.069 Open AI Token 00:19:05.069 --> 00:19:06.269 Subscription That You Will Plug 00:19:06.269 --> 00:19:07.430 Into Everything And Use To Access 00:19:07.430 --> 00:19:08.609 Everything And You 00:19:08.609 --> 00:19:09.630 Have Running For You All The Time 00:19:09.630 --> 00:19:11.599 Doing This Amazing Stuff I 00:19:11.599 --> 00:19:12.640 Kind Of Don't Think 00:19:12.640 --> 00:19:13.700 At Least Right Now 00:19:13.700 --> 00:19:14.720 The Right Way For Us 00:19:14.720 --> 00:19:15.819 To Analogize That Is 00:19:15.819 --> 00:19:17.140 We Are Selling Intelligence 00:19:17.140 --> 00:19:19.349 Because People Just Like Somehow 00:19:19.349 --> 00:19:20.549 Not Resonating I 00:19:20.549 --> 00:19:21.950 Don't Know What Our Equivalent 00:19:21.950 --> 00:19:23.690 Of Were Selling You 00:19:23.690 --> 00:19:24.750 Light At Night It's Going To Be 00:19:25.309 --> 00:19:26.730 But If We Will Become 00:19:26.730 --> 00:19:28.529 A New Utility We Need 00:19:28.529 --> 00:19:29.869 To Find A Way Explain To 00:19:29.869 --> 00:19:31.069 The World What It Means To 00:19:31.069 --> 00:19:32.789 Have This Like Intelligent 00:19:32.789 --> 00:19:33.809 Pike That You 00:19:33.809 --> 00:19:34.829 Can Just Do Whatever You Like With 00:19:34.829 --> 00:19:39.309 It So Question 00:19:39.309 --> 00:19:40.869 That Has Emerged An 00:19:40.869 --> 00:19:41.950 Immersion Property Of This Class 00:19:41.950 --> 00:19:43.009 Is Having A 00:19:43.009 --> 00:19:44.049 Diversity Of 00:19:44.049 --> 00:19:45.869 Different Speakers As The Utility 00:19:45.869 --> 00:19:47.049 Analogy Has Come Up Several Times 00:19:47.049 --> 00:19:48.289 But In Reference To Different 00:19:48.289 --> 00:19:50.470 Things So Jensen Likened 00:19:50.470 --> 00:19:51.589 Your Tip Like Compute 00:19:52.119 --> 00:19:54.119 To Our Utility And 00:19:54.119 --> 00:19:55.160 Why There Should Be Access 00:19:55.160 --> 00:19:56.240 And Talk About How 00:19:56.240 --> 00:19:58.059 Stanford Should Pull Budget And 00:19:59.000 --> 00:20:00.319 Procure That As A Utility 00:20:00.319 --> 00:20:01.980 For Everybody On Campus Whereas 00:20:01.980 --> 00:20:03.420 You Just Likened The Intelligence 00:20:03.420 --> 00:20:04.480 Part To Your Utilities 00:20:04.480 --> 00:20:05.559 Are Both Of These Things 00:20:05.559 --> 00:20:07.680 True Is One More Likely 00:20:07.680 --> 00:20:08.779 How Should People Reason About 00:20:08.779 --> 00:20:10.099 Compute As A Utilty Versus 00:20:10.099 --> 00:20:11.180 Tokens And 00:20:11.720 --> 00:20:12.779 By Compute I 00:20:12.779 --> 00:20:14.019 Mean Here Chips Vs. 00:20:14.099 --> 00:20:15.200 Tokens Does That 00:20:15.200 --> 00:20:16.480 Make Sense I 00:20:16.579 --> 00:20:17.779 Think As A Consumer 00:20:17.779 --> 00:20:18.980 As Like A Business 00:20:19.460 --> 00:20:21.680 Or An Individual You 00:20:21.680 --> 00:20:22.720 Will Think In Something 00:20:29.579 --> 00:20:30.940 Like The Hardware 00:20:30.940 --> 00:20:32.119 Is What Particular Chip 00:20:32.119 --> 00:20:33.160 It Is What's Powering And 00:20:33.160 --> 00:20:34.640 I Think That Stuff Will Be Abstracted 00:20:34.640 --> 00:20:36.460 Out And What You Care About 00:20:36.460 --> 00:20:38.660 When Your Interacting 00:20:38.660 --> 00:20:41.869 With This System Can 00:20:41.869 --> 00:20:43.009 Use A Lot As Cheap 00:20:43.009 --> 00:20:45.049 Doing Good Job So 00:20:45.049 --> 00:20:46.170 Right Now Its 00:20:46.170 --> 00:20:47.470 Like Tokens It 00:20:47.470 --> 00:20:48.609 May Get As We 00:20:48.609 --> 00:20:50.470 Move Into World Were We 00:20:50.470 --> 00:20:52.349 All Have Constant Agent Running 00:20:52.349 --> 00:20:53.430 For Us Being Useful To 00:20:53.430 --> 00:20:54.809 Us All Of The Time You 00:20:54.809 --> 00:20:56.049 May Think About It Is 00:20:56.049 --> 00:20:57.450 Even One Level Up But 00:20:57.450 --> 00:20:58.869 Yeah My Guess As New 00:20:59.920 --> 00:21:00.980 When You Like Pay For 00:21:00.980 --> 00:21:02.940 Your Cell Phone Bill You're 00:21:02.940 --> 00:21:04.039 Like All Right I'm Buying 00:21:04.039 --> 00:21:05.380 Access To Air 00:21:05.380 --> 00:21:06.460 Time And Some Number 00:21:06.460 --> 00:21:07.599 Of Gigabytes And 00:21:07.599 --> 00:21:08.660 You Know Its Going Do These 00:21:08.660 --> 00:21:09.819 Things And I'll Use All These Apps 00:21:09.819 --> 00:21:10.980 Whatever Else But What 00:21:10.980 --> 00:21:12.420 You Think About Pain 00:21:12.420 --> 00:21:13.779 For That Kind Internet Utility 00:21:13.779 --> 00:21:25.660 In This Case Um I 00:21:25.660 --> 00:21:26.720 Know I Could Nerd Out About 00:21:26.720 --> 00:21:27.839 Utility Infrastructure For 00:21:27.839 --> 00:21:29.039 A Long Time But I Want To Make Sure We 00:21:29.039 --> 00:21:30.200 Switch A Little Bit To Being Relevant 00:21:30.200 --> 00:21:31.539 For The Students Usually 00:21:31.539 --> 00:21:33.339 We Have Questions Where 00:21:33.339 --> 00:21:34.539 Were Not Hearing Those Today Well 00:21:34.539 --> 00:21:35.859 Unless You're Comfortable Oh 00:21:35.859 --> 00:21:36.900 Okay Great How 00:21:36.900 --> 00:21:37.920 About That Improv 00:21:37.920 --> 00:21:40.160 Ok Uh So 00:21:40.440 --> 00:21:41.640 One Final Question Start 00:21:41.640 --> 00:21:42.880 Getting The Creative Juices Flowing 00:21:42.880 --> 00:21:44.180 Is Um The Final Project 00:21:44.180 --> 00:21:45.940 For This Class Or 00:21:45.940 --> 00:21:46.960 Partly Private 183 00:21:46.960 --> 00:21:48.319 Is The One Person Frontier 00:21:48.319 --> 00:21:49.900 Lab So Everybody Here Is 00:21:49.900 --> 00:21:50.940 Working On Projects 00:21:50.940 --> 00:21:52.079 Where They Are Simulating Being 00:21:52.079 --> 00:21:54.380 An Individual Uh As 00:21:54.380 --> 00:21:55.579 A Lab With Access To 00:21:55.579 --> 00:21:56.839 All Right Tools I've Got Hundreds 00:21:56.839 --> 00:21:58.000 Of Thousands Dollars Of Credits From 00:21:58.000 --> 00:21:59.039 Cloud Flare I 00:21:59.039 --> 00:22:00.259 Think We Have Some Open Ai 00:22:00.259 --> 00:22:01.680 Tokens Maybe But 00:22:01.680 --> 00:22:03.079 There's A Bunch Of Compute At Their Disposal 00:22:03.079 --> 00:22:04.680 Um What Would 00:22:04.680 --> 00:22:06.259 You If You Were In The Class 00:22:06.259 --> 00:22:07.319 What Would You Be Working 00:22:07.319 --> 00:22:08.759 On For Your One Person Frontier 00:22:08.759 --> 00:22:10.680 Lab Project First Of 00:22:10.680 --> 00:22:11.779 All I Think That's An Awesome Project 00:22:11.779 --> 00:22:15.119 Um I Think 00:22:15.119 --> 00:22:16.319 This Is Top Of Mind Because 00:22:16.759 --> 00:22:19.220 Uh You We Were 00:22:19.220 --> 00:22:20.440 Just Like Talking About Utility 00:22:20.920 --> 00:22:22.880 Frame Frameworks And 00:22:22.880 --> 00:22:24.119 There Was A Lot Very Smart People 00:22:24.119 --> 00:22:25.519 Working On Uh 00:22:26.859 --> 00:22:28.480 Training Ideas And 00:22:28.480 --> 00:22:29.640 We're Going To Have Incredible 00:22:29.640 --> 00:22:30.980 Models No Matter 00:22:30.980 --> 00:22:32.140 What You Do We Have Incredible 00:22:32.140 --> 00:22:33.279 Models I Promise Here 00:22:33.279 --> 00:22:34.380 Uh Like Pretty Quickly 00:22:36.920 --> 00:22:37.980 And I Think We Have 00:22:37.980 --> 00:22:39.059 Not Invested Enough In 00:22:39.059 --> 00:22:40.380 Being Able To Deliver 00:22:40.759 --> 00:22:42.000 At Scale Huge 00:22:42.000 --> 00:22:43.660 Amounts Of Cheap Intelligence So 00:22:43.660 --> 00:22:44.720 Maybe Go Work On The 00:22:44.720 --> 00:22:45.759 Influence Part Of This Stack 00:22:45.759 --> 00:22:46.839 And How 00:22:46.839 --> 00:22:48.619 Are Going To Get This 00:22:48.619 --> 00:22:49.779 Incredible Intelligence To 00:22:49.779 --> 00:22:51.460 Be Cheap And Abundant Uh 00:22:51.460 --> 00:22:52.900 I Think That's Under Invested 00:22:52.900 --> 00:22:54.420 In An N I Think 00:22:54.420 --> 00:22:55.700 All Of Frontier Labs Are 00:22:55.700 --> 00:22:57.160 Going To Have Become Inference 00:22:57.160 --> 00:22:58.240 Companies To A Senior Finger 00:22:58.240 --> 00:23:00.619 Degree Um Okay 00:23:01.990 --> 00:23:03.029 Might Be Too Late To 00:23:03.029 --> 00:23:04.130 Pivot Your Projects But 00:23:04.650 --> 00:23:05.730 Better Late Than Never Work On 00:23:05.730 --> 00:23:07.750 Whatever You Want To Work Out Uh 00:23:08.200 --> 00:23:09.460 Ok Let's Start Taking Questions 00:23:09.460 --> 00:23:11.079 And I'm Gonna Moderate And 00:23:11.079 --> 00:23:12.660 Try To Not You Know Please 00:23:12.660 --> 00:23:13.779 Try To Productive And Not Spicy 00:23:13.779 --> 00:23:15.539 Etc Remember It'S 00:23:15.539 --> 00:23:17.000 The CS Class But Up 00:23:17.000 --> 00:23:27.299 Do Sam If You 00:23:27.299 --> 00:23:28.339 First 00:23:29.910 --> 00:23:31.589 Of All In Terms Achieving 00:23:31.589 --> 00:23:33.309 Human Level Intelligence These 00:23:33.309 --> 00:23:34.769 Models Have Already Far 00:23:34.769 --> 00:23:36.150 Surpassed Human Intelligence 00:23:36.150 --> 00:23:37.190 And Some Ways And Then 00:23:37.190 --> 00:23:38.369 They're Wildly Worse Than Others 00:23:38.369 --> 00:23:39.650 Like For Exam 00:23:39.650 --> 00:23:41.009 Pl Seem Much Wors 00:23:41.009 --> 00:23:42.569 Th An People Are At 00:23:42.569 --> 00:23:43.890 Very Long 00:23:43.890 --> 00:23:46.029 Horizon Kind 00:23:46.509 --> 00:23:47.890 Of High Judgment 00:23:47.890 --> 00:23:49.069 Signal 00:23:49.230 --> 00:23:50.529 And Tasks Uh 00:23:50.529 --> 00:23:51.670 On The 00:23:51.670 --> 00:23:53.490 Other Hand Yesterday We 00:23:53.490 --> 00:23:54.589 Had One Of 00:23:54.589 --> 00:23:57.369 Our Models Discover Disprove 00:23:57.369 --> 00:23:58.529 A Conjecture One 00:23:58.529 --> 00:23:59.589 Of The Erdős Problem That Had 00:24:00.339 --> 00:24:01.380 Smart People Have Worked 00:24:01.380 --> 00:24:03.259 On For Long Time And A 00:24:03.259 --> 00:24:04.319 Lot Of People A Lot 00:24:04.319 --> 00:24:05.420 Of Smart Scientists I Don't Know If 00:24:05.420 --> 00:24:07.799 Lacuna Is One Or Not Even 00:24:07.799 --> 00:24:09.000 Quite Recently Said Something 00:24:09.000 --> 00:24:10.240 Like That Was Not Going To Happen 00:24:10.640 --> 00:24:12.220 And Then Like The 00:24:12.220 --> 00:24:13.259 Model Just Did It And 00:24:13.259 --> 00:24:14.319 You Know Now You Have All 00:24:14.319 --> 00:24:15.339 These Mathematicians Saying Like 00:24:15.779 --> 00:24:16.859 Is Math Over What Does This 00:24:16.859 --> 00:24:18.000 Mean For Our Field So 00:24:18.000 --> 00:24:19.900 Clearly LLMs 00:24:19.900 --> 00:24:21.180 Are Capable Of 00:24:22.059 --> 00:24:23.180 Figuring Out New Knowledge 00:24:23.180 --> 00:24:24.380 And Clearly They 00:24:24.380 --> 00:24:25.420 Are Capable Doing Some 00:24:25.420 --> 00:24:27.059 Things That Some Intelligence 00:24:27.059 --> 00:24:28.259 Tasks That Humans Just 00:24:28.259 --> 00:24:30.220 Can't Do There Going 00:24:30.220 --> 00:24:31.359 To Scale Much Further So 00:24:31.359 --> 00:24:32.400 How Much Better And 00:24:32.400 --> 00:24:33.519 What Distribution The Task 00:24:33.519 --> 00:24:34.559 They Can Do Better Than Humans 00:24:35.000 --> 00:24:36.319 Will Find Out But I Suspect 00:24:36.319 --> 00:24:38.500 It's A Lot In The You 00:24:38.500 --> 00:24:39.619 Know In Terms Of This Like 00:24:40.240 --> 00:24:41.299 Lack Of A Belief 00:24:41.299 --> 00:24:42.359 And The Exponential We Were Talking 00:24:42.359 --> 00:24:43.700 About Earlier I 00:24:44.299 --> 00:24:45.640 Think The Field Was Honestly 00:24:45.640 --> 00:24:47.480 Held Back By A 00:24:47.480 --> 00:24:48.660 Generation Of Scientists 00:24:48.660 --> 00:24:49.759 Who Just Were Way Too 00:24:49.759 --> 00:24:50.920 Certain On What Wouldn't 00:24:50.920 --> 00:24:52.500 It What Scaling Is Not 00:24:52.500 --> 00:24:53.839 Going To Produce And 00:24:53.839 --> 00:24:55.099 Then Some People Looked 00:24:55.099 --> 00:24:56.299 At Graph S And Said Well Looks Like 00:24:56.299 --> 00:24:57.539 It's Continuing Beautifully Let'S 00:24:57.539 --> 00:25:01.089 Keep Going I Think 00:25:01.089 --> 00:25:02.190 World Models Are 00:25:02.190 --> 00:25:03.289 Clearly Important 00:25:03.829 --> 00:25:06.210 And To Will Need 00:25:06.210 --> 00:25:07.529 That For Things Like Robotics 00:25:07.990 --> 00:25:10.950 But Betting Against 00:25:11.690 --> 00:25:12.809 LLM Scaling At This 00:25:12.809 --> 00:25:16.549 Point Feels Quite Misguided 00:25:16.549 --> 00:25:19.400 To Me It Doesn't Get 00:25:19.400 --> 00:25:20.500 Annoying To Be The I Told You 00:25:20.500 --> 00:25:21.680 So Guy Not I 00:25:21.680 --> 00:25:25.119 Mean There Are These Light 00:25:25.119 --> 00:25:26.160 Twitter Trolls That 00:25:26.160 --> 00:25:27.240 You Know Four Years Have 00:25:27.240 --> 00:25:28.380 Just Been Like Is Not 00:25:28.380 --> 00:25:29.440 Going Work Its Not Gonna Work 00:25:29.440 --> 00:25:30.579 This Is So Dumb Like You 00:25:30.579 --> 00:25:31.720 Know This Is A Fraud This 00:25:31.720 --> 00:25:32.819 Company Is Gonna Fail This Research 00:25:32.819 --> 00:25:34.079 Approach Is Going To Fail And 00:25:34.079 --> 00:25:35.319 I Used To Get More Bothered 00:25:35.319 --> 00:25:36.900 By Them But I Don't Even Feel 00:25:36.900 --> 00:25:37.960 The I Told You At This 00:25:37.960 --> 00:25:38.980 Point It's Like You Were Just 00:25:38.980 --> 00:25:40.079 There She Was Never On 00:25:40.079 --> 00:25:41.180 Your Still Going About It 00:25:41.180 --> 00:25:42.859 Like The Data 00:25:42.859 --> 00:25:46.099 Is Quite Strong 00:25:46.099 --> 00:25:48.660 Our Side Only 00:25:48.660 --> 00:25:49.799 Be That Fun To Say I Told 00:25:49.799 --> 00:25:51.000 You So And Also The Fact 00:25:51.000 --> 00:25:52.059 That Your Like Still Saying We're 00:25:52.059 --> 00:25:53.640 Wrong Doesn't Really Bother Me I 00:25:53.640 --> 00:25:54.680 Think There's A Kind Of Move On 00:25:54.680 --> 00:25:55.779 This Is Saying That 00:25:55.779 --> 00:25:57.359 Like Insanity Is Doing 00:25:57.359 --> 00:25:59.160 The Same Thing Over Again When Presented 00:25:59.160 --> 00:26:00.220 With Data That It'S Not 00:26:00.220 --> 00:26:02.240 Working In A Sense Its 00:26:02.819 --> 00:26:04.240 Form Of Insane 00:26:04.240 --> 00:26:05.579 T E I Think I Think 00:26:05.579 --> 00:26:07.240 Something Happens Which If 00:26:07.240 --> 00:26:08.259 You Make Your Ident 00:26:08.259 --> 00:26:09.799 Ty About A Particular 00:26:10.779 --> 00:26:11.799 Thing Is Going To 00:26:11.799 --> 00:26:14.710 Work Or Not Work And 00:26:14.710 --> 00:26:16.009 You Associate Yourself 00:26:16.009 --> 00:26:17.430 With That Belief And 00:26:17.430 --> 00:26:18.569 Then The Science Are The 00:26:18.569 --> 00:26:19.789 Empirical Results Disprove 00:26:19.789 --> 00:26:21.400 You And You're Like 00:26:21.400 --> 00:26:22.759 Too Hung Up On 00:26:22.759 --> 00:26:23.779 Your Ident E Can't Let 00:26:23.779 --> 00:26:24.880 It Go You Can See The 00:26:24.880 --> 00:26:26.000 Truth Yeah And 00:26:26.000 --> 00:26:27.359 I Think This Is Like An Important 00:26:27.359 --> 00:26:28.539 Reminder In Both 00:26:28.539 --> 00:26:29.599 Directions How 00:26:30.769 --> 00:26:33.579 Do You See Education Um 00:26:34.359 --> 00:26:35.599 It Clearly Has 00:26:35.599 --> 00:26:36.819 Super Adapt And 00:26:36.819 --> 00:26:38.039 I Am Worried I Thought 00:26:38.039 --> 00:26:39.160 By Now It Would Have 00:26:39.539 --> 00:26:40.680 Um The The 00:26:41.259 --> 00:26:42.299 I Think If We 00:26:42.299 --> 00:26:43.559 Continue To Teach 00:26:43.559 --> 00:26:44.700 An Evaluate Students 00:26:45.960 --> 00:26:47.599 As In A Pre-AGI 00:26:47.599 --> 00:26:50.099 World Um Its 00:26:50.099 --> 00:26:51.200 Not Going To Work And Is 00:26:51.200 --> 00:26:52.519 Gonna Lead Into Like Atrophy 00:26:52.519 --> 00:26:53.740 Of Learning How To Think 00:26:53.740 --> 00:26:54.940 Or Whatever And I Thought That 00:26:54.940 --> 00:26:56.160 Was Obvious Enough 00:26:56.160 --> 00:26:57.319 That Wasn't That 00:26:57.319 --> 00:26:58.460 Worryed You Know When Chat 00:26:58.460 --> 00:26:59.519 GBT Launch I Was Like 00:26:59.519 --> 00:27:08.180 Yeah And We're Going To Teach People So 00:27:08.180 --> 00:27:09.200 Much Better You 00:27:09.200 --> 00:27:11.839 Know People Are Going Really Get 00:27:11.839 --> 00:27:13.000 Projects Where They Have 00:27:13.000 --> 00:27:14.779 To Use AI But 00:27:15.400 --> 00:27:16.579 Still Like Stretch 00:27:16.579 --> 00:27:17.779 Their Brain More And Think More 00:27:17.779 --> 00:27:18.920 And Figure Out New Things 00:27:18.920 --> 00:27:21.559 To Do And Honestly I 00:27:21.559 --> 00:27:22.740 Struggle To Point 00:27:22.740 --> 00:27:24.900 Any Significant Systemic 00:27:24.900 --> 00:27:26.359 Change That I've Seen In 00:27:26.359 --> 00:27:27.460 The Education System At 00:27:27.460 --> 00:27:30.160 Large Since Chat 00:27:30.160 --> 00:27:31.359 GBT Launch And I Thought 00:27:32.599 --> 00:27:33.619 Would Happened 00:27:33.619 --> 00:27:39.019 So I Have Done 00:27:39.019 --> 00:27:40.099 With Every Other Technological 00:27:40.099 --> 00:27:41.740 Leap Before Redesign 00:27:41.740 --> 00:27:42.960 How Education Works 00:27:42.960 --> 00:27:43.980 So That You Still 00:27:43.980 --> 00:27:45.579 Have To Learn How Think And 00:27:45.579 --> 00:27:46.720 There Will Be Some Things Like 00:27:46.720 --> 00:27:48.519 I Am A 00:27:48.519 --> 00:27:50.519 Person Who Thinks By Writing 00:27:51.019 --> 00:27:52.759 And Write Stuff That 00:27:52.759 --> 00:27:53.799 Is Never Showed 00:27:53.799 --> 00:27:55.079 Anyone Else But It's Important For 00:27:55.079 --> 00:27:56.099 Me To Figure Something Out And So 00:27:56.099 --> 00:27:58.160 I'm Grateful People Say 00:27:58.160 --> 00:27:59.200 The Same Thing About Programming 00:27:59.980 --> 00:28:01.259 So We Teach 00:28:01.740 --> 00:28:03.079 People To Do That Machines 00:28:03.079 --> 00:28:04.259 Can Do Better 00:28:04.259 --> 00:28:05.319 Just Because It's 00:28:05.319 --> 00:28:06.740 Helpful To Teach 00:28:06.740 --> 00:28:08.410 Them The Meta Skill 00:28:08.410 --> 00:28:09.549 Of Thinking And Learning And 00:28:09.549 --> 00:28:10.769 That Makes Sense But There 00:28:10.769 --> 00:28:11.849 Are A Lot Other Things Where 00:28:11.849 --> 00:28:14.890 We Should Totally Change How 00:28:14.890 --> 00:28:16.089 We Teach Or Learn Or 00:28:16.089 --> 00:28:17.549 Evaluate If 00:28:18.839 --> 00:28:19.880 You Don't Do That 00:28:19.880 --> 00:28:20.920 I Think There Will Be Like 00:28:21.579 --> 00:28:22.680 Significant Atrophy In 00:28:22.680 --> 00:28:23.839 People Critical Thinking Skills 00:28:24.559 --> 00:28:25.579 Question Is What Was 00:28:25.579 --> 00:28:26.700 Your Favorite Class And What Would 00:28:26.700 --> 00:28:28.460 Wish You Did When Stanford 00:28:29.430 --> 00:28:30.670 Does Stamford Still Do Intro 00:28:30.670 --> 00:28:31.710 Sims I 00:28:32.059 --> 00:28:33.200 Did Like All The I 00:28:33.200 --> 00:28:34.240 Did Like Three Interest Times 00:28:34.240 --> 00:28:35.700 A Quarter My Freshman Year Like 00:28:35.700 --> 00:28:36.839 And I Loved All 00:28:36.839 --> 00:28:38.059 Of Them They Were Super 00:28:38.059 --> 00:28:39.980 Different Eye 00:28:41.900 --> 00:28:43.420 Looking Back That 00:28:43.420 --> 00:28:45.859 Fact That I Was 00:28:45.859 --> 00:28:47.000 Able To Get Such Abroad 00:28:47.000 --> 00:28:48.059 Exposure To Stuff 00:28:48.059 --> 00:28:49.559 In Have Like A Very 00:28:49.559 --> 00:28:51.019 Shall Understanding Of 00:28:51.019 --> 00:28:52.740 Lots Of Different Fields Was 00:28:52.740 --> 00:28:53.799 An Incredible Thing If 00:28:53.799 --> 00:28:55.019 It Not Been For That I Would Have 00:28:55.019 --> 00:28:56.059 Taken Cs And 00:28:56.059 --> 00:28:57.140 Physics Classes Which Still 00:28:57.140 --> 00:28:58.240 Would Be Great But 00:28:58.240 --> 00:29:01.349 Um I Think 00:29:01.349 --> 00:29:02.390 More About The 00:29:02.390 --> 00:29:03.609 Stuff The Classes 00:29:03.609 --> 00:29:04.750 I Took That 00:29:04.750 --> 00:29:06.089 Were Like Totally Random 00:29:06.089 --> 00:29:07.250 An Unrelated To What 00:29:07.250 --> 00:29:08.589 I Do Now In Some Important 00:29:08.589 --> 00:29:11.029 Way Gave Me A 00:29:11.029 --> 00:29:12.190 Perspective Than I Did 00:29:12.190 --> 00:29:13.890 I Think I Would Of Learned 00:29:13.890 --> 00:29:15.589 Program No Matter What 00:29:15.589 --> 00:29:17.680 So I 00:29:18.579 --> 00:29:19.960 Didn't Think At Time Was 00:29:19.960 --> 00:29:22.170 Like Kind Of You Know It's 00:29:22.170 --> 00:29:23.329 This Stuff Is All Cool 00:29:23.329 --> 00:29:24.710 But Mostly Going To Be About Learning 00:29:24.710 --> 00:29:26.849 Cs I Only 00:29:26.849 --> 00:29:27.910 Did Two Years Of 00:29:27.910 --> 00:29:29.410 School So There Was A Lot Of Stuff That 00:29:29.410 --> 00:29:30.490 Wanted Take That Didn't Get Too 00:29:30.990 --> 00:29:35.420 The Question What 00:29:36.259 --> 00:29:37.299 Is Your Spiciest 00:29:37.299 --> 00:29:38.420 Steak Of All 00:29:43.059 --> 00:29:44.500 I Think With More 00:29:44.500 --> 00:29:45.720 Time To Think I 00:29:46.200 --> 00:29:48.829 Could Come Up With Much Spicer 00:29:48.829 --> 00:29:53.529 One But AI Is 00:29:53.529 --> 00:29:55.250 Just Gonna Keep Going And 00:29:55.279 --> 00:29:57.880 I This Is 00:29:57.880 --> 00:30:00.740 Considered I Don't 00:30:00.740 --> 00:30:01.759 Think This Like Widely 00:30:01.759 --> 00:30:03.019 Believed Yet If It 00:30:03.019 --> 00:30:04.059 Were Widly Believd 00:30:04.059 --> 00:30:05.980 There Would Be Significantly 00:30:05.980 --> 00:30:07.039 More Reverberations 00:30:07.480 --> 00:30:08.640 That Are Happening Through Society 00:30:08.640 --> 00:30:09.940 Right Now Maybe 00:30:10.279 --> 00:30:11.480 I Don't Have A Spicer Take Actually 00:30:11.480 --> 00:30:12.539 Maybe This Is The High Order Bit 00:30:12.539 --> 00:30:13.720 But If AI 00:30:13.720 --> 00:30:15.799 Progress Continues On 00:30:15.799 --> 00:30:16.920 The Exponential That Its One 00:30:17.849 --> 00:30:20.400 For Another Three 00:30:20.400 --> 00:30:21.500 Half Years Since Tragic 00:30:21.500 --> 00:30:29.019 Metaphory Even The That Society 00:30:29.019 --> 00:30:31.039 What Is Capable 00:30:31.039 --> 00:30:32.460 Of Are Just Completely Different 00:30:32.460 --> 00:30:33.619 Well Let Me 00:30:33.619 --> 00:30:35.019 Try To Prompt With More Thinking Tokens 00:30:35.019 --> 00:30:38.259 On That One You Have 00:30:38.259 --> 00:30:39.339 If We Treated 00:30:39.339 --> 00:30:40.779 As A Model Like Frontier 00:30:40.779 --> 00:30:43.000 Model And Do Some Inherent 00:30:43.000 --> 00:30:44.039 Capabilities I Mean 00:30:44.039 --> 00:30:46.109 It Will Try To Elicit People 00:30:46.579 --> 00:30:47.630 Don't Know About For Next 00:30:47.630 --> 00:30:49.450 Two Minutes One Of 00:30:49.450 --> 00:30:50.509 Them Is That Even Post Train 00:30:50.509 --> 00:30:52.150 Now On Even 00:30:52.640 --> 00:30:53.789 Continuously Rled On Opening Eye 00:30:53.789 --> 00:30:55.769 As Well Is The External Feedback 00:30:55.769 --> 00:30:56.789 Loop Of World On What 00:30:56.789 --> 00:30:58.049 Doesn't Work And Works 00:30:58.049 --> 00:30:59.329 In Does Not Work So 00:30:59.329 --> 00:31:00.349 Now If We're Going To Treat You As 00:31:00.349 --> 00:31:02.470 A Prediction Engine For Sec The 00:31:02.470 --> 00:31:04.369 Prompt Is One Are Three 00:31:04.369 --> 00:31:05.569 Most Likely Forks 00:31:05.569 --> 00:31:06.589 Of The Universe You See Over 00:31:06.589 --> 00:31:08.349 Next Ten Years And 00:31:08.349 --> 00:31:10.089 What Is Your Probability 00:31:10.089 --> 00:31:11.250 Assessment On Each Of Those 00:31:11.250 --> 00:31:13.819 That Make Sense One 00:31:13.819 --> 00:31:15.380 It Feels Very 00:31:15.380 --> 00:31:17.930 Important Is Like 00:31:17.930 --> 00:31:19.230 How Much Of This Technology 00:31:19.230 --> 00:31:20.609 Going To Be Very 00:31:20.609 --> 00:31:21.710 Widely Democratized 00:31:21.710 --> 00:31:22.789 Versus How Much It 00:31:22.789 --> 00:31:24.049 Will Sit In 00:31:24.049 --> 00:31:25.359 A Few 00:31:25.359 --> 00:31:26.579 Companies I Think 00:31:26.579 --> 00:31:27.619 Our World There 00:31:27.619 --> 00:31:28.799 Are All These Reasons Why 00:31:28.799 --> 00:31:30.039 You Could Imagine The Default 00:31:30.039 --> 00:31:31.440 Is That This Gets Concentrated 00:31:31.440 --> 00:31:33.240 To A Few Companies And 00:31:33.240 --> 00:31:34.440 They Become Like 00:31:34.440 --> 00:31:35.700 You Know Significant 00:31:35.700 --> 00:31:36.859 Fraction Of Wealth On Earth 00:31:37.559 --> 00:31:39.000 Now Would Obviously Terrible 00:31:39.000 --> 00:31:40.019 And Work Super Hard 00:31:40.019 --> 00:31:49.779 To Push To Kind 00:31:49.779 --> 00:31:50.819 Of 00:31:50.819 --> 00:31:51.940 Utility Model Of The World 00:31:51.940 --> 00:31:53.660 Is That A 00:31:53.690 --> 00:31:54.829 It's Quite Unstable 00:31:54.829 --> 00:31:55.869 And Bad And Will 00:31:55.869 --> 00:31:56.890 Feel Unfair If 00:31:56.890 --> 00:31:58.029 A Few Companies Have All This But 00:31:58.029 --> 00:31:59.170 B I 00:31:59.170 --> 00:32:00.589 Think There Is Real Alignment Failure 00:32:00.589 --> 00:32:01.890 In A Very Fragile 00:32:01.890 --> 00:32:03.029 World And 00:32:03.029 --> 00:32:04.170 The Best Way To 00:32:04.170 --> 00:32:05.250 Get To A World 00:32:05.250 --> 00:32:06.490 We Want That Represents Like 00:32:06.490 --> 00:32:07.670 Everybody Winning And 00:32:07.670 --> 00:32:08.890 Every Values Being Represented 00:32:08.890 --> 00:32:10.269 Having An Agency Is 00:32:10.269 --> 00:32:11.549 Just Push This Technology 00:32:11.549 --> 00:32:13.250 Out Into The World 00:32:13.829 --> 00:32:14.910 But There 00:32:14.910 --> 00:32:16.130 Will Be A Very Strong Argument Against 00:32:16.130 --> 00:32:17.150 That Around Sort 00:32:17.150 --> 00:32:19.069 Of Safety Instability And 00:32:19.529 --> 00:32:20.930 I Think That Would Be Big Fork 00:32:20.930 --> 00:32:21.970 It's Very Important 00:32:21.970 --> 00:32:23.150 Encourage All You 00:32:23.150 --> 00:32:24.210 In Your Careers To Push 00:32:24.210 --> 00:32:25.289 Hard That This 00:32:25.289 --> 00:32:26.430 Is A Technology Can 00:32:27.589 --> 00:32:29.029 Bring Us An Incredible Sci-Fi 00:32:29.029 --> 00:32:31.009 Future Life Can Be Unbelievably 00:32:31.009 --> 00:32:32.190 Much Better We Are Going 00:32:32.190 --> 00:32:33.210 To Incur Some Risk To 00:32:33.210 --> 00:32:34.390 Get There But The Risk 00:32:34.390 --> 00:32:36.390 Of Keeping Us Concentrated 00:32:36.390 --> 00:32:37.589 In A Handful Of Companies Even 00:32:37.589 --> 00:32:38.869 Though We Would Be One These Companies 00:32:39.470 --> 00:32:40.609 Is Not Something You Should Tolerate 00:32:40.609 --> 00:32:42.009 So I Think That Will Be Big Fork 00:32:42.609 --> 00:32:44.109 In Terms Of Probability 00:32:44.109 --> 00:32:47.259 I Think It's The 00:32:47.259 --> 00:32:48.700 World Should Have Such An Interest 00:32:48.700 --> 00:32:50.339 And Happening 00:32:50.339 --> 00:32:52.740 This Way But I Think 80% 00:32:52.740 --> 00:32:54.059 End Up On The Democratic Path 00:32:54.839 --> 00:32:56.140 There Will Be Very Strong 00:32:56.140 --> 00:32:57.420 Safety Message And 00:32:57.900 --> 00:33:09.900 You Have An 00:33:09.900 --> 00:33:11.339 Direction But If That 00:33:11.339 --> 00:33:12.599 Fork Then You Have An Agency 00:33:12.599 --> 00:33:14.059 To Affect The 00:33:14.059 --> 00:33:15.960 Forecasts Well I 00:33:15.960 --> 00:33:17.079 Mean We Are Clear What 00:33:17.079 --> 00:33:18.119 Is Our Agency For 00:33:18.119 --> 00:33:19.140 Like This Is What We Believe 00:33:19.140 --> 00:33:21.200 In We Think That Uh You 00:33:21.200 --> 00:33:22.319 Know We Can Do Everything We Can 00:33:22.319 --> 00:33:23.680 Push It In This Direction 00:33:23.680 --> 00:33:30.329 Maybe Related There's 00:33:30.329 --> 00:33:38.490 A Lot Of Talk About Me Like 00:33:38.490 --> 00:33:41.069 Are We Gonna Capitalism With 00:33:41.069 --> 00:33:42.369 No Change Is It Like Full 00:33:42.369 --> 00:33:43.970 On Communism There's 00:33:43.970 --> 00:33:44.990 A Lot Of Talk About This 00:33:44.990 --> 00:33:46.089 One Thing That I Think 00:33:46.089 --> 00:33:47.190 Has Not Talked 00:33:47.190 --> 00:33:49.240 About Much Is How 00:33:49.240 --> 00:33:50.279 Specifically How 00:33:50.279 --> 00:33:52.319 We Distribute Computes So 00:33:52.839 --> 00:33:53.940 Maybe Alot Of The Economy 00:33:54.420 --> 00:33:56.269 Can Work In A 00:33:56.269 --> 00:33:57.450 Way That Its Going To Work And Actually I've 00:33:57.450 --> 00:33:59.130 Become Much Less Even Short-Term 00:33:59.130 --> 00:34:00.230 Jobs Doomer Have 00:34:00.230 --> 00:34:01.329 Always Been Optimistic We Find 00:34:01.329 --> 00:34:03.170 New Things To Do But This 00:34:03.170 --> 00:34:04.910 May Not Be As Disrupted 00:34:04.910 --> 00:34:05.930 Is I Originally Thought In The 00:34:05.930 --> 00:34:08.750 Short-Term But We 00:34:08.750 --> 00:34:09.989 Are Seeing Compute Shortages 00:34:09.989 --> 00:34:11.769 Now I Can Imagine 00:34:11.769 --> 00:34:13.150 Them Getting Much Worse And 00:34:13.150 --> 00:34:14.489 I Can Imagine Compete 00:34:14.489 --> 00:34:16.030 Being Like The Most Important 00:34:16.030 --> 00:34:17.469 Utility That People Need 00:34:17.909 --> 00:34:18.949 So If The 00:34:18.949 --> 00:34:20.030 Price Of Computes From A 00:34:20.030 --> 00:34:21.190 Supply Demand Perspective 00:34:21.190 --> 00:34:22.849 Gets Way Out Of Whack Then 00:34:22.849 --> 00:34:24.070 There Will Be An Interesting 00:34:24.070 --> 00:34:26.000 Fork About What 00:34:26.000 --> 00:34:27.280 It Means To Equitably Distribute 00:34:27.280 --> 00:34:28.940 Compete So You Did 00:34:28.940 --> 00:34:30.000 Do Very Interesting Things 00:34:30.000 --> 00:34:31.119 There Which He Said On 00:34:31.119 --> 00:34:32.280 The Economic Side 00:34:32.940 --> 00:34:34.260 We Might Have Need Universal 00:34:34.260 --> 00:34:36.139 Basic Income Everybody Wants 00:34:36.139 --> 00:34:37.239 A Piece Of Shares You 00:34:37.239 --> 00:34:38.460 Know One Of Speakers In This Class 00:34:38.460 --> 00:34:41.230 Is Nicolai 00:34:41.230 --> 00:34:42.349 Tangent Who Runs 00:34:42.349 --> 00:34:43.389 The Norwegian Sovereign Well Fund 00:34:43.389 --> 00:34:44.670 These Awesome His Awesome 00:34:45.369 --> 00:34:46.469 Yeah That's No Region Sovereig 00:34:46.469 --> 00:34:47.869 And Weld Fun Owns 1.5% 00:34:47.869 --> 00:34:48.949 Of All Public Trade Companies 00:34:48.949 --> 00:34:50.010 On The Planet They 00:34:50.010 --> 00:34:51.210 Also Have Effectively Universal 00:34:51.210 --> 00:34:52.250 Basic Income You 00:34:52.250 --> 00:34:53.329 Could Argue There's Flavors Of This 00:34:53.329 --> 00:34:54.349 Already Today Because 00:34:54.730 --> 00:34:56.170 You Know The Largest Employer 00:34:56.170 --> 00:34:57.289 Now In United States Is The Government 00:34:57.750 --> 00:34:58.809 And You Can Argo Like 00:34:58.809 --> 00:34:59.969 Large Sections Of That Or 00:34:59.969 --> 00:35:01.050 A Way For The Government 00:35:01.050 --> 00:35:02.210 To Redistribute Income From 00:35:02.210 --> 00:35:03.809 Taxpayers So Are These 00:35:03.809 --> 00:35:04.929 Solutions That 00:35:04.929 --> 00:35:06.269 Actually Need Be Novel 00:35:06.269 --> 00:35:08.289 Or Just Reimplemented For 00:35:08.289 --> 00:35:09.309 This Era How 00:35:09.309 --> 00:35:10.369 Do You Think About The Novelty Of 00:35:10.369 --> 00:35:11.869 Those Solutions Where We Often 00:35:12.289 --> 00:35:13.730 In Silicon Valley Have 00:35:13.730 --> 00:35:15.829 A Tendency To Be Like Reinvent 00:35:15.829 --> 00:35:16.909 Old Things 00:35:16.909 --> 00:35:18.429 From First Principles And 00:35:18.429 --> 00:35:19.730 So Should Look At Existing 00:35:19.730 --> 00:35:20.889 Systems And Tweak Them I 00:35:20.889 --> 00:35:22.409 Don't Think That These Require 00:35:23.550 --> 00:35:24.610 Deeply New Ideas 00:35:24.610 --> 00:35:25.730 Although I Will Say 00:35:26.489 --> 00:35:28.840 I Am 00:35:28.840 --> 00:35:29.980 Much More Excited 00:35:29.980 --> 00:35:31.159 About People Having 00:35:31.159 --> 00:35:39.099 Some Sort Of Funded 00:35:39.099 --> 00:35:40.239 Like A Big Universal 00:35:40.239 --> 00:35:41.280 Basic Income Study A While 00:35:41.280 --> 00:35:42.380 Ago I Have 00:35:43.090 --> 00:35:44.260 Also Watch What Happens 00:35:44.260 --> 00:35:45.320 When People Invest 00:35:45.320 --> 00:35:46.519 In Startups 00:35:46.519 --> 00:35:47.679 And Know Which Model 00:35:47.679 --> 00:35:48.820 Hits Human 00:35:48.820 --> 00:35:50.260 Psychology Better So 00:35:50.260 --> 00:35:51.320 What I Would Love To 00:35:51.320 --> 00:35:54.480 See Is As Leverage 00:35:54.480 --> 00:35:55.699 In The World Shifts From 00:35:56.429 --> 00:36:00.099 Labor To Capital That We 00:36:00.099 --> 00:36:01.800 Find A Way To 00:36:02.400 --> 00:36:03.760 Have Something Like Citizens Well 00:36:03.760 --> 00:36:04.780 Fun Than The Country 00:36:04.780 --> 00:36:06.380 Or In The World Eventually Where 00:36:07.059 --> 00:36:08.239 You Like Basically 00:36:08.239 --> 00:36:09.280 Own A Slice Of 00:36:09.280 --> 00:36:10.400 Capitalism Right On The Slices 00:36:10.400 --> 00:36:11.480 These Companies And 00:36:11.480 --> 00:36:12.579 Then In The Second Fork There 00:36:12.579 --> 00:36:13.579 Compute 00:36:13.579 --> 00:36:15.820 Bottlenecks At Some 00:36:15.820 --> 00:36:16.880 Point When Computes Prices Get 00:36:16.880 --> 00:36:18.500 Out Of Whack Between January 00:36:18.500 --> 00:36:20.480 This Year My Current 00:36:20.480 --> 00:36:21.519 Understanding Is Based 00:36:21.519 --> 00:36:22.940 On Data We Have Seen That 00:36:22.940 --> 00:36:24.360 H100 Price And 00:36:24.360 --> 00:36:26.699 Blackwell Spreads 00:36:26.699 --> 00:36:27.860 Between Long-Term Reservations 00:36:27.860 --> 00:36:29.360 And Spot Is 5X 00:36:29.820 --> 00:36:30.920 I Don't Know If It's That 00:36:30.920 --> 00:36:32.139 High Anymore I Think 00:36:32.139 --> 00:36:33.219 Got A Little Better But Yeah 00:36:33.219 --> 00:36:34.940 Time Or If You 00:36:34.940 --> 00:36:35.960 Can Even Find H100 Because 00:36:35.960 --> 00:36:37.099 They're Pretty Much All Gone 00:36:37.099 --> 00:36:38.199 For This Year Does Sound 00:36:38.199 --> 00:36:40.380 Right No Argument 00:36:40.380 --> 00:36:42.159 There Is A Gigantic Computer Shortage 00:36:42.159 --> 00:36:43.539 Out So The That 00:36:43.539 --> 00:36:44.579 Is Good Example Of An 00:36:44.579 --> 00:36:45.920 Other Systems Problem 00:36:45.920 --> 00:36:47.039 Right Now That Live 00:36:47.559 --> 00:36:48.619 At Least To Some 00:36:48.619 --> 00:36:49.780 Folks It Feels Like Covid 00:36:50.380 --> 00:36:51.460 You Know For The 00:36:51.460 --> 00:36:52.539 Compute Era Like All The 00:36:52.539 --> 00:36:55.079 Toilet Papers Gone Yeah Why 00:36:55.079 --> 00:36:56.219 Are People Not Freaking 00:36:56.219 --> 00:36:57.920 Out About This Well 00:36:57.920 --> 00:36:58.980 I Think People Assume We 00:36:58.980 --> 00:37:00.000 Will Make Big 00:37:00.000 --> 00:37:01.099 Inference Gains On Hardware 00:37:01.099 --> 00:37:02.179 We Have I Also 00:37:02.179 --> 00:37:03.420 Think There Is A Tsunami 00:37:03.420 --> 00:37:04.940 Of Hardware Coming But 00:37:05.460 --> 00:37:06.539 Maybe That Demand 00:37:06.539 --> 00:37:07.639 Tsunamis Even Bigger And Pete 00:37:07.639 --> 00:37:08.780 I Think People Should Be Freakin Out 00:37:08.780 --> 00:37:10.219 Somewhat And Would 00:37:10.219 --> 00:37:11.519 You Say It's Fair How 00:37:11.519 --> 00:37:12.699 Long Exist 00:37:12.699 --> 00:37:14.179 In A Compute Shortage At 00:37:14.530 --> 00:37:15.750 Least You 00:37:15.750 --> 00:37:17.349 Know Based On Current Data 00:37:17.349 --> 00:37:20.380 You Have I Think Like 00:37:20.380 --> 00:37:22.190 Other You Can't 00:37:22.190 --> 00:37:23.289 Talk Really About Like 00:37:23.289 --> 00:37:24.889 Worldwide Demand For Electricity 00:37:24.889 --> 00:37:25.909 Without Talking About 00:37:25.909 --> 00:37:27.190 The Price Like 00:37:27.190 --> 00:37:29.289 It's There'S An Extremely 00:37:29.289 --> 00:37:30.530 Different Demand About 00:37:30.530 --> 00:37:31.670 How Much Energy People Use 00:37:31.670 --> 00:37:32.869 In The World If The Price Comes Down 00:37:32.869 --> 00:37:33.889 By Factor Of 10 Or Goes 00:37:33.889 --> 00:37:36.030 Up By Factor Of Ten And 00:37:37.780 --> 00:37:38.900 Ai Is Like That 00:37:40.139 --> 00:37:41.820 The If 00:37:42.300 --> 00:37:43.360 We Can Make 00:37:44.239 --> 00:37:47.719 Models Sufficiently 00:37:47.719 --> 00:37:48.980 Smart And 00:37:48.980 --> 00:37:50.219 It'S A Sufficiency Low Cost 00:37:50.219 --> 00:37:51.360 I Think Demand 00:37:51.360 --> 00:37:52.400 Is Like Kind Of Uncapped 00:37:52.400 --> 00:37:53.559 And So In Some 00:37:53.559 --> 00:37:54.619 Sense As Long As We 00:37:54.619 --> 00:37:55.960 Continue To Make Progress 00:37:55.960 --> 00:37:57.000 On This There Will Be A Shortage 00:37:57.000 --> 00:37:58.639 Forever And Things Will 00:37:58.639 --> 00:38:00.179 Be Bid Among Above 00:38:00.550 --> 00:38:02.019 What The Price Should 00:38:03.250 --> 00:38:04.280 Be Even Though 00:38:04.280 --> 00:38:06.199 People Are Getting Better Smarter 00:38:06.199 --> 00:38:07.960 More Whatever Intelligence Just 00:38:07.960 --> 00:38:13.119 Because You Can Use A 00:38:13.119 --> 00:38:14.219 You Can Have 10 Of 00:38:14.219 --> 00:38:15.440 Them Running Or Working For 100 00:38:15.920 --> 00:38:19.510 All The Think It'S 00:38:19.510 --> 00:38:20.969 A Lot Of Inference That Will 00:38:20.969 --> 00:38:22.070 Conclude Awesome With 00:38:22.070 --> 00:38:23.090 That I'm Give You The Swag 00:38:23.090 --> 00:38:28.900 Which Is Thank You 00:38:28.900 --> 00:38:29.900 For Coming Thank You