0:00 An eighty-billion parameter Qwen model is now running on a Mac using just four-point-three gigabytes of RAM.0:06 Last week, we talked about the big league performance of models like Qwen3.8-Max and the hardware headaches they cause — well, this week, a project called Swiftlet just flipped that entire script.0:19 Suddenly the question isn't what monster server you need to run AI, but what you can do with the supercomputer already sitting on your desk.0:27 This is the story of the week, maybe the month.0:30 It’s about the tools getting impossibly small, impossibly accessible.0:35 And it forces a much bigger question that’s all over the place right now: as the tools get better, what is the human's job?0:42 But first, let's sweep the rest of the landscape, because everything seems to be pointing in the same direction.0:49 While you were trying to get a 7-billion parameter model to run without crashing your laptop, OpenAI dropped a paper detailing how their Astra model solved ten, TEN, major open problems in mathematics and theoretical computer science.1:04 These aren't just puzzles.1:06 We're talking about disproving longstanding conjectures, things mathematicians have been stuck on for decades.1:13 And the total compute cost to do it?1:15 Roughly two thousand dollars.1:17 That's it.1:17 For a discovery that would get a human a tenured position for life.1:22 Then you have Cloudflare.1:23 They’re running some of the biggest models out there, like Kimi, at a scale you can't even imagine.1:29 And this week they announced how they're making it faster and cheaper.1:34 They're doing something called quantizing the KV cache — basically, shrinking the memory footprint of the model's "short-term memory." They went from 16-bit to 8-bit floating point numbers.1:46 The result?1:46 They can now fit double the context into the same amount of VRAM.1:50 For the Kimi K2.6 model, that means going from about 686,000 tokens of context to almost one-point-four MILLION.1:57 This isn't just a small tweak.1:59 It means they can handle twice as many users at once, boosting throughput by forty-one percent.2:05 It's the plumbing, the deep, unglamorous engineering that actually makes this AI revolution possible for everyone else.2:13 And in the middle of all this high-tech, high-speed progress, the community took a moment to celebrate something...2:20 slow.2:20 The document converter Pandoc just had its twentieth anniversary.2:24 If you've ever needed to turn a Markdown file into a PDF, or a Word doc into a webpage, you've probably used it, maybe without even knowing.2:33 Its creator, John MacFarlane, wrote a post about how it started in 2006 as a three-thousand-line Haskell program.2:40 It's a reminder that some of the most essential tools in our world aren't flashy.2:45 They're just reliable.2:47 They’re built on a solid architecture — in this case, an abstract syntax tree — that just works, year after year.2:54 Finally, in a moment of almost poetic contrast, a story from 1950 started trending.2:59 Ray Bradbury's "There Will Come Soft Rains." It's about a fully automated house that just...3:05 keeps going.3:05 Making breakfast, cleaning, reading poems to an empty living room after the family, and all of humanity, is gone.3:13 The story ends with the house, the last functioning thing on Earth, finally succumbing to a fire, whispering the date over and over into the dawn.3:22 Yeah.3:22 It’s a heavy read, especially this week.3:25 So.3:25 Let's pull on the main thread here.3:27 You have models that can run on your phone.3:30 You have models solving problems that stumped the smartest humans alive.3:34 And you have the engineering to make it all run at planetary scale.3:38 What does it all add up to?3:40 It adds up to a massive, sprawling, and frankly, confusing debate that hit the top of Hacker News this week, sparked by a single, provocative sentence: "The most important skill in prompting is expertise in the domain you’re prompting for." Now, at first, that sounds obvious, right?3:58 Of course a doctor will be better at getting medical advice from an AI than a random person.4:04 But the debate is SO much deeper than that.4:06 For the last two years, we've been obsessed with "prompt engineering." The idea that you could learn a secret language, a set of magic words to unlock the AI's power, regardless of what you were asking about.4:19 It created a whole cottage industry of prompt gurus and courses.4:23 This discussion says that was a mirage.4:26 A temporary phase.4:27 The real, durable skill isn't the prompting; it's the knowing.4:31 The best example in the thread is a breakdown of how Fields Medalist Terence Tao talks to ChatGPT.4:37 He was probing it about a potential counterexample to the Jacobian Conjecture.4:42 His prompts are...4:43 nothing.4:44 They're short.4:44 Terse.4:45 He doesn't use any fancy formatting.4:47 He's not telling it to "act as a world-class mathematician." He just starts talking to it like one.4:53 And the model's behavior completely changes.4:56 As the post's author puts it, by signaling his own expertise, Tao "shunts the model into ‘talking-to-mathematicians’ mode, not ‘explaining-to-amateurs’ mode." The AI stops hedging.5:07 It stops over-explaining basic concepts.5:10 It becomes a collaborator, not a tutor.5:12 When the model makes a mistake, Tao doesn't get frustrated.5:16 He just points it out, bluntly.5:18 "That seems wrong." And the model corrects.5:21 He's not engineering a prompt; he's conducting a peer review.5:24 This is where we've seen this pattern before.5:27 Think about the arrival of CAD software in architecture and engineering.5:32 For a while, the hot skill was just knowing how to use the software.5:36 Being a "CAD jockey." But that didn't last.5:39 Soon, every architect and engineer knew how to use it.5:42 The software became invisible, just part of the job.5:45 And the value returned to where it always was: in the deep knowledge of structural engineering, of materials, of aesthetics.5:53 The tool didn't replace the expert; it supercharged them.5:57 It took away the tedious drafting and freed them up to think about the actual architecture.6:02 That's what seems to be happening right now with LLMs.6:06 The "prompt engineering" phase is just the "learning the CAD interface" phase.6:11 It's temporary.6:12 The real, lasting value is in domain expertise.6:15 And this connects directly back to those OpenAI math breakthroughs.6:19 OpenAI didn't just turn on Astra and find ten new theorems.6:23 The process was a partnership.6:24 Human mathematicians and computer scientists chose the problems.6:28 They guided the AI.6:30 They used their expertise to recognize which of the AI's outputs were promising and which were dead ends.6:36 The AI was a phenomenal, tireless, and occasionally brilliant research assistant.6:41 But it needed a principal investigator.6:44 It needed an expert.6:45 The paper says the solutions were accompanied by "narrated reasoning walkthroughs." That's the key.6:51 The AI can find a pattern, a path through a trillion possibilities that a human would never have the time to explore.6:59 But a human expert is still required to understand why that path is the correct one, to narrate its logic, and to verify its conclusion.7:07 Without that verification—without the human expert in the loop—the AI’s answer is just a high-tech "trust me, bro." So where does this leave us with projects like Swiftlet, the one that lets you run an 80-billion parameter model on your Mac?7:22 It's not just a cool party trick.7:24 It represents a fundamental shift in access.7:27 This is the desktop publishing revolution all over again.7:31 Before the Mac and PageMaker, if you wanted to publish something, you needed a printing press, a typesetter, a whole industrial setup.7:39 It was expensive and slow.7:41 Then, suddenly, one person with a computer could do it all.7:44 It didn't eliminate the need for good writers or designers—in fact, it made them MORE valuable because now their work could be distributed instantly.7:54 It just destroyed the value of owning a printing press.7:57 Swiftlet, and projects like it, are doing the same for AI.8:01 They are destroying the value of simply having access to a massive GPU cluster.8:06 That's becoming a commodity.8:08 What's NOT a commodity is what you do with it.8:10 When anyone can run a powerful model locally, the advantage shifts from those who have the hardware to those who have the best questions.8:19 The advantage shifts to the expert.8:21 The biologist who can use it to simulate protein folding on her laptop.8:26 The lawyer who can review ten thousand documents on a flight.8:30 The programmer who can have an AI pair-programmer running locally, with zero latency, that understands the entire context of their private codebase.8:39 The fear for the last couple of years has been that AI would turn us all into mediocre generalists.8:45 That deep expertise would be devalued.8:47 This week's developments suggest the exact opposite is happening.8:52 The tools are becoming so good and so accessible that they are fading into the background.8:57 And what’s left is the human.8:59 The expert.9:00 The one who knows which questions to ask, which answers to trust, and what it all means.9:05 So what does this week set up?9:07 We're about to see an explosion of expert-driven AI applications.9:11 Not giant, generic chatbots, but small, specialized tools built by people who deeply understand their own fields.9:18 The focus is shifting from building bigger models to building better workflows.9:23 It's a move from brute force to intellectual leverage.9:27 The new scarcity isn't compute; it's judgment.
