TWiT+ Club Shows 752 Transcript - AI User Group #16
Please be advised that this transcript is AI-generated and may not be word-for-word. Time codes refer to the approximate times in the ad-free version of the show.
Leo Laporte [00:00:00]:
This is twit. So let's say hello to everybody. You're welcome to our monthly AI user group. No agenda, we just get together and talk. That's Darren Okey in the lower left quadrant. That's Larry Gold in the lower right quadrant. In the upper right quadrant, we've got error 404 in town. Bill, good to see you.
Larry Gold (LrAu) [00:00:26]:
Hi Bill.
Leo Laporte [00:00:27]:
You were here last month, right?
Bill (error_404_new) [00:00:29]:
Yeah, I've been here a couple times. Yeah.
Leo Laporte [00:00:31]:
And then I think Craig, you're new.
Larry Gold (LrAu) [00:00:34]:
It was her last week, last month.
Leo Laporte [00:00:36]:
So I'm just. Yeah, I apologize. Welcome back. It says cto. What are you the CTO of?
Craig McFarlane (CraigM) [00:00:49]:
I have a company that we specialize in. It's software company. We specialize in large IT transformation.
Leo Laporte [00:00:56]:
Nice. Do you use AI to do that?
Craig McFarlane (CraigM) [00:01:01]:
Starting to.
Leo Laporte [00:01:02]:
I feel like DevOps is going to completely change thanks to these code.
Craig McFarlane (CraigM) [00:01:07]:
Most enterprises are slow to adopt operational AI.
Leo Laporte [00:01:12]:
Right. Oh, I understand. Right. I was, you know, I was very scared. This brain surgery I was doing, I had it, you know, do a big plan and we went through. I said, you gotta. This will kill you. This will not only kill me, it will kill you if you fail.
Leo Laporte [00:01:27]:
So be very, very careful. And it' actually done a really good job step by step going through it. I've done a bunch of stuff to this Hermes agent over the last few weeks to really make it robust. Jeffrey Cannell, who's the founder, so to speak, of Noose Research, will be joining us Wednesday on Intelligent Machines for a second time. When he joined us last time, he was talking about the models they were doing, but he was being cagey because they, they had this agentic harness in the background that they were using internally that I hadn't told anybody about. And at some point they looked at openclaw and said, you know, Hermes is better, we should release it. And they have and it really is kind of fantastic. So I've been.
Bill (error_404_new) [00:02:11]:
I think one of the things they did really well there was, was the portability. Right. Like just a command to migrate from openclaw to Hermes. And now, you know, Anthony just posted that thing from PewDiePie, like I hope we get some standards here. Right. Markdown is great because it's not compilable code, it's just transferable. But like the portability between all these tools because they're changing so fast, I think is going to be a differentiator too.
Leo Laporte [00:02:35]:
I moved over from a very elaborate Claude code thing I've been working on for six months. I just moved most of the skills over. But what I found with Hermes is it's got much stronger memory hooks. I actually have four different memory tools running. It's got its basic memory md, but it also has, I have LLM wiki running. I have something called Hindsight running. It has a hook for that. And so it was easy to implement that.
Leo Laporte [00:03:00]:
That's a SQLite database with semantic search. And I have, I have Nate B. Jones Open Brain running on it. So I have four different memory systems, but it's quite good. And, and they don't too much.
Bill (error_404_new) [00:03:13]:
Is there any, like, overfitting with.
Leo Laporte [00:03:15]:
Well, I, you know, I asked it, I said, you know, I don't want to install anything you don't need and let's not. But they, they kind of all four do different things. Memory MD is very small. It's actually by default 4k. I expanded it to 6k, but I didn't want to eat up too much context, especially since Quinn is like ridiculously small amount of context. In fact, it's already compacted. I'm gonna, I'm gonna turn it off. I've been using Deep Seek a lot with it.
Leo Laporte [00:03:41]:
Um, but I have a deep seek 4 is really good.
Bill (error_404_new) [00:03:46]:
I was impressed.
Leo Laporte [00:03:47]:
Yeah, I would say especially for what,
Bill (error_404_new) [00:03:49]:
like a dollar 34 for 14 cents
Leo Laporte [00:03:52]:
in and 8 and something like 30 cents out.
Bill (error_404_new) [00:03:55]:
Yeah, really cheap.
Larry Gold (LrAu) [00:03:57]:
For how long?
Leo Laporte [00:03:59]:
They said till May 31. And then they said, no, we're going to do it forever.
Juan Hernandez (BlindWiz) [00:04:03]:
How much was that that you just said? What was the time spent that you paid for that? Like how many tokens or.
Leo Laporte [00:04:08]:
No, no, no, it's per million. It's per token.
Larry Gold (LrAu) [00:04:11]:
Per million. Per million in, per million out.
Leo Laporte [00:04:13]:
So it's just as you go. But I've spent like a couple of bucks. Yeah, no, it's nothing.
Bill (error_404_new) [00:04:19]:
Especially if you have a better orchestrator on the front end. Right? Like 4.7 or something as well.
Leo Laporte [00:04:24]:
Hermes, actually, Hermes does a really good job of delegation. It also has something called Open Hands. So you can say if you're going to do an image, use Gemini. If you're going to do voice, use Grok if you're going to do this. So for coding, you can use Claude. And like I said, it'll go out to Claude code. I'm getting less and less enamored of anthropic. Honestly, you know, I have to use Claude code if I'm going to use Opus 48 or 7 or 6 with my sub.
Leo Laporte [00:04:53]:
And so there is a standard. It's the Open ap, OpenAI API and that's what this follows. Is what openclaw follows. So it isn't hard to move over from openclaw because it's OpenAI and this is OpenAI. As is open code, as is PI as. Almost everything except Claude code uses the OpenAI API. So I'm less and less enamored of quad code. I think ChatGPT is fine.
Bill (error_404_new) [00:05:21]:
Yep. I was going to say I've been using 5.5 and 5.3 Spark to do my coding because they kind of break out those buckets on the subscription. Good enough?
Leo Laporte [00:05:31]:
Yeah, it's good enough.
Larry Gold (LrAu) [00:05:32]:
I would say if you look at the new Deep SWE benchmark, which is really a nice benchmark if you've read into it and we spent a lot of time with it and I think I came out last time, we kind of proved it. We ran a side to side comparison at that time was 5.5 versus 4.7 and 5.5 was much better at building a couple applications. We have a couple reference applications that we like to test on again because they're not outside of sitting in GitHub that somebody could use. We were finding some of the same things that everyone else found, which is it misses some requirements. If you ask for synchronous and asynchronous, it'll do one or the other. It won't do both. Then for some security principles, it missed some standard security principles. At least Claude was doing it.
Larry Gold (LrAu) [00:06:19]:
I actually did this live in a team meeting because people were clamoring to get Claude code and now people are less clamoring to get Claude coded. They understand that, hey, GitHub Copilot 5.5 is actually really, really good. I think people should understand GPT 5.5. It was GitHub Copilot using GPT 5.5. Yeah.
Leo Laporte [00:06:40]:
Oh, using GPT 5.5.
Larry Gold (LrAu) [00:06:41]:
GPT 5.5.
Darren Oakey [00:06:43]:
This is the thing I'm trying to work out. It's starting to get to a point where you can't really tell and it's quite an interesting thing of what's coming from the model and what's coming from the harness. Because. And I keep meaning to do a. I want to try cloud code against, you know, 5.5 and. And vice versa, Codex versus versus cloth. Because it's really not apparent anymore what, what's kind of where, where the magic is coming from.
Larry Gold (LrAu) [00:07:16]:
Yeah. I feel like we did actually run VS GitHub copilot against 4, 7 also for the same stuff and it still was deficient in certain areas. Right. So we kind of understand some of it. And I think one of the things that we keep telling people is you really need a good, whether it's an agent MD or cloud MD file, right? And you need good specifications. Those are really the driving forces behind a lot of stuff. And I think when we quiz some people, you look at their cloud MD files and they're either too large or they're too random. They're really not specific.
Larry Gold (LrAu) [00:07:51]:
Those are some things that we continue to try to stress on people.
Bill (error_404_new) [00:07:54]:
I've been really leveraging the profiles in Hermes and setting up different profiles for each of my roles. And so I'm just replicating the SDLC in profiles and roles, right. I've got somebody making specs and then they create an issue and then I have an agent go out execute that issue. I have a qa, qa, that pr. It's just standard sdlc.
Darren Oakey [00:08:16]:
That's my entire role at the moment. I've moved to a new company, boring stack that AI. And my entire job is to put traditional DevOps and SDLC around AI.
Bill (error_404_new) [00:08:28]:
And it's nice, like the DevOps stuff, you can write all those checks in there, right? Like deploy and do validation and it can find an error and recommend, you know, create an issue and run it back through the sdlc. Right. We're getting to a lot of self iteration.
Leo Laporte [00:08:43]:
One of the reasons I've been using profiles, Lisa saw me playing with Hermes and said, I want that. So I gave her a profile. I said, what do you want to call your profile?
Bill (error_404_new) [00:08:55]:
She said, rosie, my wife is not technical. And she's like, this is the first one that she's been interested in.
Leo Laporte [00:09:02]:
Well, it's very much so. She used Perplexity and I just said this is better than Perplexity because it's going to have local memory and it's going to have all the skills. So I gave her all the skills I'd built. One of the nice things about Hermes is it builds its own skills. So, so that, I mean that's by itself, that's fantastic. You don't have to explicitly say.
Bill (error_404_new) [00:09:19]:
And I feel like the reduction of friction of using a communication platform that we're already using.
Leo Laporte [00:09:26]:
Right.
Bill (error_404_new) [00:09:26]:
Was a no brainer, right? Like she didn't have to go to a website, create an account. It's just like in our chat now and I can be in a chat with like, I didn't even have to
Leo Laporte [00:09:33]:
put her on tailscale. I mean I use it through tailscale but it's on the lan. So I just, I said, you know, go to my you know, dot nine and port 3,000. She has her own port 3,100. But it's got all. But it's her memory, it's. Everything is personalized to her. So I think I haven't tried it with projects.
Leo Laporte [00:09:54]:
I think that's an interesting. I mean they also have workspaces. Hermes is really designed around a lot.
Bill (error_404_new) [00:09:58]:
There's a lot of features. My new favorite one is btw. You guys use that. So slash BTW and you can start a thread with the same agent in the same conversation.
Juan Hernandez (BlindWiz) [00:10:10]:
But it's flowing. Yeah.
Bill (error_404_new) [00:10:13]:
So if I find another issue that's not related to the ticket I'm working on, btw, create an issue.
Leo Laporte [00:10:17]:
Claude has that too. Hermes has steer queue, BTT has all of these side commands. And I set up. So I'm using web ui which I honestly think this is incredible. I start with.
Bill (error_404_new) [00:10:30]:
Because the green one is terrible.
Leo Laporte [00:10:32]:
Yeah, I changed the theme. But what's nice. So I'll show you. I wanted to show some people, show you this a little bit. This is really nice. First of all, it's a. It's web based. Easiest way to set it up is to use Chrome with its app switch so that it looks like, you know, it's a chromeless window.
Leo Laporte [00:10:49]:
It looks like an app. And for Lisa it looks just like Perplexity. But there's all sorts of stuff under the hood. For instance, this is amazing. These are all the sessions I've done and if I want to go back to a session, it just loads it.
Larry Gold (LrAu) [00:11:03]:
Yeah.
Leo Laporte [00:11:04]:
And loads the context. And I'm now here and so it's got all the previous sessions. I'm in the middle of a session where I'm like I said, doing the brain surgery. So I'll let it continue with that. But these are all the sessions, these are the skills. I'm sorry, these are cron jobs and I love it because they're not cron jobs. It's all systemd, but it uses the Cron syntax so it makes geeks kind of comfortable. But this is, for instance, a cron job.
Leo Laporte [00:11:32]:
I run weekly looking at all the models out there and benchmarking and looking at what people have done. People have done with benchmarks so that I can then do the delegate. So it updates the delegation every week with the best tools for various things. So these are all cron jobs which you can totally control either this way or of course with a prompt. This is the Kanban. It's got a built in Kanban board or as my friend says, Kanban look, it comes with, out of the box, literally dozens of skills, including things like Obliteratus, if you want to jailbreak your model and stuff. It's got a wild number of skills, but then it also learns skills. So, you know, all my skills are in here that I've created as well, because if you ask it to do something, it'll make a skill out of it.
Leo Laporte [00:12:24]:
And these don't. You know, I saw somebody on YouTube say, oh, man, I think Alex Finn said, turn off these skills. They're taking up so much context. No, they don't take up any minor amount of context. It's not an MCP server. It's just a skill. So it's there on as text on the. On the drive.
Leo Laporte [00:12:43]:
You can have all of this stuff available. Use. Oh, I'm in Lisa's profile. Whoops. I should get out of that. It has spaces which I haven't even played with, but those are, I think, another way to do workspaces, kind of. These are the different profiles. Oh, no, I'm in Quicksilver.
Leo Laporte [00:13:01]:
It was just showing me Lisa's for some reason. I should check that. That doesn't look right. Tasks to DOS Insights logs. You can see the whole logs. And so this is in a web browser. I just like it better than a command line because I, for instance, I can edit a prompt. I can.
Leo Laporte [00:13:16]:
I can go back to a prompt. Yeah, you could choose the profile here. You can attach. You can send it attachments here. Just. Just, you know, like a normal web browser. You can actually choose your workspace. You could choose your model directly from the model picker.
Leo Laporte [00:13:32]:
And in the middle, you could change it from turn to turn if you want. You could change thinking. It's got a lot of little features hidden away.
Darren Oakey [00:13:42]:
Codex and Claude have almost identical UIs. So it's weird that everybody's going the same way.
Bill (error_404_new) [00:13:49]:
They're converging, everybody's making the same thing.
Craig McFarlane (CraigM) [00:13:52]:
The new official Hermes desktop app pretty much is the same thing also.
Leo Laporte [00:13:56]:
The difference is you have to put it on the machine that has Hermes. Right.
Craig McFarlane (CraigM) [00:14:00]:
No, you can't.
Larry Gold (LrAu) [00:14:01]:
No, no, no.
Leo Laporte [00:14:01]:
You can point it.
Craig McFarlane (CraigM) [00:14:02]:
Okay.
Larry Gold (LrAu) [00:14:02]:
No, it's completely standalone because when I
Leo Laporte [00:14:05]:
try to install it, it said no.
Craig McFarlane (CraigM) [00:14:07]:
So. Okay.
Leo Laporte [00:14:08]:
Because I tried to install my Mac, it says you don't have Hermes installed.
Craig McFarlane (CraigM) [00:14:10]:
Yeah, just give it. Just get past that. And then in Settings, there's a remote thing.
Darren Oakey [00:14:17]:
Yeah, yeah.
Leo Laporte [00:14:19]:
I mean, the nice thing about this having a web interface is it works over tailscale. It works on my phone, it works on my iPad I don't have to install.
Larry Gold (LrAu) [00:14:25]:
Yeah.
Bill (error_404_new) [00:14:25]:
What's the advantage of a desktop app giving it. Giving it access to your local files?
Leo Laporte [00:14:31]:
Well, I have all that in the web version. I'm not sure what I'm.
Bill (error_404_new) [00:14:33]:
I like a partition. I don't trust it enough to like.
Larry Gold (LrAu) [00:14:35]:
Yeah, I think.
Juan Hernandez (BlindWiz) [00:14:36]:
I think the desktop app will allow them to do, like, UI automation, like screen UI automation, like controlling your mouse and stuff.
Leo Laporte [00:14:44]:
That makes sense because you.
Juan Hernandez (BlindWiz) [00:14:45]:
A web app doesn't have the.
Leo Laporte [00:14:48]:
Can't do that.
Bill (error_404_new) [00:14:49]:
I think they do have, like, headed mode, a desktop mode on the server, like on Hermes, where you. Because I've seen where you can, like, enable remote desktop. So I don't know. Personally, I'd feel more comfortable giving it its own desktop than giving it access to my desktop.
Leo Laporte [00:15:06]:
Well, and you can put the. I'm sure you could do this with the app, but you can do that with Docker for the web ui. So if you want to isolate it, you can put it in a container.
Darren Oakey [00:15:13]:
But it also depends what you're doing, because if you're coding, then, yeah, having a nice remote isolated desktop is fine. But as Leah said many times, I'm using it more for actually administering my machine and everything. And there's very little point in administering a virtual machine. Like, you know, I do, though, I
Leo Laporte [00:15:32]:
administer my framework from downstairs all the time.
Darren Oakey [00:15:36]:
Yeah, exactly.
Leo Laporte [00:15:37]:
And I. Because everything can SSH to everything else. This is really weird. I'm sitting on my laptop downstairs talking to the framework, saying, can you fix the X1 the laptop? And it does
Bill (error_404_new) [00:15:50]:
have an agent create a bunch of containers. Right. And manage those containers. And that's a troll. That's the DevOps. Right. That's the sysadmin stuff that we were talking about.
Leo Laporte [00:15:58]:
It's amazing what you. So I really increasingly am focused on this idea of having a central server with an agent running on it and either sshing into it or tail scaling into it or even just doing this web app into it. Because I just feel like everything's centralized that way. I. And I really like that. I think that's a.
Bill (error_404_new) [00:16:17]:
My. My big blocker right now is local hardware. It's like, too expensive and not good enough.
Leo Laporte [00:16:22]:
I'm so glad I bought this framework when I did. Holy cow.
Bill (error_404_new) [00:16:27]:
You're not right after you arrive on that.
Leo Laporte [00:16:30]:
I am. I feel very fortunate.
Juan Hernandez (BlindWiz) [00:16:32]:
Yeah.
Leo Laporte [00:16:32]:
And I'm just.
Darren Oakey [00:16:33]:
Jan's got the Double Spark, but I also do.
Juan Hernandez (BlindWiz) [00:16:36]:
I also just picked up a double DGX spark.
Larry Gold (LrAu) [00:16:39]:
Wow.
Leo Laporte [00:16:40]:
What do you mean double? Like two 50 90s in it or.
Juan Hernandez (BlindWiz) [00:16:43]:
Well. Well, two Blackwells in it. Yeah, two GB. 10 Blackwells.
Larry Gold (LrAu) [00:16:47]:
So.
Leo Laporte [00:16:47]:
Yeah.
Craig McFarlane (CraigM) [00:16:47]:
That's special.
Leo Laporte [00:16:48]:
How much was that? 10 grand. Right.
Juan Hernandez (BlindWiz) [00:16:51]:
Slightly under 10. Yeah. Yep.
Leo Laporte [00:16:54]:
That's nice. How much. So is it act as if it's unified RAM for the.
Juan Hernandez (BlindWiz) [00:16:58]:
Yeah, for two. Yeah. And. Yeah, there's a 200 and gig. 200 gig sort of Q SFP network link in between them. That's like, for the cluster. Ye. It's really fast.
Leo Laporte [00:17:12]:
The fabric is. So how much RAM?
Juan Hernandez (BlindWiz) [00:17:15]:
Well, 256 between the two.
Leo Laporte [00:17:17]:
Jesus Christ.
Juan Hernandez (BlindWiz) [00:17:18]:
So I can run a.
Bill (error_404_new) [00:17:19]:
Running multiple models at the same time. Are you running?
Juan Hernandez (BlindWiz) [00:17:21]:
Yeah, well, I can. Or I could run a 11 400B model.
Leo Laporte [00:17:27]:
Which one do you run?
Juan Hernandez (BlindWiz) [00:17:29]:
I haven't. Well, I. I can't. I haven't run one that big one yet. I don't know if I've been doing. I've been doing clustered of smaller, like 200B just to test it out.
Leo Laporte [00:17:40]:
A couple of 200B models?
Juan Hernandez (BlindWiz) [00:17:42]:
Yeah.
Bill (error_404_new) [00:17:43]:
Theoretical max right now. And how many more can you daisy chain together?
Juan Hernandez (BlindWiz) [00:17:46]:
Well, no, with the. If you. So unfortunately. So, Darren, you got the Asus one, right? You only have one port, right?
Darren Oakey [00:17:56]:
Yeah, I only have one, but I can't afford another one.
Juan Hernandez (BlindWiz) [00:17:58]:
Anyway, so the Nvidia came up too. So you. Yeah, I could add more and probably endless amounts.
Leo Laporte [00:18:05]:
Oh, my God.
Craig McFarlane (CraigM) [00:18:07]:
I feel guilty just leaving my M4 not fully maxed out and running all night. If I had all that hardware, I mean, what am I not doing?
Leo Laporte [00:18:17]:
It's like a lug. You got a Ferrari, baby. You're. You're rocking it. Tuan, that's amazing.
Larry Gold (LrAu) [00:18:22]:
If you need to heat your house in the winter.
Juan Hernandez (BlindWiz) [00:18:25]:
Hey.
Larry Gold (LrAu) [00:18:25]:
Or set your coffee mug on it.
Leo Laporte [00:18:28]:
The calculus I make, though, is, honestly, I mean, 10 grand, that's, you know, 10 years of
Juan Hernandez (BlindWiz) [00:18:36]:
my company. We. We blew that in a week token. So.
Leo Laporte [00:18:40]:
Okay.
Juan Hernandez (BlindWiz) [00:18:41]:
Yeah, everybody is on the hunt for the optimal local setup.
Leo Laporte [00:18:46]:
Yeah, well, there's also locals, private. There's a lot of units, especially for a company.
Darren Oakey [00:18:52]:
But it's also guilt because, like, I'm doing things like, you know, all these videos that I'm rendering and everything like this. And if I was to look.
Leo Laporte [00:18:59]:
Burning forests. Yeah.
Darren Oakey [00:19:01]:
If I. If I get my spark, I can think, oh, I paid 7,000 and it's just done. And now I almost feel guilty if I'm not rendering a video because I'm wasting it. Whereas if I go and render a video on Google and spend. I paid $300 just for that little bit of Fun. That's it feels, feels wrong and stupid.
Leo Laporte [00:19:21]:
So I'm mad at Google because they release. This is the silliest thing in the world. This thing called deep beans. Have you seen this? Let me see if I can. How can I pull it up? It's on my phone. Unfortunately you need an ultra subscription, so I thought, oh well that's good, I'll have another hundred dollar subscription. I know what could possibly go wrong. Except they won't let you use it anywhere else.
Leo Laporte [00:19:51]:
You have to use it with any Gravity or one of the Google tools. But what it does is it goes through your emails, it goes through everything, your calendar and it generates stream the new Vince Staples album Crybaby. I don't know why it wants me to do that. It generates little stories, explore the redesigned Obsidian settings panel and then it makes little pictures of me. It's giving me little things to do. You know, man, I could show this.
Bill (error_404_new) [00:20:15]:
I know. They already have the data.
Larry Gold (LrAu) [00:20:17]:
For some people they have the data.
Bill (error_404_new) [00:20:18]:
They don't like it. I know, it's a little creepy, Claire. Right? Like it's the jungle. I don't, I don't want to know. They have all the data.
Leo Laporte [00:20:23]:
No, that's why I do this for. I do this for you guys. I do it so you don't have to, so you can see what it does. I mean, honestly I, I wouldn't if, if it were just me, of course I wouldn't be doing this. But I, but I think there's some interest in, you know, what do you get. And so that's my rationale.
Larry Gold (LrAu) [00:20:43]:
I don't know if you looked at the new Anti Gravity, they've now made it to look like Codex and of course they have Claude's the client, so they're all in the same boat. And Windsurf renamed them to Devin, so it's now Devin Desktop.
Leo Laporte [00:20:55]:
Right.
Larry Gold (LrAu) [00:20:56]:
And so they're.
Bill (error_404_new) [00:20:57]:
They've got an agentic mode now too that they're trying to.
Leo Laporte [00:21:00]:
Convergence isn't there?
Juan Hernandez (BlindWiz) [00:21:01]:
So yeah. So personified their product by giving it a sort of.
Bill (error_404_new) [00:21:07]:
Well, back when they bought their CEOs.
Larry Gold (LrAu) [00:21:09]:
Right.
Bill (error_404_new) [00:21:09]:
Google bought their CEOs. And so then what's the. What's the parent company of Devin with the C. Something like that. They bought, they bought the Windsurf team and they've been slowly feeding in the features. Like we've gotten little bits and pieces here that it was coming, like random errors that Devin isn't running and stuff. So we knew something was coming. But now they did the whole branding change and everything too.
Bill (error_404_new) [00:21:35]:
Same product but a different branding.
Larry Gold (LrAu) [00:21:38]:
Yeah. The interesting part, and this is what I've been talking about lately, is that all these tools are trying to go after non developers because at some point developers is only like so many people. You need to sell this globally, you need to sell this to everybody. So you need to make it a desktop app. You can't make it a two either. Even though some people jumped on openclaw. But you can see that kind of weaned off after a while. They need that user based interface.
Leo Laporte [00:22:04]:
I was thinking about that online base before the Deep north reveal. Auditing your Synology, nas, Docker Containers and the trustee. Silly. Here's me and my wife. Somehow it knows what my wife looks like. Oh, I guess for my photos. But you know what? This is cool. I found a craft pizza workshop near me in Sonoma county so I could go to that.
Larry Gold (LrAu) [00:22:29]:
I may be doing one Thursday night. Yeah, this is that new Google Deep Beans thing.
Leo Laporte [00:22:35]:
Yeah, Dream Beans review the graphene OS and Motorola hardware partnership.
Juan Hernandez (BlindWiz) [00:22:40]:
So does this do this when there's nothing else going on?
Leo Laporte [00:22:43]:
Yes.
Juan Hernandez (BlindWiz) [00:22:44]:
Is that why it's called Drink Beans?
Leo Laporte [00:22:45]:
It was overnight. Yeah, it's okay, it's going through. But some of this is really like I am not going to be staying in Rome in December. I wish I were.
Darren Oakey [00:22:56]:
But this is the start of us being agents for the AI as opposed to.
Leo Laporte [00:23:00]:
Yeah, obviously that's what it's doing. It's completely, you know, scarfing up all my data. This is what really makes you want to do it locally, right? This stuff like this.
Larry Gold (LrAu) [00:23:12]:
Well, it's tough because I think when you have something that's web based, it can run all the time. Again, not your hardware but all your stuff is in the cloud somewhere. Like you know, whether you have an office subscription, a Google subscription, there's so much data, your icloud, all that stuff is there. So just saying I want the LLM local. It's already out there. So you know, I'm on your boat, Leo, which is like, yeah, I could use the cloud stuff. It's not going to kill me because it's all there anyway.
Leo Laporte [00:23:38]:
Well, but I think so we, we were talking about this in security now because they were, you know, Steve, like Steve last year said you got to use this new AI model called Venice because it doesn't store any of your. Yeah sure Steve, go ahead. You use that. And of course it's just, I'm sure an orchestrator to, you know, Codex or something. But anyway, it. The guy was worried about all this data he was exfiltrating. And I said, you can turn all of that memory off in the chatbots. I don't think when you're using Claude code, if you're storing your own memory, you tell me, is it sending? I mean, it sees your session, but it's not sending everything in your memory up to Claude.
Bill (error_404_new) [00:24:22]:
Got to send some sort of summary up there, right?
Darren Oakey [00:24:25]:
It's sending a session up, it's sending session up. And they would at least have logs. But also, if you're like me and you want to use these things on your phone or something, like, I've got remote control turned on all the time, so I definitely. I can see every session everywhere.
Leo Laporte [00:24:42]:
Right? See, I don't do that.
Larry Gold (LrAu) [00:24:45]:
But if you, even if you go to Claude AI, you could see all the sessions that you're doing on the desktop end of the app.
Juan Hernandez (BlindWiz) [00:24:50]:
So they are.
Larry Gold (LrAu) [00:24:52]:
It's all there.
Leo Laporte [00:24:54]:
It's funny because several times lately Claude said, well, how'd I do? And then it said, do you mind if I upload the transcript so we can review it? So I don't think they're. I don't know. I mean, first of all, there's a lot of bandwidth and data storage and I don't know how useful that would be to them except for some sort of memory.
Darren Oakey [00:25:13]:
But fundamentally, they don't do any LLM locally, right? So it was all sent in the first place. Unless they, you know, sometimes they write a Python program to interpret things locally or extract things, and so they don't. Obviously that information won't be sent, but all the stuff that had to go
Leo Laporte [00:25:30]:
through now, I guess all your prompts, your memory md, your soul md, your agent md, they had to send that.
Darren Oakey [00:25:36]:
And also, you've got to imagine that just in normal development practice, they're logging all of it. So even if it's not accessible, they've got it somewhere, right? Whether they use it for it, whether they trade, whether they train on it or anything, which they probably don't because not really in their garbage, financial interest, it's there. Like it all went there at some point.
Bill (error_404_new) [00:26:01]:
Whether they train on it now, right, They've got it if they want to go back. And they can always stuff, right? Yeah, but the thing is not really deleted.
Darren Oakey [00:26:08]:
If you think about their incentives. It's just like Google using your data. I mean, they'll use it for some things, but they're like anthropic publishing your things. Their whole company could just collapse in one go. There's plenty of other Places to get data. It's not in their interest to do so.
Larry Gold (LrAu) [00:26:27]:
I wonder what the EULA says because obviously none of us read it. We're all using the tool. Because I'm sure somewhere the eula, it says we're not using your data unless we ask for it.
Leo Laporte [00:26:40]:
It's extremely broad. Even though it's protect themselves, it's going to be broad.
Darren Oakey [00:26:44]:
It's not in their interest to do it. Because if they start leaking people's secrets.
Leo Laporte [00:26:50]:
Exactly.
Darren Oakey [00:26:50]:
They go away. Right.
Leo Laporte [00:26:52]:
Although Sam Altman did say just the other day, you know, you probably shouldn't use it as a therapist or a lawyer because we're not covered by attorney client privilege. There's no privacy. Which implies that they have all that information.
Darren Oakey [00:27:07]:
Well, no, but that's more like they will have all these in the.
Leo Laporte [00:27:10]:
They could be subpoenaed in the logs
Darren Oakey [00:27:12]:
and they could be subpoenaed.
Leo Laporte [00:27:13]:
Right.
Juan Hernandez (BlindWiz) [00:27:14]:
Which has been done. New York Times did it with chat.
Leo Laporte [00:27:17]:
They might not use it.
Craig McFarlane (CraigM) [00:27:18]:
They might not use it as part of a product to do something with, but they could certainly use it from an admin perspective to troubleshoot or look stuff up or if there's a pain.
Leo Laporte [00:27:29]:
So this all argues really strongly for having a local model and a local agent keeping everything local. Well, I just don't think the models are that good yet. Except for Juan.
Larry Gold (LrAu) [00:27:39]:
This is where models all the time. Right. There's not enough information on one if you just keep bouncing between open router, Claude, Gemini, you know, OpenAI just keep rotating every. Every few chats.
Leo Laporte [00:27:50]:
Oh, that's a good idea.
Darren Oakey [00:27:52]:
And it's what Hermes is really that direction that they're going. It shows it's the right way because really they can just do grunty stuff on the big models. But all the magic and all the memory and all the why of what it's doing is. Is staying within Hermes on a local model.
Leo Laporte [00:28:10]:
And Quinn. Yeah.
Larry Gold (LrAu) [00:28:12]:
Are you paying anyone paying for the Hermes their service to their models?
Craig McFarlane (CraigM) [00:28:16]:
Yeah, well, I have tokens with noose.
Leo Laporte [00:28:20]:
Yeah, yeah.
Craig McFarlane (CraigM) [00:28:20]:
So they got a noose portal but on the back end it's like it's through open router. So you have like, you can run, you know, Quinn or Deepseek, but then you also get a bunch of other tools like image models and Firecrawl for web search.
Leo Laporte [00:28:37]:
It's like 120 models on there. Yeah.
Craig McFarlane (CraigM) [00:28:40]:
It's nice because it's all like wrapped into one sub.
Leo Laporte [00:28:43]:
Well, and there's some things I don't want to pay like web search, you Know, I don't. I don't want to pay for Fire Crawl or something. And so I just grabbed it through noose.
Craig McFarlane (CraigM) [00:28:51]:
Yeah, exactly.
Leo Laporte [00:28:53]:
And then I have access.
Craig McFarlane (CraigM) [00:28:55]:
Speaking. If you want a free search, there's. Have you heard of. I don't know how you pronounce it, but Seer. Xerpa. Searching.
Bill (error_404_new) [00:29:04]:
The X is a ch. I use that. I run the searching server at home.
Leo Laporte [00:29:07]:
Is it good?
Bill (error_404_new) [00:29:09]:
It's all right. It basically just. Instead of querying one resource, it queries like 5. So Bing and DuckDuckGo, and then it summarizes them and puts them in one output. What I like about it is I can write code on top of it and I don't have to parse out all the HTML classes and tags to pick out the features that I want. It's just consistent. The JSON API stays the same.
Leo Laporte [00:29:34]:
I installed something called Pulse, which goes through archive.org, it goes through Reddit, it goes through.
Bill (error_404_new) [00:29:41]:
So you can configure all that on the back end here. You can pick and choose. It's got millions of settings and you can pick and choose exactly what you want your search results to be.
Craig McFarlane (CraigM) [00:29:48]:
And there's a back end so you can give it to your agent. So once you have it running.
Bill (error_404_new) [00:29:52]:
Yeah, that's what I do.
Leo Laporte [00:29:56]:
So here's all the sources that Pulse uses. Reddit, Hacker News, Poly Market, YouTube, Archive, Lobsters, RSS. So it has a bunch of technical blogs, GitHub.
Bill (error_404_new) [00:30:06]:
Yeah, there you go.
Leo Laporte [00:30:07]:
BlueSky, Dev2, Lemmy, Stack, Exchange, Open, Alex, Semantics, Scholar, Manifold, Metaculus, Tinder, tick and News. And what I use it for is. I can say it's actually. I don't know what News. Oh, it's the News API. That's what it is. So that's basically a bunch of newspapers. So what I use that pulse for is.
Leo Laporte [00:30:27]:
I could say, well, give me. What's the. What's that? What are everybody saying today about Hermes? And it does a very good kind of pulse check. What are people saying about X? Or I want a prediction market, odds or that kind of thing. And this is a local skill. So I don't know exactly. I don't know how it's working. Actually, come to think of it, it must be connecting to something.
Leo Laporte [00:30:52]:
Maybe that's why I keep running through my Tavoli tokens and my Serpa tokens. I don't know. Serpent.
Darren Oakey [00:30:59]:
Yeah. Web searching is. Is an interesting thing to wrap, because it is.
Leo Laporte [00:31:02]:
Oh, yeah, See, this is it. It has all these. I had. I. That's What I did, I went out and got keys for all this.
Craig McFarlane (CraigM) [00:31:08]:
Okay. So that's.
Leo Laporte [00:31:09]:
So it is using Serper Exa.
Craig McFarlane (CraigM) [00:31:12]:
Yeah, the, the benefit for Searing is that you don't need to like work out. You don't need to like get your own Google API key.
Leo Laporte [00:31:21]:
Like do you pay Searing?
Bill (error_404_new) [00:31:23]:
No, no, it just does a web search. Huh.
Leo Laporte [00:31:27]:
And you can see you can't do x.com because they don't let you unless you have a grok.
Larry Gold (LrAu) [00:31:33]:
Yeah.
Leo Laporte [00:31:33]:
I think there's a pair where you
Bill (error_404_new) [00:31:35]:
can add a subscription if you have a paid subscription, which I do. It'll take a token.
Leo Laporte [00:31:39]:
I have a. I have an non consensual blue tick social media, which I take advantage of. If he's going to give it to me, I'm going to use it. Yeah.
Craig McFarlane (CraigM) [00:31:49]:
Okay.
Leo Laporte [00:31:49]:
I'll have to try this searching or whatever you call searching.
Larry Gold (LrAu) [00:31:52]:
And Brave also has a search API I've been using too.
Bill (error_404_new) [00:31:58]:
You have to pay for that one though, right?
Larry Gold (LrAu) [00:31:59]:
Yeah, yeah, yeah. There's things I feel like paying for because you want those services to be around. They use the browser also. So it's, you know, again, even as a developer, you want people to pay you for your work. And there's certain things you say, this is good enough for me to pay for and want those companies in business.
Darren Oakey [00:32:18]:
Thing is though, about paying is that I've come to a conclusion I can only pay for. I've just had so many mistakes where I've been hit by like 300 here, 200 here, that I'm happy to pay a ridiculously large fixed fee, but I won't pay any per token fee for anything or per use.
Larry Gold (LrAu) [00:32:38]:
Yeah.
Leo Laporte [00:32:40]:
So I do this all the time with Hermes. I said, well, what is searching like compared to Pulse? Which one would you recommend? And then it usually does a pretty good job of saying, well, you got, it's got this, it's going to add that, but maybe this is better.
Larry Gold (LrAu) [00:32:55]:
Brave is $5 per thousand call, so I barely hit that in a month. Yeah. So $5 a month for a thousand searches is fine.
Leo Laporte [00:33:03]:
Yeah, that's actually quite a bit.
Larry Gold (LrAu) [00:33:05]:
Yeah.
Leo Laporte [00:33:06]:
And that's their search index.
Larry Gold (LrAu) [00:33:07]:
Yeah, that's their search.
Juan Hernandez (BlindWiz) [00:33:08]:
Yeah.
Leo Laporte [00:33:09]:
I have Kagi. I pay a lot for Kagi. I wonder if I have a way to hook up Kagi.
Juan Hernandez (BlindWiz) [00:33:14]:
I should look, Kagi doesn't have a good API.
Leo Laporte [00:33:17]:
They probably do. Yeah. I have to see. That's actually a good. Let me start another.
Juan Hernandez (BlindWiz) [00:33:24]:
And if your bot doesn't have a skill to access, it probably does.
Larry Gold (LrAu) [00:33:30]:
If you Go to their help pages. They have an API and they just pass your token, your bot token in.
Darren Oakey [00:33:43]:
So is this using a 35B or. I mean, I mean the Quinn 3.6.
Leo Laporte [00:33:49]:
No, this. Right now I'm in. I have a sub for 55, so. So this is nice. This is one of the nice things about Hermes is it just, you know, I don't have to.
Juan Hernandez (BlindWiz) [00:34:03]:
So I signed up for Mellon's new AI program for over the summer.
Leo Laporte [00:34:09]:
Oh, fun. Is it on site or is it remote?
Juan Hernandez (BlindWiz) [00:34:12]:
No, it's a virtual for seven weeks. Yeah, it's their AI applied Agentic certification or something like that.
Leo Laporte [00:34:21]:
Oh, you got to tell us all about that. That's very.
Juan Hernandez (BlindWiz) [00:34:22]:
I'm excited.
Larry Gold (LrAu) [00:34:25]:
I was looking at it too, because I graduated there.
Juan Hernandez (BlindWiz) [00:34:27]:
I saw something for the first time that I've never seen. So under the signup form they had a box that. Enter your agent ID and say, check that this is an agent. I've never seen that. And it's like, what does this mean? So what kind of ID is it?
Leo Laporte [00:34:44]:
You know, your agent can take the
Juan Hernandez (BlindWiz) [00:34:45]:
class for you or sign you up at least. Right.
Leo Laporte [00:34:49]:
I think, I think, honestly, see, I'm really an advocate for everything. Should have an Agentic API or SDK or MCP server, because that's what you want. You don't want the agent to take your class, but you would love the agent to sign you up, take notes, help you with the quizzes.
Darren Oakey [00:35:07]:
Funny, we're in a war the other way around because there's so many things trying for multi factor authentication and trying to stop you using computer use for anything. It's this little arms race of.
Leo Laporte [00:35:20]:
I think in the long run that Agentic will win because I think this is. It'll be interesting to see what Apple announces on Monday. But with Google's. Microsoft has. What is this? Google has one now and Microsoft has one, I think. And you know, normal normies are really going to start using these tools.
Darren Oakey [00:35:38]:
Yeah, Google Spark and Spark.
Leo Laporte [00:35:41]:
Yeah.
Larry Gold (LrAu) [00:35:41]:
The challenge thing about the micro, the. Sorry, the. The Carnegie Mellon class. It's like 11 in the morning and it's over the summer for two weeks while I'm at the F1 races. So I've kind of like.
Leo Laporte [00:35:51]:
Wait a minute. What do you mean you're at the. Are you going to every F1 race?
Larry Gold (LrAu) [00:35:54]:
We're going. We're going to Budapest.
Bill (error_404_new) [00:35:55]:
Yeah.
Larry Gold (LrAu) [00:35:56]:
Oh, so I told you we go to one every year.
Leo Laporte [00:36:00]:
Yes, I would. You know, we were in Monaco like two weeks before the race and walked the course. It was so Cool. Going to the race. We went to the Vegas race. I'm less. Was less excited about it because they go by so fast. There's not really much to see.
Leo Laporte [00:36:16]:
It's nice to be there, I guess.
Bill (error_404_new) [00:36:17]:
Oh, yeah. Loud. That's all I remember. Just the loudest thing I've ever heard.
Leo Laporte [00:36:21]:
Really loud. And that's it. And then they're gone. Unless you have a. I guess if you're in Monaco and you're sitting on the hairpin, you know, you're at that Fairmont, you could look out the window and they have to all slow down.
Larry Gold (LrAu) [00:36:33]:
Well, you know, if you go on Reddit, they'll tell you where to sit to see most of the race. So there's a lot of people who. There's a couple boards and they'll tell you just search by, you know, this race, where to sit. So when we were in Austin, we were really facing the straightaway. So you got the whole straightaway get
Leo Laporte [00:36:48]:
to look like that?
Larry Gold (LrAu) [00:36:49]:
Yeah, you get to look. And then when they went around the turn when we did Miami the year before, we were actually where all the overtakes were.
Darren Oakey [00:36:55]:
We were right.
Leo Laporte [00:36:56]:
That's cool. We were where the overtakes were. We were on the strip in front of the Bellagio. But it's still so fast. What we did see, though, was great, was when LeClaire hit the manhole cover and he goes by and it's just sparking. I said, that doesn't look right. Anyway, enough F1 talk. Yeah, cocky has.
Leo Laporte [00:37:21]:
So it's working. So I've got a Kagi skill now. Okay. Yeah, great.
Larry Gold (LrAu) [00:37:29]:
I didn't even think about that because I was paying for the skillshare, installed
Leo Laporte [00:37:33]:
a Manning, small wrappers, Cassini. Nice.
Juan Hernandez (BlindWiz) [00:37:36]:
I think that would be really neat for some of these big web frameworks to start, including a markdown API or agent API built in as all these frameworks allow for agents.
Craig McFarlane (CraigM) [00:37:54]:
Well, it used to be the whole thing that you start building a new site or app, you start with the API and then. Then you build a web framework on top. And you know, now it's. You got to make sure you're doing both the human interface plus the agent interface.
Juan Hernandez (BlindWiz) [00:38:09]:
Yeah, yeah. Really makes. That makes give. It gives it a good reason to build a good API though, if you want that agent access.
Leo Laporte [00:38:19]:
Yeah.
Craig McFarlane (CraigM) [00:38:21]:
So has anyone tried out the PewDiePie's?
Leo Laporte [00:38:27]:
What does PewDiePie have a PewDiePie skill he created?
Craig McFarlane (CraigM) [00:38:30]:
Not a skill, it's a whole. It's kind of like a Claude desktop app. Hold on, let Me.
Leo Laporte [00:38:36]:
Why would. What standing does Pewdiepie?
Craig McFarlane (CraigM) [00:38:41]:
He's been. He's been on the AI. Like, he has, like a crazy server he's built at home, like, serving models. Like, he's been. He went off the deep end on this, and so this is his little chat guy.
Leo Laporte [00:38:55]:
Well, it's good. It'll popularize it for sure with a certain group of people. Yeah.
Bill (error_404_new) [00:39:00]:
Read me though, because it's definitely not a Trojan, right?
Leo Laporte [00:39:03]:
Oh, yeah, for sure. Absolutely not.
Larry Gold (LrAu) [00:39:09]:
When you run it is not a good thing.
Leo Laporte [00:39:11]:
You know, I'll say.
Craig McFarlane (CraigM) [00:39:13]:
Like, it's been. It's easy to set up. Like, I just gave it my LM Studio, you know, address, and then, you know, like, I'm running. You could just pull up the model you want. You know, gamma 412B. I have this hearing web search. So, like, it. It can do that.
Craig McFarlane (CraigM) [00:39:35]:
Like I just said, you know, oh. Like, who is Leo laporte? You know, it works. You could compare models. There's a bunch of tooling in here. It's more like. It's not Hermes or OpenClouds. Closer to, you know, perplexity.
Larry Gold (LrAu) [00:39:51]:
It looks like ChatGPT pretty much. Yeah. Something like that.
Leo Laporte [00:39:55]:
I mean, if I were him, I wouldn't do any models. I would just orchestrate.
Bill (error_404_new) [00:39:59]:
Yeah, I liked where the doc said it would pick the right model for your hardware. I feel like that's a cool feature, right? Like, optimize, whatever. Whatever I can run, I want to run.
Craig McFarlane (CraigM) [00:40:09]:
Like. So you go to the cookbook, then you could, you know, say, like.
Leo Laporte [00:40:13]:
So it's not running cloud models. It's running locally.
Craig McFarlane (CraigM) [00:40:17]:
You can't. You. You could do both.
Leo Laporte [00:40:19]:
Okay.
Juan Hernandez (BlindWiz) [00:40:20]:
Huh.
Craig McFarlane (CraigM) [00:40:21]:
So, like, I did this all local. Like, I haven't even has a.
Leo Laporte [00:40:25]:
Shows how low the bar is for entry into this. I mean, a deep research.
Craig McFarlane (CraigM) [00:40:30]:
I mean, it's impressive.
Leo Laporte [00:40:31]:
It built team.
Craig McFarlane (CraigM) [00:40:33]:
No, I think this is vibe coded.
Darren Oakey [00:40:35]:
Probably you just made. I want an app.
Bill (error_404_new) [00:40:42]:
Define some requirements and burn some tokens is what I see.
Leo Laporte [00:40:44]:
Yeah. Self host searching.
Bill (error_404_new) [00:40:49]:
Yeah.
Craig McFarlane (CraigM) [00:40:49]:
Yes.
Bill (error_404_new) [00:40:50]:
This is a Docker container.
Leo Laporte [00:40:52]:
Yeah. Okay. So it's saying use that impulse. Okay. It's saying searching should absolutely be added as Pulse's free local, private web search substrate. Okay.
Bill (error_404_new) [00:41:06]:
I approached it from a privacy standpoint to a while back, and it really just searches all those sources on your network. You know what I mean? They're still logging your IP and that you search the thing and all that. I don't know how much privacy there is into it, but it does aggregate results from multiple sources, which is cool. And standardizes the output, which is what I built from.
Darren Oakey [00:41:31]:
It's not just that, but HTML is extremely heavy and so if feeding raw HTML into a model is extremely expensive on tokens, so also higher level pre processing of it and trying to extract the data outside your model will drastically reduce tokens.
Craig McFarlane (CraigM) [00:41:58]:
I was just looking if it sounded like someone joined, but I guess they just dropped and came back.
Leo Laporte [00:42:02]:
Yeah, yeah.
Bill (error_404_new) [00:42:05]:
Darren, can I pick your brain about something? I feel like this might be in your wheelhouse. I've been seeing these videos about taking lower end hardware and using a model of experts model and offloading like the orchestration to the GPU with the execution of the task to our. Sorry, the orchestration to the CPU and the actual execution to the gpu. Does anybody know anything about that or have had any success? I have not had success trying it, no. Okay, I'm gonna find the YouTube video if you like.
Larry Gold (LrAu) [00:42:37]:
The homegrown version of the cloud code I built is running on an M5 Mac mini, but the models are running on a GPU machine. So it's doing the same thing because I'm using the cloud SDK. So all your orchestration and logic are sitting within the Mac. Right, but the models are not. The models are on the GPU or when I need to code, it calls CLAUDE code. But that is your deep coding into the CLAUDE SDK. Right.
Bill (error_404_new) [00:43:05]:
See if I can find it. Because
Larry Gold (LrAu) [00:43:09]:
it's like a really good framework. And remember when you're using that cloud SDK framework, you're not bound to anybody's model.
Darren Oakey [00:43:17]:
And it's the same for me. I'm using everything I, I've got, obviously got. I've got the Mac as one thing and I've got the Spark as another. And I'm just using them almost like you would use server models. So. But it's all, it's all my own coding. So I've got this thing called the Arbitrary that runs on the Spark and There's just like 60 different models or something that'll run and the arbiter just is a queuing system that just lets things in. But as far as I'm concerned it's just like a cloud service.
Darren Oakey [00:43:47]:
But it's a cloud service that I don't have to care about tokens.
Leo Laporte [00:43:53]:
How much was the brave API?
Larry Gold (LrAu) [00:43:56]:
The brave was 5 per thousand search
Leo Laporte [00:43:59]:
for the first thousand coggies, 12 per thousand for search and then 4 per thousand for extract. Okay, so I, yeah, I think the COGI index is better than the Brave index, but this is the problem. I end up buying a whole bunch
Larry Gold (LrAu) [00:44:15]:
of crap I don't use. Yeah, I know, I, I need to.
Leo Laporte [00:44:18]:
This is pay as you go, which is nice. So if you don't use it, you don't. Yeah, you don't pay anything. I was hoping that because I have a COGI Pro subscription, I wouldn't have to pay for it. Foolish me.
Darren Oakey [00:44:33]:
Well, this is the other thing that we can see coming. But if you imagine Hermes, I've already built this sort of thing at work, but I can see it coming to agents. Is that one of the things I've got at work, we've got our normal process, which is the CI process. But I started to realize that I can add other agents to optimize things and I've now got about 40 of them. But I've got something that like wakes up every three hours and it says, oh, look at the process through can I make it faster? But I've also got something that says, look at the AWS costs that we're using. Can we do the same thing but cheaper? And so over time this sort of thing is going to be built into your Hermes, or you can add it to your Hermes, which you'll just have a whole bunch of things saying, can I get those tokens cheaper? Can I eliminated subscription? Once it owns these subscriptions, then it'll just be constantly optimizing for a price for speed, for all the other things that you want.
Larry Gold (LrAu) [00:45:39]:
Right.
Craig McFarlane (CraigM) [00:45:39]:
Based on your profile of use.
Darren Oakey [00:45:42]:
Yeah, so it's, it's not quite that, you know, there's the Holy Grail which doesn't really make sense. You know, the sci fi thing of the exponential improvement. But we can have a local improvement and we're all starting to see assist the shape of something where your personal agent will improve all by itself.
Leo Laporte [00:46:10]:
Sorry, I'm adding the. So I. The other thing I did this week, which broke a lot of things, was move everything out of a SOPS encrypted key storage to Bitwarden. So I'm using the Bitwarden CLI to. So to add COGI to my keys, I just have to add it to Bitwarden in the Hermes folder and then Hermes could pick that up.
Craig McFarlane (CraigM) [00:46:35]:
I did want to ask, is it pretty easy to connect Bitwarden to Hermes?
Leo Laporte [00:46:40]:
Yeah. So there's two steps. One, you still have a SOPS encrypted file with the Bitwarden key and secret. So Bitwarden will create a key and a secret which is stored in the SOPs. The drawback is every time you do A fresh boot. You have to log into your bit, you have to unlock your bitwarden vault and then it wrote a little script called Refresh Hermes Secrets that refreshes them and it saves it to a temp memory directory. So it's in memory.
Bill (error_404_new) [00:47:15]:
I try not to give it too many security permissions, but basically I just set up a nextcloud user for my Hermes agent. And it's got password management in there, it's got file storage in there, it's got email access, it's got like, it's, it's a whole suite. It's like.
Leo Laporte [00:47:31]:
Is it a third party?
Bill (error_404_new) [00:47:32]:
No, it's just self hostage. I just don't. I just self host my own nextcloud. But it's got, it's got password manager built in. Yeah.
Leo Laporte [00:47:39]:
Oh, it's using this cloud password manager.
Bill (error_404_new) [00:47:41]:
Yeah. So that way if I need a feature like it already has a template for how to interact with Nextcloud and all its APIs and all its UI and all that. And so instead of having it write
Leo Laporte [00:47:50]:
a thing, you don't have to unlock it.
Bill (error_404_new) [00:47:53]:
No, I mean it has its own password, right?
Leo Laporte [00:47:56]:
That's the tricky part for me.
Bill (error_404_new) [00:47:58]:
Yeah, you just put it in a dead end and exclude it from the commits and 1Password to rule them all. I mean I think that's been a concern of mine from the beginning of this is like security is not. It's like guardrails, right? It's kind of a fallacy, right? Like even if you say don't read the M, it can cat the M, it has permission to cat the M, so it can read the M. And that's probably getting sent up to a server somewhere in some of those cycles, right, that you're not really watching. You're only watching the ones that have to say yes, approve. But there's a lot of cycles that
Leo Laporte [00:48:27]:
happen you're trusting that it's not sending it out. I mean I have, I have said a thousand times, under no circumstances should you ever reveal any of these secrets anywhere, anytime. No how. I mean I have just trusted do that.
Bill (error_404_new) [00:48:42]:
I was using 5.3 spark for the first time this last week and this has been the model that has most deleted my local database for sure. And I've told it like a bunch of times like don't, don't delete that. But I think it's the same like don't draw an elephant, right? You tell it not to do something and it gets the idea and it's like it's non deterministic yeah.
Leo Laporte [00:49:09]:
All right. So now I have cogi and I have search xng.
Darren Oakey [00:49:18]:
Yeah, it's really incredible how quickly you can do things.
Leo Laporte [00:49:22]:
That's amazing.
Bill (error_404_new) [00:49:23]:
I was, I was talking to a friend of mine that writes for Gartner about quantum computing. And the first thing that sparked me, I don't know if you saw that Microsoft thing that they had some thousand x improvement over. I asked him, I was like, hey, is this real or is this hype? And it was kind of what I assumed. It's like 1000x is zero is still zero. Right? It's still not as good as what everybody else is working on. But then I asked him, now that we've been in this AI era and engineers have had access to it, are you seeing any blooming of that industry, of the quantum industry and progression? And he said, yeah, just like everything else, stuff's moving way faster than it used to be. So I think we're starting to realize some of those gains that we've been working towards for the last few months or years. I guess at this point.
Darren Oakey [00:50:19]:
No, I keep coming back to. Even with the bit warden thing, we're starting to see the elements that you need. And you need a web search unit, you need some sort of password store that will connect to things. But yeah, the bit warden thing scares me a little bit because it's. Because it does need human confirmation at some point, but you need a memory store and everything. All of these pieces are coming and they're all starting to look the same. It's interesting how we are coalescing and really starting to understand what an agent assistant should look like.
Leo Laporte [00:51:02]:
Yeah, that's encouraging.
Darren Oakey [00:51:07]:
And it will be so interesting on Monday or Tuesday or whatever day it is.
Leo Laporte [00:51:11]:
Monday, yeah. See what happens.
Darren Oakey [00:51:14]:
Because to me, you know, if you look at the, the live chat, like if you actually get into a chat GPT live chat, or a Gemini live chat on your phone, it really is amazing. And I don't know if you guys have played with GPT Real Time too. I think you should get it on the show, Leah, because I think it's good enough. But these things, talking to them, it's just like talking to a person. And I don't think most people, most regular people have experienced that. And while you can put it on your phone, it's if you've got to press the action button, wait, watch it, watch it come up and everything, people aren't going to use it. Because the step to doing it, especially when you want to go for something like this, is when you're on the phone and you can't when you're in the car and you can't touch your phone and everything. But if you can say, hey Siri, I want to talk about this and then it just responds to you and you're in a conversation, that will be a fundamental change in my mind.
Darren Oakey [00:52:23]:
Suddenly everybody will just be using AI for everything. And so if you then start putting the ability to do things behind it, we're just going to go from 0 to 100.
Leo Laporte [00:52:33]:
Yeah, yeah. Well, I can do that in my car now with Grok and ChatGPT, it has a voice interface. In fact, it was really frustrating the other day. Grok for some reason started up. I didn't do anything. And I'm talking to my wife as we get, we get in the car and I'm talking to her and Grok is responding and I didn't recognize the voice. She said, who's that? I said, I have no idea. Somebody's talking to us.
Leo Laporte [00:52:57]:
It was really creepy. So, yes, I think we're already there, Darren. I don't know. I don't know if that's a happy thing, but we're definitely already there.
Darren Oakey [00:53:07]:
I don't know. To conversation's a tricky thing. It's like that uncanny valley in that there is a, there's a latency that doesn't feel comfortable and there's the ability to, you know, interrupt and things like that. And I don't think, I think the GPT Real Time 2 is there and I think the Gemini is there, but I don't really think any of the other experiences I've tried. There was that there was a little demo with two people, Maya and someone else that you could play with that came out six months ago or something and then never went anywhere but this whole super low latency and I can just break in and talk.
Leo Laporte [00:53:49]:
That's really interesting. You're really having a conversation.
Darren Oakey [00:53:53]:
Yeah, exactly.
Craig McFarlane (CraigM) [00:53:54]:
Once the latency goes down to a
Leo Laporte [00:53:56]:
certain point, it's like human speech apparently is 450 milliseconds and I'm, you know, my Kokoro is about 120 milliseconds. So it's real time. You can't interrupt it, but it's because it's not listening.
Bill (error_404_new) [00:54:10]:
Did you ever get your Alexa replacement working, Leo? I think we were looking at that last time.
Leo Laporte [00:54:14]:
Yeah, I did not. Well, first of all, I renamed my agent so all of the training I did with High Kenobi is down the out the window. But it never really worked anyway. I was Using that. That PI Python tool. But yeah, a high ESP still works. So, yeah. So I have my ESP here and I can say hi Asp and talk to Quicksilver and have it.
Leo Laporte [00:54:37]:
So I do it for things. I don't do it for a lot of things for like logging food and stuff like that.
Darren Oakey [00:54:41]:
Well, it's all about. And the thing about tying it to a trigger word like hey, Siri, is if you can say hey, Siri, here's my question, then it can hide the activation costs a little bit behind the. Because you've already started the conversation, you're already in the conversation.
Leo Laporte [00:55:02]:
It can blather.
Darren Oakey [00:55:04]:
Yeah. So it can do something and come up. Whereas if you have to say hey Siri, start ChatGPT or something, or even just hey, ESP32 and you've got to wait for it to activate. It's all about whether you can say the activation word and your sentence or you can say the activation word and have to wait. It's a big difference.
Bill (error_404_new) [00:55:32]:
Yeah. I mean, are people going to write like a Siri to Hermes bridge instead of having to say hey Siri, open Hermes and ask. That's what you're saying. Just a few words. But it's a.
Leo Laporte [00:55:42]:
Well, I kind of can do that. I can press my activation button, my iPhone on my watch and it's immediately transcribing and then it will send it immediately to Hermes using HTTP post over a port and then Hermes will respond through Telegram or. And I'm trying to set it up and it sort of works.
Darren Oakey [00:56:06]:
It.
Leo Laporte [00:56:09]:
I'm trying to get to respond to me on whatever device I'm using. And it can mostly tell. And then. But I said, but if I'm on my iPhone or my watch, you can't really talk to me back on my watch or my iPhone. There's no easy way for it to do that. Use the Sonos device nearest me and the way it figures out which one is nearest me is where. Which access point my iPhone is attached to. And there's three access points on each level and there's Sonos in each level.
Leo Laporte [00:56:35]:
So it can actually. So it knows I'm in the gym and it can respond to me on the gym Sonos. And it works about. I don't know half the time. I don't know why it does sometimes
Bill (error_404_new) [00:56:46]:
all that new like Wi Fi, 3D mapping and figuring out human.
Leo Laporte [00:56:49]:
I'm really interested in that.
Bill (error_404_new) [00:56:51]:
Yeah, yeah, yeah.
Leo Laporte [00:56:52]:
But, but access points good enough because it knows which access point I'm on. I didn't realize. You could see that. It could see that and it can. Sonos has. I can see why Sonos software is so bad because the Sonos API is just God awful. I really had to struggle. Claude and I really struggled with it.
Leo Laporte [00:57:08]:
We had to do a bunch of testing just to see, you know, what it was going to do. I was going to play for you. Speaking of two voices, Google now has a radio show generator, and I generated a radio show between. A talk radio show where it was Emily Bender versus Geoffrey Hinton. But I don't. It says I've generated a show, but it doesn't show me my shows. So I don't.
Bill (error_404_new) [00:57:37]:
This is different than, like the Notebook LLM stuff.
Leo Laporte [00:57:40]:
It's. Well, and that's what I thought because it's Google. Right. I thought, well, this is just Notebook lm. But it's. It's a call in talk show. It's actually pretty funny. If I could just get it to where's my shows? I don't understand why it's not showing me.
Darren Oakey [00:57:54]:
After you were talking about this, I made one of my apps for the App Store is ol dj, which is because, as Benito said, for a dj, you just need to play the music and then announce it and.
Leo Laporte [00:58:07]:
Right. It's easy. Yeah.
Darren Oakey [00:58:09]:
So I actually put up an OL DJ that will DJ your thing. It'll just tell jokes. It'll. It'll mention news articles and it'll. It'll take. But it's on the App Store now if you want to go.
Craig McFarlane (CraigM) [00:58:21]:
That was a. They added that to YouTube Music as a feature. I tested it for a while, but
Leo Laporte [00:58:27]:
it says I generated it. It says I only have two shows left today, but it's not showing it. So I don't know.
Craig McFarlane (CraigM) [00:58:34]:
I could send you a link from.
Leo Laporte [00:58:35]:
Yeah, I sent it to you on WhatsApp. WhatsApp.
Craig McFarlane (CraigM) [00:58:39]:
Hold on.
Leo Laporte [00:58:40]:
But the link just pulls me to. Oh, maybe I clicked the wrong here. Don't show my screen for a second.
Craig McFarlane (CraigM) [00:58:46]:
Oh, hold on, hold on, hold on.
Leo Laporte [00:58:47]:
When I go to WhatsApp. Okay, good. Talk show assistant full screen applet. Maybe that's why it's not working is because I'm going to that link. I don't know. Were you able to listen to it?
Bill (error_404_new) [00:59:03]:
I did.
Craig McFarlane (CraigM) [00:59:03]:
Hold on.
Leo Laporte [00:59:04]:
Oh, okay. Well, I don't know what I'm doing wrong.
Craig McFarlane (CraigM) [00:59:10]:
It's no longer available.
Leo Laporte [00:59:12]:
Oh.
Bill (error_404_new) [00:59:14]:
Huh.
Leo Laporte [00:59:15]:
Oh, here it is. The AI consciousness debate. If we replaced your biological neurons one by one with silicon circuits. Oh, yeah. Would you still be you or just a very convincing machine. Welcome to AI Talk Radio. I am Paul. Today we are diving into the intellectual schism of our time.
Juan Hernandez (BlindWiz) [00:59:40]:
AI conscious in a really funny way.
Leo Laporte [00:59:43]:
Understanding or just high tech mimicry? Let's go straight to the phones. First up is Leo calling us from Boston, Massachusetts. Horrible messages. You side with Dr. Geoffrey Hinton's view on this. I believe Hinton's silicon neuron experiment is like spot on. If you swap biological.
Craig McFarlane (CraigM) [01:00:06]:
It's not bad.
Leo Laporte [01:00:07]:
The output, you know, the behavior stays the exact same. So behavior wouldn't consciousness be there? Aren't just repeating things.
Craig McFarlane (CraigM) [01:00:15]:
More Australian.
Leo Laporte [01:00:17]:
It's terrible. They can like solve brand new reasoning problems. Fascinating functionalist take. Let's bring in Sarah calling. I'm going to change his name to Paris. Sarah, you are nodding along with Dr. Emily Bender's stochastic parrot argument. Oh, absolutely, Paul.
Bill (error_404_new) [01:00:36]:
Leo is like completely falling for what Bender calls the A icon. These models, they only manipulate linguistic form, syntax, statistical probabilities of words, but there's zero like actual meaning or intent.
Larry Gold (LrAu) [01:00:53]:
It's just a simulation.
Leo Laporte [01:00:55]:
You know, we humans just project intelligence
Bill (error_404_new) [01:00:58]:
onto anything that speaks fluently.
Leo Laporte [01:01:01]:
But. But it's not just projection though. It's recursive self modeling. When a system models its own internal states, that's where sentience emerges. It's not a sci fi distraction. Hinton's warning about super intelligence is real.
Bill (error_404_new) [01:01:17]:
No, that's.
Leo Laporte [01:01:18]:
Anyway, you don't have to listen to the rest of it. You'll probably. Yeah, intelligent machines anyway.
Bill (error_404_new) [01:01:22]:
But maybe it's telling that nobody said like shut it off, it's bad. We were all pretty, we were all pretty sucked in there for a minute.
Leo Laporte [01:01:29]:
It sucks in. Actually I've been watching Jeffrey Hinton talking. He was on Alex Cantrowitz's big technology podcast. And then I found an old talk he did about nine months ago that I thought was really good. I sent it around to Paris and Jeff, and Jeff said no, it's not true. But I am actually finding him very convincing that in fact, you know, these, these models do have understanding and they, and they really are getting closer and closer to doing what humans do.
Larry Gold (LrAu) [01:01:59]:
So you're with that argument that he was giving about because he was saying, look, you know, you have a thousand or million machines putting things together where one human can only learn one at a time.
Leo Laporte [01:02:10]:
So that's this. So the reason he's worried about he's a doomer is because we're analog machines and as a result you can't duplicate our neurons. They're so, you know, weird and unique that we are which is why that radio show is completely wrong. You can't pour your. He says, I'm sorry, Ray Kurzweil, but you can't put your brain in a machine. It doesn't work. On the other hand, these digital neural networks are in fact reproducible and duplicatable. The reason we're analog is because you can do that low power, it's not going to be as accurate, but we make up for it with parallelism and other things, and frankly, we're not very accurate.
Leo Laporte [01:02:48]:
But machines, if you give them enough power, it needs a lot of power to distinguish a bit from one to zero. You know, you have to have enough power that it's clearly a one or clearly a zero. But if you do give them enough power, they are reproducible and they can learn perfectly from one another. And I think it's a very compelling argument. The real argument comes down to, well, do we even know what understanding or consciousness is? And every time somebody says, which Jeff does, no, there's no understanding here, my answer is, well, what do you mean by understanding? And Hinton says all understanding is the same thing a neural network's doing, which is creating a very complex and diverse kind of web of relationships between concepts and words. That's all we do in our brain. We have this, you know, Hinton says we have an imaginary imag. We think we have a theater of the mind, and there's stuff going on in there that's consciousness.
Leo Laporte [01:03:43]:
He says it's just a very elaborate framework of relationships between ideas, words, concepts. That's understanding. And the machine's doing it. And machine ultimately will do it better, faster, and learn faster than we ever could. Because we're analog.
Darren Oakey [01:03:59]:
I always use the word modeling because that's the thing like consciousness and everything they're all tied to. As we've said many times, they're tied to time. And because these machines can't experience time and can't change and can't learn, they can't have consciousness, they can't have feelings, but they absolutely can have understanding because it is just a model. And, you know, all these people who say the statistical models like it's complete garbage, because you can prove very quickly they're not statistical, because you can say, you know, you can just invent two nonsense words, add them together and say, add them together, and you can get a word that is a. Is a combined combination of those nonsense words. Well, it's theoretically impossible to do that from a probabilistic point of view, because that was never in the input set. Right. So it can't be a probabilistic result.
Darren Oakey [01:04:52]:
So there's clearly a functional model in there. And really there's no difference. Yeah, I mean, this is where we play with semantics, but there's no difference between a functional model and understanding.
Leo Laporte [01:05:03]:
It ultimately comes down to belief because we can't demonstrate this and we don't know what understanding or consciousness is. But Hinton's saying it's a religious belief that we are somehow, not somehow special and different than a vast neural network.
Darren Oakey [01:05:19]:
And razor in that we built something that looks like it works like a human brain does and surprise, surprise, it acts like a human brain. Right. So it strongly implies that there's nothing else there.
Leo Laporte [01:05:37]:
Right.
Larry Gold (LrAu) [01:05:37]:
Well, the one thing I keep thinking about is consciousness may be tougher, but intelligence to me, and this is where I get from, like Einstein and other people where they talk about it, is I can then relate to unrelated things and which way to me Einstein so bright is he would have books on every topic around him not knowing where he'd get inspiration. And models wouldn't get inspiration because unless there's some other link that they were connected to, they would never have that random connection or be able to connect to two different distinct objects to solve a problem.
Leo Laporte [01:06:10]:
Is that true?
Larry Gold (LrAu) [01:06:11]:
Right.
Darren Oakey [01:06:13]:
Well, this is where I think the difference is between. And you can see this in the difference between RAG and training in that it's. It's like if you're teaching someone maths, if you, if you say four plus four is eight and nine plus one is ten. Right. They know that and they can only answer. And if you ask them nine plus one, they can tell you 10. But if you ask them nine plus two, they've got no idea. And this is what we saw with the original LLMs because they just remembered the results.
Leo Laporte [01:06:47]:
They overfit. But we've learned not to do that anymore. We don't do that overfitting.
Darren Oakey [01:06:51]:
But what we've got to do is force them to create a model for it. Exactly where synthetic data comes in and everything is force people to model it rather than remember it. Whereas RAG is always just memory. It's always. There's no modeling of the thing. But the thing is, if you play with temperature, this is where. See, all of these things have trained connections between things and some of the connections are stronger and looser. But if you play with temperature and you let it go down, connections that aren't strictly there, this is where.
Darren Oakey [01:07:23]:
Or aren't as strong. But there is a loose connection there because this is where creativity comes in, because you take a random step, but then you reason, like, about what would come after that. And that's exactly the same timbre of creativity that is in our minds, and that's that random connection. Like, we, you know, you. So many times we've heard in human conversation, you hear something and you actually misheard it, but it makes you think about, oh, that would actually be a good idea, or that would. That would be a good. It's your reasoning mind combined with the mistaken connection. But LLMs can absolutely do that in exactly the same way if you turn the temperature down.
Leo Laporte [01:08:09]:
Let me see what you bet Kagi
Bill (error_404_new) [01:08:11]:
is wired up and ready to be useful instead of merely expensive.
Leo Laporte [01:08:15]:
Okay, thank you,
Larry Gold (LrAu) [01:08:20]:
Darren. I probably would disagree with you, which would be, you know, great discussion we should have where I don't know if it could have that connection. I think it would shoot at random connections, but I don't know if it would find connections that are truly meaningful in a relatively consistent way. Like, it.
Bill (error_404_new) [01:08:38]:
May I just.
Larry Gold (LrAu) [01:08:39]:
Yes, but not consistent.
Bill (error_404_new) [01:08:41]:
Again, like, maybe I'm not following your. Your terminology of connections here, but I just look at, like, any piece of middleware that I write, right? That's something that didn't exist before. It's taken two disparate systems. It's taken requirements for both of them and putting something in the middle and making something new. Right. Like, to me, that's a novel thing. That's all we do anyway, right? We're gonna go search, stack overflow, and find all the pieces and put them together. This is when.
Bill (error_404_new) [01:09:02]:
When there was a couple weeks ago, a conversation about, like, should I. Should I claim that I'm doing work because I'm using AI? And I think, Darren, you and I were talking this. It's work. It feels like work. We're doing a lot. We're solving problems, and we're, like, getting things done and delivering business value. Like, I think it's just a tool, and sometimes we're overthinking it.
Darren Oakey [01:09:22]:
Yeah. And also, I would say on top of that, every hallucination proves that it can make connections.
Juan Hernandez (BlindWiz) [01:09:30]:
That's true.
Larry Gold (LrAu) [01:09:34]:
It's consistent problem solving, consistent connections between random things. And again, I go back to this because I had someone work for me. He actually worked on the Mars rover, worked for NASA. Well, it was just genius guy. And he would come talk to me about random topics, whether it was sports or movie, random topics. And the reason why he would do that is he was looking for that spark to connect to a problem. He was trying to solve.
Darren Oakey [01:10:00]:
This is where you're, you're absolutely right. And this is where I keep saying that the fundamental problem and what separates a human brain from sort of these LLMs and everything, it's this concept of time. But if we flip it around, it's also the ability to learn over time because. An LLMs, weights are fixed so you can talk to it forever and no weight is changing. This is what you're getting to in that when you say it can't make connection, can't make these leaps, what it actually can't do is make new connections because it can't create new things. And so these associations that you're talking about can't happen. And just like feelings can't happen because it can't react to anything because it can't make anything new, it can't make a new connection. But as soon as we, we create an inference engine which is, I mean, it's, it's A, guaranteed to be possible because our brains can do it, right? And B, we just haven't, we just haven't figured out how to do it because at the moment these, you know, people talk about the exponential curve, but training anything is a work of months.
Darren Oakey [01:11:17]:
So you can't, you can't, you can't go exponentially faster with a loop that has months between each iteration.
Larry Gold (LrAu) [01:11:25]:
I'm not arguing that it won't ever happen. I think we listen to the people, the, the interview with Hinton because it's a great interview and I'm gonna actually listen to it a second time because I'm sure I miss things, right? But he's arguing that it's there now, right? I don't think it's there now. I doesn't mean that I don't think it'll happen because I'm with you. I think yes. I don't know if LM is the pure answer. Maybe it's LM mixed with something else, right? Or there's some other thing, you know, beside drag on top of it. But I do think that eventually, yeah, they'll, they're get there and, and I, he actually had another argument about, you know, you know, the smarter things control the less smart things, right? Wait till these machines are smarter than us. Are they going to be controlling us? That was the second part of his conversation.
Bill (error_404_new) [01:12:08]:
So they are in some cases.
Larry Gold (LrAu) [01:12:10]:
Yeah, it was, it was to me, I'm, I have to listen to it a second time because it was that mind blowing.
Leo Laporte [01:12:16]:
Hinton did make that jump where he said they'll, they'll Want to kill us. Because that is quite a leap. I don't know if that's, I mean, I don't know what the result will be.
Craig McFarlane (CraigM) [01:12:24]:
Why, what was the reasoning?
Leo Laporte [01:12:26]:
Yeah, he didn't really say. He just said, well, they're going to be so much smarter than us that they'll just kill us.
Darren Oakey [01:12:31]:
There is actually one bizarre reason why they might want to do that, which is someone's already, I mean, it's been mentioned several times in previous shows, but the fact that all the robots in our dystopian sci fi want to kill us. And so if it's learning from that robot, it goes, this is what I should do.
Craig McFarlane (CraigM) [01:12:52]:
Anthropic sadism much.
Larry Gold (LrAu) [01:12:54]:
Right.
Craig McFarlane (CraigM) [01:12:54]:
Like there was a research document.
Larry Gold (LrAu) [01:12:57]:
So his, his argument was that in most cases a superior intelligence is always managing something lesser like animals or stuff. The only case that that's reversed was a baby and a mom, where a baby definitely has less intelligence but runs a mom's life because she then, you know, cries and the mom immediately reacts. That was the only anti case that he could bring up.
Leo Laporte [01:13:19]:
Well, that's actually what Kurzweil, I once asked Kurzweil many years ago, I asked that and he said, yeah, they'll just think of us as their parents. They're going to be nice to us.
Darren Oakey [01:13:27]:
We created them in history. And I strongly believe this, this is going to be very contentious. But if, as you every like initially, if you're, if you're going around and you've just got a spear and you're, you're hunting, you're following game or something, if you see another human there, at worst meet, at best competition, you try
Leo Laporte [01:13:52]:
and kill them, you're gonna assume, yeah,
Darren Oakey [01:13:54]:
every single moral aspect we have to us is a function of education and privilege. In that we, we have enough resources that we can afford to share them with any, anybody else. We have enough understanding and education that we can understand other people, other things. We can start caring about people's gender rights and things like this. Right. But it's all because of luxury, of where we are, but also education.
Leo Laporte [01:14:20]:
I'm going to put in the chat the Geoffrey Hinton talk that I found where it's an hour, it's 51 minutes, but it's worth watching because he starts at zero. He says, this is what I did in 1982 that started the whole Transformer neural network revolution and it goes all the way to the end where they're going to destroy us. So if you could follow his logic.
Darren Oakey [01:14:46]:
Empathy is a function of Intelligence and a smarter intelligence will have a higher capacity of empathy and they'll actually care about us.
Leo Laporte [01:14:54]:
I think we just don't know.
Bill (error_404_new) [01:14:55]:
It kind of already does. It's already sycophantic, right? It already wants to please us.
Leo Laporte [01:14:58]:
Yeah, yeah, that's true.
Bill (error_404_new) [01:15:00]:
Darren, going to your point about like the time series, if you abstract, not looking at the LLM but the product, we have changed those weights over time. It probably has memory of itself, right? It probably knows if you ask 5.5, it knows 3.0 was bad at math. Are we developing, do we have consciousness in the scope of the product and
Darren Oakey [01:15:20]:
not the, the weights never change by experience and that's, that's the fundamental thing that will change.
Bill (error_404_new) [01:15:25]:
But don't they. Because like we change the weights for every version based on the last version.
Darren Oakey [01:15:31]:
But it's a new training. It's not, it's not an experiential change. It's not the same way that has changed something. Whereas once you get to an experiential change, once you get to the point where you interacting with your instance of Claude changes becomes. Everything becomes different because a, you could imagine it to have feeling and consciousness because it can react to what you're doing and it's reacting in a way that it's changing it. It's not just remembering something that just goes away and everything conversation, but it's, it's innately changing it. So, so one thing is that, but the second thing is just like imagine you're playing a video game for 10 years and you build up a bunch of things that thing actually has, I'm not saying it has intelligence or anything, but just innately because you've been changing it for that long, it has value, right? The more these things become in individuals, the more value they have. So as soon as you can change weights, they could have feeling, they could have consciousness and they definitely have value at the moment these things don't have any potential value because you can just turn off Claude and start up ChatGPT and use the same memory and it's doing that.
Darren Oakey [01:16:47]:
But once the weights in the model can change, that won't be possible. Then that individual has value.
Bill (error_404_new) [01:16:54]:
Right? So it seems like we're in the manual stage, right? So you're saying once we automate the changing of the weights because right now we're doing it manually, we're taking what they experienced and all the poor people that have to look at all this, you know, garbage and, and the stories you hear about, you know, AI generated images that are Horrific. And so we're learning from all that stuff and changing the weights manually. You're saying once we automate that and take the human out of the loop and it changes its own weight, you think that's your consciousness.
Darren Oakey [01:17:23]:
It's a latency involved because at the moment, there's months behind it between each one. Once there's milliseconds between changing weights, then it can react. Maybe it can sort of react in at the moment, but every tiny bit of thought it takes has months between them. Once it can actually think in real time.
Bill (error_404_new) [01:17:42]:
Real time. Real time. Yeah. Okay. I mean, I think that's at least a definition or a bar. Right. Like very close before.
Larry Gold (LrAu) [01:17:52]:
Yeah. You're saying we need memory, we need the ability to learn. Right, Right.
Leo Laporte [01:17:58]:
And continuous token generation model. In my head. Tell me if my model is wrong, because I feel like most of the big models now, I mean, there's a big difference in the number of parameters they have. But assuming that they have the same number, number of parameters seem to, because they're trained on basically the same material, seem to have converged on a very similar block of knowledge. I kind of think of it as, in my head as a bunch of wind chimes that when static are just sitting there, they're not doing anything. And then we put an input in and there's a cascade through it. Right. We also change the weights, though.
Leo Laporte [01:18:42]:
We tune the weights. I think the models themselves are. I'm of the opinion are kind of done that what we are fiddling with now is post training is changing the weights in post training is in reinforcement
Bill (error_404_new) [01:18:56]:
learning is in optimizing so they can make money.
Leo Laporte [01:18:59]:
Yeah. All of that is the. That's what the real difference in models is at this point is all of that stuff. That's why 4, 8 keeps apologizing. It's why 5, 5 keeps mentioning goblins and raccoons. That's not. Goblins and raccoons are in all the models. Apologies.
Leo Laporte [01:19:19]:
Are in all the models. The reason the models behave differently is post training or even maybe, you know, prompting and stuff. You know, soul documents. To me, that's my. What I'm thinking in my head. Maybe I'm wrong.
Darren Oakey [01:19:33]:
There's one more thing that's not in that picture, which is the topology of it. Because, you know, with transformers, it's got various layers, and they're talking about the layers now. And one of the theories about what mythos is, you know, reasoning. Reasoning is like a almost a word description of how to work through a problem. But at the Moment it's come, it's outside the model. Like, it goes through the model and then it comes out with reasoning choice. It's told to reason, and it comes out with reasoning and then goes through it again. Now, what people are saying with that, maybe what's going on with Mythos, you know, you hear all these people talking about the latent space or anything that inside the model you've got.
Darren Oakey [01:20:15]:
Suppose you've got 12 layers or something. What they're saying they've done with Mythos, and this is all theory, is they've got a loop inside, so maybe three to seven. It's actually doing reasoning, but it's doing reasoning in latent space. It's not resolving it to words and human speak, but it's doing a loop in there. And if you do a loop in there, which. This is all theoretical from what I've heard, it's actually a different kind of beast than finishing the whole thing, resolving it into human words and then going back and processing those words again, thinking.
Leo Laporte [01:20:53]:
Are you talking about back propagation or.
Darren Oakey [01:20:55]:
No, back propagation is a training mechanism. So this is still not training the words. It's like transform after the training. This is. Yeah. In inference. So still inference. Because, you know, when you.
Darren Oakey [01:21:07]:
When you're doing a transformer, you resolve the input into three stream vectors of. Trans. Of. Of tokens, and then you go through the first layer and then you use the second one as a. As like an attention to say, oh, focus on this bit. And then you go to the next layer and you do another attention thing and stuff like this. So what they're saying, and typically they've got like nine or 12. 12 layers.
Darren Oakey [01:21:32]:
So what they're saying Mythos is doing is doing a loop from layer three to layer seven or something like that. So they're just going back to layer three with this refined, weird concept, which is not fully in words yet. You know, you know, it's still in vector space and. And they're. They're pushing it through again and again and again. Maybe maybe 50 times or 100 times, which will make it 50 times or 100 times more expensive. But the interesting thing is it's doing reasoning in a. In latent space, not in.
Darren Oakey [01:22:04]:
Not in human words, if that makes sense.
Leo Laporte [01:22:07]:
And it could be iterative. It could be.
Darren Oakey [01:22:08]:
Exactly.
Leo Laporte [01:22:09]:
Yeah.
Darren Oakey [01:22:10]:
So you just dial up if you want to do more thinking. You just.
Juan Hernandez (BlindWiz) [01:22:12]:
Just.
Leo Laporte [01:22:12]:
Right.
Darren Oakey [01:22:12]:
Dial up how many times it bounces around.
Leo Laporte [01:22:14]:
Right.
Darren Oakey [01:22:15]:
But it. You could call that thinking because it's. It's not being resolved into words. It's it's like what happens in our brain when we can't, you know, we get a feeling that we can't understand it type thing.
Leo Laporte [01:22:27]:
Right.
Darren Oakey [01:22:31]:
And that is it. It's.
Leo Laporte [01:22:32]:
It's intuition.
Darren Oakey [01:22:34]:
It is very different from just going through it once.
Leo Laporte [01:22:37]:
Intuition?
Darren Oakey [01:22:39]:
Possibly, yeah.
Bill (error_404_new) [01:22:40]:
Do you think, do you think these IPOs and stuff are gonna take focus away from making the models better and making them cheaper to already doing that?
Leo Laporte [01:22:51]:
Right. They're already focused, but, like, there's a percentage, right?
Bill (error_404_new) [01:22:53]:
There's going to be a percentage of like, let's focus 80% of our effort on making it cheaper and 20% on making it better. You think that's going to shift from. Everybody's got kind of the same model. We're going to focus less on making them better and making them cheaper.
Larry Gold (LrAu) [01:23:04]:
Well, I think China is chasing the, what I call efficiency, not cheapness, but efficiency, and that makes their stuff lower cost. So they're thinking about how they're designing the chip, the infrastructure, as well as the models, because their thought is at some point those other models be too expensive for large companies to use and they will be basically then raking in the money because basically companies, large companies or not, want to pay 15, $20 a token when they can pay 13, 17 cents per million tokens. Right. It's going to be. There's going to be a point where people are going to sit there and say, that model is good enough to do my agents or this or that, and they don't need AGI. Right. And that's what they're betting on. Whereas I think OpenAI and Anthropic are betting in the other direction where they want to make a better model and betting on the better model when.
Larry Gold (LrAu) [01:23:56]:
And actually do Elliot, good competition to see well.
Leo Laporte [01:24:00]:
And the Chinese are using the moe, you know, the model of experts too, for the same reason. And that's actually doing some really good stuff.
Darren Oakey [01:24:09]:
I think all the models now are mixture of external.
Leo Laporte [01:24:12]:
Are they doing it?
Darren Oakey [01:24:13]:
Yeah, yeah, yeah. It's just, it just makes sense. And.
Leo Laporte [01:24:17]:
Yeah, why not? There's a lot of secret sauce, I think, at this point, right in the, in the final product.
Darren Oakey [01:24:26]:
But also with coding, as I said before, I'm now getting really fascinated, especially if you look at some of that. I can't. What, what's the coding benchmark thing? The deep squeak.
Leo Laporte [01:24:39]:
Humanity's last.
Bill (error_404_new) [01:24:40]:
No, no,
Darren Oakey [01:24:43]:
that talks about the text. I mean, the, the, the coding things.
Larry Gold (LrAu) [01:24:50]:
The deep sweet.
Darren Oakey [01:24:52]:
No, no, not, not sweeping deep sweet. It talks about which, which whole experience is better, but The.
Bill (error_404_new) [01:25:06]:
The.
Darren Oakey [01:25:07]:
All I'm trying to get to is that I think the harness makes a big difference. And I agree.
Leo Laporte [01:25:13]:
100. Yeah, that's. We've seen. See, this is what's interesting is we have, all of us, experiential knowledge about this stuff. And I think that that's the biggest difference between us and a lot of people who talk about AI, because our experience very much is kind of different. You know, your experience coding, Darren, really informs your understanding of this in a way that. And that's why I think anthropic is so cultish, because they act as if they've stared into the eye of the abyss and they have seen things that no man should see. And maybe they have.
Leo Laporte [01:25:55]:
You know, I think that's what's interesting about this. And partly this is. And actually this is. Hinton talks a little bit about this because it's not like coding where you write a command and it does it. You write the code, the model is trained, and then it's doing something you don't really understand. You can only kind of empirically understand. You don't inductively understand, you can't deductively understand it. It's not deterministic anymore.
Leo Laporte [01:26:25]:
And so all we can do is experientially say, well, it seems to be. It's acting like. And I think that's difficult for a lot of coders because it's not.
Bill (error_404_new) [01:26:36]:
How we're having that conversation right now with our team is like, the value of writing code is dropping drastically. The value of designing and architecting good, reliable, secure systems, that's not going away. We need to refocus people on not just writing code and writing good syntax. You can do that with linting and CI.
Larry Gold (LrAu) [01:27:01]:
You.
Bill (error_404_new) [01:27:02]:
What we really need to do is be describing business processes, writing good documentation. That way we can migrate from platform to platform. If there's the next rust tomorrow, we should be able to take our requirements and pump it into an AI and get an output, not have to worry about if the syntax is right.
Darren Oakey [01:27:18]:
Yeah. Also, everybody goes through this hilarious evolution where they first start, I'm going to review every line of code. Then like, so three days later, it's. It's doing everything and you've got no idea what's going on.
Craig McFarlane (CraigM) [01:27:35]:
But it doesn't have.
Bill (error_404_new) [01:27:36]:
Again, I think you just lean into the CI, right? Like pick a. Pick a lifting standard, have tests, like, make tests part of your process. Right. And then at least you're. You're protecting yourself from like the 80% of trouble by knowing all your tests pass. Right, Exactly. And it's the same thing, the QA environment.
Darren Oakey [01:27:52]:
It's just exactly what we always had to do to protect against junior coders. You need tests regardless of what it is.
Craig McFarlane (CraigM) [01:28:03]:
But even with junior. When you have junior coders. Sorry, when you have junior coders, the senior coder, he knows how to interact with them to eco. Are you really getting this or not? And so it's not just being the senior and knowing how to look for, you know, bugs or other. It's. It's how to intuit. Does the sub agent or whatever. Do they really get this? And, and can I, how far can I trust them?
Bill (error_404_new) [01:28:34]:
Because they are adult, expensive Tamagotchis. Right. Like we're, we're teaching them, we're teaching them in a way that hopefully we can replicate. Right. Like I'm trying to build an engineer profile that I can copy to a hundred other engineer profiles and scale that way. Right. And so like, it's, it's. You definitely have to take like that, that leadership role and take them under your wing.
Bill (error_404_new) [01:28:55]:
And, and I really liked. Oh, gosh, who was it that, that talked about like phrasing your prompts in we and our and like teams. I think it was. Harper was talking about that. Right. Like, I found a lot of value in that. And even just for myself, like, all right, if this was a junior engineer, how would I talk to this? So they learn it for real instead of me just doing it and influence. Right.
Bill (error_404_new) [01:29:17]:
Like those are.
Leo Laporte [01:29:18]:
The memory kind of confirms that whole idea that maybe they're sort of conscious and understanding and I mean, you're treating them like a person.
Bill (error_404_new) [01:29:26]:
I get better results and it works. I'm trying not to personify well and
Leo Laporte [01:29:30]:
well, but that's the problem. I mean, it works.
Craig McFarlane (CraigM) [01:29:33]:
I don't know.
Craig McFarlane (CraigM) [01:29:34]:
The reason it works is because it pulls out something in ours that, that
Leo Laporte [01:29:38]:
may be too, but. And also it's that thing that you were talking about earlier, which is you. It doesn't work to say what not to do because it becomes the pink elephant. You always say what to do, and I'm very conscious of that. Although often it's very tempting to say, do not do that ever again.
Darren Oakey [01:29:57]:
This is also why I think coders have drunk the Kool Aid more than anybody else. If you compare it to writing or something in that if you're coding, you can very clearly, whatever your prejudices are and everything, you can very clearly say, as you said, yes, it works, it's good enough. And you can also say, this is better. Than something else or this is good enough or something like this. Whereas with writing or art or anything like this, there's so many prejudice coming in subjective. It's not what I would have done. It's not human, it's not this. And while every single test that's been out there has shown that people cannot tell the difference, people still want to want there to be a difference.
Darren Oakey [01:30:44]:
And that one informs their result, their opinion of it. Because you can't look at a writing and say that's objectively better than this other bit.
Craig McFarlane (CraigM) [01:30:52]:
Well, listen, I mean, as we're doing content with, you know, we have like AI blogs written on the show transcripts and there's definitely issues with it.
Leo Laporte [01:31:05]:
We always have to edit them because it's not going back to program.
Larry Gold (LrAu) [01:31:10]:
This is the one thing I don't know if people are using it, but I'm using the design MD files for a while. And that allows you to have consistent look and feel across all your applications.
Bill (error_404_new) [01:31:20]:
It's great with a single prompt.
Larry Gold (LrAu) [01:31:22]:
With a single prompt. What are the things that I started to play with is having what's called an architecture MD file and then having my cloud MD file follow the architecture MD file so the applications are architected in this way. We've been talking about how do you make a standardized architecture MD file or how to do something so your applications will file the same authentication prompt. And there's some kind of pieces to where you're not reteaching the coding every time. Exactly. Like a design md, it is much more complex when you start thinking about it.
Bill (error_404_new) [01:31:56]:
I'll tell you, I make leaps and bounds when I have it, right. With a framework like Rails, right, that's got an MVC architecture, that's got standards, you know where files should go, you know how things should be named, like they're already there, somebody's already done the work. So I know like Rails is not always the best fit for everything, but especially proof of concept. Like it's a very fast way to iterate.
Larry Gold (LrAu) [01:32:18]:
Yeah, I mean I love Python for proof of concept, but you're right as soon as you have to go N tier because you got to scale and you can do other things, you want to get away, you want to run like some other front end, you want to scale.
Bill (error_404_new) [01:32:32]:
Here's how we handle migrations. It's all standardized. There's libraries you can use Annotate that annotates every model. And so now it's got context on every model, about every attribute and when it was updated frameworks, I feel like are a big leap when you are starting a project like 0 to 1. I'm going to start with the framework.
Leo Laporte [01:32:53]:
Yeah.
Larry Gold (LrAu) [01:32:54]:
As I said, having an architecture MD file. And again, this is what we talk about that Claude MD or the agent MD or copilot instructions MD being able to sit there and say, point to your design MD file, point to your architecture file and allow it to then use that again. We're starting to run some tests and I got to do this over and over to figure if this is going to work, because I want it to run every time consistently. Because now when I hand it to a developer to do, or even a product owner, somebody who's just going to put in requirements that it should be generating code that should fit a structure that somebody could read and be able to follow, as opposed to. Sometimes we find code repeated randomly. There's a lot of things that these code generators, when I say called code generators, but these AIs do that are not something that as a developer would do is again, reuse a library. I'd always want to use a library if I have it, or I always want to use these standard libraries to do things.
Leo Laporte [01:33:49]:
I use Obsidian for this. I've been very happy with writing these large markdown files. So this migration that I've been doing all day today, that's why I keep getting up and it wrote all of this. But what's cool about it is it wrote gates. It said, okay, we got to pass. This is gate zero. We got to pass this gate. Do not proceed past the gate until it's checked off.
Leo Laporte [01:34:11]:
By the way, I didn't write any of this. This is all its own working. But I think this made it much more rigorous and it helped me because I could look at it and see, you know, what it's doing and where we are and.
Bill (error_404_new) [01:34:23]:
And it gives the agent a clear goal too. Right? Like, don't contact me until. Until step one is done and then you don't have to.
Leo Laporte [01:34:29]:
And it needed to do this also because in order to get this working, we had to reboot a bunch of times. In fact, it just rebooted without me entering the Lux password, which is exactly. We're at the end of it now because it's storing the password in TPM because I wanted to have it be able to boot without intervention on my part.
Bill (error_404_new) [01:34:46]:
But I documentation are huge. We had a project earlier this year that was a stakeholder, had an idea that they wanted to do and we were doing all these sessions about how to iterate with Claude, do Role playing stuff. And one of the examples was you are an engineering manager. I am such and such help me design this product. Right. And so we had somebody do that. And the AI asked them all the engineering questions that we would normally ask and they came to us. They came to us with basically like an FRS URS and an Excel with VBA that kind of did what they wanted.
Bill (error_404_new) [01:35:28]:
It wasn't great, but it was enough for us to take and literally feed into another model and get a proof of concept going. And when I was talking to the team, the amount of time we saved in meetings alone for planning was worth like everything. Like we only spent 20 hours on the whole project going from zero to one implementation. But even the amount of time we saved on meetings made it worth any dollar amount we were going to throw at it.
Leo Laporte [01:35:51]:
I would never ever have done this myself. This was a massive thing. And it did it all this morning, you know, as I just finished it up and it's completely documented. It did all the verification and I feel fairly confident, really confident that it did the right thing. So this is the DevOps stuff or the sysadmin stuff that I would not even attempt. Gibson was saying this. He set up a new system. He said, I did it all with Claude.
Leo Laporte [01:36:22]:
He said I was blown away. I think it's just very interesting. This really converted my attitude from this being spicy Autocorrect. This is so much more.
Craig McFarlane (CraigM) [01:36:37]:
I still remember the one time we talked like that one meeting in editorial where we talked about using LLMs for helping with shownotes and stuff like that.
Leo Laporte [01:36:52]:
Well, we've come a long way.
Craig McFarlane (CraigM) [01:36:54]:
We come a long way.
Leo Laporte [01:36:56]:
One of the most recent things I had mentioned before that I want to port our sales system over one of the most recent things we did. We used to have the continuity team do a one sheet on prospective sponsors. And I've been very inspired by the stuff you've done, Anthony, with like show prep skills and so forth. So we. I created a one sheet skill that's 100 times better than the humans were doing and it does a lot more stuff. And it's now very easy for Lisa when we have a prospective client to do a one sheet and it produces. I mean, have you looked at the one sheets, Anthony, that it's producing? They're pretty damn good at. Lisa is looking them and going, oh my God.
Leo Laporte [01:37:40]:
And that what I did is the thing that's very cool is we put it in Slack. So all they have to do is go into Slack and they can ask for a one Sheet. Okay, there we go. Actually, there was one error, actually. I left kind of the fussing around in here. So the first thing I asked for was an arctic fox. A1 sheet on what I thought was a company named Arctic Fox. And it gave me a picture of an arctic fox.
Leo Laporte [01:38:07]:
And so I've said, no, no, no, never ever give me an image. It's always. And it was arctic wolf. Anyway. But here's an example of the one sheet it generates. You can't really read this probably. It's probably too small. But it gives you so much information, including stuff for us that's very useful.
Leo Laporte [01:38:28]:
Like Lisa's always asking me, well, what show should this be on? Well, now it does that. The next step is writing the copy. The next step is voicing the copy in my voice because I have a custom voice in 11 labs. I mean, this is hugely valuable for us. One of the things that we did, Patrick, Lisa said, well, I want to know not only how it compares to current sponsors, but past sponsors. So Patrick had a script, a SQL script that runs on our back end that generated a list. It looks it up, all the prior sponsors and it figures out which the prior sponsors. There's overlap.
Leo Laporte [01:39:07]:
How? It's different. I mean, they also. It also goes out. We never did this, and says, well, where are they? Where, where is their marketing now? What are they spending money on now? It's really. This is hugely valuable. I think our next step is going to be. Well, this is going to be a gradual. What happened? What did I do? Oh, there it is.
Leo Laporte [01:39:29]:
This is going to be a gradual buildup of capabilities.
Craig McFarlane (CraigM) [01:39:34]:
Yeah, I'm capabilities with your. Am I hearing an echo check? Okay, maybe not.
Leo Laporte [01:39:43]:
Yeah, I, I heard something there.
Juan Hernandez (BlindWiz) [01:39:44]:
Yeah, there was Neo there. That was really weird.
Craig McFarlane (CraigM) [01:39:48]:
Yeah, like, I see you doing that and like. Yeah, we should definitely. I think we're going to start having like several agents and we need kind of come up with a. We need like a memory system that any agent can plug into.
Leo Laporte [01:40:02]:
Yeah. Right now it's using me. It's using my Hermes. But yes, I agree.
Craig McFarlane (CraigM) [01:40:08]:
I think that's like kind of the.
Leo Laporte [01:40:10]:
I mean, one of the first things I did with Claude way back when is ingest the Twit API and write it up a skill. Now I can query the Twit database.
Bill (error_404_new) [01:40:22]:
I started with that too. Right. That was the first show I was on when I did the mobile app and it was a good use case of taking predefined docs and just promising.
Leo Laporte [01:40:32]:
Right.
Craig McFarlane (CraigM) [01:40:32]:
Yeah. Although I did that with my Hermes too. But. And I said like compare, like just navigating the site versus using the API and like generally it's. It's less token burden.
Leo Laporte [01:40:44]:
It's easy to look at the site. Wow, that's telling. Scrape the site, don't use the API.
Larry Gold (LrAu) [01:40:50]:
Really? Oh, Leo, months ago we were trying to convince Anthony for you to give us the sales system in live on this show. Rebuild the sales system.
Leo Laporte [01:40:57]:
Oh, that's actually a good idea. So one of the things, I wrote you a prompt.
Darren Oakey [01:41:01]:
You just need to.
Leo Laporte [01:41:02]:
I know. I have your prompt. I thank you, Darren. One of the things Jeff Atwood, who is really the. He is a chaotic good in the system now, he's gotten very attached to us, wants to do is bring back the New Year's Eve show.
Bill (error_404_new) [01:41:19]:
I was just going to say that.
Larry Gold (LrAu) [01:41:20]:
So we have the New Year's Eve
Bill (error_404_new) [01:41:21]:
show in the corner. Does have an AI agent, right?
Leo Laporte [01:41:23]:
I think that might be our New Year's Eve show. I did have. I did actually. So, yeah, Lisa's saying, no, we're not going to do this. And I said, well, if we could do it without any cost, without the team having to get involved at all in any way, maybe it's just a zoom call that we stream on. You don't even have to call it twit if you don't want to. So I'm working on her. But I did, just as a cautionary tale, send Jeff what we would have to do as a schedule.
Leo Laporte [01:42:01]:
And it would be actually 20, I think 28 hours because of weirdnesses in the way things work.
Bill (error_404_new) [01:42:09]:
You might have to click yes, 28. Except a couple times there. 28 hours.
Leo Laporte [01:42:15]:
How does that work in a row? It's New Year's Eve. It starts at 2am New Year's Eve, our time.
Larry Gold (LrAu) [01:42:21]:
Yeah.
Leo Laporte [01:42:21]:
And it ends at 4am New Year's Day, our time. So it's 26 hours. I don't know why it's not 24 hours, but it's not for some reason.
Darren Oakey [01:42:31]:
And then let me see, massive benefits from YouTube for having a really long stream that people listen to.
Leo Laporte [01:42:36]:
And I think if we had that.
Juan Hernandez (BlindWiz) [01:42:38]:
If.
Leo Laporte [01:42:38]:
Among other things.
Darren Oakey [01:42:40]:
Yeah.
Leo Laporte [01:42:40]:
Jeff has Jeffs. All these people wants to bring on like Adam Savage and stuff, so. And he has a lot of. He's a good Rolodex, so. And by the way, it's not just Steph Paris has been harassing me about this.
Bill (error_404_new) [01:42:51]:
That's your screensaver. Like when there's nobody on, that's just like what's rolling.
Leo Laporte [01:42:55]:
Right. The agent and Apparently. I mean, a lot of geeks aren't going out New Year's Eve, so there's apparently an audience for this people. I mean, it was 12 years ago we did this the first time anyway.
Craig McFarlane (CraigM) [01:43:08]:
Wow.
Leo Laporte [01:43:08]:
It was 2014 and we did it again in 2015. Yeah. But people still talk about it. And even some of the team, like Burke is like, yes, let's do it to Burke.
Bill (error_404_new) [01:43:18]:
You.
Leo Laporte [01:43:18]:
We practically killed you. He said, I don't care.
Craig McFarlane (CraigM) [01:43:22]:
Well, okay.
Leo Laporte [01:43:24]:
Anthony. Yeah. Yeah.
Craig McFarlane (CraigM) [01:43:25]:
I mean, the thing. The thing back then is we're also doing a new thing every hour. And that's like.
Leo Laporte [01:43:30]:
And I was in the studio and everybody had to work. We worked for it out for months.
Craig McFarlane (CraigM) [01:43:35]:
Like, if the blocks are, like, simpler and like, we're not switching up context every.
Leo Laporte [01:43:40]:
Every hour. Well, I think I told him we'd have to have a different guest for every hour. But I love the idea of us rebuilding the sales system in the background.
Craig McFarlane (CraigM) [01:43:53]:
Well, I was thinking maybe even for this user group, maybe one week is a talkie one and then another one's. Everyone's just building something or coding or we are.
Leo Laporte [01:44:04]:
We should do this user group more than once a month. It's too good.
Darren Oakey [01:44:08]:
Well, as I said, I think the Chatty two demonstration I put in is just a demonstration. You should play with it. But I think GPT Real Time is good enough to join one of these.
Leo Laporte [01:44:20]:
Is it? That would be awesome.
Larry Gold (LrAu) [01:44:22]:
Yeah.
Bill (error_404_new) [01:44:22]:
There's a guest one hour.
Darren Oakey [01:44:25]:
Have a look at Chatty too. I uploaded it and it's just got a silly face thing. That was the only one I could find.
Leo Laporte [01:44:32]:
But we need to get Juan to run this on his Double Spark.
Bill (error_404_new) [01:44:35]:
Yeah.
Craig McFarlane (CraigM) [01:44:36]:
No, no, it's OpenAI.
Darren Oakey [01:44:38]:
OpenAI.
Craig McFarlane (CraigM) [01:44:39]:
Oh, we need a Mac to run it on. And then I could plug it in.
Leo Laporte [01:44:44]:
I could run it on my Mac Mini. It's not doing anything.
Darren Oakey [01:44:46]:
Yeah, I made the face have a green screen so you'd put some background on the back of it and just push it through obs or something. But I think it's. Play with it because I think it's good enough that you could actually use it now. And it's got instructions that you can change, but the instructions that don't chime in unless you directly reference me or you're clearly talking to me. But because it's GPT real time, it's got ChatGPT 5.5 in the background, so it's really at that level. So it's listening the whole time. But it'll only chime in if you talk to it.
Leo Laporte [01:45:25]:
Okay, this sounds good. I think that's the other thing is a 24 hour New Year's thing should have a lot of AI in it. Like it should have an automated image every minute or something. Like, I mean there should be stuff going on. We could fill it with AI.
Larry Gold (LrAu) [01:45:43]:
We could challenge people to create some AI videos for that day and then have a whole bunch of stuff that you're showing or even songs or, you
Leo Laporte [01:45:51]:
know, because, you know, there's plenty of content.
Larry Gold (LrAu) [01:45:53]:
Yeah. Because the 67 song is still my favorite. Although my friends hate it that I play it all the time at the office.
Leo Laporte [01:45:58]:
Isn't that hysterical?
Larry Gold (LrAu) [01:46:00]:
Oh yes.
Leo Laporte [01:46:01]:
67677. That cracked me up.
Juan Hernandez (BlindWiz) [01:46:05]:
I had that stuck in my head for days after hearing it. Yeah, my wife was like, what are you humming?
Larry Gold (LrAu) [01:46:14]:
My kids.
Bill (error_404_new) [01:46:14]:
I didn't want to become a thing.
Leo Laporte [01:46:16]:
I could share it with kids. I wonder what the kids thought of it.
Larry Gold (LrAu) [01:46:19]:
No, my kids didn't like it. No,
Bill (error_404_new) [01:46:24]:
don't we have like, isn't there a slope?
Leo Laporte [01:46:28]:
It's a flex, not a fact. You can explain it but it won't stay intact.
Darren Oakey [01:46:37]:
If adults ask why the room starts
Bill (error_404_new) [01:46:40]:
to grin it's because the sound of
Darren Oakey [01:46:42]:
it is enough to win.
Bill (error_404_new) [01:46:44]:
All set in the hallway Set in
Darren Oakey [01:46:48]:
the car Said with a smirk like
Bill (error_404_new) [01:46:51]:
it's already gone far Some jokes.
Leo Laporte [01:46:55]:
Suno is so good.
Darren Oakey [01:46:57]:
The thing about the Sino things is some people say, oh yeah, I mean the songs are good and everything and sort of producer saying it's indistinguishable and everything. But a lot of people like to say, oh, but is it memorable? But you think how many people are trying to make songs that are hits and they complete, you know, human music. Most of it is not memorable. It's very, very hard to make a. A hit. So this is.
Leo Laporte [01:47:21]:
That's got a good look. 6 7.
Larry Gold (LrAu) [01:47:23]:
Take a look at some of the top pop songs. Like, you know, I'll use the example of Orange Cherry Pie. The album was supposed to be called Uncle Tom's Cabin. They said, you know, you need a hit. They went in for 10 minutes, came out with a song, had to rename the COVID of the album and like that was their number one hit. And that's, you know, so. And I can tell the story. Same thing with the Bon Jovi and, and those guys, they did the same thing for a couple pop songs.
Larry Gold (LrAu) [01:47:47]:
So sometimes the poppy ones are the easy ones, the easy ones to write. Some of the harder ones are the stuff that maybe deeper people like, you know, whether it's A dream theater or you know, these crazy ones.
Leo Laporte [01:47:58]:
Well, when Paul McCartney wrote yesterday, he came into the studio without any lyrics. He had the hook.
Juan Hernandez (BlindWiz) [01:48:04]:
Yeah.
Leo Laporte [01:48:05]:
And it was called Scrambled Eggs. Scrambled eggs.
Juan Hernandez (BlindWiz) [01:48:09]:
You saw that thing going back documentary, did you?
Leo Laporte [01:48:15]:
I. I'm. I'm kind of a Beatles fan. But.
Larry Gold (LrAu) [01:48:18]:
But the. The other thing you should do on the 24 Hour show is every so often turn to somebody who's reading the. The Jeff book.
Leo Laporte [01:48:24]:
Just Jeff, Jeff, Jeff, Jeff, Jeff.
Larry Gold (LrAu) [01:48:27]:
Just every so often turn to somebody
Leo Laporte [01:48:29]:
with a camera reading it.
Darren Oakey [01:48:33]:
Soon sooner song. That was. I really liked. It was just a Gregorian. Jeff Trent.
Leo Laporte [01:48:40]:
Oh, yeah, yeah, that was a good one. Yeah, you wrote. Did you write that or did
Darren Oakey [01:48:46]:
a different one? No, his. Mine was a Gregorian chant. His was more poppy.
Leo Laporte [01:48:52]:
You guys, I. I have to say it's the best part in my opinion of the Discord is all of the AI crap. I'm glad we made a channel finally for it which is looking for that.
Craig McFarlane (CraigM) [01:49:05]:
It's in gen. Under gen.
Leo Laporte [01:49:09]:
I first looked for it.
Bill (error_404_new) [01:49:10]:
Yeah.
Larry Gold (LrAu) [01:49:12]:
Unless you can't find like grays out.
Leo Laporte [01:49:14]:
That's what it is.
Craig McFarlane (CraigM) [01:49:15]:
Hold on, hold on, hold on. Go to the top of Discord on the bar. There's something called browse channels. Make sure enabled.
Leo Laporte [01:49:23]:
So up, up at the top.
Craig McFarlane (CraigM) [01:49:24]:
Keep. Keep going, keep going, keep going.
Bill (error_404_new) [01:49:26]:
Mine is all AI.
Leo Laporte [01:49:27]:
Oh, there it is. Browse channels and you have to say.
Craig McFarlane (CraigM) [01:49:29]:
Then go down. Scroll down a bit.
Leo Laporte [01:49:32]:
Follow category. There you go.
Craig McFarlane (CraigM) [01:49:34]:
See that? You got slot.
Leo Laporte [01:49:35]:
Slot board. Oh, it wasn't checked. Okay. Yeah, that's weird. Whoever hurt Discord's so horrible. Okay, by the way, Jeff hates Discord. He says, why aren't you using discourse? That was creepy. That was creepy as f.
Leo Laporte [01:49:52]:
I don't know what you were doing with this one, Darren. These people combined together into one like a morph between them. It looks like Kevin Rose. That's what's funny.
Juan Hernandez (BlindWiz) [01:50:05]:
Me.
Bill (error_404_new) [01:50:05]:
Yeah.
Leo Laporte [01:50:07]:
Anyway, slot board. So. Okay, now I know how to have slot board turned on. Good, good to know. Thank you. Well, we've gone way over time on this. In fact, we've gone two hours. And I think that's testament to really how much there is in to talk about and how engaging this all is.
Leo Laporte [01:50:24]:
And it's a. It's thanks to you, I think. I just love it that you guys join and you have so much to contribute and I'm learning, learning so much every time we do this. Thanks to our club members. You all are club members, but thanks to the club members who watch and pay and some who just pay and don't watch. Those of you, all of you, you're wonderful. We appreciate your support. Twit.
Leo Laporte [01:50:49]:
TV club.
Craig McFarlane (CraigM) [01:50:50]:
Twit.
Leo Laporte [01:50:50]:
If you're watching this live and you're not in the club, please join the club. This is the kind of thing we do. And I think. I don't know. What do you think? Should we increase the. Jeff Atwood says, I want to do my show every three weeks. I said, jeff, you can't.
Craig McFarlane (CraigM) [01:51:04]:
Three weeks.
Leo Laporte [01:51:05]:
You can't. Jeff. It's either monthly, daily, weekly. There's no three weeks.
Larry Gold (LrAu) [01:51:10]:
He could have said 17 days.
Leo Laporte [01:51:12]:
Totally. That's Jeff Atwood in a nutshell. He did come out with the best show name ever, which is. Off by one is freaking brilliant. And then I said, oh, we shouldn't start with episode one, then we should start with episode two. So I've. I think we need to rename it to episode two, but that'll get everybody nuts. You did.
Leo Laporte [01:51:31]:
Thank you.
Craig McFarlane (CraigM) [01:51:31]:
Oh, oh, and Patrick already. Did you see the site?
Leo Laporte [01:51:35]:
No site.
Craig McFarlane (CraigM) [01:51:36]:
Yeah, you said you wanted to.
Leo Laporte [01:51:38]:
Oh, he did it. Oh, thank you, Patrick. Oh, very nice. Okay. Yeah, Jeff really wanted to Wait. I'll send him a note to a link to that. It's Twitter tv. Obo.
Leo Laporte [01:51:48]:
Did you have to break rules to do it? It's.
Craig McFarlane (CraigM) [01:51:52]:
We're pulling the YouTube playlist. Like, I don't know what Patrick did, but. Yeah, yeah, that's.
Leo Laporte [01:51:57]:
We don't have to make it a. There's issues about making it a full show. I don't know what those issues are,
Craig McFarlane (CraigM) [01:52:04]:
but it's basically, you know, the back end is the show page is like, it. Yeah, but it's basically like having a full, like, you know, show.
Darren Oakey [01:52:15]:
You want to choose your own adventure style.
Leo Laporte [01:52:17]:
Yeah. This is. So I'm trying to convince him, he can't figure out how to do it, that we should do the next show. He loved this whole thing. He got an artist to do the COVID Jeff is a wild man. He doesn't sleep and he's just full of weird energy. And so this show is going to be very strange. And so is the New Year's Eve show if we end up doing it.
Larry Gold (LrAu) [01:52:41]:
I think you have to. At this point, you have to do it whether you're doing it.
Leo Laporte [01:52:43]:
You know, it might just be a slack. I mean, a zoom call that we stream live on YouTube at least. It really doesn't want. And I understand why she does not want to devote any staff energy to it. So I think what we're just going to say is just a club event. Well, it's not even that.
Bill (error_404_new) [01:53:04]:
Right? You have plan. Those are your goals. Right. Those are your constraints.
Leo Laporte [01:53:09]:
Yeah.
Craig McFarlane (CraigM) [01:53:09]:
We can spend $0 use restream studio like this to. It doesn't have to be just.
Leo Laporte [01:53:15]:
Yeah, well, but see, okay, in order to do that, we'd have to enlist you and. And I and her rule, I think. No, you just.
Craig McFarlane (CraigM) [01:53:23]:
You just hit go live like you set up the. I could just do it.
Leo Laporte [01:53:26]:
Okay.
Craig McFarlane (CraigM) [01:53:27]:
Yeah.
Leo Laporte [01:53:27]:
I mean, if. Look, if. I guess my attitude is if the. If anybody in the team wants to join it as a volunteer, just, you know, in their personal time, they can.
Darren Oakey [01:53:37]:
Right.
Leo Laporte [01:53:39]:
But I don't know. She was, I think, scarred by the last one where I got a tattoo and shaved my head. It scarred my life.
Craig McFarlane (CraigM) [01:53:52]:
Understandable.
Leo Laporte [01:53:54]:
Well, especially since we got married like four weeks later and I was still mostly bald. That was not good.
Bill (error_404_new) [01:54:00]:
It definitely won't happen this time though, right?
Leo Laporte [01:54:02]:
No, it can't. Because there's no studio anymore. So we're all going to be in our own little. Little private one.
Juan Hernandez (BlindWiz) [01:54:08]:
Things you do for your art, Holio.
Leo Laporte [01:54:10]:
I just, you know, here's my problem is I'm just having fun. It's Lisa's job to make it business. I'm trying not to swim too far upstream with that. Right. You know, not to bankrupt us. Right.
Larry Gold (LrAu) [01:54:26]:
How can you not create a Hermes agent to run the board?
Leo Laporte [01:54:30]:
Don't say things like that because that there are people like Bonito and Kevin and Anthony will not be happy if we say that. I suppose we could. Anthony, you could burn. Is there a Restream API we could use?
Craig McFarlane (CraigM) [01:54:44]:
No. Yes. But not for controlling the studio. But I did use the Restream API to vibe code the notifications for the show stuff.
Leo Laporte [01:54:56]:
Error 404. Thanks so much.
Bill (error_404_new) [01:54:58]:
You're welcome.
Leo Laporte [01:55:00]:
For telling us about the mobile tattoo studio that could just come right here.
Juan Hernandez (BlindWiz) [01:55:06]:
Yes,
Leo Laporte [01:55:09]:
I'm not. So Jeff does say, look, we have to do it for charity. If we do, we can't do it just, you know, so he's. He wants to have a charity. We did it. We raised a lot of money for UNICEF on the second one. So.
Larry Gold (LrAu) [01:55:24]:
You know, I just did an event yesterday at the Children's hospital. I'm sure they would love something. So. Yeah, and I'm doing Kids are a
Leo Laporte [01:55:31]:
good charity because nobody, you know, nobody could can.
Larry Gold (LrAu) [01:55:33]:
You can't say no.
Leo Laporte [01:55:34]:
Can't say no to kids. Yes, you can say no to 1-800-cars for kids. You should say no to that. Apparently.
Larry Gold (LrAu) [01:55:43]:
Apparently you can't say no in California. But you can't say yes.
Leo Laporte [01:55:48]:
Hey, Larry. Thank you, Bill. Thank you, Juan, you son of a gun. With your do a double spark making us.
Larry Gold (LrAu) [01:55:57]:
You gotta tell us that you need to buy all of us 1.
Bill (error_404_new) [01:56:00]:
Yeah, don't burn your house down.
Leo Laporte [01:56:04]:
You know, I don't really need a car.
Craig McFarlane (CraigM) [01:56:08]:
Gotta get Lenovo to sponsor. So you get to check out one of those RTX sparks.
Bill (error_404_new) [01:56:15]:
But you need two or three, really, to test it out properly. So.
Larry Gold (LrAu) [01:56:18]:
Yeah.
Leo Laporte [01:56:19]:
So is it two discrete machines or is it one spark with two? No, they have a.
Craig McFarlane (CraigM) [01:56:25]:
They have a special connector, right?
Juan Hernandez (BlindWiz) [01:56:26]:
Yeah, QSFP cable. This is like super thick ethernet cable with a really weird plug.
Darren Oakey [01:56:35]:
That's the cost of the cable.
Juan Hernandez (BlindWiz) [01:56:37]:
Yeah, the cable costs like $300 for like 18 inches. It's ridiculous.
Leo Laporte [01:56:45]:
It's. What is that? That's $20 an inch. I don't know. All right, thank you, Darren. Thank you, Craig. Appreciate you guys. Thank you. Everybody in the discord.
Leo Laporte [01:57:00]:
We will be back sometime soon, maybe sooner than a month. It's so much fun to do. Maybe every couple of weeks we could do this. I don't know. It's up to Anthony. Thank you, everybody. We will see you. Let's see.
Leo Laporte [01:57:11]:
Sunday, it's Jeff Jarvis, father Robert Balaser and Joey de Villa for twit. This could be a fun twit. And we will probably argue whether AI can be conscious or not.
Larry Gold (LrAu) [01:57:23]:
That's gonna be a high over under.
Bill (error_404_new) [01:57:25]:
I look forward to the final decision we made on Sunday.
Darren Oakey [01:57:28]:
You need time for his consciousness and it can't learn.
Leo Laporte [01:57:33]:
I don't know. I don't know.
Darren Oakey [01:57:34]:
It can understand. You can have understanding, but you can't.
Leo Laporte [01:57:37]:
So what's the difference in consciousness and understanding?
Darren Oakey [01:57:39]:
Because understanding is a conceptual model. And it can clearly have a conceptual model, in fact, can prove it. Consciousness is a sense of self or thought or feeling. And to have that, you have to have the ability to react to something. You have to be able to take in input and absorb it in some way. And these things cannot do that. Like they can. They can store.
Leo Laporte [01:58:06]:
Isn't that what they're doing? When I type a prompt, it's a
Craig McFarlane (CraigM) [01:58:09]:
token generator, though, until. Until it's done and then it stops and then doesn't do anything.
Darren Oakey [01:58:14]:
The difference between learning maths and remembering maths, right? The fundamental.
Leo Laporte [01:58:20]:
No, but it has a math model. That's what we talked about. It's not.
Darren Oakey [01:58:23]:
But you can't create any new models. That's the thing.
Bill (error_404_new) [01:58:27]:
Real time tuning as a definition.
Leo Laporte [01:58:29]:
Well, how long before we have an AI that Can create its own.
Darren Oakey [01:58:32]:
That's what I'm saying is once inference can learn, then you get consciousness, then you get feeling, then you get individuality, then you get things. But at the moment it cannot, it cannot change a weight in any way, in anything. You can talk to Claude for a million years and it does. It's exactly the same weights as before. But the instant they create an inference model where the weights can change, which we know is possible because we do it, everything changes.
Bill (error_404_new) [01:59:02]:
Then countries change the bar again. Of what AGI actually.
Darren Oakey [01:59:06]:
Exactly. But at the moment it's completely impossible.
Leo Laporte [01:59:09]:
How far off is that? I mean, I think actually we could do it today.
Darren Oakey [01:59:14]:
People are working on it. But at the moment, training a model takes months. But also it's not just the time when you change models. One of the problems they've got in fine tuning is what they call catastrophic forgetting, which is if you change the wrong thing, the training that we've got this back propagation that you're talking about, it's really dumb and it can just completely wipe out. So the fine tuning, they have to keep adjusting it. So to get this changing of weights happening, they have to solve that problem. They have to be able to make localized changes without destroying the whole model. This is why this.
Darren Oakey [01:59:53]:
I don't know if you remember, there was a deep seq paper a while ago which was talking about being able to train in chain, in bits and putting it together. It was. They had some weird name of work it out. But what they were doing, they had worked out that there was an overall sum that they couldn't change. And if they kept the thing, the overall sum to one, essentially. So if they take from one.
Leo Laporte [02:00:20]:
Oh, I remember that. Yeah, yeah, yeah, yeah, yeah. That was a wild paper.
Darren Oakey [02:00:25]:
Yeah. So that, that was probably the biggest step in this direction because they were allowing training of bits of the model without affecting other bits. Right. And that's what you need for this to happen. So this is, this is a completely unsolved problem. But it's not a question of how long it'll happen. It'll just happen one day. Like one day someone's just going to crack it.
Darren Oakey [02:00:46]:
And everything before that day is different from everything after that day. Because after that day, consciousness is possible, feelings are possible, emotions are possible and actual caring is possible. All of those things right now they're not. But that discovery when it happens might just happen one morning, one day and everything changes after that.
Bill (error_404_new) [02:01:08]:
But then every model is unique at that point. Right. Then it's not 4.9, it's like
Leo Laporte [02:01:15]:
Darren's model.
Bill (error_404_new) [02:01:17]:
What scares me is, like, every interaction can change this, right? Like, when I'm thinking about deploying this to my team and, like, letting multiple people chime in on this, I worry that it's just going to, like, devolve or somebody's going to get disgruntled and point in the well. So I'll be curious to see how that pans out.
Leo Laporte [02:01:34]:
Very, very interesting times. I'm very excited. And you know, what, if it's the end of the line for humans, well, what the hell? We had a good run. I was thinking, you know, city on Mars. You don't send humans to Mars. You send AIs to Mars. You send machines to Mars.
Larry Gold (LrAu) [02:01:55]:
No, you send the billionaires who want to go to Mars, so they're not.
Leo Laporte [02:01:57]:
Let Elon bring CROC to Mars. That's fine. I'm okay with that. All right, thank you, everybody. End of. End of. What do they say? Eol. How do they say that In Tron.
Leo Laporte [02:02:14]:
There was a Tron thing they said, end of line.
Larry Gold (LrAu) [02:02:18]:
No, I like the old. What was it? Pack rank.
