Untitled Linux Show 249 Transcript

Please be advised that this transcript is AI-generated and may not be word-for-word. Time codes refer to the approximate times in the ad-free version of the show.

Jonathan Bennett [00:00:00]:
This week we're doing an April 1st roundup. The fake stories, the half and half, and those that surprisingly were real. And then there's of course some Linux 7 and 7.1 news to cover. There's kind of been a shock in the Steam Linux percentages and oh yes, the software supply chain is terribly, horribly broken. We get into all that and more. So stay tuned. Podcasts you love from people you trust. This is to it.

Jonathan Bennett [00:00:35]:
This is the Untitled Linux show, episode 249, recorded Saturday, April 4. Do it on a potato. Hey folks, it's Saturday and you know what that means. It's time for the Untitled Linux show and it's going to be a blast today. This week has been kind of different because among other things that happened this week, we had April 1st this week. So you know, the Internet kind of goes crazy on that day. I do have a co host with me and this is not an April Fool's joke. We got Mr.

Jonathan Bennett [00:01:03]:
Jeff Mat, Cassie and

Jeff Massie [00:01:05]:
just the two of us just we can make it if we try.

Jonathan Bennett [00:01:09]:
There you go.

Jeff Massie [00:01:10]:
So yeah, and I'm hoping a lot of people now just have that song stuck in their head.

Jonathan Bennett [00:01:15]:
Probably, probably. It's not one that I'm super familiar with though, so I have to go and check it out after the show. Anyway, this week was April 1st and there were some very fun April 1st things around the Internet and we're going to go through a few of them, some that were completely jokes, some that were 50 50. And then there was at least one thing that I was quite sure was an April Fool's joke that turns out, I think to be a real story. So first off, there were some interesting patches to the kernel, one of which just searched for any binary named Emacs and returns a specific error. The comment there being disgusting. Return emacs. There was another one where related to the age verification stuff, a patch for the VFS kernel file system files would not be created on the file system unless there is a PR setbirthdate value with the user's birthdate.

Jonathan Bennett [00:02:22]:
And if that was blank or if the user was under 18 years old, VFS would not allow new files to be created. And this of course also was in a April 1, not a real patch. And also kind of a throwback to all the age verification stuff that's been going around. So those were fun. And then you get into the half and half. Gentoo put out a notification that it was going to switch away from a Linux kernel to a junior herd kernel. So was going to be. It was going to be Gentoo GNU Herd, I guess Gentoo slash heard, I'm not sure.

Jonathan Bennett [00:03:07]:
And so that was the April Fool's joke. But the reality is that they are actually making a Gentoo GNU Herd system and it's not going to be their primary system. So the Linux based Gentoo is not going away, but it is now going to be possible to run the Gentoo user space on top of GNU Herd. And so that was a 5050 on that particular April 1 fooler story. Yeah, their original April Fool's post was we're proud to announce that Gentoo plans to switch to GNU Herd as its primary kernel. And then they've gone back and amended it to say, no, no, it's just a port. So that one was fun. And then here's the one that you.

Jonathan Bennett [00:03:59]:
Go ahead.

Jeff Massie [00:03:59]:
One quick question. What is the advantage of running the, the Herd kernel? I mean, I know people kind of talk about it, but it seems like it's never really panned out.

Jonathan Bennett [00:04:13]:
I think, I think it's gotten a lot more sophisticated in the past few months and, and they're finally getting like their 64 bit support ironed out really well, which, you know, we've had that in the Linux kernel for decades now. I think the biggest like advantage of it is that it's not Linux. And other than that, I'm not sure. I've not done a deep dive on

Jeff Massie [00:04:37]:
gnuherd because isn't it the microkernel instead of the monolithic kernel? It's the.

Jonathan Bennett [00:04:47]:
I think so. I do remember back when Torvalds released the 0.1 version of Linux, he said back then that it was not going to be a big professional kernel that everybody uses, like GNU Heard was likely going to be. And so, you know, there's some fun stuff around that. But yeah, I don't know, maybe it's day it's coming. Maybe we'll have to start a new show here on TWiT. The Untitled Herd show doesn't really roll off the tongue quite in the same way.

Jeff Massie [00:05:14]:
No, the Herd Roundup or something, you

Jonathan Bennett [00:05:16]:
know, there you go.

Jeff Massie [00:05:17]:
That one I got a cowboy hat I could put on.

Jonathan Bennett [00:05:20]:
Yeah, yeah. Chasing the herd. Anyway, so there was another. There's another fun story. Actually, this one's less fun. Cloudflare announced em dash or em dash on April 1st as the spiritual successor to WordPress for solving plugin security. And I saw this and I thought, oh, surely that's an April Fool's joke, because like, why would Cloudflare want to make their own successor to WordPress. And then em dash, em dash, that's the extra long dash that all the LLMs overuse.

Jonathan Bennett [00:06:00]:
Like, that's the sign when you're reading something and it has EM dashes in it. That's the sign that it must be written by the LLM, written by the robot. So I'm like, oh, surely nobody, nobody would seriously name their project that, right? They did. And I think they're leaning into the AI element of it. They are, they are doing a lot of this with, with AI. It is fully open. M Dash is fully open source, it's MIT licensed, it's available on GitHub. And if you click through, there is actually a.

Jonathan Bennett [00:06:33]:
There is actually code there. And apparently it actually works. And the key point that they make here is that WordPress has a problem with its plugins and its plugin security. And that's true. Almost every problem in WordPress, every security problem for the past few years has been a plugin either has a vulnerability in it or a plugin author lost their token and so malware got shipped in some update, you know, all kinds of stuff like that. So, like, fixing that on the Internet would be great. I just, I really thought of the don't release. Don't release your product on April 1st.

Jonathan Bennett [00:07:16]:
And then there was one other announcement, and this one has a lot to. Well, not directly to do with Jeff, but Jeff will have comments on this one, I'm sure. Raspberry PI, you know, they make. They make a lot of devices with LPDDR4DRAM, like the Raspberry PI 4 and the Raspberry PI 5, and they once again are raising the price on them. And so, like, the 16 gigabyte Raspberry PI 5 is going up by $100. The Raspberry PI 500 plus is going up by $150. Some of the other ones are very, very much less hit. Like, for example, the compute module 4 is only going up by $12.50.

Jonathan Bennett [00:08:03]:
Not everything here is breaking the bank. And they did come up with an interesting workaround. They now have a Raspberry PI 4 with 3 gigabytes of RAM priced at $83.75. For now. Who knows how much that'll go up to before everything sort of settles down.

Jeff Massie [00:08:26]:
Well, if current trends can be trusted, it's kind of flattening now.

Jonathan Bennett [00:08:31]:
Yes, well, so there was an AI story that I think it was Google published some research that said, we did it. We now have an LLM that Doesn't take gigabytes and gigabytes of ram and there was some stock sell off as a result of that. Everybody's sort of losing their minds temporarily. I know Jeff Geerling put out a video that basically said SBCs are dead. Single board computers are dead for hobbyists. And at this price, I mean, I get the point, like that's it's difficult to, it's difficult to make this make sense for a hobbyist. But yeah, hopefully the price of RAM is finally going to come back down to some, some somewhere below the stratosphere.

Jeff Massie [00:09:16]:
Not this year I don't think because with the, with the Google, you know, we're. Oh, we're not going to need it. No, they're going to make their models have more tokens so they can utilize even more while still using the memory. I don't see them trying to cut back on what they're purchasing.

Jonathan Bennett [00:09:38]:
Yeah, I talked with someone else also in sort of the same industry and he was talking about the fact that demand got a bit of a haircut because of the pricing. And so you have all this pent up demand where people would love to buy a laptop or Raspberry PI or whatever, but now they've gone up so expensive you can't. And he was basically making the, making the comment that if the price comes down on this at all, that pent up demand is going to immediately eat it all up. And so, you know, it's going to keep the price at least somewhat high and everything's still going to be hard to get a hold of. So we're not, unfortunately we're not on the other side of the AI RAM apocalypse. Not yet.

Jeff Massie [00:10:21]:
Yeah, and a lot of this stuff too is long term contracts and every wafer that goes out the door is sold. It's just so much demand. Now there are several fabs coming online in about the next year. And keep in mind, when they come online, it isn't like, let's totally make up some numbers here. Let's just say a fab can do 100,000 wafers a month. And when they go, okay, our fab is running, it might only be doing 2000 wafers or 5000 wafers a month until they fully build it out. Because not only do you have to have the building and all the facilities around it, it takes a while to even fill that thing up. When you say, hey asml, I need a whole bunch of scanners, you're on.

Jeff Massie [00:11:16]:
So does everybody else. You're on a list. And yeah, there's lead times and you can get them as fast as you get them. And so it, it's when they come online, it's probably still another year or, you know, well, six, six months to a year and a half, two years before you get full output of what that FAB can actually do.

Jonathan Bennett [00:11:39]:
Yeah. I mean, you have, you have issues with that even like getting everyone in there trained 100%, getting your error rate where you want them to be, because that takes work too. Part of that is the training of the people. Yeah.

Jeff Massie [00:11:52]:
And you can say, well, but we've got this mature fab. We've got everything dialed in perfectly. I can tell you that you have that same exact recipe on the same exact tool in a different location. It's not going to behave the same way. There's going to be elevation, there's going to be minute differences in the tool. There's going to be, you know, it's just like processors, no. 2, you know, they're not identical even when they were made side by side. It's like.

Jeff Massie [00:12:20]:
Well, there's just a little bit of variation happens in the tools as well. So there's a little bit of tweaking that goes on and optimizations to get your. To get your yield.

Jonathan Bennett [00:12:31]:
Yeah, yeah, absolutely. All right, well, that is the, that's the April Fool's Roundup. And like we said, some of that was just for fun and some of it's not very fun at all. But there is. Go ahead.

Jeff Massie [00:12:44]:
I'll say my favorite was the Emacs.

Jonathan Bennett [00:12:46]:
Yeah.

Jeff Massie [00:12:48]:
And for the record, I'm not an Emacs or VI or VIM person. I'm a Kate or Fresh person, if it's in the command line.

Jonathan Bennett [00:12:59]:
Yep, I run Nano. That's how cool I am.

Jeff Massie [00:13:04]:
I've went to Fresh, over which I did a command line tip a few weeks ago, maybe a couple months ago. I'm. I'm not using Nano anymore. I went to Fresh.

Jonathan Bennett [00:13:13]:
I have to look at it. Look at it again. You want to talk about AMD and P states?

Jeff Massie [00:13:20]:
I would. Since we're talking silicon now, it isn't going to be too long before the 7.0 kernel is released. You know, possibly the 12th of this month, or maybe the 19th, depending on how the RCS go and an effort and if an extra one is added because of the size of the RC patches. Side note, Linus Torvalds, he'd mentioned that there's a whole bunch of drive or file system updates coming in and the ext4 and xfs or xsf, whichever way it is, they both had a whole lot of patches just fixing bugs and it's stuff that people won't normally run into. It's kind of weird corner case stuff. But anyway, it could push out the 7.0 release. But you know, once, once it is out though, then we have the merge window open for 7.1, which, you know, I bet a majority of our audience understands how this all flows. Well, AMD has already got a pull request for 7.1, which Linus likes to have those ahead of time.

Jeff Massie [00:14:27]:
So he appreciates when you have a pull request before the merge windows open. But they're going to update their P state driver with some new features. So what is a P state driver? It's software that controls the performance of a CPU by changing the frequency of the cpu. A lot of the time, you know, your CPU is just idling along saving power and not making so much heat because you're not stressing the cpu. You know, if you're typing a document, the CPU is going to idle because it's, you know, even at idle it's more than fast enough. And you know, going full speed isn't going to make any perceptible change other than it's going to suck a lot more power and make a lot more heat, you know. But when you're playing an intensive game or you're doing something that stresses the cpu, the frequency ramps to the maximum it can. Now maximum is limited by CPU temp and power available.

Jeff Massie [00:15:19]:
And you know, there's, it's not always going to be the same for everybody but you know, you think of it like a throttle on your car and you know, when you're sitting in traffic, your, your engine is just idling when accelerating. Well, you put your foot to the floor to get more power and it the engine speeds up, but it also takes more fuel to make that more power. So you know, it P state basically makes sure you have the right amount for, for your tasks at hand. Now Mario Lumoncello, I apologize for that, of amd, is now the formal AMD P state driver maintainer and he sent out the pull request as I mentioned. And because he did that, now we can see the new features which are going to be coming. So collaborative processor performance control, or it's commonly known as CPPC performance priority is being added. So now this feature allows user space to specify different floor performance levels for different CPUs. The platform firmware takes these different floor performance levels into consideration while throttling the CPU under power thermal constraints.

Jeff Massie [00:16:31]:
Basically the ability for the User or user space daemon or something similar to be able to set desired performance minimums for different CPU cores. And that'll be taken into account when making CPU thermal changes. So even if you're not doing something really stressful, if you lock a high performance task to a core, you want to make sure the core is and you might want to make sure the core is running at full speed or at least a moderate speed. You can set that even though the rest of the cores then can throttle down to what are, you know, an idle. Basically because they're not being utilized, you just have certain ones you specify, or vice versa. You set a low speed for a lower priority task that's locked onto a core. So maybe then you're thinking you can get more performance out of your cores that you're playing a game on. If you lock a bunch of other low priority tasks to other cores, they're going to be running slow, so they're not generating as much heat.

Jeff Massie [00:17:40]:
And you can have the heat being generated by the high performance ones. Another feature is dynamic energy performance preference, or also known as epp. This allows the CPU to change its performance preference based on if it's running on battery or plugged in, for example. There's also a new RAW feature for EPP which instead of a few power levels normally four are defined. You can now, it now allows an integer number from 0 to 255. Basically it's making the control much more granular than it was before. Now this pull request also includes some fixes for preemptive kernels and there's some bugs which other bugs which have been squashed. So you want the full details.

Jeff Massie [00:18:26]:
Take a look at the article linked in the show notes. It's got all the information you could ever want and links to the pull request. And now you can just put the pedal to the metal.

Jonathan Bennett [00:18:37]:
The pedal to the metal. So I'm curious about something and I think I understand, I think I know the answer to this. But why does a CPU use more power when you have it clocked faster? Is this. And I'll tell you what I think it is and tell me how close I am. What I think is that there's physics going on inside of it. I know that's part of the answer. But when you have a, you know, a binary 0 versus a binary 1, those are two different voltages and there is a, there is a literal delay time for that voltage inside of those transistor gates and other places to ramp from one to the other. And so my thought is you probably have to push more voltage at it if you want that to ramp faster.

Jonathan Bennett [00:19:23]:
And so if you're able to clock it down then you can have these lower voltage and it'll ramp up to your, your threshold slower and that's fine. But if you really want all of that stuff to happen very quickly, you got to really push the voltage to get those ramp speeds where you need them. Am I, am I close?

Jeff Massie [00:19:41]:
Yeah, it's known as the slew rate. And if you, here's a little trivia bit or for you non engineers, if you want to accurately measure your slew rate, you have to make sure your sampling frequency is, is at least 10 times your frequency of whatever you're measuring. So you can catch the, the exact ramp rate and figure out what, what it's going to look like, how much overshoot and undershoot. And you know, they always show ones and zeros, you know, when they're teaching somebody and oh, here's a basic 0-5-v and they show a nice square wave. Well it very high frequency, those get really rounded and they start looking more like sine waves. Well when the sine wave goes up and goes down and you trace them over each other, that's where you get your eye diagrams and that's why they look like an eye is because you don't have nice square waves that just are going straight vertical. They're, you're measuring them at speed and they're ramping at the kind of the limitations of physics to go, you know, up and down and, and there's tricks you can do. There's various device physics, there's, there's a bunch of other stuff in there that you can open up your eye diagram and reasons it closes down and jitter and you know, all sorts of stuff like that.

Jeff Massie [00:21:04]:
But yeah, that's cool and, and that's one of the reasons they ramp the voltage when they, you know. All right, we're going faster.

Jonathan Bennett [00:21:13]:
Yeah, yep, gotta get it. All right, let's talk then. Actually, actually I think it's time for a quick break. And then we've got something else about the Linux 7.0 kernel that's really interesting. Right down the same alley and that's why we put it right here. But we'll get to that right after a quick break. So not all is wonderful in linux land, particularly 7.0 if you're running a database. And this, this story comes via Pharonix and this is an AWS and Amazon engineer found a rather severe regression in postgresql and the Linux 7.0 release literally runs at half the throughput in this particular benchmark of PostgreSQL and it's sort of alarming, you don't see this size of regression that often.

Jonathan Bennett [00:22:19]:
And it was as simple as they changed the preemption modes for the kernel and so they removed the Preempt none preemption mode and they're focusing on full preemption and lazy preemption now. I've actually run into bugs with drivers around the preemption mode before. I remember I had a Sangoma hardware anyway, some telephone hardware and I was testing it before doing an install with it and I had a CentOS server, it was like CentOS 5, it's been years and years ago. And I installed their official driver and it would seg fault after a while and I don't remember would it bring this machine down when it did so because it was running in the kernel, it was not good for the machine. And so I called up their tech support. We went back and forth the guy ssh into my machine and spent a couple hours looking at it, trying to figure it out. And you know, it was one of these deals where it's like you have a machine here. I have the exact same machine in my office.

Jonathan Bennett [00:23:29]:
Your machine is barfing all over itself. My machine is running fine. Several hours we worked on it that evening, never could figure out why. And I eventually doing some more stuff with that machine. I eventually discovered that my install of CentOS was set to a different preemption mode. I was using Preempt Full and I think he was using Preempt none and that was the difference. And that driver had a bug in it that it would crash during preemption. And so I'm familiar with this causing problems.

Jonathan Bennett [00:24:02]:
Well, in this case it is Postgres doesn't know how to properly handle this different preemption mode. It's not necessarily a bug, but it's just it really causes the performance problem. And so the request has been made. Well, let's please re enable Preempt none so that we can fix this terrible, terrible problem. And the response has been so far, no, we're not going to back to Preempt none as the default. What you need to do is use and it's a new feature in the kernel, the restartable sequences time slice extension, which I believe basically just means

Jeff Massie [00:24:56]:
the

Jonathan Bennett [00:24:56]:
comment here is that it limits exposure to lockholder preemption. And so this is apparent, this is just a way for the two you know, the process to talk to the kernel and say, I need a little bit more time to finish the thing that I was doing before you steal the system back from me. And that's essentially it appears what is happening here is that PostgreSQL is in the middle of doing something. The kernel says no, it is time to preempt. And then as bad as this performance is, probably what's happening is that when PostgreSQL gets execution back into user space, it has to restart is my guess. And you know, it has to undo everything it just did and then try to do it again before the kernel takes back over that sort of thing, if it's not that. Exactly. And so there is this, this feature now in 7.0 that hopefully will fit.

Jonathan Bennett [00:25:53]:
But this is a terrible regret, like 50% performance, I don't know. And we're getting really close to 7.0, going stable about two weeks. Personally, I think it's going to be reverted because this is such a terrible performance issue.

Jeff Massie [00:26:12]:
Yeah. Well, I guess it depends too how hard it is to build that new feature into postgres SQL, right?

Jonathan Bennett [00:26:27]:
Indeed. That's not something you can just snap your fingers and have happened. The comment here about this is how you should do it. The response is maybe we should. But requiring the use of a new low level facility that was only introduced in the 7.0 kernel to address a regression that only exists in the 7.0 kernel. Seems not great. Yes.

Jeff Massie [00:26:53]:
Yeah, you know, that's valid argument.

Jonathan Bennett [00:26:57]:
It seems not great. I would agree with that.

Jeff Massie [00:27:01]:
Why do I have to fix what you broke?

Jonathan Bennett [00:27:03]:
You know, I mean, I thought that

Jeff Massie [00:27:05]:
was the kernels whole thing.

Jonathan Bennett [00:27:06]:
They don't break user space. They broke user space.

Jeff Massie [00:27:11]:
Yeah. Would that. Yeah, I guess it would be considered user space. Yeah, that's. And that is a mantra.

Jonathan Bennett [00:27:19]:
Yes, it is. That's funny. We'll see. We'll keep an eye on that. We'll see what happens there.

Jeff Massie [00:27:25]:
Yeah. And with the amount of infrastructure that runs on Linux and, and the amount

Jonathan Bennett [00:27:32]:
of infrastructure that PostgreSQL runs. Yes, that's a big deal.

Jeff Massie [00:27:37]:
Yeah, major deal.

Jonathan Bennett [00:27:39]:
Yes. I don't, I don't think we're quite knocking airplanes out of the sky with this one, but.

Jeff Massie [00:27:46]:
No, well, and realistically, anything that goes in that's really server related would not just jump on 7. 0. But still, I mean even for everyday people, it's like that's something you can notice, you know.

Jonathan Bennett [00:27:59]:
Yes.

Jeff Massie [00:28:00]:
That's throwing the anchor out the window. You're gonna. Something is very Wrong. And you will know.

Jonathan Bennett [00:28:05]:
Yeah. Also reading through this, it's. It appears that the preempt none is not removed, it's just no longer the default. And so you know, if you, if you run into this bug and you figure out that's what it is, you can just go set preempt none and be back in business.

Jeff Massie [00:28:23]:
Oh, okay.

Jonathan Bennett [00:28:23]:
So there,

Jeff Massie [00:28:27]:
that's. Yeah.

Jonathan Bennett [00:28:28]:
So we'll. Anyway, we'll see what's. Yeah, yeah. Something, something I'll have to give. Yeah. All right. There was another story. I don't think it was on April 1, but maybe it was.

Jonathan Bennett [00:28:40]:
It was pretty surprising.

Jeff Massie [00:28:41]:
You have the Steam story April 2nd.

Jonathan Bennett [00:28:44]:
I looked.

Jeff Massie [00:28:48]:
So every so often we report on Steam's market share numbers and each time we need to say take it with a big grain of salt. There's a lot of variables which we don't know that can affect the number, such as total number of users and things like Linux users could double for a month, but if the overall users doubled, well, it looked like the percentage stayed flat, which while mathematically correct, would not show the increase in Linux people. So there's some because we don't know the absolute numbers. These are, like I said, take them with a grain of salt. So pre Steam deck, the total percentage of Steam users running Linux hovered around 1%. As Linux got better and the Steam deck was released, the numbers climbed. Linux ended at 2025 at 3.5% of the users. In the next couple months there was a slide in February, was it 2.23%.

Jeff Massie [00:29:49]:
This time though there was a big jump in the numbers and the Steam survey shows Linux at 5.33%. Now this is an all time high for Linux and has never been even close to this before. That was a 3.1% jump in the numbers. Why? Is the next question. It looks like Steam corrected their China numbers. You know we can see this because simplified Chinese language use showed an almost 32% drop and English went up almost 17% and other, other languages they track also went up. Now I don't know the, the details of the exact reason, but I know when something like this happened before it was because of Internet cafes in China. Each time a person goes to one of the cafes in China and they sit at a computer, they have a fresh OS loaded.

Jeff Massie [00:30:40]:
So when you sit down you're not going to have any issues, problems or anything from the person before you. Basically it's a brand new load every time you get on there because it's a fresh load. Running Steam would get a high Number of hardware surveys and an Internet cafe would have a disproportionate number of counts since they almost always ran Windows. It inflates the numbers. So one machine, you know, you just keep hitting it every day with the hardware survey. So it kind of skews the numbers with the shift in the numbers. Because of this correction. MacOS also jumped up from about 1% to a little over 2%.

Jeff Massie [00:31:21]:
Windows fell 4.28% which takes windows is still just a touch over 92% market share. Now when you look at the different distributions, SteamOS was at the top OS for Linux with almost 25% of the users. There is a zero space 64 bit which has 17.6% and it jumped by that much because it's a new category now I don't know what it represents. You know, there was say there's distributions that are missing that some say maybe it's cashy or a group of distributions or probably some kind of error in there. Of the recognizable distributions, arch is almost 9% and mint is about the same if you count the two different versions listed on the chart. Ubuntu is at almost 5% again adding all the variations, you know, you add them all up. Manjaro is at 1.45% and like I said, there's also a 64 bit in there as well. And there's a whole lot of speculation what that is.

Jeff Massie [00:32:36]:
So those two have a pretty good sizable chunk. But also there's no cashy, there's no fedora, there's, you know, so there's a lot of distributions missing. It could be a catch all bucket now and they haven't broke them out or it's hard saying now we will have to see if this sticks but you know, because is are we going to stay up at above 5%? We'll see, you know, or was this a anomaly in the numbers now you could say why would it jump? Well, it can't hurt that there are a lot of tech influencers such as Linus from ltt Linus Tech tips doing a Linux challenge, you know, anymore I can't turn around on the Internet without seeing someone. You know, I'm trying Linux Challenge and I think with the constant improvements to Linux for ease of use, more people are sticking with Linux for the long haul. Which I mean if you think about it, you have a hundred people and you know, just a few out of that hundred stick. Well there's your 5% where you know, you still have a majority that I don't like it Or I couldn't get it to work or I don't want to learn anything new. But with Windows doing all their missteps, we'll say people are really starting to look at Linux and especially when you have some of these influential people trying it and saying it works, it does what I want it to do. It's definitely a lot of good PR for Linux now.

Jeff Massie [00:34:13]:
We'll learn a lot more next month when the numbers come out and you know, like I said, we'll, we'll see if things hold or if this was a weird anomaly anomaly or whatever it is. But take a look at the article linked in the show Notes for more details. And you know, there's, there's a link in there to the actual Steam survey where you can do your own research and find out, you know, which even things like monitor resolution, CPU cores, graphics cards, all sorts of stuff in there. So dig away and in the meantime, happy gaming.

Jonathan Bennett [00:34:47]:
Yeah, so there's something that really sticks out to me and I don't know how much to make of this with the simplified Chinese language changing, but Microsoft has really been pushing hard to get people off of Windows 10 and the raw statistics say that not all of those people are going to Windows 11.

Jeff Massie [00:35:07]:
No, it's not going well. And I know there's a lot of people that look at it and

Jonathan Bennett [00:35:18]:
there's

Jeff Massie [00:35:19]:
things like, okay, I have to use Windows 11 for work. I can't have a vertical taskbar. It's 2026 and I can't have a vertical taskbar. Really. Is this back in 1995 for the UI? There's a lot of little things like that. That's like, what is going on? Why can't I have some basic features?

Jonathan Bennett [00:35:44]:
Yeah, I thought it was more recent than this, but apparently it's been a couple of months ago. But Microsoft came out and there was a lot of news articles about this. Microsoft came out and essentially said everything about Windows 11 is broken in one way or another. And they're now working to rebuild trust in Windows. Getting back to their. Back to the basics, we'll say yeah, and.

Jeff Massie [00:36:10]:
And they just had a thing. Where was it? Copilot for Microsoft was putting ads in. Was it git, polls or git? I don't remember all the details but they were injecting ads into the code or to get polls or it was basically they're torquing more people off. It's like. And we've talked about it before, I don't think they really care. It's a locked in Cash cow is what they're thinking and they're focusing on the AI and the cloud and other things.

Jonathan Bennett [00:36:44]:
Yeah, I think the explanation there was that they were testing these features and that it was not intended to be released to the public. And. And everybody looked at that and went, why did you think that was ever a good idea ever to start with?

Jeff Massie [00:36:59]:
It's either wild vibe coding that no one checked or they're saving that feature for Windows 12, I guess.

Jonathan Bennett [00:37:09]:
Yeah. So there is an update to this. Microsoft is indeed fixing, promising to fix. This is March 21st. Microsoft is promising to fix the biggest issues in Windows 11 from AI slop to pushy. Windows updates is a tech radar story and taskbar customization is apparently coming back and all sorts of different stuff. So maybe Windows 11 will get better.

Jeff Massie [00:37:42]:
Somebody can post a picture of me in the chat, all blue and saying with the caption, Jeff holding his breath, waiting for Windows fixes.

Jonathan Bennett [00:37:52]:
Waiting for Microsoft to fix Windows. Yeah. Goodness.

Jeff Massie [00:37:57]:
Yeah.

Jonathan Bennett [00:37:58]:
Interesting stuff. I suspect that it'll stick though.

Jeff Massie [00:38:05]:
I think it will because I think what happened is because people say, well, wait, it went down in January and February, but I think that was because the China users were getting overrepresented. And I'd also heard that it doesn't make a lot of sense there's so many Chinese users because supposedly there's a Steam version for China because certain games aren't allowed in China.

Jonathan Bennett [00:38:29]:
Makes sense.

Jeff Massie [00:38:30]:
So I could see where if it's. Especially if it's an Internet cafe or something like that where it's just over representing because you get your. Like I said, you're getting a refreshed machine every time you sit it at your station.

Jonathan Bennett [00:38:44]:
Yeah, yeah, that makes sense.

Jeff Massie [00:38:47]:
That's what it was last time when they had to correct the numbers.

Jonathan Bennett [00:38:50]:
Yeah, there's been a thing a few times. Yeah. All right, well, we're going to talk about the supply chain next. Not the hardware supply chain, but the software supply chain. We got to take a quick break and we'll be right back. So when we talk about open source supply chain, software supply chain, we're essentially talking about the fact that everything is built out of libraries. Right. So you can, you can imagine that I've got this cool library that does.

Jonathan Bennett [00:39:23]:
It formats text at a certain way. Well, your. When your website renders, when you go to Twit TV, the LeT website renders, it pulls in a whole bunch of JavaScript libraries. One of those might be my little tiny library that renders text in a certain way. That. That was a real Thing, by the way, that was left pad and this was one that was one of our first sort of inklings that there could be trouble in paradise when the. When the guy. When the developer behind left pad or was it right pad? Whichever yanked that library and a quarter of the Internet stopped working, stopped displaying correctly.

Jonathan Bennett [00:39:59]:
We sort of figured out then that we might have a problem and it's not gotten better since then. So there's been a couple of. And so that was not necessarily malicious. There's been a couple of malicious problems with the software supply chain in the past week. I've got links off to a few of them. So one of which was a project called axios, the Axios HTTP project and it's called a developer friendly JavaScript HTTP interface for Node JS and basically it's just a fancier way to interact with JavaScript stuff. Don't look at me like that. I'm not a JavaScript developer.

Jonathan Bennett [00:40:43]:
I don't know what I need this stuff, okay? I'm not a JavaScript developer. I don't have to be. Anyways, this project got compromised and the developer believes that what happened was someone said let's collaborate and ended up getting him to clone a malicious repo. And when you do that and you're using something like VS code, it's actually rather easy to trick someone. Well, you clone the malicious repository and VS code will automatically run some scripts as a result. And so don't clone random repositories and open them up in VS code is sort of the message there's. But once the main developer there, his machine, his local machine got compromised. The bit of malware that ran went looking for tokens and found essentially a security token that allowed uploading malicious copies of the Axios NPM libraries.

Jonathan Bennett [00:41:51]:
And so bad guy got into one machine, got the token, pushed out malicious code to NPM and then all of a sudden lots and lots of different developer machines pulled down nasty malware as a result. And so a whole bunch of people got got hit. And so that was one. There was actually another one, I believe a few days before. I don't keep my finger quite on the pulse of security the way that I used to, since I'm not writing the hackaday security column these days. But there was a vulnerability, a few vulnerability scanners that got hit in similar ways. So Trivi and kics, a Python SDK of Telnix, all of these places got hit sort of in the same wave of supply chain attacks, believed to be by a group called Team pcp, which I Think they're out of North Korea. Again, that is what is believed.

Jonathan Bennett [00:42:57]:
These guys don't leave calling cards. So a lot of this is, you know, putting the, putting the, connecting the dots and putting clues together and trying to figure out, you know, here's where this, this group is probably out of the government that's probably supporting them. But, and when, when one of these things happens, usually what'll happen is every developer machine that then installs the malicious library, a script will run, look through that machine for security tokens, bitcoin wallets, all of that stuff, do an upload to the malicious command and control server and then install something like a wiper and a command and control bot to be able to do nasty things on that computer in the future. It's, it's, it's not great. The trajectory here is not great. This is a thing that keeps happening and nobody really has really good answers for how to stop it. I also have a link off to GitHub's blog post about it also on April 1, not on April Fools. And he talks about the.

Jonathan Bennett [00:44:12]:
Zachary Steinler wrote this and he talks about the fact that in a lot of cases these things happen through GitHub Action. And lots of projects use GitHub Actions for their continuous integration. And so something like GitHub Actions. What will often happen there is you've got your publicly accessible GitHub repository, anybody can come along and add a pull request to it. Well, you want your automated testing to run on that pull request, right? You want to try to build a project and then run through your test suite. Well, what if the incoming code on this pull request is malicious? Is there something nasty that someone can do inside your continuous integration environment? And in some cases the answer is yes. So there's in this particular blog post from GitHub, there's even like directions on how to better secure your GitHub Actions CI runs. And it's things like just don't use pull request target and make sure you're using the full length commit the commit shaws short codes are brute forceable, whereas the full length shaw is not.

Jonathan Bennett [00:45:24]:
And then of course look at people's pull requests and make sure that there's not any obvious script injections, things like that. And then GitHub is of course continuing to work on this and they are trying, and they're trying very hard to come up with some ways to prevent NPM. In particular, because NPM is owned by GitHub, you didn't know that. And then GitHub as a whole from quite as bad being hit by these attacks. It's not a good thing for everyone's confidence in GitHub, npm or even the open source ecosystem of libraries as a whole. It's really turning into a problem and I don't know what the answer is, but that's kind of where we're at. And you've got some really smart people at GitHub trying to figure it out. And other place, many, many other places, Amazon and Google, if security engineers and all those places that are trying to figure out what can we do about this.

Jonathan Bennett [00:46:21]:
But it's, it's really challenging.

Jeff Massie [00:46:24]:
Yeah, I mean it's, it's tough to collaborate without. How do you vet everybody and say okay, yeah, we, we're all in this together.

Jonathan Bennett [00:46:33]:
Yeah, so what you're doing.

Jeff Massie [00:46:34]:
Except for that person.

Jonathan Bennett [00:46:35]:
Yes, for that person. So what you're doing, open source is about trust and you may trust the person, but what, what you, what we're sort of figuring out as a result of this stuff is the fact that you trust the person does not necessarily translate directly to you get to trust that person's machine, you get to trust that person's account because something malicious can be done in that person's name. Which I guess this is true in more than just computing. Right. But when you have face to face interactions with people that, that you have sort of, that built in mechanism of trust. Whereas when you're just sending emails and doing things through GitHub, those mechanisms are more hackable.

Jeff Massie [00:47:21]:
Well, and okay, you trust that person, but did that person collaborate with somebody else on their code that you normally wouldn't trust? But maybe they're judgment wasn't the best and so they thought the person was trustworthy.

Jonathan Bennett [00:47:36]:
You know, absolutely. Love Tia John, the, the hacker that almost, almost owned everybody's computers through that SSH hack, that one we got, we got this close to that being a really big problem. It's the same thing. It's exactly what you described. You know, there was, there was one guy who was trustworthy and then he trusted somebody else that he shouldn't have. And that was the, that was the crack. That's where things got in.

Jeff Massie [00:48:07]:
Yeah, I mean it's like you could almost have an analogy of food, you know, oh, I trust all these people, but you probably have one buddy or one co worker that you're going, you know, I'm probably not going to eat whatever they cook. You know, they might mean well, but yeah, there might be some standards in there that I'm not copacetic. With and you know that that chain of trust has to go all the way and yeah, be hard.

Jonathan Bennett [00:48:38]:
I think part of what's going to have to happen here with this is it's going to be sort of an extension on top of. What do they call them? The. The manifest of everything that goes into a software project. I forget there's a term for this and I forget what they call it. It's the list of all the libraries that's inside of a. You know, inside of a system or inside of a piece of software. I think probably what we're going to end up seeing is that instead of that being descriptive at some point that's going to have to be prescriptive. Yes.

Jonathan Bennett [00:49:15]:
An SBoM. Thank you. Mashed Potato Software Bill of Materials Nestbom. It's going to have to become prescriptive, meaning that not only does this describe what is in there, but it also will be part of generating it to say only this software and only exactly these versions will be in there. I've seen this on an open source project that I work in meshtastic and doing firmware there. We pull in a bunch of different libraries because it's part of the use the Arduino ecosystem. When you do this, when you describe these, you can say I want the library that is 1.2, right? So you say this sensor at 1.2 or you can say this sensor at carat 1.2 and caret 1.2 means that version or any future compatible version. And so if there's a 1.3, well, let's say 1.2.1 is really what you would see.

Jonathan Bennett [00:50:18]:
And so then if that author released a 1.2.2 of that library, then when next time you build it would just go ahead and pull in that X version because it's compatible. And what we found is that they're not always compatible. You again, you're sort of trusting these random library authors to, to version things correctly. They don't always. So we sort of learned there that no, no, you leave the carrot out. Pin it. We want exactly 1.2.1 always every time that we build this and if we go in and we do an update, we want to do it on purpose with lots and lots of testing. The next iteration of that, which we've had to move into in a few cases is instead of using a version number, use a.

Jonathan Bennett [00:51:09]:
Use a version hash. And so instead of 1.2.1. No, no, we want GitHub.com the name of the project archive. And then like 32 bytes of SHA256/ or SHA512. What's 32 times 8 anyway? 286. Yeah. To say exactly this git thing, we want the code at exactly this git commit. And when you do that, and if you actually check it, then you can be fairly confident that no one has slipped anything in.

Jonathan Bennett [00:51:48]:
And this is something that GitHub talked about too, that idea of using the full hashes and not just the short hash. I think there is, there's a future we're seeing where for things like this that really matter, it's going to be something like an S bom but prescriptive and saying exactly this code, exactly this code. And you can imagine like some sort of testing that happens during compile where you check and make sure that that code that you just download does indeed equate to that same hash. And that sort of takes care of the problem. But man, it's a lot of work changes the way we do software.

Jeff Massie [00:52:26]:
Well, I guess, I guess it strengthens the chain, right? Because that's, you can say, okay, you're pulling in a library. Who are all the people that work on the library? Oh, there's a bunch of sub, you know, routines in the library who all worked on those and did they even get help working with somebody to, you know, it just kind of spider webs out and you kind of. Each level is going to have to be checked for trust almost.

Jonathan Bennett [00:52:52]:
Yeah.

Jeff Massie [00:52:52]:
And becomes cumbersome. But I mean, maybe there's, you know, that's where AI and automated tools can come in to help ease the pain in this realm.

Jonathan Bennett [00:53:03]:
For so long now, we've just essentially said, I'm sure it'll be fine. And come to find out, sometimes it's not fine.

Jeff Massie [00:53:11]:
Well, and it comes with, you know, okay, as Linux gets more popular now, it's more of a target. I mean, for a while there, people that, yeah, it's not worth it. It's not, you know, we're going after the easy stuff and hey, Linux is out there more and more, becoming more and more of a target.

Jonathan Bennett [00:53:31]:
Some of this is not, not just Linux specific like the npm, the node package manager. That's all JavaScript. That's your browser. Interestingly, your browser has sort of become its own OS outside of, outside of Linux. But yeah, we'll see what happens with all of that. All right, so coming up next, what do we have? Oh, Jeff. Ubuntu. Yes, Ubuntu has.

Jonathan Bennett [00:54:01]:
It's not just Windows, it's not just Microsoft with Windows 12. What is Ubuntu doing this time?

Jeff Massie [00:54:08]:
Well, Ubuntu, without any fanfare, has raised the minimum requirements for its distribution. Now, this isn't the first time this has happened. It's been happening several times over the past. Like with 12.04, 32 bit versions, it needed 384 megabytes of RAM, and the 64 bit version needed. It wanted 512 megabytes of RAM. Along with that was a single core, 1 GHz CPU and 5 gigabytes of disk space. And that would have you well on your way. Then in 2014, you know, with 2404, the RAM was raised to 1 gigabyte for all versions.

Jeff Massie [00:54:49]:
Fast forward, you know, 1704 they stopped supporting 32 bit CPUs, and in 1804 they wanted 4 gigabytes of RAM, which would all be 64 bit. Now, a 2 GHz CPU with 2 or more cores and 25 gigabytes of disk space. The history though, gives us a glimpse of the present and the future. 1804 initially had 2 gigabytes of RAM minimum, but the 1804 download page was updated in 2019 to bump this to 4 gigabyte following community feedback. Well, now with 2604 they say you need 6 gigabytes of RAM, while the rest of the requirements stay the same. But actually this is like 1804 where there isn't anything, which is taking more resources. They're just being more honest on what a smooth running system is going to look like. Sure, you can bring it up with 4 gigabytes of RAM, but when you load in a browser or other programs, you know you're going to want a little more room because it's going to be swap and you know it's going to be hitting that swap file pretty hard.

Jeff Massie [00:55:52]:
So like in the past, they're just being a little more realistic and honest on what levels of hardware is needed to have a good experience. The article does say that 4 gigabytes will work and the system will run. Like I said, it's just not going to be a smooth experience as the memory, you know, they said a lot of data swapping. It does go on to say if you're stuck at 4 gigabytes of RAM because it's soldered down, or some other reason you can't upgrade. They're all alternatives, such as Lubuntu, which requires less resources. They also talk about the manual install option using the netboot installer. Now, this is for Ubuntu, and then you can just get the core installed and then build on just only the things that you really need. And it even also they even mentioned the Server Edition which has lower requirements if your use case can get by with less items loaded.

Jeff Massie [00:56:43]:
Because the Server Edition can be installed with 1.5 gigabytes of RAM and 5 gigabytes of disk space. But with that loadout there's going to be a lot of things which going to be cut out of the full FAT distribution. So you're going to be trimming out a lot of things. But if you take a look at the article linked in the show notes for full details on system resources and options if you want or need to use a system with less resources. So happy installing.

Jonathan Bennett [00:57:11]:
Yeah, I think probably good for them for encouraging people that you really ought to have six gigs of ram. And I've said for a long time that dual core 64 bit is really where it's at for the sort of the weakest CPU you can get by with for to have a usable system.

Jeff Massie [00:57:33]:
Yeah, and, and it will be kind of slow, but it's usable. But I mean other operating systems have said oh here's our bare minimum. And yeah, it'll do it, but when you boot it up, it's like if you try to do much of anything else, it just is painful and slow and clunky. So like I said, they're just being a little more honest and you know, and there are things like say the web, like we just said, the web browser. Over the years it's, they've gotten a lot bigger because they've become more complex and like you said, they're almost an operating system onto themselves and it takes a little more to make all that work work.

Jonathan Bennett [00:58:15]:
Well, yeah, so Windows 11 requires 64 gigs of storage, but they do support 4 gigs of RAM, which is why you see those terrible little laptops with only 4 gigs of RAM still being made. I wonder if those laptops are still made because Windows says it'll run on four gigs of ram. Or if Windows says it'll run on four gigs of ram because they've got vendors making those terrible laptops for sale in places like Walmart.

Jeff Massie [00:58:49]:
Well, it probably just, hey, we can hit a price point and it will technically work.

Jonathan Bennett [00:58:54]:
Technically six gigs is going to be now so obviously like we know you can put a, depending upon what you want to do with it, a very usable Linux install on way less than this. I mean you could do a Linux from scratch. I wonder what the limitations are for that. I would, I would I would be pretty comfortable saying that you could do a Linux from Scratch build on like 512 megs of RAM and a couple. A few gigs. You can do it in Singapore.

Jeff Massie [00:59:25]:
Yeah, I could see that. And you know, and it's kind of going back to. Even like in the story about Ubuntu Server, you're probably not loading a desktop on. You're, you know, there's a lot of stuff you're not loading in. It's just the bare bones of what you need. But you know, maybe, maybe that's good enough for what you're doing. And Linux from Scratch would let you just tie in even more. You know, you can just really get

Jonathan Bennett [00:59:53]:
the

Jeff Massie [00:59:55]:
knobs to tweak it.

Jonathan Bennett [01:00:00]:
Linux from scratch. Oh, this is the host system requirements. They suggest eight gigs of memory. No, I want to know, I want to know the target system requirements. I don't know if I'll be able to find this quickly. Yeah, does not look like I'll be able to find it quickly, but I am sure that you. Because I know I've done it. You can run Linux systems on much lighter than that.

Jeff Massie [01:00:34]:
And yeah, I mean that's, that's as kind of in the comments here, you know, like Harold Finch, you know. Yeah, there's, you've got some pretty good system specs, but even, even for your old PC. But I mean there's, there's a lot of people like. And you know, Ken, he runs some really old hardware and it's, he, he's adamant about keeping that stuff going, you know, not making e waste. And that's where some of that stuff really matters. You know, he had, he had a single core that he was running some stuff on for a while.

Jonathan Bennett [01:01:16]:
Yeah. So as Linux from Scratch does have a comment here, 64 build is only 3% faster, blah, blah, blah. If you plan to use LFS links from scratches, a LAMP Linux, Apache, MySQL, PHP as a LAMP server firewall, a 32 bit CPU may be good enough. And there are some of the BLFs beyond Linux from Scratch packages that now need more than four gigs of RAM to be built and. Or run. Yeah, you can do it. You can do it cheaper, lighter. Lighter than that.

Jonathan Bennett [01:01:48]:
And on 32 bit the entire build size is only 3.6 gigs. That's hardware. Yeah, you can do it on a potato. Yeah. All right, well, we're going to talk about some hardware that is really sort of a potato. This actually feeds very nicely into our next topic. We're going to talk about routers and the legal reasons why routers are in the news. We're going to take a quick break, though, and then we'll dive into it.

Jonathan Bennett [01:02:17]:
All right, so we didn't talk about this last week. We could have. We're going to talk about it this week. The FCC has probably banned your router and. Well, not my router, but some of my routers. It's banned a lot of routers that I've sold to clients, and yet at the same time, it hasn't banned any of them at all. So let's actually look at the detail. So what specifically happened is the FCC banned the sale of new routers, new models of home routers not made in the US and to be.

Jonathan Bennett [01:02:53]:
To be even more specific, what they've done is they've come out and said they're not going to approve any new routers that are not already on the market, that are not made in the United States for the US Market. Of course, if you live outside the United States, this doesn't mean a whole lot to you, but for those of us in the US it probably, probably will. So the reason why is obviously the cybersecurity risk, and this is a pretty legitimate concern. Again, I've spent several years now, I've covered security stuff, and it's amazing how terrible the security problems in some of these routers are. I've also done some hacking on them myself, doing things like bringing OpenWRT onto TP Link routers. And I remember in one case it was the TP link router firmware. There was a field to, I think it was to connect to a new WI FI network. And so it was just a text string where you would type in the name of the WI FI network.

Jonathan Bennett [01:04:06]:
And in Bash, you can use backticks to run a command and then do replace the command with the results of that command. And so, you know, it's useful in scripting to be able to do things like, I don't know, get your total, your current date and punch that into the name of a file, something like that. It turned out that on this TP link firmware that was run as a command somewhere inside of there, it was run as an argument on the command line on Bash. And so you could literally use in this field backticks as part of this imaginary network name that it was going to search for, and it would just execute what was in the backticks. And so you can find a very old Reddit post of me giving people instructions on how to unlock a router to be able to put OpenWRT on it by telling it to connect to several of these imaginary, very weirdly named WI fi networks. And what we were doing is we were building a little script in TMP and then executing the script. Finally, all kinds of weird security problems. And in covering these security problems over the years, you do actually you find some places where security researchers will go, these are so terrible and so bad, it's not difficult to imagine that they're put there on purpose.

Jonathan Bennett [01:05:41]:
So that is part of the landscape here is that some of these really do have problems that are terrible and look somewhat purposeful. There are a lot of these really cheap routers that are part of botnets. I've had to go to customers offices before where the whole network was acting weird and come to find out the DNS servers in the router had been changed. And that's because it was possible to pop those routers from the outside and somebody did, did it to a bunch of them, got into it from the outside, changed the DNS servers and then, you know, half of your lookups was going off to somewhere malicious. So this really is a real problem. Now what specifically is going to change immediately because of this new FCC rule? And that is nothing. All of the same routers that have already been approved are still approved, they're still being sold. Where this really becomes a problem is when TP Link or you know, any of these other companies say here's my new model of router.

Jonathan Bennett [01:06:47]:
And if this, if this executive order stands up, if this new ruling stands up, then you will, they will not be able to sell those new models inside the us. And so I've got actually linked off here to the Software Freedom Conservancy. They've got a comment on this, a blog post on it. And one of the reasons that SFC has skin in this game is that SFC is one of the groups behind the OpenWRT one, which I have around here somewhere, unfortunately not within arm's reach. I'd show it to you, but that is the OpenWRT router. Really nice hardware. That one has already gotten FCC approval. So no changes with that.

Jonathan Bennett [01:07:28]:
But if they wanted to do a revision on that, like a version two of it, that's when you might run into some problems. And of course sfc, again being some of the folks behind Open wrt, supporters of it, they make the point here that making routers in the US does not actually make the security any better. And I have to agree, we've got some US based companies that make security Appliances that also run into terrible security problems that you want. Wonder how they're there. Go read the Watchtower blog for more information on that. Those guys are hilarious. Covering security issues. So anyway, all of that to say this is still sort of up in the air.

Jonathan Bennett [01:08:10]:
We don't know for sure if this is going to stick or if, if it's going to be modified to be a little bit better. I really wish that there was some rule about making the source code on these things open source. Like that would actually help. But good luck with that.

Jeff Massie [01:08:27]:
This is stupid on so many levels because.

Jonathan Bennett [01:08:31]:
Okay, I was not sure about it. Jeff's not going to be nice about it.

Jeff Massie [01:08:36]:
Oh, no, no. I'm the older crustier guy, so it's just home routers. So Enterprise.

Jonathan Bennett [01:08:45]:
Oh yeah.

Jeff Massie [01:08:46]:
Get it wherever you want. Doesn't matter. Well, hey, seems like that would be even bigger security risk. Two, oh, we don't really make them in the US So good luck with that. Or we're just going to, you know, okay, we'll approve them. Nobody's. Whatever, whatever that means, you know, I mean, like, nobody's going to look at it. Three, okay, somebody's going to start up and we're going to make it in the U.S.

Jeff Massie [01:09:09]:
where do you think the chips are coming from? The chips are still coming from overseas. We're not making every little transistor resistor, you know, logic chip in the U.S. no, that's not happening. Or if we say, okay, we're going to. Well, now we got about a three to five year wait while they build out everything. If, if companies have, would have the money to do it, you know, and then they're talking about, well, no more software updates. Oh gee, let's just make the problems even worse. How can we pour gasoline on the stupid, you know, just, we need accelerant on it.

Jeff Massie [01:09:46]:
Because now you can't patch the problems. It's like, this is so terrible at so many levels. I, it's just crazy.

Jonathan Bennett [01:09:58]:
Yeah, I, I think, I think there's a nugget of a good idea. No, that's probably not even accurate. There's a nugget of a really good intention in here. And that is, let's try to make some of this available within the country and not have a supply chain, you know, have some level of supply chain that does not exist outside and try to make these things a bit more secure. Like, I like that,

Jeff Massie [01:10:31]:
I like the concept.

Jonathan Bennett [01:10:32]:
The concept, yes. Of better security as a goal. I like that as a goal. I don't think that this Particular approach really gets us any closer to that goal. Because I don't think you can do it is really the problem.

Jeff Massie [01:10:50]:
Well, and because we're too much of a global society now, stuff is assembled with parts globally. And I agree with you first, if it was me, okay, everybody voted for me and I'm in charge. First thing is like, okay, if you buy a home router, you have to support it for at least 10 years after the last one is sold and you have to do updates. You have to be on top of security two. Okay, it's past the 10 years now. You need to open source the firmware. You need to just say, okay, if you want to take this yourself or somebody wants to do it, here you go. And you know, thing, basically there's gotta be a lot more visibility and you need, you need to really say, okay, these should be updating firmware themselves or at least have the ability to go, when you're surfing around or something, every so often, pop up a window in your browser and go, you probably need to update your firmware now because, hey, there's a new one out here.

Jeff Massie [01:11:54]:
And because how many times do we hear about security updates where something is either it's an old router that, you know, somebody forgot about and it hasn't been updated for forever, or it hasn't been updated because it's been out of support for forever, but it's still quietly chugging away in the, in the closet, in the hall, on top of the bookcase, wherever it is. You know, nobody, nobody thinks about it.

Jonathan Bennett [01:12:18]:
So,

Jeff Massie [01:12:20]:
yeah, there are so many better ways that you could do it. If you really wanted to get fancy, you could go, you know what? We need a hardware review of what's in all these, on, in all these routers. And you send us one with your hardware list for the government so you don't have to worry about losing IP or whatever. They can validate all the chips in there. And then every so often they go out and go, okay, we're going to buy one off the shelf and make sure everything matches those original schematics and make sure there's no other hardware in there. I mean, now you can, you can get really into the weeds and say, well, somebody could design little things into the chips and you'd never know. I mean, this is true. But at some point you have to say, we're catching a lot of the stuff.

Jeff Massie [01:13:03]:
Because, because, because realistically, they're not going after, you know, deep state kind of stuff. They're going after, I think more of the the botnets and things like that, that I think if we really forced better firmware and updates and better stuff, you know, say this is a computer. It's like, you know, the EU has rules for your cell phones about how long they have to support them and things like that. This is the same thing, it's a mini computer and it needs to be supported and have those firmware updates and not just, yeah, I know it's two years old but we're not supporting it anymore, so good luck and just ignore them.

Jonathan Bennett [01:13:42]:
It is, it is worth pointing out that there is a submission process for conditional approval for new routers produced by foreign countries. Right. And so there's the document here of information requested and some of it's just normal boilerplate, you know, your corporate structure and all that. But it's also got things like the bill of materials for the router, the country of origin for those components, the entities responsible for the IP ownership and software updates for the router. So you know, somebody was thinking. But then the third, the third section here is to get one of these approved, you must include your US Manufacturing and onshoring plan, a detailed time bound plan to establish or expand manufacturing in the United States for the router for which the applicant is seeking conditional approval. So it's like

Jeff Massie [01:14:33]:
good luck. And then you get into like what is made in the US meant. Oh well, we put the last stickers on and we screw the little WI FI antennas, hey, assembled in America.

Jonathan Bennett [01:14:42]:
You know, in some cases, yes, that is enough. Which is sort of ridiculous.

Jeff Massie [01:14:47]:
And, and realistically it's probably gonna, these, a lot of this stuff's gonna be looked at by people that have no idea and they're, they're not really understanding what's going on or what they're looking at or.

Jonathan Bennett [01:14:58]:
All right.

Jeff Massie [01:15:01]:
To me it's just terrible. And it, and it should include Enterprise if you're going to be serious about it. Include the Enterprise.

Jonathan Bennett [01:15:08]:
Yeah. I think if they're going particularly after the botnets, the hope was that your enterprises are going to take care of it themselves. It's just, I've seen too many don't, it doesn't work that way.

Jeff Massie [01:15:24]:
Right. To me it just seems like if you're going to do it, get serious about it and just do it and include the enterprise and really have hardware reviews, really look into this stuff. But I think a lot more of it's like you said, it's coming from the software and they're just lazy firmware software. It, you know, that's good enough. Ship it look, you can buy it. Look at all the blinky lights. It's cool, you know.

Jonathan Bennett [01:15:53]:
Yeah.

Jeff Massie [01:15:54]:
Versus we're, we're secure. That's why my, the, the router I have now runs OPEN WRT because I, my last one was still good, but it was a couple years out of compliance, you know, they stopped supporting it and I was dead in the water and I couldn't run open WRT on it. Well, realistically I couldn't. It was always an experimental state and was kind of semi updated and. But anyway, I picked one that I could run open WRT on so I can keep it updated software wise until the hardware dies.

Jonathan Bennett [01:16:34]:
Yeah, yeah, absolutely. You know, I could, I can imagine ways that you could go about writing this that would actually work, but I think honestly it's a difficult problem to solve and most any of the solutions for this are going to be a pain. I mean you look at the European law around this and there's wailing and gnashing of teeth about that as well. And it's written better than this is. Yeah.

Jeff Massie [01:17:01]:
And realistically you need to come at this because this problem is not something you go, oh, we got one rule, we're going to one rule to solve it all. No, it's going to take take like a phased approach and start going, look, you're going to have to support these for longer. You're going to have to find better ways to have these updated. And you know, maybe not push updates, but you have to at least notify, you have to do something so that these don't get forgotten about because you don't want a bad update to suddenly trash a whole bunch of systems. But on the other hand, it's like you need to update your firmware, you need to keep this stuff secure and there's gotta, you know, there's, there's a lot of different ways you could do that. And I think that's where you start phasing this stuff in and go take care of it, you know, support it.

Jonathan Bennett [01:17:57]:
Put you, you could. Yeah, I'm thinking, I'm thinking of all kinds of crazy things that may or may not work. Like, well, I mean, you know, you go, go ahead, send a working copy of the open WRT source code for your router to the government and then when you're ready to retire, that gets released automatically. You know, you could do fun things like that. To be able to force the, force opening it up, to be able to put something more modern on it if people wanted to.

Jeff Massie [01:18:26]:
Well, I'm just even thinking like if you go to a hotel or something, you want to use the wifi, it pops up a screen that says, hey, you have to agree to this to use our wifi and blah, blah, blah.

Jonathan Bennett [01:18:37]:
So I have to support that from time to time. And that is terrible and does not work about half the time because HTTPs. I'm just saying.

Jeff Massie [01:18:48]:
Right.

Jonathan Bennett [01:18:49]:
And there's things built into your Browsers and like iOS and Android that are supposed to go look for that well known URL. And man, I've had so many times that that doesn't work. Various people have problems with it or I have problems with it and for whatever reason it doesn't work. So when you say that and you're like, let's just do it like this, my immediate thought is, oh, it's a terrible. It'.

Jeff Massie [01:19:10]:
Well, it's, it maybe has to be mandated in the browsers or it has to. I mean there's, I'm just saying there's a lot of different ways you could solve this to say, hey people, you have to update your firmware. We have a, you know, we have bug fixes, we have, you know, vulnerabilities, we've got a patch, we've got a, you know, and maybe, maybe your router starts blinking funny colors, it sends a message out, it starts beeping at you. Yeah, it's something. Just so you know, oh, I need to update this. And just something you can for, for, you know, people that don't know. Grandpa doesn't know, but he knows that, oh, my router says I need to update it. So I push the button and then it updates.

Jonathan Bennett [01:19:53]:
Yeah. Yeah. Well, we are not going to solve it today, but I thought it would be interesting to let everybody know about what, what's going on in the router world. There is another world where some really weird things are happening and that's the office world. Not like the place that we go to work, but the office suite world with only Office and it's only Office and Nextcloud, isn't it? Or own Cloud Zero.

Jeff Massie [01:20:24]:
Office, only Office, Nextcloud, LibreOffice and Collabora.

Jonathan Bennett [01:20:30]:
Office. Oh, all of them are involved, just all the buzzwords. All right, walk us through this. Tell us what's going on. I don't even know the full story here. I just don't beat parts of it.

Jeff Massie [01:20:41]:
Well, we have about three things going on. So normally when we talk about things which cause a controversy, we're not talking about an office suite. Gnome versus kde, Emacs versus Vim, you know, Ubuntu versus Arch, you know, whatever you know. But when you talk about an Office suite, that's normally the boring part. Unless they come out with a new feature that everybody's waiting for, you know, oh, I can highlight in rainbows or whatever it is, you know. Well, not so this week. So a new web based Office suite called EuroOffice was recently introduced by a group of European companies, including nextcloud and Ionos. Their goal is to offer a collaborative document editing platform that supports Europe's digital independence.

Jeff Massie [01:21:30]:
A technical preview is already available and the first full release is planned for the summer of 2026. Euroo office is built on the open source code base of OnlyOffice, which is distributed under the AGPL version 3, a license that allows anyone to use, modify and share the software as long as they follow the license terms. But Almost immediately after EuroOffice was announced, OnlyOffice accused the new project of violating those terms. OnlyOffice says that its version of the software includes additional rules, especially around branding, credit and license notices. According to OnlyOffice, these requirements must remain intact whenever the software is shared or modified. The company claims that EuroOffice removed or ignored some of these conditions and doing so breaks the license and ends EuroOffice's right to redistribute the software. Eurooffice disagrees. Its position is that some of the Only Office extra conditions cannot legally be enforced if they conflict the AGPL version 3 itself.

Jeff Massie [01:22:38]:
The heart of the dispute is whether Only Office has added rules are valid, you know, the additional terms under the license or the valid terms go beyond or the additional terms go beyond what the AGPL allows. If they go too far, Euroo Office argues, then a project like theirs is permitted to remove them. So in the in the first link in the show notes, EuroOffice goes into into details of why they say they can't work with OnlyOffice. One of the points they make is that Only Office is a Russian company and there are quite a few number of binary blobs or and compiled or obfuscated code blobs. So the very bottom line summary is a lot of users and customers require software that is not potentially influenced or controlled by the Russian government. Now because of this disagreement when the two organization between the two organizations, only Office has suspended its eight year old partnership with NextCloud because like I said, NextCloud's helped start EuroOffice citing license violations after the Euro Office Coalition forked the Only Office repo. The partnership between both platforms allowed nextcloud users to edit and collaborate on Office documents right inside their own instance, which made it a big deal for self hosters. OnlyOffice was favored over LibreOffice Collabora because of its native Microsoft Office format compatibility.

Jeff Massie [01:24:12]:
You know, docx xlsx, PP, PowerPoint, you know, the PPTX, you know, and superior browser performance. Now that's from one of the articles linked in the show notes, but I should say LibreOffice does support the Microsoft Document standard or to the best ability because even Microsoft says, you know, they publish standards but they don't publish at all. They keep some of the standards in their back pocket. Now I don't have the room in this story, but I should also mention the Document foundation tdf. Their membership committee has decided to eject from membership all Collabora staff and partners. That includes over 30 people who have contributed faithfully to LibreOffice for many years. Without going into the details and the political intrigue, their collaborative is going to create a new and cut down version of their Office suite. And when I say that politics and intrigue, there's a lot of thoughts though the article I read might be biased, that they're putting too many people who have no technical skills in charge and a bunch of other we'll call it shenanigans.

Jeff Massie [01:25:31]:
But I that was just one article. It could be very biased, but there will be a Collaborative Classic and a new Collaborative Office. So you they're keeping both versions. The new version is going to be entirely new, cut down a version for users that want a smoother, more friendly and less feature dense product than their classic product. Now from Collabora they say this gives a chance to innovate faster in a separate place on a smaller, more focused code base with fewer build configurations and much less latency. No Java, no database, just web based toolkit and more. So now since this article is getting a little long, take a look at all the links I have in the show notes I for many more details. And each of the articles has links in them as well.

Jeff Massie [01:26:21]:
So there's a lot, I mean a megaton of information and details I've left out, you know, but maybe going forward I'll be need to be looking at using Euro Office or Collaborative's new Office for my future needs. You know, who knew that some of the more boring aspects of software would suddenly become one of the largest controversies in open source and multiple locations at all about the same time. Who knew? We'll let everybody know more when we know more about how this is all going to shake out.

Jonathan Bennett [01:26:56]:
Interesting. So you've got the Document foundation and Collabora are feuding, apparently over. Oh, you have to read between the lines here to figure this out. But Document Foundation's post suggests that things Clobbera was doing could have led to the Document foundation losing its nonprofit status. And then also that employees of companies involved in legal disputes with the Document foundation must be removed from TDF membership. Which seems pretty reasonable to me. And then I'm sure Collabora has. Yes, Collabora, let's kick the developers out is their spin on it.

Jonathan Bennett [01:27:40]:
And then only Office and Euro Office are fighting. So there's one thing, putting all of the immediate mess aside, there's one thing that's really interesting here from a licensing standpoint, and that's that question about adding additional licensing terms to open source licenses like the agpl. This is something I've wondered about with Red Hat before, because if you read the gpl, it's got things in there about you do not get to add additional license restrictions. And I've looked at what Red Hat does with its. Oh, I forget what they call it, terms of service or whatever. And it's like some of these really seem to me like they're additional licensing restrictions. It would be very interesting to see that question go to court and what would become of it that would potentially change the open source landscape a bit.

Jeff Massie [01:28:31]:
Yeah, and that was mentioned in one of the articles that it really kind of brings to a head of like, well, how far can you push this license? When does it stop? Yeah, there was a lot more reasons why they, you know, Euro Office said they couldn't work with Open Office. It's kind of a closed ecosystem and Europe is very against because it's very Russia based. But there was also some other restrictions in there of how things are done and how it's kind of like I said, they've got various code blobs, binary blobs in there that nobody knows what they do.

Jonathan Bennett [01:29:14]:
The Russia based in the code blobs, that's only Office.

Jeff Massie [01:29:19]:
Yes.

Jonathan Bennett [01:29:19]:
Okay. Not Open Office, not Liber Office. Let's make sure, make sure we get those, keep those straight.

Jeff Massie [01:29:25]:
No, it's only, yeah, only open only. It's the code blobs and everything are only in. Only Office. Euro Office did not like that. So that's why they forked it. And we're now having this license disagreement and at the same Time Collabora and LibreOffice are looks like splitting ways and there's a fight between the Document foundation and Collabora. But you know, I, I'd Be interesting to see how that's going to fall because Collabora has done a lot for. Lot of.

Jeff Massie [01:30:04]:
Lot of code for LibreOffice.

Jonathan Bennett [01:30:07]:
So. Yeah, and then you also have one more layer to this. If you go to the Document Foundation's blog, they have. And they have an article here. So the Document foundation, the people behind LibreOffice, have a doc, a blog post about Euro Office and basically ragging on them for using Microsoft formats by default instead of something like odf, which makes sense to me. It don't.

Jeff Massie [01:30:38]:
Well, and they say in there, it's not ideological, it's kind of what the standard is globally. And they do support the. It's like LibreOffice, they support the Open Document format by default, but they also support the Microsoft versions as well, because that's what a huge chunk of the world uses. It's kind of the de facto standard for most things. You know, the PowerPoint, Excel and Word. It's what it is. So it's either. So they make a special point.

Jeff Massie [01:31:18]:
It's not ideological cycle. It's like we're just supporting the most common standard, but we do support all the open ones as well. And then you look at it like, should you ideologically support that open standard no matter what and kind of make people choose the other format? Or do you just say, well, here's the easiest for compatibility between users and different groups, or, you know,

Jonathan Bennett [01:31:48]:
I think we should all go back to ASCII text. That would just take care of all of it.

Jeff Massie [01:31:55]:
Well, back in the good old days,

Jonathan Bennett [01:31:57]:
back in the good old days, we had TXTS and we liked it.

Jeff Massie [01:32:00]:
Yeah, give me the VT200 back.

Jonathan Bennett [01:32:03]:
Yeah, absolutely.

Jeff Massie [01:32:04]:
Type F keyboard.

Jonathan Bennett [01:32:05]:
All right, one more, one more news story I'm gonna sneak in here. And so after all of this rigamarole, and in some cases is pretty, pretty downer news, finally, finally, something is landing that we can all be excited about, and that is Wayland Protocols 1.48 includes XDG session management. And what in the world does that mean? Well, several things are in there, but one of the ones is window locations and restoring Windows positions and states. And so this is. And if you're running Wayland, you probably have seen this. Your browser either crashes or you have to restart it because of a system update or a browser update, what have you. And particularly if you run a system like mine, where you've got multiple monitors and multiple browsers going around, goes down when it comes up, you just have them all stacked right on top of each other right in the middle of the screen, it's like, ah, this is terrible. And you got to, you know, move to the various people places where they go, and it's just a pain.

Jonathan Bennett [01:33:10]:
And I know I'm overselling the frustration just a little bit, but bear with me. It's a pain. It's terrible. And on X11 and on Windows, these browsers were smart enough to go back to where they're supposed to be. They remembered their last location and would go back. And this is something that the free desktop GitLab and the Wayland guys have been fighting there for years now. A very long time those discussions have been going. And finally it has all landed in 1.4.8.

Jonathan Bennett [01:33:44]:
So the, the fact, the reason this matters is now that it's technically in Wayland protocols, you'll see support for it land in Chrome, you'll see support for it land in Firefox, you'll see support for it in GNOME and in kde. And so, you know, this time, six months from now or next year, it'll all just work. And so you can restart your browser and everything will go right back to where it's supposed to. And there'll be some other fun features that come as a result of this, but it has been a very long time in the making and so good for them for finally getting it out and putting it into the bike. Shedding over this.

Jeff Massie [01:34:26]:
Yeah, definitely good news.

Jonathan Bennett [01:34:28]:
Yep, finally. All right, we've got a couple of command line tips. Mine is going to be a lot shorter than I expected it to be because I found out I needed to hack on the tool that I was going to be recommending. So we're going to do half of it today and half of it next week. And I'm going to between now and then get some patches sent in. So you guys will get to run my code. Anyway, quick break and then we'll be back to talk about command line tips. All right, Jeff, your command is VM stat.

Jeff Massie [01:35:01]:
Yeah. So VM stat is a memory monitoring command. It gives information about process, processes, memory, paging block, IO traps, disk and CPU activity. I mean, simple. And the simplest way to use it is just type VMSTAT and it'll give you some basic information to get more information. Like most command line tips, you start using switches. For example, if you type VM stats space dash A, it gives active and inactive memory. If you use the dash D switch, it will show disk information.

Jeff Massie [01:35:35]:
Dash F will show the number of forks currently on the system and the dash M, which you will need to run the sudo but it will give Slab Slab info Slab. Now, I should say slab allocation is Memory management is a memory management technique used in the Linux kernel to efficiently allocate memory for frequently used small kernel objects. Basically, it reduces fragmentation and speeds up allocations by maintaining dedicated caches for each object type. Take a look at the article linked in the show Notes for more details and switches as there's more information that VM Stack can pull and a lot more switches that are formatting options. So you can make the output like how you want. But basically it's kind of interesting for just seeing what your memory's doing and various counts of other items.

Jonathan Bennett [01:36:28]:
Yeah, probably super useful for trying to figure out why a machine is misbehaving in one way or another.

Jeff Massie [01:36:34]:
Yeah, and it gives you pretty good detail about all sorts of stats of what's going on. So it's not just like, oh, here's your normal free memory. I mean, it's hitting cache and swap and you can get counts of how many times things have happened. It's pretty interesting.

Jonathan Bennett [01:36:56]:
Very cool. All right, so last week we talked Grafana and we are back MQTT and Grafana and we are back to do a little bit more with this. And this is where basically I've got things running again. Basically where we stopped last week and from. Well, here we'll do it from the home, from the Grafana home. We've got the little hamburger menu here and one of the options is to explore. We'll make sure that our MQTT data source is the one that we're looking at. And then you have to pick a topic and MQTT topics are kind of a pain, honestly.

Jonathan Bennett [01:37:33]:
There's a. There's a really easy shortcut here. You can just put in a hashtag or pound sign or a splat, whatever you want to call it, and then we can go up here and tell it to run this query and it will. It'll show us how many. In fact, we can automatically run it every five seconds and it will show us how many things returned that match. And right now we have zero things returned that batch. Well, that means that there is nothing there for us to look at. Let's find something there to look at.

Jonathan Bennett [01:38:07]:
So we are going to kick over to a console window here and there are a couple of commands. They come with mosquitto, but there are a couple of separate commands. One is mosquitosub, and this subscribes to an MQTT server. And so in this case we're telling it to subscribe to the local MQTT server, the 127.0.0.1. And if we run this, nothing happens because again, we don't have any data flowing there. So there is another command that is mosquito pub. And so we are just going to publish again to localhost. We're using the topic of test topic and we're just going to send a message of hello mqtt.

Jonathan Bennett [01:38:56]:
We run that and then we kick back over to the sub and you see, suddenly we've got hello MQTT just popped up. Now let's go back to Grafana and see if we have it there. And the answer is that yes, actually we do. Let's see if I can. Yeah, there we go. You'd see it there. We got the time, which right now on the 4th and the value hello MQTT, it shows up. We're not going to dive into adding this to a dashboard today.

Jonathan Bennett [01:39:32]:
We will. I've got some real world data that I want to put into MQTT and then into Grafana and the tool that I wanted to talk about. One of the things that you do with mqtt, all right, so you have something running that pushes data, individual messages to mqtt. Something that you, you see that happens is you'll run a separate program that'll pull that message out, do something to it and put it right back on the MQTT server on a different topic. And so I've got data that I can put on MQTT that's actually encrypted and I need a program that will decrypt it and then turn it into JSON and then put it back. That's the one that was not working. So I got to, I got to troubleshoot encryption. It was, it was generating a nonce incorrectly inside of AES.

Jonathan Bennett [01:40:22]:
So fun nerdy encryption stuff. I have it working, but ran out of time to finish getting the demo ready. So we'll be back next week, we'll talk about it. And so if you don't remember, I'm actually slated to talk at the Ubuntu conference in London that is going to correspond with the release of Ubuntu 26.04 LTS. I've got a 45 minute talk there that's going to be partially a workshop doing a live demo. And part of what we're doing here is I'm actually stepping through building my live demo and you guys get to watch it as it comes together. This is not all yet fully polished. You get to.

Jonathan Bennett [01:40:59]:
This is the spit and Polish procedure that you guys get to watch. So it's fun, though. This is coming together, and I've got a little bit more of it working on the machine behind me, and it is pretty cool. And I'm learning Grafana, which is also neat.

Jeff Massie [01:41:12]:
You're testing in production?

Jonathan Bennett [01:41:15]:
Yeah, sort of testing. Doing research, development for the show. We'll put it that way, yes.

Jeff Massie [01:41:20]:
Yeah, well, that's what we try to be real here. There's no fancy polish on this. We're just who we are. And you see, what you see is what you get.

Jonathan Bennett [01:41:29]:
What you see is what you get. Yeah. All right. Anything you want to plug for the end of the show? Want to get the last word in on something?

Jeff Massie [01:41:37]:
Just a couple little things. You know, I know some people are really stressed out about things, and, you know, just sometimes you just got to go outside and take a breath and. Or sit quietly and drink some coffee or tea and just relax and just step back from everything and just, you know, be calm. That being said, I just have a little poetry login. Incorrect. Only perfect spellers may enter this system. Thank you, everybody. Have a great week.

Jonathan Bennett [01:42:15]:
That's great. All right. Appreciate you being here, Jeff. It's been a lot of fun.

Jeff Massie [01:42:19]:
I love it.

Jonathan Bennett [01:42:20]:
Yeah. All right. If you want to find more of me, there's, of course, hackaday. We skipped a week with Floss weekly, but I think we have somebody scheduled for this Tuesday and then working to get the schedule filled up after that. And then, of course, you can keep track of all the stuff going on at the meshtastic project. That's the other thing that keeps me busy these days. Other than that, just want to say thank you. We appreciate everybody that's here whether you watch.

Jeff Massie [01:42:44]:
Sure.

Jonathan Bennett [01:42:44]:
Listen, get us live or on the download, and we'll be back next week on the Untitled Linux Show.