Security Now 983 transcript
Please be advised this transcript is AI-generated and may not be word for word. Time codes refer to the approximate times in the ad-supported version of the show
0:00:00 - Leo Laporte
It's time for security. Now Steve Gibson is here. We have some really interesting things to talk about. An update on the polyfillio attack. Can content delivery networks be used safely? That ransomware attack against 15,000 auto dealers what have we learned there? And you won't believe who still uses Entrust for their certificates. I'll give you a hint it's got a gov domain. Finally, we'll talk about the Snowflake breach. Steve's not sure he believes the stories. He's going to get to the bottom of this one All coming up next on Security Now.
0:00:40 - Steve Gibson
Podcasts you love From people you trust.
0:00:43 - Leo Laporte
This is Twit from people you trust. This is Twit. This is Security Now with Steve Gibson, Episode 983. Recorded Tuesday, July 16th 2024. A snowflake's chance. It's time for Security Now, the show you wait all week for. You collect the little tidbits in the back of your brain at least I do going, oh, I wonder what Steve's going to say about that. Now, here's your chance. Steve Gibson is here. Our master of ceremonies. Hello, Steve, Yo Leo, Good to see you, Good to be with you.
0:01:18 - Steve Gibson
Yes, one of the final episodes we're recording with you in the In the old studio.
0:01:27 - Leo Laporte
That's right, there'll be uh, let's see, there'll be two, three more, because we're leaving august 8th, so august 7th will be the last episode of security now from uh, the, the east side studio, which has been it's been a good one after uh uh this week in google on wednesday yes, yes, be the last report, then that's when I'm going to and that gives you a few days.
To spread the accelerant. Thursday, Friday and Saturday Stuff the newspapers in the various crevices, Light the flame. No, we're not going to do that. We're not going to burn it to the ground.
0:01:59 - Steve Gibson
Okay, so we got a lot of fun stuff to talk about and some sort of some interesting takeaways. I think Today's podcast number 983, for this July 16th is titled A Snowflake's Chance. This, of course, named after the firm. I don't know why they named themselves Snowflake. It's not a good name. It just seems kind of like a flaky yeah, flaky no, it's not a good name Now that you mention it. I don't know that. I want to store all of my customer data at Snowflake.
0:02:36 - Leo Laporte
That's the one that has a chance in hell. I mean, that's not a good name.
0:02:40 - Steve Gibson
Yes, no and at least 350 companies are now very sorry that they did store all their data there. You know Ticketmaster, of course, famously now AT&T, with 110,000 of their customers' mobile phone metadata, and the people that use the AT&T system and which AT&T resells to. Anyway, big disaster there. Some interesting takeaways from that, but we have a lot of other stuff to talk about. The discussion last week of the polyfillio attack caused people to say can content delivery networks be used safely? Uh, can content delivery networks be used safely? Because, of course, the problem was that that polyfillio, that the domain got purchased by a now known to be malicious Chinese company. So what's to prevent that from happening elsewhere? Um, also, what do we learn from the ransomware attack that took 15,000 auto dealers down a couple weeks ago? And, interestingly, I heard from three of our listeners who were directly impacted by this. Oh, I mean, so this thing was widespread. Also, guess who uses an Entrust certificate and when it expires? We're going to look at that.
0:04:08 - Leo Laporte
Oh, that's interesting.
0:04:11 - Steve Gibson
Can't wait to hear that one. Don't go over to the IRSgov until you catch up on it. Also, how worried should we be about the polyfillio attack aftermath? That is, do we have to reformat our hard drives or what? And a listener really brought up an interesting view that I'm going to share about whose side Microsoft is really on, by taking a look at their history. I'm going to comment about how GRC's new security now mailing is going and then, as I said, we're finally going to take a look at this company named for who knows why, snowflake, which was the epicenter of what has now become the largest series of corporate data breaches in history. And, as we know, unfortunately that's saying something. So there's been a lot of finger pointing who's saying what, what appears to be most likely, and always we have a fun but puzzling picture of the week. I understand what's going on. I've already had feedback from those who received Security Now's email a couple hours ago saying what. Anyway, we'll explain it.
0:05:33 - Leo Laporte
I haven't seen it yet. We will see it together for the first time, I think everyone is going to like it.
0:05:38 - Steve Gibson
They're going to find these next two hours have been well spent.
0:05:42 - Leo Laporte
I think that's the case as always, mr G. Two hours have been well spent. I think that's the case as always, mr G, and we have Adam watching on Facebook today. Thank you, adam, it's very nice to have you. You don't know this, but we are streaming. Maybe you do know this, steve, I don't know. We're streaming now everywhere.
0:05:59 - Steve Gibson
But we're using Zoom.
0:06:00 - Leo Laporte
We are, but we have a service called Restream that we've started using. That's going to be. We used it a couple of weeks ago. Remember I was pulling up chat and it was distracting you.
0:06:10 - Steve Gibson
So you can either use Restream for the conferencing or use Zoom as the front end and then Restream on the back end.
0:06:18 - Leo Laporte
That's exactly what we're doing. We're going to have Zoom ISO in the front end. We're going to have Ecamm, so the technical director, our producers, will also be switching the show. I won't have to do that anymore, which is interesting. Actually, we'll be doing less. I guess that's good, because those restream fades were not good.
No, these will be much nicer, but we do have restream, taking our video and putting it everywhere. So it's still YouTube. Youtubecom slash twit, slash live. But it's still YouTube. Youtubecom slash twit, slash live. But it's now twitchtv slash twit. It's on Facebook, it's on LinkedIn, it's on xcom, all over the place. So Adam is watching on Facebook and I guess hasn't seen the video or doesn't remember, but he said what's that speaking spell right behind Steve there, but there's a story with that right.
0:07:07 - Steve Gibson
Yep, I was involved in its creation.
0:07:10 - Leo Laporte
Isn't that amazing. So that's one of Steve's products.
0:07:14 - Steve Gibson
I guess you could say Well, no, it's a linear predictive coding. Speech synthesis was the technology used, and that's part of what happened when I was at the Stanford AI Lab.
0:07:28 - Leo Laporte
It's kind of amazing. I mean, here we are. What was that? 40,? 50 years ago, 50, I guess.
0:07:34 - Steve Gibson
Yeah, it was. 73 was when I graduated high school and I was at the AI Lab for the couple years before that it's kind of amazing and look where AI is now.
0:07:43 - Leo Laporte
I mean, yeah, the sound of the voice synthesis and the speak and spell, versus what 11 Labs is doing now with celebrity voices. I mean it's incredible. We have lived in interesting times, Mr Gibson.
0:07:57 - Steve Gibson
One of our listeners one of our female listeners, took offense to my little rant last week about man in the middle and needing to call it what was it not? Attacker in the middle, adversary in the middle? And she fed what I said into some generative AI asking it how could women find this offensive? And it was astonishing. I mean I really it put me in my place. I thought, well, maybe you just need to go. You know we need to turn you out in the pasture because you're obviously you know you need to have a serious sit down with human resources or at least least with chat.
0:08:44 - Leo Laporte
Gpt Gemini wants a word with you. Mr Gibson, let's pause before we get to the picture of the week. This week I'll look at it for the first time with you and talk about our first sponsor of the show today Vanta Vanta name you should know. I was driving up 101 in Silicon Valley yesterday and I saw a billboard that says Vanta compliance. That doesn't SOC too much. Get it, I love it.
Whether you're starting or scaling your company's security program, demonstrating top-notch security practices and establishing trust is more important than ever. Your customers expect it, but Vanta makes it easy. They automate compliance, not just for SOC 2, by the way, but for ISO 27001 and, of course, many more, saving you time, saving you money while helping you build customer trust. Plus, you can streamline security reviews by automating questionnaires demonstrating your security posture with a customer-facing trust center. You know, put that link on your front page, check our trust center, all powered by Vanta AI. So it doesn't mean more work for you, it just means customers that are reassured, with over 7,000 global companies like Atlassian, flow, health, quora and many, many more using Vanta to manage risk and prove security in real time. You know this is the real deal and we're going to get you a great deal.
Speaking of deals $1,000 off Vanta when you go to Vanta. V-a-n-t-a dot com slash security now. V-a-n-t-a Vanta dot com slash security now. One thousand dollars off vantacom security now. All right, I have, uh, the picture of the week right here on my laptop and I am ready to to pull it up. Do you want prelude?
0:10:42 - Steve Gibson
prelude, so I gave this one the title. Does anyone wonder how to lock this bathroom door?
0:10:53 - Leo Laporte
oh, my imagination is reeling. Let us let us look together. Apparently no one knows how to lock this bathroom door. There are one, two, three, four, five, six signs, all of which say do not turn, push to lock please, yes, we have one most.
0:11:17 - Steve Gibson
Five of the six are in a sans serif font of varying sizes, apparently having been incrementally added to the door. Five of them were on the door, one is over on the wall. Um, somebody came on the one that is the serif font, which reads simply push to lock, where those four words fill the entire page. Had someone came along and gave it some extra underlining in a felt-tip marker.
Simply push. So we have just push to lock, do not turn, with three exclamation points Push to lock, do not turn. Oh, that one was actually. That looks like it's pretty much the same.
0:11:57 - Leo Laporte
That's replicated. The first one says just push to lock.
0:12:01 - Steve Gibson
The second one says push to lock. So apparently that wasn't sufficient, so they added the just push to lock. Second one says push to lock. So apparently that wasn't sufficient, so they added just push to don't turn it now we actually have a paragraph down at the bottom. If you just push the button straight in without turning, the door is locked period now what this makes me wonder what happens if you turn it.
0:12:23 - Leo Laporte
In fact, honestly, being the rebel that I am looking at these signs, I would turn it.
0:12:29 - Steve Gibson
Apparently people do. I know what's going on here. Oh good, If you push in and now, so we just explained. It's got sort of an L-shaped handle coming off to one side and if you push it down the door will open and you can leave. Yeah, but the button is actually sort of a thumb screw.
0:12:55 - Leo Laporte
It looks like you should turn it. It's got a little affordance for turning it.
0:12:58 - Steve Gibson
Oh, it wants you. It is saying turn me, turn me, yes, everything about this, yes. So you push it in and it stays in. Then when you push the handle down, it pops out Right. But if you push it in and turn it, it locks the button in.
0:13:20 - Leo Laporte
It locks you out after you go through the door.
0:13:23 - Steve Gibson
Well, yes, exactly. You're going to get you out after you go through the door. Well, yes, exactly.
So what people are doing is because this was a poorly chosen handle for a bathroom. They're pushing it in and turning it, thinking, okay, I don't want anyone coming in on me when I'm in the middle of doing my business, right of doing my business, right but then they leave, and because they twisted the door locks behind them and nobody could ever get in again. Now, now, I don't know if, if, if, uh, if, residential home door jam locks still have this, but when, leo, leo, I know that when you and I were young, if you looked at the door jam on the front door of your home, there would often be two buttons there, yes, and those two buttons would lock or unlock the, like the thumb lever on the outside of the door, the point being that it would keep the door locked from the outside, whether or not you locked it from the inside. It's exactly this technology, I see.
0:14:34 - Leo Laporte
Unfortunately, they made this much too easy to use and they put the wrong affordance on it because they put a little turning thing on it which you shouldn't have. By the way, if you've not read this book, the Great Don Norman's Design of Everyday Things and the cover tells you all that you need to know. It's a teapot with a handle on the same side as the spout, which obviously is poor design. He talks about this all the time. Doors that invite you to push it because they have a push bar or, worse, have a handle for pulling but do the opposite, is very common, right? So they have to put a sign up that says push, don't pull. Because you pull it, nothing happens and you look like an idiot. So, uh, this is very common. And he says don't blame yourself. This is just common. And he says don't blame yourself, this is just poor design.
0:15:26 - Steve Gibson
Well, and what this company, whoever they are, should do, is just hire a locksmith or a doorknob person to come out and put a button. Here I mean because you can buy the same handle with a button and then you don't have this problem.
0:15:43 - Leo Laporte
Yep. And that's the other thing Dave Reddick up saying no amount of signage is going to solve that. If you have, that's, you know, a little hint. If you have eight or nine signs on the door explaining in the same way how to do it. It's messages and getting through it's.
0:16:00 - Steve Gibson
It's just poor design and presumably somebody has to go when they come into this bathroom, so they're not taking any time to like, read the dictionary of proper knob operation.
0:16:16 - Leo Laporte
Well, dave also points out, the other thing they may do is push it and then test the handle and then inadvertently unlock it, which could also lead to embarrassment. So we need a better design.
0:16:32 - Steve Gibson
Better design, exactly okay, so using content delivery networks safely. Looking back upon last week's polyfillio attack topic I, I can imagine that I may have come off as being very anti-third party when it comes to sourcing potentially dangerous content, such as code libraries from third parties, you know, such as high performance content delivery networks, cdns. It was never my intent to rain on the idea you know the concept of CDNs in general for this purpose, because the web designers have made ample provisions for safely pulling code into web pages from remote sites. And a number of our astute listeners sent me notes asking variations of uh, steve, did you perhaps forget about asset integrity pinning? Actually, no, I didn't, but those questions also raised a very good point. So, rather than answering each of those notes separately, and since it's a terrific topic for this podcast to cover in the wake of the polyfillio news, I wanted to talk about how third-party content can be delivered safely and why the polyfillio facility was never able to take advantage of that. Okay, so the formal name for the facility is sub-resource integrity, abbreviated SRI, where the concept and implementation could hardly be clearer, cleaner and simpler. You know, as the best things are, the same HTML script tag that contains the URL of some remote third-party code or style sheet, because it can be used for link tags also, where the URL is what the browser is being asked to remotely load can also optionally contain another name-value pair, specifically integrity, equals, and then a big quoted string. The format is the word integrity, followed by an equal sign, then any one of the prefixes SHA-256, sha-384, or SHA-512. And our astute listeners will already be going ah, I know what's coming. What's coming is a dash followed by the specified hash of the expected URL resource which has been hashed under the specified hash, then encoded from its binary into base64 ASCII. Okay, so here's what all that means.
When a web designer wishes to pull some remote resource from a remote content delivery network, you know, or really from anywhere, that where they do not directly control that resource, they want to be absolutely certain that the resource they want has not been changed from what they expect to receive. So they first go to the SRI hash generator site very handy, wwwsrihashorg, and this is formally specified, like at the W3 consortium, mozilla talks about it. This is sort of like the reference site for generating these resource protection hashes, again wwwsrihashorg. Or you could open SSL or any other utility that can create base64 encoded hashes. The SRI hashorg site is handy since when given a URL, it will fetch the resource for you, perform the hashing, perform the base64 encoding and return the snippet of code tag, which already is set up, to drop into your own web page to perform all of the proper matching. It defaults to SHA-384, sort of the one of medium strength, but really SHA-256 is plenty strong. Also, it creates a somewhat shorter hash, but you know, these days web pages have become so out of control no one cares. Shorter hash, but you know, these days web pages have become so out of control no one cares Anyway. So our designer goes there, gets the hash, drops the URL of the jQuery library they wish to use, you know into SRI hashorg to receive the hash. Then they add this hash, along with that integrity keyword, into the jQuery fetching script tag in their web pages and they are henceforth protected from any modification of that code. When their page is delivered to a user's web browser, the browser reads the pages HTML, sees the script or link tags and fetches the resource referred to by the URL. But because that script or link tag also includes an integrity argument, before the browser does anything with the freshly downloaded resource it takes its own hash of what it just downloaded. Base64 encodes that and compares the result with the hash that follows the integrity keyword, and only if the hashes match will the browser allow that code to enter the browser's inner sanctum, to be trusted and used. So what all of this does is very nicely and cleanly allow web designers to protect their site's users from both inadvertent or deliberate alteration of the resource that they're requesting.
Also, note that, since code libraries are constantly evolving, jquery is currently at, for example, version 3.7.1. I noted that the jQuery my email system is using I think it's 3.6.0. So jQuery is a moving target. Because of that, it's also necessary for the jQuery or whatever library specification to indicate the exact version that is being desired so that the hash will match. Since the CDNs will always continue to offer all older releases, a site will continue to use that version, the one that's known to work, until its designer changes the version number in the URL and obtains that newer releases matching hash to add to the, that newer releases matching hash to add to the invocation tag in the HTML. Okay, so we now see how it's completely possible to safely obtain potentially dangerous script code from any other service that the designer does not control All of the advantages a CDN has to offer. You know, like nearby points of presence. So you get super fast content, delivery of potentially large content without delay, that can all be used without any risk at all by specifying the hash that you expect that content that you receive to have, and the browser itself will just say nope and will not use it if the hash doesn't match.
But as I said earlier, unfortunately this very slick protection was not available to users of polyfillio. I touched on this briefly last week but since it wasn't our focus I didn't elaborate or highlight its significance. So just now, when I was putting the show notes together, I used our trusty Web Archives Wayback Machine to show the polyfillio website homepage. They were clearly they the poly filled on IO people, when they originally created this, were clearly very proud of what they'd created and they were a little tricky. The homepage of the site says just the poly fills you need for your site, tailored to each browser. And then they said copy the code to unleash the magic.
And what we see is a script tag with a source equals and then a URL which doesn't have any specific subversion numbering. It says V2, but otherwise no polyfillminjs. Then it says polyfillio reads the user agent header of each request and returns polyfills that are suitable for the requesting browser. They said tailor the response based on the features you're using in your app and see our live examples to get started quickly. In other words, they are customizing what they return depending upon the browser.
The user is using no version numbers, so the way the polyfillio site always worked was that it generated and delivered custom polyfill JavaScript code specifically tuned to the make, model and version of the web browser each individual user was using and Leo, I'm going to need you in about a minute. This always made it actively hostile to the web's sub-resource integrity system, which prevented any of the Polyfillio's great many users from supplying a hash code because they would receive entirely different code each time. Anyway, that said, our listeners were 100% correct to point out the power and value of sub-resource integrity protection. It's been universally supported by every web browser for many years, so it's something that all web designers whose web pages are pulling code, which should never change without notice from any third party, should be adding to their bag of tricks.
So I did want to just bring it to everyone's attention. You know that very cool website that I mentioned, srihashorg will give you the hashes and the code to drop in to any remote resources you're pulling, so it is entirely possible to protect yourself. So again, I thank our listeners for bringing my attention back to that. I wanted to make sure that I explained that it's absolutely possible to safely pull remote content from CDNs not from polyfillio, because it was always delivering customized code, but otherwise yes, Interesting.
0:28:39 - Leo Laporte
Yeah, I mean, how often do we install stuff that says, well, just copy and paste this line, that will download and install software. And you know there's a program I put on every Mac called Homebrew, and that's how you're supposed to install it. I mean you can look at the bash script, I guess, but it's just very nerve-wracking. You nailed it, though, last week on talking about the polyfill. It's very nerve-wracking. You nailed it, though, last week on talking about the polyfill. In fact, I wanted to mention that in our Twit forums, at Twitcommunity, Pseudo Random Noise said Another great episode. And as a web developer, it was nice to hear how incredibly correct Steve was and how well he understands the subject matter. So I just thought I'd pass that along. People really appreciate your deeper understanding of this stuff. Thank you, I guess because you're a coder, you know you know about libraries and you know how tempting they are to load and, as it turns out thanks to supply chain attacks, how risky.
0:29:40 - Steve Gibson
Well, yeah, and you know, I'm a bit of a web developer myself. I don't have, as everyone knows, a fancy website that is taking advantage of all kinds of stuff.
0:29:49 - Leo Laporte
You're not using React or anything, but it's pretty good, it's nice, it looks good. I'm not even using JavaScript, it's all text, all HTML.
0:29:59 - Steve Gibson
That's right. Even the GRC's web, I mean our website menus, are just using CSS, with no scripting, in order to perform their magic.
0:30:12 - Leo Laporte
Well, that's all you need. Frankly, that's good enough, yeah.
0:30:19 - Steve Gibson
It works, be a big one about this massive CDK global ransomware attack that impacted three of our listeners. Oh dear.
0:30:32 - Leo Laporte
Wow. All right, we'll get to that in just a moment, but first a word from our sponsor, panoptica. Panoptica is Cisco's cloud application security solution. Got it. Got it Provides end-to-end lifecycle protection for cloud-native application environments. I think, listening to this show, you think maybe I might need that right.
Panoptica empowers organizations to safeguard their APIs, their serverless functions, containers, their Kubernetes environments. Panoptica ensures comprehensive cloud security compliance and monitoring at scale, offering deep visibility, contextual risk assessments and actionable remediation insights for all your cloud assets. It's powered by graph-based technology and this is their attack path engine. Specifically, panoptica's attack path engine prioritizes and offers dynamic remediation for vulnerable attack vectors in real time, helping security teams quickly identify and remediate potential risks across all their cloud infrastructures. And because it's a unified cloud native security platform, you're minimizing gaps from having multiple security solutions. It gives you centralized management and reduces those non-critical vulnerabilities from fragmented systems.
You know the little gaps, the interstices between your solutions. Panoptica, it's nice to have all in one, provides several key benefits for businesses at any stage of cloud maturity, including advanced CNAP, multi-cloud compliance, end-to-end visualization, the ability to prioritize with precision and context, dynamic remediation, increased efficiency with reduced overheads. Here's the deal. Find out more by just going to the website panopticaapp Okay, not com Panopticaapp to learn more P-A-N-O-P-T-I-C-A Panopticaapp. We thank Cisco and Panoptica for their support of this fine podcast. And now another breach.
0:32:48 - Steve Gibson
Steve. Ah, yes, toward the end of June, I heard from three of our listeners whose lives had been affected by a recent major attack on a very large automotive dealer network a very large automotive dealer network. On June 21st, I received two notes from our listeners. First one said hello, mr Gibson. My name is Sean and I'm an automotive technician at a GM dealership and have been listening since about 2016. Awesome.
0:33:23 - Leo Laporte
Oh, I know what he's going to talk about. Oh, I know what he's going to talk. This hit my dealer too. Yep, okay.
0:33:31 - Steve Gibson
He says I love when your world of security crosses over into my world of automotive. My dealership, as well as thousands of others yes, 15,000 of others is affected by the CDK cyber attack that happened yesterday. Yuck, when the details come out, I would love to hear your take on it. This is the first time a cyber attack has had a direct effect on me. He said, parens, we get paid by what we do, and this is slowing everything down as we have to go back to manual ROs and quotes, lowering my booked hours. Oh, that's too bad. Thanks, sean Yikes. And then we actually heard from, apparently, an owner of a dealership. On the same day, on June 21st, I received Steve. Thanks for all the years of podcasts.
I've been a listener from the beginning and a watcher from the tech TV days. I hope to hear some coverage of the CDK global incident. Sales of auto repair parts from the dealer side of the industry have come to a screeching halt as they're unable to create invoices nor tell us our cost for a part. I was told today from one dealer that they hope to be able to sell me parts next week with some form of paper invoice. The delivery box truck that stopped was almost empty today, and he said I only got my parts today as they had already been ordered and invoiced prior to the issue. And then, finally, daughter-in-law are both remote workers for a dealership network that has been brought to a standstill by this cyber attack. Usa Today is reporting 15,000 dealerships across the United States are affected and may not be back online until the end of the month. Color me cynical, but I'm fond of the saying there is no cloud. You are just someone else's computer. Usually, that other computer is better secured than your own, but, as you so frequently say, it's not a matter of if but when. I appreciate the work you and Leo put into the podcast each week. Best regards, richard in Clemens, north Carolina. Okay, so what's the scoop on this? We have a situation where 15,000 operating dealerships were dependent upon a single MSP, a managed service provider, also sometimes referred to as SAAS or delivering, offering SAAS software as a service, for the dealerships were dependent upon this for all of their I guess we still call it paperwork processing, though it's virtual paper. We'll be taking or talking rather more about software as a service when we get into today's discussion of the Snowflake disaster, but in this case, a Russia-based drive-encrypting ransomware cyberattack took down hard the entire network of 15,000 auto dealerships which needed that network to operate.
I found a terrific piece posted on Medium by someone who's been in the auto industry and writing about it for some time, speaking from her long experience. Kathy's headline is the CDK Cyber Attack. Recovery Will fall squarely on the accounting office, she writes, are often the cleanup crew when several types of problems arise. There are still systems and procedure hiccups that happen today, but thanks to technology and automation there are fewer in number. Then came the CDK cyber attack.
This CDK cyber attack is on a whole different level. This breach is a very different type of problem. But in the end, when things begin to settle, which may take months, it will be the accounting office who will be asked to gather the thousands of dealership puzzle pieces from sales, service and parts and methodically match them up together to form some semblance of financial order. The end of the month is here, so she was writing this at the end of last month, June. She said new car dealerships are required to produce a monthly financial statement as mandated by the manufacturer and certain lenders. It's unclear as of this writing if a June financial statement will be available. I would say the chances are slim. So why did the CDK cyber attack happen? There was once a company called ADP Dealer Services who were a great DMS provider. Dms is Dealer Management System, which is the generic term in the industry. They got rolled into a company called Cobalt that sold mostly digital marketing services. Then all of that got rolled into CDK Global and with that came yep, wait for it private equity investments.
Now this is her speaking, not me, and I'll address this a little bit later in a second. She said the first thing to get cut when private equity rolls through the front door is cost centers, and InfoSec, aka information security, is viewed as a cost center. The main people who defend the gates of the village, the company, from the barbarians, the hackers, are the first sent off to exile. When there's a ransomware attack, it's revealed with clockwork-like precision that no one has tested the backups for six months and half the legacy systems cannot be resuscitated, as a cybersecurity expert told me last week, a few days after the attack happened. Quote it's been at least two days since the ransomware attack, with no fix in sight, which tells me he said. A few things on this list have to be true A, they have no backups. Or B, if they do have backups, they're outdated or never tested, which is effectively the same as having no backups. C no one knows how to restore the backups. D, there's no disaster recovery plan or if it exists, it's outdated to the point of uselessness. E, multiple, single points of failure are baked into the infrastructure. Or, finally, f, they have no idea how compromised they are.
So she says, I'm very angry about how ADP Dealer Services. Once a great company has been raped and pillaged by private equity, the real pain is suffered by the rank and file at the dealerships who still have to care for customers and sell to make a paycheck. According to recent reporting, cdk will be paying the tens of millions of dollars in ransom, and I got some follow up reporting on that I'll share in a second. So I'll just note that information security doesn't seem like such a waste of money now at this moment, does it? So she says how did the CDK cyber attack happen? She says CDK is an ancient program. Not a lot has been done to upgrade the original version for decades. This is standard operating procedure when companies or private equity buy legacy companies. Innovation is not the goal. They slap on a new paint job or buff out the dents and package it as the new, improved version. That is always more expensive but quote worth the investment. Unquote. Ask any dealer how they feel about CDK and other DMS fees. These days, these corporate raiders' goal is to cut costs at all costs and in this debacle it's clear. They stripped the car for parts and left the data vulnerable to cyber criminals. Now I'll interrupt to just say, as we know, it's very difficult to completely protect any large organization from intrusion.
But her earlier point about recovery is unassailable. Any organization today whose survival would be threatened by a significant, protracted network outage should certainly arrange to get back on the air after any attack. Anyway, kathy continues, theoretically, a mature dealer management system provider should be able to lose any single critical part of their core business and be able to restore functionality within 24 hours, you know, barring a massive natural disaster or personnel losses. Instead, they have no backups, no redundancy, no separate servers and no silo databases which, when lost, are a pain to retrieve. But at least it's only one silo and not the entire client roster of 15,000 locations.
How does a dealer restore their records once the breach is contained? Once CDK pays the ransom, it may take weeks or even months to get all the data in order. After they receive the keys to the ransomware, she writes the database will likely have holes in it. That will add to the arduous restoration process. She says there's been a lot of talk online about just getting a new DMS vendor. While that seems like a good solution, the problem is that your data is being held hostage by whoever attacked CDK. Without the data, you have nothing to convert to the new DMS, but the idea of other DMS solutions is a good one. That should be explored. Once the dealership CDK records are restored, when the dealership comes back online, that's when the fun starts for the accounting office. During the outage, all employees continue to serve customers to the best of their ability using manual documents and a patchwork of software support. A bunch of paper slips.
Exactly Well, like remember when we're in a restaurant and their credit card processing goes out and it's like whoops.
0:45:16 - Leo Laporte
Yeah.
0:45:18 - Steve Gibson
So she says, when operations is functional again, all the business they produced new and used car sales, service parts, internals, warranty, anything that happened during the downtime will need to be assembled and manually input into the system. It could take a few weeks or a few months to match everything up and it will be a lot of work just to get back to normal. Organization is key. If it's a busier store think 150 plus cars per month or over 500K in monthly service labor it will take a considerable amount of time to input due to the sheer volume of transactions. Vehicle inventories new cars, used cars will need to be counted to verify every unit's whereabouts. Parts inventory should also be verified unless the store had some kind of redundant system that kept track of it during the outage.
Untracked inventories are ripe for theft. If all the manual input goes well and I do mean if she writes then all entries should land in their respective general ledger accounts. Schedules and other general ledger reports should be run to determine what it all actually looks like and to make sure all the monies that were collected are posted to their respective accounts. One surefire place to start is bank reconciliation. If you can balance your books to your bank, you'll have a roadmap to a decent amount of checks and balances. It will not be pretty, but with the always-present perseverance of dealership accounting office staff, sounds like she was once a dealership accountant.
0:47:10 - Leo Laporte
Well, this is definitely a post traumatic stress syndrome from somebody who's been on that front line for sure.
0:47:17 - Steve Gibson
For sure. She says it will ultimately come together. And she says I'm just so appalled that this event happened. When I first heard about it, I said to my colleagues in what universe is it okay to manage data in such an irresponsible way? And I'll have something to say about that too. In a second she said most dealership employees have never had to perform their job without the use of technology. It's a strong reminder that technology is only a tool for efficiency and it's only as good as its infrastructure and established crisis protocols.
There will be lawsuits, of course. The only question is how many and from whom. Certainly, I would expect claims against CDK from and she names three dealers for impending commerce and negligence in data loss, among other things. Consumers for the massive data breach of extremely sensitive information, and employees for data privacy and lost compensation. She says now is a good time for dealers to contact their cyber liability policy carrier, check to see if you have contingent business interruption coverage and put the carrier on notice. No need to file a claim just yet, but it's worth having a conversation to know if you're covered and for how much. Okay, so that's Kathy's take. In subsequent reporting, cnn Business reported under their headline how did the auto dealer outage end? Cdk almost certainly paid a $25 million ransom. So CNN wrote oh, my God.
Yep $25 million to get back online. Cdk global uh. Cnn said cdk global, a software firm serving car dealerships across the us that was roiled by a cyber attack last month, appears to have paid a 25 million dollar ransom to hackers. Multiple sources familiar with the matter told CNN the company has declined to discuss the matter. Pinpointing exactly who sends a cryptocurrency payment can be complicated by the relative anonymity that some crypto services offer, but data on the blockchain that underpins cryptocurrency payments also tells its own story. On June 21st, about 387 Bitcoin then the equivalent of roughly $25 million was sent to a cryptocurrency account controlled by hackers affiliated with a type of ransomware called BlackSuit. A week after the payment was made, cdk said that it was bringing car dealers back online to its software platform. They write Cryptocurrency allows for the exchange of digital assets outside of the traditional banking system, but a record of those transactions is accessible on the blockchain. Three sources closely tracking the incident confirmed that a roughly $25 million payment had been made to BlackSuit affiliates and that CDK was very likely the source of that payment. Those sources spoke on the condition of anonymity because of the sensitive nature of the investigation. The cryptocurrency account that sent the ransom payment is affiliated with a firm that helps victims respond to ransom attacks. One of the sources said declining to identify the firm. Okay, so will the payment of that $25 million affect CDK's behavior going forward? Who knows? The greatest cost is likely their reputation damage is likely their reputation damage.
We've previously seen the consequences of MSPs managed service providers being penetrated to allow malicious attacks against their clients thanks to the MSPs' access into those clients' networks. But that's not what happened here. The problem here was that 15,000 auto dealers had come to so depend upon the networked services provided by their massive MSP and I'm sure that was both at the MSP's urging and the dealer's willingness to avoid redundant work willingness to avoid redundant work that when that MSP was taken down by a ransomware attack the second order consequences were so widespread that at least three listeners of this podcast were directly affected and wrote to me. Whether or not this was a consequence of profiteering by private equity owners who stripped the organization of what they felt were excessive cost centers is irrelevant here. We've certainly seen many organizations attacked with devastating consequences when their owners were fully invested in their company's success and infrastructure security and, as we know, mistakes happen. Could profiteering ownership have been a contributing factor? Okay, sure, maybe, but we would need to have much more information about CDK Global's history to render any judgment about that. Judgment about that the point Kathy made in her article about there needing to be some explanation for the fact that CDK Global was unable to recover immediately without paying a multi-million dollar ransom that was certainly a good one, but we have no idea. You know what's going on behind the scenes and whether you know and what one way or another was the case, and to my mind, that's really beside the point. What I think we have here is another consequence of a theme we saw last week with polyfillio, where so many websites were pulling unverifiable code from a central source. This is another example, and just wait till we look at the snowflake disaster in a minute. One way to describe all of these widely different problems would be as the danger of the promise of a free lunch, or, stated another way, it's very rare that you get something for nothing. Remember that XKCD cartoon we showed last week, where a massive construction of blocks was ultimately resting on an endangered twig.
In the case of the CDK Global MSP outage, 15,000 auto dealerships had become dependent upon this single service provider for virtually all of their daily operations, and it's entirely human for this to happen over time, if CDK's service had been so reliably delivered for so long that the maintenance of any backup plan in the event of a CDK service outage seemed entirely redundant. For all we know, there were such plans in place 15 years ago, but staff changed, people who knew how to fall back to a manual system retired and left the dealerships, and new hires were only trained on and knew how to use the automated system. You know, just press this button and follow the on-screen prompts. So what gradually grew over time was a deepening dependence upon this miraculous new system that had, after all, demonstrated to be dependable enough to be depended upon, right up until the day its plug got pulled and without it, a massive network of auto dealerships were marooned. Kathy was correct in her prediction that class action lawsuits would be filed against CDK. Some already have been, and I think that's unfortunate, because the whole truth is this sort of free lunch failure is what comes with the territory. Class action lawsuits, after the fact, when the free lunch needs to be paid for, is just sour grapes.
Having tasted and grown accustomed to the power of the service provided by CDK, it would be safe to predict that not a single dealer is going to return to a manual in-house operation? Was the pain that great? No, not nearly enough. Might some switch to an alternative provider? I'd bet that even that is rare. Everyone is now breathing a huge sigh of relief with the network and the automation that it provides coming back up and business resuming as normal. Cdk's CEO has apologized. He's promised to improve their cybersecurity posture and has even offered some financial restitution to their 15,000 dealerships for the loss of sales and service revenue that they suffered. And life goes on.
So the message I'd like to take from this perfect example of what can go wrong is that, in the final analysis, it's all worth it. I don't take the opportunity to remind us of that often enough, the opportunity to remind us of that often enough. You know we're only doing all of this cyber stuff because it really does make sense. It really is phenomenally powerful. It really is improving people's lives.
Sure, there's a two steps forward, one step backward sense, and that faltering backward step can be painful, but the net effect is still one step forward. This still doesn't mean that a truly massive catastrophe is not possible. From all the evidence we continually see, we can feel the very real possibility of that in our guts and, following from my analogy last week of hoping for minor earthquake tremors, the hope is that other competing DMS dealer management system providers are looking at what just happened at CDK and shuddering while suddenly feeling better about the size of their own information security budgetary line item, and they ask their IT staff with renewed attention whether they are safe from the same thing happening to them and if not, what more do they need to do? Thanks to the network effects of this event, a great deal of press coverage and attention was given to this, so let's hope that some lessons were learned to better prepare other similar organizations to respond if it should happen to them.
0:59:05 - Leo Laporte
Yeah, but what lessons can be learned? No dealership is going to turn to a tool that they create themselves, nor would that solve the problem. Um, cdk, I'm sure, says well, we're going to make sure we're more secure, but are they and will they and can they?
0:59:23 - Steve Gibson
I don't think it matters, leo. I think that that that dealerships are saving so much right by using, by basically subcontracting out I mean the very fact that they were completely crippled demonstrates how much of the work.
0:59:41 - Leo Laporte
How useful it is.
0:59:42 - Steve Gibson
Yeah, yes, how much of their work. Cdk was successfully automating, right. So, yes, this was not good successfully automating. So, yes, this was not good. They were inconvenienced, you know, like the restaurant whose computers go down and they have to take orders by hand and process their customers' credit slips. It's inconvenient. Do they stay on paper afterwards? No, they recover and life goes on.
Yeah, and that's really the message I wanted to convey here is, yes, this was big, this was awful, but ultimately this was a minor earthquake tremor. This is what we want. We want to remind people these things can fail and failure. We would like it not to happen, but it does. But even so, when you step back from it, all the dealers are going to stay with CDK. They're going to say, well, the CEO says they're going to improve their security. And look, we got a check that didn't begin to cover our losses, but at least it's something to cover our losses, but at least it's something. And besides, it would be far too painful to have to train our staff how to do this themselves if we didn't have automation. And we don't really want to move to a different DMS provider because all of our stuff is here. So, look, it's working again. The lights are on. Let's just move forward.
1:01:11 - Leo Laporte
Yeah, and I don't even know what you could do. It's, I mean, it's almost like saying well, don't use computers, you know, go back to that paper system that never crashes. Nobody's going to do that.
1:01:25 - Steve Gibson
Right Now, I mean, I guess the only thing you could do if you really cared cared, would be to go to the expense of running your own system and not using a managed service provider. However, there may very well be significant advantages from being tied into CDK. There was some stuff Kathy talked about that was so deep in the weeds that I didn't include it, but it had to do with CDK's stature as a preferred vendor to auto manufacturers, some privilege status which all of those 15,000 dealers inherited as a consequence of using their dealer management system. So there actually is benefit that an independent dealer, being entirely independent even from automation, would not have.
1:02:28 - Leo Laporte
Yeah, and you can't even really fault cdk. I mean I wish they. I almost want to say don't. It's too bad. They paid the ransom, but there was no alternative.
1:02:39 - Steve Gibson
That was the cheapest way of them getting those desperate dealers back online yes, and and and we can fault them for not having, for, for, I mean, and they're probably faulting themselves for, for not having the um, the, the infotech, the, the information, uh, technology security to deal with this. They've been around a long time, for decades, before ransomware became a problem. I'll bet you this is pure inertia. Is there hoped it would never happen to them? And you know they, they were using backup technology and security.
That's a was a decade old yeah so when this thing took them out and encrypted their servers, they were to use the phrase that's common these days screwed, yeah, all right we'll have more in a bit.
1:03:36 - Leo Laporte
um, when I when I ask, when I say, well, what can you do? There are things you can do. Just listen to our sponsors. I mean we, that's. This is specifically why advertisers come to security now, because they're trying to reach out to companies like cdk saying, before it's too late, our show today, brought to you by lookout today, every company is a data company. Cdk maybe thought they were oh no, we're in the, uh, we're in the database. I don't know what they thought, but every company is a data company and every company's at risk cyber threats, breaches, leaks. I mean, this is the new norm.
And, of course, cyber criminals are not sitting back on their laurels. Every time you pay $25 million, they invest that money to get more sophisticated. They go oh, there's gold in them, narhills. But here's the problem. We live in a time now where there are no boundaries to your data, right, what it means for your data to be secured has completely changed. You can't put it in a vault. You can't take all your dealer records and lock them up every night.
Enter Lookout From the first phishing text to the final data grab. Lookout stops modern breaches as swiftly as they unfold, whether on a device in the cloud, across networks, working remotely at the local coffee shop. Lookout gives you clear visibility into all your data, at rest and in motion. You can monitor, assess and protect and here's the key without sacrificing productivity for security. With a single, unified cloud platform. Lookout simplifies and strengthens reimagining security for the world that will be today. It's happening right now. You can't live 10 years ago. You can't. You can't live in the past. Visit lookoutcom today to learn how to safeguard your data, how to secure hybrid work, how to reduce IT complexity. That's lookoutcom, and I will offer the CEO of CDK a free consultation at lookoutcom. All right, thank you, steve, for letting us pause and thank you, lookout, for supporting the important work Steve does here. Let's move on.
1:05:51 - Steve Gibson
So a listener, knox North. He said I listened to the Entrust story with interest, even though professionally I use DigiCert, I figured I'd never encountered Entrust. But I went to https, colon slash, slash, irsgov and guess who issued their cert? No, now Knox's observation made me curious so I went over to the irsgov website to see for myself. First of all, sure enough, the irs has been purchasing its websites tls certificates from entrust. Presumably that end. But what caught my eye was exactly when it will end. The certificate that's presently being sent to any visiting web browser is displaying a not valid after date of October 26th of this year. Now we might expect Entrust to attempt to renew any certificates they can before the Halloween drop-dead date. And since the IRS's current certificate will need renewing before October 26th, it will be interesting to see whether they remain with Entrust, as they certainly could for another year, or whether a policy somewhere deep within the bureaucracy triggers a change. We'll see. We won't have long to wait because by Halloween they will be recertified. We just don't know who will sign their certificate. Jonathan said Hello Steve, their certificate. Jonathan said, hello Steve.
I found a connection from my iPhone to one of the polyfill-related domains, cdnstaticfileorg. There was one lookup in my next DNS logs on June 24th. Obviously, it would be difficult, if not impossible, to locate the source of the lookup on an iPhone. I look for information on how to respond to this potential compromise, but all I can find is information for site operators. You know, remove dependencies on polyfill. I see no other connections to the known indications of compromised domains in my logs. What would you recommend at this point to make sure I'm not hacked? I'm thinking of wiping and reinstalling the OS, a backup, or starting fresh. Thank you. And he says from an undisclosed location near Washington DC oh it was the president Okay.
So it's 100% true that we don't know what we don't know, and the reason the Polyfillio event was so significant was mostly how bad an attack could have been. Indications are again within what we know that, for whatever reason, fun Null chose to only use this immense power. They had to launch highly targeted and selective attacks against users of mobile devices who were selected by the make and model of the handset they were using, and only when visiting specific websites. Fun Null's missed opportunity is the massively large bullet that we appeared to have dodged. Fun Null may have imagined that their hack would never be discovered, so they may have been in no hurry to do more damage, and they likely figured that as long as they continued to deliver the proper polyfills to nearly everyone who asked, their deception would go unseen. So my point is based upon everything we know. My point is based upon everything we know. The actual likelihood that you, jonathan, or I or anyone would have ever been subjected to Fun Null's malicious code truly seems vanishingly small. You know, I'm an avid iPhone and iPad user and I haven't given it a second thought. I'm an avid iPhone and iPad user and I haven't given it a second thought. None of the forensic analysis that's been done after this was discovered has revealed any more than those very tightly targeted attacks. It may have only ever been a handful of users who got this malicious JavaScript. But also Jonathan asked what would you recommend at this point to make sure I'm not hacked? I'm thinking of wiping and reinstalling the OS, a backup or starting fresh.
There's no indication that the malicious JavaScript, even if the targeting happened to match with you and somewhere you went was exploiting a vulnerability in the platform you were using. So when we say it was malicious, we don't necessarily mean that it was exploiting a vulnerability. It's almost certain that even in those who were penetrated, nothing about their browser or OS was ever compromised. That's a whole different end. The attack would have just used JavaScript code running in the browser at that website to whatever, grab their login credentials or their browser's session cookie to impersonate them or something of similar value to the attackers, as I noted last week, since the browser was loading the polyfillio code in the browser's first-party context and giving it access to the browser's DOM, you know, the document, object model, the web page's guts that code could do whatever it wished, but probably almost certainly only within the bounds of what any JavaScript code could do. In other words, your browser and OS would not be damaged at all. Therefore, first of all, incredibly unlikely that you ever actually received any malicious JavaScript and even if you did, especially on an iOS device vanishingly small chance that a compromise was required. It just wasn't necessary in order to probably get what they wanted, which would have been login credentials or a session cookie, something like that.
Something like that, okay, from Bud in West Virginia. I want to share a longer than usual piece of thoughtful feedback from a listener think is a factually supported case for Microsoft clearly placing their own profit well ahead of the needs of the users of their Windows desktop. And while, yes, okay, maybe that's all obvious to all of us, the conclusion that he draws and what I think he predicts is worth looking at. So I'm going to share what Bud wrote, but first what Bud wrote and then I'll discuss it. So he says Hi, steve, I realize this is a bit long and tried the best I could for brevity. When I first heard about recall, I thought it could be a useful tool, but also expected it to be a mess. So far, I'd say that's accurate and after listening to your coverage about recall, I think it's going to be even worse than I originally thought. You've said multiple times recently that Microsoft has not shown malicious intent, but I believe that they have. Let's look at three Microsoft products and then I'll share my thoughts about recall and how it might be. What finally makes me switch everything I'm responsible for away from Microsoft products and services. Yes, I believe it's that bad.
First, let's look at Windows 10. I tend to be an early adopter, he writes, and Windows 10 was no exception for me. When it was released, I was working in a small IT services company with customers in small business, local, local government and home end users. As for win 10 upgrades, some people didn't want change and some couldn't change due to a dependency on something not supported by Windows 10. Microsoft's rollout of Windows 10 basically went like this first, hey, windows 10 is a free upgrade for seven or eight, then you haven't upgraded to 10, let's schedule it. And then I'm going to schedule the upgrade, unless you click in the fine print which, he says, tricked a bunch of users to upgrade. And finally, no notification, no choice. Some users went to bed with Windows 7 or 8, and on their computer they woke up to have Windows 10. He says OK.
Next, let's look at OneDrive. Last year, microsoft started asking users to back up their desktop and other folders to OneDrive. Backup their desktop and other folders to OneDrive. Then, after saying no, some users found that when trying to delete something from their desktop, they'd get a message stating that items deleted from OneDrive could be recovered. Microsoft has now started asking in Windows 11 initial setup, but then turning on folder backup, even if the user selects not to. And I'll just note that I've been listening to Paul Theriot lamenting this more recent behavior of Microsoft's, which has also been driving him crazy. Microsoft is indeed ignoring these settings, even when they're arguably privacy-oriented and should be entirely within the user's control, but, as Paul keeps saying, they're just ignoring him. So Bud continues.
And finally, the kludge, wonderful word that is Microsoft Edge. Microsoft Edge, he says. Parens chromium started off as a great browser. He says I used it for a few years, but now it's so bad that I'll use literally anything but Edge. There's too much content here to choose from, so I'll just choose the latest that has impacted me. He says I've worked in DevOps for several years and redeploy Windows VMs, often for several years. And redeploy Windows VMs often.
The startup screens for Edge have, over the past couple of years, gone from please sign in to your Microsoft account to this infuriating mess. And he has four points Sign in to your Microsoft account, then let's sign in to your Google account to pull in that data. Then we're scheduling to pull data from other browsers on a regular basis, which he says is enabled by default. And finally, let's make your experience better for you. And he says really meaning better for Microsoft to track you and target you with ads. And he says, finally, the coup de grace, after already turning off the let's make your experience better setting. Opening the browser sometime later will open a small notification that Microsoft has made your experience better anyway. And if you don't like it, go to settings and change it again. He says, every time I get that notification, I'm already typing and some key hits. Ok, if you aren't paying attention, you could easily miss it.
So how does all this come together and apply? To recall, microsoft has clearly demonstrated that they can and will pressure, trick, countermand and or silently change settings to what will benefit them. And Microsoft has heavily invested in AI and needs some return on that AI investment. He says I don't think they're just setting the stage for an ad-supported version of Windows. They are going to want all Windows systems to have recall enabled so they can have hundreds of millions of computers that can be targeted for advertisements, advertising and everything the user does. Not only web browsing activity will be monitored, it will be everything they do.
Microsoft needs that AI return on investment and this is how I think their deployment will likely go. First, hey, windows recall is ready and you should turn it on now. Then I'm going to go ahead and enable Recall. You can disable it in Settings. And finally, windows Recall now works on non-Copilot Plus PCs. Let's enable it now, he says. And when all that isn't enough, Microsoft will just silently enable it anyway and people won't know until search exclaims your search is now enhanced by AI and Windows Recall, he finishes with.
I sincerely hope Microsoft does not take this path, but given their track record as I've outlined it above, I think it's all too likely. He says I'm in the market to replace an old laptop and would love to get an Elite X-based system, but I'm waiting until Linux is an option for it and for AMD's next-gen system to be released, because I simply do not trust Microsoft to be content with leaving recall disabled. They have an established history of breaking the workarounds and he says looking at you, edge, I'd like to hit. So he says to me I'd like to hear your thoughts on this review of recall and Microsoft's intentions, given their history. Thanks again for all you and Leo and the other hosts do to provide great shows every week, signed Bud, and I have to say it's difficult to argue with Bud's assessment. As I said at the start of this, he makes a strong, evidence-based case for what Microsoft seems very likely to do with recall in the future. One thing we do know is that it's been very clear from everything they've said that they are very determined to push recall onto the desktop, which really does beg the question why what's in it for them? Which really does beg the question why what's in it for them? Why is Microsoft so anxious to push everyone into using recall if it's just to give us better search? That doesn't really track. I've heard from other listeners whose opinions are more aligned with Bud's in this regard, so I wanted to share Bud's well-reasoned perspective and I will reiterate that should or when this comes to pass, I will make a definitive recall blocker available as a piece of lightweight GRC assembly language freeware See Assembly Language Freeware.
And one last note Thomas Tomchak said I have a disproportionate amount of joy for you having a newsletter via email. Thank you for putting the work in to make it happen and to do so on your own terms. And so, thomas, I just should say I just chose one quote. I'm getting constant approbation from our listeners, who are just delighted to receive GRC's weekly emailing in advance of this podcast. I did have, for what it's worth, a problem this week, which was interesting. The emailing contains a thumbnail of the picture of the week which, as we know, was that crazy bathroom door lock issue issue. About 80 or so of our recipients had the email bounce, claiming that it contained a virus. Apparently, clam AV had a false positive match on the binary of the JPEG and thought it was malicious.
And so 80 of our listeners although that's 80 out of currently 6,471 who have signed up for the week's newsletter Conveniently almost exactly one-eighths One Yep eights, one, yep, um, so about, uh, anyways, uh, I just wanted to say for those of you who did not receive it, though you are signed up, it's because your email provider is using a, an anti-virus system, virus system Clam AV, which identified the thumbnail as being a virus, so it bounced. I had like two last week and 80, something like 80 this week. So, and I'm using the same template, so like nothing changed except a different picture, and the only reason I could see that you know any that AV would trigger would be when you look at the body of the email. It is a binary blob, because that's the JPEG thumbnail and you know I could omit it, but it's fun to have that picture in email and other people get to have pictures in email, so why not me? I'm just going to chalk it up to you know, so why not me?
I'm just going to chalk it up to you know, hopefully a rare occurrence, one 80th of course You'll get email.
1:24:56 - Leo Laporte
Yeah, one, 80th of your entire group. Right, let us break. And then the meat of the matter the snowflakes chance. We'll talk about the snowflake breach. Who's at fault for that, after all, the AT&T breach? But first a word from our sponsor, delete Me A sponsor we know well because we started using it when our CEO quote started emailing her direct reports saying oh, could you please buy some Amazon gift cards and send them to this address? We need to get this done and I'm in a meeting right now. Of course, our team is smart. They didn't fall for it. But what scared us was the knowledge that the bad guys were able to figure out our CEO's cell phone number, her direct reports and their phone numbers so they could craft that phishing scam. And actually that's what's really interesting, why you really need Delete Me, especially as a business.
All that information that data brokers have can be used to manipulate you, to trick you, to defraud you. Have you ever searched for your name online? Don't. If you haven't, just save yourself the angst. You won't believe how much of your personal information is available. And if you don't believe what I'm saying, go ahead and do it, you'll see. You'll see.
Maintaining privacy. It's not just a personal concern, it's a family affair. Delete Me has family plans that make sure everyone in the family feels safe online. So Delete Me helps reduce risk from identity theft, from cybersecurity threats, as I mentioned, from harassment and from more, and it really works. Our subscription to Deleteme got Lisa's name off of all those brokers' lists. Deleteme's experts will find and remove your information from hundreds of data brokers. They will go to each and every one, remove your information and then continue to scan and remove your information regularly Addresses, photos, emails, relatives, phone numbers, social media, property values and more. And if you're getting the family plan, there are easy-to-use controls, which means the account owner can manage privacy settings for the whole family. Protect yourself, reclaim your privacy. Family. Protect yourself. Reclaim your privacy and, frankly, avoid getting scammed by going to joindelete mecom slash twit. If you use the offer code twit, you'll get 20% off. That's a great deal. Joindelete mecom slash twit the offer code TWIT for 20% off. Okay, steve, let's talk about Snowflake.
1:27:36 - Steve Gibson
Okay. So there's undeniable logic in the proposition that a third-party organization specializing in some aspect of business operations can, within a limited sphere, do a better job and a more cost-effective job than a company whose business is not doing that, you know. So the idea of farming out to a subcontractor some chunk of work becomes appealing when that's not your business's main focus. You know like, for example, when a building is being built, you use a subcontractor who specializes in laying foundations to do that work. You don't ask your painter to do that and the commercial plumbers install the plumbing and the HVAC guys run the air ducting and install the equipment on the roof and so on. So from a theoretical standpoint, the model is sound. It can and has gone wrong. Of course, if a contractor is discovered to be doing substandard work, it's certainly prudent to go back and look at the previous buildings they worked on to determine whether those might also have been impacted. Now, as we know, today, the cloud is all the rage.
I told the story of participating in a DigiCert customer summit seven years ago where all of the other techies looked at me like I had two heads when I casually mentioned my rack of servers at level three, one of them saying to me Steve, no one does hardware anymore. No one does hardware anymore, right? What's been happening for at least the past several decades or more is that a few nerds who know each other will get together over some pizza to discuss ways to make a bazillion dollars. The framework of their idea is nothing new create a business plan and present it to some venture capitalists in order to obtain seed capital and form a classic startup. Work 24 7 to create some Everyone needs then start it running, watch it grow, create demand, then either take it public or sell it off to a much bigger fish. The venture capitalists are happy, the co-founders are rich and everyone wins. So in a world where I'm told that no one does hardware anymore, it was only natural for those nerds to turn their attention to offering various sorts of cloud services, and the model there is more intoxicating than anywhere else, since not only do their future customers not want to do hardware, neither do they and they don't need to, since massive data centers already exist where doing hardware is all they do. Again, another example of increasing specialization. So these nerds write a bunch of code to do whatever it is they think companies will not be able to live without. Once they see what their new service is capable of doing for them, they rent some servers, spin up a bunch of virtual machines, launch their website, make an offer for trying it for free before committing, and start looking for and signing up new customers.
Everything I've just shared before I went to Wikipedia to see what Wikipedia had specifically to say about Snowflake. I promise that I really did write all of that with zero specific knowledge of Snowflake. So here's what the start of Wikipedia's page on Snowflake says. Wikipedia writes Snowflake Inc is an American cloud computing-based data cloud company based in Bozeman, montana. It was founded in July of 2012 and was publicly launched in October 2014 after two years in stealth mode. The firm offers a cloud-based data storage and analytics service, generally termed data as a service. It allows corporate users to store and analyze data using cloud-based hardware and software. The Snowflake service's main features are separation of storage and compute, on-the-fly scalable compute, data sharing, data cloning and support for third-party tools. It has run on Amazon Web Services since 2014,. On Microsoft Azure since 2018, and on the Google Cloud Platform since 2019. The company was ranked first on the Forbes Cloud 100 in 2019. The company's initial public offering raised $3.4 billion in September of 2020, one of the largest software IPOs in history, writes Wikipedia.
Snowflake Inc was founded in July 2012 in San Mateo, california, by three data warehousing experts two who previously worked as data architects at Oracle Corporation and the third a co-founder of a venture startup, vectorwise. The company's first CEO was Mike Spicer, a venture capitalist at Sutter Hill Ventures. So pretty much exactly what I said is the way this all happens these days. So the point I can now make from what was my first blind writing is without any specific knowledge of Snowflake is that, indeed, this is the way today's cloud-based service ventures are being born and, as Wikipedia's details have shown us, in this case, the founding three were absolutely correct about the need for, and the appeal for, their service. Since we're going to be talking about what happened in a minute, it's worth getting a little more specific information about this company. So Wikipedia continues from where I had left off.
In June 2014, the company appointed former Microsoft executive Bob Muglia as CEO. In October 2014, it raised $26 million and came out of stealth mode being used by 80 organizations. In June of 2015, the company raised an additional $45 million and launched its first product, its cloud data warehouse, to the public. It raised another $100 million in April 2017. In January 2018, the company announced a $263 million financing round at a $1.5 billion valuation, making it a unicorn. For those who don't know, a unicorn is a startup company valued at over $1 billion which is still privately owned and not listed on any market. Wikipedia says in October 2018, it raised another $450 million in a round led by Sequoia Capital, raising its valuation to $3.5 billion. In May of 2019, frank Schlutman, the retired former CEO of ServiceNow, joined Snowflake as its CEO, and Michael Scarpelli, the former CFO of ServiceNow, joined the company as CFO. In June 2019, the company launched Snowflake Data Exchange. In September 2019, it was ranked first on LinkedIn's 2019 US list of top startups.
On February 7, 2020, the company raised another $479 million. At that time, it had 3,400 active customers. Okay, four and a half years ago, 3,400 active customers. On September 16th 2020, snowflake became a public company via an initial public offering. Became a public company via an initial public offering, raising $3.4 billion, one of the largest software IPOs and the largest to double on its first day of trading. So four and a half years ago, back in February 2020, snowflake had 3,400 active customers and the sky's the limit. Everything looks great. We can presume that four and a half years later, that number has only grown.
So I wanted to start by painting that generic picture of the relatively new phenomenon of an entirely cloud-basedves of data. Much of it sensitive, suggests that we're not yet fully equipped to deal with the consequences of this new and essentially virtual cloud-based industry. A ton of information about what can now only be described as an historic data breach exists on the Internet, so I've spent a great deal of time following the links and reading original sources in an attempt to make sense of what happened. I think I finally have it worked out, and it's not quite the narrative that has taken hold throughout the industry, due to a bit of subtlety, as well as contracts and non-disclosure agreements.
Snowflake is blaming its customers for having their Snowflake login credentials used to log in to their Snowflake did not require any stronger login authentication, as it certainly could have. But it seems to me that the real question, which Snowflake appears to want to avoid answering by deflecting about multi-factor authentication, while its security contractors may be bound by agreements not to disclose, is how were many hundreds of its customers' login credentials obtained by these attackers in the first place? In the first place. The facts strongly suggest that something happened where, in short order, attackers obtained the login names and passwords belonging to a large number hundreds of Snowflakes customers where present. The attackers were apparently unable to obtain the account's MFA secrets, which is why MFA protected those customers who were using it. But somewhere around 350 of Snowflake's customers who were not using MFA suddenly found that all of the proprietary data they had shared with Snowflake had been exfiltrated to parts unknown. So whose fault was it? Was it Snowflake's customers for not extra protecting themselves from what appears to be a major, precipitating breach of authentication credentials at Snowflake? Or did Snowflake make some mistake themselves which, to be clear, they are denying strongly and that preceding breach allowed a large set of their customers' login credentials to fall into the hands of the bad guys? We know mistakes happen, that's a fact. But the narrative that's taken hold in the industry, which many articles quote Snowflake's spokespersons saying, is that the actual fault lies with Ticketmaster, with Advance Auto Parts, with Santander Bank, with LendingTree and now with AT&T, as well as apparently more than 340 others, for not using multi-factor authentication. That's a nice sleight of hand on Snowflake's part, but I'm not sure it's fair.
Security researcher Kevin Beaumont often summarizes things with more technical detail than other publications. In this case, back toward the beginning of June, under his headline Snowflake at Center of World's Largest Data Bre breach, kevin posted on Medium Cloud AI data platform. Snowflake are having a bad month due to teenage threat actors and cybersecurity of its own customers and its own cybersecurity too. In terms of optics, there are several large data breaches playing out in the media currently. For example, he writes Ticketmaster, owner of Live Nation, filed an 8K with the SEC for potentially the largest data breach ever, claimed to be 560 million customers. They finger Snowflake as part of the data breach. Kevin cites Tex Crunch's article with the headline Live Nation confirms Ticketmaster was hacked. Says personal information stolen in data breach, stolen in data breach. Then Kevin says additionally, incidents are running at multiple other cyber companies who are Snowflake customers where full databases have been taken. He says I've spoken to people in multiple industries at large corporations where they've had significant data exfiltration in May via Snowflake. The Australian security services have issued an advisory high alert act quickly. They say they are quote aware of successful compromises of several companies using Snowflake environments. Unquote. He says Snowflake themselves have put out indicators of compromise for threat activity over the weekend saying to look for connections into their platform from the user agent Rapeflake. Additionally, a threat actor claims they gained access to Snowflake itself and their customers using InfoStealers.
Okay, so let's pause here, because what happened has been interesting. The security research firm Hudson Rock first told the story of the penetration of Snowflake but quickly received a takedown order from Snowflake's legal beagles. You know the quote. We're going to sue you if you don't stop saying this unquote. So Hudson Rock complied and the industry was then forced to reference the Internet Archive's Wayback Machine record of their write-up until it was hit with a similar order requiring it to block that URL from access. So not really a good look for Snowflake. What Hudson Rock had to say was interesting, so we'll circle back to that in a minute.
Referring to what Kevin read in Hudson Rock's piece he wrote, the threat actor makes various claims which sound questionable, but well, snowflake have confirmed some of it is true, while crowing to the media and customers that the blog is not true. It is Schrodinger's blog. The threat actors here, from what I've managed to establish, from what I've managed to establish, were a teen crimeware group who've been publicly active on Telegram for a while, you know thus Rape Flake or whatever it was that they called their agent. Was it Rape Flake? Yeah, rape Flake. As you know, snowflake, rape Flake. So okay, in other words, you know, a bunch of kids did all of this damage.
When Kevin writes, let's recap. He says we have what appears to be the world's biggest data breach in terms of impacted individuals playing out, with Snowflake as the vendor linking the victims. A lot of data has gone. Walkies Snowflake, for those who don't know, is an AI data platform where you shove vast amounts of data in and then use it. It allows you to do this with effectively no security. He says I feel bad for Snowflake on a human level, as they're in a bad situation. This is a potentially business-ending event for them, so they have to use every lever possible to point the fingers at their own customers as being negligent over quote rape flake, unquote activity to avoid responsibility and, to be clear, some of this is their customers' responsibility, but also Snowflake have to own this issue and face straight into it to survive, as there's an extremely high chance this is going to play out publicly over coming weeks and months, and boy was Kevin prescient about that one. He wrote this more than a month before the AT&T breach announcement.
Then he writes and this is so perfect. He says Note that in the age of SAAS, software as a service, your providers will throw you under the bus to save themselves. When you transfer your security risk to a provider, they don't accept your risk, they just take your money. He says what you're sold versus what you get often don't align. He says I've worked for a cloud provider. You don't want to see how the sausage is made and there's no real accountability for the provider. There will be much more of this to come with cloud data providers in the future, is what I'm saying.
So what actually happened? Despite Snowflake saying the Hudson Rock blog is inaccurate, and he says and parts most probably are the Snowflake credentials bit is accurate. Snowflake say quote we did find evidence that a threat actor obtained personal credentials to and accessed demo accounts belonging to a former Snowflake employee. It did not contain sensitive data. Demo accounts are not connected to Snowflake's corporate and production systems. And I'm just reminded didn't LastPass go on and on about how safe everything was because their development systems were completely isolated from their production and corporate systems? That sounds like a familiar tune we've heard before Whoops.
He says Snowflake have incident response stood up with CrowdStrike and Mandiant involved. They say the cause of the malicious activity, in other words database downloads, is, quote this appears to be a targeted campaign directed at users with single-factor authentication. As part of this campaign, threat actors have leveraged credentials previously purchased or obtained through info-stealing malware. Okay, so to me this is curious, since they're not saying from where a huge number of their customers' single-factor authentication credentials may have been info-stolen. There's only one place in the entire world where all of those otherwise completely independent customer credentials would all be gathered into one place. I wonder where that could be.
As Kevin wrote, in the age of SAAS, your providers will throw you under the bus to save themselves. Your providers will throw you under the bus to save themselves. Okay, so as he says, so what happens essentially is info stealers were used to gain access to snowflake databases using their customers stolen credentials using the cluster. The client named rape flake and then he said side note to to Threat Actor over that name. Really, anyway, he finishes. Snowflake themselves fell into this trap by both not using multi-factor authentication on their demo environment and failing to disable an ex-employee's access. Stuff happens, incidents happen, and while Snowflake may present themselves as having no platform breach. They themselves also fell into the same problem and in terms of optics, it isn't great. As they can point out, customers messed up, but then they messed up too, but then they messed up too. So he then went on a little bit of a digression about what he feels is a hugely important topic of InfoStealers and since it's Kevin, I'm going to share that. He said you may know about InfoStealers as I recently wrote about them being a huge threat when it comes to Microsoft Copilot Plus, recall allowing full data threat of everything you've ever viewed, a feature you should absolutely disable in Windows 11. Then he says Mandiant themselves have this to say about InfoStealers this weekend. And then I've got I grabbed a picture of Kevin's snap from Mandiant's site where they write here are some of he says here are some of Mandiant's observations related to InfoStealers from the past few years.
Since the beginning of 2020, employees and contractors working from home increasingly use their personal computers to access corporate systems. People often synchronize their web browsers on their work computers and personal computers. People, or their children, sometimes inadvertently install software laced with info-stealing malware on their personal computers. The malware can capture credentials from their web browsers. Threat actors opportunistically search for corporate credentials stolen by info-stealing malware to use them to compromise enterprises, steal data and conduct extortion. In other words, although it's not a straight line, it's a series of interconnections. A worker at home is using his personal machine on the enterprise network. The personal machine is synchronizing browsers with the enterprise network. At the other end, someone may install info-stealing malware on the home machine. That home machine can steal the local credentials from the browser which, being synchronized with the with the enterprise browser, gets the enterprise credentials and then the info stealing machine on the home computer is able to whisk those off somewhere.
So Kevin says, if you use snowflake, you need to first of all. So Kevin says you cannot rely on Snowflake doing this for you. He says InfoStealers are a significant problem. It's long since outpaced botnets and so forth in the real world and the only real solution is robust multi-factor authentication and, ideally, getting rid of passwords altogether by replacing them with secure authentication, in other words, pass keys. He says there are companies offering services where you can buy your own stolen credentials back Whoa, and then you can change users' passwords. He says I don't like this approach. The reason is those vendors often buy those credentials from credential brokers, which translates to funding the criminal hackers who steal them in the first place. As a customer, you end up proxy funding the threat actors you're trying to deal with. Additionally, it is a huge user impact to have their password changed and it doesn't fix the problem. He says tightening authentication fixes the problem. Ask the Snowflake victims how they have fixed the problem. It's through robust multi-factor authentication.
The wider problem is that something is wrong at Snowflake when it comes to authentication. Snowflake themselves fell victim to this incident, albeit with a demo tenant. They need to, at an engineering and secure by design level, go back and review how authentication works, as it's pretty transparent that, given the number of victims and the scale of the breach, that the status quo has not worked. Secure authentication should not be optional and they've got to be completely transparent about steps they're taking off the back of this incident to strengthen things For cloud providers in general. They need to be more robust in terms of secure defaults or risk being dragged into this kind of situation. For Microsoft, he just couldn't resist finishing with this. For Microsoft, they need to recall, recall or they will pour petrol onto the flames and make the InfoStealer problem far worse. And after Kevin posted this piece, they will pour petrol onto the flames and make the info stealer problem far worse.
And after Kevin posted this piece, he put he added a quick followup. He said people are pinging me to say there's more to this story than I've disclosed. And he says I know it will be a developing story and all eyes are on Snowflake so maybe, for example, there was some knowledge of the AT&T problem. So anyway, you know Snowflake insists that it didn't happen, that they were breached. Maybe that's true. Seems suspicious to me, since all of a sudden, someone got a hold of 350 plus of their customers non-mfa single factor authentication, um, uh login data and used it to breach their technologies. Uh, it'll be interesting to see if anything more happens.
And you know how Snowflake fares. They are certainly a huge cloud provider. It's also interesting and here a little egg is on AT&T. The data that was stolen from AT&T was apparently two years old. Was apparently two years old, that is, this was back from 2022. At&t left old 110 million customers' worth of individual transaction data at Snowflake from two years back and could have pressed a button to delete it, but didn't. So you know plenty of blame to go around. But wow, we now have broken all records in terms of amount of data lost in a massive breach.
1:58:14 - Leo Laporte
So do you think that the credentials of those 350 companies were revealed or that there is something more likely? There's a flaw in the authentication process, right that, whatever it is, it takes in the password and then lets the person in Wasn't working properly. There was a hole in it. We don't know, do we?
1:58:36 - Steve Gibson
I don't think I, the everybody who is surrounding this thinks something is fishy. Yeah, um, the, the, what snowflake. This is hard to keep saying that name, what a name.
1:58:52 - Leo Laporte
Maybe they'll change it they thought it was funny at the time. You know, hey, we're a unicorn snowflake huh.
1:58:58 - Steve Gibson
Um, what they're. What they're saying is that their customers were the. Their customers were the victims of infooStealers which found Snowflake credentials on their customers' computers and then, because those customers weren't using multi-factor authentication, just finding static authentication username and password allowed them to log into Snowflake?
If that's the case, then why not? Everything else I mean, and why all of a sudden you know 340, 350 customers? It seems much more likely that Snowflake was infiltrated and the database of those you know that authentication was exfiltrated and then used because those customers did not have multi-factor authentication.
2:00:01 - Leo Laporte
That was all they needed.
2:00:02 - Steve Gibson
It was used to log in as them and grab all their data. We just don't know.
2:00:07 - Leo Laporte
We don't know. It's possible that the hacking group was actually looking for Snowflake credentials on all those customer computers.
2:00:16 - Steve Gibson
Yes, it absolutely is possible.
2:00:18 - Leo Laporte
You know, they knew they were targeting Snowflake and they were looking for that kind of credentials. That's why it was all Snowflake. I mean, it's unknown. Somebody needs to come forward and say what happened, but Snowflake probably not.
2:00:30 - Steve Gibson
Well, actually the record, the publication of the record, said this. They said, according to the original post, the intruders were able to sign into a Snowflake employees ServiceNow account using stolen credentials and from there were able to generate session tokens. Hudson rock wrote to put it bluntly, a single credential resulted in the exfiltration of potentially hundreds of companies that stored their data using snowflake, with the threat actor himself suggesting 400 companies were impacted. In a post on Friday, snowflake did not respond directly to the researchers' claims, but denied that a vulnerability within its systems was to blame for the accessing of customer data. The company said it is investigating an increase in cyber threat activity.
2:01:39 - Leo Laporte
A large increase.
2:01:40 - Steve Gibson
Uh-huh. So, basically, hudson Rock posted this claim by the threat actor themselves and they received a legal takedown notice. Oh, and they received a legal takedown notice. And then the web archive was similarly forced to block their archive link.
2:02:01 - Leo Laporte
Well, that's, that's max of cover up. Yes, I mean, I guess you could say for security reasons, we don't want anybody to know what, how they did this, or PR reasons. Or PR reasons, wow, I hope we get to the bottom of this at some point. I'm sure if we do, you will let us know. Absolutely what a great story, great in an interesting way. Not a good way for anybody involved. Thank you, steve Gibson, once again elucidating the dark corners of the internet. Security Now must listen.
Every Tuesday we start around well right after MacBreak Weekly. So that varies, but around 1.30 Pacific, 4.30 Eastern, 20.30 UTC. As I mentioned, we are streaming live everywhere YouTube, twitch, facebook, linkedin, x. I'm sure I've forgotten Kik. I'm sure I've forgotten Kik. I'm sure I've forgotten some. So if you have a favorite site that has live streaming, look for Twit, and when we're doing a live show, you'll be able to watch it, which we love. If you like what you hear, if you're new to the network or maybe you're a longtime listener but not yet a club member, please consider joining the club. That makes all the difference in our ability to stick around. You may have noticed there's some vital stuff missing from the set behind me, I've had to start to sell all my memorabilia one by one. No, I'm bringing some home. This is the stuff that's going to stay. That would be sad, though. Please, would you like to buy a podcast award from 2014? Please, only $5. It's like selling your Oscar, what else? So watch us live, subscribe, twittv slash Club, twit Lots of benefits.
It's all there. It's all laid out there. You can pay as little as seven dollars a month, or as much as you want. It really helps and we really appreciate. You'll be in a great community of people too. You can also but we still offer free versions of the show. I mean, this is a public benefit. We've got to put this out. So you can find it on the web.
There's a YouTube channel. There's twittv slash sn, or you can get it from the man himself. Just go to grccom. That's Steve's home on the web. You can get the podcast. He has the traditional 64-kilobit audio version as an MP3. He also has a 16-kilobit MP3 for people with scratchy ears. He also has transcripts written by a real live human being, elaine Ferris, and they're great. So you can read along as you listen, or you can use them to search All that is at GRCcom. While you're there, you might want to check out. A copy of Spinrite Version 6.1 is out. It's new, it's fresh, it's better than ever the world's finest mass storage, performance, improver maintenance utility and recovery tool Spinrite. And it's also Steve's bread and butter at GRCcom. There's lots of other stuff there. Steve's got a lot of free utilities. There's the forums and if you'd like to email Steve or subscribe to his newsletters, go to GRccom, slash email and, uh, get validated. Uh, steve's, uh also. Uh, are you? Should we say you're on x, or is that? Is that kind of fading into the, into the past?
2:05:26 - Steve Gibson
I'm reminding people who send me things that, uh, email is my preferred conduit, and I'm hearing from so many listeners who refused to do X. I don't blame them, it's really a net positive. So I am posting the notes every week to anyone who follows me on X.
2:05:47 - Leo Laporte
Yes, Ironically, on Twitch a guy named 60 frames per second says I only listen to the 16 kilobit version. When it comes to video, he wants 60 FPS. But if it's audio, 16 kilobits, that's just fine. That's just fine. We have it at our website, twittv, slash SN. We have video too there and you can subscribe, if you want, to the video or the audio version. What else is there to say? But thank you for being here. Make sure you come back next Tuesday. It will be not quite the penultimate edition of Security Now from the studio. The last one will be in August. August 6th, I think, is the last one.
2:06:34 - Steve Gibson
Thank you, steve Gibson. And then, a couple of weeks later, we'll be having our birthday, wow 19 years going on to 20. August 20th we will be how exciting it's, one day past the August 19th of the 2005 first podcast.
2:06:51 - Leo Laporte
It's been a lot of fun. Getting there, yep, and that one will be from our new TwitAddict studio, so we'll see you there. Thank you, steve.
2:06:59 - Steve Gibson
Bye, next week Bye.