A couple SNAFUs conspired to delay this morning's package of fun. We apologize for the delay, and hope to have something posted before too long.
This was one part of the problem:
CLOUDFLARE OUTAGE CAUSES ERROR MESSAGES ACROSS THE INTERNET
US company that defends millions of websites against malicious attacks suffers unidentified problem
by Robert Booth
A key piece of the internet’s usually hidden infrastructure suffered a global outage on Tuesday, causing error messages to flash up across websites.
Cloudflare, a US company whose services include defending millions of websites against malicious attacks, experienced an unidentified problem, which meant internet users could not access some of its customers’ websites.
Some site owners could not access their performance dashboards. Sites including X and OpenAI suffered increased outages at the same time as Cloudflare’s problems, according to Downdetector.
The outage is ongoing but as of 12.21pm GMT, the company said: “We are seeing services recover, but customers may continue to observe higher-than-normal error rates as we continue remediation efforts.”
A further message said: “Update: we are continuing to investigate this issue.”
A spokesperson for Cloudflare said: “We saw a spike in unusual traffic to one of Cloudflare’s services beginning at 11.20am. That caused some traffic passing through Cloudflare’s network to experience errors. While most traffic for most services continued to flow as normal, there were elevated errors across multiple Cloudflare services.
“We do not yet know the cause of the spike in unusual traffic. We are all hands on deck to make sure all traffic is served without errors. After that, we will turn our attention to investigating the cause of the unusual spike in traffic.”
Cloudflare’s engineers had been scheduled to carry out maintenance on Tuesday on datacentres in Tahiti, Los Angeles, Atlanta and Santiago in Chile, but it is not clear if their activities were related to the outage.
As it tries to fix the problem it disabled an encryption service called Warp in London and said: “Users in London trying to access the internet via Warp will see a failure to connect.”
Cloudflare was described as “the biggest company you’ve never heard of” by Prof Alan Woodward of the Surrey Centre for Cyber Security. The company says it provides services to “protect your websites, apps, APIs, and AI workloads while accelerating performance”.
Woodward described it as a “gatekeeper” and said its roles included monitoring traffic to sites to defend them against distributed denial of service attacks when malicious actors try to overwhelm sites with requests. It also checks users are human.
The problems at Cloudflare come less than a month after an outage of Amazon Web Services brought down thousands of sites.
“We’re seeing how few of these companies there are in the infrastructure of the internet, so that when one of them fails it becomes really obvious quickly,” Woodward said.
While the cause remains unclear, Woodward said it was unlikely to be a cyber-attack as a service so large was unlikely to have a single point of failure.
(theguardian.com)
Nickname: Where Did “Zukini” Come From? (Now it can be told…)
It’s an underwhelming story (but I guess I just made it into an epic):
Back in the 70’s as a kid in Indiana reading underground newspapers, and even with a subscription to the Berkeley Tribe, I noticed in The Chicago Seed for example (I had read the Indianapolis Free Press once at a Unitarian youth group weekend, found an address, and ordered a big selection of underground newspapers, including the San Francisco Oracle, The Great Speckled Bird from the south somewhere (Atlanta), Northwest Passage from Bellingham, and so on as I became further radicalized and ripe for something, but what?) that people had their made up names, hippie names as it were and I wondered what could mine be?
Well, I thought, I’m into food, um, what’s some food items, umm, zucchini, okay Zukini, I’ll be Zukini! I came west, left the Zukini behind, then when I serrandipitously ended up at The Big House there was another Paul there, that odd bird Paul Carlson, and so I started up the Zukini shuffle again.
(I had run into my sister at a food conspiracy store in Tucson after taking mushrooms in Palenque and hitchhiking seven days straight to Arizona, she ended up in a three-sided cabin at Nooning Creek, me crashing at the old school house in Whitethorn, and I was not feeling it in the middle of the winter of ‘73 and was thinking about heading back to Indiana when I saw a sign up at the Whitethorn post office for a Walk For Edible Plants down the Yellow Dirt Road.
Ray Raphael was doing it with Timothy Clark and others, I met the big house EST freaks there, got invited back for dinner that night, the classic move, and stayed for six weeks working in their nursery school. (Remember when the kid got lost?)
Anyway, that’s the cliff notes from my epic story Wandering, 7000 words about 1973 when I was nineteen and traveling from Indiana to the Gulch to Mexico and ended up in New York for parts of a couple years where I drove taxi among other things.
So Zukini, I tried to discourage it after a while but do feel a faint good vibe when I hear it, usually as Paul Zukini, though when I was pitching for the softball team they once hung an oversized zucchini behind the backstop and chanted Nuke the Zuke.
(Other variations, Zukananda…)
[email protected]
A primary reason that sites of every size, from the behemoth X right down to the venerable, albeit tiny, AVA use Cloudflare is to ameliorate the effects of DDoS (Distributed Denial of Service) attacks.
The principle of a DDoS attack is simple: Flood the target with phony requests so that it is unable to respond to legitimate requests in a timely fashion, effectively making the target inaccessible. These attacks are distributed across a network of hijacked devices, often IoT (Internet of Things) devices like security cameras, smart TVs, automatic vacuum cleaners, wireless range extenders, etc. These devices have in common an always-on internet connection and a large attack surface. A collection of hijacked devices is known as a botnet.
The attack surface of an IoT device is the sum of the flaws that the device can be targeted by a remote attacker. Preset passwords meant to be, but never changed by the user, weak passwords, zero day security flaws that are never patched, and manufacturer installed backdoors, are examples of flaws that make up an attack surface. Attacks are automated by widely available hacking tools, lowering the barrier of entry to assembling a botnet down to simply obtaining and running a tool. Devices that have become part of a botnet usually continue to fulfill their intended purpose, so the owner is almost always unaware of the compromise.
A site like the AVA would never be able to operate without Cloudflare. Without the protection from DDoS attacks that Cloudflare offers, a single person with an agenda could keep the AVA permanently offline with only a minor effort and little technical expertise.
It appears that Cloudflare’s failure this morning was due to an internal programming error that cascaded into a widespread failure. We can expect similar problems due to internal and external causes becoming a regular feature of the internet for the foreseeable future.
Of course the problem we saw earlier today is a consequence of having a sole provider of DDoS protection. The reason that we don’t have a number of providers is one of scale. Only an entity as large as Cloudflare has the capacity to absorb large DDoS attacks that can peak at several trillion bytes per second.