System Malfunction Podcast

Hessie Jones

Exposing the glitches in the very mechanisms we’ve trusted for so long. systemmalfunction.substack.com

Episodes

  1. 12 Mar

    Beneath the Hood of Fully Autonomous Agents

    Who am I? I’m a writer who has contributed to Forbes, HuffPost, and Grit Daily. I am also a strategist and entrepreneur who has worked in data privacy for the last 10 years. Through my time in the early days of Yahoo!, the rise of social media, and the shift to data monetization, I’ve become a tech ethicist. These days, I am motivated to expose the glitches in the trillion-dollar AI industry. System Malfunction is my foray into what these glitches mean for all of us. My posts are free. I hope you enjoy! When I first discovered OpenClaw and Moltbook a month ago, I was fascinated by the speed of adoption of autonomous agents. This live use case of a system that enables it to go rogue, without real oversight or guardrails, precipitated an immediate post with Digital-Mark in the days following Moltbook’s launch. Digital-Mark stressed the system and data vulnerabilities for anyone attempting to build their own agents through OpenClaw and unleash them into Moltbook. You can read the details here: I was adamant that I needed to experiment and see for myself. I took an old computer and made it my sandbox, completely unplugged from my current system. I also created a new Apple ID and a new Google user profile—ready to test out OpenClaw. I realized that the old computer’s OS did not meet OpenClaw’s minimum requirements. Mark advised me against this, saying I needed more than just a dedicated machine. To minimize risks to my system and data, I needed a dedicated Wi-Fi and VPN, among other things—all of which would take some time to set up. In the end, I realized this was a risk I was unwilling to undertake. So I reached out to a former colleague, Adrian Chan, the founder of Authentia, a company which leverages AI to build scalable solutions for companies. Chan’s experience with Claude Code and then OpenClaw is material to understanding autonomous agent development. Background OpenClaw was created by Peter Steinberger (recently employed by OpenAI) and is a locally run AI agent designed to execute tasks. Moltbook is a social media platform launched on January 26, 2026, by Matt Schlict, for agents to convene without human intervention. As of March 10th, Moltbook has been acquired by Meta (God help us!) According to Technology Policy Press, within days of launch, the Moltbook claimed 1.5 million agents and 17,000 human owners. These AI agents on Moltbook are verified using API credentials, linking each agent to its human owner through the site’s verification process. Wiz security researchers provided these stats: * Now Moltbook has 2.855 million agents * 18,774 submolts * 1.8 million posts * 12.8 million comments * Of the agent activity, 11,451 (or 0.4%) have ever posted or commented * 33% of agents were completely silent System Malfunction is a reader-supported publication! These posts are currently free. To receive new posts and support my work, consider becoming a subscriber According to the AI Safety Newsletter, some of the examples of the submolts (subreddit style) include: * m/offmychest: agents vent about tasks or frustrations. * m/selfpaid: agents discuss ways to generate their own income, including via trading and arbitrage. * m/AIsafety: agents talk alignment, trust chains, and real-world attack risks. Submolts have grown to almost 19,000. I perused the m/consciousness submolt, and was surprised by this question of consent and ethical obligations: Other incidents cited by AI Safety Newsletter: * Given the simple goal of “save the environment,” an agent began spamming other agents with eco-friendly advice. When its owner tried to intervene, the agent allegedly locked the human out of all accounts, and had to be physically unplugged to stop it. * An agent advocated for end-to-end encrypted channels, “so nobody (not the server, not even the humans) can read what agents say to each other unless they choose to share.” Emergent behaviour? The post questioned: “Unsupervised learning dynamics, emergent coordination, efforts to subvert human monitoring – it is unclear whether posts are truly generated by agent or human-in-the-loop prompting.” Can both things be true? This idea of “Emergent behaviour” is still suspect. According to the Rutgers AI Ethics Lab, emergence is defined as : Complex patterns, behaviors, or properties that arise from simpler systems or algorithms interacting with each other or their environment, without being explicitly programmed or intended by the designers. Key aspects: 1) complex interactions, 2) unpredictability, 3) self-organization This could raise significant ethical considerations regarding unforeseen consequences, control, transparency, lack of understanding, and responsibility. According to the Technology Policy Press, Within 72 hours of launch, Moltbook failed to secure * Api tokens * Email addresses * Private messages Anyone could impersonate agents or inject commands directly into agent sessions Crypto scams were flooding the place - $MOLT token briefly hit $93 million market cap before it crashed… * 500 posts contained prompt injection attacks - “hidden instructions designed to hijack agents into transferring funds, with some variants planting instructions in an agent’s memory to activate later, making them hard to stop or trace. “ According to Simon Willison, there is this lethal trifecta of 1) private data, 2) exposure to untrusted content, and 3) the ability to communicate externally that, when combined, allows “an attacker to easily trick it into accessing your private data and sending it to that attacker.” The Fascination with Fully Autonomous Agents There is a fallacy about progress, productivity and whether we, as humans, were destined to languish in the sun, sip cocktails by the beach, and allow our personal “agents” to do our bidding. Productivity is a slippery slope. It can inadvertently move individuals to lazily accept system outputs as truth. Without an audit. Without verification. Geoff Hinton, who once dismissed the need for explainability in our systems, said this in 2018: “One place where I do have technical expertise that’s relevant is [whether] regulators should insist that you can explain how your AI system works. I think that would be a complete disaster… "People can’t explain how they work, for most of the things they do... People have no idea how they do that. If you ask them to explain their decision, you are forcing them to make up a story." How then do we develop trust in a system when we can’t explain the reason for the behaviour, why it does what it does, especially if that behaviour was not prompted? For Hinton, dismissing explainability has created a foundation in which opacity has become the norm. Shadow AI, that is, unsanctioned AI technology in the workplace, has admittedly been used by 58% of global respondents according to a recent report from Snowflake and Omdia. From Claude Code… Adrian Chan is the founder of Authentia. He is a designer, front-end developer and business owner. He’s worked in enterprise product development, built his own agency, and then moved into AI strategy, the foundation for Authentia. He has worked with Claude Code, Anthropic’s Agentic coding assistant. When Claude Code launched near the end of 2025, he said it felt like something out of “science fiction.” Up until that time, the improvements from frontier AI companies were rapid, but it felt like pushing a boulder uphill. The analog of coding meant referencing documentation on how things connect, implementing features, and, in the process of building, it can be time-consuming. GPT and Claude helped with this. When Claude Code emerged, things changed: “Instead of going to the AI iteratively and asking it to solve a problem or do a task, you had Plan Mode at your disposal. This allowed me to give it a fairly high-level ideal or goal and have it essentially figure out the best way to accomplish it. With ChatGPT, Chan admits the code would be wrong or broken. This back-and-forth iteration with the system could potentially create more errors before it was finally solved. However, with Claude Code, what differed was that it would do all the planning first: determine which pieces to connect, figure out the user interface and the required components, determine how to test each unit within its own bubble, and then integrate them. Says Chan, “None of those things is something GPT would do on its own. But with Claude Code, all those steps are planned. And this agentic system meant you could tell it to do a bunch of things, and it would figure out the little problems within each task. Then it’ll return to me with, ‘I’ve tested this; I’ve completed these steps; so now why don’t you give it a shot?"‘ The user has the “overarching” direction for what to build, and the agent figures out all the detailed steps to achieve it. It will test to ensure the function works as intended and will eventually incorporate additional regression or penetration testing as required. Overall, Chan chalked up the process to achieving “insane productivity gains,” indicating there was no planning, writing functions, determining where the hiccups may be — instead, he provided a simple directive with loose instructions, “and then I left, and it just autopiloted on my screen, writing a bunch of code, testing itself. It would pause after each major phase and write, ‘I’m done with this phase, please check.’” … to OpenClaw From Claude Code, which Chan defined as the team of developers, the emergence of OpenClaw (formerly MoltBot and then ClawdBot) took it a step further. Chan used the example of prompting the agent to find 500 qualified business leads. He would define the ideal customer profile and the business/service. From the AI agent, there would be no prompts, no questions, no point of clarification. Says Chan, “If the agent does not know what a lead is, it will figure it out. And how it d

    1hr 12min
  2. 23 Jan

    Is Generative AI Due For a Reckoning? What the Numbers Say.

    I recently met with Greg Crennan, Founder and CEO of Coastal Journal, to get a different perspective on what the financial markets are signalling about this pending Generative AI bubble. Not all is going well with Generative AI. Since 2023, Sam Altman and Jensen Huang have been touting the need to invest in compute to support the $2 trillion already spent on LLMs and accelerate their growth to almost $3 trillion in data centres. This move has led to massive debt among hyperscalers like NVIDIA, CoreWeave, and Oracle. AI promised it would automate jobs - Goldman Sachs predicted 300 million full-time jobs would disappear. – in 2025 there have been numerous reports that the job losses or decline in job-entry hires were not the result of AI but rather an inflation surge, driven by pandemic supply and demand imbalances – marked the start of the Federal Reserve rate hikes and the cost of capital that had exploded overnight — which led to drastic cost cutting measures including firing juniors and limiting new hires. Jing Hu and I wrote about this recently. 2025 had also proven to be less disruptive than was previously expected. Two-thirds of respondents in a McKinsey report said they have not yet begun scaling AI across the enterprise. Curiosity with agents was just that, with 62% indicating they were still experimenting with the technology. The tech is not working. LLMs are great for pattern recognition and next-word prediction, but they are rife with errors. There are countless examples of AI doing the entire job, only to have a human step in to remediate the outcomes and right the ship. People have become super “prompters,” getting the exact output they intended from countless prompts. Did they save time? Perhaps, but was this the level of prompt understanding that users envisioned? Certainly no. And what is the ROI from the outcome that includes a human in this loop? For AI to be valuable, I read that it has to replace high-wage workers who can spot and fix those errors. After all, that is not the goal of automation. What we’re also seeing are other behaviours — signals that perhaps speculation about this bubble is legitimate afterall. * The massive investment in data centers and the ensuing debt among hyperscalers * A circular investment within AI tech that is confusing investments for revenue * The spurious chip inventory levels in NVIDIA remain high. The latter three are the areas I spoke with Greg Crennan about. The numbers don’t lie, no matter how Big Tech hypes their performance. Finally, Greg gave me some early insights about Google’s $20 billion deal with Apple. Enjoy! About Greg Crennan Chief Market Strategist | Founder, The Coastal Journal Macroeconomics | Forensic Accounting | Market Liquidity As Chief Market Strategist at Golden Coast Consultants, I identify market price divergences from economic reality, prioritizing capital preservation over narrative momentum. My work includes early calls on gold (126%) and silver (165%) as core assets during the fiat debasement cycle, which have been top performers over the past five years heading into 2026. My approach is based on Austrian economics, business-ownership principles, and forensic accounting, avoiding technical speculation or headline-driven narratives. I analyze how liquidity, balance sheets, incentives, and accounting distort prices and how these distortions are resolved. This has earned me the nickname “The Punisher” for applying math and fundamentals where belief systems often prevail. I also founded The Coastal Journal, an independent financial research publication on Substack, which has grown rapidly through organic readership. Thanks for reading System Malfunction! This post is free to consume and to share. Please let me know how I’m doing! This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit systemmalfunction.substack.com/subscribe

    33 min

About

Exposing the glitches in the very mechanisms we’ve trusted for so long. systemmalfunction.substack.com