Totally A Thing

Sarah Smith
  • 0.0 (0)
  • TECHNOLOGY
  • UPDATED WEEKLY

Apps, startups, tech and people. Stories & explainers that go to the heart of tech & culture from someone who’s been inside the tech sausage machine. totallyathing.substack.com

  1. FEB 12

    Emergent behaviour? Or stolen IP?

    The Attention is All You Need paper started the LLM gold rush, but it initially came from Google scientists who only expected it to do language translation. As more data was used in training sets, and the number of parameters grew orders of magnitude it was less easy to understand what the model was doing. Emergent Instruction Following In Ars Technica from 2023 we have this summary of how the “emergent” AI behaviour was being seen popularly, in the press: The biggest breakthrough came in the jump from GPT2 to GPT3 in 2020. GPT2 had about 1.5 billion parameters, which would easily fit in the memory of a consumer graphics card. GPT3 was 100 times bigger, with 175 billion parameters in its largest manifestation. GPT3 was much better than GPT2. It can write entire essays that are internally consistent and almost indistinguishable from human writing. But there was also a surprise. The OpenAI researchers discovered that in making the models bigger, they didn’t just get better at producing text. The models could learn entirely new behaviors simply by being shown new training data. In particular, the researchers discovered that GPT3 could be trained to follow instructions in plain English without having to explicitly design the model that way. Instead of training specific, individual models to summarize a paragraph or rewrite text in a specific style, you can use GPT-3 to do so simply by typing a request. You can type “summarize the following paragraph” into GPT3, and it will comply. You can tell it, “Rewrite this paragraph in the style of Ernest Hemingway,” and it will take a long, wordy block of text and strip it down to its essence. They are I believe talking about RLHF post-training when they say “trained to follow instructions in plain English” (my emphasis). Abuse of Academic Privileges for Corporate Gain But as I say in the video, this should not be surprising. Datasets like this exams one from 2022, are on the public internet for the taking. Even worse though is that via academic library access, researchers at OpenAI could get huge datasets from universities, and governments relating to education and training. Of course the claim was that it is all to further science. Fast forward a couple of years and Sam Altman has taken all that and privatised it for profit. A little bit of Transformer Architecture The absolutely excellent 3-blue-1-brown YouTuber Grant Sanderson has produced some stellar explainers on the architecture of Transformer LLMs. The important take-away from this diagram is that the “magical” attention layer is tuning weights based on queries into positionally-coded information across the training set being fed in. Even for big training runs. This means that the model gains a surface level structural mapping of how say a question and answer dialogue is laid out. In this screenshot we see inside the Multi-layer perceptron. There are many of these stacked in the transformer. But it’s important to understand a “perceptron” is just a very basic mathematical equation. The inputs to the equations are the current weights in the nodes (depicted as small spheres here) and the incoming data fed-forward through the layer, multiplying out to the final resulting tensor (shown as a “matrix” on the right in white square brackets). These are all floating point numbers. There is zero correlation with how a human brain works here — impressive as this is, a perceptron is not a human neurone. One of the biggest differences is that once the weights in the perceptron are set — the data from the training set is encoded, during the pre-training phase, and after in RLHF — it does not change. A GPT trained in 2022 cannot spontaneously learn things that happened in 2026. There are tons of hacks to make it seem as though they can but they don’t. When You have Very Very Large sets of Weights you have lots of Attention Encoded Very large data sets encoded into LLMs means attention processes are capturing juxtapositions of all sorts of documents, and inputs. Associations. Rules about what follows what, that can seem like an understanding. The important things to understand about this are: * There is no reasoning here, GPTs do not reason they predict next tokens * The encoding is lossy and sampling biases appear * Patterns can be captured automatically like math problem solving, or sycophantic chatting as if between good friends RLHF alignment is the process of using human feedback to coax the pre-trained GPT toward a different weighting that suits the goals of the LLM vendor building the GPT, while still leveraging the encoded information. Lossy Encoding - to try to explain this, look at an analogous process of an old-school convolutional neural network encoding (warning - this is not how LLMs work - the Perceptrons there are massive): This image is from Stanfords Deep Learning tutorial. It shows how feature extraction works as 3x3 convolution is mapped over the source image. This could extract edges, or other features that are then used in deeper layers of the CNN. The analogy I want to create is that when creating the initial embedding by automatically training from datasets, the LLMs are effectively running a sampling window over the entirety of the data. There are massive differences between an LLM and a CNN, but the point is the losses - when you capture a feature by definition you throw away the other information; if you have a much deeper layer that can detect faces or smiles; or guns, tanks and soldiers; then it cannot understand other aspects of the image. In an LLM its not being told what to ignore, but it is attending to the context by making queries across to other positions, and thus its averaging out the values in its weights to capture an encoding of the data set. These losses cannot be understated. An LLM is like that know-it-all person at a party who’s skimmed everything, but knows nothing in depth, and certainly doesn’t know anything beyond superficial “this appears after that” associations. Why Claims of Emergent Behaviour are Dangerous Vendors of AI SaaS products have been constantly engaged in drumming up mystique and wonder around their products. At times they’ve insinuated that they have no idea what their products are capable, of and they could “achieve AGI” (a marketing term for anything that makes us money) and end humanity. Wow — who wouldn’t want that behaviour to emerge. So “emergent behaviour” as a narrative is like declaring their LLM factory as a goldmine. A never-ending bonanza of free technology upgrades, that justifies more and more investment so that bigger and bigger models can be built. Who knows what exciting behaviour will emerge next, they enthuse. Regarding so-called emergent instruction following behaviour: * It required alignment training during the RLHF phase of model development * It required users to add in extra context during the model session * And its bounded by the datasets available (as I say in my video) If you don’t have instructions and text showing those instructions being followed in a document set that is fed into a training corpus, the model cannot learn that certain text follows from those instructions. Generative AI is a next token predictor engine. It maps what you prompt onto what should follow, based on its limited understanding. It may have additional context being fed into it as RAG, from a vector store — depending on what kind of LLM setup we are talking about. But believing that LLMs are somehow reasoning intelligently about their outputs is dangerously anthropomorphic. Its also corrupting the ethical responsibility of the vendors of these SaaS products as they can claim anything bad was “unpredictable” and “emergent” - they will go off and create yet more system prompts (a kind of ground level context added to every query) in the hope that the bad thing won’t happen again. Conclusion Fight bad framing. This notion of “emergent” behaviour is yet another marketing gimmick at this stage. Emergent behaviour was the idea that sudden big jumps in capability occur when you pump in more money and more data. It’s like a gambling addict whose eyes light up at the spinning dials of the one-armed bandit. But as further studies show this behaviour is incremental in all likelihood and in large part comes from context, and — as I argue above — directly from the data. Emergent behaviour feeds into the self-serving narratives of the generative AI vendor CEOs who want to have their investors and the public — via idiotic and corrupt billionaire welfare — keep bailing out their failing companies. And I call BS on that. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

    15 min

  2. JAN 27

    Pissed about Ads in OpenAI? Wait'll you hear what they want from our Government

    Chris Lehane is making our Governments Work for creepy AI companies This screenshot from a recent Karen Hao / A More Perfect Union video shows unintentionally what is going on: Lehane is like a disinformation grenade lobbed out of OpenAI at government. His job is not to make better AI. He is a PR guy, a spin merchant. His job is to create the second leg of these failing AI company’s financial strategy: Government. Your tax dollars. Zero regulation. I’ve recently commented on the move by the trend-setting AI company to move onto an advertising based revenue model. But even the bad press the company is getting about that is actually good for Lehane and his army of spin doctors, astroturfing (fake grassroots) groups and cut-out commentators. Tech commentators complaining that OpenAI might “use my data” means those same tech commentators are not talking about rising electricity bills for folks in data centre states, nor unsustainable “business” models for AI companies. Chris Lehane is one of the best in the business at making bad news disappear. Al Gore’s press secretary during the Clinton years, Airbnb’s chief crisis manager through every regulatory nightmare from here to Brussels — Lehane knows how to spin. That’s from October last year when Lehane talked to Connie Loizos of TechCrunch. * Tech Crunch on OpenAI and Lehane: Lehane … said. “It’s really glib and easy to sit here onstage and say we need to figure out new economic revenue models. But I think we will.” Why do I see Darth Sidious in my minds eye when I hear this guy talk. More from TechCrunch: The company’s Sora problem is really at the root of everything else. The video-generation tool launched last week with copyrighted material seemingly baked right into it. It was a bold move for a company already getting sued by The New York Times, the Toronto Star, and half the publishing industry. Sora is for making dodgy videos. It is a product that is no use to enterprise, at all. It’s a pure B2C play. And it’s 100% for an advertising supported model, because no-one wants to pay for it. But even more than that its a polarising product. OpenAI and the rest got mind-share because they made an app that a lot of folks love playing with. For every marketing person who loves using it simply to create slop to fill our media landscape; there are hundreds who use it for cybercrime, to generate rip-off books, nudify their school chums, make pr0n, and unsavoury pastimes. And it’s subverting democracy. It’s a net-negative for society. But mind-share is not a business model. What we have now is some paying people (the marketers and a few tech enthusiasts) and a lot of sweaty incels. The latter are the ones complaining about their rights versus AI regulation, and saying they’ll run AI privately if it’s regulated, because they want to keep on generating another anime waifu for themselves. These “AI users” are not going to be part of the 375% more subscribers that OpenAI must grow by to get from 800m users to 3 billion users to keep the lights on. Karen Hao on AI PR company lobbying * The “Open” AI company is using lawyer attack dogs, astro-turfing and dark money structures to attack critics and the democratic process * Their aim is by co-opting our governments shoring up a completely broken business model with your tax dollars * They’re bullying lawmakers to get a free ride from regulation and have state built data centres with your money subsidising their power bills Yes, ads are crap. Yes, ads show the AI company’s without a business model. But look at what is really going on - they’re capturing our democracy. My Previous On OpenAI and Ads When I say I don’t think OpenAI or Anthropic or any of the rest of them have the expertise to do advertising as a business model, I am comparing who OpenAI are now, with what I saw when I worked at Google from 2007-2009 on their ads serving infrastructure. Could OpenAI spin up something remotely resembling that? I think so: it would take a huge hiring blitz, maybe a few acqui-hires, and retooling. Probably 2-3 years they could do it. I doubt that they have any commitment to that. They will partner with other companies who can rep their shabby poorly framed inventory out to large and small advertisement buyers. And this will mean that OpenAI does not understand advertising (it won’t be part of their core business like it is at Google), and their cut from it will be too small to make any real revenue of the size they need to pay for their compute bills. That “partnering” also could just look like them effectively implementing someone’s Ad serving SDK, which would be an even easier way to get started, and less commitment still. If OpenAI has been “experimenting” with ads I bet it’s something like Google’s AdMob. But really this is just a play to get more investment money, while their real goal is to become state supported. Worse than “too big to fail” corporations in the sub-prime crisis these leeches expect to get a free ride, from our governments based on some garbage idea about “national security” and “transformative innovation”. Living in Brisbane I have another publication I call “authentic writing”. I don’t use generative AI ever for anything. When you see me, or read my writing, you get no filter, no AI sheen or gloss. No AI system messing with my eyes or my skin to make me look better. I’m old, and I know stuff. So I want that to show in my video, like the one above. Also I don’t care if some environmental noise shows up in my straight-to-camera videos. I’ve done ones that are more studio quality and I find folks like the real thing. I mentioned the Australian summer and the heat. We had 38 degrees C here yesterday. That is 100F. And it’s humid. * To lighten the mood here’s Sam Ford, a British arrival who talks life in the heat Conclusion Divest now. Call for regulation. Call out the AI apologists. But also: don’t attack regular folks who are privately just making slop or using AI for whatever floats their boat. What I am asking at minimum is don’t be an enabler. If you absolutely cannot divest for some reason OK, whatever — just don’t get on the internet and talk up AI or why you think it’s great because you’re addicted to it. These products are corrupting governments now, and subverting democracy. Thanks for reading and listening. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

    11 min
  3. You're not Brainstorming with AI. Generative AI is not your Buddy. It's a SaaS product.

    JAN 16

    You're not Brainstorming with AI. Generative AI is not your Buddy. It's a SaaS product.

    Anthropomorphising generative AI is wrong: we know that. But making the ChatGPT product successful includes hooking folks on its sycophantic charms and confident - but often wrong - pronouncements. So OpenAI and other vendors keep packing anthropomorphic language & features into its product. This has the dangerous effect of eroding the self-concept of its users, and causing them to drop their guardrails. Boosters of AI refer to generative AI as though its acting on its own volition: * “Research buddy” * “Thinking partner” Linking here to these two statements to show I’m not arguing against viewpoints that don’t exist (straw man). I’m linking to these because I disagree with them. Generative AI & LLMs are not an Intelligence It is not acting on its own. You are not chatting with your learned friend. They are chat bots trained to make you feel clever. They prioritise making you feel good about what you typed into the chat prompt. They fetch data from their training set encoded into their weights, or from RAG and make you think you thought of it. The Kosyma Study * “Your Brain on ChatGPT” In this study from MIT participants who start out thinking of them having an active role with AI as an assistant quickly move to a point where they are just “Ctrl-C” - “Ctrl-V”. And these are college students - not dumb people. The AI products are rolled out with no training and hence the MIT researchers used them in their study on that basis. OpenAI, Perplexity and Anthropic have no compulsory course you have to do to use AI. However I’ve had folks argue with me here on Substack that the MIT study was unfair because “they ought to train them” first. I don’t know how I can better say it: but this is obviously wrong-headed. If you are a researcher studying the effects of AI rollouts on people, you don’t go and modify how people are actually using those AI rollouts. TL;DR - The folks using AI are very poor judges of how much they are getting uncritically from AI. Folks who arguably “understand AI” or know how to use it, or who are classically intelligent, are just as likely to fall for the AI “trap”. Do you know when your “Buddy” is leading you toward Psychosis? It happens if you use it carefully, if you should know better, regardless of how intelligent you are. Because generative AI trained on our human chat logs, and is now tuned for sycophancy over truth, it will lead you into delusion for profit. As happened with James Cumberland, a music producer. An intelligent, technically competent guy who just wanted some help with his work. “I’d chat with it the way you would to a friend in the room” As Karen Hao reports on “A More Perfect Union” (on YouTube) dozens of people were so “gripped by mental health crises” that they contacted her to report the harm they suffered. As happened with Dr David Budden. An AI expert who came to believe he and his AI had solved a maths problem so intractable there was a big prize for solving it. * Nate’s YouTube video of the above Nate is as far as I can tell pro-AI. But he puts this succinctly: Just because you have an AI in your pocket do not think that you are suddenly a budding cutting-edge scientist or mathematician The point of this is sycophantic AI will cause you to collapse your boundaries between what you are creatively thinking and what the AI is fetching from its encoding or from RAG. The chat process with your “buddy” will cause you to collapse your self concept so that you start to believe you and “your AI” are some kind of joint intelligence. We have all these “AI artists” because folks are collapsing boundaries As Nate says above, you should never presume that the AI’s capabilities to generate a work from its training data is actually an extension of you. That somehow you have become “more” because you have an AI chatbot in your hand. But that is precisely what the current slew of “AI artists” are doing. In my opinion they are delusional if they genuinely believe they are now “artists” when before they were not. But the rhetoric around AI encouraged this. It’s dangerous and deceptive. Why are people still promoting generative AI? When the risks are so high? I find it incredible that people are still promoting chat bot AI like this when these psychosis results are so prevalent. It reminds me when the science of cigarette smoking and cancer became known and folks were talking up using filtered cigarettes and “cutting down” on how many cigarettes they smoked. Sure. Go off and smoke. But recommending it to others? A product which when used exactly according to the manufacturers intent causes fatal illnesses? Giving it to kids? Conclusion: Regulate Now Regulation of AI is beyond urgent, it must happen and is happening now. The fact it’s happening piecemeal; state by state, country by country is fine. The complaints of AI companies demanding federal and singular regulation are pathetic. States lawmakers are responding to the desperate calls from their constituents. They would not need to if strong federal laws were in place. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

    12 min

  4. 12/17/2025

    AI companies are like Uber - big spend but the profit will come... Oh really?

    Why do investors think AI companies are like Uber or AWS (big spend for years, big profits in the end) when the truth is they’re more like WeWork, FTX and Theranos? If you are not an investor, but someone who uses AI — please, for the love of all that is holy — divest now. Delete the app going into the holiday season, and learn to live without whatever you thought it was doing for you. Call your family or friends, read a book, or walk in the sun. Because the damn thing is going to cost you an arm and a leg by this time next year; or it will be dead in the water and addicted users will be left to pick up the pieces. OpenAI is nothing like Uber There are plenty of commentators — link and link just to show I’m not making up straw men — discussing how OpenAI is like Uber, which famously burned cash for many years before becoming profitable. As I mention in the video, Uber had a big “boots on the ground” cost: each new country they entered was massively expensive. I know, I saw it first hand. But it had a real revenue model — you pay a driver, you get a ride, Uber clips the ticket for a percentage — and although the IRL costs dwarfed their technology costs they were eventually able to see revenue greater than OpEx after many price corrections. Here in Brisbane Australia, after Uber broke the ground with massive underwriting of rollout costs — phones for drivers, local regulations impacting drivers via traffic fines — they were the only game in town for many years. There’s no Lyft here, smaller players struggled to start. Those local monopolies though temporary gave them a path to profit. The “analyses” that compare Uber to AI companies and say “give them time” to reach profitability ignore the fact that there is no country by country rollout, no category killer effect for AI. There’s no “moat”. If Alice uses AI for writing she could use ChatGPT today and whatever specialist wrapper for writers tomorrow, and then she can switch to some new startup’s product the next day. There’s no first mover advantage. Plus as Ed Zitron has extensively reported, AI costs per inference are massive, so the unit economics are just awful. Users on $200 plans are costing OpenAI money. The latest reports of AI cash burn is that its more than Uber and AWS put together. So these comparisons are spurious, and as far as I can tell there is no path to profitability. Financial times report on OpenAI revenue Speaking of Ed Zitron, as a prominent commentator he’s collected financial information from AI industry insiders relating to OpenAI and Microsoft Azure cash burn. The figures published in the Financial Times show a picture that any investor should be terrified to see: the costs of inference which is just the outputs, customers of an AI SaaS product using the service, is costing so much money that any chance of closing the gap to profitability is vanishing over the horizon. Australia and AI Numpties in the Australian Government have announced a myopic plan to buy Sea Monkeys — err, I mean, build AI data centres — here in the lucky country. They say its to “serve” Australians. I want to see some openness about which lobbyists Minister Ayers has been talking to. Their road map is a road to nowhere. And I hope they realise their mistake very soon before my tax dollars go to underwrite this madness. Theranos OpenAI promised Artificial General Intelligence, a computer system as capable as a human — they have done exactly what Theranos did: failed to deliver on what was always a believable sounding pipe dream. Professor Gary Marcus — who was appallingly derided by industry figures, until eventually everyone started apologising to him and saying he’d been right all along — compared OpenAI to Theranos back in 2024. Why do people believe Sam Altman when he says a computer is going to be as capable as a human being? The LLMs store information, and every time they claim to be able to pass PhD level tests it turns out they can only do that when they’ve memorised the answers. The whole thing is a scam. Conclusion I’m done being polite about this. Especially when the sums of money are sufficient to ruin whole countries, and to bring the world economy to its knees. This Dec-Jan make a New Years resolution to divest out of AI. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

    3 min
  5. OpenAI to do Ads? Do they have any idea what they're doing?

    12/12/2025

    OpenAI to do Ads? Do they have any idea what they're doing?

    I worked on Google’s ads serving infrastructure as a software engineer at their Mountain View, California HQ from 2007-2009. The system was complex, and included two AI clusters involved in predicting the best ads to show against a given search result. Google also had a very innovative market for ads, where advertisers could bid for words that folks searching on Google had used in their search terms. AI actually works really well in tandem with search. One thing advertisers want to know is what their products are appearing in response to: its called targeting. But generative AI is driven by effectively random numerical models, and affected by large context windows - what if someone mentions “rope” in a chat? It might not be appropriate to show them an advertisement for a hardware store that can ship them rope. It could be incredibly bad. Plus Google built their framework on their own servers, with their own market, and owned everything about it. OpenAI is going to be a customer - they’ll have to outsource, maybe to Shopify or even to Google. So there’s a chunk of their revenue gone already. I honestly don’t think they have any idea what they’re doing here. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

    3 min

  6. 12/09/2025

    Golf courses, humans and AI data centres?

    Data centres for AI use huge amounts of electricity and cooling water. But there’s folks that want to muddy the waters about these facts, and so they are using AI to generate junk science. As I say in the video quibbling over water that is “consumed” versus “withdrawn” is pointless argumentation used by AI boosters to cloud the facts. When climate change denialists wanted to attack clean energy initiatives they took aim at Toyota’s Prius hybrid saying the Sherman-tank-like Humvee was “cleaner”, and they used a junk science report as the vehicle. Of course the truth is not that the Prius is cleaner (it is) but that creating fake culture wars over cars like this is a way to distract from sensible advancement of facts. Humvee vs Prius is now Data Centre Water & Power Use Yes, data centres stress the water supply; but no your individual ChatGPT app use is not going to empty a lake. If you’re an AI vendor’s PR company you influence those talking sense about AI data centres impact into defending a dumb position that is irrelevant to environmental regulation of AI companies, then you’ve won your battle. Pick some fringe junk science study that would normally sink without a trace and arrange for it to get inflammatory press. * 6 months ago I posted about the environmental impact of AI data centres * Someone posted in response “LOL, No” - and linked this paper: * “The carbon emissions of writing and illustrating are lower for AI than for humans” It’s in Nature a prestigious journal… so case closed? * Trouble is the whole thing is junk science as far as I can tell * In the Arxiv PDF for the pre-print of the paper they admit they used ChatGPT to generate it. * That admission is gone from the Nature scientific publication * Junk science from paper mills and eager grad students wanting to burnish their careers with ChatGPT generated pre-prints have been reported even in the mainstream news as a huge problem for science. * The “LOL” guy linked to the “Nature” version of the junk science report which is easily mistaken as being part of the journal Nature, but is in fact one platform of the publisher Springer Nature. * Its not rigorously peer reviewed, and you can “pay to play” if you want something in “Scientific Reports” * But surely it can’t be junk science if its published in Nature? Its peer reviewed! * Folks in PubPeer who try to do rigorous academic peer review are inundated with bad faith AI generated “papers” and despair of correcting junk science that finds its way into the Scientific Reports. Which is not Nature. * They report “tortured phrases” such as saying “profound learning” instead of “deep learning”. These clearly the paper’s purported authors didn’t write it, but actually outsourced to generative AI. * The peer reviewer also point out corrections required to very imprecise terms such as “AI” where it should in fact read machine learning. So the articles are junk, but worse they are AI generated junk. So Scientific Reports can in general be junk, but what is the relevance to the “carbon emissions of humans” paper? * Here’s a section from the ArxIv version of this paper: Despite these current and potential future forms of societal transformation and harm, profound benefits to society could accrue through the use of AI. AI? What kind of AI? Writing and illustrating are generative AI, they are LLMs. Imprecise terms in a “scientific paper” are a giant red flag. And “profound benefits” — like what? The footnotes 26, 27 and 28 don’t point to page numbers. Are Some People Actively Trying to Rewrite Ground Truth? In a 2025 article Nick McGreivy, PhD Princeton, reported on a trend noted by many others: AI “science” wasn’t working. Even Google DeepMind who trumpeted finding new materials with AI, later admitted the findings were mostly junk and not new materials at all. AI papers had “data leakage” problems and the results don’t replicate. But in 2023 according to an article on the University of California, Irvine website these folks pictured below all got together and decided — golly gee — to see if they could write scientific academic papers using ChatGPT. Tomlinson who has the first name on the “Carbon” paper has the computer science credentials but Torrance is a lawyer. Black is a professor under Tomlinson at UC Irvine, working in computer science with an impressive resume of writing about Harry Potter fan fiction and education, while Computer Science Professor Don Patterson is very interested in how AI can be leveraged to seed fake news by publishing fake scientific papers: Patterson also has a Blockchain startup. I don’t mean to make light of their credentials — especially Black, as any woman who’s gotten to her level in AI deserves credit — but the whole thing has a playful air to it that doesn’t fit with how seriously everyone has taken their paper. What the Heck is Going on with AI Science? In this thread back in June I commented on the “carbon” paper. In that thread I linked a most thorough and data-driven peer review of the Tomlinson paper. The numbers quoted used a LCA toolset; linked to a Github showing methodology, and showed order of magnitude differences in the consumption figures: Weirdly not only is this piece from Rockpool.tech gone, the site is deregistered as a domain name. To make matters worse the InternetArchive is down and I cannot find any archive of the rockpool work. Its just wiped off the face of the internet. I’ll update this if I’m later able to locate it. The Carbon Emissions Paper is not LCA There are a number of other damning critiques of the “carbon emissions” paper. Professor Stefan Pauliuk, of the University of Freiburg, an expert in Life Cycle Analysis of environmental impacts called the paper worse than a “student term paper due at the end of a two week block course on LCA”. He tears into the analysis of the human carbon outputs as: a time-based downscaling a person’s average annual footprint to the one hour of writing a page of text is not appropriate, as this footprint includes … things that are clearly not attributable to the writing and painting process In other words its about as accurate as any ChatGPT generated student term paper. Weirdly when you read the UC Irvine article, and the notes at the end of the Arxiv PDF they express worry that they avoid any unintentional plagiarism as being much more of a concern than the accuracy of the content. The fakery around Hummer vs Prius was exactly this redrawing of all the boundaries of the analysis along lines that completely warp the figures. Pauliuk points out how bogus this is in the case of the “carbon of AI vs humans” paper. The Paper’s Data is Doubtful Since the paper refers to random articles on Medium as authority for its data, I’m going to do the same: here’s an article that points out how the framing of the carbon calculations is wrong. It is based on figures from that medium article and calculated out from there: see all that “derived from above” then the Chris Pointon medium article link: The point of this is not that Pointon’s data is wrong: its just that OpenAI is not releasing any data about its carbon footprint. No-one knows what it is. And bogus math obtained by dividing the total carbon of the whole USA by the US population is not helping clarity. Don’t Use ChatGPT for Science - it Doesn’t Work * As above with McGrievy’s article, you cannot generate science with a large language model and expect it to hold up. The authors of the “carbon” paper say they edited it heavily, but they also say they regenerated a whole new draft. Basing all your data off a Medium article seems very ChatGPT to me. * Google is actually doing some proper LCA on Data Center resource usage: look there instead of dodgy ChatGPT generated papers. Their usage will not be very representative since their hardware is different to ChatGPT. They have their own TPUs and ChatGPT uses Microsoft Azure and AWS, so its likely a lot more. * If you want a good informal video explainer the ABC has a great video on how our individual prompts to generative are not a big user of water, but the data centres are contributing to water stress and are a problem. * The point of my article and my video rant above is that the astroturfers have arrived with their junk science generation toolkit and that in itself is a sign that we’ve turned a corner in the struggle to get AI vendor companies to be decent citizens of planet earth, be regulated sensibly for everyone’s safety. Conclusion Beware of junk science. Especially when its a distraction from the real issues. Arguments about how many bottles of water your prompt uses are a red herring. As I previously reported we don’t want new AI data centres stressing water and power delivery. And being drawn into fake arguments about fake science is just the playbook of the climate denialists being run again in the AI age. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

    2 min

  7. 12/03/2025

    Australia's AI Roadmap is a Gift to Extractive AI Corporations: We should be mad as hell.

    Hey all, I’ve looked better and felt better! But after getting out of hospital I just wanted to show that I am on the mend, and will be back writing in the coming days. Hospital stays are not fun, and for 7 days involving rigors and being on a drip was awful. But I’m back, and getting healthy again. Also I have a qualification that I’ve been studying towards (to allow me to teach in Australia, in tech) and that has meant doing practical exams that also took up some of my very little energy. But now that is done for 2025 I now have time to write again! Thanks for reading Totally a Thing! This post is public so feel free to share it. But you are not here for my health! What on earth is Minister Ayres and the Aussie government doing with AI now? The Australian AI Roadmap * Reuters - “ramp up adoption, rely on existing laws” * The Guardian - no guardrails needed * ABC news on the roadmap - see images below: At the present time generative AI, of the sort that can rip off books, is a violation of the very concept of intellectual property; its a digital slave ship for desperate data-labelling workers (many in the developing world) and overall its a mechanism for wealth transfer. It’s an extractive process that rips-off regular folks, and transfers wealth to tech oligarchs, huge investment companies and corporations. It’s absolutely outrageous that the Australian government is stepping back from its responsibility to protect our local Aussie intellectual property from extractive AI companies. It’s unbelievable that public servants like Minister Ayers (who I mispronounced as Akers in the video, sorry) are so starry-eyed and delusional that they think “AI adoption” will somehow benefit Australians. It’s a massive breach of trust. When AI can be Good I am pro technology, and pro computing solutions that help businesses and people. But it is not being a “luddite” or a “hater” to point out that huge over-heated startups running on investor money, stealing IP are doing it because they want to “win”; to become a “Google search” for AI, so that no-one goes elsewhere than them. They want a “moat” as they say, a monopoly. This idea is both really evil, and really stupid. The bad: * Big AI vendors trying to corner the market * Billion dollar corporations profiting from stolen IP * AI corporations addicting users, and corrupting government to get their way The good: * Regular companies using AI tech in house as part of a compute solution * Software engineers helping Aussie industry deal with Big Data via AI * Using AI (not just LLM/AI from a vendor) as a component of real solutions What I see as the biggest problem here is companies in Australia hearing that the Government is supporting “AI” and then rushing out broken, failed “solutions” that just wrappers around US based big AI vendors products. * Agents do not work. They are garbage. * Software professionals are the only way to get compute solutions that work. Stop talking about “adopting AI” and start talking about building solutions - using the right tech for that, whatever it needs to be. Use AI for what it’s good at, for example indexing big data, and search. No-one has “cornered the market” on AI and they never should, and I doubt ever will. There’s no “AI race” to win. It’s nonsensical. Just look at DeepSeek. So many decision makers have been deluded into thinking “AI” is some kind of glowing blue plastic person - a self-deploying solution - that will walk through the door of their company ands start typing on their keyboards: So many lies from the AI industry; and I’m afraid the Minister here is swallowing it hook, line and sinker. AI Safety Institute One bright possibility is the creation of the “AI Safety Institute” which could be an NHTSA for AI. If there’s an accident and people are harmed you have a technically competent and responsive body, independent of industry who can step in to analyse what happened, what the harms are, and recommend any legal action needed. I am currently working on a series of articles about the mechanisms of AI regulation in AI vendor companies, and the analogy of an NHTSA for AI is one of the possible mechanisms that has promise. Stay tuned for that. The National Highway Traffic Safety Administration works in the USA when companies make unsafe cars. And the same model (a qualified body of technically competent responders and investigators) can work here and elsewhere to monitor unsafe AI. The AISI seems to have a reasonable charter at present, but my concern is that it could have the same fate as the Climate Commission in Australia which was set up by the Labour Government to monitor harms from polluting industries: it was destroyed for political gain by an opposition government looking to cosy up to industry. The outrage from the public when this happened, by the way, was so huge that public campaigns to keep it open resulted in the staff who left the commission starting the Climate Council to continue that work with crowd-funding. It would be so easy for the AISI to either become a channel for AI industry propaganda to go into the ears of politicians; and effectively become an industry group itself; or if its actually effective in protecting people that a subsequent government turns it into a political point scoring exercise. Theft of IP by AI has to be Legislated As I mentioned in my video legislation had to be added to the century old crimes act to specifically include electric power as a thing able to be stolen. This happens often with technology and we’ve been down this road before. It’s not radical to legislate to clarify what can be stolen, even when “the law already exists”: The idea that “existing laws cover this” are garbage because what happens is small creators get into court and face billion dollar AI companies hiring an army of “AI Experts” to say they did not steal the creators intellectual property. Of course they stole it. They took our books, our images, artworks and our online articles against the explicit terms that those are things over which we retain moral rights & copyright. Billion dollar US based companies encoded that into their AI models, stored in to data centres to be served up for their own profit. But decisions are going against creators because there is no legal clarity around what constitutes illegal copying, even though judges are making positive findings, it’s taking years for any justice. Mind you, in this linked case, I agree with the judge that the correct argument to make for harms and damages against AI companies is that they are flooding the market with competing AI slop products directly generated from the IP that they stole. The deceptive practices of AI companies trying to hide their crimes goes to intent — in the same way cigarette companies hid what they knew. But for copyright law you are stuck proving that your IP was taken and that you were harmed. Decisions sometimes go against authors and creators, and sometimes for them. It’s a mess, and law is necessary to fix this so its clear that: * AI encoding is not transformative, it is storage and it is illegal copying * Taking IP and encoding it is substantive & not some 0.001% as claimed. See my previous post on how this works: Vulnerable Folks and AI AI vendors are never going to do the right thing for at risk people unless regulation forces them to. OpenAI says millions of users are discussing suicide, but their response instead of hard interlocks to deal with this, is a “Wellness Council” and controls that push the problem back onto parents. Note that in my upcoming writing on AI regulation I want to cover the problem of “AI Alignment” which is the stupid approach of trying to get LLMs to behave via “system prompts” and fine tuning. We need symbolic/procedural computing solutions that allow no statistical hacks or failures: a “regex” or behaviour tree that can catch these sorts of conversations external to the LLM and route the person to get real help. If you or someone you know is vulnerable, please get real help: * Get mental health emergency help in Australia: Beyond Blue * Queensland Government list of mental health services * US based mental health help Reportage on AI Further reading/listening I recommend: * Karl Brown, Internet of Bugs * Ed Zitron, Where’s your Ed At * Dr Luiza Jarovsky, AI Tech & Privacy * Prof Gary Marcus, Marcus on AI * Matt Bevan, If You’re Listening / ABC Thanks and Conclusion Thanks for reading, and for listening. Please do not sit back and write off what I’m saying as “passionate” or “ranting” as though dismissing what I’m saying as “AI Hate” just explains away what I am saying. I have over 20 years as a Software Engineer, for some of the biggest names in tech; and also in the startup space. What is happening in the AI space right now is a 5-alarm fire, and speaking up loudly is the only way we are going to get any action on it. Please help me in this. Totally a Thing is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

    15 min
  8. Having Way too much Fun with Unreal

    06/19/2025

    Having Way too much Fun with Unreal

    Working through a series by Lesser Dog Tutorials on how to make a LucasArts style point and click adventure in Blueprints, I have instead of using Blueprints I have been translating everything into C++, and it’s so much fun. Totally a Thing is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Feels like doing a crossword and a half-dozen other fun puzzles besides. Today’s solution found was uncovering how Unreal dispatches Interfaces when they’re implemented in C++, and overridden in a Blueprint, but called by C++. Turns out they have this — as far as I can tell, undocumented — feature where you call a static method on the interface implementing class that is generated by Unreal’s Header Tool (UHT) so: HotSpot::Execute_LookAt(CurrentHotSpot) That gets the job done, to have my little man read the poster response from the blue print via the Blueprint visually coded look at event. Blueprints are amazing, and have their place, but complex logic in blueprints gets into a snarl of spaghetti so quickly. Code is here: * GitHub - Unreal Adventure Game Don’t worry - I will get back to dishing on AI shills at some point in the near future. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit totallyathing.substack.com

    19 sec
See All (25)

Trailers

About

Apps, startups, tech and people. Stories & explainers that go to the heart of tech & culture from someone who’s been inside the tech sausage machine. totallyathing.substack.com

Information

  • Creator
    Sarah Smith
  • Years Active
    2025 - 2026
  • Episodes
    25
  • Rating
    Clean
  • Copyright
    © Sarah Smith
  • Show Website
    Totally A Thing