Content Operations

Scriptorium - The Content Strategy Experts

The Content Operations podcast from Scriptorium delivers industry-leading insights for scalable, global, AI-optimized content.

  1. 3d ago

    From clay tablets to LLMs: Applying communication theory to AI-optimized content ops

    In the second episode of The Uninvited Author podcast series, authors Sarah O’Keefe and Carlos Evia examine how foundational communication theories help structure modern content operations. They unpack how AI adds new intentionality to creating and delivering content, providing the theoretical groundwork creators and leaders need to manage workflow risks and build sustainable AI-optimized content ops. Sarah O’Keefe: It’s not that you have to go talk to your management or your CEO or whoever you’re dealing with about Umberto Echo, although that might work. It is that this needs to be in the back of your mind so that when somebody pushes back with something that is clearly nonsensical and is just simply not going to work, you can explain why. You have the foundation that allows you to move forward and say, “Okay, so as we’re building out the system, as we’re bringing AI into the equation, here’s where it’s going to do well and here’s where it’s risky.” Related links: Terje Hillesund Marshall T Poe Umberto Eco Daniel Dennett Digital sovereignty in the age of AI From ad hoc to autonomous: The AI content ops maturity model Check out other episodes in The Uninvited Author podcast series LinkedIn: Sarah O’Keefe Carlos Evia Transcript: Disclaimer: This is a machine-generated transcript with edits. Introduction with ambient background music Christine Cuellar: From Scriptorium, this is Content Operations, a show that delivers industry-leading insights for global organizations. Bill Swallow: In the end, you have a unified experience so that people aren’t relearning how to engage with your content in every context you produce it. Sarah O’Keefe: Change is perceived as being risky; you have to convince me that making the change is less risky than not making the change. Alan Pringle: And at some point, you are going to have tools, technology, and processes that no longer support your needs, so if you think about that ahead of time, you’re going to be much better off. End of introduction SO: Hey everybody, welcome back. This is Sarah O’Keefe again, and I am here once again with Carlos Evia. CE: Hello. SO: And this is part two of our podcast series about the book that we’ve just released called The Uninvited Author. And we want to talk in this episode about some of the core concepts that we used in breaking down how we look at the process of creating, managing, and delivering content and what that looks like. So Carlos, what were some of the key concepts that we that we brought into this discussion? CE: Well, we were looking at things that were a bit broader than just one type of content. We talk about content a lot in our lives. I mean, you in your work and you do as well in my work from slightly different perspectives, but with a core of what unites our experiences is technical content or enabling content, that content that at the end of the day, you’re gonna read not for entertainment, but when you need assistance, you need information to complete or perform a task. But we wanted this book to be a bit broader than just specific cases about technical or enabling content. So instead of just jumping to the life cycle of technical content, that many versions of that exist. I think you were the one who found this text cycle that came from a professor in Norway.  SO: Yeah. CE: And my Norwegian is pretty bad, so I will attempt to pronounce Terje Hillesund. I hope that is correct. We might get some tomatoes thrown at us for mispronouncing this media studies professor’s name. and he does research from where we can tell he’s still active. He does research on digital publishing, especially when it comes to electronic books and EPUBs. So he proposed a text cycle that had some very clear stages. What he calls them stages, right? Or phases? What does he? I think he’s stages. SO: I’m not actually sure. And also it probably sounded better in the original Norwegian. CE: We need to do an AI translation of this episode into Norwegian and see how it goes. So those let’s call them stages, this is our adaptation of it, are pretty much life cycle like, but they’re not just specific about one type or one combination of elements of content. So they’re very broad for any type of communication that is particularly being written, but not just delivered in text. We also applied it to multimedia and other digital approaches to text. And we have a whole section in the book saying: what do we mean by text? It’s not just words written on a page, but it can be audio and visual content, anything that is contained in bits and is delivered through electronic channels, even if you at the end of the day get it in a piece of paper that had to come from a file that was printed. So do you want to talk about the stages or should I talk about the stages, whatever you wanna do? SO: So breaking down the stages. As Carlos is saying, it’s not just text, words on a page per se. and in fact, you could apply the so-called text cycle to spoken communication, which kind of makes me twitch a little bit.  CE: Mm-hmm. SO: If you think about this, and of course we’ve got a bunch of images for this in the book, but there are six phases to this text process or cycle. The first one is write, which is really having the idea. Then there’s produce, which is something like writing it down. There’s storing, which, you know, in a digital content world would be parking it in a file or a database. There’s representation. To represent it is to format it and make it look a certain way. And then you distribute it, and finally you read it, or you consume it. Now, most of us, if you’ve ever thought about this, which most regular people don’t have any need to, think about writing and reading. There’s an author; they write some stuff, and then there’s a reader, and they read the stuff. But as you start breaking down this process and thinking about what really goes into that, if you go back to the earliest communication, so now we’re talking about spoken communication. Well, the process of writing, of creating the idea, of packaging that idea of storing it, of representing it, which is to say, to use your words to speak it. And then you use your words also, your voice to distribute.  And then you’ve got the reader, the consumer, they’re sitting right there listening to you. And so this entire six-phase process is really just two, right? It’s two actions. I speak, you listen. And then all these other pieces that are represented in this text cycle are all embedded in that little two-step process. The minute you introduce actual writing, the process of putting something on a clay tablet or piece of paper, which of course came much, much later, well, now I have an idea in my head, and then I take a pen or a stylus or a chisel or something, and I actually encode that information onto a tablet or we’ll go with paper, right? We put it on a piece of paper. So I’ve now produced it using visible marks, and I’ve stored it on that piece of paper, and I’ve represented it on that piece of paper. But the key thing is that I have not yet distributed it because I haven’t distributed the idea. Now if I hand you the piece of paper and you drive across town and hand it to somebody else, that’s distribution, and then they read it. SO: And so you can see this process of decoupling the phases starting to happen. And the key thing is when you look at digital content, we decouple more pieces. It gets more and more fragmented, and there are more and more people involved, more and more different roles involved in the process of having an idea, encoding that idea in some way and then representing it in some way and delivering it downstream to the person who finally consumes it. And so that was kind of the fundamental starting point for this book was to look at this text cycle and ask the question of: okay, well, what does it look like to have this text cycle in typically now a digital workflow?  And now we introduce AI. And when you look at the six phases that we have here, it turns out that you can use AI in different ways at every point in the text cycle. And I think this is why, as authors, as content creators, looking at AI is so difficult because there are lots and lots of different things we can do with it. It’s not just go online, open up a chat bot and ask a question, which is how most people, the vast majority of everybody, that is how they interact with AI. But as an author, there’s a whole bunch of other stuff going on. so did you want to talk a little bit about what it looks like at some of the points in this text cycle to introduce AI. CE: Yeah, but I think what something that I would like to mention is that we didn’t just use that one lens to analyze how content and communication has evolved based on this life cycle, the the text cycle, to get us to where we are right now working with artificial intelligence. So instead of that one lens, we combine it with other perspectives from different disciplines and particularly from history, we borrow the work of Marshall T. Poe about the ages of human communication. So Poe, who by the way is from Alabama, not from Norway, the develop a framework for studying human communication across history. So Poe had five ages of communication, which include the age of speech, the age of manuscripts, the age of print, the age of audio and visual media, and the age of the internet. So what we did is to put a Hillison’s text cycle across the ages that Poe documents in his model to expand that and get to propose a new age of human communication that we surprised called the age of human communication with AI. And we were able to I want to say map the stages of the text cycle into what communication looks like now with artificial intelligence. And we start the book in very, I don’t want to say naive, but conservative ways of presenting artificial i

  2. Oct 12

    Bridging the gap between content design, content ops, and AI

    Is AI the catalyst that will finally bring content design and content ops together? In this episode, Sarah O’Keefe speaks with Kate Agena, Vice President of Content Design at SAP, about why fluency is cheap while meaning remains expensive in an AI-driven world. Kate Agena: But users don’t care if content is coming from marketing, product, or whatever: they want whatever it is that they need at the time they need it. If everything is just kind of in one pile of information, that’s not going to come out correctly. So I think ops is just so incredibly important now, and content ops in particular are going to be far more important than they were. Related links: KateAgena.com Paz Parez: Context architecture Digital sovereignty in the age of AI From ad hoc to autonomous: The AI content ops maturity model LinkedIn: Sarah O’Keefe Kate Agena Transcript: Disclaimer: This is a machine-generated transcript with edits. Introduction with ambient background music Christine Cuellar: From Scriptorium, this is Content Operations, a show that delivers industry-leading insights for global organizations. Bill Swallow: In the end, you have a unified experience so that people aren’t relearning how to engage with your content in every context you produce it. Sarah O’Keefe: Change is perceived as being risky; you have to convince me that making the change is less risky than not making the change. Alan Pringle: And at some point, you are going to have tools, technology, and processes that no longer support your needs, so if you think about that ahead of time, you’re going to be much better off. End of introduction SO: Hi everyone, I’m Sarah O’Keefe. I’m here today with Kate Agena, who is Vice President of Content Design at SAP and also founder of the Content Design Leaders Membership Community. She has a PhD from Purdue, and in doing her PhD, she studied how tech writers could use iterative development processes to impact software design, otherwise known as, at least to me, basically waterfall versus agile. So she’s got some really interesting background as a tech writer and kind of started out there via I think information development at IBM and then landed in content design. So welcome, Kate. Kate Agena: Thank you so much, Sarah. Happy to be here. SO: Yeah, we’re glad to have you and to connect some of the maybe more technical content enabling content side of things to content design. our focus today is actually on that, on the fact that content design and content ops are related, or maybe should be, but somehow aren’t connected. And Kate has a theory that AI is actually going to for once be a force for good and maybe bring these disciplines together. So we should talk about that. But first, give us your definition, your short definition of content design. What is that? KA: Yeah, so for me, UX content design is a UX design practice, first of all. And it is the practice of designing the information and the language within a product experience so people can understand what’s happening, they can make decisions better, and they can accomplish whatever it is that they’re trying to do. SO: Yeah, and I think I would argue that content ops is the tooling and the processes that sit underneath that, that this content design process or discipline flows over or inside. so I was reading up on some of your stuff, and I’ve included Kate’s personal website in our show notes. One of the things that you said, and I’ve seen this in longer phrasing from other people, but you had a very succinct definition of this. You said fluency is now cheap and meaning is not.  So, we’ve said for many, many years, that it is not true that anyone can write. However, it turns out that with AI, if you know, if writing counts as let the AI generate the text, then I guess it is actually true that anyone can write. The problem is that we evaluate content on fluency. If it’s poorly written, if it’s hard to understand, if it’s ungrammatical, we assume it’s low quality. But that isn’t true anymore because AI will always generate something that looks good, looks plausible. So it sounds good, it’s well written for values of the mechanics are correct, but it’s not necessarily accurate. So when you say that fluency is cheap because of AI, but meaning is not. What what is meaning and why is it still expensive? KA: Yeah. So I mean, for me, I think coming from the techcomm world where, you know, and from rhetoric where obviously I’m always worried about things like audience purpose and context. What is it that the person is needing? What are they trying to do? And then design really makes you think about psychology and you know, where is this in a user’s journey and maybe in the user’s day and additional, you know, a lot of different context around whatever the content is. And that emphasizes so much user research. You understand if you’re, you know, if you have experience doing content design, you have to like, you know, changing one phrase, changing one word can make a huge difference on, you know, even product success, but certainly on how a user reacts to that content.  And so it’s, you know, it’s very different for an AI to just kind of generate a phrase that has the general meaning, supposedly, that you need in that piece of UI, than to really do all of the thinking about the human context that’s around that. And I think there’s a lot of discernment that happens that a content designer brings in, or even a technical writer brings in to documentation, maybe.  As they’re kind of interpreting a lot of times what devs have put in, that any tech writer will tell you that they do a lot of work to get maybe specs or you know, JIRA information or a description that a developer gave them into a format that users can actually understand. And with content design, it’s the same that you really have a lot of human context that you need to take into account. And that’s a lot of discernment that I think. You know, you can’t give all of that context to the AI so easily. and if you try to, I think you it’s it’s gonna get lost in the context window at some point. SO: So then as we’re working through this and we’re saying, well, you know, writing and discernment and humans are more important than ever, you know, I had yet another note from a colleague who’s facing a layoff. And it’s pretty clear that at the individual level, people are losing their jobs because of AI. the AI can do it, we’re getting rid of our entire team. The AI can do it, you’re out. So there are lots of content people looking for work on sort of both sides of this fence that we’re talking about, but there are also new opportunities. So acknowledging that there’s huge disruption and a lot of pain, where do you see some potential and some new jobs and some new specializations evolving? KA: Yeah, I think there’s a lot. And I I feel like there should be kind of niches that people can find something that feels a little bit closer to their own skill set to move into. In content design, a lot of folks are moving into content systems, which is a little closer to content ops. Also, content ops, I think, should be something that people could be moving into more content strategies, you know, kind of ideas that we would have thought about as content strategy a while back. So there’s definitely the system side, which is just expanding because now we’re talking about things like SAP calls it a compositional design system that would allow a machine to use that design system to generate UIs on the fly based on user intent. And to do that, you need an awful lot of information to help that machine kind of go through a tree of decision-making. And find patterns. What are the good patterns for doing this specific thing? How do you understand what the user is actually looking to do? And then things that you know used to be a human would be involved, and so they would bring in all of their human knowledge. You need to encode that into components and things as well. So there’s, you know, kind of system-level voice and tone, personality, behavioral things, but also, you know, on the component level, there needs to be a pattern for what the copy should be on that specific component. So I think content systems is a huge new area. Where, you know, before I think we always wanted to put resource into that kind of work and there wasn’t any appetite for resourcing it. And now it’s kind of table stakes that we need to get there. If they want AI to do these things, we need to have a lot more effort put into content systems and governance. You know, it’s one thing to build it, and it’s another thing to keep it going.  SO: A bad word. KA: So content systems, yes, content systems is a big one. A second big one for content designers is model design, or there’s a lot of different names going around for it now. Paz Paris called it model design when she first kind of identified it when she was at Google working on, actually, I think it was working on Bard at the time, before it was Gemini. And that’s work that content designers have been kind of really a natural fit for. So that’s things like system prompts and skills and evals and determining all of that stuff around kind of how the model will behave with folks. so model design is a second one, content engineering people call it, that kind of thing. So that’s a second big one, I think, for content designers.  I don’t think static UI content design is going to fully go away, but I don’t think anyone will be doing exclusively that. I think there’s a new need that you have to have patterns and guidelines that fit your static UI, but you have to have all of those also articulated for the machines. And then everything has to go together. So I think on the content design side, that’s where most people are trying to learn skills. And then I see this ad

  3. Sep 14

    The Uninvited Author: AI, intent, and the future of content

    In this podcast, Sarah O’Keefe and Carlos Evia discuss their upcoming book: The Uninvited Author: AI, Intent, and the Future of Content. Carlos Evia: One of the things that we really want to invite people who end up reading this book to do is engage critically with AI tools across the text cycle. It’s not magic. It’s a collection of ugly computers in water-chugging data centers that work primarily on predictability approaches. And I truly think that we need to be critical in our engagement with AI as authors and as readers of content. Related links: Digital sovereignty in the age of AI From ad hoc to autonomous: The AI content ops maturity model Subscribe to our newsletter for updates on Sarah O’Keefe and Carlos Evia’s book: The Uninvited Author LinkedIn: Sarah O’Keefe Carlos Evia Transcript: Disclaimer: This is a machine-generated transcript with edits. Introduction with ambient background music Christine Cuellar: From Scriptorium, this is Content Operations, a show that delivers industry-leading insights for global organizations. Bill Swallow: In the end, you have a unified experience so that people aren’t relearning how to engage with your content in every context you produce it. Sarah O’Keefe: Change is perceived as being risky; you have to convince me that making the change is less risky than not making the change. Alan Pringle: And at some point, you are going to have tools, technology, and processes that no longer support your needs, so if you think about that ahead of time, you’re going to be much better off. End of introduction Sarah O’Keefe: Hey everyone, this is Sarah O’Keefe. I’m here with Carlos Evia. Hi, Carlos. Carlos Evia: Hello! SO: So, Carlos, for those of you who don’t know, is the co-author of this book that we’re working on and also professor at Virginia Tech. His official title, his official titles are Associate Dean of Strategic Initiatives at Virginia Tech, and he is also the CTO of the College of Liberal Arts and Human Sciences. I think, Carlos, that means you’re the computer guy in the humanities world. CE: That means that people come and complain to me about curriculum and also about their computers. So yes. SO: Okay, cool. And so with that, Carlos and I have written this book. And in this first episode to this special series, we wanted to give you an overview of what it is that we’ve done, or what it is we hope we’ve done, we are trying to achieve here. And probably the easiest way to do that is actually to break down the official book titles. So what is our official book title? CE: The Uninvited Author: AI, Intent, and the Future of Content. And I don’t know if I got it right from memory. SO: No, I think that’s right, only because I wrote it down. So I’ve been staring at it nonstop for, you know, four to six months now, but still.  CE: Yeah. yeah. I see it now. Ha ha ha. SO: You know, can we remember our own book title at this point in the writing process? And we should tell you that we’re recording this as we’re approaching sort of ninety percent done, which means that we still have approximately ninety percent of the work to do. And we’re a little punchy. CE: My co-author today woke up being very optimistic about progress on the books. But I agree, 90% sounds accurate. Yes, I’m not gonna argue. SO: Yes, you’re well, your co-author is slightly delusional, so we’ll just set that aside and move on. All right, so let’s talk about this. The main title is the uninvited author. So who is the uninvited author? What are we talking about here? CE: Well, the uninvited author, surprise, is the AI author. We have seen how AI has been taking over the generation of content at different types of content, content for marketing purposes, content for technical purposes, content in video, in text, in audio. And in some of those cases, it’s creeping into people’s workflow without an invitation. And now, and we know people, I mean, across the different facets of content work that now are, on purpose, inviting AI agents to contribute or to take over and start driving the car of the content operations. But particularly for the type of content that we do in our little corner of technical communication, AI, authors kind of started coming as uninvited guests that will appear when a user will use a chatbot, say going to ChatGPT or going to Claude asking how do I do this instead of going to the published official manual of the name of the appliance or name, the software application.  And what we realized is that what was happening is that as the company, I write my documentation, I put it on the website, I put it on a book if I’m in 1985, and I give that to my users. And I hope that that’s what they use. I am the author. I bring it to you. But now when a user prompts a chat button and says, how do I turn on this machine? How do I save a file in this software application?  Now there’s a new author that is going to give information that might be accurate, that might reflect what I wrote, or might be inaccurate. It might be, like you said, delulu and inventing some other content. So that is the uninvited author. An author that is already here and is adding to what other humans have created or written or flat-out making up stuff out of nowhere that didn’t exist and that was not created by humans before. SO: And I think that this this theme of loss of control for the humans returns in maybe every page of this every page of this book. So we have the AI that is now delivering… well, it was delivering content, right? And when you had a search engine, you would ask it a question and it would return answers, but in general, those answers were, “Here’s what I found in this document,” or “Here’s a collection of links,” or whatever. The what’s new is that the AI is synthesizing answers from its sources, and we don’t know what those sources are necessarily. Sometimes they’re cited, sometimes they’re not.  I asked the chatbot something the other day, and it said it was something very straightforward like how old is somebody, like famous person. And it gave me an answer, and I said, cite your sources. And it said, this is just general knowledge. I’m sorry. How is this general knowledge? It it would not and then it said, but you know, it got kind of passive-aggressive. It was like, well, if you insist, I can give you a couple of websites that list this kind of information. but essentially it was like, how dare you question my knowledge of just the general universe? Well, I I use computers for a living, so I question everything. All right, so the uninvited author is the AI, and now the chatbot is authoring content, accurately or not, whether we like it or not. So our subtitle then is AI Intent and the Future of Content. When we look at AI in the context of content, what are we looking at here? How does AI get involved in content operations, in the content process? CE: Well, we have been chatting, ha ha, we have been chatting about chatbots so far, but the chatbot, it’s really only scratching the surface of what AI already is for content operations. I don’t even want to think about right now, let’s save it for the next section of the title of what it’s going to do or what it could do. CE: And the chatbot is the easiest representation of it. And yesterday I was teaching, I’m teaching this semester a senior seminar in communication on the topic of what AI is doing to the profession of communication, for good or for bad. And something that my students tell me is that they use ChatGPT or a similar chatbot as a replacement for the Google search box. So they go there and they go ahead and ask and prompt without prompting, ask, they query things that up to two or three years ago, people would just ask Uncle Google. SO: Now, in their defense, it’s getting very difficult to actually find the search box.  CE: If you go to Google, yeah, you don’t know what’s Google, and you don’t know what’s Gemini. They’re all married and combined together. And I think that that is the very interesting part. But the chatbot in that box that resembles the Google Omni box is only one component of what we mean by AI. AI comes in many other flavors that I think my co-author in this book even has a fancy table inserted on page number here about the different forms of AI engagement in content operations. And at the end of that spectrum, we have agentic approaches to AI in which you, as the developer or you as the author of the content, are going to be sending out these AI agents to be taking care of processes or supporting processes that human beings are doing throughout the life cycle of your content.  So I want to emphasize that what we’re talking about here is not just ChatGPT, not just Gemini or even Claude, but many in-house developed and maintained in a little or not so little computer in one organization implementation of not just a large language model, but many other machine learning and data mining approaches that could create something that smells or looks like artificial intelligence beyond basic automation. So AI means many things, and we have more than one section in the book that tries to address that AI is many things, even though for the majority of people who probably are listening to this AI is ChatGPT in that box. SO: And one of I think one of the really tricky parts about this is that, as content people, we see AI in lots of different places in our in our job roles. So what I mean by that is that mean, we’ll start at the end. The content consumer, it used to be get your content on the website, do some search engine optimization, keywords, whatever, make sure that people find your content. Or you might be privileged that your content is ships with the product as online help or a companion document. Again, if it’s nineteen eighty five, but you know, it’s the supporting con

  4. Aug 31

    The limits of automation and AI in content workflows

    What are the limits of automation and AI in content systems? In this episode, Sarah O’Keefe and Bill Swallow dive into traditional formatting, workflow challenges, and what happens when AI is introduced. The limit of automation here is not really the automation. It’s the people who say, “I don’t like the way this looks, and I’m gonna find a way to fix it. I know all sorts of ways of bending this stuff to my will.” If you’re generating content at scale, you can’t afford to do any of this stuff. Related links: Digital sovereignty in the age of AI From ad hoc to autonomous: The AI content ops maturity model Subscribe to our newsletter for updates on Sarah O’Keefe and Carlos Evia’s book: The Uninvited Author LinkedIn: Sarah O’Keefe Bill Swallow Transcript: Disclaimer: This is a machine-generated transcript with edits. Introduction with ambient background music Christine Cuellar: From Scriptorium, this is Content Operations, a show that delivers industry-leading insights for global organizations. Bill Swallow: In the end, you have a unified experience so that people aren’t relearning how to engage with your content in every context you produce it. Sarah O’Keefe: Change is perceived as being risky; you have to convince me that making the change is less risky than not making the change. Alan Pringle: And at some point, you are going to have tools, technology, and processes that no longer support your needs, so if you think about that ahead of time, you’re going to be much better off. End of introduction BS: Hi, I’m Bill Swallow. SO: And I’m Sarah O’Keefe. BS: And today we’re gonna talk a bit about automation, and more specifically what the limits are for automation. SO: Yeah, they let us out of our cages. And so here we are. And I think that in this world of  I’m not going to get through the first sentence without saying AI. In this world of AI, we are suddenly facing, you know, increasing automation across every facet of everything that you could imagine. And I think it’s important to take a step back and talk a little bit about limits and what automation cannot do and maybe more importantly, why it cannot do certain things. So before we get to, you know, the AI in the room,  Bill, what are the limits when we talk about generalized publishing in the pre-AI world where we’ve done a ton of formatting automation and we talk about all those kinds of things, what are some of the limits that we’ve run up against in terms of automation? BS: Yeah, automation is something that we’ve been doing quite a bit of  for the past, what, twenty years, I think, or more. I’ve only been around for thirteen years. I’m thirteen years young,  so… SO: Yeah,  this is lies. BS: But no, a lot of our work around automation is around publishing.  That is, I think first and foremost, the most common case.  And with automation and publishing, you can save a lot of time and save a lot of effort, but a lot goes into building that capability because you can’t really work around edge cases or you know, fancy hand-cobbled formatting,  hand page breaks,  all of that thing. You have to kind of rely on the software to understand the content that you’re feeding it and it’s going to produce something that you know is a finished product: a PDF, a web page, what have you.  And you know, if you deviate from the structures and conventions that it’s expecting, then all bets are off as to whether or not your automation will be successful. SO: Right. And so, from our point of view, doing lots and lots and lots of structured content work,  i setting aside the tools and the technologies, what structured content really does is limit variability. Right? It limits the variability of the information or of the markup really going into your system, which then means in turn that the system can be automated and can automatically output all the outputs, all the deliverables, all the file formats that you need. And so  when you have weird edge cases, and I am of course the worst offender in terms of actually finding ways to bend the software to my will,  but only when I’m an author, right? As a system configuration, I’m, yeah, I think you should all follow the rules. Yeah.  So here we are, and we basically say we’re gonna limit the variability of the input and standardize the input and therefore we will get standardized output. Cool. But as you said, that only works if you don’t have so what is a weird edge case or what are some of the things you’ve run into that just bollocks up automation? BS: One good case is working with content that maybe has valid structures in place. So they’re not, you know, the content itself validates against a validator. So there’s nothing wrong with it. But they’re using, let’s say, different in this case, DITA elements in a let’s say creative way. You know, so they’re you know  I was on vacation last week, so that wasn’t me.  SO: I see you were looking over my shoulder last week again. Yeah, that’s true. Well,  what I actually ran into was I had a DITA map; it was valid, it validated, and then I used Oxygen’s validator, which you know really goes a little bit deeper and looks at things. Everything was fine, but it crashed. And eventually what I realized or what I found after some digging was that somebody who was definitely me had inserted a draft comment and the draft comment was in a table, maybe under the title, but before the table group kind of thing. It was in an unusual location and it just the processor just died. It just laid down and gave up.  now I don’t know exactly whether that was because of the, you know, the core processing or something that we did in the the  plugin that I was running. It doesn’t really matter. The point is it was it was valid. But it didn’t pass the processor because the processor was like,  Why would anybody put a draft comment in this location? This is dumb. And then I, you know, felt berated by the processor and I moved it and then it all worked. BS: Interesting. Yeah. I would have enjoyed being there for that.  SO: Yeah. Mm hmm.  So anyway, we fixed it, and it was fine, and you know, and off we go. Let’s talk about pagination! BS: Pagination’s a fun one.  SO: Pagination’s my favorite.  I get very upset with bad line breaks or bad page breaks. BS: Yeah. And you know, we can build in rules that say, you know, only, you know, to control widows and orphans, that type of thing. You know, tell it, you know, break a table leaving X many cells. If you don’t have X many cells to break, then move the entire table to the next page, all that fun stuff. But if you have, you know, specific places where you need to break to a new page for whatever reason, that always can’t be necessarily automated, depending on what the rules are for producing your output. You know, the processor’s not going to know that, you know, you may arbitrarily want to break a page at this particular location. So you know, in many cases we cobble together a little tag that says, you know, essentially break the page here,  and the processor knows to you know, when it sees that to say, okay, stop processing this page, move to the next one.  And you know, it works most of the time.  SO: I would never. Yeah, the problem with inserting page breaks, as I’ve learned to my great sorrow, is that, of course, later you add more content and now you have a page break a third of the way down the page because you hard-coded it in. So this is bad, and you shouldn’t do it. But if you insist on doing it, do it l as late in the process as possible. Don’t try to fix your pages when you’re 80% of the way there, because you will have to reinsert them over and over and over again.  also. BS: Right. SO: Inserting empty tags that have a non-breaking space in them to introduce vertical space is wrong, and you shouldn’t do it. BS: Ha ha. I will agree with that one. Also, I mean, doing these types of things to force a page break or to force extra space, it really flies in the face of automation because technically you have to create the output in order to know where you need to insert your page breaks and then go back and add them and then automate your output again. SO: Ha ha. It looks bad. BS: Yeah. SO: So yeah. So really the limit of automation here is not really the automation, right?  It’s, well, it’s me, right? It’s the people who are like, I don’t like the way this looks, and I’m gonna find a way to fix it. And I have lots more demented tricks up my sleeve. I know all sorts of ways of bending this stuff to my will. And if you’re generating content at scale, you basically can’t afford to do any of this stuff, right? It’s one thing if you’re producing, you know, one document and it’s short and/or it’s marketing content, and we’re really concerned about the appearance because of people making buying decisions. But if you’re producing, you know, ten, twenty, fifty thousand pages a year, then you just need an engine that produces this stuff. So BS: Mm-hmm. Yeah. And the page break problem is actually a good one to speak to at scale because if you’re in an environment where you’re sharing a bunch of different content and you’re reusing pieces, if you insert a page break for your own personal preference, suddenly that’s going to be in everyone else’s document that also uses that particular topic. SO: All right. So I’m hearing that page breaks are bad and I shouldn’t do them. BS: No, they’re great. It’s just that you have to be smart about it. SO: Okay. Sneak them in. Don’t get caught. Got it.  So, while we’re on the subject of recalcitrant authors. Such is definitely not me. What about automation in a scenario where your content production system is not being used by the authors? BS: Yes. That’s a completely different problem. Yeah, th

  5. Aug 3

    Digital sovereignty in the age of AI

    Are you in control of your digital destiny? In this episode, Alan Pringle and Sarah O’Keefe define digital sovereignty. They break down what organizations need to consider as they bring AI into content operations, from data leakage and competitive intelligence to shifting international regulations. Sarah O’Keefe: The working definition that I’m using is that digital sovereignty is control over your own digital assets, digital destiny. That could be you personally, it could be you as an organization, or it could be you as a country or as a group of nations. And when I say group of nations, probably 98% of the time I’m talking about the EU, which has some laws in this regard. Digital sovereignty is your ability to control, own, and manage your digital assets and how they are used, reused, processed, resold, repurposed, and all the rest of it. Related links: AI in the content lifecycle: three years later From ad hoc to autonomous: The AI content ops maturity model Upcoming AI book: The Uninvited Author LinkedIn: Alan Pringle Sarah O’Keefe Transcript: Disclaimer: This is a machine-generated transcript with edits. Introduction with ambient background music Christine Cuellar: From Scriptorium, this is Content Operations, a show that delivers industry-leading insights for global organizations. Bill Swallow: In the end, you have a unified experience so that people aren’t relearning how to engage with your content in every context you produce it. Sarah O’Keefe: Change is perceived as being risky; you have to convince me that making the change is less risky than not making the change. Alan Pringle: And at some point, you are going to have tools, technology, and processes that no longer support your needs, so if you think about that ahead of time, you’re going to be much better off. End of introduction AP: Hey everybody, I’m Alan Pringle. SO: And I’m Sarah O’Keefe, hello. AP: Hey there. And today Sarah and I want to talk about something that’s really starting to come to the forefront with all of the AI things that are going on in our world. And that is digital sovereignty. And before we get too deep in that, I need to throw up many, many disclaimers. Sarah and I are not lawyers, nor do we play them on television. And we are absolutely not lawyers or experts on anything in regard to international intellectual property. So with those disclaimers out there, Sarah, if you would, would you define what digital sovereignty is? SO: The working definition that I’m using is that digital sovereignty is control over your own digital assets, digital destiny. That could be you personally, it could be you, the organization, or it could be you as a country or as a group of nations. And when I say group of nations, probably 98% of the time I’m talking about the EU, which has some laws in this regard. So it is your ability to control, own, and manage your digital assets and how they are used, reused, processed, resold, repurposed, and all the rest of it. AP: And I’m going to give a very basic example from my personal life in regard to email. Years ago, actually decades ago, when I got set up with an internet service provider, they provided me with an email address. It had their company name in the domain.com. And I used it for years. But when I switched my ISP, guess what? I had to make a decision. Do I continue to pay that old ISP, basically rent, to maintain that old email address? Or should I move to one of maybe the free email providers? So I did a little research, and my ultimate decision was I created my own domain with my name, and I kind of just decided not to ever use again the email provided by an ISP because if you switch it, you’re going to possibly lose control of that email address. And at the time that I made the switch, a lot of the free providers, they would scan your email to provide targeted ads and some other things that was kind of unsavory to me.  So I ultimately made the decision I was going to own the domain and set that up myself and pick my own software that I hooked up to it to use to to read it. And I did not use a third-party email provider to basically pull in or reference my account. I am using an open-source standalone email client to kind of minimize who can poke into my email. So that’s one very basic example of how I kind of took control of my digital life as it were. SO: Right. And if we apply that to content ops, again, staying pretty general, you think about cloud systems, like the cloud universe versus on-prem. AP: Yes. SO: And when you talk about, let’s say, a CCMS, a component content management system that is on-premises, the the argument was always, well, that way all of our content lives on our servers, in our organization, we have complete control over it. Along come the cloud services, the cloud-based CCMSs and everything else. And they say, well, yes, but it’s much cheaper for you to put this into the cloud on our systems, which are shared. And you know, there’s advantages of upgrading, and there’s all sorts of advantages to cloud systems, mostly around IT overhead. from a digital sovereignty point of view. You are delegating to that cloud system and you have some sort of a contract or a service level agreement. And again, we are not lawyers, but you have this agreement that says we the cloud provider promise to not scan your information or not use it for evil or not, you know, there’s a bunch of stuff in that contract that governs how cloud provider is or is not allowed to handle your content, your personal information, your credit card information that you might be putting in there and all the rest of it. And with software as a service, with cloud systems, we have for the most part cut over into cloud. You know, that’s that’s kind of the default these days. There’s hardly anybody that is still putting their content, their content and their content management systems on their own proprietary in-house servers. AP: Yes, correct. SO: So we have decided that for cloud, you know, cloud writ large generally, that the advantages of cloud outweigh the disadvantages. Now, moving this a little bit more towards content and slowly towards AI, where things get really interesting with digital sovereignty, if we talk about machine translation for a minute. There are a couple of different ways of doing machine translation, obviously, but big picture, you can have your in-house machine translation system and database, and you can control that and govern it and do things with it. Or you can take your content and you can throw it at a public-facing machine translation system. And the risk that you run when you throw something at a public system is that they will take your proprietary confidential content. And use it. So I throw at it a sentence that says, the XYZ company has developed a special new thing, right? And I need that translated into various languages and I get it translated. But as a result of that, I am leaking information. I am leaking my confidential, potentially information into the machine translation services. And there are some really interesting security issues around that and how you might be able to. As a competitor, extract that back out. But just dialing it back for a for a minute, we understand the concept of leaking information by using public-facing machine translation, right? Publicly available machine translation. But one thing that I think is very often overlooked is that machine translation, the fact that I ask for a specific language, never mind the content, But the fact that I am now asking for a new language provides competitive intelligence in the sense that that means that I or my organization now cares about that locale, that that language. So let’s say that I’ve been consistently submitting European languages for machine translation, right? If you but if you look at my record of what I’m asking for, you will see that all of a sudden, about three months ago, I started adding a bunch of Asian languages. Well, what does that tell you about what I’m up to? Either I’ve decided that Asian languages are interesting and fun, or my organization, I mean, presumably I’m doing this for work and not fun, or my organization is launching into Asia. And I don’t actually need to see the content to sort of get that piece of competitive intelligence out of it. So essentially. The way that I’m using the machine translation, even if we protect or have a contract that says you you are not allowed to look at what I’m processing, but if you can look at the parameters of what I’m processing, that might give you enough information to tell you something about what I’m up to. AP: Yeah, so basically what you request or don’t request, as the case may be, can give away clues that you may not want floating out in the world. SO: Right, exactly. So now we take this to AI and we think about digital sovereignty for AI, and it gets very much more complicated. So t first, taking this example of machine translation, if you think about AI chatbots and prompting and public-facing models, then in the same way that requesting a particular language so well, let’s back up.  Let’s say that we have a contract with XYZ model provider and it says we will not use your input as training data. We will not use your output as training data. Cool. But are you going to use my prompts as competitive intelligence? Because think about what I’m prompting on. Hey, tell me about the intellectual property laws in Vietnam. Tell me about how to go to market in a particular country. Tell me about strategy for pricing tiers, right? If I’m doing those kinds of prompts, then you, as my competitor or adversary or whatever we’re dealing with here, can get an awful lot of information out of what I’m up to, right? You can figure out what I’m up to just by looking at my prompts. S

  6. Jul 13

    The debt crisis: AI edition

    What happens when you feed years of messy content into AI? In this episode, Bill Swallow and Alan Pringle dig into the content debt crisis, including increased system costs, neglected localization, and the fallout of “just use AI” mandates. They share practical insights to help organizations get back on track. Alan Pringle: Is your content updated? Does it reflect the latest information? Is it created for all the different locales that your company serves? Is it in different languages? That is another pile of debt that when you start looking at AI, all the problems will be very brutally magnified, and you’re going to have to address them to really have a large language model that works at all. Related links: Balancing automation, accuracy, and authenticity: AI in localization (podcast) Forbes: AI Costs More Than The People It Replaced Taming AI: Using AI for content conversion at scale (podcast) LinkedIn: Bill Swallow Alan Pringle Transcript: Disclaimer: This is a machine-generated transcript with edits. Introduction with ambient background music Christine Cuellar: From Scriptorium, this is Content Operations, a show that delivers industry-leading insights for global organizations. Bill Swallow: In the end, you have a unified experience so that people aren’t relearning how to engage with your content in every context you produce it. Sarah O’Keefe: Change is perceived as being risky; you have to convince me that making the change is less risky than not making the change. Alan Pringle: And at some point, you are going to have tools, technology, and processes that no longer support your needs, so if you think about that ahead of time, you’re going to be much better off. End of introduction Bill Swallow: Hi everybody, I’m Bill Swallow. Alan Pringle: And I’m Alan Pringle. BS: And today’s episode is going to focus more on the content debt crisis, AI edition. AP: And it’s also going to be the complaint edition, surprise, surprise, because there’s a lot of things in the AI world right now that are still making me cranky. And I am sure we will talk about them at some point. BS: Yeah. So with the rise of AI, I think it’s kind of holding a microscope to a lot of the technical debt that we’ve been seeing over the years in content operations in general, whether you have outdated authoring formats or content that’s not being updated on a regular basis, new delivery formats not necessarily meeting the needs of the users, and so forth. And all of that is kind of compiling or snowballing into a bigger problem once you start feeding all of this stuff into AI. AP: Right. And it’s interesting to me how everyone’s talking about AI as being this productivity tool. In a lot of ways it is, but that’s not what the focus of this is. In a way, it is also sort of a consultant for you. As you just mentioned, Bill, when you start looking at AI and delivering it, treating it as a delivery endpoint for your content, a distribution endpoint, you are going to start to discover that your processes on the back end for creating and distributing your content are not what they should be. So it’s kind of like this consultant saying, Hey, you need to do better over here. And that is where a lot of this debt is coming from, from my point of view. BS: Mm-hmm. The unfortunate part of that consultant is that it’s not offering advice on how to fix it, but it certainly is pointing out the issues. AP: It’s like, “This is screwed up. Full stop.” So, I mean, part of why we’re here is to talk about some of those kinds of debt. And let’s just start with one technical debt. And I’m saying technical in the sense of the way that you perhaps use software to put together your content. Let’s kind of focus on the content operations world. BS: Mm-hmm. AP: For example, if you are delivering content via unstructured desktop authoring tools, of which there are many, and you can templatize things and make your content seem more consistent, but there’s a problem with a lot of desktop published or content generated from the desktop publishing world. It’s more focused on look and feel and fit and finish, particularly if you’re delivering for PDF. And yes, people are still doing that. So there’s a lot of time and effort spent on that look and feel, that fit and finish. And frankly, that time should have been invested in adding intelligence to the content to explain, you know, things under the covers about what user is this for? What is the model of this particular thing? All of that kind of metadata, that kind of categorization. Desktop publishing, at least from my point of view, doesn’t really do a great job of helping you catalog that kind of stuff. So that’s a problem. BS: No. It is a problem. Also, with desktop publishing, you can kind of confuse AI a bit if you’re using desktop publishing inconsistently. So if you’re using formatting tools to override formatting to make things look like headings or make certain paragraphs look like children of another paragraph, doing those manual finesses is great for print because, as you know, most people will look at that and understand the hierarchy of information, understand what’s going on with the content that they’re reading. But anything digital isn’t necessarily going to pick that up.  AP: Right. Yeah. BS: Especially if you’re looking at something like bare bones HTML, if you’re using CSS to override the size and prominence of a standard paragraph as opposed to using a heading, that is not necessarily going to be picked up as a heading, even though a reader would actually see that as a heading while looking at the HTML page. AP: Right. What a human reader can figure out from formatting cues, if those cues aren’t set up in a way that a large language model, a computer, can understand, there’s that huge disconnect, and that’s where that debt starts piling up. And then another angle here in this content creation world, if you are using multiple different tools to create your content, there’s a good chance that content under the covers is not going to be processed the same by a large language model. So there’s another deficiency right there on top of that. And this is very common, for example, if you have had mergers, acquisitions, and you’ve basically created a larger company from many different companies. And they all still have, especially in legacy content, things created the “old way,” and all the old ways start to pile up and cause problems because your large language model can’t properly basically figure out what is the heading in this particular chunk of docs versus what it is over here. So it can’t parse it as well. And again, this all goes all the way back to the way that you created that content. And it’s a clue, hey, you need you need to fix this. And I I think beyond that more technical, the way that you create it, there also there are also issues with the content itself. BS: Mm-hmm. AP: Is it updated? Does it reflect the latest information? Is it created for all the different locales that your company serves? Is it in different languages? That is another pile of debt that when you start looking at AI, it’s gonna be all the problems in regard to that are also gonna be just very brutally magnified, and you’re going to have to address them to really have a large language model that works at all. BS: Yeah, because not only do you have the technical debt on the source content side and on the published content side, but you also now have technical debt growing on the AI side because you need to spend more time and energy refining the how that model works with your content in order to achieve the correct results. AP: Right. So you’re having to do a lot of overrides and we I don’t know if overrides is the right word, but you’re having to do a lot of additional processing and figuring out so it will parse things correctly. And there are a lot of companies today that still have problems keeping content updated to the latest and greatest. So unfortunately, people turn around and call support. Or today they start hitting up the chatbot. But guess what?  BS: Mm-hmm. AP: If the chatbot doesn’t have access to the latest and greatest because frankly it doesn’t exist or it’s not hooked up to it. It’s it’s just like the poor people in support. It’s not going to know what to do and it’s going to spit out probably very authoritatively wrong outdated information. Again, yeah, it’s be it’s goes all the way back to BS: That’s yeah. AP: In your content creation process, how are you accounting for updates? How quickly are you getting them in place? How are you handling them in both your source language and how are you handling them in your other languages? It’s one thing I do want to bring up here, and and and I may be biased here, but I don’t think localization is getting enough attention on the AI distribution side. It’s been talked about for a very long time in regard to machine translation, AI assisted translation.  BS: Mm-hmm. AP: But I don’t just like I think sometimes localization is a second thought for a lot of companies, which still blows my mind in 2026. And by the way, we’re recording this in July 2026. So what we say right now may be outdated next month. Who knows?  BS: Who knows? AP: There is that issue. I’m curious, you have a more of a localization background than I do, but I do see that being a potential debt problem, part of this debt crisis that we’re talking about here. BS: Definitely, because if you’re pushing your content out to AI, do you have targeted audiences in mind? Or is it going to be a free-for-all of people going in and using that AI to get answers to their questions? if you are localizing your content, I think probably the best practice here would be to somehow bundle for consumption all of the guides in all of th

  7. Jun 22

    From ad hoc to autonomous: The AI content ops maturity model

    There are five levels of maturity for AI-driven content operations. Which level are you in? In this episode, Sarah O’Keefe and Bill Swallow walk through the AI content ops maturity model, from ad hoc experimentation to fully autonomous workflows. Sarah O’Keefe: We want this automation, right? We want the ability to go in and extract release notes and do something with them. We have to have a certain level of maturity on the software development process so that we can grab the appropriate information. The same thing is true on the content side. You have to have a certain level of maturity in your content development processes, in your content management, so that you can identify the right things to process and the right things to access. Related links: Want to know more about Sarah’s nifty little side project? Register for our upcoming webinar. AI in the content lifecycle Enterprise content strategy maturity model LinkedIn: Sarah O’Keefe Bill Swallow Transcript: Introduction with ambient background music Christine Cuellar: From Scriptorium, this is Content Operations, a show that delivers industry-leading insights for global organizations. Bill Swallow: In the end, you have a unified experience so that people aren’t relearning how to engage with your content in every context you produce it. Sarah O’Keefe: Change is perceived as being risky; you have to convince me that making the change is less risky than not making the change. Alan Pringle: And at some point, you are going to have tools, technology, and processes that no longer support your needs, so if you think about that ahead of time, you’re going to be much better off. End of introduction Bill Swallow: I am Bill Swallow. Sarah O’Keefe: And I’m Sarah O’Keefe. BS: And today we’re going to talk about AI in content operations, or more specifically, a maturity model for AI. SO: Everything needs a maturity model, even AI. BS: Even me. SO: I have no comment. BS: My maturity model is written in crayon, what can I say? So okay, so we need a maturity model for AI as far as content operations are concerned, and probably in you know, many different degrees, but we’ll focus on content operations. So what might that look like? SO: I’ve been thinking about this and what it looks like to employ AI as a tool to help you with content. And as I was thinking about what this looks like, you know, you always fall back on that standard five-step model where one is basically mass chaos, and five is the perfect world, generally. also, one is nearly always cheap, and five is nearly always expensive enterprise things. But, you know, let’s go a little beyond mass chaos versus governed, regulated, etc., and sort of sort of back up a little bit and talk about what this might look like. So level one in every maturity model typically is ad hoc. And what that means is that in this case, AI is being used sporadically by some people. It’s inconsistent. And I would say that when we look at AI and content specifically, BS: Mm-hmm. SO: This is going to be things like reprocessing your content using public-facing models. So I wrote a draft of something, I shove it into ChatGPT and I ask it to shorten it or tighten it up or identify areas that are problematic. Or I just say, hey, you know, write my article for me. The outcome that you’re gonna get on an ad hoc model is going to depend on an ad hoc level one. BS: Mm-hmm. SO: AI thing is going to depend on how good you are on the individual’s expertise and their level of interest. So if you want to just go in there and say, hey, I have a bio and it’s too long, and I’ve been asked to produce one that’s only 50 words for a particular conference, for example, then, you know, this is this is actually a really good example of ad hoc, right?  BS: Mm-hmm. SO: We have these long multi-paragraph bios and every conference I’ve been to has a different requirement for how that bio needs to be shaped. And the fastest way to success is to just shove it into a chatbot and say, give me a 50-word version. And and then read it and make sure it didn’t invent things or give you a PhD or anything like that, and then ship it off to the conference organizer. But this is much, much faster than rewriting it from scratch by hand. And also I think, it’s a good example of something where I have the extended version and I’m going to summarize down. And that usually works pretty well. So level one is ad hoc. It’s kind of sporadic. There’s no standard across the organization. It’s just me saying, this looks useful, or you’ve probably got some use cases in this space as well. BS: Right. So it it kind of aligns with I guess level one of the the content maturity model that we talked about a while back, where level one is is simply content exists. Could be, you know, someone typing stuff up in Word or, you know, using a myriad of different tools, no style guide, just kind of getting content out there because people need it. SO: Yep. So level two is tactical. And tactical is sort of like we’re using this tool to solve some specific problems. And what you’re going to see here is something like that Bill has invented some nifty time saving tool and he has shared it with other people. Or in a larger organization, maybe somebody invented a nifty validate or or something like that and they’ve rolled it out across maybe the department, probably not the entire organization. Aomething like AI support is being rolled out. Maybe the organization has created a chatbot internally for customers, right? So there’s a chatbot, it’s sitting on the company website, people can use it to get answers, but it’s really bad. And the reason it’s really bad is because nobody thought too carefully about the content going into the chatbot, because again, we’re tactical. So probably this looked like the AI team just raided the local SharePoint, grabbed a bunch of content, did not pay a whole lot of attention to the question of whether this content was up to date in release status. Those things don’t exist, right? It’s just, look, a bucket of PDFs. Cool. Let’s dump them into the AI and go for it. BS: Mm-hmm. SO: And tragically, in many cases, the techcomm team is sitting on rigorous, structured, vetted, approved content, and nobody remembered to go ask them, can we have your content? Or where is your official content source? Or how do I know what version belongs with which document? BS: Right, because you know, in in their point of view there’s a PDF of it, so I don’t need to ask them. SO: Yeah, it’s it’s just PDF. How hard could it be? So something like AI was support was rolled out, but nobody really thought about it. Maybe it’s at a departmental level, probably it’s not enterprise-wide. And nobody has really thought about connecting this AI thing to the assets inside the organization in a reasonable, rigorous, governed, organized kind of manner. BS: And I suppose that’s where you get to the next tier. SO: Right. So the next tier after tactical comes strategic, right? So we have an actual strategy. Now, one of the difficulties in talking about AI is that AI is a tool and it’s kind of like talking about electricity. You can apply it to lots of places and it’s more sensible in some places than others. But when we say what’s your AI strategy, like how do you use water? I mean, come on, and the answer is of course to drive the AI and you know destroy the environment. But there are things that you can do with AI that are useful for content. There are also things that you can do that are not. So if you have a strategic approach to this, a strategic approach to use of AI, backing up to the authors again rather than the delivery side, maybe this looks like a collection of prompts that have been built that are shared. BS: Not with electricity. SO: Maybe this looks like saying this is the workflow that you employ. These are the kinds of things that we do to actually test whether this thing is working. these are the metrics that we’re following. So there’s an actual overarching bigger picture that somebody’s thinking about that goes beyond, let me go shove this into chatbot of the day. BS: Mm-hmm. Right, right. SO: So there’s an actual strategy for the public-facing chatbots. Somebody has thought about the back end. The authors have useful AI tools that add to their you know their productivity. One of the things that I’m hearing a lot now, you know, low-hanging fruit, release notes. Nobody wants to write release notes. It’s a terrible drudge task. It’s and it needs to be done. Well, BS: Mm-hmm. SO: There’s now there are now a lot of solutions that look like look at the diff in the code, look at the delta from you know version one to version one dot one, find the diff in the code, find the changes that have been made, look at the JIRA tickets that have been addressed, that have been solved in release one dot one, and then consolidate that all into a set of release notes that say, here’s what’s been done. And that’s probably 90% of the work, and the last 10% of the work is read that and make sure it’s accurate. Right? Don’t please don’t skip that step. Like actually look at what the thing is generating. Now, what’s interesting to me about level three, this sort of more strategic approach, is that what you’re gonna start to see is that you have prerequisites for this. You can’t do this.  BS: Yes. SO: So release notes are good example. Let’s say that hypothetically, and this is gonna sound insane, but let’s say that hypothetically, you have software development and you have no source control. BS: Hmm. SO: Everybody’s screaming, right? Because this is nuts, and why would you ever do this? Okay. But hypothetically, you have no source control. Okay. How do you know what’s changed between version one and version one point one? BS: It’

  8. Jun 1

    Tool selection and the unpredictable variable

    How do you really choose the right documentation tool? In this podcast episode, Sarah O’Keefe (Scriptorium) talks with Paweł Kowaluk and Michał Skowron (Guidewire Software) about building a successful tool selection process, the realities of docs as code, and what happens when the technology becomes the unpredictable variable. Paweł Kowaluk: It’s funny how programming used to be deterministic, and it was the people who were messy. We always knew that people are going to be whimsical and maybe harder to rein in, but the technology is going to be predictable. Whereas now, technology is not predictable anymore, and you give it a prompt and you hope it’s going to do what you want. You adjust the system prompts and change the weight of things which are retrieved versus metadata, et cetera, and it doesn’t always work the way you expect it to. Sarah O’Keefe: And now the people are being asked to be the deterministic layer, right? To be the QA on top of the AI. Paweł Kowaluk: That’s actually very insightful. I like that. That is true. The human in the loop or whatever you call it, that’s supposed to be the voice of reason. Related links: Scriptorium: AI in the content lifecycle Tech Writer Koduje podcast Tech Writer Koduje: DITA as code – a modern approach to the classic standard Tech Writer Koduje: Are people abandoning docs as code? Tech Writer Koduje: A tech writing CCMS can also be a broken promise LinkedIn: Host: Sarah O’Keefe Guest: Paweł Kowaluk Guest: Michał Skowron Tech Writer Koduje LinkedIn profile Transcript: Introduction with ambient background music Christine Cuellar: From Scriptorium, this is Content Operations, a show that delivers industry-leading insights for global organizations. Bill Swallow: In the end, you have a unified experience so that people aren’t relearning how to engage with your content in every context you produce it. Sarah O’Keefe: Change is perceived as being risky; you have to convince me that making the change is less risky than not making the change. Alan Pringle: And at some point, you are going to have tools, technology, and processes that no longer support your needs, so if you think about that ahead of time, you’re going to be much better off. End of introduction Sarah O’Keefe: Hey, everyone. I’m Sarah O’Keefe, and welcome to the podcast. In this episode, we are going to talk about tool selection with a couple of special guests. With me today are Paweł Kowaluk, who is a software architect at Guidewire Software, and Michał Skowron, who is a documentation tools developer, also at Guidewire. Both of them are based in Poland. Welcome. Paweł Kowaluk: Hi. Michał Skowron: Hello. SO: I am glad to have you. For those of you on this podcast that speak Polish, you’re probably already aware that they have the one and only techcomm podcast in Polish that is available out there, and Michał and Paweł are also experts on doc process and tool selection, so that’s what we wanted to focus on today. So I will start and throw it to Michał and ask you the big picture question, which is what does a good tool selection process actually look like? MS: For me, good selection tool process would be divided in three stages. The first one would be gathering requirements, looking what’s out there, defining what you want to basically achieve with this new tool. Then I would go to a pilot project where you can actually test the selected tool in the real world. Manufacturers and producers of software will tell you that it can do anything and it will promise that, “Okay, you can meet all your requirements easily and we can fix that, we can improve that, we can adjust that,” so everything can be done is usually what we hear, but then you want to test it in real world on a real project, so that will be a pilot project for you and your team. And the third phase that depends on the outcome of the second phase, which is you either productize the selected solution or you just say, “Okay, that was a bad choice and we don’t need that.” Then we need to go back to the first stage and then say, “Okay, we need to select another tool,” and again, requirements, et cetera, et cetera. So for me, that’s the whole process, and the first stage would be probably the longest one because you need to make sure that you are meeting all your goals. SO: So what’s the most common reason that a pilot doesn’t succeed, that you have to go back and say, “That didn’t work. We have to try something different”? MS: It’s usually because you didn’t see everything when you were planning. For example, you have some projects that are very specific or you didn’t see all the problems or things that are coming your way. It’s hard to say exactly what the reason is, but it can be multiple reasons. For example, using of, I don’t know, branching, let’s say, in a specific tool. When you have multiple versions of your product and you want to keep them separate when it comes to documentation, it can turn out that the feature says, “Okay, you can use branching and then you can do it easily,” and then you start using it and it turns out that it doesn’t work the way you expect it. This is actually a real life example because we had a system that… I’m not going to mention any names or anything like that, but there was a system and they promised us… That was years ago and it was a vendor that promised us that they’re going to introduce a feature called branching, and it turned out after they did that that it wasn’t what we expected. So it can turn out in many cases, in many ways, it can be the problem, but branching is just an example, but it can be many other things that can go wrong. PK: Hey, if I can jump in here, I got a couple of examples. One is I could call it releasing strategy or versioning strategy overall, which is very hard to test in a pilot project. It’s very hard to scope for requirements because the little problems come out after a while, after a year of publishing, after two years of publishing. And another example which is related is reuse, and this one is down to formulating the requirements correctly. Because I think just saying, “We want to reuse something,” is not enough, because you have to say exactly what you want to reuse and how you want to reuse it and what you want the result to be. So for example, if you say, “I want to reuse notes and warnings and things like that.” We sometimes call them admonition. So, “I want to reuse these notes in my docs, and if I update a note, I want it to update in every published version of the doc.” Then only if you have these details, like I want to update it once and then want it to automatically update and publish docs, then you will see it’s not working the way you expect it. Because if you just say, “I want to reuse notes,” every system can reuse notes. Even in docs as code, there’s scripts and macros that allow you to reuse notes. MS: It can be also another thing that, for example, you compare the benefits with the actual cost of implementation, and it can turn out it’s not worth it because people are, for example, reluctant to use your new tool. The training, the cost of licensing, the cost of support is too big, and then you realize, okay, we want to achieve a goal like, I don’t know, reduce the time to market, and then it turns out it doesn’t work because people are struggling with using the product on a real project. And on paper, everything looks cool and you have all these features, you can use them, like Paweł mentioned, for example, reuse conditional formatting and things like that. And then it turns out it’s very hard to use, it doesn’t serve its purpose, and then you have to pay for every additional stuff and people don’t want to use it, so what’s the point? SO: Yeah. And I think we find that the technical problems in general, if you’ve done your requirements work, then usually, the technical problems are solvable if the people engaged in the project want to solve them, and that’s where you run into the change management issues that you’re talking about, that if the team that is being asked to pilot, to test, to try things out is sufficiently disinterested in making it succeed, they will find a way to make it fail. And the reverse is true as well. If they want it to succeed, you can implement tools that are … Well, all tools are imperfect, but you can implement tools that are not perfect solutions and succeed if the team is behind you, and if they’re not, bad, bad things will happen. PK: Oh yeah, that’s true. And I’ve been on projects where we did not do proper change management and I’ve been on projects where we really did it well. If you start early and you involve people, like get the biggest troublemakers, people who are the most opposed to any change, get them on the team, and if you can convince them, that means, one, you are making the right choice because you’re convincing people who are skeptical, and then two, you are set up for success. These people are going to be your biggest champions of the new solution. MS: But it’s good that you mentioned it because I think it’s worth emphasizing that the goal of the pilot project is not to succeed. That’s not the actual goal. The goal is to verify the requirements against the real project, and so the failure is also a success to some extent. It’s not like you have to do everything to prove that the selected tool is the right one. No, you should be aware that the pilot can end up with your let’s call it failure, and then you realize that it’s either a bad choice or you don’t need that at all. For example, you don’t need that tool at all. It can be also the outcome of the pilot project. So there are many different outcomes, so don’t be fixated on the success path that is the only right way. It means, okay, the pilot project was a su

See All (205)

Ratings & Reviews

4.3
out of 5
7 Ratings

About

The Content Operations podcast from Scriptorium delivers industry-leading insights for scalable, global, AI-optimized content.

Information

  • Creator
    Scriptorium - The Content Strategy Experts
  • Years Active
    2017 - 2026
  • Episodes
    205
  • Rating
    Clean
  • Copyright
    © Scriptorium Publishing Services, Inc.
  • Show Website
    Content Operations