AI Security Ops

Black Hills Information Security

Join in on weekly podcasts that aim to illuminate how AI transforms cybersecurity—exploring emerging threats, tools, and trends—while equipping viewers with knowledge they can use practically (e.g., for secure coding or business risk mitigation).

  1. 3d ago ·  Video

    Interview with Josh Mason | Episode 62

    In this episode of BHIS Presents: AI Security Ops, Bronwen Aker and Ethan sit down with Josh Mason for a wide-ranging conversation about cybersecurity careers, AI, small business security, and what it actually takes to help organizations make practical security decisions. How do small businesses think about security when they do not have a full-time CISO? And what happens when AI starts lowering the barrier for research, planning, governance, and security operations? Not hypothetically. Not as a buzzword. But in the real world — where companies are trying to understand SOC 2, HIPAA, incident response, cyber insurance, AI governance, hallucinations, and the risks of letting tools make decisions they do not fully understand. Josh brings a practical perspective from his background as a C-130 pilot, cyber leader, instructor, sales engineer, vCISO, consultant, and founder of Noob Village at DEF CON. We dig into:- What Noob Village is and why DEF CON needs an on-ramp for new people- Josh’s path from Air Force pilot to cyber leadership- Why communication and translation matter so much in cybersecurity- What a vCISO actually does for small businesses- How smaller companies think through SOC 2, HIPAA, GRC, pen testing, and incident response- How AI can speed up research, planning, and draft creation- Why AI-generated work still needs human review and source validation- How companies are trying to govern employee use of AI tools- Why cyber insurance, E&O coverage, and AI hallucinations are starting to overlap- Where RAG and guardrails can help reduce risk- How AI may reshape the work small businesses can do on their own- Why trust, relationships, and human judgment still matter in consulting- How hacker community, mentorship, and D&D all somehow fit together This episode explores a practical shift in AI security: AI is not just changing the tools defenders use. It is changing how small businesses learn, make decisions, evaluate risk, and decide when they need expert help. The takeaway: AI can make security work more accessible, but it does not replace experience, judgment, validation, or trust. The organizations that benefit most are the ones that use AI to accelerate good decisions — not outsource thinking entirely. Chapters(00:00) - Meet Josh Mason (01:27) - Hacker Summer Camp and Noob Village (06:45) - From Air Force Pilot to Cyber Leadership (13:08) - What a vCISO Does for Small Businesses (19:55) - Using AI for Research and Incident Response Planning (24:46) - Small Business AI Security and Governance (27:47) - Cyber Insurance, Hallucinations, and Guardrails (31:43) - How AI Is Reshaping Small Business Security (42:15) - Where to Find Josh Click here to watch this episode on YouTube. Creators & Guests Ethan Robish - Guest Bronwen Aker - Host Josh Mason - Guest Brought to you by: Black Hills Information Security  https://www.blackhillsinfosec.com ☯️ Introducing BHIS Fusion Penetration Testinghttps://www.blackhillsinfosec.com/fusion-penetration-testing/ Antisyphon Training https://www.antisyphontraining.com/ Active Countermeasures https://www.activecountermeasures.com Wild West Hackin Fest https://wildwesthackinfest.com 🔗 Register for FREE Infosec Webcasts, Anti-casts & Summitshttps://poweredbybhis.com Click here to view the episode transcript.

    Interview with Josh Mason | Episode 62

  2. Jul 10 ·  Video

    Are Foreign Open Weight Models a Security Risk? | Episode 61

    In this episode of AI Security Ops, the team tackles one of the most common questions security teams are asking about open-weight AI models: Are foreign open-weight models actually a security risk? Not in the vague “AI is scary” sense. Not in the headline-driven “it must be spyware” sense. But in the practical, security-operations sense: if you download a model like Qwen or DeepSeek and run it locally, what risks are real, which ones are overblown, and what should defenders actually care about? The answer is more nuanced than “ban them” or “they’re totally fine.” Open-weight models can be cheap, capable, and private when they run on your own hardware. But “open-weight” does not mean “open source,” and running a foreign model locally does not automatically mean it is phoning home. The bigger risks are often in the runtime, file format, download source, tooling chain, model behavior, and how much trust you place in the output. We dig into:- What “open-weight” actually means, and why it is not the same as open source- Why the “phone home” fear is usually the wrong threat model for local weights- The difference between a hosted AI service and a locally run model- Why model delivery, runtime, and tooling matter more than the weights themselves- How pickle files, unsafe formats, and poisoned packages create real supply-chain risk- Why typosquatting and fake model repos are a practical concern- Why safetensors and verified sources matter- How bias and censorship can show up in foreign and domestic models- Why model behavior, refusals, and blind spots can become integrity risks- What sleeper-agent research tells us about hidden triggers and model backdoors- Why country of origin matters, but does not replace basic security hygiene- How to safely evaluate and use open-weight models in real workflows This episode explores a critical shift in AI security: the risk is not just where a model comes from. It is how you download it, how you run it, what data it can access, what actions it can take, and whether your pipeline assumes the output is trustworthy. For security teams, the practical takeaway is simple: do not treat any model as inherently safe just because it runs locally, and do not treat every foreign model as magic spyware. Build the workflow so the model can be useful without becoming a single point of trust. — Key Concepts & Topics Open-Weight Models- Local model weights and inference engines- Open-weight versus open source- Qwen, DeepSeek, and foreign model adoption Threat Modeling- Local models versus hosted AI services- The difference between weights, wrappers, and APIs- Why “phoning home” is usually a runtime or tooling issue Supply-Chain Risk- Unsafe model formats- Pickle files and arbitrary code execution- Typosquatting and poisoned repositories- Package and dependency compromise Safer Model Handling- Prefer safetensors over risky serialized formats- Download from verified sources- Pin hashes and validate model artifacts- Use containers and restrict unnecessary network access Bias and Censorship- Model behavior shaped by training data- Political, cultural, and regulatory influence- Refusals, blind spots, and subtle output bias- Matching model behavior to the use case Sleeper Agents and Backdoors- Hidden trigger behavior in model outputs- Why behavioral testing may miss certain risks- The difference between lab demonstrations and real-world evidence- Designing workflows so hidden triggers have limited impact Defensive Strategy- Treat model output as untrusted input- Do not pipe outputs directly into shells, databases, or production systems- Avoid unsupervised code execution or autonomous production access- Make adoption decisions based on threat model, compliance, and use case Learn more about Black Hills Information Security:https://www.blackhillsinfosec.com/ Check out Antisyphon Training:https://www.antisyphontraining.com/ #AISecurity #CyberSecurity #LLMSecurity #ArtificialIntelligence #InfoSec #BHIS #Antisyphon #OpenWeightModels #SupplyChainSecurity (00:00) - Intro: Foreign Open-Weight Models and Security Risk (01:50) - What Open-Weight Actually Means (03:35) - The Phone Home Concern (07:44) - Pickle Files and Supply-Chain Risk (14:34) - Bias, Censorship, and Model Behavior (19:21) - Sleeper Agents and Hidden Triggers (24:41) - Country of Origin vs Security Practices (25:20) - Practical Checklist and Final Takeaways Click here to watch this episode on YouTube. Creators & Guests Brian Fehrman - Host Ethan Robish - Guest Derek Banks - Host Brought to you by: Black Hills Information Security  https://www.blackhillsinfosec.com ☯️ Introducing BHIS Fusion Penetration Testinghttps://www.blackhillsinfosec.com/fusion-penetration-testing/ Antisyphon Training https://www.antisyphontraining.com/ Active Countermeasures https://www.activecountermeasures.com Wild West Hackin Fest https://wildwesthackinfest.com 🔗 Register for FREE Infosec Webcasts, Anti-casts & Summitshttps://poweredbybhis.com Click here to view the episode transcript.

    Are Foreign Open Weight Models a Security Risk? | Episode 61

  3. Jul 3 ·  Video

    Hey Skippy! | Episode 60

    This episode takes a break from the usual AI security news roundup for a show-and-tell discussion centered on "Skippy," an AI-powered personal assistant built to automate cybersecurity workflows. The conversation covers how the project evolved from an OpenClaw experiment into a system that tracks AI and cybersecurity news, generates daily intelligence briefs, documents its own code, recommends training updates, assists with content creation, and performs automated vulnerability research. The hosts also discuss practical AI workflows, prompt engineering, model selection, and lessons learned from integrating LLMs into day-to-day security operations. Key Concepts and Topics * The origin and evolution of the "Skippy" AI assistant* Building an AI agent with OpenClaw and Telegram* Automating AI and cybersecurity news aggregation* Daily intelligence briefs and trend analysis* Self-documenting AI-assisted software development* Personalizing AI behavior with custom instructions* AI-assisted content creation and documentation* Identifying training and course update opportunities* Automated vulnerability research against open-source projects* Comparing open and commercial LLMs for security workflows (00:00) - Intro - Show and Tell (00:55) - Introducing My Bot Skippy! (03:56) - Why the name Skippy? (06:55) - The Skippy Dashboard (09:00) - Questions about Skippy? (09:57) - Other Features, customization and automation (16:28) - AI creates more work for you to do (20:33) - What can we do next? Click here to watch this episode on YouTube. Creators & Guests Derek Banks - Host Brian Fehrman - Host Bronwen Aker - Host Brought to you by: Black Hills Information Security  https://www.blackhillsinfosec.com ☯️ Introducing BHIS Fusion Penetration Testinghttps://www.blackhillsinfosec.com/fusion-penetration-testing/ Antisyphon Training https://www.antisyphontraining.com/ Active Countermeasures https://www.activecountermeasures.com Wild West Hackin Fest https://wildwesthackinfest.com 🔗 Register for FREE Infosec Webcasts, Anti-casts & Summitshttps://poweredbybhis.com Click here to view the episode transcript.

    Hey Skippy! | Episode 60

  4. Jun 26 ·  Video

    Mythos and Fable Pulled | Episode 59

    In this episode of BHIS Presents: AI Security Ops, the team tackles a first-of-its-kind moment in AI security and regulation: What happens when the U.S. government orders a company to pull its most powerful AI models off the market? Not the chips. Not the infrastructure. The models themselves. On June 12th, 2026, Anthropic disabled Fable-5 and Mythos-5 worldwide after receiving a federal export-control directive tied to foreign-national access. The models were only three days old, and the shutdown raises a much bigger question for security teams, builders, and defenders: Are frontier AI models now controlled technology? This episode breaks down the order, the export-control mechanism behind it, the cybersecurity concerns around jailbreaks, and what this means for anyone building security workflows on top of hosted AI models. We dig into:• Why Anthropic pulled Fable-5 and Mythos-5 for all customers• How foreign-national access rules forced an all-or-nothing shutdown• What EAR export controls are, and why ITAR keeps coming up• The history of encryption, PGP, and software as controlled technology• Why Fable-5 and Mythos-5 triggered cyberweapon concerns• The difference between guarded and less-guarded model releases• Why jailbreaks are central to the government’s justification• Why “all LLMs can be jailbroken” matters for policy and enforcement• Whether Anthropic’s safety messaging created regulatory risk• How competition and AI industry politics may shape regulation• Why model redundancy is becoming a security resilience requirement• What security teams should learn from a hosted model disappearing overnight• Why taking powerful AI away from defenders may make security worse, not better This episode explores a critical shift in AI security: frontier models are no longer just another SaaS dependency. They are becoming part of the security supply chain, subject to policy, export controls, national-security concerns, and sudden access loss. For security teams, the question is no longer just which model performs best. It is what happens when the model your workflow depends on disappears, and what that model could see while it was running. — Key Concepts & Topics AI Export Controls• Federal action targeting AI models instead of chips• Foreign-national access restrictions• Frontier models as controlled technology EAR, ITAR, and Software Regulation• Dual-use technology under Commerce Department authority• Historical parallels to encryption and PGP• Why software can become a national-security control point Fable-5 and Mythos-5• Guarded and less-guarded model access• Safety classifiers and cyber capability concerns• Public release versus vetted access models Jailbreaks and AI Security• Bypassing model safeguards• Universal versus narrow jailbreaks• Why perfect jailbreak resistance is not realistic Security Resilience• Model redundancy as a practical requirement• Avoiding single-model dependency• Planning for sudden access loss, policy changes, and vendor shutdowns Defensive Strategy• Understanding where AI lives in your workflows• Thinking through AI blast radius• Balancing model capability, access, monitoring, and risk Learn more about Black Hills Information Security:https://www.blackhillsinfosec.com/ Introducing BHIS Fusion Penetration Testinghttps://www.blackhillsinfosec.com/fusion-penetration-testing/ Check out Antisyphon Training:https://www.antisyphontraining.com/ #AISecurity #CyberSecurity #LLMSecurity #ArtificialIntelligence #InfoSec #BHIS #Antisyphon #AIRegulation #ExportControls ----------------------------------------------------------------------------------------------🎧 Subscribe to the Podcast:https://aisecurityops.transistor.fm About Brian Fehrman - https://www.blackhillsinfosec.com/team/brian-fehrman/About Bronwen Aker - https://www.blackhillsinfosec.com/team/bronwen-aker/About Derek Banks - https://www.blackhillsinfosec.com/team/derek-banks/About Ethan Robish - https://www.blackhillsinfosec.com/team/ethan-robish/About Ben Bowman - https://www.blackhillsinfosec.com/team/ben-bowman/ (00:00) - Intro: The First AI Model Export Control (01:38) - The Anthropic Order and Foreign-National Access (03:19) - EAR, ITAR, and Software as Controlled Technology (04:39) - Mythos-5, Fable-5, and Guarded Model Access (06:32) - Jailbreaks and Cyberweapon Concerns (08:58) - Competition, Regulation, and AI Industry Politics (10:54) - Model Redundancy as a Security Requirement (13:21) - Defensive AI Use and Final Takeaways Click here to watch this episode on YouTube. Creators & Guests Brian Fehrman - Host Bronwen Aker - Host Brought to you by: Black Hills Information Security  https://www.blackhillsinfosec.com ☯️ Introducing BHIS Fusion Penetration Testinghttps://www.blackhillsinfosec.com/fusion-penetration-testing/ Antisyphon Training https://www.antisyphontraining.com/ Active Countermeasures https://www.activecountermeasures.com Wild West Hackin Fest https://wildwesthackinfest.com 🔗 Register for FREE Infosec Webcasts, Anti-casts & Summitshttps://poweredbybhis.com Click here to view the episode transcript.

    Mythos and Fable Pulled | Episode 59

  5. Jun 20 ·  Video

    Agentic Security: The Maturity Model — From Wild West to Locked Down | Episode 58

    In this episode of BHIS Presents: AI Security Ops, the team tackles one of the most urgent — and misunderstood — problems in modern security: How do you actually secure AI agents? Not hypothetically. Not in theory. But in the real world — where agents have access to your filesystem, your credentials, your network… and are making decisions on their own. The answer isn’t a single control or tool — it’s a maturity model. From “YOLO agent with full access” to fully instrumented, controlled, and observable systems, this episode walks through a five-level maturity model for agentic security — and what it actually takes to move up each stage. We dig into:• Why agentic AI introduces a completely different security model• What “Level 0” chaos looks like in real organizations• The risks of giving agents unrestricted access to systems• Why containment is the first real step toward security• How sandboxing changes the risk equation• The importance of logging, monitoring, and visibility• Where most organizations are actually operating today• Why skipping steps in maturity creates hidden risk• How to think about blast radius in agent design• What “fully enforced” agentic security actually looks like This episode explores a critical shift in AI security: you’re not just securing models anymore — you’re securing autonomous systems. ⸻ 📚 Key Concepts & Topics Agentic Security• AI agents with system-level access• Autonomous decision-making and execution• Expanding attack surface beyond prompts Security Maturity Model• Level 0 → Level 4 progression• Incremental risk reduction strategies• Why maturity matters more than tools Containment & Sandboxing• Limiting blast radius• Isolating agent execution environments• Preventing lateral movement Monitoring & Observability• Logging agent actions and decisions• Detecting misuse or unexpected behavior• Building visibility into autonomous systems Defensive Strategy• Designing for least privilege• Avoiding “full access by default”• Treating agents like untrusted users #AISecurity #CyberSecurity #AIAgents #LLMSecurity #ArtificialIntelligence #InfoSec #BHIS #AppSec #AgenticAI----------------------------------------------------------------------------------------------About Brian Fehrman - https://www.blackhillsinfosec.com/team/brian-fehrman/About Bronwen Aker - https://www.blackhillsinfosec.com/team/bronwen-aker/About Derek Banks - https://www.blackhillsinfosec.com/team/derek-banks/About Ethan Robish - https://www.blackhillsinfosec.com/team/ethan-robish/About Ben Bowman - https://www.blackhillsinfosec.com/team/ben-bowman/ (00:00) - Intro: The Reality of Unsecured AI Agents (00:24) - The Agentic Security Maturity Model Explained (07:20) - Level 0: Total Chaos (Unrestricted Agents) (11:24) - Level 1: Containment and Basic Guardrails (13:24) - Level 2: Controlled Execution (20:32) - Level 3: Monitoring, Logging, and Visibility (27:00) - Level 4: Fully Enforced Agent Security (28:00) - Final Takeaways: Maturity Over Hype Click here to watch this episode on YouTube. Creators & Guests Bronwen Aker - Host Brian Fehrman - Host Derek Banks - Host Ethan Robish - Guest Brought to you by: Black Hills Information Security  https://www.blackhillsinfosec.com ☯️ Introducing BHIS Fusion Penetration Testinghttps://www.blackhillsinfosec.com/fusion-penetration-testing/ Antisyphon Training https://www.antisyphontraining.com/ Active Countermeasures https://www.activecountermeasures.com Wild West Hackin Fest https://wildwesthackinfest.com 🔗 Register for FREE Infosec Webcasts, Anti-casts & Summitshttps://poweredbybhis.com Click here to view the episode transcript.

    Agentic Security: The Maturity Model — From Wild West to Locked Down | Episode 58

  6. Jun 19 ·  Bonus Video

    Introducing Fusion AI Pentest | Episode 57

    In this episode of BHIS Presents: AI Security Ops, the team introduces a new approach to offensive security: Fusion AI Pentesting.https://www.blackhillsinfosec.com/fusion-penetration-testing/ As AI continues to reshape cybersecurity, one question keeps coming up — is AI replacing pentesters, or just changing how they work? This episode answers that directly. Rather than replacing human expertise, Fusion combines AI-driven discovery with human-led validation and exploitation, creating a workflow that’s faster, more scalable, and far more effective than either approach alone. The result isn’t just more findings — it’s better findings, faster, with real-world impact. We dig into:• What “Fusion AI Pentesting” actually means in practice• Why AI alone isn’t enough for real security testing• How human + AI collaboration outperforms either independently• The difference between finding vulnerabilities and proving impact• Where AI excels in offensive security workflows• Where human intuition and experience still matter most• How this approach scales continuous testing and red teaming• Why traditional pentesting models are starting to break down• How organizations should think about integrating AI into security testing• What this means for the future of offensive security This episode highlights a key shift in cybersecurity: AI doesn’t replace the pentester — it changes what a great pentester looks like. ⸻ 📚 Key Concepts & Topics Fusion AI Pentesting• Combining AI discovery with human validation• Augmenting—not replacing—pentesters• Faster, more scalable offensive workflows AI in Offensive Security• Automated vulnerability discovery• Pattern matching vs real-world exploitation• Limits of AI-only approaches Human + AI Collaboration• Human intuition and domain expertise• Chaining vulnerabilities for real impact• Validating and prioritizing findings Security Testing Evolution• Continuous testing vs point-in-time pentests• Red teaming with AI-assisted workflows• Changing expectations for coverage and speed Defensive Implications• Better signal vs noise in findings• Faster identification of real risk• Preparing for AI-augmented attackers #AISecurity #CyberSecurity #Pentesting #ArtificialIntelligence #LLMSecurity #InfoSec #BHIS #RedTeaming #AIAgents ----------------------------------------------------------------------------------------------About Melisa Wachs - https://www.blackhillsinfosec.com/team/melisa-wachsAbout Brian Fehrman - https://www.blackhillsinfosec.com/team/brian-fehrman/About Bronwen Aker - https://www.blackhillsinfosec.com/team/bronwen-aker/About Derek Banks - https://www.blackhillsinfosec.com/team/derek-banks/About Ethan Robish - https://www.blackhillsinfosec.com/team/ethan-robish/About Ben Bowman - https://www.blackhillsinfosec.com/team/ben-bowman/ (00:00) - Intro: A Different Kind of AI Sec Ops Episode (01:59) - Introducing Fusion AI Pentesting (03:34) - Why AI Alone Isn’t Enough (05:59) - Human vs AI: Strengths and Limitations (09:12) - Finding vs Exploiting Vulnerabilities (11:43) - How Fusion Improves Speed and Coverage (15:06) - Scaling Offensive Security with AI (18:12) - Final Takeaways: The Future of Pentesting Click here to watch this episode on YouTube. Creators & Guests Brian Fehrman - Host Derek Banks - Host Melisa Wachs - Guest Brought to you by: Black Hills Information Security  https://www.blackhillsinfosec.com Antisyphon Training https://www.antisyphontraining.com/ Active Countermeasures https://www.activecountermeasures.com Wild West Hackin Fest https://wildwesthackinfest.com 🔗 Register for FREE Infosec Webcasts, Anti-casts & Summitshttps://poweredbybhis.com Click here to view the episode transcript.

    Introducing Fusion AI Pentest | Episode 57

  7. Jun 13 ·  Video

    Open Weight Models and Open Source Harnesses | Episode 56

    In this episode of BHIS Presents: AI Security Ops, the team looks at what it actually means to own your AI stack. Open-weight models and open-source harnesses are no longer just lab toys. They are becoming practical options for security teams that care about where their prompts, code, client data, findings, and tooling actually live. The core question: when your work depends on AI, how much control are you willing to give away? We dig into:- What data sovereignty means for security teams- Why token sovereignty matters in agentic workflows- How provider terms can become a business risk- Open-weight models vs. truly open-source AI- Why harnesses like Hermes and OpenCode matter- Where cloud providers may apply fewer restrictions- The tradeoff between local control and hosted capability- Supply chain risk in models, harnesses, and plugins- Running local models with Ollama, VLLM, and similar tools- Why “local” does not automatically mean “safe”- How to start experimenting without buying expensive hardware- The next risk frontier: local prompt injection Owning your AI stack does not magically eliminate risk. It moves the risk. Hosted models create exposure around data, terms, pricing, and availability. Local models create exposure around maintenance, supply chain, permissions, and prompt injection. The security win is not blindly choosing local or cloud — it is knowing which layer you need to control, and why. ⸻ 📚 Key Concepts & Topics Data & Terms Risk- Prompts can contain code, client data, findings, and operational context- Hosted providers may inspect, retain, or restrict usage- Terms changes can affect entire security workflows- “Allowed yesterday” does not guarantee “allowed tomorrow” Token Sovereignty- Agentic workflows burn far more tokens than simple chat- Rate limits, usage windows, and pricing changes become operational dependencies- Local hardware shifts the constraint from API quota to compute capacity- Cost control is part of architecture, not just procurement Models vs. Harnesses- Open-weight models provide downloadable weights, not always full training transparency- Harnesses provide the tool loop, permissions, memory, and provider adapters- Hermes, OpenCode, Claude Code, Codex, and similar tools shape what the model can actually do- Risk often lives in the harness around the model Local Stack Tradeoffs- Local models improve control over sensitive data- Self-hosting adds maintenance, patching, networking, and monitoring responsibilities- Tools like Ollama, VLLM, and Llama.cpp lower the barrier to experimentation- Expensive hardware helps, but it is not required to start learning Supply Chain & Prompt Injection- Model weights, plugins, skills, and MCP servers are all supply chain decisions- Local agents with shell access can turn prompt injection into local impact- “No provider guardrails” means you own the safety controls- Permissions, sandboxing, and audit logs matter more as the stack gets more autonomous Practical Starting Point- Pick one harness and go deep before chasing every new tool- Test real tasks, not toy demos- Compare hosted and local workflows honestly- Decide which layers you need to own before you need an emergency exit #AISecurity #LLMSecurity #CyberSecurity #ArtificialIntelligence #OpenSourceAI #LocalLLM #AIAgents #SecOps #InfoSec #BHIS #AppSec #PromptInjection #SecurityArchitecture ----------------------------------------------------------------------------------------------About Brian Fehrman - https://www.blackhillsinfosec.com/team/brian-fehrman/About Bronwen Aker - https://www.blackhillsinfosec.com/team/bronwen-aker/About Derek Banks - https://www.blackhillsinfosec.com/team/derek-banks/About Ethan Robish - https://www.blackhillsinfosec.com/team/ethan-robish/About Ben Bowman - https://www.blackhillsinfosec.com/team/ben-bowman/ (00:00) - Intro: Owning Your AI Stack (01:43) - Data Sovereignty, Token Sovereignty & Terms Risk (03:38) - Provider Inspection, Prompt Data & Business Exposure (08:09) - Where the Guardrails Live: Model, Harness, or API (12:12) - Open Weights, Frontier Providers & the Innovation Race (14:53) - Local Models, Open Harnesses & Real Hardware Tradeoffs (24:24) - Self-Hosting Reality: VLLM, Ollama, VPNs & Maintenance (31:25) - Getting Started: Pick a Harness and Run Real Tasks Click here to watch this episode on YouTube. Creators & Guests Bronwen Aker - Host Derek Banks - Host Ethan Robish - Guest Brought to you by: Black Hills Information Security  https://www.blackhillsinfosec.com Antisyphon Training https://www.antisyphontraining.com/ Active Countermeasures https://www.activecountermeasures.com Wild West Hackin Fest https://wildwesthackinfest.com 🔗 Register for FREE Infosec Webcasts, Anti-casts & Summitshttps://poweredbybhis.com Click here to view the episode transcript.

    Open Weight Models and Open Source Harnesses | Episode 56

  8. Jun 4 ·  Video

    AI Cost Saving Tips | Episode 55

    In this episode of BHIS Presents: AI Security Ops, the team digs into a problem every AI-enabled SOC eventually hits: The demo looked great — until the inference bill showed up! AI in SecOps gets expensive because security data is huge, repetitive, and constant. Logs, alerts, runbooks, tool definitions, and historical context all get pushed into models again and again. That burns money, slows systems down, and often makes answers worse. The fix is not exotic. It is basic engineering: use smaller models where they work, cache what repeats, stop dumping raw logs, and save expensive reasoning for the cases that actually need it. We dig into:• Why AI SecOps workloads get expensive fast  • When smaller models are good enough  • Where frontier models still make sense  • How grouping alerts into cases reduces waste  • Using strong models to judge cheaper models  • Why prompt caching can be a major cost lever  • How small prompt changes can break caching  • Batch APIs for non-urgent security work  • Why raw logs make prompts noisy and expensive  • RAG, deduplication, and cached verdicts  • Budget caps, circuit breakers, and stolen-key risk  • When deterministic code beats another model call  AI cost control is not just a budgeting exercise. It is a security architecture issue. If every alert goes to the biggest model with no caching, no limits, and no measurement, the system is not just expensive — it is uncontrolled. Good AI SecOps design means scoping the model, reducing unnecessary context, measuring spend, and putting guardrails around how AI is allowed to operate. ⸻ 📚 Key Concepts & Topics AI Cost Architecture  • SecOps cost comes from large inputs, repeated context, and high alert volume  • Model selection should match task difficulty  • Routine triage can often use smaller models  • Hard correlation and judgment may justify stronger models  Model Evaluation  • Test smaller models against real historical cases  • Use stronger models as judges when appropriate  • Compare quality before moving workloads  • Do not assume the biggest model is always necessary  Prompt & Context Design  • Cache static instructions, tool definitions, and repeated context  • Keep cacheable sections stable  • Avoid changing static prompts with unnecessary variables  • Better prompt structure can reduce both cost and noise  Data Reduction & Retrieval  • Do not send entire logs when only a few fields matter  • Preprocess alerts before model calls  • Use RAG instead of stuffing whole libraries into prompts  • Cache repeated verdicts for repeated alert patterns  Operational Guardrails  • Track AI spend by workload  • Set hard caps and circuit breakers  • Use limits to reduce stolen-key blast radius  • Treat AI pipelines like production security systems  Deterministic Workflows  • Not every task needs inference  • Repeatable logic should become code  • AI can help write that code  • Once the workflow is deterministic, stop paying the model to repeat it  #AISecurity #LLMSecurity #CyberSecurity #ArtificialIntelligence #SecOps #SOC #InfoSec #BHIS #AppSec #PromptEngineering #securityarchitecture ----------------------------------------------------------------------------------------------About Brian Fehrman - https://www.blackhillsinfosec.com/team/brian-fehrman/About Bronwen Aker - https://www.blackhillsinfosec.com/team/bronwen-aker/About Derek Banks - https://www.blackhillsinfosec.com/team/derek-banks/About Ethan Robish - https://www.blackhillsinfosec.com/team/ethan-robish/About Ben Bowman - https://www.blackhillsinfosec.com/team/ben-bowman/ (00:00) - Intro: When the AI Triage Assistant Gets Expensive (01:27) - The Setup: Saving Money Without Killing the Workflow (02:22) - Right-Size the Model: Cheap for Routine, Big for Hard (05:36) - Testing Smaller Models, Judges & Real SOC Workflows (13:46) - Prompt Caching: The Big Lever Hiding in Plain Sight (18:37) - Batch APIs: Half the Urgency, Lower the Cost (20:19) - Stop Dumping Logs: Less Noise, Better Answers (24:20) - RAG, Dedupe, Budgets & the Deterministic Code Bonus Click here to watch this episode on YouTube. Creators & Guests Ethan Robish - Guest Derek Banks - Host Brian Fehrman - Host Brought to you by: Black Hills Information Security  https://www.blackhillsinfosec.com Antisyphon Training https://www.antisyphontraining.com/ Active Countermeasures https://www.activecountermeasures.com Wild West Hackin Fest https://wildwesthackinfest.com 🔗 Register for FREE Infosec Webcasts, Anti-casts & Summitshttps://poweredbybhis.com Click here to view the episode transcript.

    AI Cost Saving Tips | Episode 55
See All (63)

Trailer

2.1
out of 5
13 Ratings

About

Join in on weekly podcasts that aim to illuminate how AI transforms cybersecurity—exploring emerging threats, tools, and trends—while equipping viewers with knowledge they can use practically (e.g., for secure coding or business risk mitigation).

Information

  • Creator
    Black Hills Information Security
  • Years Active
    2025 - 2026
  • Episodes
    63
  • Rating
    Clean
  • Copyright
    © 2025 Black Hills Information Security
  • Show Website
    AI Security Ops

You Might Also Like