All Episodes

"The agent did it" won't hold up in court: Santosh Kaveti on AI agent accountability for founders

with Santosh Kaveti · ProArch

September 16, 202600:52:44Atlanta, GA

"The agent did it" won't hold up in court: Santosh Kaveti on AI agent accountability for founders

0:000:00

Skillz from the Episode

Agent Roll Call

Builds a living register of every AI agent, copilot and AI-enabled vendor feature in your company, assigns each one a named human owner, and scores its risk. It then runs a trace-back test on your riskiest agents to show whether you could explain their decisions.

When to use it: Use it when you have no clear answer to “what AI is running in my business?” It's also the right move before an insurance renewal, a board meeting, a security questionnaire, or a federal contract bid.

Benefits: You get an inventory that matches where the Stop Rogue AI Act is heading, a ranked fix list for your biggest gaps, and an answer ready for the day a customer, insurer, or judge asks why an agent did what it did. Save the download as agent-roll-call/SKILL.md in your Claude or ChatGPT skills folder, then run it before someone else asks first.

Click to steal this skill 👇

Show Notes

Funding stage: Growth. ProArch took a $25 million debt and equity growth investment from MSouth Capital in February 2022, after roughly 16 self-funded years, and now employs somewhere between 400 and 500 people — a mature, late-stage company well past the range a Series A label usually describes, so it's tracked here as growth.

Congress wants a roll call of your AI agents. Could you produce one by Friday?

In early September, Reps. Josh Gottheimer and Mike Lawler introduced the Stop Rogue AI Act. The bill would have NIST write standards for tracking and logging AI agents, including a live, machine-readable inventory of every agent on a network. Meanwhile, carriers like Berkshire Hathaway, Chubb, Travelers and AIG have been writing AI exclusions into standard commercial policies. The era of “give everyone a Claude seat and see what happens” is getting expensive.

Santosh Kaveti has a sharper take on that roll call. He is the founder and CEO of ProArch, the Atlanta firm he built over two decades to bring cloud, data, security and AI into a single conversation. In his view, a list of agents tells you very little. The question that matters is whether you can trace any AI decision back to why it happened. “The agent did it” will not survive a courtroom or a customer call.

His fix sounds almost old-fashioned: treat agents like people. Give each one an ID, onboard it, monitor it, and put a named human on the hook for what it does. Then baseline your risk across infrastructure, applications, data and AI. That includes the agents your ERP vendor quietly shipped last quarter that nobody on your team evaluated.

Santosh and Ryan also get into why the “AI will kill coding” story keeps missing. Santosh is watching great programmers turn into 100X AI workflow engineers. Teams without checks and balances, on the other hand, are piling up AI slop that costs millions to unwind. He explains why a services firm pours money into R&D, pointing out that end of life is written on every MSP and MSSP. He also shares why ProArch hires for curiosity and for people who solve problems beyond their own silo.

Then it gets personal. Santosh protects six to seven hours of sleep and treats tonight's crisis as tomorrow's problem. He grew up around entrepreneurs on both sides of his family, who taught him to build conviction, do the work and stop waiting on results. He grows turmeric on his farm in India and bought a farmhouse in Ellijay, Georgia last year. His next dream is mountain land in Costa Rica planted with coffee and mangoes.

If one of your systems made a bad call tomorrow, could you explain it? Press play before someone asks.

Q&A: What Founders Ask After This Episode

Who is Santosh Kaveti?

Santosh Kaveti is the founder and CEO of ProArch, an Atlanta-based technology firm that helps power producers, manufacturers, healthcare organizations and software companies secure and scale their cloud, data, security and AI. He also founded the Yashoda Foundation, a volunteer-led nonprofit supporting underprivileged children and families.

What is the Stop Rogue AI Act and what does it mean for startups?

It's a bipartisan House bill from Reps. Josh Gottheimer and Mike Lawler that would have NIST create standards for deploying AI agents, including live agent inventories and tamper-proof logs. Santosh's argument is that traceability of AI decisions matters more than the list itself.

Do I need to inventory my AI agents now if the bill isn't law yet?

Yes. The Stop Rogue AI Act would make NIST standards voluntary for most companies, with federal contractors pushed to comply. Insurers and enterprise customers are already asking the same questions, though, and Santosh says very few companies could produce an accurate list today. Starting now is cheap. Starting after an incident is not.

Who is accountable when an AI agent makes a bad call?

A person. Until AGI arrives, Santosh says a human in the loop who is accountable is a must. That person should question the agent's recommendation, check the data it used, and be able to explain the decision. “The agent did it” should never be an acceptable answer.

How do I secure AI without slowing my company down?

Baseline your risk across infrastructure, applications, data and AI, then build on that foundation. Santosh argues security becomes your differentiator and business enabler rather than a tax. That baseline should include the AI features your existing vendors have added, which most teams never evaluate.

Is AI going to replace my engineers?

Santosh doesn't see it happening in practice. The best programmers are becoming AI workflow engineers who design systems that produce quality code at 10X to 100X speed. Teams that adopt coding agents without checks and balances pile up technical debt and AI slop that can cost millions to fix.

How do I stay sane running a company at this pace?

Treat problems as challenges to solve rather than threats, protect six to seven hours of sleep, and build a routine that separates you from the day. Santosh's rule is that whatever happened today is tomorrow's problem. Taking care of your body feeds your mental health too.

URLs Mentioned in the Episode

  • ProArchhttps://www.proarch.com

    Santosh Kaveti's Atlanta-based technology firm bringing cloud, data, security, and AI into one practice.

  • Santosh Kaveti on LinkedInhttps://www.linkedin.com/in/santoshkaveti/

    Connect with this episode's guest.

  • Loftie smart alarm clockhttps://byloftie.com

    The alarm clock Santosh credits with protecting his six to seven hours of sleep.

  • Stop Rogue AI Act announcementhttps://gottheimer.house.gov/posts/release-gottheimer-introduces-bipartisan-bill-to-stop-rogue-ai-agents-and-keep-people-in-control

    The bipartisan House bill that would direct NIST to set AI agent tracking and logging standards.

  • Ryan Estes on LinkedInhttps://www.linkedin.com/in/estesryan/

    Connect with the host.

  • AI for Foundershttps://aiforfounders.co

    The newsletter and podcast this episode comes from.

  • Inbox Alchemyhttps://inboxalchemy.co/

    Ryan's tool for building a newsletter audience you own instead of renting from a platform.

  • CEOfriendhttps://bit.ly/3SrjgL1

    Free competitor analysis from this episode's AI advisory board sponsor.

  • AI Native Studenthttps://ainativestudent.com/

    AI skills training for job seekers, this episode's sponsor.

  • Momentoushttps://crrnt.app/MOME/8RDrnXDd

    35% off any supplement subscription, this episode's sponsor.

  • Taelorhttps://taelor.style/

    An AI men's fashion stylist and this episode's sponsor; use code RYAN30 for $30 off.

  • Get Howdihttps://gethowdi.com/

    An autonomous AI sponsorship agent for podcasts, this episode's sponsor.

Named Frameworks

The Trace-Back Test

How to check whether an AI decision can survive scrutiny.

  • →For any consequential AI decision, ask what it decided and why.
  • →Go to the data point or source the agent relied on.
  • →Check whether the decision passes a human gut check.
  • →If the only answer is “the agent said so,” the system fails.

Onboard Agents Like Hires

Santosh's governance model for AI agents.

  • →Every agent gets an ID.
  • →Every agent goes through onboarding, just like a new employee.
  • →Every agent is monitored the way a person would be.
  • →Every agent has a named, accountable human owner.
  • →A list of agents is the starting point, not the finish line.

The Four-Layer Risk Baseline

Where to start securing AI.

  • →Infrastructure risk: where is your security maturity today?
  • →Application risk: including the AI your vendors bolted on.
  • →Data risk: quality, ownership, and protection.
  • →AI risk: agentic and non-agentic systems alike.

The Amplifier Principle

Why AI incidents hit harder than pre-AI ones.

  • →AI magnifies whatever already exists in your company, bad and ugly included.
  • →Pre-AI incidents were usually survivable, but Santosh expects AI incidents to be far harder to recover from.

Security as the Enabler

Reframing security from cost center to advantage.

  • →A solid, secure foundation removes a large share of AI risk before you build anything.
  • →Companies that take the thoughtful path are already standing out from competitors.

The 100X Workflow Engineer

What actually happens to great programmers under AI.

  • →The core skill shifts from writing code to designing the system that writes it.
  • →Experienced programmers who embrace AI become more valuable.
  • →Output scales 10X, 50X, or 100X when the workflow is designed well.
  • →Without checks and balances, AI code turns into expensive technical debt.

One Room, One Lens

How ProArch structures its own teams.

  • →Bring IT, the CISO's office, and business owners into a single conversation.
  • →Innovation teams mix security, app dev, data, and cloud people so nothing becomes an afterthought.

Outrun the Disruption Curve

Why a services firm keeps reinvesting in itself.

  • →Santosh says end of life is written on every MSP and MSSP, so ProArch invests heavily in R&D.
  • →The goal is to thrive on disruption and measure AI by ROI rather than license counts or agent counts.

Tomorrow's Problem

Santosh's rule for staying sane at this pace.

  • →End every day, good or ugly, by detaching from it.
  • →Tonight's crisis belongs to tomorrow's version of you.
  • →A simple routine helps: family dinner, non-work conversation, a show before bed.
  • →Aim for six to seven hours of sleep. Sleep debt comes back to haunt you.

Conviction Without Expectation

The entrepreneurial lesson from his family.

  • →Build conviction and go after it.
  • →Keep doing the work without demanding results.
  • →Stay persistent and consistent.
  • →Share what you have.

Glossary

Agentic AI

AI systems that take actions and make decisions, not just generate content.

AI agent

A software system that performs tasks with limited or no human supervision. Santosh notes there is no universally accepted legal definition.

Agentic sprawl

Agents multiplying across a company faster than anyone can track or govern them.

Human in the loop (HITL)

A person who reviews, approves, or can override an AI system's decisions.

Traceability

The ability to follow an AI decision back to the data and logic behind it.

Auditability

The ability for someone else to inspect and verify how an AI system behaved.

Stop Rogue AI Act

A 2026 bipartisan House bill directing NIST to set security standards for deploying AI agents.

NIST

National Institute of Standards and Technology, the federal standards body within the Commerce Department.

CISA

Cybersecurity and Infrastructure Security Agency.

Machine-readable inventory

A continuously updated agent list that software can query.

Tamper-proof log

An activity record that can't be quietly altered after the fact.

AI exclusion

Insurance policy language that removes coverage for AI-related claims.

ERP

Enterprise resource planning software that runs core operations such as finance and supply chain.

ISV

Independent software vendor, a company that builds and sells software products.

MSP / MSSP

Managed service provider / managed security service provider.

CISO

Chief information security officer.

AGI

Artificial general intelligence, AI that matches humans across most tasks.

Technical debt

The future cost of shortcuts taken in code today.

AI slop

Low-quality AI-generated output that creates cleanup work.

Copilot licenses

Seats for Microsoft 365 Copilot, which Santosh says are a weak proxy for AI ROI.

AI workflow engineer

A developer who designs AI systems that produce code and other output.