DataRobot's Venky Veeraraghavan On Agentic AI ROI

At the AI Agent Conference in New York this past week, I sat in on a session that reframed how I think about enterprise AI deployment. Venky Veeraraghavan, Chief Product Officer at DataRobot, gave one of the more technically grounded talks of the two-day event — and the central argument was this: the reason enterprise AI fails to deliver ROI has almost nothing to do with models, prompts, or frameworks. It has everything to do with what he called "everything below the waterline."

What Is the "Below the Waterline" Problem?

The metaphor is simple and it stuck. When you look at a ship, you see what's above the water. The hull, the deck, the structure that matters visually. What makes the ship actually function — ballast, propulsion, structural integrity — is invisible.

Enterprise AI works the same way. The demo is above the waterline. Everyone can build one. The infrastructure that makes a demo into a production system — identity, authorization, orchestration, observability, token economics, deployment architecture — is entirely below it.

"The demo works great," Veeraraghavan said. "Running it in production all day is much harder. The hard part is not the demo."

The problem isn't that organizations don't know this work exists. It's that none of it is value-additive on its own. Businesses don't care about auth systems or observability pipelines. They care about ROI, productivity, and automation outcomes. But those outcomes are impossible without the infrastructure. Every piece of below-the-waterline work is a gating condition — invisible, non-differentiating, and absolutely mandatory.

"None of this is value-add by itself," he said. And yet without it, none of the actual value gets delivered.

Agent Identity: The Problem Nobody Has Solved Yet

The section of Veeraraghavan's talk that generated the most attention in the room was on agent identity. He called it plainly: "The most important problem is identity."

Here's the core issue. Traditional enterprise authorization systems were designed for human users. A person logs in, has a defined role, gets a set of permissions. Clean, auditable, understood. Agents break every assumption in that model.

Agents act autonomously. They access systems independently. They perform actions on behalf of users — but they're not the user. They may need to aggregate data from multiple systems, combine information from sources that individual users could access separately, and derive new outputs from that synthesis. That creates governance and compliance problems that IAM systems simply weren't designed to handle.

Veeraraghavan raised the questions every enterprise deploying agents needs to answer: Who is the agent acting as? What can it access? What permissions should it inherit — and what should it specifically not inherit? Can it act independently of the user? How is that access audited?

"It's not enough to just pass through the user API key," he said. "The user may see something the agent should not."

That last line is the one that should concern compliance teams. Authorization isn't just about what the user can see — it's about what the agent can derive, synthesize, or surface from combinations of data that individually seem permissioned but together create exposure.

Authorization Gets More Complex With Every Integration

Related to identity is the broader authorization problem, which compounds as agents become more capable and more integrated.

Agents increasingly orchestrate across multiple enterprise systems — CRM, ERP, HR platforms, financial data, communication tools. Each integration adds a new surface area for permission ambiguity. An agent that legitimately accesses three separate systems and combines their outputs may be creating information that no individual in the organization is supposed to see in aggregate.

"Authorization is not just what the user can see," Veeraraghavan noted. This is the kind of problem that gets discovered post-deployment during a compliance audit, not during development. Building scoped, workflow-level authorization from the start is harder and slower — but it's the work that has to happen before enterprise organizations can deploy agents at scale with confidence.

Developer Experience Is Workflow Engineering, Not Prompting

Veeraraghavan made a point about developer experience that challenges how most AI teams are organized. Building enterprise-grade agents is primarily workflow engineering, platform engineering, and operational architecture. It is not prompting.

"The hard part is making agents enterprise-ready. You have to think about workflows, authorization, and operations." He was direct about the failure mode: "If you do it all yourself, you'll get poor results."

The implication is that the teams winning at enterprise AI deployment aren't necessarily the ones with the best prompt engineers or the most sophisticated model configurations. They're the ones who've built or adopted the operational infrastructure that makes agents reliable, governable, and scalable. That's a different hiring profile, a different investment priority, and a different definition of technical excellence than most AI teams currently have.

Token Economics Will Determine Who Can Scale

The third major pillar Veeraraghavan covered was cost infrastructure — specifically, token economics and workload routing.

"The third pillar is cost. Not every task needs the most expensive model."

This is obvious once stated but frequently ignored in practice. Production AI systems handle a wide range of task types: classification, summarization, orchestration, reasoning, lightweight automation. These have vastly different latency requirements, accuracy requirements, and economic profiles. Routing all of them through the same large, expensive model is neither economically sensible nor technically necessary.

"You need workload-aware model selection. You need flexibility." The production architecture that actually scales uses dynamic routing — matching task type to the appropriate model based on what the task actually requires. That means combining hosted proprietary models, open-source alternatives, and specialized fine-tuned models in a hybrid stack.

"You probably don't want one model for everything. Different workloads require different models. You need architectural flexibility."

For organizations currently running all their AI workloads through a single API, this is where significant cost reduction lives — alongside significant architectural complexity that has to be built and maintained.

The Operational Infrastructure Is the Product

Veeraraghavan closed with a philosophical point that applies beyond just AI engineering: the visible experience sits on top of invisible operational systems, and those invisible systems determine everything that actually matters — reliability, scale, ROI, governance, adoption.

"The platform fails without the infrastructure. The real work happens below the waterline."

This is the reframing that I think most organizations need. Enterprise AI is not primarily an AI problem. It's an operational engineering problem where AI is the capability layer and infrastructure is the foundation. Organizations that invest heavily in models and prompts without building the operational foundation beneath them will keep getting impressive demos that don't make it to production.

The question Veeraraghavan was really asking the room: how much of your AI investment is above the waterline, and how much is below it?

Venky Veeraraghavan presented at the AI Agent Conference 2026 in New York. He is Chief Product Officer at DataRobot.