In the rapidly evolving landscape of enterprise technology, the last 12 months have fundamentally altered the blueprint for enterprise architecture. At the Dell Technologies World conference in Las Vegas, John Roese, Dell's global chief technology officer, shared insights into how agentic artificial intelligence (AI) is driving this transformation. Instead of treating AI as a single-task tool like a chatbot, enterprises are now delegating complex objectives to autonomous AI agents that can plan, execute, and refine tasks without human intervention.

Roese emphasized that agentic AI represents a paradigm shift from earlier generative AI applications. 'We have shifted our assumption that the use of AI is no longer a one-shot task like a chatbot,' he said. 'It's about handing objectives to the AI system, and that's what agents do today.' He pointed to Google's redesigned search engine as a prime example, where an agentic framework gathers information, processes it, and presents a fully compiled page to the user. This seamless experience—where the human becomes an instructor rather than a doer—is causing enterprises to abandon their old generative AI use cases and rebuild them as agentic workflows.

The GPU training myth

One of the most persistent misconceptions in the enterprise AI space is the belief that companies need massive clusters of graphics processing units (GPUs) to get value from AI. Roese debunked this myth, explaining that the infrastructure needs of most enterprises are vastly different from those of hyperscalers like AWS, Microsoft Azure, or Google Cloud. 'The myth out there is that enterprises need thousands of GPUs,' he said. 'Our biggest workload inside of Dell only sits on 16 GPUs and supports 40,000 people. You don't need thousands of GPUs in an enterprise because for each workload, agent, or project you only need a handful, sometimes half a GPU.'

The reason is that the vast majority of enterprise AI workloads involve inference—the process of running a trained model to make predictions or generate responses—rather than training new models from scratch. 'For agents, you only need inference,' Roese explained. 'There's no training for agents.' This realization is reshaping procurement strategies and infrastructure planning across industries. However, even inference workloads are evolving. When chatbots were the primary use case, they placed a very light load on central processing units (CPUs). But AI agents require external tools, communication protocols, and knowledge graphs—components that do not naturally live in the GPU. 'When you move to agentic, it's almost balanced,' said Roese. 'The number of CPUs and GPUs are very similar. For every two GPUs you have a CPU. You don't just build an AI infrastructure with a pile of GPUs—you build it with GPUs and traditional CPU compute.'

Air-gapped frontier models and the edge

Another major shift is how powerful AI models are being deployed. A year ago, the most capable frontier models—those with billions of parameters—were only accessible through cloud-based application programming interfaces (APIs). Now, hyperscalers are enabling those same models to run on-premise through services like Google Distributed Cloud and AWS Outposts. Roese noted that enterprises have multiple topology options: consuming models in a virtual private cloud, deploying them in their own data centers, or even air-gapping them from external networks for security and compliance. 'We didn't have any of those options, except the API one, a year ago,' he said.

Simultaneously, AI is moving to the edge in a structured and scalable way. The emergence of agentic frameworks such as OpenClaw, which run natively on devices and AI PCs, is enabling real-time inference without relying on cloud connectivity. Roese described this development as 'incredibly powerful, and not a fad.' Edge AI agents can process data locally, reducing latency and bandwidth costs while enabling use cases in manufacturing, retail, healthcare, and autonomous systems.

Re-architecting the data layer

Data management strategies are also evolving in lockstep with agentic AI. Traditional storage systems bolted onto AI compute clusters are no longer sufficient to meet the performance demands of agents that need to access vector databases, graph databases, and data annotation tools in real time. These knowledge and context layers cannot sit isolated; they must be deeply integrated into the compute fabric. Roese warned that one of the biggest performance bottlenecks is data movement. 'The GPUs you're paying for are sitting idle, waiting for data,' he said.

To address this, Dell's AI data platform is now directly plumbed into Nvidia's Cuda-X interfaces, allowing data layer services to operate at GPU speed. This tight coupling reduces latency and ensures that agents can access the information they need without delay. Enterprises are also investing in data pipelines that preprocess, index, and annotate data to make it agent-ready. The shift from batch processing to real-time data flows is critical for agentic AI to function effectively.

Mastering tokenomics and model routing

With multiple model deployment options available at different pricing mechanisms, IT leaders must now manage the cost of AI consumption—a discipline Roese calls 'tokenomics.' Although the cost per token (the unit of input/output for AI models) is expected to decline over time, the overall cost of AI will not get cheaper because usage will scale exponentially. 'There's no path where it becomes cheaper to do AI,' he said. Enterprises must therefore treat AI workloads as an arbitrage game, routing each task to the most cost-effective model.

For example, in specification-driven development—where AI writes software based on a markdown document—a single agentic framework can spawn dozens of coding tasks. If all those tasks are blindly sent to top-tier frontier models like GPT-4 or Claude, the bill can skyrocket. However, with intelligent model routing, enterprises can ensure that complex planning and architecture tasks go to expensive frontier models, while routine coding and testing tasks are handled by smaller, on-premise open-source models where energy is the only operational cost. 'Building a piece of software and doing spec-driven development might have four or five different economic paths to ultimately get to the best overall economic efficiency,' said Roese. Mastering model routing, he added, will be a competitive differentiator that lowers the cost of product development.

The human element

Despite all the technology changes, the hardest part of operationalizing agentic AI remains the human element. Roese described the traditional human job as a 'container of work' that includes hygiene tasks, productivity tasks, coordination tasks, and expert tasks. AI agents cannot perform an entire job, but they are highly capable of executing specific types of work within that container. Dell audited 6,400 jobs across its own business to understand how agents would impact its workforce. 'The first thing we realized is every single job in the company is going to change,' Roese said. 'I'm taking work out of the job and removing stuff from the container. If the container is now only half full, do I need half the number of people, or do I expand that by half? Am I able to do more expert work?'

The impact of AI on the workplace is so profound that change management has become a key remit of IT leadership. Roese noted that for the past four months, he has spent 50% of his time dealing with human dynamics. 'AI has ceased being a technology and an ROI discussion. It's now very much an organizational and human dynamic discussion. You simply can't use these things unless you fully understand how you're going to adapt the human population around them.' This requires upskilling programs, transparent communication about role changes, and a cultural shift toward continuous learning. Enterprises that neglect the human dimension risk resistance, burnout, and failed implementations.

The rise of agentic AI is also influencing how vendors design their products. Dell, for instance, is embedding AI agents into its own support systems, allowing customers to resolve issues without waiting for human technicians. Similarly, companies like Salesforce, SAP, and ServiceNow are building agentic layers into their platforms. The next wave of enterprise software will be defined not by the features of an application but by the intelligence of the agents that orchestrate workflows across them.

As enterprises navigate this transformation, they must also contend with the need for new governance frameworks. Agentic AI systems operate with a degree of autonomy that demands robust guardrails—both for technical reliability and ethical compliance. Organizations are establishing AI centers of excellence that include legal, compliance, and HR representatives to oversee agent behavior. The conversation is shifting from 'Can we build it?' to 'Should we let it run?' and 'Who is accountable when something goes wrong?'

The infrastructure requirements for agentic AI also have implications for sustainability. While individual agents may be efficient, the aggregate energy consumption of millions of agents running 24/7 could be significant. Enterprises are exploring energy-efficient hardware, such as custom ASICs for inference, and leveraging renewable energy sources for data centers. Dell's Roese predicts that as agentic AI matures, the industry will see a convergence of compute, storage, and networking into unified platforms that can dynamically allocate resources based on agent demand. This will reduce waste and improve total cost of ownership.

Source: ComputerWeekly.com News