Enterprise generative AI is entering a more complicated phase: instead of routing every task through one frontier model, organizations are increasingly mixing models, controlling inference costs and rebuilding their data pipelines. A new Hyperscience study, conducted with The Harris Poll, finds that 80% of surveyed organizations are moving away from a single-large-model strategy, while 52% say GenAI infrastructure costs have exceeded expectations.
The early enterprise generative AI playbook was relatively straightforward: choose a powerful foundation model, connect it to business applications and expand usage. As deployments move from experiments into production, however, organizations are discovering that model capability is only one part of the equation.
The economics of inference, data quality and increasingly complex AI workflows are forcing enterprises to reconsider how they build AI infrastructure.
That shift is highlighted in a new study from Hyperscience, based on an online survey of more than 400 public- and private-sector decision makers conducted by The Harris Poll between July and August 2026. Hyperscience reports that 80% of organizations are moving away from a single-large-model strategy, while 52% say their GenAI infrastructure costs have been higher than expected. The study also found that 86% say data-quality problems are hurting the performance of their GenAI applications.
The findings point toward a more heterogeneous AI architecture in which enterprises choose models according to workload rather than treating a single large language model as the default for every application.
The economics of inference change the architecture
Model pricing has become a more visible component of enterprise AI strategy as organizations move beyond isolated chatbot interactions toward agents and multistep workflows.
Hyperscience says 62% of surveyed organizations are optimizing model selection and routing, while 57% are evaluating cheaper alternative models. It also reports that 77% already route non-critical workloads to smaller models and that 89% either use or are developing workload routing that sends tasks to different models based on their characteristics.
That approach is increasingly reflected in the broader AI infrastructure market.
Gartner said in August that inference spending worldwide is expected to reach $23.3 billion in 2026, surpassing the $19 billion forecast for AI training. Gartner also expects AI-optimized infrastructure spending to reach about $42.3 billion this year.
The implication is that enterprises are no longer optimizing only for model quality. Latency, token consumption, throughput, reliability and cost per completed task are becoming infrastructure considerations.
Gartner has separately described model routing as a decision-control layer that determines which models handle particular workflows and under what cost and policy constraints. Its July research also argues that inference tiering is becoming more important as AI deployments become more complex.
For companies using platforms from Microsoft, Google, Amazon and other cloud providers, this creates a more complex technology stack. Applications can combine frontier models for difficult reasoning with smaller or specialized models for classification, extraction, summarization and routine automation.
Data quality moves to the center
Model selection, however, cannot solve poor enterprise data.
Hyperscience reports that 86% of respondents say data quality issues are damaging GenAI application performance. The company also says 56% characterize their data extraction and preparation processes as inefficient or manual.
That problem is particularly relevant for enterprise AI applications built around documents and other unstructured information. Contracts, invoices, claims, correspondence, forms and reports often contain the information that AI systems need, but those materials may require extraction, normalization, classification and validation before they can reliably support automated decisions.
McKinsey has similarly identified data management as a major barrier to scaling generative AI. In its research on GenAI data foundations, 70% of surveyed top performers reported difficulties integrating data into AI models, including challenges involving data quality, governance and training data.
As AI agents become more autonomous, the consequences of poor data can become more significant. An inaccurate answer from a chatbot is one problem; an autonomous workflow acting on inaccurate customer, financial or operational information can propagate the error across multiple systems.
This is pushing AI development frameworks and AI automation platforms toward stronger data preparation, observability, governance and evaluation capabilities.
The rise of the multi-model enterprise
The emerging architecture does not necessarily mean enterprises are abandoning frontier models. Instead, the role of those models is changing.
Hyperscience says only 19% of surveyed organizations expect to rely on a single large foundational model over the next 12 to 18 months. At the same time, 78% expect to increase their GenAI investment during the next year.
That combination is notable: organizations can be increasing AI investment while becoming more selective about where expensive models are used.
Gartner’s latest market research similarly points toward growing demand for specialized models. The firm forecasts spending on domain-specific and specialized GenAI models to grow 210% in 2026, while total spending on AI models and platforms is projected to reach $64.3 billion.
For AI infrastructure providers, that creates demand for model gateways, inference optimization, routing engines, evaluation systems and governance layers. NVIDIA remains central to the underlying compute market, while hyperscalers and enterprise software vendors are building increasingly broad AI platforms around model access, data and workflow orchestration.
The resulting enterprise AI stack is therefore becoming less about picking a single winner among large language models and more about managing a portfolio of models and infrastructure components.
Hyperscience’s research frames this as a response to the economics of production GenAI. The broader market evidence points in the same direction: falling unit costs can encourage more sophisticated and token-intensive AI workflows, potentially offsetting efficiency gains.
For enterprise technology teams, the practical question is increasingly how to match the right model, data and compute resources to each job while maintaining accuracy, governance and predictable costs. That architecture could become a defining layer of the next stage of enterprise generative AI.
Market Landscape
Enterprise AI is moving toward multi-model architectures, inference routing and specialized models as organizations balance capability with cost and reliability. Gartner expects worldwide AI spending to reach $2.7 trillion in 2026, a 49.5% year-over-year increase, while AI infrastructure remains the largest spending category.
At the same time, Gartner forecasts that inference costs per agentic workflow will increase more than fivefold through 2028 as increasingly sophisticated AI agents consume more tokens and perform multistep reasoning. Gartner says enterprises will need inference tiering, routing and orchestration to manage those economics.
This creates opportunities for AI infrastructure, AI cloud platforms, model-routing systems, AI development frameworks, enterprise data platforms and AI automation software. The competitive focus is shifting from access to a single powerful model toward controlling the complete path from enterprise data to model inference to business action.
Top Insights
- 80% of surveyed organizations are moving away from a single-large-model strategy as enterprise GenAI economics become harder to manage.
- Hyperscience reports that 86% of organizations experience GenAI performance problems linked to data quality, making data foundations increasingly important.
- Gartner expects inference spending to surpass AI training spending globally in 2026 as production AI workloads expand.
- Specialized and domain-specific GenAI models are gaining traction as enterprises match model capabilities and costs to individual workloads.
- Multi-model routing can help enterprises reserve expensive frontier models for complex tasks while using smaller models for routine workloads.
Power Tomorrow’s Intelligence — Build It with TechEdgeAI
