As enterprises deploy increasingly complex AI agents, infrastructure providers are facing a challenge that extends beyond supplying powerful GPUs. Large-scale inference also depends on memory capacity, data movement, cooling efficiency, storage performance and software orchestration. At SC26 in Chicago, MiTAC Computing Technology unveiled a full-stack AI infrastructure ecosystem designed to address these constraints. Through collaborations with AMD, NVIDIA, Intel, Skymizer, Canonical, DDN and Rafay, the company is combining disaggregated inference, liquid-cooled rack systems and GPU resource management into an architecture aimed at scaling enterprise agentic AI.
Disaggregated Inference Separates Compute From Memory
One of MiTAC’s more technically distinctive demonstrations is its collaboration with Skymizer on a disaggregated inference architecture. The design separates two stages of large language model inference: prefill, which processes incoming prompts and creates the initial key-value (KV) cache, and decode, which generates the response token by token.
These stages have different hardware requirements. Prefill is computationally intensive and benefits from GPU acceleration, while decode can be constrained by memory capacity and bandwidth. Running both on the same accelerator infrastructure can leave resources underutilized or make it harder to optimize each stage independently.
MiTAC and Skymizer propose assigning prefill to the G8825Z5 server, equipped with AMD EPYC processors and Instinct MI350X GPUs. The KV cache is then transferred over high-speed networking to a G4520G6 decode server containing eight Skymizer HTX301 accelerator cards.
Each card is specified with 256 GB of memory, providing 2 TB of aggregate accelerator memory in the proposed configuration. MiTAC and Skymizer say this architecture is designed to run large models, including the 2.8-trillion-parameter Kimi K3, on premises.
The approach could appeal to enterprises that want to operate large models within their own infrastructure while allocating resources separately for prompt processing and token generation. It also reflects a broader shift toward specialized inference systems as organizations deploy AI agents that make repeated model calls and handle longer-running tasks.
There is, however, an important qualification. Skymizer’s November announcement says HTX301 chip production is expected to begin in the first quarter of 2027, with platform development and validation still underway. The Kimi K3 demonstration should therefore be understood as a proposed and developing configuration, not evidence of broadly available production infrastructure.
Software Orchestration and Data Infrastructure
MiTAC’s full-stack proposition also includes the software required to manage infrastructure across different hardware and workloads.
Canonical Ubuntu provides the operating system foundation, while DDN Infinia is intended to supply high-throughput storage for data-intensive AI workloads. Rafay’s orchestration technology supports GPU-as-a-Service management, allowing infrastructure operators to allocate accelerator resources across teams and distributed workloads.
MiTAC’s own MiCoreView software provides data-center and POD-level telemetry, giving operators a consolidated view of infrastructure operations.
The combination addresses a common scaling problem: expensive accelerators can remain underused when data cannot be delivered quickly enough or workloads are not scheduled efficiently. Storage performance, cluster orchestration and infrastructure monitoring therefore become part of the AI performance equation.
Rack-Scale Designs Target Density and Thermal Limits
The company is also demonstrating several rack configurations aimed at different infrastructure requirements.
One 52U liquid-cooled rack combines 12 G4826Z5 servers with 96 AMD Instinct MI355X GPUs. MiTAC claims the configuration achieves 50% higher GPU density than conventional deployments.
A second proof of concept, developed with ZutaCore, uses waterless two-phase direct liquid cooling. The system vaporizes dielectric fluid at the chip to remove heat, with MiTAC citing support for thermal design power exceeding 4,000 watts and an annualized partial Power Usage Effectiveness (pPUE) of 1.03. These are vendor-reported figures for the demonstrated configuration, not independently verified results across operating environments.
A third rack focuses on storage performance, using DDN Infinia and seven MiTAC GC68C-B8056 storage servers to support continuous data delivery to AI workloads.
The company is also developing an AMD Helion Agentic AI Rack based on an open ORv3 design. AMD describes Helion as a flexible rack-scale blueprint for agent execution and inference, with CPU-only and integrated CPU/GPU configurations. The architecture can scale to 32,768 Zen 6 CPU cores, according to AMD.
Infrastructure Competition Moves Up the Stack
MiTAC’s announcement reflects a competitive shift among server manufacturers and infrastructure providers. As AI deployments become more complex, vendors are differentiating through cooling, storage, networking, orchestration and workload-specific system design rather than relying exclusively on accelerator specifications.
The broader ecosystem includes AMD, NVIDIA, Intel, Canonical, DDN, Rafay and Skymizer, with each partner addressing a different layer of the infrastructure stack.
For enterprise buyers, the attraction is the possibility of deploying a more integrated AI environment while retaining control over models, data and infrastructure. The challenge will be validating performance across the complete system, including networking overhead, KV-cache transfer, software compatibility, cooling efficiency and workload economics.
MiTAC’s SC26 showcase makes a clear argument: scaling agentic AI requires infrastructure designed around the entire inference lifecycle. Whether its proposed systems deliver the promised efficiency and density at production scale will depend on engineering validation, component availability and real-world deployments.
Market Landscape
MiTAC’s announcement highlights five developments shaping AI infrastructure:
1. Disaggregated AI inference: Separating prefill and decode workloads allows organizations to optimize compute-intensive and memory-intensive stages independently.
2. Agentic AI infrastructure: Autonomous agents create sustained inference demand through repeated model calls, multi-step reasoning and workflow execution.
3. High-density liquid cooling: Two-phase direct liquid cooling and other advanced thermal technologies are becoming important as rack power requirements increase.
4. AI storage and orchestration: High-throughput data platforms and GPU resource management are critical to reducing idle accelerator capacity.
5. Open rack-scale architectures: AMD Helion and OCP-based designs support more flexible infrastructure configurations for hyperscale and enterprise deployments.
Competitive landscape: MiTAC is positioning itself alongside server and infrastructure providers such as Supermicro, HPE, Lenovo and Dell Technologies. Its strategy emphasizes system integration and specialized inference architectures rather than competing solely on GPU availability.
Top Insights
- MiTAC is combining compute, storage, networking, cooling and orchestration into a full-stack infrastructure proposition for enterprise agentic AI.
- Its Skymizer partnership separates GPU-based prefill processing from memory-intensive decode using dedicated accelerator cards.
- The proposed decode server provides 2 TB of aggregate accelerator memory across eight HTX301 cards.
- A 96-GPU liquid-cooled rack and a two-phase cooling proof of concept demonstrate MiTAC’s focus on infrastructure density.
- AMD Helion extends the portfolio toward CPU-heavy agent execution, with a proposed rack architecture supporting up to 32,768 Zen 6 cores.
Power Tomorrow’s Intelligence — Build It with TechEdgeAI
