As large language models move toward trillion-parameter scale, reducing the cost of running them is becoming almost as important as improving their accuracy. Nota AI is targeting that problem with two research papers accepted at EMNLP 2026, including one selected for the conference’s Main Conference. Both studies focus on a difficult optimization problem in Mixture-of-Experts (MoE) models: how to reduce memory and compute requirements through quantization without disrupting the routing decisions that determine model quality.

The next phase of generative AI may depend less on making models simply larger and more on making them economical enough to operate.

Nota AI, a South Korean AI model optimization company, says two of its papers on large-scale model optimization have been accepted at EMNLP 2026, one of the major international conferences for natural language processing. One paper was selected for the Main Conference, while the second was accepted into the Findings track.

The company said roughly 18,000 papers were submitted this year, with the Main Conference accepting about 15.4% of submissions. That makes the Main Conference selection particularly significant, although conference acceptance should be viewed as evidence of peer-reviewed research quality rather than proof of commercial superiority.

The research tackles quantization of Mixture-of-Experts models, an architecture increasingly associated with large language models such as Qwen and Kimi.

MoE models divide a neural network into multiple specialist components, or “experts.” Instead of activating every expert for every token, a routing mechanism selects a subset that is considered most relevant to the input. The architecture can therefore increase total model capacity without requiring every expert to participate in every computation.

There is a catch.

Even when only a fraction of experts are active at a given moment, the complete model still needs to be stored or made accessible in memory. At very large parameter counts, that creates substantial GPU memory and infrastructure requirements.

Quantization is one of the main techniques used to address the problem. By representing model weights and related values using fewer bits, operators can reduce memory consumption and potentially increase inference efficiency.

But MoE models introduce another source of sensitivity.

Small numerical changes caused by quantization can alter the ranking of candidate experts. A token that previously activated one expert could instead be routed to another. Across billions of tokens, those seemingly minor routing changes can accumulate into measurable degradation in model output.

Preserving the routing decision

Nota AI’s Main Conference paper introduces MENDS-MoE, a methodology designed to account for the effect of quantization on expert selection across subsequent model operations.

The approach focuses particularly on experts close to the selection boundary — the experts that could be swapped in or out as numerical values change.

According to Nota AI, experiments involving 3-bit and 4-bit quantization across three MoE models produced higher average accuracy and stronger language-model performance than competing methods across most evaluation settings.

The second paper, OPERA, takes a different approach to the same underlying problem.

Rather than attempting to correct every change in expert selection, OPERA concentrates optimization on routing changes that have a meaningful effect on the model’s final output.

That distinction is important for production AI systems. Not every internal numerical difference necessarily matters to the user. A practical optimization technique should ideally spend computational and engineering effort correcting the changes that actually affect response quality.

The two approaches therefore address the same broad challenge from complementary directions: preserving useful routing behavior after quantization without unnecessarily sacrificing the efficiency gains that quantization is intended to deliver.

Why MoE optimization matters for AI infrastructure

The research arrives as AI infrastructure operators face a growing gap between model scale and available computing resources.

Large language models are increasingly being built with hundreds of billions or even trillions of parameters. While MoE architectures can make inference more computationally efficient than activating the entire parameter set for every token, their memory footprint can remain substantial.

That makes optimization techniques such as pruning, quantization and efficient routing increasingly important to the economics of AI inference.

The competitive landscape includes proprietary optimization stacks from companies such as NVIDIA, alongside open-source frameworks and model-specific techniques developed by Google, Meta, Microsoft and model developers such as Alibaba’s Qwen team and Moonshot AI.

NVIDIA’s TensorRT-LLM, for example, provides optimization techniques for large-scale inference across NVIDIA GPU infrastructure. The broader industry is also experimenting with lower-precision formats, speculative decoding, structured sparsity and specialized accelerators.

Nota AI’s research is positioned in a narrower but strategically important area: reducing the cost of running large models while maintaining their behavior after compression.

From academic research to deployment

The company has also been attempting to connect its research with practical inference workloads.

Earlier in 2026, Nota AI placed third among approximately 40 teams in the Efficient Qwen Competition, held as part of the Resource-Adaptive Foundation Model Inference workshop at ICML 2026. The company says it ran Qwen3.5-4B on a single NVIDIA A10G GPU while maintaining model performance and achieving an average inference speedup of 6,978 times over the competition baseline.

The baseline and measurement methodology matter when interpreting such a figure, so the speedup should not be treated as a general claim that Qwen inference can simply become thousands of times faster in production.

Nota AI says it has published or presented more than 50 papers covering model pruning and quantization, edge AI, generative AI, vision-language models and LLMs.

Its recent commercial examples point toward the same objective. The company says its optimization technology reduced the GPU requirements for Qwen3.8-Max, a model exceeding one trillion parameters, from 24 NVIDIA B300 GPUs to four. It also reports reducing the requirements for Moonshot AI’s Kimi K3 from eight B300 GPUs to four and Upstage’s Solar Open 2 from eight NVIDIA H100 GPUs to two.

Those figures are company-reported deployment results rather than independently audited benchmarks, but they illustrate why model optimization has become an infrastructure issue rather than simply a research exercise.

The emerging AI efficiency race

The AI industry’s first competitive wave centered on model capability: larger parameter counts, stronger benchmarks and increasingly sophisticated reasoning.

The next phase is more operational.

Inference providers need to control GPU costs, memory consumption, power use and latency while maintaining output quality. For enterprises deploying AI at scale, those factors directly influence the cost per query and the infrastructure required to serve production workloads.

That makes quantization and routing optimization particularly relevant to AI factories built around expensive accelerators from NVIDIA and AMD.

The challenge will be maintaining that efficiency as model architectures continue to evolve. Techniques that work well for one MoE architecture may not transfer directly to another, particularly as models introduce more complex routing, multimodal inputs and reasoning workloads.

Nota AI’s EMNLP results therefore matter less as a standalone corporate milestone than as evidence of a broader industry transition: AI optimization is becoming a core layer of the model stack.

As models approach trillion-parameter scale, the companies that can preserve useful model behavior while substantially reducing compute and memory requirements could have an increasingly important role in determining the economics of enterprise AI.

Market Landscape

The AI infrastructure market is moving toward an efficiency-per-token model rather than a simple race for maximum model size.

Several optimization layers are competing:

• Quantization: Reduces numerical precision to lower memory and compute requirements.
• Pruning: Removes parameters or computational structures that contribute relatively little to model output.
• MoE routing optimization: Improves how experts are selected while minimizing unnecessary computation.
• Inference engines: Platforms such as NVIDIA TensorRT-LLM optimize execution across accelerator hardware.
• Model architecture: Developers increasingly design models around sparse activation and efficient inference from the outset.

The importance of these techniques is reinforced by the scale of AI infrastructure spending. Gartner forecasts worldwide AI spending will reach $2.59 trillion in 2026, with AI infrastructure accounting for more than 45% of that spending.

For enterprises, the practical metric is increasingly useful output per GPU, per watt and per dollar. That favors optimization technologies capable of reducing hardware requirements without creating unacceptable accuracy or latency trade-offs.

Top Insights

• Nota AI secured two EMNLP 2026 acceptances focused on MoE quantization, highlighting growing academic attention to efficient large-language-model inference.
• MENDS-MoE targets quantization-induced expert-routing changes, while OPERA prioritizes routing errors that materially influence final model output quality.
• The research addresses a central AI infrastructure challenge: reducing GPU memory requirements while preserving the behavior and accuracy enterprises expect from large models.
• Nota AI reports major GPU reductions for Qwen3.8-Max, Kimi K3 and Solar Open 2, though these deployment figures remain company-reported benchmarks.
• As trillion-parameter models proliferate, quantization, pruning and inference optimization could become increasingly important to AI cloud economics and enterprise adoption.

Power Tomorrow’s Intelligence — Build It with TechEdgeAI