\n
![](\"https:\/\/images.ctfassets.net\/jdtwqhzvc2n1\/c8LjxKPtjJumRUwmgig3W\/3ceb88204d1ec334f3ce3719b80793eb\/6Ud5A3f_99gyh14m2ffZr.jpg?w=300&q=30\")
<\/p>\n
\n<\/p>\n
Chinese AI startup Zhipu AI aka Z.ai has released its GLM-4.6V series<\/b><\/a>, a new generation of open-source vision-language models (VLMs) optimized for multimodal reasoning, frontend automation, and high-efficiency deployment. <\/p>\n The release includes two models in "large" and "small" sizes: <\/p>\n GLM-4.6V (106B)<\/b>, a larger 106-billion parameter model aimed at cloud-scale inference<\/p>\n<\/li>\n GLM-4.6V-Flash (9B)<\/b>, a smaller model of only 9 billion parameters designed for low-latency, local applications<\/p>\n<\/li>\n<\/ol>\n Recall that generally speaking, models with more parameters \u2014 or internal settings governing their behavior, i.e. weights and biases \u2014 are more powerful, performant, and capable of performing at a higher general level across more varied tasks.<\/p>\n However, smaller models can offer better efficiency for edge or real-time applications where latency and resource constraints are critical.<\/p>\n The defining innovation in this series is the introduction of native function calling<\/b> in a vision-language model\u2014enabling direct use of tools such as search, cropping, or chart recognition with visual inputs. <\/p>\n With a 128,000 token context length (equivalent to a 300-page novel's worth of text exchanged in a single input\/output interaction with the user) and state-of-the-art (SoTA) results across more than 20 benchmarks, the GLM-4.6V series positions itself as a highly competitive alternative to both closed and open-source VLMs. It's available in the following formats:<\/p>\n API access<\/a> via OpenAI-compatible interface<\/p>\n<\/li>\n Try the demo<\/a> on Zhipu\u2019s web interface<\/p>\n<\/li>\n Download weights<\/a> from Hugging Face<\/p>\n<\/li>\n Desktop assistant app available on Hugging Face Spaces<\/a><\/p>\n<\/li>\n<\/ul>\n GLM\u20114.6V and GLM\u20114.6V\u2011Flash are distributed under the MIT license<\/a>, a permissive open-source license that allows free commercial and non-commercial use, modification, redistribution, and local deployment without obligation to open-source derivative works. <\/p>\n This licensing model makes the series suitable for enterprise adoption, including scenarios that require full control over infrastructure, compliance with internal governance, or air-gapped environments.<\/p>\n Model weights and documentation are publicly hosted on Hugging Face<\/a>, with supporting code and tooling available on GitHub<\/a>. <\/p>\n The MIT license ensures maximum flexibility for integration into proprietary systems, including internal tools, production pipelines, and edge deployments.<\/p>\n The GLM-4.6V models follow a conventional encoder-decoder architecture with significant adaptations for multimodal input. <\/p>\n Both models incorporate a Vision Transformer (ViT) encoder\u2014based on AIMv2-Huge\u2014and an MLP projector to align visual features with a large language model (LLM) decoder. <\/p>\n Video inputs benefit from 3D convolutions and temporal compression, while spatial encoding is handled using 2D-RoPE and bicubic interpolation of absolute positional embeddings.<\/p>\n A key technical feature is the system\u2019s support for arbitrary image resolutions and aspect ratios, including wide panoramic inputs up to 200:1. <\/p>\n In addition to static image and document parsing, GLM-4.6V can ingest temporal sequences of video frames with explicit timestamp tokens, enabling robust temporal reasoning.<\/p>\n On the decoding side, the model supports token generation aligned with function-calling protocols, allowing for structured reasoning across text, image, and tool outputs. This is supported by extended tokenizer vocabulary and output formatting templates to ensure consistent API or agent compatibility.<\/p>\n GLM-4.6V introduces native multimodal function calling, allowing visual assets\u2014such as screenshots, images, and documents\u2014to be passed directly as parameters to tools. This eliminates the need for intermediate text-only conversions, which have historically introduced information loss and complexity.<\/p>\n The tool invocation mechanism works bi-directionally:<\/p>\n Input tools can be passed images or videos directly (e.g., document pages to crop or analyze).<\/p>\n<\/li>\n Output tools such as chart renderers or web snapshot utilities return visual data, which GLM-4.6V integrates directly into the reasoning chain.<\/p>\n<\/li>\n<\/ul>\n In practice, this means GLM-4.6V can complete tasks such as:<\/p>\n Generating structured reports from mixed-format documents<\/p>\n<\/li>\n Performing visual audit of candidate images<\/p>\n<\/li>\n Automatically cropping figures from papers during generation<\/p>\n<\/li>\n Conducting visual web search and answering multimodal queries<\/p>\n<\/li>\n<\/ul>\n GLM-4.6V was evaluated across more than 20 public benchmarks covering general VQA, chart understanding, OCR, STEM reasoning, frontend replication, and multimodal agents. <\/p>\n According to the benchmark chart released by Zhipu AI:<\/p>\n GLM-4.6V (106B) achieves SoTA or near-SoTA scores among open-source models of comparable size (106B) on MMBench, MathVista, MMLongBench, ChartQAPro, RefCOCO, TreeBench, and more.<\/p>\n<\/li>\n GLM-4.6V-Flash (9B) outperforms other lightweight models (e.g., Qwen3-VL-8B, GLM-4.1V-9B) across almost all categories tested.<\/p>\n<\/li>\n The 106B model\u2019s 128K-token window allows it to outperform larger models like Step-3 (321B) and Qwen3-VL-235B on long-context document tasks, video summarization, and structured multimodal reasoning.<\/p>\n<\/li>\n<\/ul>\n Example scores from the leaderboard include:<\/p>\n MathVista: 88.2 (GLM-4.6V) vs. 84.6 (GLM-4.5V) vs. 81.4 (Qwen3-VL-8B)<\/p>\n<\/li>\n WebVoyager: 81.0 vs. 68.4 (Qwen3-VL-8B)<\/p>\n<\/li>\n Ref-L4-test: 88.9 vs. 89.5 (GLM-4.5V), but with better grounding fidelity at 87.7 (Flash) vs. 86.8<\/p>\n<\/li>\n<\/ul>\n Both models were evaluated using the vLLM inference backend and support SGLang for video-based tasks.<\/p>\n Zhipu AI emphasized GLM-4.6V\u2019s ability to support frontend development workflows. The model can:<\/p>\n Replicate pixel-accurate HTML\/CSS\/JS from UI screenshots<\/p>\n<\/li>\n Accept natural language editing commands to modify layouts<\/p>\n<\/li>\n Identify and manipulate specific UI components visually<\/p>\n<\/li>\n<\/ul>\n This capability is integrated into an end-to-end visual programming interface, where the model iterates on layout, design intent, and output code using its native understanding of screen captures.<\/p>\n In long-document scenarios, GLM-4.6V can process up to 128,000 tokens\u2014enabling a single inference pass across:<\/p>\n 150 pages of text (input)<\/p>\n<\/li>\n 200 slide decks<\/p>\n<\/li>\n 1-hour videos<\/p>\n<\/li>\n<\/ul>\n Zhipu AI reported successful use of the model in financial analysis across multi-document corpora and in summarizing full-length sports broadcasts with timestamped event detection.<\/p>\n The model was trained using multi-stage pre-training followed by supervised fine-tuning (SFT) and reinforcement learning (RL). Key innovations include:<\/p>\n Curriculum Sampling (RLCS): Dynamically adjusts the difficulty of training samples based on model progress<\/p>\n<\/li>\n Multi-domain reward systems: Task-specific verifiers for STEM, chart reasoning, GUI agents, video QA, and spatial grounding<\/p>\n<\/li>\n Function-aware training: Uses structured tags (e.g., <think>, <answer>, <|begin_of_box|>) to align reasoning and answer formatting<\/p>\n<\/li>\n<\/ul>\n The reinforcement learning pipeline emphasizes verifiable rewards (RLVR) over human feedback (RLHF) for scalability, and avoids KL\/entropy losses to stabilize training across multimodal domains<\/p>\n Zhipu AI offers competitive pricing for the GLM-4.6V series, with both the flagship model and its lightweight variant positioned for high accessibility.<\/p>\n GLM-4.6V: $0.30 (input) \/ $0.90 (output) per 1M tokens<\/p>\n<\/li>\n GLM-4.6V-Flash: Free<\/p>\n<\/li>\n<\/ul>\n Compared to major vision-capable and text-first LLMs, GLM-4.6V is among the most cost-efficient for multimodal reasoning at scale. Below is a comparative snapshot of pricing across providers:<\/p>\n USD per 1M tokens \u2014 sorted lowest \u2192 highest total cost<\/i><\/p>\n\n
\n
Licensing and Enterprise Use<\/b><\/h2>\n
Architecture and Technical Capabilities<\/b><\/h2>\n
Native Multimodal Tool Use<\/b><\/h2>\n
\n
\n
High Performance Benchmarks Compared to Other Similar-Sized Models<\/b><\/h2>\n
\n
\n
Frontend Automation and Long-Context Workflows<\/b><\/h2>\n
\n
\n
Training and Reinforcement Learning<\/b><\/h2>\n
\n
Pricing (API)<\/b><\/h2>\n
\n