Inference Brew

GLM 5.2 Token Optimization and Local Performance Metrics

00:00 / --:--

← Back to home

GLM 5.2 Token Optimization and Local Performance Metrics

1. GLM 5.2 Token Optimization and Local Performance Metrics

GLM 5.2 expands its reasoning capabilities with a token limit increase to 36.7k, up from GLM 5.1's 16.7k. A technical report from z.ai indicates that developers can optimize local and API deployments by utilizing the 'high' effort level instead of 'max', which retains 98% of coding task performance while cutting token consumption by more than half. While local testing on multi-GPU setups (such as 6x RTX 3090s) yields acceptable generation speeds of 7.8 tokens per second at 90K context, users note that Xeon-based hardware currently suffers from extreme response latency.

  • GLM 5.2's reasoning tokens have increased to 36.7k from 16.7k in GLM 5.1.
  • A z.ai technical report shows the 'high' effort level delivers 98% of 'max' level intelligence on coding tasks with under half the token usage.
  • Local testing of GLM 5.2 with Q4 quantization at 'high' effort produced math results comparable to 'max' level.
  • Community benchmarks on 6x RTX 3090 GPUs achieved 7.8 tokens per second for generation using unsloth UD-IQ2_M quantization at 90K context.
  • Users report that GLM 5.2 is currently unusable on Xeon hardware due to excessive response wait times.

This allows developers to drastically cut API costs and latency in daily workflows while maintaining high coding quality.

SOURCES

2. Cloudflare Introduces Temporary Accounts for AI Agent Deployments

Cloudflare has introduced Temporary Cloudflare Accounts for Agents, a feature designed to facilitate fully autonomous agentic workflows. By executing `wrangler deploy --temporary`, AI agents can deploy websites, APIs, and Cloudflare Workers without needing a pre-existing account or manual authentication. These temporary deployments remain active for 60 minutes, giving developers or agents a window to claim the resources permanently before they are automatically deleted.

  • AI agents can initiate a temporary deployment by running the command `wrangler deploy --temporary`.
  • Temporary deployments remain active for 60 minutes, during which the account and resources can be claimed to become permanent.
  • Unclaimed temporary accounts and their associated resources are automatically deleted after 60 minutes.
  • The Wrangler CLI has been updated to prompt agents with information about the `--temporary` flag when authentication barriers are met.
  • The feature is designed to support rapid, iterative agentic workflows without human intervention.

This enables frictionless, fully automated write-deploy-verify cycles for coding agents, removing human authentication bottlenecks.

SOURCES

3. Cisco AI Releases FAPO Prompt Optimization System

Cisco AI has open-sourced FAPO (Fully Automated Prompt Optimization), a Claude Code-driven system designed to optimize multi-step LLM pipelines. Released under the Apache 2.0 license, FAPO utilizes step-level failure attribution to classify pipeline errors into retrieval, cascading, format, or reasoning failures. It then optimizes the pipeline across three escalating levels: prompt edits, parameter adjustments, and structural changes. In evaluations, FAPO outperformed the GEPA optimizer by a mean of 14.1 percentage points, and up to 33.8 percentage points on benchmarks requiring structural changes, while employing guardrails like training-split-only inspection to prevent overfitting.

  • FAPO (Fully Automated Prompt Optimization) is open-source under the Apache 2.0 license.
  • The system supports Claude Code and Codex as optimization agents to refine multi-step LLM pipelines.
  • It uses step-level failure attribution to classify errors into retrieval, cascading, format, or reasoning failures.
  • FAPO optimizes pipelines across three levels: prompt edits, parameter adjustments, and structural changes.
  • In benchmarks, FAPO outperformed the GEPA optimizer with a mean gain of 14.1 percentage points, rising to 33.8 points on structural benchmarks like HoVer.
  • Built-in guardrails against overfitting include training-split-only inspection and independent proposal reviews.

Developers can automatically optimize complex LLM pipelines through prompt edits, parameter adjustments, and structural changes with built-in overfitting guardrails.

SOURCES

4. Nous Research Adds Blank Slate Mode to Hermes Agent

Nous Research has updated its open-source Hermes Agent with a new 'Blank Slate' configuration mode designed for security-sensitive and auditable deployments. When enabled, Blank Slate initializes the agent with only basic provider settings, File Operations, and Terminal access, while disabling default capabilities such as web browsing, code execution, vision, and MCP servers. To ensure configuration durability across software updates, the mode writes explicit `disabled_toolsets` and `platform_toolsets.cli` files directly to disk.

  • Blank Slate mode initializes Hermes Agent with only provider/model settings, File Operations, and Terminal enabled.
  • Default features like web access, browser, code execution, vision, memory, and MCP servers are disabled.
  • The configuration is made durable across updates by writing explicit 'disabled_toolsets' and 'platform_toolsets.cli' lists to disk.
  • Hermes Agent requires an LLM with a minimum context window of 64,000 tokens.
  • The mode is designed for security-sensitive deployments, reproducible team environments, and educational audits.

This provides a highly secure, reproducible, and auditable environment for running agents in sensitive production settings.

SOURCES

5. Cosine Post-Trains Kimi K2.6 for Active Pen Testing

Cosine (YC W23) has launched a cyber-security AI tool designed to actively identify system vulnerabilities. Built on a Kimi K2.6 base model, the tool was post-trained on a decade of capture-the-flag (CTF) contest data using supervised fine-tuning (SFT) and reinforcement learning (RL) with verifiable rewards to prevent standard safety refusals during security testing. Operating via a CLI, the tool features a read-only local code scanner and an active pen-testing mode that attempts to exploit sandboxed systems, providing developers with concrete proof of vulnerabilities by displaying the exact exploit request and response.

  • The tool is built on a Kimi K2.6 base model post-trained on a decade of capture-the-flag (CTF) data using SFT and RL with verifiable rewards.
  • It operates via a CLI and offers a read-only local code scan mode and an active pen test mode for sandboxed systems.
  • The active pen test mode attempts to break live systems and provides proof of vulnerabilities by displaying the exploit request and response.
  • The system uses a multi-agent swarm harness where an orchestrator distributes tasks across parallel subagents.
  • It is free to install and includes a free tier for scans up to 2 million tokens, with context sent to an inference API over TLS.

Developers can run automated, active pen tests on sandboxed systems to find and verify vulnerabilities with concrete exploit proofs.

SOURCES

6. Self-Hosted Web Access for Local Agents via SearXNG and Scrapling

A developer has designed a self-hosted web access pipeline for local-first AI agents that bypasses paid search and scraping APIs. The architecture deploys SearXNG in Docker to serve search results via JSON, then uses Scrapling to extract page content. It features a fast browserless path alongside a stealth headless browser path for challenging sites, converting the final HTML to Markdown via Trafilatura to minimize token consumption. The pipeline also includes SSRF guards to block agents from scanning internal network ranges.

  • The self-hosted setup uses SearXNG deployed in Docker to provide search results via a JSON endpoint.
  • Page extraction utilizes a fast path without a browser and a stealth path with a headless browser for blocked pages.
  • Trafilatura converts extracted HTML into clean Markdown to optimize readability and token usage for local LLMs.
  • The system integrates pypdf for PDF processing and SSRF guards to prevent agents from accessing internal network ranges.
  • The setup avoids paid API keys and vendor-specific search services, though search quality depends on upstream engines.

Developers can build fully local, privacy-preserving web-browsing agents without relying on expensive, vendor-locked search or scraping APIs.

SOURCES

7. Running Claude Code with GLM-5.2 via Hugging Face

A new integration guide details how to configure Anthropic's Claude Code CLI to run with the GLM-5.2 model. By routing requests through Hugging Face Inference Providers, developers can utilize the open-weights model within their agentic terminal workflows. The setup requires generating a fine-grained Hugging Face token with inference provider permissions and configuring specific environment variables to establish the connection.

  • Claude Code can be integrated with GLM-5.2 using Hugging Face Inference Providers.
  • Configuration requires a fine-grained Hugging Face token with 'Make calls to Inference Providers' permissions.
  • The setup is completed by configuring specific environment variables to route Claude Code's API calls.

This allows developers to leverage Claude Code's agentic capabilities with alternative open-weights models like GLM-5.2.

SOURCES

8. TimeCopilot Simplifies Forecasting Pipelines with Foundation Models

A new tutorial outlines how to build end-to-end forecasting pipelines using TimeCopilot, a tool designed to streamline forecasting workflows. The pipeline evaluates statistical, foundation, and GPU-based forecasting models using rolling cross-validation to rank them on MAE, RMSE, and MAPE metrics. Additionally, TimeCopilot integrates an optional LLM agent that can automatically select the best-performing models and generate analytical interpretations using OpenAI or Anthropic API keys.

  • TimeCopilot is a tool designed for building end-to-end forecasting workflows, including data prep and anomaly detection.
  • The pipeline evaluates statistical, foundation, and GPU-based forecasting models using rolling cross-validation.
  • It generates 12-month probabilistic forecasts with 80% and 95% prediction intervals.
  • TimeCopilot features an optional LLM agent that can select models and provide analytical interpretations using OpenAI or Anthropic APIs.

Developers can quickly build and evaluate complex forecasting pipelines that combine traditional statistical models with LLM-driven analysis.

SOURCES

9. Yandex Open-Sources YaFF Zero-Copy Serialization Library

Yandex has open-sourced YaFF (Yet another Flat Format), a high-performance C++ serialization library released under the Apache 2.0 license. YaFF introduces a zero-copy wire format for the Protobuf ecosystem, allowing developers to maintain their existing `.proto` files as the single source of truth. Benchmarks on an AMD EPYC 7713 show that YaFF's Flat Layout reads hot data 3.8 times faster than FlatBuffers and 22 times faster than standard Protobuf, yielding 10–20% CPU savings when deployed at production scale in recommendation engines.

  • YaFF (Yet another Flat Format) is open-sourced under the Apache 2.0 license.
  • The library provides a zero-copy wire format for the Protobuf ecosystem while keeping the .proto file as the single source of truth.
  • In benchmarks on an AMD EPYC 7713, YaFF's Flat Layout read hot data 3.8 times faster than FlatBuffers and 22 times faster than Protobuf.
  • Implementing YaFF in Yandex's advertising recommendation system resulted in 10–20% CPU savings at production scale.
  • YaFF supports incremental adoption via two-way conversion between Protobuf and YaFF at system edges.

Developers building high-throughput server-side runtimes can achieve significant CPU savings and faster read speeds for recommendation engines and memory-mapped indexes.

SOURCES

10. Noema Atlas Launches P2P Local LLM Weight Distribution Network

Noema Atlas has introduced an open-source, Rust-based peer-to-peer network software designed for sharing local LLM weights. Released under the Apache-2.0 license, Noema Atlas utilizes the Iroh protocol for peer-to-peer transfers and verifies file integrity via BLAKE3 content hashing. The system optimizes local storage by deduplicating identical files across model variants using reflinks or hardlinks, and allows developers to share and rescue models that have been removed from centralized platforms like Hugging Face.

  • Noema Atlas is built in Rust and released under the Apache-2.0 open-source license.
  • The software uses Iroh for peer-to-peer transfers and verifies file integrity using BLAKE3 content hashing.
  • It deduplicates identical files across model variants or mirrors using reflinks or hardlinks.
  • Users can rescue and share models removed from platforms like Hugging Face by providing a title and license.
  • The project includes a native desktop app for macOS, Windows, and Linux, a CLI, and a companion app called Noema Atlas Studio.

Developers can securely share, mirror, and download large model weights directly from peers, bypassing centralized bandwidth bottlenecks or model takedowns.

SOURCES

11. EXL3 Quantization Support Arrives on Apple Silicon Macs

The open-source PonyExl3 project has brought EXL3 quantization support to Apple Silicon Macs with 64GB or more of RAM. Previously restricted to CUDA-enabled RTX hardware, EXL3 models can now be run, inferred, and converted on macOS. Initial testing on MiniCPM5 and Qwen3.6-27B models demonstrates that macOS-converted models achieve a mean Kullback-Leibler divergence (KLD) on par with those converted on NVIDIA RTX cards, while delivering approximately half a bit per weight improvement in quality-to-weight performance over MLX quantization.

  • EXL3 models, previously restricted to CUDA and expensive RTX hardware, can now run and convert on Apple Silicon Macs with 64GB+ RAM.
  • The open-source PonyExl3 project is available on GitHub under an Apache 2.0 license.
  • Testing on MiniCPM5 and Qwen3.6-27B models on macOS showed mean KLD on par with conversions done on RTX cards.
  • EXL3 quantization provides approximately half a bit per weight improvement in quality-to-weight performance compared to MLX.

This unlocks high-performance EXL3 quantization on macOS, offering better quality-to-weight performance than MLX quantization.

SOURCES

Inference Brew in your inbox

5 minutes a day. Free, unsubscribe anytime.

Inference Brew in your inbox

5 minutes a day. Free, unsubscribe anytime.