1. Shanghai AI Lab Releases Intern-S2-397B Multimodal Model
Shanghai AI Lab has released Intern-S2-397B, a new open-weights multimodal foundation model tailored for scientific intelligence and long-horizon agent tasks. The model employs a unique vision-language pre-training paradigm that learns directly from raw pages of scientific literature to capture complex visual relationships and symbolic semantics. Trained across more than 20 scientific domains and optimized using black-box agentic reinforcement learning in sandboxed environments, Intern-S2-397B is designed to excel at specialized tasks like biomolecular interaction design and material structure generation.
- • Intern-S2-397B is a new multimodal foundation model designed for scientific intelligence and long-horizon agent tasks.
- • The model utilizes a vision-language pre-training paradigm that learns directly from raw pages of scientific literature to model symbolic semantics.
- • It was trained on diverse scientific reinforcement-learning tasks across more than 20 domains, including biomolecular design.
- • The model employs black-box agentic reinforcement learning in sandboxed environments to improve generalization.
- • It scales across three dimensions: pre-training, reinforcement-learning task coverage, and interactive agent environments.
It provides developers with a massive, open-weights model capable of handling complex scientific reasoning and sandboxed agent environments.
2. AWS Releases Pizza Bot, an Open-Source Inbox for Background AI Agents
AWS has launched Pizza Bot, an open-source, self-hosted application designed to manage background AI agents. Licensed under Apache 2.0, Pizza Bot provides an email-style inbox interface to organize task results, thread histories, and pending decisions. Built on DeepAgents and LangGraph, the system supports human-in-the-loop approvals for external tools via MCP servers and integrates with major model providers including Bedrock, Anthropic, Gemini, OpenAI, OpenRouter, and Ollama.
- • Pizza Bot is an open-source, self-hosted application licensed under Apache 2.0, with desktop builds for macOS, Windows, and Linux.
- • The application organizes AI task results and pending decisions into an email-style inbox with categories for thread history and completed work.
- • It utilizes DeepAgents and LangGraph for stateful execution, using checkpoints to maintain thread state and approval pauses.
- • The system is compatible with multiple model providers, including Amazon Bedrock, Anthropic, Google Gemini, OpenAI, OpenRouter, and Ollama.
- • Users can manage external tools via MCP servers and configure policies to approve, edit, or reject proposed actions.
- • An always-on backend is required for tasks to continue running if the desktop client is closed.
It gives developers a ready-to-use, self-hosted dashboard to monitor background agents, review pending actions, and manage tool execution policies.
3. New Context Engineering Patterns for Long-Horizon AI Agents
As autonomous agents face performance degradation in long-horizon tasks, the industry has moved beyond simple breadcrumb workflows to more robust context engineering. New developments include OpenAI's server-side context caching, AWS AgentCore's parallel MicroVM subagent architecture, and specialized filesystem offloading in Deep Agents, providing developers with more effective tools to maintain agent focus and control costs.
- • OpenAI's Responses API now includes server-side compaction via 'context_caching'.
- • AWS AgentCore utilizes parallel MicroVM subagents to reduce task runtime by up to 3x.
- • Deep Agents implements filesystem offloading for tool responses exceeding 20,000 tokens.
- • Claude Code continues to refine selective compaction to preserve architectural decisions.
These advancements provide developers with standardized, scalable architectural patterns to mitigate context window limits, moving past the experimental 'breadcrumb' approaches previously identified.
4. Llama.cpp Fork Adds Hot-Swappable Speculative Decoding and Adaptive KV Streaming
A new fork of Raymond's llama.cpp KV cache streaming project has been released, introducing adaptive KV streaming and hot-swappable speculative decoding. Designed to maximize performance on limited hardware, the system utilizes a VRAM memory pool that streams context from host RAM when VRAM capacity is exceeded. Crucially, it allows developers to dynamically eject speculative models (such as MTP or DFlash2) when context is high and reload them when context is low, enabling faster local execution of models like Qwen 3.8 27B on a 16GB GPU.
- • The fork adds hot-swappable speculative decoding to Raymond's llama.cpp KV cache streaming project.
- • It uses a VRAM memory pool that streams context from host RAM when it exceeds VRAM capacity, outperforming standard offloading.
- • The system allows for the ejection of speculative models (like MTP or DFlash2) when context is high, reloading them when context is low.
- • The implementation was tested using the Qwen 3.8 27B UD-IQ4_XS model on a 16GB 5060Ti GPU.
- • The project repository is hosted publicly at https://github.com/troed/llama.cpp-adaptive-kv-streaming.
It allows developers to run larger models locally with higher performance on memory-constrained hardware.
5. VLLM Configuration Enables High-Speed Qwen3.8 27B Inference on RTX 3090
A developer benchmark has demonstrated that running the Qwen3.8-27B-int4-AutoRound model on vLLM 0.27.1 can achieve high-speed local inference on a single RTX 3090 GPU. By configuring an FP8 E4M3 KV cache, the setup supports a massive 147,456-token context window, delivering an average prefill speed of 871.93 tokens per second and a decode speed of 38.39 tokens per second. The model scored 94.7% on the BenchLocal-CLI benchmark for tool calling and instruction following, utilizing optimized vLLM flags like prefix caching and bfloat16 precision.
- • The benchmark utilized Qwen3.8-27B-int4-AutoRound on vLLM 0.27.1 running on WSL2 with an RTX 3090 GPU.
- • The configuration supported a context window of 147,456 tokens using an FP8 E4M3 KV cache.
- • Prefill performance averaged 871.93 tokens per second, and decode performance averaged 38.39 tokens per second.
- • The model achieved a score of 71/75 (94.7%) on the BenchLocal-CLI benchmark, which tests tool calling, instruction following, and reasoning.
- • Key vLLM settings used include --dtype bfloat16, --gpu-memory-utilization 0.9475, and --enable-prefix-caching.
It provides a concrete, highly optimized configuration for developers to run a 27B model with a 144K context window on standard consumer hardware.