huggingface-daily-paper-feed/feed_weekly.xml

<?xml version='1.0' encoding='UTF-8'?>
<rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" version="2.0">
  <channel>
    <title>Hugging Face Daily Papers</title>
    <link>https://huggingface.co/papers</link>
    <description>Daily research papers curated by the Hugging Face community.</description>
    <docs>http://www.rssboard.org/rss-specification</docs>
    <generator>python-feedgen</generator>
    <language>en</language>
    <lastBuildDate>Mon, 16 Jun 2025 00:12:07 +0000</lastBuildDate>
    <item>
      <title>Will It Still Be True Tomorrow? Multilingual Evergreen Question   Classification to Improve Trustworthy QA</title>
      <link>https://arxiv.org/abs/2505.21115</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.21115.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Sergey Pletenev, Maria Marina, Nikolay Ivanov, Daria Galimzianova, Nikita Krayko, Mikhail Salnikov, Vasily Konovalov, Alexander Panchenko, Viktor Moskvoretskii&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 124&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large Language Models (LLMs) often hallucinate in question answering (QA) tasks. A key yet underexplored factor contributing to this is the temporality of questions -- whether they are evergreen (answers remain stable over time) or mutable (answers change). In this work, we introduce EverGreenQA, the first multilingual QA dataset with evergreen labels, supporting both evaluation and training. Using EverGreenQA, we benchmark 12 modern LLMs to assess whether they encode question temporality explicitly (via verbalized judgments) or implicitly (via uncertainty signals). We also train EG-E5, a lightweight multilingual classifier that achieves SoTA performance on this task. Finally, we demonstrate the practical utility of evergreen classification across three applications: improving self-knowledge estimation, filtering QA datasets, and explaining GPT-4o retrieval behavior.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.21115</guid>
      <pubDate>Tue, 27 May 2025 12:35:13 +0000</pubDate>
    </item>
    <item>
      <title>Model Immunization from a Condition Number Perspective</title>
      <link>https://arxiv.org/abs/2505.23760</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.23760.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Amber Yijia Zheng, Cedar Site Bai, Brian Bullins, Raymond A. Yeh&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 8&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Model immunization aims to pre-train models that are difficult to fine-tune on harmful tasks while retaining their utility on other non-harmful tasks. Though prior work has shown empirical evidence for immunizing text-to-image models, the key understanding of when immunization is possible and a precise definition of an immunized model remain unclear. In this work, we propose a framework, based on the condition number of a Hessian matrix, to analyze model immunization for linear models. Building on this framework, we design an algorithm with regularization terms to control the resulting condition numbers after pre-training. Empirical results on linear models and non-linear deep-nets demonstrate the effectiveness of the proposed algorithm on model immunization. The code is available at https://github.com/amberyzheng/model-immunization-cond-num.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.23760</guid>
      <pubDate>Thu, 29 May 2025 17:59:48 +0000</pubDate>
    </item>
    <item>
      <title>FusionAudio-1.2M: Towards Fine-grained Audio Captioning with Multimodal   Contextual Fusion</title>
      <link>https://arxiv.org/abs/2506.01111</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.01111.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shunian Chen, Xinyuan Xie, Zheshu Chen, Liyan Zhao, Owen Lee, Zhan Su, Qilin Sun, Benyou Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 29&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; High-quality, large-scale audio captioning is crucial for advancing audio understanding, yet current automated methods often generate captions that lack fine-grained detail and contextual accuracy, primarily due to their reliance on limited unimodal or superficial multimodal information. Drawing inspiration from human auditory perception, which adeptly integrates cross-modal cues and performs sophisticated auditory scene analysis, we introduce a novel two-stage automated pipeline. This pipeline first employs specialized pretrained models to extract diverse contextual cues (e.g., speech, music, general sounds, and visual information from associated video). A large language model (LLM) then synthesizes these rich, multimodal inputs to generate detailed and context-aware audio captions. Key contributions of this work include: (1) the proposed scalable method for fine-grained audio caption generation; (2) FusionAudio, a new large-scale dataset comprising 1.2 million such detailed captions, combined with 6 million QA pairs; and (3) enhanced audio models developed using FusionAudio, specifically a CLAP-based audio encoder with superior audio-text alignment and instruction following. This paper paves the way for more nuanced and accurate automated understanding of complex audio environments. Code and data can be found in https://github.com/satsuki2486441738/FusionAudio.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.01111</guid>
      <pubDate>Sun, 01 Jun 2025 18:29:17 +0000</pubDate>
    </item>
    <item>
      <title>ExpertLongBench: Benchmarking Language Models on Expert-Level Long-Form   Generation Tasks with Structured Checklists</title>
      <link>https://arxiv.org/abs/2506.01241</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.01241.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jie Ruan, Inderjeet Nair, Shuyang Cao, Amy Liu, Sheza Munir, Micah Pollens-Dempsey, Tiffany Chiang, Lucy Kates, Nicholas David, Sihan Chen, Ruxin Yang, Yuqian Yang, Jasmine Gump, Tessa Bialek, Vivek Sankaran, Margo Schlanger, Lu Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 8&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; This paper introduces ExpertLongBench, an expert-level benchmark containing 11 tasks from 9 domains that reflect realistic expert workflows and applications. Beyond question answering, the application-driven tasks in ExpertLongBench demand long-form outputs that can exceed 5,000 tokens and strict adherence to domain-specific requirements. Notably, each task in ExpertLongBench includes a rubric, designed or validated by domain experts, to specify task requirements and guide output evaluation. Furthermore, we propose CLEAR, an evaluation framework that supports accurate evaluation of long-form model outputs in our benchmark. To achieve fine-grained, expert-aligned evaluation, CLEAR derives checklists from both model outputs and references by extracting information corresponding to items in the task-specific rubric. Checklist items for model outputs are then compared with corresponding items for reference outputs to assess their correctness, enabling grounded evaluation. We benchmark 11 large language models (LLMs) and analyze components in CLEAR, showing that (1) existing LLMs, with the top performer achieving only a 26.8% F1 score, require significant improvement for expert-level tasks; (2) models can generate content corresponding to the required aspects, though often not accurately; and (3) accurate checklist extraction and comparison in CLEAR can be achieved by open-weight models for more scalable and low-cost usage.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.01241</guid>
      <pubDate>Mon, 02 Jun 2025 01:39:02 +0000</pubDate>
    </item>
    <item>
      <title>Is Extending Modality The Right Path Towards Omni-Modality?</title>
      <link>https://arxiv.org/abs/2506.01872</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.01872.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Tinghui Zhu, Kai Zhang, Muhao Chen, Yu Su&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 21&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Omni-modal language models (OLMs) aim to integrate and reason over diverse input modalities--such as text, images, video, and audio--while maintaining strong language capabilities. Despite recent advancements, existing models, especially open-source ones, remain far from true omni-modality, struggling to generalize beyond the specific modality pairs they are trained on or to achieve strong performance when processing multi-modal inputs. We study the effect of extending modality, the dominant technique for training multimodal models, where an off-the-shelf language model is fine-tuned on target-domain and language data. Specifically, we investigate three key questions: (1) Does modality extension compromise core language abilities? (2) Can model merging effectively integrate independently fine-tuned modality-specific models to achieve omni-modality? (3) Does omni-modality extension lead to better knowledge sharing and generalization compared to sequential extension? Through extensive experiments, we analyze these trade-offs and provide insights into the feasibility of achieving true omni-modality using current approaches.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.01872</guid>
      <pubDate>Mon, 02 Jun 2025 17:01:40 +0000</pubDate>
    </item>
    <item>
      <title>Medical World Model: Generative Simulation of Tumor Evolution for   Treatment Planning</title>
      <link>https://arxiv.org/abs/2506.02327</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.02327.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yijun Yang, Zhao-Yang Wang, Qiuping Liu, Shuwen Sun, Kang Wang, Rama Chellappa, Zongwei Zhou, Alan Yuille, Lei Zhu, Yu-Dong Zhang, Jieneng Chen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 16&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Providing effective treatment and making informed clinical decisions are essential goals of modern medicine and clinical care. We are interested in simulating disease dynamics for clinical decision-making, leveraging recent advances in large generative models. To this end, we introduce the Medical World Model (MeWM), the first world model in medicine that visually predicts future disease states based on clinical decisions. MeWM comprises (i) vision-language models to serve as policy models, and (ii) tumor generative models as dynamics models. The policy model generates action plans, such as clinical treatments, while the dynamics model simulates tumor progression or regression under given treatment conditions. Building on this, we propose the inverse dynamics model that applies survival analysis to the simulated post-treatment tumor, enabling the evaluation of treatment efficacy and the selection of the optimal clinical action plan. As a result, the proposed MeWM simulates disease dynamics by synthesizing post-treatment tumors, with state-of-the-art specificity in Turing tests evaluated by radiologists. Simultaneously, its inverse dynamics model outperforms medical-specialized GPTs in optimizing individualized treatment protocols across all metrics. Notably, MeWM improves clinical decision-making for interventional physicians, boosting F1-score in selecting the optimal TACE protocol by 13%, paving the way for future integration of medical world models as the second readers.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.02327</guid>
      <pubDate>Mon, 02 Jun 2025 23:50:40 +0000</pubDate>
    </item>
    <item>
      <title>Splatting Physical Scenes: End-to-End Real-to-Sim from Imperfect Robot   Data</title>
      <link>https://arxiv.org/abs/2506.04120</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.04120.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Ben Moran, Mauro Comi, Steven Bohez, Tom Erez, Zhibin Li, Leonard Hasenclever&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 7&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Creating accurate, physical simulations directly from real-world robot motion holds great value for safe, scalable, and affordable robot learning, yet remains exceptionally challenging. Real robot data suffers from occlusions, noisy camera poses, dynamic scene elements, which hinder the creation of geometrically accurate and photorealistic digital twins of unseen objects. We introduce a novel real-to-sim framework tackling all these challenges at once. Our key insight is a hybrid scene representation merging the photorealistic rendering of 3D Gaussian Splatting with explicit object meshes suitable for physics simulation within a single representation. We propose an end-to-end optimization pipeline that leverages differentiable rendering and differentiable physics within MuJoCo to jointly refine all scene components - from object geometry and appearance to robot poses and physical parameters - directly from raw and imprecise robot trajectories. This unified optimization allows us to simultaneously achieve high-fidelity object mesh reconstruction, generate photorealistic novel views, and perform annotation-free robot pose calibration. We demonstrate the effectiveness of our approach both in simulation and on challenging real-world sequences using an ALOHA 2 bi-manual manipulator, enabling more practical and robust real-to-simulation pipelines.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.04120</guid>
      <pubDate>Wed, 04 Jun 2025 16:14:31 +0000</pubDate>
    </item>
    <item>
      <title>Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error   Diagnosis in GUI Automation</title>
      <link>https://arxiv.org/abs/2506.04614</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.04614.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuyang Wanyan, Xi Zhang, Haiyang Xu, Haowei Liu, Junyang Wang, Jiabo Ye, Yutong Kou, Ming Yan, Fei Huang, Xiaoshan Yang, Weiming Dong, Changsheng Xu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 15&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; In recent years, Multimodal Large Language Models (MLLMs) have been extensively utilized for multimodal reasoning tasks, including Graphical User Interface (GUI) automation. Unlike general offline multimodal tasks, GUI automation is executed in online interactive environments, necessitating step-by-step decision-making based on real-time status of the environment. This task has a lower tolerance for decision-making errors at each step, as any mistakes may cumulatively disrupt the process and potentially lead to irreversible outcomes like deletions or payments. To address these issues, we introduce a pre-operative critic mechanism that provides effective feedback prior to the actual execution, by reasoning about the potential outcome and correctness of actions. Specifically, we propose a Suggestion-aware Gradient Relative Policy Optimization (S-GRPO) strategy to construct our pre-operative critic model GUI-Critic-R1, incorporating a novel suggestion reward to enhance the reliability of the model's feedback. Furthermore, we develop a reasoning-bootstrapping based data collection pipeline to create a GUI-Critic-Train and a GUI-Critic-Test, filling existing gaps in GUI critic data. Static experiments on the GUI-Critic-Test across both mobile and web domains reveal that our GUI-Critic-R1 offers significant advantages in critic accuracy compared to current MLLMs. Dynamic evaluation on GUI automation benchmark further highlights the effectiveness and superiority of our model, as evidenced by improved success rates and operational efficiency.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.04614</guid>
      <pubDate>Thu, 05 Jun 2025 04:12:36 +0000</pubDate>
    </item>
    <item>
      <title>Agents of Change: Self-Evolving LLM Agents for Strategic Planning</title>
      <link>https://arxiv.org/abs/2506.04651</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.04651.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Nikolas Belle, Dakota Barnes, Alfonso Amayuelas, Ivan Bercovich, Xin Eric Wang, William Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 6&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in LLMs have enabled their use as autonomous agents across a range of tasks, yet they continue to struggle with formulating and adhering to coherent long-term strategies. In this paper, we investigate whether LLM agents can self-improve when placed in environments that explicitly challenge their strategic planning abilities. Using the board game Settlers of Catan, accessed through the open-source Catanatron framework, we benchmark a progression of LLM-based agents, from a simple game-playing agent to systems capable of autonomously rewriting their own prompts and their player agent's code. We introduce a multi-agent architecture in which specialized roles (Analyzer, Researcher, Coder, and Player) collaborate to iteratively analyze gameplay, research new strategies, and modify the agent's logic or prompt. By comparing manually crafted agents to those evolved entirely by LLMs, we evaluate how effectively these systems can diagnose failure and adapt over time. Our results show that self-evolving agents, particularly when powered by models like Claude 3.7 and GPT-4o, outperform static baselines by autonomously adopting their strategies, passing along sample behavior to game-playing agents, and demonstrating adaptive reasoning over multiple iterations.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.04651</guid>
      <pubDate>Thu, 05 Jun 2025 05:45:24 +0000</pubDate>
    </item>
    <item>
      <title>Truth in the Few: High-Value Data Selection for Efficient Multi-Modal   Reasoning</title>
      <link>https://arxiv.org/abs/2506.04755</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.04755.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shenshen Li, Kaiyuan Deng, Lei Wang, Hao Yang, Chong Peng, Peng Yan, Fumin Shen, Heng Tao Shen, Xing Xu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; While multi-modal large language models (MLLMs) have made significant progress in complex reasoning tasks via reinforcement learning, it is commonly believed that extensive training data is necessary for improving multi-modal reasoning ability, inevitably leading to data redundancy and substantial computational costs. However, can smaller high-value datasets match or outperform full corpora for multi-modal reasoning in MLLMs? In this work, we challenge this assumption through a key observation: meaningful multi-modal reasoning is triggered by only a sparse subset of training samples, termed cognitive samples, whereas the majority contribute marginally. Building on this insight, we propose a novel data selection paradigm termed Reasoning Activation Potential (RAP), which identifies cognitive samples by estimating each sample's potential to stimulate genuine multi-modal reasoning by two complementary estimators: 1) Causal Discrepancy Estimator (CDE) based on the potential outcome model principle, eliminates samples that overly rely on language priors by comparing outputs between multi-modal and text-only inputs; 2) Attention Confidence Estimator (ACE), which exploits token-level self-attention to discard samples dominated by irrelevant but over-emphasized tokens in intermediate reasoning stages. Moreover, we introduce a Difficulty-aware Replacement Module (DRM) to substitute trivial instances with cognitively challenging ones, thereby ensuring complexity for robust multi-modal reasoning. Experiments on six datasets show that our RAP method consistently achieves superior performance using only 9.3% of the training data, while reducing computational costs by over 43%. Our code is available at https://github.com/Leo-ssl/RAP.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.04755</guid>
      <pubDate>Thu, 05 Jun 2025 08:40:24 +0000</pubDate>
    </item>
    <item>
      <title>Debatable Intelligence: Benchmarking LLM Judges via Debate Speech   Evaluation</title>
      <link>https://arxiv.org/abs/2506.05062</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05062.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Noy Sternlicht, Ariel Gera, Roy Bar-Haim, Tom Hope, Noam Slonim&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 14&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce Debate Speech Evaluation as a novel and challenging benchmark for assessing LLM judges. Evaluating debate speeches requires a deep understanding of the speech at multiple levels, including argument strength and relevance, the coherence and organization of the speech, the appropriateness of its style and tone, and so on. This task involves a unique set of cognitive abilities that have previously received limited attention in systematic LLM benchmarking. To explore such skills, we leverage a dataset of over 600 meticulously annotated debate speeches and present the first in-depth analysis of how state-of-the-art LLMs compare to human judges on this task. Our findings reveal a nuanced picture: while larger models can approximate individual human judgments in some respects, they differ substantially in their overall judgment behavior. We also investigate the ability of frontier LLMs to generate persuasive, opinionated speeches, showing that models may perform at a human level on this task.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05062</guid>
      <pubDate>Thu, 05 Jun 2025 14:06:51 +0000</pubDate>
    </item>
    <item>
      <title>Sentinel: SOTA model to protect against prompt injections</title>
      <link>https://arxiv.org/abs/2506.05446</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05446.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Dror Ivry, Oran Nahum&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 22&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large Language Models (LLMs) are increasingly powerful but remain vulnerable to prompt injection attacks, where malicious inputs cause the model to deviate from its intended instructions. This paper introduces Sentinel, a novel detection model, qualifire/prompt-injection-sentinel, based on the \answerdotai/ModernBERT-large architecture. By leveraging ModernBERT's advanced features and fine-tuning on an extensive and diverse dataset comprising a few open-source and private collections, Sentinel achieves state-of-the-art performance. This dataset amalgamates varied attack types, from role-playing and instruction hijacking to attempts to generate biased content, alongside a broad spectrum of benign instructions, with private datasets specifically targeting nuanced error correction and real-world misclassifications. On a comprehensive, unseen internal test set, Sentinel demonstrates an average accuracy of 0.987 and an F1-score of 0.980. Furthermore, when evaluated on public benchmarks, it consistently outperforms strong baselines like protectai/deberta-v3-base-prompt-injection-v2. This work details Sentinel's architecture, its meticulous dataset curation, its training methodology, and a thorough evaluation, highlighting its superior detection capabilities.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05446</guid>
      <pubDate>Thu, 05 Jun 2025 14:07:15 +0000</pubDate>
    </item>
    <item>
      <title>ECoRAG: Evidentiality-guided Compression for Long Context RAG</title>
      <link>https://arxiv.org/abs/2506.05167</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05167.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yeonseok Jeong, Jinsu Kim, Dohyeon Lee, Seung-won Hwang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 9&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large Language Models (LLMs) have shown remarkable performance in Open-Domain Question Answering (ODQA) by leveraging external documents through Retrieval-Augmented Generation (RAG). To reduce RAG overhead, from longer context, context compression is necessary. However, prior compression methods do not focus on filtering out non-evidential information, which limit the performance in LLM-based RAG. We thus propose Evidentiality-guided RAG, or ECoRAG framework. ECoRAG improves LLM performance by compressing retrieved documents based on evidentiality, ensuring whether answer generation is supported by the correct evidence. As an additional step, ECoRAG reflects whether the compressed content provides sufficient evidence, and if not, retrieves more until sufficient. Experiments show that ECoRAG improves LLM performance on ODQA tasks, outperforming existing compression methods. Furthermore, ECoRAG is highly cost-efficient, as it not only reduces latency but also minimizes token usage by retaining only the necessary information to generate the correct answer. Code is available at https://github.com/ldilab/ECoRAG.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05167</guid>
      <pubDate>Thu, 05 Jun 2025 15:43:49 +0000</pubDate>
    </item>
    <item>
      <title>Time to Talk: LLM Agents for Asynchronous Group Communication in Mafia   Games</title>
      <link>https://arxiv.org/abs/2506.05309</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05309.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Niv Eckhaus, Uri Berger, Gabriel Stanovsky&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 13&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; LLMs are used predominantly in synchronous communication, where a human user and a model communicate in alternating turns. In contrast, many real-world settings are inherently asynchronous. For example, in group chats, online team meetings, or social games, there is no inherent notion of turns; therefore, the decision of when to speak forms a crucial part of the participant's decision making. In this work, we develop an adaptive asynchronous LLM-agent which, in addition to determining what to say, also decides when to say it. To evaluate our agent, we collect a unique dataset of online Mafia games, including both human participants, as well as our asynchronous agent. Overall, our agent performs on par with human players, both in game performance, as well as in its ability to blend in with the other human players. Our analysis shows that the agent's behavior in deciding when to speak closely mirrors human patterns, although differences emerge in message content. We release all our data and code to support and encourage further research for more realistic asynchronous communication between LLM agents. This work paves the way for integration of LLMs into realistic human group settings, from assistance in team discussions to educational and professional environments where complex social dynamics must be navigated.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05309</guid>
      <pubDate>Thu, 05 Jun 2025 17:53:44 +0000</pubDate>
    </item>
    <item>
      <title>MORSE-500: A Programmatically Controllable Video Benchmark to   Stress-Test Multimodal Reasoning</title>
      <link>https://arxiv.org/abs/2506.05523</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05523.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zikui Cai, Andrew Wang, Anirudh Satheesh, Ankit Nakhawa, Hyunwoo Jae, Keenan Powell, Minghui Liu, Neel Jay, Sungbin Oh, Xiyao Wang, Yongyuan Liang, Tom Goldstein, Furong Huang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 32&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Despite rapid advances in vision-language models (VLMs), current benchmarks for multimodal reasoning fall short in three key dimensions. First, they overwhelmingly rely on static images, failing to capture the temporal complexity of real-world environments. Second, they narrowly focus on mathematical problem-solving, neglecting the broader spectrum of reasoning skills -- including abstract, physical, planning, spatial, and temporal capabilities -- required for robust multimodal intelligence. Third, many benchmarks quickly saturate, offering limited headroom for diagnosing failure modes or measuring continued progress. We introduce MORSE-500 (Multimodal Reasoning Stress-test Environment), a video benchmark composed of 500 fully scripted clips with embedded questions spanning six complementary reasoning categories. Each instance is programmatically generated using deterministic Python scripts (via Manim, Matplotlib, MoviePy), generative video models, and curated real footage. This script-driven design allows fine-grained control over visual complexity, distractor density, and temporal dynamics -- enabling difficulty to be scaled systematically as models improve. Unlike static benchmarks that become obsolete once saturated, MORSE-500 is built to evolve: its controllable generation pipeline supports the creation of arbitrarily challenging new instances, making it ideally suited for stress-testing next-generation models. Initial experiments with state-of-the-art systems -- including various Gemini 2.5 Pro and OpenAI o3 which represent the strongest available at the time, alongside strong open-source models -- reveal substantial performance gaps across all categories, with particularly large deficits in abstract and planning tasks. We release the full dataset, generation scripts, and evaluation harness to support transparent, reproducible, and forward-looking multimodal reasoning research.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05523</guid>
      <pubDate>Thu, 05 Jun 2025 19:12:45 +0000</pubDate>
    </item>
    <item>
      <title>Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models</title>
      <link>https://arxiv.org/abs/2506.06395</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06395.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Pengyi Li, Matvey Skripkin, Alexander Zubrey, Andrey Kuznetsov, Ivan Oseledets&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 107&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large language models (LLMs) excel at reasoning, yet post-training remains critical for aligning their behavior with task goals. Existing reinforcement learning (RL) methods often depend on costly human annotations or external reward models. We propose Reinforcement Learning via Self-Confidence (RLSC), which uses the model's own confidence as reward signals-eliminating the need for labels, preference models, or reward engineering. Applied to Qwen2.5-Math-7B with only 16 samples per question and 10 or 20 training steps, RLSC improves accuracy by +13.4% on AIME2024, +21.2% on MATH500, +21.7% on Minerva Math, +20.8% on Olympiadbench, and +9.7% on AMC23. RLSC provides a simple, scalable post-training method for inference models, requiring only a small number of samples and unlabelled supervision.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06395</guid>
      <pubDate>Thu, 05 Jun 2025 19:55:15 +0000</pubDate>
    </item>
    <item>
      <title>PartCrafter: Structured 3D Mesh Generation via Compositional Latent   Diffusion Transformers</title>
      <link>https://arxiv.org/abs/2506.05573</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05573.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuchen Lin, Chenguo Lin, Panwang Pan, Honglei Yan, Yiqiang Feng, Yadong Mu, Katerina Fragkiadaki&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 58&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce PartCrafter, the first structured 3D generative model that jointly synthesizes multiple semantically meaningful and geometrically distinct 3D meshes from a single RGB image. Unlike existing methods that either produce monolithic 3D shapes or follow two-stage pipelines, i.e., first segmenting an image and then reconstructing each segment, PartCrafter adopts a unified, compositional generation architecture that does not rely on pre-segmented inputs. Conditioned on a single image, it simultaneously denoises multiple 3D parts, enabling end-to-end part-aware generation of both individual objects and complex multi-object scenes. PartCrafter builds upon a pretrained 3D mesh diffusion transformer (DiT) trained on whole objects, inheriting the pretrained weights, encoder, and decoder, and introduces two key innovations: (1) A compositional latent space, where each 3D part is represented by a set of disentangled latent tokens; (2) A hierarchical attention mechanism that enables structured information flow both within individual parts and across all parts, ensuring global coherence while preserving part-level detail during generation. To support part-level supervision, we curate a new dataset by mining part-level annotations from large-scale 3D object datasets. Experiments show that PartCrafter outperforms existing approaches in generating decomposable 3D meshes, including parts that are not directly visible in input images, demonstrating the strength of part-aware generative priors for 3D understanding and synthesis. Code and training data will be released.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05573</guid>
      <pubDate>Thu, 05 Jun 2025 20:30:28 +0000</pubDate>
    </item>
    <item>
      <title>SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward   Models in LLMs</title>
      <link>https://arxiv.org/abs/2506.05598</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05598.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Michael J Ryan, Omar Shaikh, Aditri Bhagirath, Daniel Frees, William Held, Diyi Yang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 7&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent calls for pluralistic alignment of Large Language Models (LLMs) encourage adapting models to diverse user preferences. However, most prior work on personalized reward models heavily rely on additional identity information, such as demographic details or a predefined set of preference categories. To this end, we introduce SynthesizeMe, an approach to inducing synthetic user personas from user interactions for personalized reward modeling. SynthesizeMe first generates and verifies reasoning to explain user preferences, then induces synthetic user personas from that reasoning, and finally filters to informative prior user interactions in order to build personalized prompts for a particular user. We show that using SynthesizeMe induced prompts improves personalized LLM-as-a-judge accuracy by 4.4% on Chatbot Arena. Combining SynthesizeMe derived prompts with a reward model achieves top performance on PersonalRewardBench: a new curation of user-stratified interactions with chatbots collected from 854 users of Chatbot Arena and PRISM.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05598</guid>
      <pubDate>Thu, 05 Jun 2025 21:23:16 +0000</pubDate>
    </item>
    <item>
      <title>Leveraging Self-Attention for Input-Dependent Soft Prompting in LLMs</title>
      <link>https://arxiv.org/abs/2506.05629</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05629.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Ananth Muppidi, Abhilash Nandy, Sambaran Bandyopadhyay&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 33&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; The performance of large language models in domain-specific tasks necessitates fine-tuning, which is computationally expensive and technically challenging. This paper focuses on parameter-efficient fine-tuning using soft prompting, a promising approach that adapts pre-trained models to downstream tasks by learning a small set of parameters. We propose a novel Input Dependent Soft Prompting technique with a self-Attention Mechanism (ID-SPAM) that generates soft prompts based on the input tokens and attends different tokens with varying importance. Our method is simple and efficient, keeping the number of trainable parameters small. We show the merits of the proposed approach compared to state-of-the-art techniques on various tasks and show the improved zero shot domain transfer capability.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05629</guid>
      <pubDate>Thu, 05 Jun 2025 23:13:22 +0000</pubDate>
    </item>
    <item>
      <title>Peer-Ranked Precision: Creating a Foundational Dataset for Fine-Tuning   Vision Models from DataSeeds' Annotated Imagery</title>
      <link>https://arxiv.org/abs/2506.05673</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05673.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Sajjad Abdoli, Freeman Lewin, Gediminas Vasiliauskas, Fabian Schonholz&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 9&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; The development of modern Artificial Intelligence (AI) models, particularly diffusion-based models employed in computer vision and image generation tasks, is undergoing a paradigmatic shift in development methodologies. Traditionally dominated by a "Model Centric" approach, in which performance gains were primarily pursued through increasingly complex model architectures and hyperparameter optimization, the field is now recognizing a more nuanced "Data-Centric" approach. This emergent framework foregrounds the quality, structure, and relevance of training data as the principal driver of model performance. To operationalize this paradigm shift, we introduce the DataSeeds.AI sample dataset (the "DSD"), initially comprised of approximately 10,610 high-quality human peer-ranked photography images accompanied by extensive multi-tier annotations. The DSD is a foundational computer vision dataset designed to usher in a new standard for commercial image datasets. Representing a small fraction of DataSeed.AI's 100 million-plus image catalog, the DSD provides a scalable foundation necessary for robust commercial and multimodal AI development. Through this in-depth exploratory analysis, we document the quantitative improvements generated by the DSD on specific models against known benchmarks and make the code and the trained models used in our evaluation publicly available.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05673</guid>
      <pubDate>Fri, 06 Jun 2025 01:50:28 +0000</pubDate>
    </item>
    <item>
      <title>CodeContests+: High-Quality Test Case Generation for Competitive   Programming</title>
      <link>https://arxiv.org/abs/2506.05817</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05817.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zihan Wang, Siyao Liu, Yang Sun, Hongyan Li, Kai Shen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 8&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Competitive programming, due to its high reasoning difficulty and precise correctness feedback, has become a key task for both training and evaluating the reasoning capabilities of large language models (LLMs). However, while a large amount of public problem data, such as problem statements and solutions, is available, the test cases of these problems are often difficult to obtain. Therefore, test case generation is a necessary task for building large-scale datasets, and the quality of the test cases directly determines the accuracy of the evaluation. In this paper, we introduce an LLM-based agent system that creates high-quality test cases for competitive programming problems. We apply this system to the CodeContests dataset and propose a new version with improved test cases, named CodeContests+. We evaluated the quality of test cases in CodeContestsPlus. First, we used 1.72 million submissions with pass/fail labels to examine the accuracy of these test cases in evaluation. The results indicated that CodeContests+ achieves significantly higher accuracy than CodeContests, particularly with a notably higher True Positive Rate (TPR). Subsequently, our experiments in LLM Reinforcement Learning (RL) further confirmed that improvements in test case quality yield considerable advantages for RL.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05817</guid>
      <pubDate>Fri, 06 Jun 2025 07:29:01 +0000</pubDate>
    </item>
    <item>
      <title>Audio-Aware Large Language Models as Judges for Speaking Styles</title>
      <link>https://arxiv.org/abs/2506.05984</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05984.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Cheng-Han Chiang, Xiaofei Wang, Chung-Ching Lin, Kevin Lin, Linjie Li, Radu Kopetz, Yao Qian, Zhendong Wang, Zhengyuan Yang, Hung-yi Lee, Lijuan Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 14&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Audio-aware large language models (ALLMs) can understand the textual and non-textual information in the audio input. In this paper, we explore using ALLMs as an automatic judge to assess the speaking styles of speeches. We use ALLM judges to evaluate the speeches generated by SLMs on two tasks: voice style instruction following and role-playing. The speaking style we consider includes emotion, volume, speaking pace, word emphasis, pitch control, and non-verbal elements. We use four spoken language models (SLMs) to complete the two tasks and use humans and ALLMs to judge the SLMs' responses. We compare two ALLM judges, GPT-4o-audio and Gemini-2.5-pro, with human evaluation results and show that the agreement between Gemini and human judges is comparable to the agreement between human evaluators. These promising results show that ALLMs can be used as a judge to evaluate SLMs. Our results also reveal that current SLMs, even GPT-4o-audio, still have room for improvement in controlling the speaking style and generating natural dialogues.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05984</guid>
      <pubDate>Fri, 06 Jun 2025 11:05:48 +0000</pubDate>
    </item>
    <item>
      <title>Bootstrapping World Models from Dynamics Models in Multimodal Foundation   Models</title>
      <link>https://arxiv.org/abs/2506.06006</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06006.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yifu Qiu, Yftah Ziser, Anna Korhonen, Shay B. Cohen, Edoardo M. Ponti&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 13&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; To what extent do vision-and-language foundation models possess a realistic world model (observation times action rightarrow observation) and a dynamics model (observation times observation rightarrow action), when actions are expressed through language? While open-source foundation models struggle with both, we find that fine-tuning them to acquire a dynamics model through supervision is significantly easier than acquiring a world model. In turn, dynamics models can be used to bootstrap world models through two main strategies: 1) weakly supervised learning from synthetic data and 2) inference time verification. Firstly, the dynamics model can annotate actions for unlabelled pairs of video frame observations to expand the training data. We further propose a new objective, where image tokens in observation pairs are weighted by their importance, as predicted by a recognition model. Secondly, the dynamics models can assign rewards to multiple samples of the world model to score them, effectively guiding search at inference time. We evaluate the world models resulting from both strategies through the task of action-centric image editing on Aurora-Bench. Our best model achieves a performance competitive with state-of-the-art image editing models, improving on them by a margin of 15% on real-world subsets according to GPT4o-as-judge, and achieving the best average human evaluation across all subsets of Aurora-Bench.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06006</guid>
      <pubDate>Fri, 06 Jun 2025 11:50:18 +0000</pubDate>
    </item>
    <item>
      <title>MIRIAD: Augmenting LLMs with millions of medical query-response pairs</title>
      <link>https://arxiv.org/abs/2506.06091</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06091.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Qinyue Zheng, Salman Abdullah, Sam Rawal, Cyril Zakka, Sophie Ostmeier, Maximilian Purk, Eduardo Reis, Eric J. Topol, Jure Leskovec, Michael Moor&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 7&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; LLMs are bound to transform healthcare with advanced decision support and flexible chat assistants. However, LLMs are prone to generate inaccurate medical content. To ground LLMs in high-quality medical knowledge, LLMs have been equipped with external knowledge via RAG, where unstructured medical knowledge is split into small text chunks that can be selectively retrieved and integrated into the LLMs context. Yet, existing RAG pipelines rely on raw, unstructured medical text, which can be noisy, uncurated and difficult for LLMs to effectively leverage. Systematic approaches to organize medical knowledge to best surface it to LLMs are generally lacking. To address these challenges, we introduce MIRIAD, a large-scale, curated corpus of 5,821,948 medical QA pairs, each rephrased from and grounded in a passage from peer-reviewed medical literature using a semi-automated pipeline combining LLM generation, filtering, grounding, and human annotation. Unlike prior medical corpora, which rely on unstructured text, MIRIAD encapsulates web-scale medical knowledge in an operationalized query-response format, which enables more targeted retrieval. Experiments on challenging medical QA benchmarks show that augmenting LLMs with MIRIAD improves accuracy up to 6.7% compared to unstructured RAG baselines with the same source corpus and with the same amount of retrieved text. Moreover, MIRIAD improved the ability of LLMs to detect medical hallucinations by 22.5 to 37% (increase in F1 score). We further introduce MIRIAD-Atlas, an interactive map of MIRIAD spanning 56 medical disciplines, enabling clinical users to visually explore, search, and refine medical knowledge. MIRIAD promises to unlock a wealth of down-stream applications, including medical information retrievers, enhanced RAG applications, and knowledge-grounded chat interfaces, which ultimately enables more reliable LLM applications in healthcare.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06091</guid>
      <pubDate>Fri, 06 Jun 2025 13:52:32 +0000</pubDate>
    </item>
    <item>
      <title>Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal   Learning</title>
      <link>https://arxiv.org/abs/2506.06205</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06205.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Sheng Chen, Peiyu He, Jiaxin Hu, Ziyang Liu, Yansheng Wang, Tao Xu, Chi Zhang, Chongchong Zhang, Chao An, Shiyu Cai, Duo Cao, Kangping Chen, Shuai Chu, Tianwei Chu, Mingdi Dan, Min Du, Weiwei Fang, Pengyou Fu, Junkai Hu, Xiaowei Jiang, Zhaodi Jiang, Fuxuan Li, Jun Li, Minghui Li, Mingyao Li, Yanchang Li, Zhibin Li, Guangming Liu, Kairui Liu, Lihao Liu, Weizhi Liu, Xiaoshun Liu, Yufei Liu, Yunfei Liu, Qiang Lu, Yuanfei Luo, Xiang Lv, Hongying Ma, Sai Ma, Lingxian Mi, Sha Sa, Hongxiang Shu, Lei Tian, Chengzhi Wang, Jiayu Wang, Kaijie Wang, Qingyi Wang, Renwen Wang, Tao Wang, Wei Wang, Xirui Wang, Chao Wei, Xuguang Wei, Zijun Xia, Zhaohao Xiao, Tingshuai Yan, Liyan Yang, Yifan Yang, Zhikai Yang, Zhong Yin, Li Yuan, Liuchun Yuan, Chi Zhang, Jinyang Zhang, Junhui Zhang, Linge Zhang, Zhenyi Zhang, Zheyu Zhang, Dongjie Zhu, Hang Li, Yangang Zhang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 27&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Modern robot navigation systems encounter difficulties in diverse and complex indoor environments. Traditional approaches rely on multiple modules with small models or rule-based systems and thus lack adaptability to new environments. To address this, we developed Astra, a comprehensive dual-model architecture, Astra-Global and Astra-Local, for mobile robot navigation. Astra-Global, a multimodal LLM, processes vision and language inputs to perform self and goal localization using a hybrid topological-semantic graph as the global map, and outperforms traditional visual place recognition methods. Astra-Local, a multitask network, handles local path planning and odometry estimation. Its 4D spatial-temporal encoder, trained through self-supervised learning, generates robust 4D features for downstream tasks. The planning head utilizes flow matching and a novel masked ESDF loss to minimize collision risks for generating local trajectories, and the odometry head integrates multi-sensor inputs via a transformer encoder to predict the relative pose of the robot. Deployed on real in-house mobile robots, Astra achieves high end-to-end mission success rate across diverse indoor environments.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06205</guid>
      <pubDate>Fri, 06 Jun 2025 16:08:47 +0000</pubDate>
    </item>
    <item>
      <title>Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence   with Egocentric-Exocentric Vision</title>
      <link>https://arxiv.org/abs/2506.06253</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06253.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuping He, Yifei Huang, Guo Chen, Lidong Lu, Baoqi Pei, Jilan Xu, Tong Lu, Yoichi Sato&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 8&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Perceiving the world from both egocentric (first-person) and exocentric (third-person) perspectives is fundamental to human cognition, enabling rich and complementary understanding of dynamic environments. In recent years, allowing the machines to leverage the synergistic potential of these dual perspectives has emerged as a compelling research direction in video understanding. In this survey, we provide a comprehensive review of video understanding from both exocentric and egocentric viewpoints. We begin by highlighting the practical applications of integrating egocentric and exocentric techniques, envisioning their potential collaboration across domains. We then identify key research tasks to realize these applications. Next, we systematically organize and review recent advancements into three main research directions: (1) leveraging egocentric data to enhance exocentric understanding, (2) utilizing exocentric data to improve egocentric analysis, and (3) joint learning frameworks that unify both perspectives. For each direction, we analyze a diverse set of tasks and relevant works. Additionally, we discuss benchmark datasets that support research in both perspectives, evaluating their scope, diversity, and applicability. Finally, we discuss limitations in current works and propose promising future research directions. By synthesizing insights from both perspectives, our goal is to inspire advancements in video understanding and artificial intelligence, bringing machines closer to perceiving the world in a human-like manner. A GitHub repo of related works can be found at https://github.com/ayiyayi/Awesome-Egocentric-and-Exocentric-Vision.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06253</guid>
      <pubDate>Fri, 06 Jun 2025 17:25:48 +0000</pubDate>
    </item>
    <item>
      <title>STARFlow: Scaling Latent Normalizing Flows for High-resolution Image   Synthesis</title>
      <link>https://arxiv.org/abs/2506.06276</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06276.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jiatao Gu, Tianrong Chen, David Berthelot, Huangjie Zheng, Yuyang Wang, Ruixiang Zhang, Laurent Dinh, Miguel Angel Bautista, Josh Susskind, Shuangfei Zhai&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 19&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We present STARFlow, a scalable generative model based on normalizing flows that achieves strong performance in high-resolution image synthesis. The core of STARFlow is Transformer Autoregressive Flow (TARFlow), which combines the expressive power of normalizing flows with the structured modeling capabilities of Autoregressive Transformers. We first establish the theoretical universality of TARFlow for modeling continuous distributions. Building on this foundation, we introduce several key architectural and algorithmic innovations to significantly enhance scalability: (1) a deep-shallow design, wherein a deep Transformer block captures most of the model representational capacity, complemented by a few shallow Transformer blocks that are computationally efficient yet substantially beneficial; (2) modeling in the latent space of pretrained autoencoders, which proves more effective than direct pixel-level modeling; and (3) a novel guidance algorithm that significantly boosts sample quality. Crucially, our model remains an end-to-end normalizing flow, enabling exact maximum likelihood training in continuous spaces without discretization. STARFlow achieves competitive performance in both class-conditional and text-conditional image generation tasks, approaching state-of-the-art diffusion models in sample quality. To our knowledge, this work is the first successful demonstration of normalizing flows operating effectively at this scale and resolution.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06276</guid>
      <pubDate>Fri, 06 Jun 2025 17:58:39 +0000</pubDate>
    </item>
    <item>
      <title>Saffron-1: Towards an Inference Scaling Paradigm for LLM Safety   Assurance</title>
      <link>https://arxiv.org/abs/2506.06444</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06444.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Ruizhong Qiu, Gaotang Li, Tianxin Wei, Jingrui He, Hanghang Tong&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 69&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Existing safety assurance research has primarily focused on training-phase alignment to instill safe behaviors into LLMs. However, recent studies have exposed these methods' susceptibility to diverse jailbreak attacks. Concurrently, inference scaling has significantly advanced LLM reasoning capabilities but remains unexplored in the context of safety assurance. Addressing this gap, our work pioneers inference scaling for robust and effective LLM safety against emerging threats. We reveal that conventional inference scaling techniques, despite their success in reasoning tasks, perform poorly in safety contexts, even falling short of basic approaches like Best-of-N Sampling. We attribute this inefficiency to a newly identified challenge, the exploration--efficiency dilemma, arising from the high computational overhead associated with frequent process reward model (PRM) evaluations. To overcome this dilemma, we propose SAFFRON, a novel inference scaling paradigm tailored explicitly for safety assurance. Central to our approach is the introduction of a multifurcation reward model (MRM) that significantly reduces the required number of reward model evaluations. To operationalize this paradigm, we further propose: (i) a partial supervision training objective for MRM, (ii) a conservative exploration constraint to prevent out-of-distribution explorations, and (iii) a Trie-based key--value caching strategy that facilitates cache sharing across sequences during tree search. Extensive experiments validate the effectiveness of our method. Additionally, we publicly release our trained multifurcation reward model (Saffron-1) and the accompanying token-level safety reward dataset (Safety4M) to accelerate future research in LLM safety. Our code, model, and data are publicly available at https://github.com/q-rz/saffron , and our project homepage is at https://q-rz.github.io/p/saffron .&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06444</guid>
      <pubDate>Fri, 06 Jun 2025 18:05:45 +0000</pubDate>
    </item>
    <item>
      <title>Geopolitical biases in LLMs: what are the "good" and the "bad" countries   according to contemporary language models</title>
      <link>https://arxiv.org/abs/2506.06751</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06751.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Mikhail Salnikov, Dmitrii Korzh, Ivan Lazichny, Elvir Karimov, Artyom Iudin, Ivan Oseledets, Oleg Y. Rogov, Alexander Panchenko, Natalia Loukachevitch, Elena Tutubalina&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 67&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; This paper evaluates geopolitical biases in LLMs with respect to various countries though an analysis of their interpretation of historical events with conflicting national perspectives (USA, UK, USSR, and China). We introduce a novel dataset with neutral event descriptions and contrasting viewpoints from different countries. Our findings show significant geopolitical biases, with models favoring specific national narratives. Additionally, simple debiasing prompts had a limited effect in reducing these biases. Experiments with manipulated participant labels reveal models' sensitivity to attribution, sometimes amplifying biases or recognizing inconsistencies, especially with swapped labels. This work highlights national narrative biases in LLMs, challenges the effectiveness of simple debiasing methods, and offers a framework and dataset for future geopolitical bias research.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06751</guid>
      <pubDate>Sat, 07 Jun 2025 10:45:17 +0000</pubDate>
    </item>
    <item>
      <title>The Illusion of Thinking: Understanding the Strengths and Limitations of   Reasoning Models via the Lens of Problem Complexity</title>
      <link>https://arxiv.org/abs/2506.06941</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06941.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, Mehrdad Farajtabar&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 12&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent generations of language models have introduced Large Reasoning Models (LRMs) that generate detailed thinking processes before providing answers. While these models demonstrate improved performance on reasoning benchmarks, their fundamental capabilities, scaling properties, and limitations remain insufficiently understood. Current evaluations primarily focus on established math and coding benchmarks, emphasizing final answer accuracy. However, this evaluation paradigm often suffers from contamination and does not provide insights into the reasoning traces. In this work, we systematically investigate these gaps with the help of controllable puzzle environments that allow precise manipulation of complexity while maintaining consistent logical structures. This setup enables the analysis of not only final answers but also the internal reasoning traces, offering insights into how LRMs think. Through extensive experiments, we show that LRMs face a complete accuracy collapse beyond certain complexities. Moreover, they exhibit a counterintuitive scaling limit: their reasoning effort increases with problem complexity up to a point, then declines despite having remaining token budget. By comparing LRMs with their standard LLM counterparts under same inference compute, we identify three performance regimes: (1) low-complexity tasks where standard models outperform LRMs, (2) medium-complexity tasks where LRMs demonstrates advantage, and (3) high-complexity tasks where both models face complete collapse. We found that LRMs have limitations in exact computation: they fail to use explicit algorithms and reason inconsistently across scales. We also investigate the reasoning traces in more depth, studying the patterns of explored solutions and analyzing the models' computational behavior, shedding light on their strengths, limitations, and raising questions about their reasoning capabilities.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06941</guid>
      <pubDate>Sat, 07 Jun 2025 22:42:29 +0000</pubDate>
    </item>
    <item>
      <title>What Makes a Good Natural Language Prompt?</title>
      <link>https://arxiv.org/abs/2506.06950</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06950.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Do Xuan Long, Duy Dinh, Ngoc-Hai Nguyen, Kenji Kawaguchi, Nancy F. Chen, Shafiq Joty, Min-Yen Kan&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 8&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; As large language models (LLMs) have progressed towards more human-like and human--AI communications have become prevalent, prompting has emerged as a decisive component. However, there is limited conceptual consensus on what exactly quantifies natural language prompts. We attempt to address this question by conducting a meta-analysis surveying more than 150 prompting-related papers from leading NLP and AI conferences from 2022 to 2025 and blogs. We propose a property- and human-centric framework for evaluating prompt quality, encompassing 21 properties categorized into six dimensions. We then examine how existing studies assess their impact on LLMs, revealing their imbalanced support across models and tasks, and substantial research gaps. Further, we analyze correlations among properties in high-quality natural language prompts, deriving prompting recommendations. We then empirically explore multi-property prompt enhancements in reasoning tasks, observing that single-property enhancements often have the greatest impact. Finally, we discover that instruction-tuning on property-enhanced prompts can result in better reasoning models. Our findings establish a foundation for property-centric prompt evaluation and optimization, bridging the gaps between human--AI communication and opening new prompting research directions.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06950</guid>
      <pubDate>Sat, 07 Jun 2025 23:19:27 +0000</pubDate>
    </item>
    <item>
      <title>LaTtE-Flow: Layerwise Timestep-Expert Flow-based Transformer</title>
      <link>https://arxiv.org/abs/2506.06952</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06952.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Ying Shen, Zhiyang Xu, Jiuhai Chen, Shizhe Diao, Jiaxin Zhang, Yuguang Yao, Joy Rimchala, Ismini Lourentzou, Lifu Huang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 10&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in multimodal foundation models unifying image understanding and generation have opened exciting avenues for tackling a wide range of vision-language tasks within a single framework. Despite progress, existing unified models typically require extensive pretraining and struggle to achieve the same level of performance compared to models dedicated to each task. Additionally, many of these models suffer from slow image generation speeds, limiting their practical deployment in real-time or resource-constrained settings. In this work, we propose Layerwise Timestep-Expert Flow-based Transformer (LaTtE-Flow), a novel and efficient architecture that unifies image understanding and generation within a single multimodal model. LaTtE-Flow builds upon powerful pretrained Vision-Language Models (VLMs) to inherit strong multimodal understanding capabilities, and extends them with a novel Layerwise Timestep Experts flow-based architecture for efficient image generation. LaTtE-Flow distributes the flow-matching process across specialized groups of Transformer layers, each responsible for a distinct subset of timesteps. This design significantly improves sampling efficiency by activating only a small subset of layers at each sampling timestep. To further enhance performance, we propose a Timestep-Conditioned Residual Attention mechanism for efficient information reuse across layers. Experiments demonstrate that LaTtE-Flow achieves strong performance on multimodal understanding tasks, while achieving competitive image generation quality with around 6x faster inference speed compared to recent unified multimodal models.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06952</guid>
      <pubDate>Sun, 08 Jun 2025 00:15:32 +0000</pubDate>
    </item>
    <item>
      <title>Lingshu: A Generalist Foundation Model for Unified Multimodal Medical   Understanding and Reasoning</title>
      <link>https://arxiv.org/abs/2506.07044</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07044.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; LASA Team, Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang, Chenghao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, Yu Sun, Junao Shen, Chaojun Wang, Jie Tan, Deli Zhao, Tingyang Xu, Hao Zhang, Yu Rong&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 100&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in understanding common visual elements, largely due to their large-scale datasets and advanced training strategies. However, their effectiveness in medical applications remains limited due to the inherent discrepancies between data and tasks in medical scenarios and those in the general domain. Concretely, existing medical MLLMs face the following critical limitations: (1) limited coverage of medical knowledge beyond imaging, (2) heightened susceptibility to hallucinations due to suboptimal data curation processes, (3) lack of reasoning capabilities tailored for complex medical scenarios. To address these challenges, we first propose a comprehensive data curation procedure that (1) efficiently acquires rich medical knowledge data not only from medical imaging but also from extensive medical texts and general-domain data; and (2) synthesizes accurate medical captions, visual question answering (VQA), and reasoning samples. As a result, we build a multimodal dataset enriched with extensive medical knowledge. Building on the curated data, we introduce our medical-specialized MLLM: Lingshu. Lingshu undergoes multi-stage training to embed medical expertise and enhance its task-solving capabilities progressively. Besides, we preliminarily explore the potential of applying reinforcement learning with verifiable rewards paradigm to enhance Lingshu's medical reasoning ability. Additionally, we develop MedEvalKit, a unified evaluation framework that consolidates leading multimodal and textual medical benchmarks for standardized, fair, and efficient model assessment. We evaluate the performance of Lingshu on three fundamental medical tasks, multimodal QA, text-based QA, and medical report generation. The results show that Lingshu consistently outperforms the existing open-source multimodal models on most tasks ...&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07044</guid>
      <pubDate>Sun, 08 Jun 2025 08:47:30 +0000</pubDate>
    </item>
    <item>
      <title>Frame Guidance: Training-Free Guidance for Frame-Level Control in Video   Diffusion Models</title>
      <link>https://arxiv.org/abs/2506.07177</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07177.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Sangwon Jang, Taekyung Ki, Jaehyeong Jo, Jaehong Yoon, Soo Ye Kim, Zhe Lin, Sung Ju Hwang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 22&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Advancements in diffusion models have significantly improved video quality, directing attention to fine-grained controllability. However, many existing methods depend on fine-tuning large-scale video models for specific tasks, which becomes increasingly impractical as model sizes continue to grow. In this work, we present Frame Guidance, a training-free guidance for controllable video generation based on frame-level signals, such as keyframes, style reference images, sketches, or depth maps. For practical training-free guidance, we propose a simple latent processing method that dramatically reduces memory usage, and apply a novel latent optimization strategy designed for globally coherent video generation. Frame Guidance enables effective control across diverse tasks, including keyframe guidance, stylization, and looping, without any training, compatible with any video models. Experimental results show that Frame Guidance can produce high-quality controlled videos for a wide range of tasks and input signals.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07177</guid>
      <pubDate>Sun, 08 Jun 2025 14:54:41 +0000</pubDate>
    </item>
    <item>
      <title>Pre-trained Large Language Models Learn Hidden Markov Models In-context</title>
      <link>https://arxiv.org/abs/2506.07298</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07298.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yijia Dai, Zhaolin Gao, Yahya Satter, Sarah Dean, Jennifer J. Sun&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 21&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Hidden Markov Models (HMMs) are foundational tools for modeling sequential data with latent Markovian structure, yet fitting them to real-world data remains computationally challenging. In this work, we show that pre-trained large language models (LLMs) can effectively model data generated by HMMs via in-context learning (ICL)x2013their ability to infer patterns from examples within a prompt. On a diverse set of synthetic HMMs, LLMs achieve predictive accuracy approaching the theoretical optimum. We uncover novel scaling trends influenced by HMM properties, and offer theoretical conjectures for these empirical observations. We also provide practical guidelines for scientists on using ICL as a diagnostic tool for complex data. On real-world animal decision-making tasks, ICL achieves competitive performance with models designed by human experts. To our knowledge, this is the first demonstration that ICL can learn and predict HMM-generated sequencesx2013an advance that deepens our understanding of in-context learning in LLMs and establishes its potential as a powerful tool for uncovering hidden structure in complex scientific data.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07298</guid>
      <pubDate>Sun, 08 Jun 2025 21:49:38 +0000</pubDate>
    </item>
    <item>
      <title>ConfQA: Answer Only If You Are Confident</title>
      <link>https://arxiv.org/abs/2506.07309</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07309.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yin Huang, Yifan Ethan Xu, Kai Sun, Vera Yan, Alicia Sun, Haidar Khan, Jimmy Nguyen, Mohammad Kachuee, Zhaojiang Lin, Yue Liu, Aaron Colak, Anuj Kumar, Wen-tau Yih, Xin Luna Dong&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 9&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Can we teach Large Language Models (LLMs) to refrain from hallucinating factual statements? In this paper we present a fine-tuning strategy that we call ConfQA, which can reduce hallucination rate from 20-40% to under 5% across multiple factuality benchmarks. The core idea is simple: when the LLM answers a question correctly, it is trained to continue with the answer; otherwise, it is trained to admit "I am unsure". But there are two key factors that make the training highly effective. First, we introduce a dampening prompt "answer only if you are confident" to explicitly guide the behavior, without which hallucination remains high as 15%-25%. Second, we leverage simple factual statements, specifically attribute values from knowledge graphs, to help LLMs calibrate the confidence, resulting in robust generalization across domains and question types. Building on this insight, we propose the Dual Neural Knowledge framework, which seamlessly select between internally parameterized neural knowledge and externally recorded symbolic knowledge based on ConfQA's confidence. The framework enables potential accuracy gains to beyond 95%, while reducing unnecessary external retrievals by over 30%.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07309</guid>
      <pubDate>Sun, 08 Jun 2025 22:51:46 +0000</pubDate>
    </item>
    <item>
      <title>Well Begun is Half Done: Low-resource Preference Alignment by   Weak-to-Strong Decoding</title>
      <link>https://arxiv.org/abs/2506.07434</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07434.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Feifan Song, Shaohang Wei, Wen Luo, Yuxuan Fan, Tianyu Liu, Guoyin Wang, Houfeng Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 7&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large Language Models (LLMs) require alignment with human preferences to avoid generating offensive, false, or meaningless content. Recently, low-resource methods for LLM alignment have been popular, while still facing challenges in obtaining both high-quality and aligned content. Motivated by the observation that the difficulty of generating aligned responses is concentrated at the beginning of decoding, we propose a novel framework, Weak-to-Strong Decoding (WSD), to enhance the alignment ability of base models by the guidance of a small aligned model. The small model first drafts well-aligned beginnings, followed by the large base model to continue the rest, controlled by a well-designed auto-switch mechanism. We also collect a new dataset, GenerAlign, to fine-tune a small-sized Pilot-3B as the draft model, which effectively enhances different base models under the WSD framework to outperform all baseline methods, while avoiding degradation on downstream tasks, termed as the alignment tax. Extensive experiments are further conducted to examine the impact of different settings and time efficiency, as well as analyses on the intrinsic mechanisms of WSD in depth.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07434</guid>
      <pubDate>Mon, 09 Jun 2025 05:21:22 +0000</pubDate>
    </item>
    <item>
      <title>CCI4.0: A Bilingual Pretraining Dataset for Enhancing Reasoning in Large   Language Models</title>
      <link>https://arxiv.org/abs/2506.07463</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07463.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Guang Liu, Liangdong Wang, Jijie Li, Yang Yu, Yao Xu, Jiabei Chen, Yu Bai, Feng Liao, Yonghua Lin&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 8&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce CCI4.0, a large-scale bilingual pre-training dataset engineered for superior data quality and diverse human-like reasoning trajectory. CCI4.0 occupies roughly 35 TB of disk space and comprises two sub-datasets: CCI4.0-M2-Base and CCI4.0-M2-CoT. CCI4.0-M2-Base combines a 5.2 TB carefully curated Chinese web corpus, a 22.5 TB English subset from Nemotron-CC, and diverse sources from math, wiki, arxiv, and code. Although these data are mostly sourced from well-processed datasets, the quality standards of various domains are dynamic and require extensive expert experience and labor to process. So, we propose a novel pipeline justifying data quality mainly based on models through two-stage deduplication, multiclassifier quality scoring, and domain-aware fluency filtering. We extract 4.5 billion pieces of CoT(Chain-of-Thought) templates, named CCI4.0-M2-CoT. Differing from the distillation of CoT from larger models, our proposed staged CoT extraction exemplifies diverse reasoning patterns and significantly decreases the possibility of hallucination. Empirical evaluations demonstrate that LLMs pre-trained in CCI4.0 benefit from cleaner, more reliable training signals, yielding consistent improvements in downstream tasks, especially in math and code reflection tasks. Our results underscore the critical role of rigorous data curation and human thinking templates in advancing LLM performance, shedding some light on automatically processing pretraining corpora.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07463</guid>
      <pubDate>Mon, 09 Jun 2025 06:14:19 +0000</pubDate>
    </item>
    <item>
      <title>SpatialLM: Training Large Language Models for Structured Indoor Modeling</title>
      <link>https://arxiv.org/abs/2506.07491</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07491.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yongsen Mao, Junhao Zhong, Chuan Fang, Jia Zheng, Rui Tang, Hao Zhu, Ping Tan, Zihan Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; SpatialLM is a large language model designed to process 3D point cloud data and generate structured 3D scene understanding outputs. These outputs include architectural elements like walls, doors, windows, and oriented object boxes with their semantic categories. Unlike previous methods which exploit task-specific network designs, our model adheres to the standard multimodal LLM architecture and is fine-tuned directly from open-source LLMs.   To train SpatialLM, we collect a large-scale, high-quality synthetic dataset consisting of the point clouds of 12,328 indoor scenes (54,778 rooms) with ground-truth 3D annotations, and conduct a careful study on various modeling and training decisions. On public benchmarks, our model gives state-of-the-art performance in layout estimation and competitive results in 3D object detection. With that, we show a feasible path for enhancing the spatial understanding capabilities of modern LLMs for applications in augmented reality, embodied robotics, and more.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07491</guid>
      <pubDate>Mon, 09 Jun 2025 07:10:58 +0000</pubDate>
    </item>
    <item>
      <title>BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulation</title>
      <link>https://arxiv.org/abs/2506.07530</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07530.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Hongyu Wang, Chuyan Xiong, Ruiping Wang, Xilin Chen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 18&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Vision-Language-Action (VLA) models have shown impressive capabilities across a wide range of robotics manipulation tasks. However, their growing model size poses significant challenges for deployment on resource-constrained robotic systems. While 1-bit pretraining has proven effective for enhancing the inference efficiency of large language models with minimal performance loss, its application to VLA models remains underexplored. In this work, we present BitVLA, the first 1-bit VLA model for robotics manipulation, in which every parameter is ternary, i.e., {-1, 0, 1}. To further reduce the memory footprint of the vision encoder, we propose the distillation-aware training strategy that compresses the full-precision encoder to 1.58-bit weights. During this process, a full-precision encoder serves as a teacher model to better align latent representations. Despite the lack of large-scale robotics pretraining, BitVLA achieves performance comparable to the state-of-the-art model OpenVLA-OFT with 4-bit post-training quantization on the LIBERO benchmark, while consuming only 29.8% of the memory. These results highlight BitVLA's promise for deployment on memory-constrained edge devices. We release the code and model weights in https://github.com/ustcwhy/BitVLA.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07530</guid>
      <pubDate>Mon, 09 Jun 2025 08:15:11 +0000</pubDate>
    </item>
    <item>
      <title>Eliciting Fine-Tuned Transformer Capabilities via Inference-Time   Techniques</title>
      <link>https://arxiv.org/abs/2506.08060</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08060.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Asankhaya Sharma&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 6&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large language models have transformed natural language processing, yet supervised fine-tuning (SFT) remains computationally intensive. This paper formally proves that capabilities acquired through SFT can be approximated by a base transformer model using inference-time techniques, specifically in-context learning (ICL), without altering model parameters, under idealized assumptions including unbounded computational resources and access to the fine-tuning dataset. We extend these results to practical scenarios with finite context lengths and partial dataset access. For text generation tasks with fixed output length l, datasets of size Oleft( m V{varepsilon^2} log m{delta} right) or, with bounded context, Oleft( l log V{varepsilon^2} log 1{delta} right) suffice to approximate fine-tuned behavior across m contexts within error varepsilon, where V is the vocabulary size and delta is the failure probability. For linear classification, datasets of size Oleft( d{varepsilon} right) or, with fixed context, Oleft( 1{varepsilon^2} log 1{delta} right) are sufficient, where d is the input dimension. Grounded in the Turing completeness of transformers, these results provide a theoretical foundation for resource-efficient deployment of large language models, with practical techniques like retrieval-augmented generation bridging theory to real-world applications.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08060</guid>
      <pubDate>Mon, 09 Jun 2025 08:37:19 +0000</pubDate>
    </item>
    <item>
      <title>GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular   Structure Recognition</title>
      <link>https://arxiv.org/abs/2506.07553</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07553.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jingchao Wang, Haote Yang, Jiang Wu, Yifan He, Xingjian Wei, Yinfan Wang, Chengjin Liu, Lingli Ge, Lijun Wu, Bin Wang, Dahua Lin, Conghui He&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 13&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Optical Chemical Structure Recognition (OCSR) is crucial for digitizing chemical knowledge by converting molecular images into machine-readable formats. While recent vision-language models (VLMs) have shown potential in this task, their image-captioning approach often struggles with complex molecular structures and inconsistent annotations. To overcome these challenges, we introduce GTR-Mol-VLM, a novel framework featuring two key innovations: (1) the Graph Traversal as Visual Chain of Thought mechanism that emulates human reasoning by incrementally parsing molecular graphs through sequential atom-bond predictions, and (2) the data-centric principle of Faithfully Recognize What You've Seen, which addresses the mismatch between abbreviated structures in images and their expanded annotations. To support model development, we constructed GTR-CoT-1.3M, a large-scale instruction-tuning dataset with meticulously corrected annotations, and introduced MolRec-Bench, the first benchmark designed for a fine-grained evaluation of graph-parsing accuracy in OCSR. Comprehensive experiments demonstrate that GTR-Mol-VLM achieves superior results compared to specialist models, chemistry-domain VLMs, and commercial general-purpose VLMs. Notably, in scenarios involving molecular images with functional group abbreviations, GTR-Mol-VLM outperforms the second-best baseline by approximately 14 percentage points, both in SMILES-based and graph-based metrics. We hope that this work will drive OCSR technology to more effectively meet real-world needs, thereby advancing the fields of cheminformatics and AI for Science. We will release GTR-CoT at https://github.com/opendatalab/GTR-CoT.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07553</guid>
      <pubDate>Mon, 09 Jun 2025 08:47:10 +0000</pubDate>
    </item>
    <item>
      <title>SAFEFLOW: A Principled Protocol for Trustworthy and Transactional   Autonomous Agent Systems</title>
      <link>https://arxiv.org/abs/2506.07564</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07564.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Peiran Li, Xinkai Zou, Zhuohang Wu, Ruifeng Li, Shuo Xing, Hanwen Zheng, Zhikai Hu, Yuping Wang, Haoxi Li, Qin Yuan, Yingmo Zhang, Zhengzhong Tu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 5&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in large language models (LLMs) and vision-language models (VLMs) have enabled powerful autonomous agents capable of complex reasoning and multi-modal tool use. Despite their growing capabilities, today's agent frameworks remain fragile, lacking principled mechanisms for secure information flow, reliability, and multi-agent coordination. In this work, we introduce SAFEFLOW, a new protocol-level framework for building trustworthy LLM/VLM-based agents. SAFEFLOW enforces fine-grained information flow control (IFC), precisely tracking provenance, integrity, and confidentiality of all the data exchanged between agents, tools, users, and environments. By constraining LLM reasoning to respect these security labels, SAFEFLOW prevents untrusted or adversarial inputs from contaminating high-integrity decisions. To ensure robustness in concurrent multi-agent settings, SAFEFLOW introduces transactional execution, conflict resolution, and secure scheduling over shared state, preserving global consistency across agents. We further introduce mechanisms, including write-ahead logging, rollback, and secure caches, that further enhance resilience against runtime errors and policy violations. To validate the performances, we built SAFEFLOWBENCH, a comprehensive benchmark suite designed to evaluate agent reliability under adversarial, noisy, and concurrent operational conditions. Extensive experiments demonstrate that agents built with SAFEFLOW maintain impressive task performance and security guarantees even in hostile environments, substantially outperforming state-of-the-art. Together, SAFEFLOW and SAFEFLOWBENCH lay the groundwork for principled, robust, and secure agent ecosystems, advancing the frontier of reliable autonomy.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07564</guid>
      <pubDate>Mon, 09 Jun 2025 09:04:37 +0000</pubDate>
    </item>
    <item>
      <title>Through the Valley: Path to Effective Long CoT Training for Small   Language Models</title>
      <link>https://arxiv.org/abs/2506.07712</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07712.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Renjie Luo, Jiaxi Li, Chen Huang, Wei Lu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 17&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Long chain-of-thought (CoT) supervision has become a common strategy to enhance reasoning in language models. While effective for large models, we identify a phenomenon we call Long CoT Degradation, in which small language models (SLMs; &lt;=3B parameters) trained on limited long CoT data experience significant performance deterioration. Through extensive experiments on the Qwen2.5, LLaMA3 and Gemma3 families, we demonstrate that this degradation is widespread across SLMs. In some settings, models trained on only 8k long CoT examples lose up to 75% of their original performance before fine-tuning. Strikingly, we further observe that for some particularly small models, even training on 220k long CoT examples fails to recover or surpass their original performance prior to fine-tuning. Our analysis attributes this effect to error accumulation: while longer responses increase the capacity for multi-step reasoning, they also amplify the risk of compounding mistakes. Furthermore, we find that Long CoT Degradation may negatively impacts downstream reinforcement learning (RL), although this can be alleviated by sufficiently scaled supervised fine-tuning (SFT). Our findings challenge common assumptions about the benefits of long CoT training for SLMs and offer practical guidance for building more effective small-scale reasoning models.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07712</guid>
      <pubDate>Mon, 09 Jun 2025 12:56:41 +0000</pubDate>
    </item>
    <item>
      <title>Image Reconstruction as a Tool for Feature Analysis</title>
      <link>https://arxiv.org/abs/2506.07803</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07803.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Eduard Allakhverdov, Dmitrii Tarasov, Elizaveta Goncharova, Andrey Kuznetsov&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 28&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Vision encoders are increasingly used in modern applications, from vision-only models to multimodal systems such as vision-language models. Despite their remarkable success, it remains unclear how these architectures represent features internally. Here, we propose a novel approach for interpreting vision features via image reconstruction. We compare two related model families, SigLIP and SigLIP2, which differ only in their training objective, and show that encoders pre-trained on image-based tasks retain significantly more image information than those trained on non-image tasks such as contrastive learning. We further apply our method to a range of vision encoders, ranking them by the informativeness of their feature representations. Finally, we demonstrate that manipulating the feature space yields predictable changes in reconstructed images, revealing that orthogonal rotations (rather than spatial transformations) control color encoding. Our approach can be applied to any vision encoder, shedding light on the inner structure of its feature space. The code and model weights to reproduce the experiments are available in GitHub.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07803</guid>
      <pubDate>Mon, 09 Jun 2025 14:32:18 +0000</pubDate>
    </item>
    <item>
      <title>MiniCPM4: Ultra-Efficient LLMs on End Devices</title>
      <link>https://arxiv.org/abs/2506.07900</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07900.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; MiniCPM Team, Chaojun Xiao, Yuxuan Li, Xu Han, Yuzhuo Bai, Jie Cai, Haotian Chen, Wentong Chen, Xin Cong, Ganqu Cui, Ning Ding, Shengdan Fan, Yewei Fang, Zixuan Fu, Wenyu Guan, Yitong Guan, Junshao Guo, Yufeng Han, Bingxiang He, Yuxiang Huang, Cunliang Kong, Qiuzuo Li, Siyuan Li, Wenhao Li, Yanghao Li, Yishan Li, Zhen Li, Dan Liu, Biyuan Lin, Yankai Lin, Xiang Long, Quanyu Lu, Yaxi Lu, Peiyan Luo, Hongya Lyu, Litu Ou, Yinxu Pan, Zekai Qu, Qundong Shi, Zijun Song, Jiayuan Su, Zhou Su, Ao Sun, Xianghui Sun, Peijun Tang, Fangzheng Wang, Feng Wang, Shuo Wang, Yudong Wang, Yesai Wu, Zhenyu Xiao, Jie Xie, Zihao Xie, Yukun Yan, Jiarui Yuan, Kaihuo Zhang, Lei Zhang, Linyue Zhang, Xueren Zhang, Yudi Zhang, Hengyu Zhao, Weilin Zhao, Weilun Zhao, Yuanqian Zhao, Zhi Zheng, Ge Zhou, Jie Zhou, Wei Zhou, Zihan Zhou, Zixuan Zhou, Zhiyuan Liu, Guoyang Zeng, Chao Jia, Dahai Li, Maosong Sun&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 74&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; This paper introduces MiniCPM4, a highly efficient large language model (LLM) designed explicitly for end-side devices. We achieve this efficiency through systematic innovation in four key dimensions: model architecture, training data, training algorithms, and inference systems. Specifically, in terms of model architecture, we propose InfLLM v2, a trainable sparse attention mechanism that accelerates both prefilling and decoding phases for long-context processing. Regarding training data, we propose UltraClean, an efficient and accurate pre-training data filtering and generation strategy, and UltraChat v2, a comprehensive supervised fine-tuning dataset. These datasets enable satisfactory model performance to be achieved using just 8 trillion training tokens. Regarding training algorithms, we propose ModelTunnel v2 for efficient pre-training strategy search, and improve existing post-training methods by introducing chunk-wise rollout for load-balanced reinforcement learning and data-efficient tenary LLM, BitCPM. Regarding inference systems, we propose CPM.cu that integrates sparse attention, model quantization, and speculative sampling to achieve efficient prefilling and decoding. To meet diverse on-device requirements, MiniCPM4 is available in two versions, with 0.5B and 8B parameters, respectively. Sufficient evaluation results show that MiniCPM4 outperforms open-source models of similar size across multiple benchmarks, highlighting both its efficiency and effectiveness. Notably, MiniCPM4-8B demonstrates significant speed improvements over Qwen3-8B when processing long sequences. Through further adaptation, MiniCPM4 successfully powers diverse applications, including trustworthy survey generation and tool use with model context protocol, clearly showcasing its broad usability.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07900</guid>
      <pubDate>Mon, 09 Jun 2025 16:16:50 +0000</pubDate>
    </item>
    <item>
      <title>Solving Inequality Proofs with Large Language Models</title>
      <link>https://arxiv.org/abs/2506.07927</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07927.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jiayi Sheng, Luna Lyu, Jikai Jin, Tony Xia, Alex Gu, James Zou, Pan Lu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 20&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Inequality proving, crucial across diverse scientific and mathematical fields, tests advanced reasoning skills such as discovering tight bounds and strategic theorem application. This makes it a distinct, demanding frontier for large language models (LLMs), offering insights beyond general mathematical problem-solving. Progress in this area is hampered by existing datasets that are often scarce, synthetic, or rigidly formal. We address this by proposing an informal yet verifiable task formulation, recasting inequality proving into two automatically checkable subtasks: bound estimation and relation prediction. Building on this, we release IneqMath, an expert-curated dataset of Olympiad-level inequalities, including a test set and training corpus enriched with step-wise solutions and theorem annotations. We also develop a novel LLM-as-judge evaluation framework, combining a final-answer judge with four step-wise judges designed to detect common reasoning flaws. A systematic evaluation of 29 leading LLMs on IneqMath reveals a surprising reality: even top models like o1 achieve less than 10% overall accuracy under step-wise scrutiny; this is a drop of up to 65.5% from their accuracy considering only final answer equivalence. This discrepancy exposes fragile deductive chains and a critical gap for current LLMs between merely finding an answer and constructing a rigorous proof. Scaling model size and increasing test-time computation yield limited gains in overall proof correctness. Instead, our findings highlight promising research directions such as theorem-guided reasoning and self-refinement. Code and data are available at https://ineqmath.github.io/.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07927</guid>
      <pubDate>Mon, 09 Jun 2025 16:43:38 +0000</pubDate>
    </item>
    <item>
      <title>Squeeze3D: Your 3D Generation Model is Secretly an Extreme Neural   Compressor</title>
      <link>https://arxiv.org/abs/2506.07932</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07932.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Rishit Dagli, Yushi Guan, Sankeerth Durvasula, Mohammadreza Mofayezi, Nandita Vijaykumar&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 10&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We propose Squeeze3D, a novel framework that leverages implicit prior knowledge learnt by existing pre-trained 3D generative models to compress 3D data at extremely high compression ratios. Our approach bridges the latent spaces between a pre-trained encoder and a pre-trained generation model through trainable mapping networks. Any 3D model represented as a mesh, point cloud, or a radiance field is first encoded by the pre-trained encoder and then transformed (i.e. compressed) into a highly compact latent code. This latent code can effectively be used as an extremely compressed representation of the mesh or point cloud. A mapping network transforms the compressed latent code into the latent space of a powerful generative model, which is then conditioned to recreate the original 3D model (i.e. decompression). Squeeze3D is trained entirely on generated synthetic data and does not require any 3D datasets. The Squeeze3D architecture can be flexibly used with existing pre-trained 3D encoders and existing generative models. It can flexibly support different formats, including meshes, point clouds, and radiance fields. Our experiments demonstrate that Squeeze3D achieves compression ratios of up to 2187x for textured meshes, 55x for point clouds, and 619x for radiance fields while maintaining visual quality comparable to many existing methods. Squeeze3D only incurs a small compression and decompression latency since it does not involve training object-specific networks to compress an object.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07932</guid>
      <pubDate>Mon, 09 Jun 2025 16:52:10 +0000</pubDate>
    </item>
    <item>
      <title>Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction</title>
      <link>https://arxiv.org/abs/2506.07976</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07976.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Junhong Shen, Hao Bai, Lunjun Zhang, Yifei Zhou, Amrith Setlur, Shengbang Tong, Diego Caples, Nan Jiang, Tong Zhang, Ameet Talwalkar, Aviral Kumar&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 6&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; The current paradigm of test-time scaling relies on generating long reasoning traces ("thinking" more) before producing a response. In agent problems that require interaction, this can be done by generating thinking traces before acting in the world. However, this process does not allow agents to acquire new information from the environment or adapt their behavior over time. In this work, we propose to scale test-time interaction, an untapped dimension of test-time scaling that increases the agent's interaction horizon to enable running rich behaviors such as exploration, backtracking, and dynamic re-planning within a single rollout. To demonstrate the promise of this scaling dimension, we study the domain of web agents. We first show that even prompting-based interaction scaling without any training can improve task success on web benchmarks non-trivially. Building on this, we introduce TTI (Test-Time Interaction), a curriculum-based online reinforcement learning (RL) approach that trains agents by adaptively adjusting their rollout lengths. Using a Gemma 3 12B model, TTI produces state-of-the-art open-source, open-data web agents on WebVoyager and WebArena benchmarks. We further show that TTI enables agents to balance exploration and exploitation adaptively. Our results establish interaction scaling as a powerful, complementary axis to scaling per-step compute, offering new avenues for training adaptive agents.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07976</guid>
      <pubDate>Mon, 09 Jun 2025 17:50:02 +0000</pubDate>
    </item>
    <item>
      <title>OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation</title>
      <link>https://arxiv.org/abs/2506.07977</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07977.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jingjing Chang, Yixiao Fang, Peng Xing, Shuhan Wu, Wei Cheng, Rui Wang, Xianfang Zeng, Gang Yu, Hai-Bao Chen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 38&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Text-to-image (T2I) models have garnered significant attention for generating high-quality images aligned with text prompts. However, rapid T2I model advancements reveal limitations in early benchmarks, lacking comprehensive evaluations, for example, the evaluation on reasoning, text rendering and style. Notably, recent state-of-the-art models, with their rich knowledge modeling capabilities, show promising results on the image generation problems requiring strong reasoning ability, yet existing evaluation systems have not adequately addressed this frontier. To systematically address these gaps, we introduce OneIG-Bench, a meticulously designed comprehensive benchmark framework for fine-grained evaluation of T2I models across multiple dimensions, including prompt-image alignment, text rendering precision, reasoning-generated content, stylization, and diversity. By structuring the evaluation, this benchmark enables in-depth analysis of model performance, helping researchers and practitioners pinpoint strengths and bottlenecks in the full pipeline of image generation. Specifically, OneIG-Bench enables flexible evaluation by allowing users to focus on a particular evaluation subset. Instead of generating images for the entire set of prompts, users can generate images only for the prompts associated with the selected dimension and complete the corresponding evaluation accordingly. Our codebase and dataset are now publicly available to facilitate reproducible evaluation studies and cross-model comparisons within the T2I research community.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07977</guid>
      <pubDate>Mon, 09 Jun 2025 17:50:21 +0000</pubDate>
    </item>
    <item>
      <title>Rethinking Cross-Modal Interaction in Multimodal Diffusion Transformers</title>
      <link>https://arxiv.org/abs/2506.07986</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07986.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zhengyao Lv, Tianlin Pan, Chenyang Si, Zhaoxi Chen, Wangmeng Zuo, Ziwei Liu, Kwan-Yee K. Wong&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 17&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Multimodal Diffusion Transformers (MM-DiTs) have achieved remarkable progress in text-driven visual generation. However, even state-of-the-art MM-DiT models like FLUX struggle with achieving precise alignment between text prompts and generated content. We identify two key issues in the attention mechanism of MM-DiT, namely 1) the suppression of cross-modal attention due to token imbalance between visual and textual modalities and 2) the lack of timestep-aware attention weighting, which hinder the alignment. To address these issues, we propose Temperature-Adjusted Cross-modal Attention (TACA), a parameter-efficient method that dynamically rebalances multimodal interactions through temperature scaling and timestep-dependent adjustment. When combined with LoRA fine-tuning, TACA significantly enhances text-image alignment on the T2I-CompBench benchmark with minimal computational overhead. We tested TACA on state-of-the-art models like FLUX and SD3.5, demonstrating its ability to improve image-text alignment in terms of object appearance, attribute binding, and spatial relationships. Our findings highlight the importance of balancing cross-modal attention in improving semantic fidelity in text-to-image diffusion models. Our codes are publicly available at https://github.com/Vchitect/TACA&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07986</guid>
      <pubDate>Mon, 09 Jun 2025 17:54:04 +0000</pubDate>
    </item>
    <item>
      <title>Aligning Text, Images, and 3D Structure Token-by-Token</title>
      <link>https://arxiv.org/abs/2506.08002</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08002.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Aadarsh Sahoo, Vansh Tibrewal, Georgia Gkioxari&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 17&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Creating machines capable of understanding the world in 3D is essential in assisting designers that build and edit 3D environments and robots navigating and interacting within a three-dimensional space. Inspired by advances in language and image modeling, we investigate the potential of autoregressive models for a new modality: structured 3D scenes. To this end, we propose a unified LLM framework that aligns language, images, and 3D scenes and provide a detailed ''cookbook'' outlining critical design choices for achieving optimal training and performance addressing key questions related to data representation, modality-specific objectives, and more. We evaluate performance across four core 3D tasks -- rendering, recognition, instruction-following, and question-answering -- and four 3D datasets, synthetic and real-world. We extend our approach to reconstruct complex 3D object shapes by enriching our 3D modality with quantized shape encodings, and show our model's effectiveness on real-world 3D object recognition tasks. Project webpage: https://glab-caltech.github.io/kyvo/&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08002</guid>
      <pubDate>Mon, 09 Jun 2025 17:59:37 +0000</pubDate>
    </item>
    <item>
      <title>Dynamic View Synthesis as an Inverse Problem</title>
      <link>https://arxiv.org/abs/2506.08004</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08004.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Hidir Yesiltepe, Pinar Yanardag&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 5&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; In this work, we address dynamic view synthesis from monocular videos as an inverse problem in a training-free setting. By redesigning the noise initialization phase of a pre-trained video diffusion model, we enable high-fidelity dynamic view synthesis without any weight updates or auxiliary modules. We begin by identifying a fundamental obstacle to deterministic inversion arising from zero-terminal signal-to-noise ratio (SNR) schedules and resolve it by introducing a novel noise representation, termed K-order Recursive Noise Representation. We derive a closed form expression for this representation, enabling precise and efficient alignment between the VAE-encoded and the DDIM inverted latents. To synthesize newly visible regions resulting from camera motion, we introduce Stochastic Latent Modulation, which performs visibility aware sampling over the latent space to complete occluded regions. Comprehensive experiments demonstrate that dynamic view synthesis can be effectively performed through structured latent manipulation in the noise initialization phase.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08004</guid>
      <pubDate>Mon, 09 Jun 2025 17:59:47 +0000</pubDate>
    </item>
    <item>
      <title>Dreamland: Controllable World Creation with Simulator and Generative   Models</title>
      <link>https://arxiv.org/abs/2506.08006</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08006.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Sicheng Mo, Ziyang Leng, Leon Liu, Weizhen Wang, Honglin He, Bolei Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 7&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large-scale video generative models can synthesize diverse and realistic visual content for dynamic world creation, but they often lack element-wise controllability, hindering their use in editing scenes and training embodied AI agents. We propose Dreamland, a hybrid world generation framework combining the granular control of a physics-based simulator and the photorealistic content output of large-scale pretrained generative models. In particular, we design a layered world abstraction that encodes both pixel-level and object-level semantics and geometry as an intermediate representation to bridge the simulator and the generative model. This approach enhances controllability, minimizes adaptation cost through early alignment with real-world distributions, and supports off-the-shelf use of existing and future pretrained generative models. We further construct a D3Sim dataset to facilitate the training and evaluation of hybrid generation pipelines. Experiments demonstrate that Dreamland outperforms existing baselines with 50.8% improved image quality, 17.9% stronger controllability, and has great potential to enhance embodied agent training. Code and data will be made available.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08006</guid>
      <pubDate>Mon, 09 Jun 2025 17:59:52 +0000</pubDate>
    </item>
    <item>
      <title>Reinforcement Pre-Training</title>
      <link>https://arxiv.org/abs/2506.08007</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08007.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Qingxiu Dong, Li Dong, Yao Tang, Tianzhu Ye, Yutao Sun, Zhifang Sui, Furu Wei&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 204&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling paradigm for large language models and reinforcement learning (RL). Specifically, we reframe next-token prediction as a reasoning task trained using RL, where it receives verifiable rewards for correctly predicting the next token for a given context. RPT offers a scalable method to leverage vast amounts of text data for general-purpose RL, rather than relying on domain-specific annotated answers. By incentivizing the capability of next-token reasoning, RPT significantly improves the language modeling accuracy of predicting the next tokens. Moreover, RPT provides a strong pre-trained foundation for further reinforcement fine-tuning. The scaling curves show that increased training compute consistently improves the next-token prediction accuracy. The results position RPT as an effective and promising scaling paradigm to advance language model pre-training.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08007</guid>
      <pubDate>Mon, 09 Jun 2025 17:59:53 +0000</pubDate>
    </item>
    <item>
      <title>Hidden in plain sight: VLMs overlook their visual representations</title>
      <link>https://arxiv.org/abs/2506.08008</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08008.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Stephanie Fu, Tyler Bonnen, Devin Guillory, Trevor Darrell&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 8&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Language provides a natural interface to specify and evaluate performance on visual tasks. To realize this possibility, vision language models (VLMs) must successfully integrate visual and linguistic information. Our work compares VLMs to a direct readout of their visual encoders to understand their ability to integrate across these modalities. Across a series of vision-centric benchmarks (e.g., depth estimation, correspondence), we find that VLMs perform substantially worse than their visual encoders, dropping to near-chance performance. We investigate these results through a series of analyses across the entire VLM: namely 1) the degradation of vision representations, 2) brittleness to task prompt, and 3) the language model's role in solving the task. We find that the bottleneck in performing these vision-centric tasks lies in this third category; VLMs are not effectively using visual information easily accessible throughout the entire model, and they inherit the language priors present in the LLM. Our work helps diagnose the failure modes of open-source VLMs, and presents a series of evaluations useful for future investigations into visual understanding within VLMs.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08008</guid>
      <pubDate>Mon, 09 Jun 2025 17:59:54 +0000</pubDate>
    </item>
    <item>
      <title>Self Forcing: Bridging the Train-Test Gap in Autoregressive Video   Diffusion</title>
      <link>https://arxiv.org/abs/2506.08009</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08009.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, Eli Shechtman&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 17&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce Self Forcing, a novel training paradigm for autoregressive video diffusion models. It addresses the longstanding issue of exposure bias, where models trained on ground-truth context must generate sequences conditioned on their own imperfect outputs during inference. Unlike prior methods that denoise future frames based on ground-truth context frames, Self Forcing conditions each frame's generation on previously self-generated outputs by performing autoregressive rollout with key-value (KV) caching during training. This strategy enables supervision through a holistic loss at the video level that directly evaluates the quality of the entire generated sequence, rather than relying solely on traditional frame-wise objectives. To ensure training efficiency, we employ a few-step diffusion model along with a stochastic gradient truncation strategy, effectively balancing computational cost and performance. We further introduce a rolling KV cache mechanism that enables efficient autoregressive video extrapolation. Extensive experiments demonstrate that our approach achieves real-time streaming video generation with sub-second latency on a single GPU, while matching or even surpassing the generation quality of significantly slower and non-causal diffusion models. Project website: http://self-forcing.github.io/&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08009</guid>
      <pubDate>Mon, 09 Jun 2025 17:59:55 +0000</pubDate>
    </item>
    <item>
      <title>GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection   Behavior</title>
      <link>https://arxiv.org/abs/2506.08012</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08012.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Penghao Wu, Shengnan Ma, Bo Wang, Jiaheng Yu, Lewei Lu, Ziwei Liu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 7&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Multimodal Large Language Models (MLLMs) have shown great potential in revolutionizing Graphical User Interface (GUI) automation. However, existing GUI models mostly rely on learning from nearly error-free offline trajectories, thus lacking reflection and error recovery capabilities. To bridge this gap, we propose GUI-Reflection, a novel framework that explicitly integrates self-reflection and error correction capabilities into end-to-end multimodal GUI models throughout dedicated training stages: GUI-specific pre-training, offline supervised fine-tuning (SFT), and online reflection tuning. GUI-reflection enables self-reflection behavior emergence with fully automated data generation and learning processes without requiring any human annotation. Specifically, 1) we first propose scalable data pipelines to automatically construct reflection and error correction data from existing successful trajectories. While existing GUI models mainly focus on grounding and UI understanding ability, we propose the GUI-Reflection Task Suite to learn and evaluate reflection-oriented abilities explicitly. 2) Furthermore, we built a diverse and efficient environment for online training and data collection of GUI models on mobile devices. 3) We also present an iterative online reflection tuning algorithm leveraging the proposed environment, enabling the model to continuously enhance its reflection and error correction abilities. Our framework equips GUI agents with self-reflection and correction capabilities, paving the way for more robust, adaptable, and intelligent GUI automation, with all data, models, environments, and tools to be released publicly.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08012</guid>
      <pubDate>Mon, 09 Jun 2025 17:59:57 +0000</pubDate>
    </item>
    <item>
      <title>Play to Generalize: Learning to Reason Through Game Play</title>
      <link>https://arxiv.org/abs/2506.08011</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08011.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yunfei Xie, Yinsong Ma, Shiyi Lan, Alan Yuille, Junfei Xiao, Chen Wei&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 13&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Developing generalizable reasoning capabilities in multimodal large language models (MLLMs) remains challenging. Motivated by cognitive science literature suggesting that gameplay promotes transferable cognitive skills, we propose a novel post-training paradigm, Visual Game Learning, or ViGaL, where MLLMs develop out-of-domain generalization of multimodal reasoning through playing arcade-like games. Specifically, we show that post-training a 7B-parameter MLLM via reinforcement learning (RL) on simple arcade-like games, e.g. Snake, significantly enhances its downstream performance on multimodal math benchmarks like MathVista, and on multi-discipline questions like MMMU, without seeing any worked solutions, equations, or diagrams during RL, suggesting the capture of transferable reasoning skills. Remarkably, our model outperforms specialist models tuned on multimodal reasoning data in multimodal reasoning benchmarks, while preserving the base model's performance on general visual benchmarks, a challenge where specialist models often fall short. Our findings suggest a new post-training paradigm: synthetic, rule-based games can serve as controllable and scalable pre-text tasks that unlock generalizable multimodal reasoning abilities in MLLMs.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08011</guid>
      <pubDate>Mon, 09 Jun 2025 17:59:57 +0000</pubDate>
    </item>
    <item>
      <title>Vision Transformers Don't Need Trained Registers</title>
      <link>https://arxiv.org/abs/2506.08010</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08010.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Nick Jiang, Amil Dravid, Alexei Efros, Yossi Gandelsman&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 16&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We investigate the mechanism underlying a previously identified phenomenon in Vision Transformers -- the emergence of high-norm tokens that lead to noisy attention maps. We observe that in multiple models (e.g., CLIP, DINOv2), a sparse set of neurons is responsible for concentrating high-norm activations on outlier tokens, leading to irregular attention patterns and degrading downstream visual processing. While the existing solution for removing these outliers involves retraining models from scratch with additional learned register tokens, we use our findings to create a training-free approach to mitigate these artifacts. By shifting the high-norm activations from our discovered register neurons into an additional untrained token, we can mimic the effect of register tokens on a model already trained without registers. We demonstrate that our method produces cleaner attention and feature maps, enhances performance over base models across multiple downstream visual tasks, and achieves results comparable to models explicitly trained with register tokens. We then extend test-time registers to off-the-shelf vision-language models to improve their interpretability. Our results suggest that test-time registers effectively take on the role of register tokens at test-time, offering a training-free solution for any pre-trained model released without them.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08010</guid>
      <pubDate>Mon, 09 Jun 2025 17:59:57 +0000</pubDate>
    </item>
    <item>
      <title>Compound AI Systems Optimization: A Survey of Methods, Challenges, and   Future Directions</title>
      <link>https://arxiv.org/abs/2506.08234</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08234.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yu-Ang Lee, Guan-Ting Yi, Mei-Yi Liu, Jui-Chao Lu, Guan-Bo Yang, Yun-Nung Chen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 8&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advancements in large language models (LLMs) and AI systems have led to a paradigm shift in the design and optimization of complex AI workflows. By integrating multiple components, compound AI systems have become increasingly adept at performing sophisticated tasks. However, as these systems grow in complexity, new challenges arise in optimizing not only individual components but also their interactions. While traditional optimization methods such as supervised fine-tuning (SFT) and reinforcement learning (RL) remain foundational, the rise of natural language feedback introduces promising new approaches, especially for optimizing non-differentiable systems. This paper provides a systematic review of recent progress in optimizing compound AI systems, encompassing both numerical and language-based techniques. We formalize the notion of compound AI system optimization, classify existing methods along several key dimensions, and highlight open research challenges and future directions in this rapidly evolving field. A list of surveyed papers is publicly available at https://github.com/MiuLab/AISysOpt-Survey.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08234</guid>
      <pubDate>Mon, 09 Jun 2025 21:04:14 +0000</pubDate>
    </item>
    <item>
      <title>Seeing Voices: Generating A-Roll Video from Audio with Mirage</title>
      <link>https://arxiv.org/abs/2506.08279</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08279.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Aditi Sundararaman, Amogh Adishesha, Andrew Jaegle, Dan Bigioi, Hyoung-Kyu Song, Jon Kyl, Justin Mao, Kevin Lan, Mojtaba Komeili, ShahRukh Athar, Sheila Babayan, Stanislau Beliasau, William Buchwalter&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 23&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; From professional filmmaking to user-generated content, creators and consumers have long recognized that the power of video depends on the harmonious integration of what we hear (the video's audio track) with what we see (the video's image sequence). Current approaches to video generation either ignore sound to focus on general-purpose but silent image sequence generation or address both visual and audio elements but focus on restricted application domains such as re-dubbing. We introduce Mirage, an audio-to-video foundation model that excels at generating realistic, expressive output imagery from scratch given an audio input. When integrated with existing methods for speech synthesis (text-to-speech, or TTS), Mirage results in compelling multimodal video. When trained on audio-video footage of people talking (A-roll) and conditioned on audio containing speech, Mirage generates video of people delivering a believable interpretation of the performance implicit in input audio. Our central technical contribution is a unified method for training self-attention-based audio-to-video generation models, either from scratch or given existing weights. This methodology allows Mirage to retain generality as an approach to audio-to-video generation while producing outputs of superior subjective quality to methods that incorporate audio-specific architectures or loss components specific to people, speech, or details of how images or audio are captured. We encourage readers to watch and listen to the results of Mirage for themselves (see paper and comments for links).&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08279</guid>
      <pubDate>Mon, 09 Jun 2025 22:56:02 +0000</pubDate>
    </item>
    <item>
      <title>Institutional Books 1.0: A 242B token dataset from Harvard Library's   collections, refined for accuracy and usability</title>
      <link>https://arxiv.org/abs/2506.08300</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08300.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Matteo Cargnelutti, Catherine Brobston, John Hess, Jack Cushman, Kristi Mukk, Aristana Scourtas, Kyle Courtney, Greg Leppert, Amanda Watson, Martha Whitehead, Jonathan Zittrain&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 6&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large language models (LLMs) use data to learn about the world in order to produce meaningful correlations and predictions. As such, the nature, scale, quality, and diversity of the datasets used to train these models, or to support their work at inference time, have a direct impact on their quality. The rapid development and adoption of LLMs of varying quality has brought into focus the scarcity of publicly available, high-quality training data and revealed an urgent need to ground the stewardship of these datasets in sustainable practices with clear provenance chains. To that end, this technical report introduces Institutional Books 1.0, a large collection of public domain books originally digitized through Harvard Library's participation in the Google Books project, beginning in 2006. Working with Harvard Library, we extracted, analyzed, and processed these volumes into an extensively-documented dataset of historic texts. This analysis covers the entirety of Harvard Library's collection scanned as part of that project, originally spanning 1,075,899 volumes written in over 250 different languages for a total of approximately 250 billion tokens. As part of this initial release, the OCR-extracted text (original and post-processed) as well as the metadata (bibliographic, source, and generated) of the 983,004 volumes, or 242B tokens, identified as being in the public domain have been made available. This report describes this project's goals and methods as well as the results of the analyses we performed, all in service of making this historical collection more accessible and easier for humans and machines alike to filter, read and use.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08300</guid>
      <pubDate>Tue, 10 Jun 2025 00:11:30 +0000</pubDate>
    </item>
    <item>
      <title>DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in   Search-Augmented LLMs</title>
      <link>https://arxiv.org/abs/2506.08500</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08500.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Arie Cattan, Alon Jacovi, Ori Ram, Jonathan Herzig, Roee Aharoni, Sasha Goldshtein, Eran Ofek, Idan Szpektor, Avi Caciularu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 7&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Retrieval Augmented Generation (RAG) is a commonly used approach for enhancing large language models (LLMs) with relevant and up-to-date information. However, the retrieved sources can often contain conflicting information and it remains unclear how models should address such discrepancies. In this work, we first propose a novel taxonomy of knowledge conflict types in RAG, along with the desired model behavior for each type. We then introduce CONFLICTS, a high-quality benchmark with expert annotations of conflict types in a realistic RAG setting. CONFLICTS is the first benchmark that enables tracking progress on how models address a wide range of knowledge conflicts. We conduct extensive experiments on this benchmark, showing that LLMs often struggle to appropriately resolve conflicts between sources. While prompting LLMs to explicitly reason about the potential conflict in the retrieved documents significantly improves the quality and appropriateness of their responses, substantial room for improvement in future research remains.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08500</guid>
      <pubDate>Tue, 10 Jun 2025 06:52:57 +0000</pubDate>
    </item>
    <item>
      <title>Auto-Regressive vs Flow-Matching: a Comparative Study of Modeling   Paradigms for Text-to-Music Generation</title>
      <link>https://arxiv.org/abs/2506.08570</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08570.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Or Tal, Felix Kreuk, Yossi Adi&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 28&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent progress in text-to-music generation has enabled models to synthesize high-quality musical segments, full compositions, and even respond to fine-grained control signals, e.g. chord progressions. State-of-the-art (SOTA) systems differ significantly across many dimensions, such as training datasets, modeling paradigms, and architectural choices. This diversity complicates efforts to evaluate models fairly and pinpoint which design choices most influence performance. While factors like data and architecture are important, in this study we focus exclusively on the modeling paradigm. We conduct a systematic empirical analysis to isolate its effects, offering insights into associated trade-offs and emergent behaviors that can guide future text-to-music generation systems. Specifically, we compare the two arguably most common modeling paradigms: Auto-Regressive decoding and Conditional Flow-Matching. We conduct a controlled comparison by training all models from scratch using identical datasets, training configurations, and similar backbone architectures. Performance is evaluated across multiple axes, including generation quality, robustness to inference configurations, scalability, adherence to both textual and temporally aligned conditioning, and editing capabilities in the form of audio inpainting. This comparative study sheds light on distinct strengths and limitations of each paradigm, providing actionable insights that can inform future architectural and training decisions in the evolving landscape of text-to-music generation. Audio sampled examples are available at: https://huggingface.co/spaces/ortal1602/ARvsFM&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08570</guid>
      <pubDate>Tue, 10 Jun 2025 08:37:45 +0000</pubDate>
    </item>
    <item>
      <title>RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic   Sampling</title>
      <link>https://arxiv.org/abs/2506.08672</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08672.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yang Liu, Jiaqi Li, Zilong Zheng&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 31&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Rule-based reasoning has been acknowledged as one of the fundamental problems in reasoning, while deviations in rule formats, types, and complexity in real-world applications pose severe challenges. Recent studies have shown that large reasoning models (LRMs) have remarkable reasoning capabilities, and their performance is substantially enhanced by reinforcement learning (RL). However, it remains an open question whether small reasoning models (SRMs) can learn rule-based reasoning effectively with robust generalization across diverse tasks and domains. To address this, we introduce Reinforced Rule-based Reasoning, a.k.a. RuleReasoner, a simple yet effective method to conduct rule-based reasoning via a wide collection of curated tasks and a novel domain-aware dynamic sampling approach. Specifically, RuleReasoner resamples each training batch by updating the sampling weights of different domains based on historical rewards. This facilitates domain augmentation and flexible online learning schedules for RL, obviating the need for pre-hoc human-engineered mix-training recipes used in existing methods. Empirical evaluations on in-distribution (ID) and out-of-distribution (OOD) benchmarks reveal that RuleReasoner outperforms frontier LRMs by a significant margin (Delta4.1% average points on eight ID tasks and Delta10.4% average points on three OOD tasks over OpenAI-o1). Notably, our approach also exhibits higher computational efficiency compared to prior dynamic sampling methods for RL.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08672</guid>
      <pubDate>Tue, 10 Jun 2025 10:31:21 +0000</pubDate>
    </item>
    <item>
      <title>SeerAttention-R: Sparse Attention Adaptation for Long Reasoning</title>
      <link>https://arxiv.org/abs/2506.08889</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08889.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yizhao Gao, Shuming Guo, Shijie Cao, Yuqing Xia, Yu Cheng, Lei Wang, Lingxiao Ma, Yutao Sun, Tianzhu Ye, Li Dong, Hayden Kwok-Hay So, Yu Hua, Ting Cao, Fan Yang, Mao Yang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 24&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce SeerAttention-R, a sparse attention framework specifically tailored for the long decoding of reasoning models. Extended from SeerAttention, SeerAttention-R retains the design of learning attention sparsity through a self-distilled gating mechanism, while removing query pooling to accommodate auto-regressive decoding. With a lightweight plug-in gating, SeerAttention-R is flexible and can be easily integrated into existing pretrained model without modifying the original parameters. We demonstrate that SeerAttention-R, trained on just 0.4B tokens, maintains near-lossless reasoning accuracy with 4K token budget in AIME benchmark under large sparse attention block sizes (64/128). Using TileLang, we develop a highly optimized sparse decoding kernel that achieves near-theoretical speedups of up to 9x over FlashAttention-3 on H100 GPU at 90% sparsity. Code is available at: https://github.com/microsoft/SeerAttention.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08889</guid>
      <pubDate>Tue, 10 Jun 2025 15:17:26 +0000</pubDate>
    </item>
    <item>
      <title>SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner</title>
      <link>https://arxiv.org/abs/2506.09003</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09003.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Lei Zhang, Jiaxi Yang, Min Yang, Jian Yang, Mouxiang Chen, Jiajun Zhang, Zeyu Cui, Binyuan Hui, Junyang Lin&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 16&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce **SWE-Flow**, a novel data synthesis framework grounded in Test-Driven Development (TDD). Unlike existing software engineering data that rely on human-submitted issues, **SWE-Flow** automatically infers incremental development steps directly from unit tests, which inherently encapsulate high-level requirements. The core of **SWE-Flow** is the construction of a Runtime Dependency Graph (RDG), which precisely captures function interactions, enabling the generation of a structured, step-by-step *development schedule*. At each step, **SWE-Flow** produces a partial codebase, the corresponding unit tests, and the necessary code modifications, resulting in fully verifiable TDD tasks. With this approach, we generated 16,061 training instances and 2,020 test instances from real-world GitHub projects, creating the **SWE-Flow-Eval** benchmark. Our experiments show that fine-tuning open model on this dataset significantly improves performance in TDD-based coding. To facilitate further research, we release all code, datasets, models, and Docker images at [Github](https://github.com/Hambaobao/SWE-Flow).&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09003</guid>
      <pubDate>Tue, 10 Jun 2025 17:23:33 +0000</pubDate>
    </item>
    <item>
      <title>Seedance 1.0: Exploring the Boundaries of Video Generation Models</title>
      <link>https://arxiv.org/abs/2506.09113</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09113.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yu Gao, Haoyuan Guo, Tuyen Hoang, Weilin Huang, Lu Jiang, Fangyuan Kong, Huixia Li, Jiashi Li, Liang Li, Xiaojie Li, Xunsong Li, Yifu Li, Shanchuan Lin, Zhijie Lin, Jiawei Liu, Shu Liu, Xiaonan Nie, Zhiwu Qing, Yuxi Ren, Li Sun, Zhi Tian, Rui Wang, Sen Wang, Guoqiang Wei, Guohong Wu, Jie Wu, Ruiqi Xia, Fei Xiao, Xuefeng Xiao, Jiangqiao Yan, Ceyuan Yang, Jianchao Yang, Runkai Yang, Tao Yang, Yihang Yang, Zilyu Ye, Xuejiao Zeng, Yan Zeng, Heng Zhang, Yang Zhao, Xiaozheng Zheng, Peihao Zhu, Jiaxin Zou, Feilong Zuo&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 65&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Notable breakthroughs in diffusion modeling have propelled rapid improvements in video generation, yet current foundational model still face critical challenges in simultaneously balancing prompt following, motion plausibility, and visual quality. In this report, we introduce Seedance 1.0, a high-performance and inference-efficient video foundation generation model that integrates several core technical improvements: (i) multi-source data curation augmented with precision and meaningful video captioning, enabling comprehensive learning across diverse scenarios; (ii) an efficient architecture design with proposed training paradigm, which allows for natively supporting multi-shot generation and jointly learning of both text-to-video and image-to-video tasks. (iii) carefully-optimized post-training approaches leveraging fine-grained supervised fine-tuning, and video-specific RLHF with multi-dimensional reward mechanisms for comprehensive performance improvements; (iv) excellent model acceleration achieving ~10x inference speedup through multi-stage distillation strategies and system-level optimizations. Seedance 1.0 can generate a 5-second video at 1080p resolution only with 41.4 seconds (NVIDIA-L20). Compared to state-of-the-art video generation models, Seedance 1.0 stands out with high-quality and fast video generation having superior spatiotemporal fluidity with structural stability, precise instruction adherence in complex multi-subject contexts, native multi-shot narrative coherence with consistent subject representation.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09113</guid>
      <pubDate>Tue, 10 Jun 2025 17:56:11 +0000</pubDate>
    </item>
    <item>
      <title>Autoregressive Semantic Visual Reconstruction Helps VLMs Understand   Better</title>
      <link>https://arxiv.org/abs/2506.09040</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09040.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Dianyi Wang, Wei Song, Yikun Wang, Siyuan Wang, Kaicheng Yu, Zhongyu Wei, Jiaqi Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 34&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Typical large vision-language models (LVLMs) apply autoregressive supervision solely to textual sequences, without fully incorporating the visual modality into the learning process. This results in three key limitations: (1) an inability to utilize images without accompanying captions, (2) the risk that captions omit critical visual details, and (3) the challenge that certain vision-centric content cannot be adequately conveyed through text. As a result, current LVLMs often prioritize vision-to-language alignment while potentially overlooking fine-grained visual information. While some prior works have explored autoregressive image generation, effectively leveraging autoregressive visual supervision to enhance image understanding remains an open challenge. In this paper, we introduce Autoregressive Semantic Visual Reconstruction (ASVR), which enables joint learning of visual and textual modalities within a unified autoregressive framework. We show that autoregressively reconstructing the raw visual appearance of images does not enhance and may even impair multimodal understanding. In contrast, autoregressively reconstructing the semantic representation of images consistently improves comprehension. Notably, we find that even when models are given continuous image features as input, they can effectively reconstruct discrete semantic tokens, resulting in stable and consistent improvements across a wide range of multimodal understanding benchmarks. Our approach delivers significant performance gains across varying data scales (556k-2M) and types of LLM bacbones. Specifically, ASVR improves LLaVA-1.5 by 5% in average scores across 14 multimodal benchmarks. The code is available at https://github.com/AlenjandroWang/ASVR.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09040</guid>
      <pubDate>Tue, 10 Jun 2025 17:57:50 +0000</pubDate>
    </item>
    <item>
      <title>Comment on The Illusion of Thinking: Understanding the Strengths and   Limitations of Reasoning Models via the Lens of Problem Complexity</title>
      <link>https://arxiv.org/abs/2506.09250</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09250.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; C. Opus, A. Lawsen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 21&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Shojaee et al. (2025) report that Large Reasoning Models (LRMs) exhibit "accuracy collapse" on planning puzzles beyond certain complexity thresholds. We demonstrate that their findings primarily reflect experimental design limitations rather than fundamental reasoning failures. Our analysis reveals three critical issues: (1) Tower of Hanoi experiments systematically exceed model output token limits at reported failure points, with models explicitly acknowledging these constraints in their outputs; (2) The authors' automated evaluation framework fails to distinguish between reasoning failures and practical constraints, leading to misclassification of model capabilities; (3) Most concerningly, their River Crossing benchmarks include mathematically impossible instances for N &gt; 5 due to insufficient boat capacity, yet models are scored as failures for not solving these unsolvable problems. When we control for these experimental artifacts, by requesting generating functions instead of exhaustive move lists, preliminary experiments across multiple models indicate high accuracy on Tower of Hanoi instances previously reported as complete failures. These findings highlight the importance of careful experimental design when evaluating AI reasoning capabilities.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09250</guid>
      <pubDate>Tue, 10 Jun 2025 21:16:53 +0000</pubDate>
    </item>
    <item>
      <title>Token Perturbation Guidance for Diffusion Models</title>
      <link>https://arxiv.org/abs/2506.10036</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10036.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Javad Rajabi, Soroush Mehraban, Seyedmorteza Sadat, Babak Taati&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 5&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Classifier-free guidance (CFG) has become an essential component of modern diffusion models to enhance both generation quality and alignment with input conditions. However, CFG requires specific training procedures and is limited to conditional generation. To address these limitations, we propose Token Perturbation Guidance (TPG), a novel method that applies perturbation matrices directly to intermediate token representations within the diffusion network. TPG employs a norm-preserving shuffling operation to provide effective and stable guidance signals that improve generation quality without architectural changes. As a result, TPG is training-free and agnostic to input conditions, making it readily applicable to both conditional and unconditional generation. We further analyze the guidance term provided by TPG and show that its effect on sampling more closely resembles CFG compared to existing training-free guidance techniques. Extensive experiments on SDXL and Stable Diffusion 2.1 show that TPG achieves nearly a 2times improvement in FID for unconditional generation over the SDXL baseline, while closely matching CFG in prompt alignment. These results establish TPG as a general, condition-agnostic guidance method that brings CFG-like benefits to a broader class of diffusion models. The code is available at https://github.com/TaatiTeam/Token-Perturbation-Guidance&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10036</guid>
      <pubDate>Tue, 10 Jun 2025 21:25:46 +0000</pubDate>
    </item>
    <item>
      <title>UFM: A Simple Path towards Unified Dense Correspondence with Flow</title>
      <link>https://arxiv.org/abs/2506.09278</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09278.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuchen Zhang, Nikhil Keetha, Chenwei Lyu, Bhuvan Jhamb, Yutian Chen, Yuheng Qiu, Jay Karhade, Shreyas Jha, Yaoyu Hu, Deva Ramanan, Sebastian Scherer, Wenshan Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 6&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Dense image correspondence is central to many applications, such as visual odometry, 3D reconstruction, object association, and re-identification. Historically, dense correspondence has been tackled separately for wide-baseline scenarios and optical flow estimation, despite the common goal of matching content between two images. In this paper, we develop a Unified Flow &amp; Matching model (UFM), which is trained on unified data for pixels that are co-visible in both source and target images. UFM uses a simple, generic transformer architecture that directly regresses the (u,v) flow. It is easier to train and more accurate for large flows compared to the typical coarse-to-fine cost volumes in prior work. UFM is 28% more accurate than state-of-the-art flow methods (Unimatch), while also having 62% less error and 6.7x faster than dense wide-baseline matchers (RoMa). UFM is the first to demonstrate that unified training can outperform specialized approaches across both domains. This result enables fast, general-purpose correspondence and opens new directions for multi-modal, long-range, and real-time correspondence tasks.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09278</guid>
      <pubDate>Tue, 10 Jun 2025 22:32:13 +0000</pubDate>
    </item>
    <item>
      <title>Ming-Omni: A Unified Multimodal Model for Perception and Generation</title>
      <link>https://arxiv.org/abs/2506.09344</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09344.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Inclusion AI, Biao Gong, Cheng Zou, Chuanyang Zheng, Chunluan Zhou, Canxiang Yan, Chunxiang Jin, Chunjie Shen, Dandan Zheng, Fudong Wang, Furong Xu, GuangMing Yao, Jun Zhou, Jingdong Chen, Jianxin Sun, Jiajia Liu, Jianjiang Zhu, Jun Peng, Kaixiang Ji, Kaiyou Song, Kaimeng Ren, Libin Wang, Lixiang Ru, Lele Xie, Longhua Tan, Lyuxin Xue, Lan Wang, Mochen Bai, Ning Gao, Pei Chen, Qingpei Guo, Qinglong Zhang, Qiang Xu, Rui Liu, Ruijie Xiong, Sirui Gao, Tinghao Liu, Taisong Li, Weilong Chai, Xinyu Xiao, Xiaomei Wang, Xiaoxue Chen, Xiao Lu, Xiaoyu Li, Xingning Dong, Xuzheng Yu, Yi Yuan, Yuting Gao, Yunxiao Sun, Yipeng Chen, Yifei Wu, Yongjie Lyu, Ziping Ma, Zipeng Feng, Zhijiang Fang, Zhihao Qiu, Ziyuan Huang, Zhengyu He&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 22&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We propose Ming-Omni, a unified multimodal model capable of processing images, text, audio, and video, while demonstrating strong proficiency in both speech and image generation. Ming-Omni employs dedicated encoders to extract tokens from different modalities, which are then processed by Ling, an MoE architecture equipped with newly proposed modality-specific routers. This design enables a single model to efficiently process and fuse multimodal inputs within a unified framework, thereby facilitating diverse tasks without requiring separate models, task-specific fine-tuning, or structural redesign. Importantly, Ming-Omni extends beyond conventional multimodal models by supporting audio and image generation. This is achieved through the integration of an advanced audio decoder for natural-sounding speech and Ming-Lite-Uni for high-quality image generation, which also allow the model to engage in context-aware chatting, perform text-to-speech conversion, and conduct versatile image editing. Our experimental results showcase Ming-Omni offers a powerful solution for unified perception and generation across all modalities. Notably, our proposed Ming-Omni is the first open-source model we are aware of to match GPT-4o in modality support, and we release all code and model weights to encourage further research and development in the community.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09344</guid>
      <pubDate>Wed, 11 Jun 2025 02:50:49 +0000</pubDate>
    </item>
    <item>
      <title>Autoregressive Adversarial Post-Training for Real-Time Interactive Video   Generation</title>
      <link>https://arxiv.org/abs/2506.09350</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09350.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shanchuan Lin, Ceyuan Yang, Hao He, Jianwen Jiang, Yuxi Ren, Xin Xia, Yang Zhao, Xuefeng Xiao, Lu Jiang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 46&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Existing large-scale video generation models are computationally intensive, preventing adoption in real-time and interactive applications. In this work, we propose autoregressive adversarial post-training (AAPT) to transform a pre-trained latent video diffusion model into a real-time, interactive video generator. Our model autoregressively generates a latent frame at a time using a single neural function evaluation (1NFE). The model can stream the result to the user in real time and receive interactive responses as controls to generate the next latent frame. Unlike existing approaches, our method explores adversarial training as an effective paradigm for autoregressive generation. This not only allows us to design an architecture that is more efficient for one-step generation while fully utilizing the KV cache, but also enables training the model in a student-forcing manner that proves to be effective in reducing error accumulation during long video generation. Our experiments demonstrate that our 8B model achieves real-time, 24fps, streaming video generation at 736x416 resolution on a single H100, or 1280x720 on 8xH100 up to a minute long (1440 frames). Visit our research website at https://seaweed-apt.com/2&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09350</guid>
      <pubDate>Wed, 11 Jun 2025 03:04:23 +0000</pubDate>
    </item>
    <item>
      <title>Give Me FP32 or Give Me Death? Challenges and Solutions for Reproducible   Reasoning</title>
      <link>https://arxiv.org/abs/2506.09501</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09501.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jiayi Yuan, Hao Li, Xinheng Ding, Wenya Xie, Yu-Jhe Li, Wentian Zhao, Kun Wan, Jing Shi, Xia Hu, Zirui Liu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 16&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large Language Models (LLMs) are now integral across various domains and have demonstrated impressive performance. Progress, however, rests on the premise that benchmark scores are both accurate and reproducible. We demonstrate that the reproducibility of LLM performance is fragile: changing system configuration such as evaluation batch size, GPU count, and GPU version can introduce significant difference in the generated responses. This issue is especially pronounced in reasoning models, where minor rounding differences in early tokens can cascade into divergent chains of thought, ultimately affecting accuracy. For instance, under bfloat16 precision with greedy decoding, a reasoning model like DeepSeek-R1-Distill-Qwen-7B can exhibit up to 9% variation in accuracy and 9,000 tokens difference in response length due to differences in GPU count, type, and evaluation batch size. We trace the root cause of this variability to the non-associative nature of floating-point arithmetic under limited numerical precision. This work presents the first systematic investigation into how numerical precision affects reproducibility in LLM inference. Through carefully controlled experiments across various hardware, software, and precision settings, we quantify when and how model outputs diverge. Our analysis reveals that floating-point precision -- while critical for reproducibility -- is often neglected in evaluation practices. Inspired by this, we develop a lightweight inference pipeline, dubbed LayerCast, that stores weights in 16-bit precision but performs all computations in FP32, balancing memory efficiency with numerical stability. Code is available at https://github.com/nanomaoli/llm_reproducibility.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09501</guid>
      <pubDate>Wed, 11 Jun 2025 08:23:53 +0000</pubDate>
    </item>
    <item>
      <title>ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical   Reasoning</title>
      <link>https://arxiv.org/abs/2506.09513</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09513.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yu Sun, Xingyu Qian, Weiwen Xu, Hao Zhang, Chenghao Xiao, Long Li, Yu Rong, Wenbing Huang, Qifeng Bai, Tingyang Xu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 81&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Though reasoning-based large language models (LLMs) have excelled in mathematics and programming, their capabilities in knowledge-intensive medical question answering remain underexplored. To address this, we introduce ReasonMed, the largest medical reasoning dataset, comprising 370k high-quality examples distilled from 1.7 million initial reasoning paths generated by various LLMs. ReasonMed is constructed through a multi-agent verification and refinement process, where we design an Error Refiner to enhance the reasoning paths by identifying and correcting error-prone steps flagged by a verifier. Leveraging ReasonMed, we systematically investigate best practices for training medical reasoning models and find that combining detailed Chain-of-Thought (CoT) reasoning with concise answer summaries yields the most effective fine-tuning strategy. Based on this strategy, we train ReasonMed-7B, which sets a new benchmark for sub-10B models, outperforming the prior best by 4.17\% and even exceeding LLaMA3.1-70B on PubMedQA by 4.60\%.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09513</guid>
      <pubDate>Wed, 11 Jun 2025 08:36:55 +0000</pubDate>
    </item>
    <item>
      <title>Vision Matters: Simple Visual Perturbations Can Boost Multimodal Math   Reasoning</title>
      <link>https://arxiv.org/abs/2506.09736</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09736.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuting Li, Lai Wei, Kaipeng Zheng, Jingyuan Huang, Linghe Kong, Lichao Sun, Weiran Huang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 10&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Despite the rapid progress of multimodal large language models (MLLMs), they have largely overlooked the importance of visual processing. In a simple yet revealing experiment, we interestingly find that language-only models, when provided with image captions, can achieve comparable or even better performance than MLLMs that consume raw visual inputs. This suggests that current MLLMs may generate accurate visual descriptions but fail to effectively integrate them during reasoning. Motivated by this, we propose a simple visual perturbation framework that enhances perceptual robustness without requiring algorithmic modifications or additional training data. Our approach introduces three targeted perturbations: distractor concatenation, dominance-preserving mixup, and random rotation, that can be easily integrated into existing post-training pipelines including SFT, DPO, and GRPO. Through extensive experiments across multiple datasets, we demonstrate consistent improvements in mathematical reasoning performance, with gains comparable to those achieved through algorithmic changes. Additionally, we achieve competitive performance among open-source 7B RL-tuned models by training Qwen2.5-VL-7B with visual perturbation. Through comprehensive ablation studies, we analyze the effectiveness of different perturbation strategies, revealing that each perturbation type contributes uniquely to different aspects of visual reasoning. Our findings highlight the critical role of visual perturbation in multimodal mathematical reasoning: better reasoning begins with better seeing. Our code is available at https://github.com/YutingLi0606/Vision-Matters.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09736</guid>
      <pubDate>Wed, 11 Jun 2025 13:39:46 +0000</pubDate>
    </item>
    <item>
      <title>ComfyUI-R1: Exploring Reasoning Models for Workflow Generation</title>
      <link>https://arxiv.org/abs/2506.09790</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09790.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zhenran Xu, Yiyu Wang, Xue Yang, Longyue Wang, Weihua Luo, Kaifu Zhang, Baotian Hu, Min Zhang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 46&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; AI-generated content has evolved from monolithic models to modular workflows, particularly on platforms like ComfyUI, enabling customization in creative pipelines. However, crafting effective workflows requires great expertise to orchestrate numerous specialized components, presenting a steep learning curve for users. To address this challenge, we introduce ComfyUI-R1, the first large reasoning model for automated workflow generation. Starting with our curated dataset of 4K workflows, we construct long chain-of-thought (CoT) reasoning data, including node selection, workflow planning, and code-level workflow representation. ComfyUI-R1 is trained through a two-stage framework: (1) CoT fine-tuning for cold start, adapting models to the ComfyUI domain; (2) reinforcement learning for incentivizing reasoning capability, guided by a fine-grained rule-metric hybrid reward, ensuring format validity, structural integrity, and node-level fidelity. Experiments show that our 7B-parameter model achieves a 97\% format validity rate, along with high pass rate, node-level and graph-level F1 scores, significantly surpassing prior state-of-the-art methods that employ leading closed-source models such as GPT-4o and Claude series. Further analysis highlights the critical role of the reasoning process and the advantage of transforming workflows into code. Qualitative comparison reveals our strength in synthesizing intricate workflows with diverse nodes, underscoring the potential of long CoT reasoning in AI art creation.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09790</guid>
      <pubDate>Wed, 11 Jun 2025 14:35:15 +0000</pubDate>
    </item>
    <item>
      <title>CoRT: Code-integrated Reasoning within Thinking</title>
      <link>https://arxiv.org/abs/2506.09820</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09820.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Chengpeng Li, Zhengyang Tang, Ziniu Li, Mingfeng Xue, Keqin Bao, Tian Ding, Ruoyu Sun, Benyou Wang, Xiang Wang, Junyang Lin, Dayiheng Liu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 17&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large Reasoning Models (LRMs) like o1 and DeepSeek-R1 have shown remarkable progress in natural language reasoning with long chain-of-thought (CoT), yet they remain inefficient or inaccurate when handling complex mathematical operations. Addressing these limitations through computational tools (e.g., computation libraries and symbolic solvers) is promising, but it introduces a technical challenge: Code Interpreter (CI) brings external knowledge beyond the model's internal text representations, thus the direct combination is not efficient. This paper introduces CoRT, a post-training framework for teaching LRMs to leverage CI effectively and efficiently. As a first step, we address the data scarcity issue by synthesizing code-integrated reasoning data through Hint-Engineering, which strategically inserts different hints at appropriate positions to optimize LRM-CI interaction. We manually create 30 high-quality samples, upon which we post-train models ranging from 1.5B to 32B parameters, with supervised fine-tuning, rejection fine-tuning and reinforcement learning. Our experimental results demonstrate that Hint-Engineering models achieve 4\% and 8\% absolute improvements on DeepSeek-R1-Distill-Qwen-32B and DeepSeek-R1-Distill-Qwen-1.5B respectively, across five challenging mathematical reasoning datasets. Furthermore, Hint-Engineering models use about 30\% fewer tokens for the 32B model and 50\% fewer tokens for the 1.5B model compared with the natural language models. The models and code are available at https://github.com/ChengpengLi1003/CoRT.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09820</guid>
      <pubDate>Wed, 11 Jun 2025 14:59:02 +0000</pubDate>
    </item>
    <item>
      <title>SAFE: Multitask Failure Detection for Vision-Language-Action Models</title>
      <link>https://arxiv.org/abs/2506.09937</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09937.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Qiao Gu, Yuanliang Ju, Shengxiang Sun, Igor Gilitschenski, Haruki Nishimura, Masha Itkina, Florian Shkurti&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 9&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; While vision-language-action models (VLAs) have shown promising robotic behaviors across a diverse set of manipulation tasks, they achieve limited success rates when deployed on novel tasks out-of-the-box. To allow these policies to safely interact with their environments, we need a failure detector that gives a timely alert such that the robot can stop, backtrack, or ask for help. However, existing failure detectors are trained and tested only on one or a few specific tasks, while VLAs require the detector to generalize and detect failures also in unseen tasks and novel environments. In this paper, we introduce the multitask failure detection problem and propose SAFE, a failure detector for generalist robot policies such as VLAs. We analyze the VLA feature space and find that VLAs have sufficient high-level knowledge about task success and failure, which is generic across different tasks. Based on this insight, we design SAFE to learn from VLA internal features and predict a single scalar indicating the likelihood of task failure. SAFE is trained on both successful and failed rollouts, and is evaluated on unseen tasks. SAFE is compatible with different policy architectures. We test it on OpenVLA, pi_0, and pi_0-FAST in both simulated and real-world environments extensively. We compare SAFE with diverse baselines and show that SAFE achieves state-of-the-art failure detection performance and the best trade-off between accuracy and detection time using conformal prediction. More qualitative results can be found at https://vla-safe.github.io/.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09937</guid>
      <pubDate>Wed, 11 Jun 2025 16:59:13 +0000</pubDate>
    </item>
    <item>
      <title>VerIF: Verification Engineering for Reinforcement Learning in   Instruction Following</title>
      <link>https://arxiv.org/abs/2506.09942</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09942.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Hao Peng, Yunjia Qi, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 6&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Reinforcement learning with verifiable rewards (RLVR) has become a key technique for enhancing large language models (LLMs), with verification engineering playing a central role. However, best practices for RL in instruction following remain underexplored. In this work, we explore the verification challenge in RL for instruction following and propose VerIF, a verification method that combines rule-based code verification with LLM-based verification from a large reasoning model (e.g., QwQ-32B). To support this approach, we construct a high-quality instruction-following dataset, VerInstruct, containing approximately 22,000 instances with associated verification signals. We apply RL training with VerIF to two models, achieving significant improvements across several representative instruction-following benchmarks. The trained models reach state-of-the-art performance among models of comparable size and generalize well to unseen constraints. We further observe that their general capabilities remain unaffected, suggesting that RL with VerIF can be integrated into existing RL recipes to enhance overall model performance. We have released our datasets, codes, and models to facilitate future research at https://github.com/THU-KEG/VerIF.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09942</guid>
      <pubDate>Wed, 11 Jun 2025 17:10:36 +0000</pubDate>
    </item>
    <item>
      <title>Resa: Transparent Reasoning Models via SAEs</title>
      <link>https://arxiv.org/abs/2506.09967</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09967.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shangshang Wang, Julian Asilis, Ömer Faruk Akgül, Enes Burak Bilgin, Ollie Liu, Deqing Fu, Willie Neiswanger&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 19&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; How cost-effectively can we elicit strong reasoning in language models by leveraging their underlying representations? We answer this question with Resa, a family of 1.5B reasoning models trained via a novel and efficient sparse autoencoder tuning (SAE-Tuning) procedure. This method first trains an SAE to capture reasoning abilities from a source model, and then uses the trained SAE to guide a standard supervised fine-tuning process to elicit such abilities in a target model, all using verified question-answer data without any reasoning traces. Notably, when applied to certain base models before further RL post-training, SAE-Tuning retains &gt;97% of its RL-trained counterpart's reasoning performance while reducing training costs by &gt;2000x to roughly \1 and training time by &gt;450x to around 20 minutes. Furthermore, when applied to lightly RL-trained models (e.g., within 1 hour on 2 GPUs), it enables reasoning performance such as 43.33% Pass@1 on AIME24 and 90% Pass@1 on AMC23 for only around 1 additional cost. Surprisingly, the reasoning abilities extracted via SAEs are potentially both generalizable and modular. Generality means abilities extracted from one dataset still elevate performance on a larger and overlapping corpus. Modularity means abilities extracted from Qwen or Qwen-Math can be attached to the R1-Distill model at test time, without any retraining, and yield comparable gains. Extensive ablations validate these findings and all artifacts are fully open-sourced.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09967</guid>
      <pubDate>Wed, 11 Jun 2025 17:44:01 +0000</pubDate>
    </item>
    <item>
      <title>Efficient Part-level 3D Object Generation via Dual Volume Packing</title>
      <link>https://arxiv.org/abs/2506.09980</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09980.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jiaxiang Tang, Ruijie Lu, Zhaoshuo Li, Zekun Hao, Xuan Li, Fangyin Wei, Shuran Song, Gang Zeng, Ming-Yu Liu, Tsung-Yi Lin&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 7&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent progress in 3D object generation has greatly improved both the quality and efficiency. However, most existing methods generate a single mesh with all parts fused together, which limits the ability to edit or manipulate individual parts. A key challenge is that different objects may have a varying number of parts. To address this, we propose a new end-to-end framework for part-level 3D object generation. Given a single input image, our method generates high-quality 3D objects with an arbitrary number of complete and semantically meaningful parts. We introduce a dual volume packing strategy that organizes all parts into two complementary volumes, allowing for the creation of complete and interleaved parts that assemble into the final object. Experiments show that our model achieves better quality, diversity, and generalization than previous image-based part-level generation methods.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09980</guid>
      <pubDate>Wed, 11 Jun 2025 17:55:03 +0000</pubDate>
    </item>
    <item>
      <title>InterActHuman: Multi-Concept Human Animation with Layout-Aligned Audio   Conditions</title>
      <link>https://arxiv.org/abs/2506.09984</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09984.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zhenzhi Wang, Jiaqi Yang, Jianwen Jiang, Chao Liang, Gaojie Lin, Zerong Zheng, Ceyuan Yang, Dahua Lin&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 13&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; End-to-end human animation with rich multi-modal conditions, e.g., text, image and audio has achieved remarkable advancements in recent years. However, most existing methods could only animate a single subject and inject conditions in a global manner, ignoring scenarios that multiple concepts could appears in the same video with rich human-human interactions and human-object interactions. Such global assumption prevents precise and per-identity control of multiple concepts including humans and objects, therefore hinders applications. In this work, we discard the single-entity assumption and introduce a novel framework that enforces strong, region-specific binding of conditions from modalities to each identity's spatiotemporal footprint. Given reference images of multiple concepts, our method could automatically infer layout information by leveraging a mask predictor to match appearance cues between the denoised video and each reference appearance. Furthermore, we inject local audio condition into its corresponding region to ensure layout-aligned modality matching in a iterative manner. This design enables the high-quality generation of controllable multi-concept human-centric videos. Empirical results and ablation studies validate the effectiveness of our explicit layout control for multi-modal conditions compared to implicit counterparts and other existing methods.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09984</guid>
      <pubDate>Wed, 11 Jun 2025 17:57:09 +0000</pubDate>
    </item>
    <item>
      <title>Multiverse: Your Language Models Secretly Decide How to Parallelize and   Merge Generation</title>
      <link>https://arxiv.org/abs/2506.09991</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09991.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Xinyu Yang, Yuwei An, Hongyi Liu, Tianqi Chen, Beidi Chen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 45&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Autoregressive Large Language Models (AR-LLMs) frequently exhibit implicit parallelism in sequential generation. Inspired by this, we introduce Multiverse, a new generative model that enables natively parallel generation. Multiverse internalizes a MapReduce paradigm, generating automatically through three stages: (i) a Map stage for adaptive task decomposition, (ii) a Process stage for parallel subtask execution, and (iii) a Reduce stage for lossless result synthesis. Next, we build a real-world Multiverse reasoning model with co-design of data, algorithm, and system, enabling rapid and seamless transfer from frontier AR-LLMs. Starting from sequential reasoning chains, we create Multiverse 1K by converting them into structured training data using an automated LLM-assisted pipeline, avoiding costly human annotations. Algorithmically, we design Multiverse Attention to separate parallel reasoning steps while keeping compatibility with causal attention for efficient training. Systematically, we implement Multiverse Engine to enable parallel inference. It features a dedicated scheduler that dynamically switches between sequential and parallel generation, triggered directly by the model. After a 3-hour fine-tuning with 1K examples, our Multiverse-32B stands as the only open-sourced non-AR model achieving performance on par with leading AR-LLMs of the same scale, evidenced by AIME24 &amp; 25 scores of 54% and 46%, respectively. Moreover, our budget control experiments show that Multiverse-32B exhibits superior scaling, outperforming AR-LLMs by 1.87% on average using the same context length. Such scaling further leads to practical efficiency gain, achieving up to 2x speedup across varying batch sizes. We have open-sourced the entire Multiverse ecosystem, including data, model weights, engine, supporting tools, as well as complete data curation prompts and detailed training and evaluation recipes.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09991</guid>
      <pubDate>Wed, 11 Jun 2025 17:59:23 +0000</pubDate>
    </item>
    <item>
      <title>Text-Aware Image Restoration with Diffusion Models</title>
      <link>https://arxiv.org/abs/2506.09993</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09993.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jaewon Min, Jin Hyeon Kim, Paul Hyunbin Cho, Jaeeun Lee, Jihye Park, Minkyu Park, Sangpil Kim, Hyunhee Park, Seungryong Kim&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Image restoration aims to recover degraded images. However, existing diffusion-based restoration methods, despite great success in natural image restoration, often struggle to faithfully reconstruct textual regions in degraded images. Those methods frequently generate plausible but incorrect text-like patterns, a phenomenon we refer to as text-image hallucination. In this paper, we introduce Text-Aware Image Restoration (TAIR), a novel restoration task that requires the simultaneous recovery of visual contents and textual fidelity. To tackle this task, we present SA-Text, a large-scale benchmark of 100K high-quality scene images densely annotated with diverse and complex text instances. Furthermore, we propose a multi-task diffusion framework, called TeReDiff, that integrates internal features from diffusion models into a text-spotting module, enabling both components to benefit from joint training. This allows for the extraction of rich text representations, which are utilized as prompts in subsequent denoising steps. Extensive experiments demonstrate that our approach consistently outperforms state-of-the-art restoration methods, achieving significant gains in text recognition accuracy. See our project page: https://cvlab-kaist.github.io/TAIR/&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09993</guid>
      <pubDate>Wed, 11 Jun 2025 17:59:46 +0000</pubDate>
    </item>
    <item>
      <title>PlayerOne: Egocentric World Simulator</title>
      <link>https://arxiv.org/abs/2506.09995</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09995.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuanpeng Tu, Hao Luo, Xi Chen, Xiang Bai, Fan Wang, Hengshuang Zhao&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 30&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce PlayerOne, the first egocentric realistic world simulator, facilitating immersive and unrestricted exploration within vividly dynamic environments. Given an egocentric scene image from the user, PlayerOne can accurately construct the corresponding world and generate egocentric videos that are strictly aligned with the real scene human motion of the user captured by an exocentric camera. PlayerOne is trained in a coarse-to-fine pipeline that first performs pretraining on large-scale egocentric text-video pairs for coarse-level egocentric understanding, followed by finetuning on synchronous motion-video data extracted from egocentric-exocentric video datasets with our automatic construction pipeline. Besides, considering the varying importance of different components, we design a part-disentangled motion injection scheme, enabling precise control of part-level movements. In addition, we devise a joint reconstruction framework that progressively models both the 4D scene and video frames, ensuring scene consistency in the long-form video generation. Experimental results demonstrate its great generalization ability in precise control of varying human movements and worldconsistent modeling of diverse scenarios. It marks the first endeavor into egocentric real-world simulation and can pave the way for the community to delve into fresh frontiers of world modeling and its diverse applications.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09995</guid>
      <pubDate>Wed, 11 Jun 2025 17:59:53 +0000</pubDate>
    </item>
    <item>
      <title>Attention, Please! Revisiting Attentive Probing for Masked Image   Modeling</title>
      <link>https://arxiv.org/abs/2506.10178</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10178.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Bill Psomas, Dionysis Christopoulos, Eirini Baltzi, Ioannis Kakogeorgiou, Tilemachos Aravanis, Nikos Komodakis, Konstantinos Karantzalos, Yannis Avrithis, Giorgos Tolias&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 8&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; As fine-tuning (FT) becomes increasingly impractical at scale, probing is emerging as the preferred evaluation protocol for self-supervised learning (SSL). Yet, the standard linear probing (LP) fails to adequately reflect the potential of models trained with Masked Image Modeling (MIM), due to the distributed nature of patch tokens. This motivates the need for attentive probing, an alternative that uses attention to selectively aggregate patch-level features. Despite its growing adoption, attentive probing remains under-explored, with existing methods suffering from excessive parameterization and poor computational efficiency.   In this work, we revisit attentive probing through the lens of the accuracy-efficiency trade-off. We conduct a systematic study of existing methods, analyzing their mechanisms and benchmarking their performance. We introduce efficient probing (EP), a multi-query cross-attention mechanism that eliminates redundant projections, reduces the number of trainable parameters, and achieves up to a 10times speed-up over conventional multi-head attention. Despite its simplicity, EP outperforms LP and prior attentive probing approaches across seven benchmarks, generalizes well beyond MIM to diverse pre-training paradigms, produces interpretable attention maps, and achieves strong gains in low-shot and layer-wise settings. Code available at https://github.com/billpsomas/efficient-probing.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10178</guid>
      <pubDate>Wed, 11 Jun 2025 21:10:26 +0000</pubDate>
    </item>
    <item>
      <title>Discrete Audio Tokens: More Than a Survey!</title>
      <link>https://arxiv.org/abs/2506.10274</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10274.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Pooneh Mousavi, Gallil Maimon, Adel Moumen, Darius Petermann, Jiatong Shi, Haibin Wu, Haici Yang, Anastasia Kuznetsova, Artem Ploujnikov, Ricard Marxer, Bhuvana Ramabhadran, Benjamin Elizalde, Loren Lugosch, Jinyu Li, Cem Subakan, Phil Woodland, Minje Kim, Hung-yi Lee, Shinji Watanabe, Yossi Adi, Mirco Ravanelli&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 27&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Discrete audio tokens are compact representations that aim to preserve perceptual quality, phonetic content, and speaker characteristics while enabling efficient storage and inference, as well as competitive performance across diverse downstream tasks.They provide a practical alternative to continuous features, enabling the integration of speech and audio into modern large language models (LLMs). As interest in token-based audio processing grows, various tokenization methods have emerged, and several surveys have reviewed the latest progress in the field. However, existing studies often focus on specific domains or tasks and lack a unified comparison across various benchmarks. This paper presents a systematic review and benchmark of discrete audio tokenizers, covering three domains: speech, music, and general audio. We propose a taxonomy of tokenization approaches based on encoder-decoder, quantization techniques, training paradigm, streamability, and application domains. We evaluate tokenizers on multiple benchmarks for reconstruction, downstream performance, and acoustic language modeling, and analyze trade-offs through controlled ablation studies. Our findings highlight key limitations, practical considerations, and open challenges, providing insight and guidance for future research in this rapidly evolving area. For more information, including our main results and tokenizer database, please refer to our website: https://poonehmousavi.github.io/dates-website/.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10274</guid>
      <pubDate>Thu, 12 Jun 2025 01:35:43 +0000</pubDate>
    </item>
    <item>
      <title>Optimus-3: Towards Generalist Multimodal Minecraft Agents with Scalable   Task Experts</title>
      <link>https://arxiv.org/abs/2506.10357</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10357.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zaijing Li, Yuquan Xie, Rui Shao, Gongwei Chen, Weili Guan, Dongmei Jiang, Liqiang Nie&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 19&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recently, agents based on multimodal large language models (MLLMs) have achieved remarkable progress across various domains. However, building a generalist agent with capabilities such as perception, planning, action, grounding, and reflection in open-world environments like Minecraft remains challenges: insufficient domain-specific data, interference among heterogeneous tasks, and visual diversity in open-world settings. In this paper, we address these challenges through three key contributions. 1) We propose a knowledge-enhanced data generation pipeline to provide scalable and high-quality training data for agent development. 2) To mitigate interference among heterogeneous tasks, we introduce a Mixture-of-Experts (MoE) architecture with task-level routing. 3) We develop a Multimodal Reasoning-Augmented Reinforcement Learning approach to enhance the agent's reasoning ability for visual diversity in Minecraft. Built upon these innovations, we present Optimus-3, a general-purpose agent for Minecraft. Extensive experimental results demonstrate that Optimus-3 surpasses both generalist multimodal large language models and existing state-of-the-art agents across a wide range of tasks in the Minecraft environment. Project page: https://cybertronagent.github.io/Optimus-3.github.io/&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10357</guid>
      <pubDate>Thu, 12 Jun 2025 05:29:40 +0000</pubDate>
    </item>
    <item>
      <title>AniMaker: Automated Multi-Agent Animated Storytelling with MCTS-Driven   Clip Generation</title>
      <link>https://arxiv.org/abs/2506.10540</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10540.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Haoyuan Shi, Yunxin Li, Xinyu Chen, Longyue Wang, Baotian Hu, Min Zhang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Despite rapid advancements in video generation models, generating coherent storytelling videos that span multiple scenes and characters remains challenging. Current methods often rigidly convert pre-generated keyframes into fixed-length clips, resulting in disjointed narratives and pacing issues. Furthermore, the inherent instability of video generation models means that even a single low-quality clip can significantly degrade the entire output animation's logical coherence and visual continuity. To overcome these obstacles, we introduce AniMaker, a multi-agent framework enabling efficient multi-candidate clip generation and storytelling-aware clip selection, thus creating globally consistent and story-coherent animation solely from text input. The framework is structured around specialized agents, including the Director Agent for storyboard generation, the Photography Agent for video clip generation, the Reviewer Agent for evaluation, and the Post-Production Agent for editing and voiceover. Central to AniMaker's approach are two key technical components: MCTS-Gen in Photography Agent, an efficient Monte Carlo Tree Search (MCTS)-inspired strategy that intelligently navigates the candidate space to generate high-potential clips while optimizing resource usage; and AniEval in Reviewer Agent, the first framework specifically designed for multi-shot animation evaluation, which assesses critical aspects such as story-level consistency, action completion, and animation-specific features by considering each clip in the context of its preceding and succeeding clips. Experiments demonstrate that AniMaker achieves superior quality as measured by popular metrics including VBench and our proposed AniEval framework, while significantly improving the efficiency of multi-candidate generation, pushing AI-generated storytelling animation closer to production standards.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10540</guid>
      <pubDate>Thu, 12 Jun 2025 10:06:21 +0000</pubDate>
    </item>
    <item>
      <title>DreamActor-H1: High-Fidelity Human-Product Demonstration Video   Generation via Motion-designed Diffusion Transformers</title>
      <link>https://arxiv.org/abs/2506.10568</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10568.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Lizhen Wang, Zhurong Xia, Tianshu Hu, Pengrui Wang, Pengfei Wang, Zerong Zheng, Ming Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 6&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; In e-commerce and digital marketing, generating high-fidelity human-product demonstration videos is important for effective product presentation. However, most existing frameworks either fail to preserve the identities of both humans and products or lack an understanding of human-product spatial relationships, leading to unrealistic representations and unnatural interactions. To address these challenges, we propose a Diffusion Transformer (DiT)-based framework. Our method simultaneously preserves human identities and product-specific details, such as logos and textures, by injecting paired human-product reference information and utilizing an additional masked cross-attention mechanism. We employ a 3D body mesh template and product bounding boxes to provide precise motion guidance, enabling intuitive alignment of hand gestures with product placements. Additionally, structured text encoding is used to incorporate category-level semantics, enhancing 3D consistency during small rotational changes across frames. Trained on a hybrid dataset with extensive data augmentation strategies, our approach outperforms state-of-the-art techniques in maintaining the identity integrity of both humans and products and generating realistic demonstration motions. Project page: https://submit2025-dream.github.io/DreamActor-H1/.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10568</guid>
      <pubDate>Thu, 12 Jun 2025 10:58:23 +0000</pubDate>
    </item>
    <item>
      <title>PosterCraft: Rethinking High-Quality Aesthetic Poster Generation in a   Unified Framework</title>
      <link>https://arxiv.org/abs/2506.10741</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10741.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; SiXiang Chen, Jianyu Lai, Jialin Gao, Tian Ye, Haoyu Chen, Hengyu Shi, Shitong Shao, Yunlong Lin, Song Fei, Zhaohu Xing, Yeying Jin, Junfeng Luo, Xiaoming Wei, Lei Zhu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 22&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Generating aesthetic posters is more challenging than simple design images: it requires not only precise text rendering but also the seamless integration of abstract artistic content, striking layouts, and overall stylistic harmony. To address this, we propose PosterCraft, a unified framework that abandons prior modular pipelines and rigid, predefined layouts, allowing the model to freely explore coherent, visually compelling compositions. PosterCraft employs a carefully designed, cascaded workflow to optimize the generation of high-aesthetic posters: (i) large-scale text-rendering optimization on our newly introduced Text-Render-2M dataset; (ii) region-aware supervised fine-tuning on HQ-Poster100K; (iii) aesthetic-text-reinforcement learning via best-of-n preference optimization; and (iv) joint vision-language feedback refinement. Each stage is supported by a fully automated data-construction pipeline tailored to its specific needs, enabling robust training without complex architectural modifications. Evaluated on multiple experiments, PosterCraft significantly outperforms open-source baselines in rendering accuracy, layout coherence, and overall visual appeal-approaching the quality of SOTA commercial systems. Our code, models, and datasets can be found in the Project page: https://ephemeral182.github.io/PosterCraft&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10741</guid>
      <pubDate>Thu, 12 Jun 2025 14:28:12 +0000</pubDate>
    </item>
    <item>
      <title>VideoDeepResearch: Long Video Understanding With Agentic Tool Using</title>
      <link>https://arxiv.org/abs/2506.10821</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10821.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Huaying Yuan, Zheng Liu, Junjie Zhou, Ji-Rong Wen, Zhicheng Dou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 17&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Long video understanding (LVU) presents a significant challenge for current multi-modal large language models (MLLMs) due to the task's inherent complexity and context window constraint. It is widely assumed that addressing LVU tasks requires foundation MLLMs with extended context windows, strong visual perception capabilities, and proficient domain expertise. In this work, we challenge this common belief by introducing VideoDeepResearch, a novel agentic framework for long video understanding. Our approach relies solely on a text-only large reasoning model (LRM) combined with a modular multi-modal toolkit, including multimodal retrievers and visual perceivers, all of which are readily available in practice. For each LVU task, the system formulates a problem-solving strategy through reasoning, while selectively accessing and utilizing essential video content via tool using. We conduct extensive experiments on popular LVU benchmarks, including MLVU, Video-MME, and LVBench. Our results demonstrate that VideoDeepResearch achieves substantial improvements over existing MLLM baselines, surpassing the previous state-of-the-art by 9.6%, 6.6%, and 3.9% on MLVU (test), LVBench, and LongVideoBench, respectively. These findings highlight the promise of agentic systems in overcoming key challenges in LVU problems.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10821</guid>
      <pubDate>Thu, 12 Jun 2025 15:39:10 +0000</pubDate>
    </item>
    <item>
      <title>VRBench: A Benchmark for Multi-Step Reasoning in Long Narrative Videos</title>
      <link>https://arxiv.org/abs/2506.10857</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10857.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jiashuo Yu, Yue Wu, Meng Chu, Zhifei Ren, Zizheng Huang, Pei Chu, Ruijie Zhang, Yinan He, Qirui Li, Songze Li, Zhenxiang Li, Zhongying Tu, Conghui He, Yu Qiao, Yali Wang, Yi Wang, Limin Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 31&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We present VRBench, the first long narrative video benchmark crafted for evaluating large models' multi-step reasoning capabilities, addressing limitations in existing evaluations that overlook temporal reasoning and procedural validity. It comprises 1,010 long videos (with an average duration of 1.6 hours), along with 9,468 human-labeled multi-step question-answering pairs and 30,292 reasoning steps with timestamps. These videos are curated via a multi-stage filtering process including expert inter-rater reviewing to prioritize plot coherence. We develop a human-AI collaborative framework that generates coherent reasoning chains, each requiring multiple temporally grounded steps, spanning seven types (e.g., event attribution, implicit inference). VRBench designs a multi-phase evaluation pipeline that assesses models at both the outcome and process levels. Apart from the MCQs for the final results, we propose a progress-level LLM-guided scoring metric to evaluate the quality of the reasoning chain from multiple dimensions comprehensively. Through extensive evaluations of 12 LLMs and 16 VLMs on VRBench, we undertake a thorough analysis and provide valuable insights that advance the field of multi-step reasoning.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10857</guid>
      <pubDate>Thu, 12 Jun 2025 16:17:17 +0000</pubDate>
    </item>
    <item>
      <title>CreatiPoster: Towards Editable and Controllable Multi-Layer Graphic   Design Generation</title>
      <link>https://arxiv.org/abs/2506.10890</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10890.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zhao Zhang, Yutao Cheng, Dexiang Hong, Maoke Yang, Gonglei Shi, Lei Ma, Hui Zhang, Jie Shao, Xinglong Wu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 11&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Graphic design plays a crucial role in both commercial and personal contexts, yet creating high-quality, editable, and aesthetically pleasing graphic compositions remains a time-consuming and skill-intensive task, especially for beginners. Current AI tools automate parts of the workflow, but struggle to accurately incorporate user-supplied assets, maintain editability, and achieve professional visual appeal. Commercial systems, like Canva Magic Design, rely on vast template libraries, which are impractical for replicate. In this paper, we introduce CreatiPoster, a framework that generates editable, multi-layer compositions from optional natural-language instructions or assets. A protocol model, an RGBA large multimodal model, first produces a JSON specification detailing every layer (text or asset) with precise layout, hierarchy, content and style, plus a concise background prompt. A conditional background model then synthesizes a coherent background conditioned on this rendered foreground layers. We construct a benchmark with automated metrics for graphic-design generation and show that CreatiPoster surpasses leading open-source approaches and proprietary commercial systems. To catalyze further research, we release a copyright-free corpus of 100,000 multi-layer designs. CreatiPoster supports diverse applications such as canvas editing, text overlay, responsive resizing, multilingual adaptation, and animated posters, advancing the democratization of AI-assisted graphic design. Project homepage: https://github.com/graphic-design-ai/creatiposter&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10890</guid>
      <pubDate>Thu, 12 Jun 2025 16:54:39 +0000</pubDate>
    </item>
    <item>
      <title>Magistral</title>
      <link>https://arxiv.org/abs/2506.10910</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10910.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Mistral-AI, Abhinav Rastogi, Albert Q. Jiang, Andy Lo, Gabrielle Berrada, Guillaume Lample, Jason Rute, Joep Barmentlo, Karmesh Yadav, Kartik Khandelwal, Khyathi Raghavi Chandu, Léonard Blier, Lucile Saulnier, Matthieu Dinot, Maxime Darrin, Neha Gupta, Roman Soletskyi, Sagar Vaze, Teven Le Scao, Yihan Wang, Adam Yang, Alexander H. Liu, Alexandre Sablayrolles, Amélie Héliou, Amélie Martin, Andy Ehrenberg, Anmol Agarwal, Antoine Roux, Arthur Darcet, Arthur Mensch, Baptiste Bout, Baptiste Rozière, Baudouin De Monicault, Chris Bamford, Christian Wallenwein, Christophe Renaudin, Clémence Lanfranchi, Darius Dabert, Devon Mizelle, Diego de las Casas, Elliot Chane-Sane, Emilien Fugier, Emma Bou Hanna, Gauthier Delerce, Gauthier Guinet, Georgii Novikov, Guillaume Martin, Himanshu Jaju, Jan Ludziejewski, Jean-Hadrien Chabran, Jean-Malo Delignon, Joachim Studnia, Jonas Amar, Josselin Somerville Roberts, Julien Denize, Karan Saxena, Kush Jain, Lingxiao Zhao, Louis Martin, Luyu Gao, Lélio Renard Lavaud, Marie Pellat, Mathilde Guillaumin, Mathis Felardos, Maximilian Augustin, Mickaël Seznec, Nikhil Raghuraman, Olivier Duchenne, Patricia Wang, Patrick von Platen, Patryk Saffer, Paul Jacob, Paul Wambergue, Paula Kurylowicz, Pavankumar Reddy Muddireddy, Philomène Chagniot, Pierre Stock, Pravesh Agrawal, Romain Sauvestre, Rémi Delacourt, Sanchit Gandhi, Sandeep Subramanian, Shashwat Dalal, Siddharth Gandhi, Soham Ghosh, Srijan Mishra, Sumukh Aithal, Szymon Antoniak, Thibault Schueller, Thibaut Lavril, Thomas Robert, Thomas Wang, Timothée Lacroix, Valeriia Nemychnikova, Victor Paltz, Virgile Richard, Wen-Ding Li, William Marshall, Xuanyu Zhang, Yunhao Tang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 41&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce Magistral, Mistral's first reasoning model and our own scalable reinforcement learning (RL) pipeline. Instead of relying on existing implementations and RL traces distilled from prior models, we follow a ground up approach, relying solely on our own models and infrastructure. Notably, we demonstrate a stack that enabled us to explore the limits of pure RL training of LLMs, present a simple method to force the reasoning language of the model, and show that RL on text data alone maintains most of the initial checkpoint's capabilities. We find that RL on text maintains or improves multimodal understanding, instruction following and function calling. We present Magistral Medium, trained for reasoning on top of Mistral Medium 3 with RL alone, and we open-source Magistral Small (Apache 2.0) which further includes cold-start data from Magistral Medium.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10910</guid>
      <pubDate>Thu, 12 Jun 2025 17:22:37 +0000</pubDate>
    </item>
    <item>
      <title>Decomposing MLP Activations into Interpretable Features via   Semi-Nonnegative Matrix Factorization</title>
      <link>https://arxiv.org/abs/2506.10920</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10920.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Or Shafran, Atticus Geiger, Mor Geva&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 5&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; A central goal for mechanistic interpretability has been to identify the right units of analysis in large language models (LLMs) that causally explain their outputs. While early work focused on individual neurons, evidence that neurons often encode multiple concepts has motivated a shift toward analyzing directions in activation space. A key question is how to find directions that capture interpretable features in an unsupervised manner. Current methods rely on dictionary learning with sparse autoencoders (SAEs), commonly trained over residual stream activations to learn directions from scratch. However, SAEs often struggle in causal evaluations and lack intrinsic interpretability, as their learning is not explicitly tied to the computations of the model. Here, we tackle these limitations by directly decomposing MLP activations with semi-nonnegative matrix factorization (SNMF), such that the learned features are (a) sparse linear combinations of co-activated neurons, and (b) mapped to their activating inputs, making them directly interpretable. Experiments on Llama 3.1, Gemma 2 and GPT-2 show that SNMF derived features outperform SAEs and a strong supervised baseline (difference-in-means) on causal steering, while aligning with human-interpretable concepts. Further analysis reveals that specific neuron combinations are reused across semantically-related features, exposing a hierarchical structure in the MLP's activation space. Together, these results position SNMF as a simple and effective tool for identifying interpretable features and dissecting concept representations in LLMs.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10920</guid>
      <pubDate>Thu, 12 Jun 2025 17:33:29 +0000</pubDate>
    </item>
    <item>
      <title>Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture   without Training</title>
      <link>https://arxiv.org/abs/2506.10952</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10952.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Mozhi Zhang, Howe Tissue, Lu Wang, Xipeng Qiu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 21&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce~Domain2Vec, a novel approach that decomposes any dataset into a linear combination of several meta-domains, a new concept designed to capture the key underlying features of datasets. Domain2Vec maintains a vocabulary of meta-domains and uses a classifier to decompose any given dataset into a domain vector that corresponds to a distribution over this vocabulary. These domain vectors enable the identification of the optimal data mixture for language model (LM) pretraining in a training-free manner under the \textbf{Distribution Alignment Assumption} (DA^{2}), which suggests that when the data distributions of the training set and the validation set are better aligned, a lower validation loss is achieved. Moreover, Domain2vec can be seamlessly integrated into previous works to model the relationship between domain vectors and LM performance, greatly enhancing the efficiency and scalability of previous methods. Extensive experiments demonstrate that Domain2Vec helps find the data mixture that enhances downstream task performance with minimal computational overhead. Specifically, Domain2Vec achieves the same validation loss on Pile-CC using only 51.5% of the computation required when training on the original mixture of The Pile dataset. Under equivalent compute budget, Domain2Vec improves downstream performance by an average of 2.83%.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10952</guid>
      <pubDate>Thu, 12 Jun 2025 17:53:51 +0000</pubDate>
    </item>
    <item>
      <title>Build the web for agents, not agents for the web</title>
      <link>https://arxiv.org/abs/2506.10953</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10953.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Xing Han Lù, Gaurav Kamath, Marius Mosbach, Siva Reddy&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 15&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advancements in Large Language Models (LLMs) and multimodal counterparts have spurred significant interest in developing web agents -- AI systems capable of autonomously navigating and completing tasks within web environments. While holding tremendous promise for automating complex web interactions, current approaches face substantial challenges due to the fundamental mismatch between human-designed interfaces and LLM capabilities. Current methods struggle with the inherent complexity of web inputs, whether processing massive DOM trees, relying on screenshots augmented with additional information, or bypassing the user interface entirely through API interactions. This position paper advocates for a paradigm shift in web agent research: rather than forcing web agents to adapt to interfaces designed for humans, we should develop a new interaction paradigm specifically optimized for agentic capabilities. To this end, we introduce the concept of an Agentic Web Interface (AWI), an interface specifically designed for agents to navigate a website. We establish six guiding principles for AWI design, emphasizing safety, efficiency, and standardization, to account for the interests of all primary stakeholders. This reframing aims to overcome fundamental limitations of existing interfaces, paving the way for more efficient, reliable, and transparent web agent design, which will be a collaborative effort involving the broader ML community.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10953</guid>
      <pubDate>Thu, 12 Jun 2025 17:53:58 +0000</pubDate>
    </item>
    <item>
      <title>SWE-Factory: Your Automated Factory for Issue Resolution Training Data   and Evaluation Benchmarks</title>
      <link>https://arxiv.org/abs/2506.10954</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10954.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Lianghong Guo, Yanlin Wang, Caihua Li, Pengyu Yang, Jiachi Chen, Wei Tao, Yingtian Zou, Duyu Tang, Zibin Zheng&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 45&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Constructing large-scale datasets for the GitHub issue resolution task is crucial for both training and evaluating the software engineering capabilities of Large Language Models (LLMs). However, the traditional process for creating such benchmarks is notoriously challenging and labor-intensive, particularly in the stages of setting up evaluation environments, grading test outcomes, and validating task instances. In this paper, we propose SWE-Factory, an automated pipeline designed to address these challenges. To tackle these issues, our pipeline integrates three core automated components. First, we introduce SWE-Builder, a multi-agent system that automates evaluation environment construction, which employs four specialized agents that work in a collaborative, iterative loop and leverages an environment memory pool to enhance efficiency. Second, we introduce a standardized, exit-code-based grading method that eliminates the need for manually writing custom parsers. Finally, we automate the fail2pass validation process using these reliable exit code signals. Experiments on 671 issues across four programming languages show that our pipeline can effectively construct valid task instances; for example, with GPT-4.1-mini, our SWE-Builder constructs 269 valid instances at 0.045 per instance, while with Gemini-2.5-flash, it achieves comparable performance at the lowest cost of 0.024 per instance. We also demonstrate that our exit-code-based grading achieves 100% accuracy compared to manual inspection, and our automated fail2pass validation reaches a precision of 0.92 and a recall of 1.00. We hope our automated pipeline will accelerate the collection of large-scale, high-quality GitHub issue resolution datasets for both training and evaluation. Our code and datasets are released at https://github.com/DeepSoftwareAnalytics/swe-factory.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10954</guid>
      <pubDate>Thu, 12 Jun 2025 17:54:17 +0000</pubDate>
    </item>
    <item>
      <title>ChineseHarm-Bench: A Chinese Harmful Content Detection Benchmark</title>
      <link>https://arxiv.org/abs/2506.10960</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10960.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Kangwei Liu, Siyuan Cheng, Bozhong Tian, Xiaozhuan Liang, Yuyang Yin, Meng Han, Ningyu Zhang, Bryan Hooi, Xi Chen, Shumin Deng&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 11&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large language models (LLMs) have been increasingly applied to automated harmful content detection tasks, assisting moderators in identifying policy violations and improving the overall efficiency and accuracy of content review. However, existing resources for harmful content detection are predominantly focused on English, with Chinese datasets remaining scarce and often limited in scope. We present a comprehensive, professionally annotated benchmark for Chinese content harm detection, which covers six representative categories and is constructed entirely from real-world data. Our annotation process further yields a knowledge rule base that provides explicit expert knowledge to assist LLMs in Chinese harmful content detection. In addition, we propose a knowledge-augmented baseline that integrates both human-annotated knowledge rules and implicit knowledge from large language models, enabling smaller models to achieve performance comparable to state-of-the-art LLMs. Code and data are available at https://github.com/zjunlp/ChineseHarm-bench.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10960</guid>
      <pubDate>Thu, 12 Jun 2025 17:57:05 +0000</pubDate>
    </item>
    <item>
      <title>AutoMind: Adaptive Knowledgeable Agent for Automated Data Science</title>
      <link>https://arxiv.org/abs/2506.10974</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10974.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yixin Ou, Yujie Luo, Jingsheng Zheng, Lanning Wei, Shuofei Qiao, Jintian Zhang, Da Zheng, Huajun Chen, Ningyu Zhang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 17&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large Language Model (LLM) agents have shown great potential in addressing real-world data science problems. LLM-driven data science agents promise to automate the entire machine learning pipeline, yet their real-world effectiveness remains limited. Existing frameworks depend on rigid, pre-defined workflows and inflexible coding strategies; consequently, they excel only on relatively simple, classical problems and fail to capture the empirical expertise that human practitioners bring to complex, innovative tasks. In this work, we introduce AutoMind, an adaptive, knowledgeable LLM-agent framework that overcomes these deficiencies through three key advances: (1) a curated expert knowledge base that grounds the agent in domain expert knowledge, (2) an agentic knowledgeable tree search algorithm that strategically explores possible solutions, and (3) a self-adaptive coding strategy that dynamically tailors code generation to task complexity. Evaluations on two automated data science benchmarks demonstrate that AutoMind delivers superior performance versus state-of-the-art baselines. Additional analyses confirm favorable effectiveness, efficiency, and qualitative solution quality, highlighting AutoMind as an efficient and robust step toward fully automated data science.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10974</guid>
      <pubDate>Thu, 12 Jun 2025 17:59:32 +0000</pubDate>
    </item>
    <item>
      <title>Fine-Grained Perturbation Guidance via Attention Head Selection</title>
      <link>https://arxiv.org/abs/2506.10978</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10978.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Donghoon Ahn, Jiwon Kang, Sanghyun Lee, Minjae Kim, Jaewon Min, Wooseok Jang, Saungwu Lee, Sayak Paul, Susung Hong, Seungryong Kim&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 22&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent guidance methods in diffusion models steer reverse sampling by perturbing the model to construct an implicit weak model and guide generation away from it. Among these approaches, attention perturbation has demonstrated strong empirical performance in unconditional scenarios where classifier-free guidance is not applicable. However, existing attention perturbation methods lack principled approaches for determining where perturbations should be applied, particularly in Diffusion Transformer (DiT) architectures where quality-relevant computations are distributed across layers. In this paper, we investigate the granularity of attention perturbations, ranging from the layer level down to individual attention heads, and discover that specific heads govern distinct visual concepts such as structure, style, and texture quality. Building on this insight, we propose "HeadHunter", a systematic framework for iteratively selecting attention heads that align with user-centric objectives, enabling fine-grained control over generation quality and visual attributes. In addition, we introduce SoftPAG, which linearly interpolates each selected head's attention map toward an identity matrix, providing a continuous knob to tune perturbation strength and suppress artifacts. Our approach not only mitigates the oversmoothing issues of existing layer-level perturbation but also enables targeted manipulation of specific visual styles through compositional head selection. We validate our method on modern large-scale DiT-based text-to-image models including Stable Diffusion 3 and FLUX.1, demonstrating superior performance in both general quality enhancement and style-specific guidance. Our work provides the first head-level analysis of attention perturbation in diffusion models, uncovering interpretable specialization within attention layers and enabling practical design of effective perturbation strategies.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10978</guid>
      <pubDate>Thu, 12 Jun 2025 17:59:51 +0000</pubDate>
    </item>
  </channel>
</rss>