huggingface-daily-paper-feed/feed_monthly.xml

<?xml version='1.0' encoding='UTF-8'?>
<rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" version="2.0">
  <channel>
    <title>Hugging Face Daily Papers</title>
    <link>https://huggingface.co/papers</link>
    <description>Daily research papers curated by the Hugging Face community.</description>
    <docs>http://www.rssboard.org/rss-specification</docs>
    <generator>python-feedgen</generator>
    <language>en</language>
    <lastBuildDate>Tue, 01 Jul 2025 00:13:58 +0000</lastBuildDate>
    <item>
      <title>Large Language Models for Data Synthesis</title>
      <link>https://arxiv.org/abs/2505.14752</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.14752.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yihong Tang, Menglin Kong, Lijun Sun&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 49&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Generating synthetic data that faithfully captures the statistical structure of real-world distributions is a fundamental challenge in data modeling. Classical approaches often depend on strong parametric assumptions or manual structural design and struggle in high-dimensional or heterogeneous domains. Recent progress in Large Language Models (LLMs) reveals their potential as flexible, high-dimensional priors over real-world distributions. However, when applied to data synthesis, standard LLM-based sampling is inefficient, constrained by fixed context limits, and fails to ensure statistical alignment. Given this, we introduce LLMSynthor, a general framework for data synthesis that transforms LLMs into structure-aware simulators guided by distributional feedback. LLMSynthor treats the LLM as a nonparametric copula simulator for modeling high-order dependencies and introduces LLM Proposal Sampling to generate grounded proposal distributions that improve sampling efficiency without requiring rejection. By minimizing discrepancies in the summary statistics space, the iterative synthesis loop aligns real and synthetic data while gradually uncovering and refining the latent generative structure. We evaluate LLMSynthor in both controlled and real-world settings using heterogeneous datasets in privacy-sensitive domains (e.g., e-commerce, population, and mobility) that encompass both structured and unstructured formats. The synthetic data produced by LLMSynthor shows high statistical fidelity, practical utility, and cross-data adaptability, positioning it as a valuable tool across economics, social science, urban studies, and beyond.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.14752</guid>
      <pubDate>Tue, 20 May 2025 13:35:38 +0000</pubDate>
    </item>
    <item>
      <title>CASS: Nvidia to AMD Transpilation with Data, Models, and Benchmark</title>
      <link>https://arxiv.org/abs/2505.16968</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.16968.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Ahmed Heakl, Sarim Hashmi, Gustavo Bertolo Stahl, Seung Hun Eddie Han, Salman Khan, Abdulrahman Mahmoud&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 41&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce CASS, the first large-scale dataset and model suite for cross-architecture GPU code transpilation, targeting both source-level (CUDA leftrightarrow HIP) and assembly-level (Nvidia SASS leftrightarrow AMD RDNA3) translation. The dataset comprises 70k verified code pairs across host and device, addressing a critical gap in low-level GPU code portability. Leveraging this resource, we train the CASS family of domain-specific language models, achieving 95% source translation accuracy and 37.5% assembly translation accuracy, substantially outperforming commercial baselines such as GPT-4o, Claude, and Hipify. Our generated code matches native performance in over 85% of test cases, preserving runtime and memory behavior. To support rigorous evaluation, we introduce CASS-Bench, a curated benchmark spanning 16 GPU domains with ground-truth execution. All data, models, and evaluation tools are released as open source to foster progress in GPU compiler tooling, binary compatibility, and LLM-guided hardware translation. Dataset and benchmark are on https://huggingface.co/datasets/MBZUAI/cass{blue{HuggingFace}}, with code at https://github.com/GustavoStahl/CASS{blue{GitHub}}.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.16968</guid>
      <pubDate>Thu, 22 May 2025 17:48:53 +0000</pubDate>
    </item>
    <item>
      <title>Don't Look Only Once: Towards Multimodal Interactive Reasoning with   Selective Visual Revisitation</title>
      <link>https://arxiv.org/abs/2505.18842</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.18842.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jiwan Chung, Junhyeok Kim, Siyeol Kim, Jaeyoung Lee, Min Soo Kim, Youngjae Yu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 37&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We present v1, a lightweight extension to Multimodal Large Language Models (MLLMs) that enables selective visual revisitation during inference. While current MLLMs typically consume visual input only once and reason purely over internal memory, v1 introduces a simple point-and-copy mechanism that allows the model to dynamically retrieve relevant image regions throughout the reasoning process. This mechanism augments existing architectures with minimal modifications, enabling contextual access to visual tokens based on the model's evolving hypotheses. To train this capability, we construct v1g, a dataset of 300K multimodal reasoning traces with interleaved visual grounding annotations. Experiments on three multimodal mathematical reasoning benchmarks -- MathVista, MathVision, and MathVerse -- demonstrate that v1 consistently improves performance over comparable baselines, particularly on tasks requiring fine-grained visual reference and multi-step reasoning. Our results suggest that dynamic visual access is a promising direction for enhancing grounded multimodal reasoning. Code, models, and data will be released to support future research.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.18842</guid>
      <pubDate>Sat, 24 May 2025 19:30:47 +0000</pubDate>
    </item>
    <item>
      <title>Will It Still Be True Tomorrow? Multilingual Evergreen Question   Classification to Improve Trustworthy QA</title>
      <link>https://arxiv.org/abs/2505.21115</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.21115.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Sergey Pletenev, Maria Marina, Nikolay Ivanov, Daria Galimzianova, Nikita Krayko, Mikhail Salnikov, Vasily Konovalov, Alexander Panchenko, Viktor Moskvoretskii&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 133&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large Language Models (LLMs) often hallucinate in question answering (QA) tasks. A key yet underexplored factor contributing to this is the temporality of questions -- whether they are evergreen (answers remain stable over time) or mutable (answers change). In this work, we introduce EverGreenQA, the first multilingual QA dataset with evergreen labels, supporting both evaluation and training. Using EverGreenQA, we benchmark 12 modern LLMs to assess whether they encode question temporality explicitly (via verbalized judgments) or implicitly (via uncertainty signals). We also train EG-E5, a lightweight multilingual classifier that achieves SoTA performance on this task. Finally, we demonstrate the practical utility of evergreen classification across three applications: improving self-knowledge estimation, filtering QA datasets, and explaining GPT-4o retrieval behavior.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.21115</guid>
      <pubDate>Tue, 27 May 2025 12:35:13 +0000</pubDate>
    </item>
    <item>
      <title>HardTests: Synthesizing High-Quality Test Cases for LLM Coding</title>
      <link>https://arxiv.org/abs/2505.24098</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.24098.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zhongmou He, Yee Man Choi, Kexun Zhang, Jiabao Ji, Junting Zhou, Dejia Xu, Ivan Bercovich, Aidan Zhang, Lei Li&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 44&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Verifiers play a crucial role in large language model (LLM) reasoning, needed by post-training techniques such as reinforcement learning. However, reliable verifiers are hard to get for difficult coding problems, because a well-disguised wrong solution may only be detected by carefully human-written edge cases that are difficult to synthesize. To address this issue, we propose HARDTESTGEN, a pipeline for high-quality test synthesis using LLMs. With this pipeline, we curate a comprehensive competitive programming dataset HARDTESTS with 47k problems and synthetic high-quality tests. Compared with existing tests, HARDTESTGEN tests demonstrate precision that is 11.3 percentage points higher and recall that is 17.5 percentage points higher when evaluating LLM-generated code. For harder problems, the improvement in precision can be as large as 40 points. HARDTESTS also proves to be more effective for model training, measured by downstream code generation performance. We will open-source our dataset and synthesis pipeline at https://leililab.github.io/HardTests/.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.24098</guid>
      <pubDate>Fri, 30 May 2025 01:00:34 +0000</pubDate>
    </item>
    <item>
      <title>CSVQA: A Chinese Multimodal Benchmark for Evaluating STEM Reasoning   Capabilities of VLMs</title>
      <link>https://arxiv.org/abs/2505.24120</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.24120.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Ai Jian, Weijie Qiu, Xiaokun Wang, Peiyu Wang, Yunzhuo Hao, Jiangbo Pei, Yichen Wei, Yi Peng, Xuchen Song&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 49&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Vision-Language Models (VLMs) have demonstrated remarkable progress in multimodal understanding, yet their capabilities for scientific reasoning remains inadequately assessed. Current multimodal benchmarks predominantly evaluate generic image comprehension or text-driven reasoning, lacking authentic scientific contexts that require domain-specific knowledge integration with visual evidence analysis. To fill this gap, we present CSVQA, a diagnostic multimodal benchmark specifically designed for evaluating scientific reasoning through domain-grounded visual question answering.Our benchmark features 1,378 carefully constructed question-answer pairs spanning diverse STEM disciplines, each demanding domain knowledge, integration of visual evidence, and higher-order reasoning. Compared to prior multimodal benchmarks, CSVQA places greater emphasis on real-world scientific content and complex reasoning.We additionally propose a rigorous evaluation protocol to systematically assess whether model predictions are substantiated by valid intermediate reasoning steps based on curated explanations. Our comprehensive evaluation of 15 VLMs on this benchmark reveals notable performance disparities, as even the top-ranked proprietary model attains only 49.6\% accuracy.This empirical evidence underscores the pressing need for advancing scientific reasoning capabilities in VLMs. Our CSVQA is released at https://huggingface.co/datasets/Skywork/CSVQA.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.24120</guid>
      <pubDate>Fri, 30 May 2025 01:34:25 +0000</pubDate>
    </item>
    <item>
      <title>FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation</title>
      <link>https://arxiv.org/abs/2505.24714</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.24714.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Junyu Luo, Zhizhuo Kou, Liming Yang, Xiao Luo, Jinsheng Huang, Zhiping Xiao, Jingshu Peng, Chengzhong Liu, Jiaming Ji, Xuanzhe Liu, Sirui Han, Ming Zhang, Yike Guo&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Multimodal Large Language Models (MLLMs) have experienced rapid development in recent years. However, in the financial domain, there is a notable lack of effective and specialized multimodal evaluation datasets. To advance the development of MLLMs in the finance domain, we introduce FinMME, encompassing more than 11,000 high-quality financial research samples across 18 financial domains and 6 asset classes, featuring 10 major chart types and 21 subtypes. We ensure data quality through 20 annotators and carefully designed validation mechanisms. Additionally, we develop FinScore, an evaluation system incorporating hallucination penalties and multi-dimensional capability assessment to provide an unbiased evaluation. Extensive experimental results demonstrate that even state-of-the-art models like GPT-4o exhibit unsatisfactory performance on FinMME, highlighting its challenging nature. The benchmark exhibits high robustness with prediction variations under different prompts remaining below 1%, demonstrating superior reliability compared to existing datasets. Our dataset and evaluation protocol are available at https://huggingface.co/datasets/luojunyu/FinMME and https://github.com/luo-junyu/FinMME.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.24714</guid>
      <pubDate>Fri, 30 May 2025 15:36:19 +0000</pubDate>
    </item>
    <item>
      <title>Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning</title>
      <link>https://arxiv.org/abs/2505.24726</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.24726.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shelly Bensal, Umar Jamil, Christopher Bryant, Melisa Russak, Kiran Kamble, Dmytro Mozolevskyi, Muayad Ali, Waseem AlShikh&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 244&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We explore a method for improving the performance of large language models through self-reflection and reinforcement learning. By incentivizing the model to generate better self-reflections when it answers incorrectly, we demonstrate that a model's ability to solve complex, verifiable tasks can be enhanced even when generating synthetic data is infeasible and only binary feedback is available. Our framework operates in two stages: first, upon failing a given task, the model generates a self-reflective commentary analyzing its previous attempt; second, the model is given another attempt at the task with the self-reflection in context. If the subsequent attempt succeeds, the tokens generated during the self-reflection phase are rewarded. Our experimental results show substantial performance gains across a variety of model architectures, as high as 34.7% improvement at math equation writing and 18.1% improvement at function calling. Notably, smaller fine-tuned models (1.5 billion to 7 billion parameters) outperform models in the same family that are 10 times larger. Our novel paradigm is thus an exciting pathway to more useful and reliable language models that can self-improve on challenging tasks with limited external feedback.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.24726</guid>
      <pubDate>Fri, 30 May 2025 15:49:42 +0000</pubDate>
    </item>
    <item>
      <title>REASONING GYM: Reasoning Environments for Reinforcement Learning with   Verifiable Rewards</title>
      <link>https://arxiv.org/abs/2505.24760</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.24760.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zafir Stojanovski, Oliver Stanley, Joe Sharratt, Richard Jones, Abdulhakeem Adefioye, Jean Kaddour, Andreas Köpf&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 64&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce Reasoning Gym (RG), a library of reasoning environments for reinforcement learning with verifiable rewards. It provides over 100 data generators and verifiers spanning multiple domains including algebra, arithmetic, computation, cognition, geometry, graph theory, logic, and various common games. Its key innovation is the ability to generate virtually infinite training data with adjustable complexity, unlike most previous reasoning datasets, which are typically fixed. This procedural generation approach allows for continuous evaluation across varying difficulty levels. Our experimental results demonstrate the efficacy of RG in both evaluating and reinforcement learning of reasoning models.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.24760</guid>
      <pubDate>Fri, 30 May 2025 16:20:18 +0000</pubDate>
    </item>
    <item>
      <title>AlphaOne: Reasoning Models Thinking Slow and Fast at Test Time</title>
      <link>https://arxiv.org/abs/2505.24863</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.24863.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Junyu Zhang, Runpei Dong, Han Wang, Xuying Ning, Haoran Geng, Peihao Li, Xialin He, Yutong Bai, Jitendra Malik, Saurabh Gupta, Huan Zhang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 95&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; This paper presents AlphaOne (alpha1), a universal framework for modulating reasoning progress in large reasoning models (LRMs) at test time. alpha1 first introduces alpha moment, which represents the scaled thinking phase with a universal parameter alpha. Within this scaled pre-alpha moment phase, it dynamically schedules slow thinking transitions by modeling the insertion of reasoning transition tokens as a Bernoulli stochastic process. After the alpha moment, alpha1 deterministically terminates slow thinking with the end-of-thinking token, thereby fostering fast reasoning and efficient answer generation. This approach unifies and generalizes existing monotonic scaling methods by enabling flexible and dense slow-to-fast reasoning modulation. Extensive empirical studies on various challenging benchmarks across mathematical, coding, and scientific domains demonstrate alpha1's superior reasoning capability and efficiency. Project page: https://alphaone-project.github.io/&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.24863</guid>
      <pubDate>Fri, 30 May 2025 17:58:36 +0000</pubDate>
    </item>
    <item>
      <title>ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in   Large Language Models</title>
      <link>https://arxiv.org/abs/2505.24864</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.24864.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, Yi Dong&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 132&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in reasoning-centric language models have highlighted reinforcement learning (RL) as a promising method for aligning models with verifiable rewards. However, it remains contentious whether RL truly expands a model's reasoning capabilities or merely amplifies high-reward outputs already latent in the base model's distribution, and whether continually scaling up RL compute reliably leads to improved reasoning performance. In this work, we challenge prevailing assumptions by demonstrating that prolonged RL (ProRL) training can uncover novel reasoning strategies that are inaccessible to base models, even under extensive sampling. We introduce ProRL, a novel training methodology that incorporates KL divergence control, reference policy resetting, and a diverse suite of tasks. Our empirical analysis reveals that RL-trained models consistently outperform base models across a wide range of pass@k evaluations, including scenarios where base models fail entirely regardless of the number of attempts. We further show that reasoning boundary improvements correlates strongly with task competence of base model and training duration, suggesting that RL can explore and populate new regions of solution space over time. These findings offer new insights into the conditions under which RL meaningfully expands reasoning boundaries in language models and establish a foundation for future work on long-horizon RL for reasoning. We release model weights to support further research: https://huggingface.co/nvidia/Nemotron-Research-Reasoning-Qwen-1.5B&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.24864</guid>
      <pubDate>Fri, 30 May 2025 17:59:01 +0000</pubDate>
    </item>
    <item>
      <title>Time Blindness: Why Video-Language Models Can't See What Humans Can?</title>
      <link>https://arxiv.org/abs/2505.24867</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2505.24867.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Ujjwal Upadhyay, Mukul Ranjan, Zhiqiang Shen, Mohamed Elhoseiny&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 78&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in vision-language models (VLMs) have made impressive strides in understanding spatio-temporal relationships in videos. However, when spatial information is obscured, these models struggle to capture purely temporal patterns. We introduce SpookyBench, a benchmark where information is encoded solely in temporal sequences of noise-like frames, mirroring natural phenomena from biological signaling to covert communication. Interestingly, while humans can recognize shapes, text, and patterns in these sequences with over 98% accuracy, state-of-the-art VLMs achieve 0% accuracy. This performance gap highlights a critical limitation: an over-reliance on frame-level spatial features and an inability to extract meaning from temporal cues. Furthermore, when trained in data sets with low spatial signal-to-noise ratios (SNR), temporal understanding of models degrades more rapidly than human perception, especially in tasks requiring fine-grained temporal reasoning. Overcoming this limitation will require novel architectures or training paradigms that decouple spatial dependencies from temporal processing. Our systematic analysis shows that this issue persists across model scales and architectures. We release SpookyBench to catalyze research in temporal pattern recognition and bridge the gap between human and machine video understanding. Dataset and code has been made available on our project website: https://timeblindness.github.io/.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2505.24867</guid>
      <pubDate>Fri, 30 May 2025 17:59:12 +0000</pubDate>
    </item>
    <item>
      <title>Temporal In-Context Fine-Tuning for Versatile Control of Video Diffusion   Models</title>
      <link>https://arxiv.org/abs/2506.00996</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.00996.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Kinam Kim, Junha Hyung, Jaegul Choo&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in text-to-video diffusion models have enabled high-quality video synthesis, but controllable generation remains challenging, particularly under limited data and compute. Existing fine-tuning methods for conditional generation often rely on external encoders or architectural modifications, which demand large datasets and are typically restricted to spatially aligned conditioning, limiting flexibility and scalability. In this work, we introduce Temporal In-Context Fine-Tuning (TIC-FT), an efficient and versatile approach for adapting pretrained video diffusion models to diverse conditional generation tasks. Our key idea is to concatenate condition and target frames along the temporal axis and insert intermediate buffer frames with progressively increasing noise levels. These buffer frames enable smooth transitions, aligning the fine-tuning process with the pretrained model's temporal dynamics. TIC-FT requires no architectural changes and achieves strong performance with as few as 10-30 training samples. We validate our method across a range of tasks, including image-to-video and video-to-video generation, using large-scale base models such as CogVideoX-5B and Wan-14B. Extensive experiments show that TIC-FT outperforms existing baselines in both condition fidelity and visual quality, while remaining highly efficient in both training and inference. For additional results, visit https://kinam0252.github.io/TIC-FT/&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.00996</guid>
      <pubDate>Sun, 01 Jun 2025 12:57:43 +0000</pubDate>
    </item>
    <item>
      <title>Taming LLMs by Scaling Learning Rates with Gradient Grouping</title>
      <link>https://arxiv.org/abs/2506.01049</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.01049.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Siyuan Li, Juanxi Tian, Zedong Wang, Xin Jin, Zicheng Liu, Wentao Zhang, Dan Xu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Training large language models (LLMs) poses challenges due to their massive scale and heterogeneous architectures. While adaptive optimizers like AdamW help address gradient variations, they still struggle with efficient and effective parameter-wise learning rate estimation, resulting in training instability, slow convergence, and poor compatibility with parameter-efficient fine-tuning (PEFT) techniques. This work introduces Scaling with Gradient Grouping (SGG), an optimizer wrapper that improves adaptive learning rate estimation by dynamic grouping and group-specific scaling. SGG first groups gradient statistics in each layer into clusters and then applies cluster-specific scaling to calibrate learning rates for each parameter, thus imposing collective group-wise constraints while maintaining precise per-parameter adaptation. Experiments on diverse (M)LLM benchmarks show that SGG integrates seamlessly with existing optimizers, and offers consistent gains and faster convergence over baselines, with various model sizes. Its stability across varying batch sizes and learning rates establishes SGG as a robust choice for LLM optimization.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.01049</guid>
      <pubDate>Sun, 01 Jun 2025 15:30:37 +0000</pubDate>
    </item>
    <item>
      <title>SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware   Reinforcement Learning</title>
      <link>https://arxiv.org/abs/2506.01713</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.01713.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zhongwei Wan, Zhihao Dou, Che Liu, Yu Zhang, Dongfei Cui, Qinjian Zhao, Hui Shen, Jing Xiong, Yi Xin, Yifan Jiang, Yangfan He, Mi Zhang, Shen Yan&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 46&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Multimodal large language models (MLLMs) have shown promising capabilities in reasoning tasks, yet still struggle with complex problems requiring explicit self-reflection and self-correction, especially compared to their unimodal text-based counterparts. Existing reflection methods are simplistic and struggle to generate meaningful and instructive feedback, as the reasoning ability and knowledge limits of pre-trained models are largely fixed during initial training. To overcome these challenges, we propose Multimodal Self-Reflection enhanced reasoning with Group Relative Policy Optimization (SRPO), a two-stage reflection-aware reinforcement learning (RL) framework explicitly designed to enhance multimodal LLM reasoning. In the first stage, we construct a high-quality, reflection-focused dataset under the guidance of an advanced MLLM, which generates reflections based on initial responses to help the policy model learn both reasoning and self-reflection. In the second stage, we introduce a novel reward mechanism within the GRPO framework that encourages concise and cognitively meaningful reflection while avoiding redundancy. Extensive experiments across multiple multimodal reasoning benchmarks, including MathVista, MathVision, MathVerse, and MMMU-Pro, using Qwen-2.5-VL-7B and Qwen-2.5-VL-32B demonstrate that SRPO significantly outperforms state-of-the-art models, achieving notable improvements in both reasoning accuracy and reflection quality.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.01713</guid>
      <pubDate>Mon, 02 Jun 2025 14:21:44 +0000</pubDate>
    </item>
    <item>
      <title>SmolVLA: A Vision-Language-Action Model for Affordable and Efficient   Robotics</title>
      <link>https://arxiv.org/abs/2506.01844</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.01844.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, Simon Alibert, Matthieu Cord, Thomas Wolf, Remi Cadene&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 106&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Vision-language models (VLMs) pretrained on large-scale multimodal datasets encode rich visual and linguistic knowledge, making them a strong foundation for robotics. Rather than training robotic policies from scratch, recent approaches adapt VLMs into vision-language-action (VLA) models that enable natural language-driven perception and control. However, existing VLAs are typically massive--often with billions of parameters--leading to high training costs and limited real-world deployability. Moreover, they rely on academic and industrial datasets, overlooking the growing availability of community-collected data from affordable robotic platforms. In this work, we present SmolVLA, a small, efficient, and community-driven VLA that drastically reduces both training and inference costs, while retaining competitive performance. SmolVLA is designed to be trained on a single GPU and deployed on consumer-grade GPUs or even CPUs. To further improve responsiveness, we introduce an asynchronous inference stack decoupling perception and action prediction from action execution, allowing higher control rates with chunked action generation. Despite its compact size, SmolVLA achieves performance comparable to VLAs that are 10x larger. We evaluate SmolVLA on a range of both simulated as well as real-world robotic benchmarks and release all code, pretrained models, and training data.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.01844</guid>
      <pubDate>Mon, 02 Jun 2025 16:30:19 +0000</pubDate>
    </item>
    <item>
      <title>SynthRL: Scaling Visual Reasoning with Verifiable Data Synthesis</title>
      <link>https://arxiv.org/abs/2506.02096</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.02096.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zijian Wu, Jinjie Ni, Xiangyan Liu, Zichen Liu, Hang Yan, Michael Qizhe Shieh&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 51&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Vision-language models (VLMs) trained via reinforcement learning with verifiable reward (RLVR) have shown notable progress in scaling test-time compute effectively. In this work, we investigate how synthesized RL data can further improve RLVR. To this end, we propose SynthRL-a scalable and guaranteed pipeline for automatic data scaling in reasoning-oriented RL training. SynthRL comprises three key stages: (1) selecting seed questions with appropriate distribution, (2) augmenting them into more challenging variants while preserving the original answers, and (3) a guaranteed verification stage that ensures near-perfect correctness and difficulty enhancement. Our empirical experiments demonstrate SynthRL's scalability and effectiveness. When applied to the MMK12 dataset, SynthRL synthesizes over 3.3K additional verifiable, challenging questions from approximately 8K seed samples. Models trained with our synthesized data achieve consistent gains across five out-of-domain visual math reasoning benchmarks, with a significant improvement over baseline models trained on seed data alone. Notably, detailed analysis reveals that the gains are more pronounced on the most challenging evaluation samples, highlighting SynthRL's effectiveness in eliciting deeper and more complex reasoning patterns.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.02096</guid>
      <pubDate>Mon, 02 Jun 2025 17:45:16 +0000</pubDate>
    </item>
    <item>
      <title>Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective   Reinforcement Learning for LLM Reasoning</title>
      <link>https://arxiv.org/abs/2506.01939</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.01939.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang, Andrew Zhao, Yang Yue, Shiji Song, Bowen Yu, Gao Huang, Junyang Lin&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 165&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a powerful approach to enhancing the reasoning capabilities of Large Language Models (LLMs), while its mechanisms are not yet well understood. In this work, we undertake a pioneering exploration of RLVR through the novel perspective of token entropy patterns, comprehensively analyzing how different tokens influence reasoning performance. By examining token entropy patterns in Chain-of-Thought (CoT) reasoning, we observe that only a small fraction of tokens exhibit high entropy, and these tokens act as critical forks that steer the model toward diverse reasoning pathways. Furthermore, studying how entropy patterns evolve during RLVR training reveals that RLVR largely adheres to the base model's entropy patterns, primarily adjusting the entropy of high-entropy tokens. These findings highlight the significance of high-entropy tokens (i.e., forking tokens) to RLVR. We ultimately improve RLVR by restricting policy gradient updates to forking tokens and uncover a finding even beyond the 80/20 rule: utilizing only 20% of the tokens while maintaining performance comparable to full-gradient updates on the Qwen3-8B base model and significantly surpassing full-gradient updates on the Qwen3-32B (+11.04 on AIME'25 and +7.71 on AIME'24) and Qwen3-14B (+4.79 on AIME'25 and +5.21 on AIME'24) base models, highlighting a strong scaling trend. In contrast, training exclusively on the 80% lowest-entropy tokens leads to a marked decline in performance. These findings indicate that the efficacy of RLVR primarily arises from optimizing the high-entropy tokens that decide reasoning directions. Collectively, our results highlight the potential to understand RLVR through a token-entropy perspective and optimize RLVR by leveraging high-entropy minority tokens to further improve LLM reasoning.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.01939</guid>
      <pubDate>Mon, 02 Jun 2025 17:54:39 +0000</pubDate>
    </item>
    <item>
      <title>VS-Bench: Evaluating VLMs for Strategic Reasoning and Decision-Making in   Multi-Agent Environments</title>
      <link>https://arxiv.org/abs/2506.02387</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.02387.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zelai Xu, Zhexuan Xu, Xiangmin Yi, Huining Yuan, Xinlei Chen, Yi Wu, Chao Yu, Yu Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 57&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advancements in Vision Language Models (VLMs) have expanded their capabilities to interactive agent tasks, yet existing benchmarks remain limited to single-agent or text-only environments. In contrast, real-world scenarios often involve multiple agents interacting within rich visual and linguistic contexts, posing challenges with both multimodal observations and strategic interactions. To bridge this gap, we introduce Visual Strategic Bench (VS-Bench), a multimodal benchmark that evaluates VLMs for strategic reasoning and decision-making in multi-agent environments. VS-Bench comprises eight vision-grounded environments spanning cooperative, competitive, and mixed-motive interactions, designed to assess agents' ability to predict others' future moves and optimize for long-term objectives. We consider two complementary evaluation dimensions, including offline evaluation of strategic reasoning by next-action prediction accuracy and online evaluation of decision-making by normalized episode return. Extensive experiments of fourteen leading VLMs reveal a significant gap between current models and optimal performance, with the best models attaining 47.8% prediction accuracy and 24.3% normalized return. We further conduct in-depth analyses on multimodal observations, test-time scaling, social behaviors, and failure cases of VLM agents. By standardizing the evaluation and highlighting the limitations of existing models, we envision VS-Bench as a foundation for future research on strategic multimodal agents. Code and data are available at https://vs-bench.github.io.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.02387</guid>
      <pubDate>Tue, 03 Jun 2025 02:57:38 +0000</pubDate>
    </item>
    <item>
      <title>OThink-R1: Intrinsic Fast/Slow Thinking Mode Switching for   Over-Reasoning Mitigation</title>
      <link>https://arxiv.org/abs/2506.02397</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.02397.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shengjia Zhang, Junjie Wu, Jiawei Chen, Changwang Zhang, Xingyu Lou, Wangchunshu Zhou, Sheng Zhou, Can Wang, Jun Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advanced large reasoning models (LRMs) leverage extended chain-of-thought (CoT) reasoning to solve complex tasks, achieving state-of-the-art performance. Despite their success, we identify a critical issue: a substantial portion of simple tasks solved by LRMs can also be addressed by non-reasoning LLMs using significantly fewer tokens, indicating the complex reasoning may not always be necessary. To address this, we systematically analyze the reasoning trajectories of LRMs and present a method utilizing identified paradigms and LLM-Judge to classify these trajectories as either Redundant Reasoning or Essential Reasoning. And we introduce OThink-R1, a method that prunes redundant reasoning steps while preserving logical validity. OThink-R1 dynamically employs the non-thinking mode (fast-thinking) for straightforward problems while engaging in deliberate thinking (slow-thinking) for complex problems. Experiments across mathematical and question-answering tasks demonstrate that OThink-R1 reduces reasoning redundancy by almost 23\% on average without compromising accuracy, offering practical guidelines for efficient reasoning models. The code is available at https://github.com/AgenticIR-Lab/OThink-R1.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.02397</guid>
      <pubDate>Tue, 03 Jun 2025 03:31:30 +0000</pubDate>
    </item>
    <item>
      <title>OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for   Vision Language Models</title>
      <link>https://arxiv.org/abs/2506.03135</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.03135.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Mengdi Jia, Zekun Qi, Shaochen Zhang, Wenyao Zhang, Xinqiang Yu, Jiawei He, He Wang, Li Yi&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 37&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Spatial reasoning is a key aspect of cognitive psychology and remains a major bottleneck for current vision-language models (VLMs). While extensive research has aimed to evaluate or improve VLMs' understanding of basic spatial relations, such as distinguishing left from right, near from far, and object counting, these tasks represent only the most fundamental level of spatial reasoning. In this work, we introduce OmniSpatial, a comprehensive and challenging benchmark for spatial reasoning, grounded in cognitive psychology. OmniSpatial covers four major categories: dynamic reasoning, complex spatial logic, spatial interaction, and perspective-taking, with 50 fine-grained subcategories. Through Internet data crawling and careful manual annotation, we construct over 1.5K question-answer pairs. Extensive experiments show that both open- and closed-source VLMs, as well as existing reasoning and spatial understanding models, exhibit significant limitations in comprehensive spatial understanding. We further analyze failure cases and propose potential directions for future research.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.03135</guid>
      <pubDate>Tue, 03 Jun 2025 17:58:29 +0000</pubDate>
    </item>
    <item>
      <title>GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents</title>
      <link>https://arxiv.org/abs/2506.03143</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.03143.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Qianhui Wu, Kanzhi Cheng, Rui Yang, Chaoyun Zhang, Jianwei Yang, Huiqiang Jiang, Jian Mu, Baolin Peng, Bo Qiao, Reuben Tan, Si Qin, Lars Liden, Qingwei Lin, Huan Zhang, Tong Zhang, Jianbing Zhang, Dongmei Zhang, Jianfeng Gao&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 48&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; One of the principal challenges in building VLM-powered GUI agents is visual grounding, i.e., localizing the appropriate screen region for action execution based on both the visual content and the textual plans. Most existing work formulates this as a text-based coordinate generation task. However, these approaches suffer from several limitations: weak spatial-semantic alignment, inability to handle ambiguous supervision targets, and a mismatch between the dense nature of screen coordinates and the coarse, patch-level granularity of visual features extracted by models like Vision Transformers. In this paper, we propose GUI-Actor, a VLM-based method for coordinate-free GUI grounding. At its core, GUI-Actor introduces an attention-based action head that learns to align a dedicated &lt;ACTOR&gt; token with all relevant visual patch tokens, enabling the model to propose one or more action regions in a single forward pass. In line with this, we further design a grounding verifier to evaluate and select the most plausible action region from the candidates proposed for action execution. Extensive experiments show that GUI-Actor outperforms prior state-of-the-art methods on multiple GUI action grounding benchmarks, with improved generalization to unseen screen resolutions and layouts. Notably, GUI-Actor-7B even surpasses UI-TARS-72B (38.1) on ScreenSpot-Pro, achieving scores of 40.7 with Qwen2-VL and 44.6 with Qwen2.5-VL as backbones. Furthermore, by incorporating the verifier, we find that fine-tuning only the newly introduced action head (~100M parameters for 7B model) while keeping the VLM backbone frozen is sufficient to achieve performance comparable to previous state-of-the-art models, highlighting that GUI-Actor can endow the underlying VLM with effective grounding capabilities without compromising its general-purpose strengths.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.03143</guid>
      <pubDate>Tue, 03 Jun 2025 17:59:08 +0000</pubDate>
    </item>
    <item>
      <title>UniWorld: High-Resolution Semantic Encoders for Unified Visual   Understanding and Generation</title>
      <link>https://arxiv.org/abs/2506.03147</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.03147.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Bin Lin, Zongjian Li, Xinhua Cheng, Yuwei Niu, Yang Ye, Xianyi He, Shenghai Yuan, Wangbo Yu, Shaodong Wang, Yunyang Ge, Yatian Pang, Li Yuan&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 58&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Although existing unified models deliver strong performance on vision-language understanding and text-to-image generation, their models are limited in exploring image perception and manipulation tasks, which are urgently desired by users for wide applications. Recently, OpenAI released their powerful GPT-4o-Image model for comprehensive image perception and manipulation, achieving expressive capability and attracting community interests. By observing the performance of GPT-4o-Image in our carefully constructed experiments, we infer that GPT-4o-Image leverages features extracted by semantic encoders instead of VAE, while VAEs are considered essential components in many image manipulation models. Motivated by such inspiring observations, we present a unified generative framework named UniWorld based on semantic features provided by powerful visual-language models and contrastive semantic encoders. As a result, we build a strong unified model using only 1% amount of BAGEL's data, which consistently outperforms BAGEL on image editing benchmarks. UniWorld also maintains competitive image understanding and generation capabilities, achieving strong performance across multiple image perception tasks. We fully open-source our models, including model weights, training and evaluation scripts, and datasets.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.03147</guid>
      <pubDate>Tue, 03 Jun 2025 17:59:33 +0000</pubDate>
    </item>
    <item>
      <title>MiMo-VL Technical Report</title>
      <link>https://arxiv.org/abs/2506.03569</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.03569.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Xiaomi LLM-Core Team, Zihao Yue, Zhenru Lin, Yifan Song, Weikun Wang, Shuhuai Ren, Shuhao Gu, Shicheng Li, Peidian Li, Liang Zhao, Lei Li, Kainan Bao, Hao Tian, Hailin Zhang, Gang Wang, Dawei Zhu, Cici, Chenhong He, Bowen Ye, Bowen Shen, Zihan Zhang, Zihan Jiang, Zhixian Zheng, Zhichao Song, Zhenbo Luo, Yue Yu, Yudong Wang, Yuanyuan Tian, Yu Tu, Yihan Yan, Yi Huang, Xu Wang, Xinzhe Xu, Xingchen Song, Xing Zhang, Xing Yong, Xin Zhang, Xiangwei Deng, Wenyu Yang, Wenhan Ma, Weiwei Lv, Weiji Zhuang, Wei Liu, Sirui Deng, Shuo Liu, Shimao Chen, Shihua Yu, Shaohui Liu, Shande Wang, Rui Ma, Qiantong Wang, Peng Wang, Nuo Chen, Menghang Zhu, Kangyang Zhou, Kang Zhou, Kai Fang, Jun Shi, Jinhao Dong, Jiebao Xiao, Jiaming Xu, Huaqiu Liu, Hongshen Xu, Heng Qu, Haochen Zhao, Hanglong Lv, Guoan Wang, Duo Zhang, Dong Zhang, Di Zhang, Chong Ma, Chang Liu, Can Cai, Bingquan Xia&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 73&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We open-source MiMo-VL-7B-SFT and MiMo-VL-7B-RL, two powerful vision-language models delivering state-of-the-art performance in both general visual understanding and multimodal reasoning. MiMo-VL-7B-RL outperforms Qwen2.5-VL-7B on 35 out of 40 evaluated tasks, and scores 59.4 on OlympiadBench, surpassing models with up to 78B parameters. For GUI grounding applications, it sets a new standard with 56.1 on OSWorld-G, even outperforming specialized models such as UI-TARS. Our training combines four-stage pre-training (2.4 trillion tokens) with Mixed On-policy Reinforcement Learning (MORL) integrating diverse reward signals. We identify the importance of incorporating high-quality reasoning data with long Chain-of-Thought into pre-training stages, and the benefits of mixed RL despite challenges in simultaneous multi-domain optimization. We also contribute a comprehensive evaluation suite covering 50+ tasks to promote reproducibility and advance the field. The model checkpoints and full evaluation suite are available at https://github.com/XiaomiMiMo/MiMo-VL.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.03569</guid>
      <pubDate>Wed, 04 Jun 2025 04:32:54 +0000</pubDate>
    </item>
    <item>
      <title>AmbiK: Dataset of Ambiguous Tasks in Kitchen Environment</title>
      <link>https://arxiv.org/abs/2506.04089</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.04089.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Anastasiia Ivanova, Eva Bakaeva, Zoya Volovikova, Alexey K. Kovalev, Aleksandr I. Panov&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 46&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; As a part of an embodied agent, Large Language Models (LLMs) are typically used for behavior planning given natural language instructions from the user. However, dealing with ambiguous instructions in real-world environments remains a challenge for LLMs. Various methods for task ambiguity detection have been proposed. However, it is difficult to compare them because they are tested on different datasets and there is no universal benchmark. For this reason, we propose AmbiK (Ambiguous Tasks in Kitchen Environment), the fully textual dataset of ambiguous instructions addressed to a robot in a kitchen environment. AmbiK was collected with the assistance of LLMs and is human-validated. It comprises 1000 pairs of ambiguous tasks and their unambiguous counterparts, categorized by ambiguity type (Human Preferences, Common Sense Knowledge, Safety), with environment descriptions, clarifying questions and answers, user intents, and task plans, for a total of 2000 tasks. We hope that AmbiK will enable researchers to perform a unified comparison of ambiguity detection methods. AmbiK is available at https://github.com/cog-model/AmbiK-dataset.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.04089</guid>
      <pubDate>Wed, 04 Jun 2025 15:47:07 +0000</pubDate>
    </item>
    <item>
      <title>OpenThoughts: Data Recipes for Reasoning Models</title>
      <link>https://arxiv.org/abs/2506.04178</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.04178.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang, Shreyas Pimpalgaonkar, Kartik Sharma, Charlie Cheng-Jie Ji, Yichuan Deng, Sarah Pratt, Vivek Ramanujan, Jon Saad-Falcon, Jeffrey Li, Achal Dave, Alon Albalak, Kushal Arora, Blake Wulfe, Chinmay Hegde, Greg Durrett, Sewoong Oh, Mohit Bansal, Saadia Gabriel, Aditya Grover, Kai-Wei Chang, Vaishaal Shankar, Aaron Gokaslan, Mike A. Merrill, Tatsunori Hashimoto, Yejin Choi, Jenia Jitsev, Reinhard Heckel, Maheswaran Sathiamoorthy, Alexandros G. Dimakis, Ludwig Schmidt&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 41&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Reasoning models have made rapid progress on many benchmarks involving math, code, and science. Yet, there are still many open questions about the best training recipes for reasoning since state-of-the-art models often rely on proprietary datasets with little to no public information available. To address this, the goal of the OpenThoughts project is to create open-source datasets for training reasoning models. After initial explorations, our OpenThoughts2-1M dataset led to OpenThinker2-32B, the first model trained on public reasoning data to match DeepSeek-R1-Distill-32B on standard reasoning benchmarks such as AIME and LiveCodeBench. We then improve our dataset further by systematically investigating each step of our data generation pipeline with 1,000+ controlled experiments, which led to OpenThoughts3. Scaling the pipeline to 1.2M examples and using QwQ-32B as teacher yields our OpenThinker3-7B model, which achieves state-of-the-art results: 53% on AIME 2025, 51% on LiveCodeBench 06/24-01/25, and 54% on GPQA Diamond. All of our datasets and models are available on https://openthoughts.ai.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.04178</guid>
      <pubDate>Wed, 04 Jun 2025 17:25:39 +0000</pubDate>
    </item>
    <item>
      <title>SuperWriter: Reflection-Driven Long-Form Generation with Large Language   Models</title>
      <link>https://arxiv.org/abs/2506.04180</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.04180.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuhao Wu, Yushi Bai, Zhiqiang Hu, Juanzi Li, Roy Ka-Wei Lee&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 32&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Long-form text generation remains a significant challenge for large language models (LLMs), particularly in maintaining coherence, ensuring logical consistency, and preserving text quality as sequence length increases. To address these limitations, we propose SuperWriter-Agent, an agent-based framework designed to enhance the quality and consistency of long-form text generation. SuperWriter-Agent introduces explicit structured thinking-through planning and refinement stages into the generation pipeline, guiding the model to follow a more deliberate and cognitively grounded process akin to that of a professional writer. Based on this framework, we construct a supervised fine-tuning dataset to train a 7B SuperWriter-LM. We further develop a hierarchical Direct Preference Optimization (DPO) procedure that uses Monte Carlo Tree Search (MCTS) to propagate final quality assessments and optimize each generation step accordingly. Empirical results across diverse benchmarks demonstrate that SuperWriter-LM achieves state-of-the-art performance, surpassing even larger-scale baseline models in both automatic evaluation and human evaluation. Furthermore, comprehensive ablation studies demonstrate the effectiveness of hierarchical DPO and underscore the value of incorporating structured thinking steps to improve the quality of long-form text generation.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.04180</guid>
      <pubDate>Wed, 04 Jun 2025 17:27:42 +0000</pubDate>
    </item>
    <item>
      <title>Advancing Multimodal Reasoning: From Optimized Cold Start to Staged   Reinforcement Learning</title>
      <link>https://arxiv.org/abs/2506.04207</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.04207.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shuang Chen, Yue Guo, Zhaochen Su, Yafu Li, Yulun Wu, Jiacheng Chen, Jiayu Chen, Weijie Wang, Xiaoye Qu, Yu Cheng&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 45&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Inspired by the remarkable reasoning capabilities of Deepseek-R1 in complex textual tasks, many works attempt to incentivize similar capabilities in Multimodal Large Language Models (MLLMs) by directly applying reinforcement learning (RL). However, they still struggle to activate complex reasoning. In this paper, rather than examining multimodal RL in isolation, we delve into current training pipelines and identify three crucial phenomena: 1) Effective cold start initialization is critical for enhancing MLLM reasoning. Intriguingly, we find that initializing with carefully selected text data alone can lead to performance surpassing many recent multimodal reasoning models, even before multimodal RL. 2) Standard GRPO applied to multimodal RL suffers from gradient stagnation, which degrades training stability and performance. 3) Subsequent text-only RL training, following the multimodal RL phase, further enhances multimodal reasoning. This staged training approach effectively balances perceptual grounding and cognitive reasoning development. By incorporating the above insights and addressing multimodal RL issues, we introduce ReVisual-R1, achieving a new state-of-the-art among open-source 7B MLLMs on challenging benchmarks including MathVerse, MathVision, WeMath, LogicVista, DynaMath, and challenging AIME2024 and AIME2025.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.04207</guid>
      <pubDate>Wed, 04 Jun 2025 17:51:08 +0000</pubDate>
    </item>
    <item>
      <title>RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language   Models for Robotics</title>
      <link>https://arxiv.org/abs/2506.04308</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.04308.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Enshen Zhou, Jingkun An, Cheng Chi, Yi Han, Shanyu Rong, Chi Zhang, Pengwei Wang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, Shanghang Zhang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 40&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Spatial referring is a fundamental capability of embodied robots to interact with the 3D physical world. However, even with the powerful pretrained vision language models (VLMs), recent approaches are still not qualified to accurately understand the complex 3D scenes and dynamically reason about the instruction-indicated locations for interaction. To this end, we propose RoboRefer, a 3D-aware VLM that can first achieve precise spatial understanding by integrating a disentangled but dedicated depth encoder via supervised fine-tuning (SFT). Moreover, RoboRefer advances generalized multi-step spatial reasoning via reinforcement fine-tuning (RFT), with metric-sensitive process reward functions tailored for spatial referring tasks. To support SFT and RFT training, we introduce RefSpatial, a large-scale dataset of 20M QA pairs (2x prior), covering 31 spatial relations (vs. 15 prior) and supporting complex reasoning processes (up to 5 steps). In addition, we introduce RefSpatial-Bench, a challenging benchmark filling the gap in evaluating spatial referring with multi-step reasoning. Experiments show that SFT-trained RoboRefer achieves state-of-the-art spatial understanding, with an average success rate of 89.6%. RFT-trained RoboRefer further outperforms all other baselines by a large margin, even surpassing Gemini-2.5-Pro by 17.4% in average accuracy on RefSpatial-Bench. Notably, RoboRefer can be integrated with various control policies to execute long-horizon, dynamic tasks across diverse robots (e,g., UR5, G1 humanoid) in cluttered real-world scenes.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.04308</guid>
      <pubDate>Wed, 04 Jun 2025 17:59:27 +0000</pubDate>
    </item>
    <item>
      <title>Truth in the Few: High-Value Data Selection for Efficient Multi-Modal   Reasoning</title>
      <link>https://arxiv.org/abs/2506.04755</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.04755.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shenshen Li, Kaiyuan Deng, Lei Wang, Hao Yang, Chong Peng, Peng Yan, Fumin Shen, Heng Tao Shen, Xing Xu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; While multi-modal large language models (MLLMs) have made significant progress in complex reasoning tasks via reinforcement learning, it is commonly believed that extensive training data is necessary for improving multi-modal reasoning ability, inevitably leading to data redundancy and substantial computational costs. However, can smaller high-value datasets match or outperform full corpora for multi-modal reasoning in MLLMs? In this work, we challenge this assumption through a key observation: meaningful multi-modal reasoning is triggered by only a sparse subset of training samples, termed cognitive samples, whereas the majority contribute marginally. Building on this insight, we propose a novel data selection paradigm termed Reasoning Activation Potential (RAP), which identifies cognitive samples by estimating each sample's potential to stimulate genuine multi-modal reasoning by two complementary estimators: 1) Causal Discrepancy Estimator (CDE) based on the potential outcome model principle, eliminates samples that overly rely on language priors by comparing outputs between multi-modal and text-only inputs; 2) Attention Confidence Estimator (ACE), which exploits token-level self-attention to discard samples dominated by irrelevant but over-emphasized tokens in intermediate reasoning stages. Moreover, we introduce a Difficulty-aware Replacement Module (DRM) to substitute trivial instances with cognitively challenging ones, thereby ensuring complexity for robust multi-modal reasoning. Experiments on six datasets show that our RAP method consistently achieves superior performance using only 9.3% of the training data, while reducing computational costs by over 43%. Our code is available at https://github.com/Leo-ssl/RAP.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.04755</guid>
      <pubDate>Thu, 05 Jun 2025 08:40:24 +0000</pubDate>
    </item>
    <item>
      <title>ComfyUI-Copilot: An Intelligent Assistant for Automated Workflow   Development</title>
      <link>https://arxiv.org/abs/2506.05010</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05010.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zhenran Xu, Xue Yang, Yiyu Wang, Qingli Hu, Zijiao Wu, Longyue Wang, Weihua Luo, Kaifu Zhang, Baotian Hu, Min Zhang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 70&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce ComfyUI-Copilot, a large language model-powered plugin designed to enhance the usability and efficiency of ComfyUI, an open-source platform for AI-driven art creation. Despite its flexibility and user-friendly interface, ComfyUI can present challenges to newcomers, including limited documentation, model misconfigurations, and the complexity of workflow design. ComfyUI-Copilot addresses these challenges by offering intelligent node and model recommendations, along with automated one-click workflow construction. At its core, the system employs a hierarchical multi-agent framework comprising a central assistant agent for task delegation and specialized worker agents for different usages, supported by our curated ComfyUI knowledge bases to streamline debugging and deployment. We validate the effectiveness of ComfyUI-Copilot through both offline quantitative evaluations and online user feedback, showing that it accurately recommends nodes and accelerates workflow development. Additionally, use cases illustrate that ComfyUI-Copilot lowers entry barriers for beginners and enhances workflow efficiency for experienced users. The ComfyUI-Copilot installation package and a demo video are available at https://github.com/AIDC-AI/ComfyUI-Copilot.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05010</guid>
      <pubDate>Thu, 05 Jun 2025 13:20:50 +0000</pubDate>
    </item>
    <item>
      <title>Qwen3 Embedding: Advancing Text Embedding and Reranking Through   Foundation Models</title>
      <link>https://arxiv.org/abs/2506.05176</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05176.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, Jingren Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 62&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; In this work, we introduce the Qwen3 Embedding series, a significant advancement over its predecessor, the GTE-Qwen series, in text embedding and reranking capabilities, built upon the Qwen3 foundation models. Leveraging the Qwen3 LLMs' robust capabilities in multilingual text understanding and generation, our innovative multi-stage training pipeline combines large-scale unsupervised pre-training with supervised fine-tuning on high-quality datasets. Effective model merging strategies further ensure the robustness and adaptability of the Qwen3 Embedding series. During the training process, the Qwen3 LLMs serve not only as backbone models but also play a crucial role in synthesizing high-quality, rich, and diverse training data across multiple domains and languages, thus enhancing the training pipeline. The Qwen3 Embedding series offers a spectrum of model sizes (0.6B, 4B, 8B) for both embedding and reranking tasks, addressing diverse deployment scenarios where users can optimize for either efficiency or effectiveness. Empirical evaluations demonstrate that the Qwen3 Embedding series achieves state-of-the-art results across diverse benchmarks. Notably, it excels on the multilingual evaluation benchmark MTEB for text embedding, as well as in various retrieval tasks, including code retrieval, cross-lingual retrieval and multilingual retrieval. To facilitate reproducibility and promote community-driven research and development, the Qwen3 Embedding models are publicly available under the Apache 2.0 license.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05176</guid>
      <pubDate>Thu, 05 Jun 2025 15:49:48 +0000</pubDate>
    </item>
    <item>
      <title>The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly   Licensed Text</title>
      <link>https://arxiv.org/abs/2506.05209</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05209.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Nikhil Kandpal, Brian Lester, Colin Raffel, Sebastian Majstorovic, Stella Biderman, Baber Abbasi, Luca Soldaini, Enrico Shippole, A. Feder Cooper, Aviya Skowron, John Kirchenbauer, Shayne Longpre, Lintang Sutawika, Alon Albalak, Zhenlin Xu, Guilherme Penedo, Loubna Ben Allal, Elie Bakouch, John David Pressman, Honglu Fan, Dashiell Stander, Guangyu Song, Aaron Gokaslan, Tom Goldstein, Brian R. Bartoldson, Bhavya Kailkhura, Tyler Murray&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 42&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large language models (LLMs) are typically trained on enormous quantities of unlicensed text, a practice that has led to scrutiny due to possible intellectual property infringement and ethical concerns. Training LLMs on openly licensed text presents a first step towards addressing these issues, but prior data collection efforts have yielded datasets too small or low-quality to produce performant LLMs. To address this gap, we collect, curate, and release the Common Pile v0.1, an eight terabyte collection of openly licensed text designed for LLM pretraining. The Common Pile comprises content from 30 sources that span diverse domains including research papers, code, books, encyclopedias, educational materials, audio transcripts, and more. Crucially, we validate our efforts by training two 7 billion parameter LLMs on text from the Common Pile: Comma v0.1-1T and Comma v0.1-2T, trained on 1 and 2 trillion tokens respectively. Both models attain competitive performance to LLMs trained on unlicensed text with similar computational budgets, such as Llama 1 and 2 7B. In addition to releasing the Common Pile v0.1 itself, we also release the code used in its creation as well as the training mixture and checkpoints for the Comma v0.1 models.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05209</guid>
      <pubDate>Thu, 05 Jun 2025 16:21:30 +0000</pubDate>
    </item>
    <item>
      <title>Diagonal Batching Unlocks Parallelism in Recurrent Memory Transformers   for Long Contexts</title>
      <link>https://arxiv.org/abs/2506.05229</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05229.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Danil Sivtsov, Ivan Rodkin, Gleb Kuzmin, Yuri Kuratov, Ivan Oseledets&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 37&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Transformer models struggle with long-context inference due to their quadratic time and linear memory complexity. Recurrent Memory Transformers (RMTs) offer a solution by reducing the asymptotic cost to linear time and constant memory usage. However, their memory update mechanism leads to sequential execution, causing a performance bottleneck.   We introduce Diagonal Batching, a scheduling scheme that unlocks parallelism across segments in RMTs while preserving exact recurrence. This approach eliminates the sequential constraint, enabling efficient GPU inference even for single long-context inputs without complex batching and pipelining techniques. Because the technique is purely a run-time computation reordering, existing RMT models adopt it with no retraining.   Applied to a LLaMA-1B ARMT model, Diagonal Batching yields a 3.3x speedup over standard full-attention LLaMA-1B and a 1.8x speedup over the sequential RMT implementation on 131,072-token sequences. By removing sequential bottleneck, Diagonal Batching reduces inference cost and latency, thereby strengthening RMTs as a practical solution for real-world, long-context applications.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05229</guid>
      <pubDate>Thu, 05 Jun 2025 16:43:48 +0000</pubDate>
    </item>
    <item>
      <title>Video World Models with Long-term Spatial Memory</title>
      <link>https://arxiv.org/abs/2506.05284</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05284.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Tong Wu, Shuai Yang, Ryan Po, Yinghao Xu, Ziwei Liu, Dahua Lin, Gordon Wetzstein&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 53&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Emerging world models autoregressively generate video frames in response to actions, such as camera movements and text prompts, among other control signals. Due to limited temporal context window sizes, these models often struggle to maintain scene consistency during revisits, leading to severe forgetting of previously generated environments. Inspired by the mechanisms of human memory, we introduce a novel framework to enhancing long-term consistency of video world models through a geometry-grounded long-term spatial memory. Our framework includes mechanisms to store and retrieve information from the long-term spatial memory and we curate custom datasets to train and evaluate world models with explicitly stored 3D memory mechanisms. Our evaluations show improved quality, consistency, and context length compared to relevant baselines, paving the way towards long-term consistent world generation.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05284</guid>
      <pubDate>Thu, 05 Jun 2025 17:42:34 +0000</pubDate>
    </item>
    <item>
      <title>SeedVR2: One-Step Video Restoration via Diffusion Adversarial   Post-Training</title>
      <link>https://arxiv.org/abs/2506.05301</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05301.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jianyi Wang, Shanchuan Lin, Zhijie Lin, Yuxi Ren, Meng Wei, Zongsheng Yue, Shangchen Zhou, Hao Chen, Yang Zhao, Ceyuan Yang, Xuefeng Xiao, Chen Change Loy, Lu Jiang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 55&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in diffusion-based video restoration (VR) demonstrate significant improvement in visual quality, yet yield a prohibitive computational cost during inference. While several distillation-based approaches have exhibited the potential of one-step image restoration, extending existing approaches to VR remains challenging and underexplored, particularly when dealing with high-resolution video in real-world settings. In this work, we propose a one-step diffusion-based VR model, termed as SeedVR2, which performs adversarial VR training against real data. To handle the challenging high-resolution VR within a single step, we introduce several enhancements to both model architecture and training procedures. Specifically, an adaptive window attention mechanism is proposed, where the window size is dynamically adjusted to fit the output resolutions, avoiding window inconsistency observed under high-resolution VR using window attention with a predefined window size. To stabilize and improve the adversarial post-training towards VR, we further verify the effectiveness of a series of losses, including a proposed feature matching loss without significantly sacrificing training efficiency. Extensive experiments show that SeedVR2 can achieve comparable or even better performance compared with existing VR approaches in a single step.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05301</guid>
      <pubDate>Thu, 05 Jun 2025 17:51:05 +0000</pubDate>
    </item>
    <item>
      <title>MORSE-500: A Programmatically Controllable Video Benchmark to   Stress-Test Multimodal Reasoning</title>
      <link>https://arxiv.org/abs/2506.05523</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05523.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zikui Cai, Andrew Wang, Anirudh Satheesh, Ankit Nakhawa, Hyunwoo Jae, Keenan Powell, Minghui Liu, Neel Jay, Sungbin Oh, Xiyao Wang, Yongyuan Liang, Tom Goldstein, Furong Huang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 33&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Despite rapid advances in vision-language models (VLMs), current benchmarks for multimodal reasoning fall short in three key dimensions. First, they overwhelmingly rely on static images, failing to capture the temporal complexity of real-world environments. Second, they narrowly focus on mathematical problem-solving, neglecting the broader spectrum of reasoning skills -- including abstract, physical, planning, spatial, and temporal capabilities -- required for robust multimodal intelligence. Third, many benchmarks quickly saturate, offering limited headroom for diagnosing failure modes or measuring continued progress. We introduce MORSE-500 (Multimodal Reasoning Stress-test Environment), a video benchmark composed of 500 fully scripted clips with embedded questions spanning six complementary reasoning categories. Each instance is programmatically generated using deterministic Python scripts (via Manim, Matplotlib, MoviePy), generative video models, and curated real footage. This script-driven design allows fine-grained control over visual complexity, distractor density, and temporal dynamics -- enabling difficulty to be scaled systematically as models improve. Unlike static benchmarks that become obsolete once saturated, MORSE-500 is built to evolve: its controllable generation pipeline supports the creation of arbitrarily challenging new instances, making it ideally suited for stress-testing next-generation models. Initial experiments with state-of-the-art systems -- including various Gemini 2.5 Pro and OpenAI o3 which represent the strongest available at the time, alongside strong open-source models -- reveal substantial performance gaps across all categories, with particularly large deficits in abstract and planning tasks. We release the full dataset, generation scripts, and evaluation harness to support transparent, reproducible, and forward-looking multimodal reasoning research.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05523</guid>
      <pubDate>Thu, 05 Jun 2025 19:12:45 +0000</pubDate>
    </item>
    <item>
      <title>Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models</title>
      <link>https://arxiv.org/abs/2506.06395</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06395.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Pengyi Li, Matvey Skripkin, Alexander Zubrey, Andrey Kuznetsov, Ivan Oseledets&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 124&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large language models (LLMs) excel at reasoning, yet post-training remains critical for aligning their behavior with task goals. Existing reinforcement learning (RL) methods often depend on costly human annotations or external reward models. We propose Reinforcement Learning via Self-Confidence (RLSC), which uses the model's own confidence as reward signals-eliminating the need for labels, preference models, or reward engineering. Applied to Qwen2.5-Math-7B with only 16 samples per question and 10 or 20 training steps, RLSC improves accuracy by +13.4% on AIME2024, +21.2% on MATH500, +21.7% on Minerva Math, +20.8% on Olympiadbench, and +9.7% on AMC23. RLSC provides a simple, scalable post-training method for inference models, requiring only a small number of samples and unlabelled supervision.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06395</guid>
      <pubDate>Thu, 05 Jun 2025 19:55:15 +0000</pubDate>
    </item>
    <item>
      <title>PartCrafter: Structured 3D Mesh Generation via Compositional Latent   Diffusion Transformers</title>
      <link>https://arxiv.org/abs/2506.05573</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05573.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuchen Lin, Chenguo Lin, Panwang Pan, Honglei Yan, Yiqiang Feng, Yadong Mu, Katerina Fragkiadaki&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 69&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce PartCrafter, the first structured 3D generative model that jointly synthesizes multiple semantically meaningful and geometrically distinct 3D meshes from a single RGB image. Unlike existing methods that either produce monolithic 3D shapes or follow two-stage pipelines, i.e., first segmenting an image and then reconstructing each segment, PartCrafter adopts a unified, compositional generation architecture that does not rely on pre-segmented inputs. Conditioned on a single image, it simultaneously denoises multiple 3D parts, enabling end-to-end part-aware generation of both individual objects and complex multi-object scenes. PartCrafter builds upon a pretrained 3D mesh diffusion transformer (DiT) trained on whole objects, inheriting the pretrained weights, encoder, and decoder, and introduces two key innovations: (1) A compositional latent space, where each 3D part is represented by a set of disentangled latent tokens; (2) A hierarchical attention mechanism that enables structured information flow both within individual parts and across all parts, ensuring global coherence while preserving part-level detail during generation. To support part-level supervision, we curate a new dataset by mining part-level annotations from large-scale 3D object datasets. Experiments show that PartCrafter outperforms existing approaches in generating decomposable 3D meshes, including parts that are not directly visible in input images, demonstrating the strength of part-aware generative priors for 3D understanding and synthesis. Code and training data will be released.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05573</guid>
      <pubDate>Thu, 05 Jun 2025 20:30:28 +0000</pubDate>
    </item>
    <item>
      <title>Leveraging Self-Attention for Input-Dependent Soft Prompting in LLMs</title>
      <link>https://arxiv.org/abs/2506.05629</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05629.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Ananth Muppidi, Abhilash Nandy, Sambaran Bandyopadhyay&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 34&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; The performance of large language models in domain-specific tasks necessitates fine-tuning, which is computationally expensive and technically challenging. This paper focuses on parameter-efficient fine-tuning using soft prompting, a promising approach that adapts pre-trained models to downstream tasks by learning a small set of parameters. We propose a novel Input Dependent Soft Prompting technique with a self-Attention Mechanism (ID-SPAM) that generates soft prompts based on the input tokens and attends different tokens with varying importance. Our method is simple and efficient, keeping the number of trainable parameters small. We show the merits of the proposed approach compared to state-of-the-art techniques on various tasks and show the improved zero shot domain transfer capability.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.05629</guid>
      <pubDate>Thu, 05 Jun 2025 23:13:22 +0000</pubDate>
    </item>
    <item>
      <title>Saffron-1: Towards an Inference Scaling Paradigm for LLM Safety   Assurance</title>
      <link>https://arxiv.org/abs/2506.06444</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06444.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Ruizhong Qiu, Gaotang Li, Tianxin Wei, Jingrui He, Hanghang Tong&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 72&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Existing safety assurance research has primarily focused on training-phase alignment to instill safe behaviors into LLMs. However, recent studies have exposed these methods' susceptibility to diverse jailbreak attacks. Concurrently, inference scaling has significantly advanced LLM reasoning capabilities but remains unexplored in the context of safety assurance. Addressing this gap, our work pioneers inference scaling for robust and effective LLM safety against emerging threats. We reveal that conventional inference scaling techniques, despite their success in reasoning tasks, perform poorly in safety contexts, even falling short of basic approaches like Best-of-N Sampling. We attribute this inefficiency to a newly identified challenge, the exploration--efficiency dilemma, arising from the high computational overhead associated with frequent process reward model (PRM) evaluations. To overcome this dilemma, we propose SAFFRON, a novel inference scaling paradigm tailored explicitly for safety assurance. Central to our approach is the introduction of a multifurcation reward model (MRM) that significantly reduces the required number of reward model evaluations. To operationalize this paradigm, we further propose: (i) a partial supervision training objective for MRM, (ii) a conservative exploration constraint to prevent out-of-distribution explorations, and (iii) a Trie-based key--value caching strategy that facilitates cache sharing across sequences during tree search. Extensive experiments validate the effectiveness of our method. Additionally, we publicly release our trained multifurcation reward model (Saffron-1) and the accompanying token-level safety reward dataset (Safety4M) to accelerate future research in LLM safety. Our code, model, and data are publicly available at https://github.com/q-rz/saffron , and our project homepage is at https://q-rz.github.io/p/saffron .&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06444</guid>
      <pubDate>Fri, 06 Jun 2025 18:05:45 +0000</pubDate>
    </item>
    <item>
      <title>Geopolitical biases in LLMs: what are the "good" and the "bad" countries   according to contemporary language models</title>
      <link>https://arxiv.org/abs/2506.06751</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.06751.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Mikhail Salnikov, Dmitrii Korzh, Ivan Lazichny, Elvir Karimov, Artyom Iudin, Ivan Oseledets, Oleg Y. Rogov, Alexander Panchenko, Natalia Loukachevitch, Elena Tutubalina&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 72&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; This paper evaluates geopolitical biases in LLMs with respect to various countries though an analysis of their interpretation of historical events with conflicting national perspectives (USA, UK, USSR, and China). We introduce a novel dataset with neutral event descriptions and contrasting viewpoints from different countries. Our findings show significant geopolitical biases, with models favoring specific national narratives. Additionally, simple debiasing prompts had a limited effect in reducing these biases. Experiments with manipulated participant labels reveal models' sensitivity to attribution, sometimes amplifying biases or recognizing inconsistencies, especially with swapped labels. This work highlights national narrative biases in LLMs, challenges the effectiveness of simple debiasing methods, and offers a framework and dataset for future geopolitical bias research.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.06751</guid>
      <pubDate>Sat, 07 Jun 2025 10:45:17 +0000</pubDate>
    </item>
    <item>
      <title>Lingshu: A Generalist Foundation Model for Unified Multimodal Medical   Understanding and Reasoning</title>
      <link>https://arxiv.org/abs/2506.07044</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07044.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; LASA Team, Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang, Chenghao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, Yu Sun, Junao Shen, Chaojun Wang, Jie Tan, Deli Zhao, Tingyang Xu, Hao Zhang, Yu Rong&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 105&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in understanding common visual elements, largely due to their large-scale datasets and advanced training strategies. However, their effectiveness in medical applications remains limited due to the inherent discrepancies between data and tasks in medical scenarios and those in the general domain. Concretely, existing medical MLLMs face the following critical limitations: (1) limited coverage of medical knowledge beyond imaging, (2) heightened susceptibility to hallucinations due to suboptimal data curation processes, (3) lack of reasoning capabilities tailored for complex medical scenarios. To address these challenges, we first propose a comprehensive data curation procedure that (1) efficiently acquires rich medical knowledge data not only from medical imaging but also from extensive medical texts and general-domain data; and (2) synthesizes accurate medical captions, visual question answering (VQA), and reasoning samples. As a result, we build a multimodal dataset enriched with extensive medical knowledge. Building on the curated data, we introduce our medical-specialized MLLM: Lingshu. Lingshu undergoes multi-stage training to embed medical expertise and enhance its task-solving capabilities progressively. Besides, we preliminarily explore the potential of applying reinforcement learning with verifiable rewards paradigm to enhance Lingshu's medical reasoning ability. Additionally, we develop MedEvalKit, a unified evaluation framework that consolidates leading multimodal and textual medical benchmarks for standardized, fair, and efficient model assessment. We evaluate the performance of Lingshu on three fundamental medical tasks, multimodal QA, text-based QA, and medical report generation. The results show that Lingshu consistently outperforms the existing open-source multimodal models on most tasks ...&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07044</guid>
      <pubDate>Sun, 08 Jun 2025 08:47:30 +0000</pubDate>
    </item>
    <item>
      <title>SpatialLM: Training Large Language Models for Structured Indoor Modeling</title>
      <link>https://arxiv.org/abs/2506.07491</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07491.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yongsen Mao, Junhao Zhong, Chuan Fang, Jia Zheng, Rui Tang, Hao Zhu, Ping Tan, Zihan Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 38&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; SpatialLM is a large language model designed to process 3D point cloud data and generate structured 3D scene understanding outputs. These outputs include architectural elements like walls, doors, windows, and oriented object boxes with their semantic categories. Unlike previous methods which exploit task-specific network designs, our model adheres to the standard multimodal LLM architecture and is fine-tuned directly from open-source LLMs.   To train SpatialLM, we collect a large-scale, high-quality synthetic dataset consisting of the point clouds of 12,328 indoor scenes (54,778 rooms) with ground-truth 3D annotations, and conduct a careful study on various modeling and training decisions. On public benchmarks, our model gives state-of-the-art performance in layout estimation and competitive results in 3D object detection. With that, we show a feasible path for enhancing the spatial understanding capabilities of modern LLMs for applications in augmented reality, embodied robotics, and more.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07491</guid>
      <pubDate>Mon, 09 Jun 2025 07:10:58 +0000</pubDate>
    </item>
    <item>
      <title>MiniCPM4: Ultra-Efficient LLMs on End Devices</title>
      <link>https://arxiv.org/abs/2506.07900</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07900.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; MiniCPM Team, Chaojun Xiao, Yuxuan Li, Xu Han, Yuzhuo Bai, Jie Cai, Haotian Chen, Wentong Chen, Xin Cong, Ganqu Cui, Ning Ding, Shengdan Fan, Yewei Fang, Zixuan Fu, Wenyu Guan, Yitong Guan, Junshao Guo, Yufeng Han, Bingxiang He, Yuxiang Huang, Cunliang Kong, Qiuzuo Li, Siyuan Li, Wenhao Li, Yanghao Li, Yishan Li, Zhen Li, Dan Liu, Biyuan Lin, Yankai Lin, Xiang Long, Quanyu Lu, Yaxi Lu, Peiyan Luo, Hongya Lyu, Litu Ou, Yinxu Pan, Zekai Qu, Qundong Shi, Zijun Song, Jiayuan Su, Zhou Su, Ao Sun, Xianghui Sun, Peijun Tang, Fangzheng Wang, Feng Wang, Shuo Wang, Yudong Wang, Yesai Wu, Zhenyu Xiao, Jie Xie, Zihao Xie, Yukun Yan, Jiarui Yuan, Kaihuo Zhang, Lei Zhang, Linyue Zhang, Xueren Zhang, Yudi Zhang, Hengyu Zhao, Weilin Zhao, Weilun Zhao, Yuanqian Zhao, Zhi Zheng, Ge Zhou, Jie Zhou, Wei Zhou, Zihan Zhou, Zixuan Zhou, Zhiyuan Liu, Guoyang Zeng, Chao Jia, Dahai Li, Maosong Sun&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 81&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; This paper introduces MiniCPM4, a highly efficient large language model (LLM) designed explicitly for end-side devices. We achieve this efficiency through systematic innovation in four key dimensions: model architecture, training data, training algorithms, and inference systems. Specifically, in terms of model architecture, we propose InfLLM v2, a trainable sparse attention mechanism that accelerates both prefilling and decoding phases for long-context processing. Regarding training data, we propose UltraClean, an efficient and accurate pre-training data filtering and generation strategy, and UltraChat v2, a comprehensive supervised fine-tuning dataset. These datasets enable satisfactory model performance to be achieved using just 8 trillion training tokens. Regarding training algorithms, we propose ModelTunnel v2 for efficient pre-training strategy search, and improve existing post-training methods by introducing chunk-wise rollout for load-balanced reinforcement learning and data-efficient tenary LLM, BitCPM. Regarding inference systems, we propose CPM.cu that integrates sparse attention, model quantization, and speculative sampling to achieve efficient prefilling and decoding. To meet diverse on-device requirements, MiniCPM4 is available in two versions, with 0.5B and 8B parameters, respectively. Sufficient evaluation results show that MiniCPM4 outperforms open-source models of similar size across multiple benchmarks, highlighting both its efficiency and effectiveness. Notably, MiniCPM4-8B demonstrates significant speed improvements over Qwen3-8B when processing long sequences. Through further adaptation, MiniCPM4 successfully powers diverse applications, including trustworthy survey generation and tool use with model context protocol, clearly showcasing its broad usability.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07900</guid>
      <pubDate>Mon, 09 Jun 2025 16:16:50 +0000</pubDate>
    </item>
    <item>
      <title>OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation</title>
      <link>https://arxiv.org/abs/2506.07977</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.07977.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jingjing Chang, Yixiao Fang, Peng Xing, Shuhan Wu, Wei Cheng, Rui Wang, Xianfang Zeng, Gang Yu, Hai-Bao Chen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 39&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Text-to-image (T2I) models have garnered significant attention for generating high-quality images aligned with text prompts. However, rapid T2I model advancements reveal limitations in early benchmarks, lacking comprehensive evaluations, for example, the evaluation on reasoning, text rendering and style. Notably, recent state-of-the-art models, with their rich knowledge modeling capabilities, show promising results on the image generation problems requiring strong reasoning ability, yet existing evaluation systems have not adequately addressed this frontier. To systematically address these gaps, we introduce OneIG-Bench, a meticulously designed comprehensive benchmark framework for fine-grained evaluation of T2I models across multiple dimensions, including prompt-image alignment, text rendering precision, reasoning-generated content, stylization, and diversity. By structuring the evaluation, this benchmark enables in-depth analysis of model performance, helping researchers and practitioners pinpoint strengths and bottlenecks in the full pipeline of image generation. Specifically, OneIG-Bench enables flexible evaluation by allowing users to focus on a particular evaluation subset. Instead of generating images for the entire set of prompts, users can generate images only for the prompts associated with the selected dimension and complete the corresponding evaluation accordingly. Our codebase and dataset are now publicly available to facilitate reproducible evaluation studies and cross-model comparisons within the T2I research community.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.07977</guid>
      <pubDate>Mon, 09 Jun 2025 17:50:21 +0000</pubDate>
    </item>
    <item>
      <title>Reinforcement Pre-Training</title>
      <link>https://arxiv.org/abs/2506.08007</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08007.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Qingxiu Dong, Li Dong, Yao Tang, Tianzhu Ye, Yutao Sun, Zhifang Sui, Furu Wei&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 236&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling paradigm for large language models and reinforcement learning (RL). Specifically, we reframe next-token prediction as a reasoning task trained using RL, where it receives verifiable rewards for correctly predicting the next token for a given context. RPT offers a scalable method to leverage vast amounts of text data for general-purpose RL, rather than relying on domain-specific annotated answers. By incentivizing the capability of next-token reasoning, RPT significantly improves the language modeling accuracy of predicting the next tokens. Moreover, RPT provides a strong pre-trained foundation for further reinforcement fine-tuning. The scaling curves show that increased training compute consistently improves the next-token prediction accuracy. The results position RPT as an effective and promising scaling paradigm to advance language model pre-training.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08007</guid>
      <pubDate>Mon, 09 Jun 2025 17:59:53 +0000</pubDate>
    </item>
    <item>
      <title>Wait, We Don't Need to "Wait"! Removing Thinking Tokens Improves   Reasoning Efficiency</title>
      <link>https://arxiv.org/abs/2506.08343</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.08343.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Chenlong Wang, Yuanning Feng, Dongping Chen, Zhaoyang Chu, Ranjay Krishna, Tianyi Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 46&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in large reasoning models have enabled complex, step-by-step reasoning but often introduce significant overthinking, resulting in verbose and redundant outputs that hinder efficiency. In this study, we examine whether explicit self-reflection, signaled by tokens such as "Wait" and "Hmm", is necessary for advanced reasoning. We propose NoWait, a simple yet effective approach that disables explicit self-reflection by suppressing these tokens during inference. Extensive experiments on ten benchmarks across textual, visual, and video reasoning tasks show that NoWait reduces chain-of-thought trajectory length by up to 27%-51% in five R1-style model series, without compromising model utility. NoWait thus offers a plug-and-play solution for efficient and utility-preserving multimodal reasoning.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.08343</guid>
      <pubDate>Tue, 10 Jun 2025 01:54:04 +0000</pubDate>
    </item>
    <item>
      <title>Seedance 1.0: Exploring the Boundaries of Video Generation Models</title>
      <link>https://arxiv.org/abs/2506.09113</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09113.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yu Gao, Haoyuan Guo, Tuyen Hoang, Weilin Huang, Lu Jiang, Fangyuan Kong, Huixia Li, Jiashi Li, Liang Li, Xiaojie Li, Xunsong Li, Yifu Li, Shanchuan Lin, Zhijie Lin, Jiawei Liu, Shu Liu, Xiaonan Nie, Zhiwu Qing, Yuxi Ren, Li Sun, Zhi Tian, Rui Wang, Sen Wang, Guoqiang Wei, Guohong Wu, Jie Wu, Ruiqi Xia, Fei Xiao, Xuefeng Xiao, Jiangqiao Yan, Ceyuan Yang, Jianchao Yang, Runkai Yang, Tao Yang, Yihang Yang, Zilyu Ye, Xuejiao Zeng, Yan Zeng, Heng Zhang, Yang Zhao, Xiaozheng Zheng, Peihao Zhu, Jiaxin Zou, Feilong Zuo&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 90&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Notable breakthroughs in diffusion modeling have propelled rapid improvements in video generation, yet current foundational model still face critical challenges in simultaneously balancing prompt following, motion plausibility, and visual quality. In this report, we introduce Seedance 1.0, a high-performance and inference-efficient video foundation generation model that integrates several core technical improvements: (i) multi-source data curation augmented with precision and meaningful video captioning, enabling comprehensive learning across diverse scenarios; (ii) an efficient architecture design with proposed training paradigm, which allows for natively supporting multi-shot generation and jointly learning of both text-to-video and image-to-video tasks. (iii) carefully-optimized post-training approaches leveraging fine-grained supervised fine-tuning, and video-specific RLHF with multi-dimensional reward mechanisms for comprehensive performance improvements; (iv) excellent model acceleration achieving ~10x inference speedup through multi-stage distillation strategies and system-level optimizations. Seedance 1.0 can generate a 5-second video at 1080p resolution only with 41.4 seconds (NVIDIA-L20). Compared to state-of-the-art video generation models, Seedance 1.0 stands out with high-quality and fast video generation having superior spatiotemporal fluidity with structural stability, precise instruction adherence in complex multi-subject contexts, native multi-shot narrative coherence with consistent subject representation.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09113</guid>
      <pubDate>Tue, 10 Jun 2025 17:56:11 +0000</pubDate>
    </item>
    <item>
      <title>Autoregressive Semantic Visual Reconstruction Helps VLMs Understand   Better</title>
      <link>https://arxiv.org/abs/2506.09040</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09040.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Dianyi Wang, Wei Song, Yikun Wang, Siyuan Wang, Kaicheng Yu, Zhongyu Wei, Jiaqi Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 35&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Typical large vision-language models (LVLMs) apply autoregressive supervision solely to textual sequences, without fully incorporating the visual modality into the learning process. This results in three key limitations: (1) an inability to utilize images without accompanying captions, (2) the risk that captions omit critical visual details, and (3) the challenge that certain vision-centric content cannot be adequately conveyed through text. As a result, current LVLMs often prioritize vision-to-language alignment while potentially overlooking fine-grained visual information. While some prior works have explored autoregressive image generation, effectively leveraging autoregressive visual supervision to enhance image understanding remains an open challenge. In this paper, we introduce Autoregressive Semantic Visual Reconstruction (ASVR), which enables joint learning of visual and textual modalities within a unified autoregressive framework. We show that autoregressively reconstructing the raw visual appearance of images does not enhance and may even impair multimodal understanding. In contrast, autoregressively reconstructing the semantic representation of images consistently improves comprehension. Notably, we find that even when models are given continuous image features as input, they can effectively reconstruct discrete semantic tokens, resulting in stable and consistent improvements across a wide range of multimodal understanding benchmarks. Our approach delivers significant performance gains across varying data scales (556k-2M) and types of LLM bacbones. Specifically, ASVR improves LLaVA-1.5 by 5% in average scores across 14 multimodal benchmarks. The code is available at https://github.com/AlenjandroWang/ASVR.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09040</guid>
      <pubDate>Tue, 10 Jun 2025 17:57:50 +0000</pubDate>
    </item>
    <item>
      <title>VIKI-R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement   Learning</title>
      <link>https://arxiv.org/abs/2506.09049</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09049.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Li Kang, Xiufeng Song, Heng Zhou, Yiran Qin, Jie Yang, Xiaohong Liu, Philip Torr, Lei Bai, Zhenfei Yin&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 33&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Coordinating multiple embodied agents in dynamic environments remains a core challenge in artificial intelligence, requiring both perception-driven reasoning and scalable cooperation strategies. While recent works have leveraged large language models (LLMs) for multi-agent planning, a few have begun to explore vision-language models (VLMs) for visual reasoning. However, these VLM-based approaches remain limited in their support for diverse embodiment types. In this work, we introduce VIKI-Bench, the first hierarchical benchmark tailored for embodied multi-agent cooperation, featuring three structured levels: agent activation, task planning, and trajectory perception. VIKI-Bench includes diverse robot embodiments, multi-view visual observations, and structured supervision signals to evaluate reasoning grounded in visual inputs. To demonstrate the utility of VIKI-Bench, we propose VIKI-R, a two-stage framework that fine-tunes a pretrained vision-language model (VLM) using Chain-of-Thought annotated demonstrations, followed by reinforcement learning under multi-level reward signals. Our extensive experiments show that VIKI-R significantly outperforms baselines method across all task levels. Furthermore, we show that reinforcement learning enables the emergence of compositional cooperation patterns among heterogeneous agents. Together, VIKI-Bench and VIKI-R offer a unified testbed and method for advancing multi-agent, visual-driven cooperation in embodied AI systems.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09049</guid>
      <pubDate>Tue, 10 Jun 2025 17:59:44 +0000</pubDate>
    </item>
    <item>
      <title>Autoregressive Adversarial Post-Training for Real-Time Interactive Video   Generation</title>
      <link>https://arxiv.org/abs/2506.09350</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09350.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shanchuan Lin, Ceyuan Yang, Hao He, Jianwen Jiang, Yuxi Ren, Xin Xia, Yang Zhao, Xuefeng Xiao, Lu Jiang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 48&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Existing large-scale video generation models are computationally intensive, preventing adoption in real-time and interactive applications. In this work, we propose autoregressive adversarial post-training (AAPT) to transform a pre-trained latent video diffusion model into a real-time, interactive video generator. Our model autoregressively generates a latent frame at a time using a single neural function evaluation (1NFE). The model can stream the result to the user in real time and receive interactive responses as controls to generate the next latent frame. Unlike existing approaches, our method explores adversarial training as an effective paradigm for autoregressive generation. This not only allows us to design an architecture that is more efficient for one-step generation while fully utilizing the KV cache, but also enables training the model in a student-forcing manner that proves to be effective in reducing error accumulation during long video generation. Our experiments demonstrate that our 8B model achieves real-time, 24fps, streaming video generation at 736x416 resolution on a single H100, or 1280x720 on 8xH100 up to a minute long (1440 frames). Visit our research website at https://seaweed-apt.com/2&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09350</guid>
      <pubDate>Wed, 11 Jun 2025 03:04:23 +0000</pubDate>
    </item>
    <item>
      <title>ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical   Reasoning</title>
      <link>https://arxiv.org/abs/2506.09513</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09513.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yu Sun, Xingyu Qian, Weiwen Xu, Hao Zhang, Chenghao Xiao, Long Li, Yu Rong, Wenbing Huang, Qifeng Bai, Tingyang Xu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 93&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Though reasoning-based large language models (LLMs) have excelled in mathematics and programming, their capabilities in knowledge-intensive medical question answering remain underexplored. To address this, we introduce ReasonMed, the largest medical reasoning dataset, comprising 370k high-quality examples distilled from 1.7 million initial reasoning paths generated by various LLMs. ReasonMed is constructed through a multi-agent verification and refinement process, where we design an Error Refiner to enhance the reasoning paths by identifying and correcting error-prone steps flagged by a verifier. Leveraging ReasonMed, we systematically investigate best practices for training medical reasoning models and find that combining detailed Chain-of-Thought (CoT) reasoning with concise answer summaries yields the most effective fine-tuning strategy. Based on this strategy, we train ReasonMed-7B, which sets a new benchmark for sub-10B models, outperforming the prior best by 4.17\% and even exceeding LLaMA3.1-70B on PubMedQA by 4.60\%.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09513</guid>
      <pubDate>Wed, 11 Jun 2025 08:36:55 +0000</pubDate>
    </item>
    <item>
      <title>Effective Red-Teaming of Policy-Adherent Agents</title>
      <link>https://arxiv.org/abs/2506.09600</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09600.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Itay Nakash, George Kour, Koren Lazar, Matan Vetzler, Guy Uziel, Ateret Anaby-Tavor&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 37&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Task-oriented LLM-based agents are increasingly used in domains with strict policies, such as refund eligibility or cancellation rules. The challenge lies in ensuring that the agent consistently adheres to these rules and policies, appropriately refusing any request that would violate them, while still maintaining a helpful and natural interaction. This calls for the development of tailored design and evaluation methodologies to ensure agent resilience against malicious user behavior. We propose a novel threat model that focuses on adversarial users aiming to exploit policy-adherent agents for personal benefit. To address this, we present CRAFT, a multi-agent red-teaming system that leverages policy-aware persuasive strategies to undermine a policy-adherent agent in a customer-service scenario, outperforming conventional jailbreak methods such as DAN prompts, emotional manipulation, and coercive. Building upon the existing tau-bench benchmark, we introduce tau-break, a complementary benchmark designed to rigorously assess the agent's robustness against manipulative user behavior. Finally, we evaluate several straightforward yet effective defense strategies. While these measures provide some protection, they fall short, highlighting the need for stronger, research-driven safeguards to protect policy-adherent agents from adversarial attacks&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09600</guid>
      <pubDate>Wed, 11 Jun 2025 10:59:47 +0000</pubDate>
    </item>
    <item>
      <title>ComfyUI-R1: Exploring Reasoning Models for Workflow Generation</title>
      <link>https://arxiv.org/abs/2506.09790</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09790.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zhenran Xu, Yiyu Wang, Xue Yang, Longyue Wang, Weihua Luo, Kaifu Zhang, Baotian Hu, Min Zhang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 52&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; AI-generated content has evolved from monolithic models to modular workflows, particularly on platforms like ComfyUI, enabling customization in creative pipelines. However, crafting effective workflows requires great expertise to orchestrate numerous specialized components, presenting a steep learning curve for users. To address this challenge, we introduce ComfyUI-R1, the first large reasoning model for automated workflow generation. Starting with our curated dataset of 4K workflows, we construct long chain-of-thought (CoT) reasoning data, including node selection, workflow planning, and code-level workflow representation. ComfyUI-R1 is trained through a two-stage framework: (1) CoT fine-tuning for cold start, adapting models to the ComfyUI domain; (2) reinforcement learning for incentivizing reasoning capability, guided by a fine-grained rule-metric hybrid reward, ensuring format validity, structural integrity, and node-level fidelity. Experiments show that our 7B-parameter model achieves a 97\% format validity rate, along with high pass rate, node-level and graph-level F1 scores, significantly surpassing prior state-of-the-art methods that employ leading closed-source models such as GPT-4o and Claude series. Further analysis highlights the critical role of the reasoning process and the advantage of transforming workflows into code. Qualitative comparison reveals our strength in synthesizing intricate workflows with diverse nodes, underscoring the potential of long CoT reasoning in AI art creation.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09790</guid>
      <pubDate>Wed, 11 Jun 2025 14:35:15 +0000</pubDate>
    </item>
    <item>
      <title>Multiverse: Your Language Models Secretly Decide How to Parallelize and   Merge Generation</title>
      <link>https://arxiv.org/abs/2506.09991</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09991.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Xinyu Yang, Yuwei An, Hongyi Liu, Tianqi Chen, Beidi Chen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 56&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Autoregressive Large Language Models (AR-LLMs) frequently exhibit implicit parallelism in sequential generation. Inspired by this, we introduce Multiverse, a new generative model that enables natively parallel generation. Multiverse internalizes a MapReduce paradigm, generating automatically through three stages: (i) a Map stage for adaptive task decomposition, (ii) a Process stage for parallel subtask execution, and (iii) a Reduce stage for lossless result synthesis. Next, we build a real-world Multiverse reasoning model with co-design of data, algorithm, and system, enabling rapid and seamless transfer from frontier AR-LLMs. Starting from sequential reasoning chains, we create Multiverse 1K by converting them into structured training data using an automated LLM-assisted pipeline, avoiding costly human annotations. Algorithmically, we design Multiverse Attention to separate parallel reasoning steps while keeping compatibility with causal attention for efficient training. Systematically, we implement Multiverse Engine to enable parallel inference. It features a dedicated scheduler that dynamically switches between sequential and parallel generation, triggered directly by the model. After a 3-hour fine-tuning with 1K examples, our Multiverse-32B stands as the only open-sourced non-AR model achieving performance on par with leading AR-LLMs of the same scale, evidenced by AIME24 &amp; 25 scores of 54% and 46%, respectively. Moreover, our budget control experiments show that Multiverse-32B exhibits superior scaling, outperforming AR-LLMs by 1.87% on average using the same context length. Such scaling further leads to practical efficiency gain, achieving up to 2x speedup across varying batch sizes. We have open-sourced the entire Multiverse ecosystem, including data, model weights, engine, supporting tools, as well as complete data curation prompts and detailed training and evaluation recipes.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09991</guid>
      <pubDate>Wed, 11 Jun 2025 17:59:23 +0000</pubDate>
    </item>
    <item>
      <title>Text-Aware Image Restoration with Diffusion Models</title>
      <link>https://arxiv.org/abs/2506.09993</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09993.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jaewon Min, Jin Hyeon Kim, Paul Hyunbin Cho, Jaeeun Lee, Jihye Park, Minkyu Park, Sangpil Kim, Hyunhee Park, Seungryong Kim&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 40&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Image restoration aims to recover degraded images. However, existing diffusion-based restoration methods, despite great success in natural image restoration, often struggle to faithfully reconstruct textual regions in degraded images. Those methods frequently generate plausible but incorrect text-like patterns, a phenomenon we refer to as text-image hallucination. In this paper, we introduce Text-Aware Image Restoration (TAIR), a novel restoration task that requires the simultaneous recovery of visual contents and textual fidelity. To tackle this task, we present SA-Text, a large-scale benchmark of 100K high-quality scene images densely annotated with diverse and complex text instances. Furthermore, we propose a multi-task diffusion framework, called TeReDiff, that integrates internal features from diffusion models into a text-spotting module, enabling both components to benefit from joint training. This allows for the extraction of rich text representations, which are utilized as prompts in subsequent denoising steps. Extensive experiments demonstrate that our approach consistently outperforms state-of-the-art restoration methods, achieving significant gains in text recognition accuracy. See our project page: https://cvlab-kaist.github.io/TAIR/&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09993</guid>
      <pubDate>Wed, 11 Jun 2025 17:59:46 +0000</pubDate>
    </item>
    <item>
      <title>PlayerOne: Egocentric World Simulator</title>
      <link>https://arxiv.org/abs/2506.09995</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.09995.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuanpeng Tu, Hao Luo, Xi Chen, Xiang Bai, Fan Wang, Hengshuang Zhao&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 33&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce PlayerOne, the first egocentric realistic world simulator, facilitating immersive and unrestricted exploration within vividly dynamic environments. Given an egocentric scene image from the user, PlayerOne can accurately construct the corresponding world and generate egocentric videos that are strictly aligned with the real scene human motion of the user captured by an exocentric camera. PlayerOne is trained in a coarse-to-fine pipeline that first performs pretraining on large-scale egocentric text-video pairs for coarse-level egocentric understanding, followed by finetuning on synchronous motion-video data extracted from egocentric-exocentric video datasets with our automatic construction pipeline. Besides, considering the varying importance of different components, we design a part-disentangled motion injection scheme, enabling precise control of part-level movements. In addition, we devise a joint reconstruction framework that progressively models both the 4D scene and video frames, ensuring scene consistency in the long-form video generation. Experimental results demonstrate its great generalization ability in precise control of varying human movements and worldconsistent modeling of diverse scenarios. It marks the first endeavor into egocentric real-world simulation and can pave the way for the community to delve into fresh frontiers of world modeling and its diverse applications.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.09995</guid>
      <pubDate>Wed, 11 Jun 2025 17:59:53 +0000</pubDate>
    </item>
    <item>
      <title>Scientists' First Exam: Probing Cognitive Abilities of MLLM via   Perception, Understanding, and Reasoning</title>
      <link>https://arxiv.org/abs/2506.10521</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10521.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuhao Zhou, Yiheng Wang, Xuming He, Ruoyao Xiao, Zhiwei Li, Qiantai Feng, Zijie Guo, Yuejin Yang, Hao Wu, Wenxuan Huang, Jiaqi Wei, Dan Si, Xiuqi Yao, Jia Bu, Haiwen Huang, Tianfan Fu, Shixiang Tang, Ben Fei, Dongzhan Zhou, Fenghua Ling, Yan Lu, Siqi Sun, Chenhui Li, Guanjie Zheng, Jiancheng Lv, Wenlong Zhang, Lei Bai&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 65&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Scientific discoveries increasingly rely on complex multimodal reasoning based on information-intensive scientific data and domain-specific expertise. Empowered by expert-level scientific benchmarks, scientific Multimodal Large Language Models (MLLMs) hold the potential to significantly enhance this discovery process in realistic workflows. However, current scientific benchmarks mostly focus on evaluating the knowledge understanding capabilities of MLLMs, leading to an inadequate assessment of their perception and reasoning abilities. To address this gap, we present the Scientists' First Exam (SFE) benchmark, designed to evaluate the scientific cognitive capacities of MLLMs through three interconnected levels: scientific signal perception, scientific attribute understanding, scientific comparative reasoning. Specifically, SFE comprises 830 expert-verified VQA pairs across three question types, spanning 66 multimodal tasks across five high-value disciplines. Extensive experiments reveal that current state-of-the-art GPT-o3 and InternVL-3 achieve only 34.08% and 26.52% on SFE, highlighting significant room for MLLMs to improve in scientific realms. We hope the insights obtained in SFE will facilitate further developments in AI-enhanced scientific discoveries.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10521</guid>
      <pubDate>Thu, 12 Jun 2025 09:29:16 +0000</pubDate>
    </item>
    <item>
      <title>AniMaker: Automated Multi-Agent Animated Storytelling with MCTS-Driven   Clip Generation</title>
      <link>https://arxiv.org/abs/2506.10540</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10540.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Haoyuan Shi, Yunxin Li, Xinyu Chen, Longyue Wang, Baotian Hu, Min Zhang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 37&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Despite rapid advancements in video generation models, generating coherent storytelling videos that span multiple scenes and characters remains challenging. Current methods often rigidly convert pre-generated keyframes into fixed-length clips, resulting in disjointed narratives and pacing issues. Furthermore, the inherent instability of video generation models means that even a single low-quality clip can significantly degrade the entire output animation's logical coherence and visual continuity. To overcome these obstacles, we introduce AniMaker, a multi-agent framework enabling efficient multi-candidate clip generation and storytelling-aware clip selection, thus creating globally consistent and story-coherent animation solely from text input. The framework is structured around specialized agents, including the Director Agent for storyboard generation, the Photography Agent for video clip generation, the Reviewer Agent for evaluation, and the Post-Production Agent for editing and voiceover. Central to AniMaker's approach are two key technical components: MCTS-Gen in Photography Agent, an efficient Monte Carlo Tree Search (MCTS)-inspired strategy that intelligently navigates the candidate space to generate high-potential clips while optimizing resource usage; and AniEval in Reviewer Agent, the first framework specifically designed for multi-shot animation evaluation, which assesses critical aspects such as story-level consistency, action completion, and animation-specific features by considering each clip in the context of its preceding and succeeding clips. Experiments demonstrate that AniMaker achieves superior quality as measured by popular metrics including VBench and our proposed AniEval framework, while significantly improving the efficiency of multi-candidate generation, pushing AI-generated storytelling animation closer to production standards.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10540</guid>
      <pubDate>Thu, 12 Jun 2025 10:06:21 +0000</pubDate>
    </item>
    <item>
      <title>The Diffusion Duality</title>
      <link>https://arxiv.org/abs/2506.10892</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10892.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin Chiu, Volodymyr Kuleshov&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Uniform-state discrete diffusion models hold the promise of fast text generation due to their inherent ability to self-correct. However, they are typically outperformed by autoregressive models and masked diffusion models. In this work, we narrow this performance gap by leveraging a key insight: Uniform-state diffusion processes naturally emerge from an underlying Gaussian diffusion. Our method, Duo, transfers powerful techniques from Gaussian diffusion to improve both training and sampling. First, we introduce a curriculum learning strategy guided by the Gaussian process, doubling training speed by reducing variance. Models trained with curriculum learning surpass autoregressive models in zero-shot perplexity on 3 of 7 benchmarks. Second, we present Discrete Consistency Distillation, which adapts consistency distillation from the continuous to the discrete setting. This algorithm unlocks few-step generation in diffusion language models by accelerating sampling by two orders of magnitude. We provide the code and model checkpoints on the project page: http://s-sahoo.github.io/duo&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10892</guid>
      <pubDate>Thu, 12 Jun 2025 16:55:35 +0000</pubDate>
    </item>
    <item>
      <title>Magistral</title>
      <link>https://arxiv.org/abs/2506.10910</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10910.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Mistral-AI, Abhinav Rastogi, Albert Q. Jiang, Andy Lo, Gabrielle Berrada, Guillaume Lample, Jason Rute, Joep Barmentlo, Karmesh Yadav, Kartik Khandelwal, Khyathi Raghavi Chandu, Léonard Blier, Lucile Saulnier, Matthieu Dinot, Maxime Darrin, Neha Gupta, Roman Soletskyi, Sagar Vaze, Teven Le Scao, Yihan Wang, Adam Yang, Alexander H. Liu, Alexandre Sablayrolles, Amélie Héliou, Amélie Martin, Andy Ehrenberg, Anmol Agarwal, Antoine Roux, Arthur Darcet, Arthur Mensch, Baptiste Bout, Baptiste Rozière, Baudouin De Monicault, Chris Bamford, Christian Wallenwein, Christophe Renaudin, Clémence Lanfranchi, Darius Dabert, Devon Mizelle, Diego de las Casas, Elliot Chane-Sane, Emilien Fugier, Emma Bou Hanna, Gauthier Delerce, Gauthier Guinet, Georgii Novikov, Guillaume Martin, Himanshu Jaju, Jan Ludziejewski, Jean-Hadrien Chabran, Jean-Malo Delignon, Joachim Studnia, Jonas Amar, Josselin Somerville Roberts, Julien Denize, Karan Saxena, Kush Jain, Lingxiao Zhao, Louis Martin, Luyu Gao, Lélio Renard Lavaud, Marie Pellat, Mathilde Guillaumin, Mathis Felardos, Maximilian Augustin, Mickaël Seznec, Nikhil Raghuraman, Olivier Duchenne, Patricia Wang, Patrick von Platen, Patryk Saffer, Paul Jacob, Paul Wambergue, Paula Kurylowicz, Pavankumar Reddy Muddireddy, Philomène Chagniot, Pierre Stock, Pravesh Agrawal, Romain Sauvestre, Rémi Delacourt, Sanchit Gandhi, Sandeep Subramanian, Shashwat Dalal, Siddharth Gandhi, Soham Ghosh, Srijan Mishra, Sumukh Aithal, Szymon Antoniak, Thibault Schueller, Thibaut Lavril, Thomas Robert, Thomas Wang, Timothée Lacroix, Valeriia Nemychnikova, Victor Paltz, Virgile Richard, Wen-Ding Li, William Marshall, Xuanyu Zhang, Yunhao Tang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 60&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce Magistral, Mistral's first reasoning model and our own scalable reinforcement learning (RL) pipeline. Instead of relying on existing implementations and RL traces distilled from prior models, we follow a ground up approach, relying solely on our own models and infrastructure. Notably, we demonstrate a stack that enabled us to explore the limits of pure RL training of LLMs, present a simple method to force the reasoning language of the model, and show that RL on text data alone maintains most of the initial checkpoint's capabilities. We find that RL on text maintains or improves multimodal understanding, instruction following and function calling. We present Magistral Medium, trained for reasoning on top of Mistral Medium 3 with RL alone, and we open-source Magistral Small (Apache 2.0) which further includes cold-start data from Magistral Medium.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10910</guid>
      <pubDate>Thu, 12 Jun 2025 17:22:37 +0000</pubDate>
    </item>
    <item>
      <title>SWE-Factory: Your Automated Factory for Issue Resolution Training Data   and Evaluation Benchmarks</title>
      <link>https://arxiv.org/abs/2506.10954</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.10954.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Lianghong Guo, Yanlin Wang, Caihua Li, Pengyu Yang, Jiachi Chen, Wei Tao, Yingtian Zou, Duyu Tang, Zibin Zheng&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 51&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Constructing large-scale datasets for the GitHub issue resolution task is crucial for both training and evaluating the software engineering capabilities of Large Language Models (LLMs). However, the traditional process for creating such benchmarks is notoriously challenging and labor-intensive, particularly in the stages of setting up evaluation environments, grading test outcomes, and validating task instances. In this paper, we propose SWE-Factory, an automated pipeline designed to address these challenges. To tackle these issues, our pipeline integrates three core automated components. First, we introduce SWE-Builder, a multi-agent system that automates evaluation environment construction, which employs four specialized agents that work in a collaborative, iterative loop and leverages an environment memory pool to enhance efficiency. Second, we introduce a standardized, exit-code-based grading method that eliminates the need for manually writing custom parsers. Finally, we automate the fail2pass validation process using these reliable exit code signals. Experiments on 671 issues across four programming languages show that our pipeline can effectively construct valid task instances; for example, with GPT-4.1-mini, our SWE-Builder constructs 269 valid instances at 0.045 per instance, while with Gemini-2.5-flash, it achieves comparable performance at the lowest cost of 0.024 per instance. We also demonstrate that our exit-code-based grading achieves 100% accuracy compared to manual inspection, and our automated fail2pass validation reaches a precision of 0.92 and a recall of 1.00. We hope our automated pipeline will accelerate the collection of large-scale, high-quality GitHub issue resolution datasets for both training and evaluation. Our code and datasets are released at https://github.com/DeepSoftwareAnalytics/swe-factory.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.10954</guid>
      <pubDate>Thu, 12 Jun 2025 17:54:17 +0000</pubDate>
    </item>
    <item>
      <title>DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents</title>
      <link>https://arxiv.org/abs/2506.11763</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.11763.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Mingxuan Du, Benfeng Xu, Chiwei Zhu, Xiaorui Wang, Zhendong Mao&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 60&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Deep Research Agents are a prominent category of LLM-based agents. By autonomously orchestrating multistep web exploration, targeted retrieval, and higher-order synthesis, they transform vast amounts of online information into analyst-grade, citation-rich reports--compressing hours of manual desk research into minutes. However, a comprehensive benchmark for systematically evaluating the capabilities of these agents remains absent. To bridge this gap, we present DeepResearch Bench, a benchmark consisting of 100 PhD-level research tasks, each meticulously crafted by domain experts across 22 distinct fields. Evaluating DRAs is inherently complex and labor-intensive. We therefore propose two novel methodologies that achieve strong alignment with human judgment. The first is a reference-based method with adaptive criteria to assess the quality of generated research reports. The other framework is introduced to evaluate DRA's information retrieval and collection capabilities by assessing its effective citation count and overall citation accuracy. We have open-sourced DeepResearch Bench and key components of these frameworks at https://github.com/Ayanami0730/deep_research_bench to accelerate the development of practical LLM-based agents.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.11763</guid>
      <pubDate>Fri, 13 Jun 2025 13:17:32 +0000</pubDate>
    </item>
    <item>
      <title>Aligned Novel View Image and Geometry Synthesis via Cross-modal   Attention Instillation</title>
      <link>https://arxiv.org/abs/2506.11924</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.11924.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Min-Seop Kwak, Junho Kim, Sangdoo Yun, Dongyoon Han, Taekyoung Kim, Seungryong Kim, Jin-Hwa Kim&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 32&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce a diffusion-based framework that performs aligned novel view image and geometry generation via a warping-and-inpainting methodology. Unlike prior methods that require dense posed images or pose-embedded generative models limited to in-domain views, our method leverages off-the-shelf geometry predictors to predict partial geometries viewed from reference images, and formulates novel-view synthesis as an inpainting task for both image and geometry. To ensure accurate alignment between generated images and geometry, we propose cross-modal attention distillation, where attention maps from the image diffusion branch are injected into a parallel geometry diffusion branch during both training and inference. This multi-task approach achieves synergistic effects, facilitating geometrically robust image synthesis as well as well-defined geometry prediction. We further introduce proximity-based mesh conditioning to integrate depth and normal cues, interpolating between point cloud and filtering erroneously predicted geometry from influencing the generation process. Empirically, our method achieves high-fidelity extrapolative view synthesis on both image and geometry across a range of unseen scenes, delivers competitive reconstruction quality under interpolation settings, and produces geometrically aligned colored point clouds for comprehensive 3D completion. Project page is available at https://cvlab-kaist.github.io/MoAI.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.11924</guid>
      <pubDate>Fri, 13 Jun 2025 16:19:00 +0000</pubDate>
    </item>
    <item>
      <title>Feedback Friction: LLMs Struggle to Fully Incorporate External Feedback</title>
      <link>https://arxiv.org/abs/2506.11930</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.11930.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Dongwei Jiang, Alvin Zhang, Andrew Wang, Nicholas Andrews, Daniel Khashabi&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 53&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent studies have shown LLMs possess some ability to improve their responses when given external feedback. However, it remains unclear how effectively and thoroughly these models can incorporate extrinsic feedback. In an ideal scenario, if LLMs receive near-perfect and complete feedback, we would expect them to fully integrate the feedback and change their incorrect answers to correct ones. In this paper, we systematically investigate LLMs' ability to incorporate feedback by designing a controlled experimental environment. For each problem, a solver model attempts a solution, then a feedback generator with access to near-complete ground-truth answers produces targeted feedback, after which the solver tries again. We evaluate this pipeline across a diverse range of tasks, including math reasoning, knowledge reasoning, scientific reasoning, and general multi-domain evaluations with state-of-the-art language models including Claude 3.7 (with and without extended thinking). Surprisingly, even under these near-ideal conditions, solver models consistently show resistance to feedback, a limitation that we term FEEDBACK FRICTION. To mitigate this limitation, we experiment with sampling-based strategies like progressive temperature increases and explicit rejection of previously attempted incorrect answers, which yield improvements but still fail to help models achieve target performance. We also perform a rigorous exploration of potential causes of FEEDBACK FRICTION, ruling out factors such as model overconfidence and data familiarity. We hope that highlighting this issue in LLMs and ruling out several apparent causes will help future research in self-improvement.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.11930</guid>
      <pubDate>Fri, 13 Jun 2025 16:31:51 +0000</pubDate>
    </item>
    <item>
      <title>CMI-Bench: A Comprehensive Benchmark for Evaluating Music Instruction   Following</title>
      <link>https://arxiv.org/abs/2506.12285</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.12285.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yinghao Ma, Siyou Li, Juntao Yu, Emmanouil Benetos, Akira Maezawa&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 53&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in audio-text large language models (LLMs) have opened new possibilities for music understanding and generation. However, existing benchmarks are limited in scope, often relying on simplified tasks or multi-choice evaluations that fail to reflect the complexity of real-world music analysis. We reinterpret a broad range of traditional MIR annotations as instruction-following formats and introduce CMI-Bench, a comprehensive music instruction following benchmark designed to evaluate audio-text LLMs on a diverse set of music information retrieval (MIR) tasks. These include genre classification, emotion regression, emotion tagging, instrument classification, pitch estimation, key detection, lyrics transcription, melody extraction, vocal technique recognition, instrument performance technique detection, music tagging, music captioning, and (down)beat tracking: reflecting core challenges in MIR research. Unlike previous benchmarks, CMI-Bench adopts standardized evaluation metrics consistent with previous state-of-the-art MIR models, ensuring direct comparability with supervised approaches. We provide an evaluation toolkit supporting all open-source audio-textual LLMs, including LTU, Qwen-audio, SALMONN, MusiLingo, etc. Experiment results reveal significant performance gaps between LLMs and supervised models, along with their culture, chronological and gender bias, highlighting the potential and limitations of current models in addressing MIR tasks. CMI-Bench establishes a unified foundation for evaluating music instruction following, driving progress in music-aware LLMs.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.12285</guid>
      <pubDate>Sat, 14 Jun 2025 00:18:44 +0000</pubDate>
    </item>
    <item>
      <title>Scaling Test-time Compute for LLM Agents</title>
      <link>https://arxiv.org/abs/2506.12928</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.12928.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; King Zhu, Hanhao Li, Siwei Wu, Tianshun Xing, Dehua Ma, Xiangru Tang, Minghao Liu, Jian Yang, Jiaheng Liu, Yuchen Eleanor Jiang, Changwang Zhang, Chenghua Lin, Jun Wang, Ge Zhang, Wangchunshu Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 59&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Scaling test time compute has shown remarkable success in improving the reasoning abilities of large language models (LLMs). In this work, we conduct the first systematic exploration of applying test-time scaling methods to language agents and investigate the extent to which it improves their effectiveness. Specifically, we explore different test-time scaling strategies, including: (1) parallel sampling algorithms; (2) sequential revision strategies; (3) verifiers and merging methods; (4)strategies for diversifying rollouts.We carefully analyze and ablate the impact of different design strategies on applying test-time scaling on language agents, and have follow findings: 1. Scaling test time compute could improve the performance of agents. 2. Knowing when to reflect is important for agents. 3. Among different verification and result merging approaches, the list-wise method performs best. 4. Increasing diversified rollouts exerts a positive effect on the agent's task performance.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.12928</guid>
      <pubDate>Sun, 15 Jun 2025 17:59:47 +0000</pubDate>
    </item>
    <item>
      <title>MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning   Attention</title>
      <link>https://arxiv.org/abs/2506.13585</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.13585.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; MiniMax, Aili Chen, Aonian Li, Bangwei Gong, Binyang Jiang, Bo Fei, Bo Yang, Boji Shan, Changqing Yu, Chao Wang, Cheng Zhu, Chengjun Xiao, Chengyu Du, Chi Zhang, Chu Qiao, Chunhao Zhang, Chunhui Du, Congchao Guo, Da Chen, Deming Ding, Dianjun Sun, Dong Li, Enwei Jiao, Haigang Zhou, Haimo Zhang, Han Ding, Haohai Sun, Haoyu Feng, Huaiguang Cai, Haichao Zhu, Jian Sun, Jiaqi Zhuang, Jiaren Cai, Jiayuan Song, Jin Zhu, Jingyang Li, Jinhao Tian, Jinli Liu, Junhao Xu, Junjie Yan, Junteng Liu, Junxian He, Kaiyi Feng, Ke Yang, Kecheng Xiao, Le Han, Leyang Wang, Lianfei Yu, Liheng Feng, Lin Li, Lin Zheng, Linge Du, Lingyu Yang, Lunbin Zeng, Minghui Yu, Mingliang Tao, Mingyuan Chi, Mozhi Zhang, Mujie Lin, Nan Hu, Nongyu Di, Peng Gao, Pengfei Li, Pengyu Zhao, Qibing Ren, Qidi Xu, Qile Li, Qin Wang, Rong Tian, Ruitao Leng, Shaoxiang Chen, Shaoyu Chen, Shengmin Shi, Shitong Weng, Shuchang Guan, Shuqi Yu, Sichen Li, Songquan Zhu, Tengfei Li, Tianchi Cai, Tianrun Liang, Weiyu Cheng, Weize Kong, Wenkai Li, Xiancai Chen, Xiangjun Song, Xiao Luo, Xiao Su, Xiaobo Li, Xiaodong Han, Xinzhu Hou, Xuan Lu, Xun Zou, Xuyang Shen, Yan Gong, Yan Ma, Yang Wang, Yiqi Shi, Yiran Zhong, Yonghong Duan, Yongxiang Fu, Yongyi Hu, Yu Gao, Yuanxiang Fan, Yufeng Yang, Yuhao Li, Yulin Hu, Yunan Huang, Yunji Li, Yunzhi Xu, Yuxin Mao, Yuxuan Shi, Yuze Wenren, Zehan Li, Zelin Li, Zhanxu Tian, Zhengmao Zhu, Zhenhua Fan, Zhenzhen Wu, Zhichao Xu, Zhihang Yu, Zhiheng Lyu, Zhuo Jiang, Zibo Gao, Zijia Wu, Zijian Song, Zijun Sun&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 248&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce MiniMax-M1, the world's first open-weight, large-scale hybrid-attention reasoning model. MiniMax-M1 is powered by a hybrid Mixture-of-Experts (MoE) architecture combined with a lightning attention mechanism. The model is developed based on our previous MiniMax-Text-01 model, which contains a total of 456 billion parameters with 45.9 billion parameters activated per token. The M1 model natively supports a context length of 1 million tokens, 8x the context size of DeepSeek R1. Furthermore, the lightning attention mechanism in MiniMax-M1 enables efficient scaling of test-time compute. These properties make M1 particularly suitable for complex tasks that require processing long inputs and thinking extensively. MiniMax-M1 is trained using large-scale reinforcement learning (RL) on diverse problems including sandbox-based, real-world software engineering environments. In addition to M1's inherent efficiency advantage for RL training, we propose CISPO, a novel RL algorithm to further enhance RL efficiency. CISPO clips importance sampling weights rather than token updates, outperforming other competitive RL variants. Combining hybrid-attention and CISPO enables MiniMax-M1's full RL training on 512 H800 GPUs to complete in only three weeks, with a rental cost of just $534,700. We release two versions of MiniMax-M1 models with 40K and 80K thinking budgets respectively, where the 40K model represents an intermediate phase of the 80K training. Experiments on standard benchmarks show that our models are comparable or superior to strong open-weight models such as the original DeepSeek-R1 and Qwen3-235B, with particular strengths in complex software engineering, tool utilization, and long-context tasks. We publicly release MiniMax-M1 at https://github.com/MiniMax-AI/MiniMax-M1.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.13585</guid>
      <pubDate>Mon, 16 Jun 2025 15:08:02 +0000</pubDate>
    </item>
    <item>
      <title>Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning</title>
      <link>https://arxiv.org/abs/2506.13654</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.13654.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shulin Tian, Ruiqi Wang, Hongming Guo, Penghao Wu, Yuhao Dong, Xiuying Wang, Jingkang Yang, Hao Zhang, Hongyuan Zhu, Ziwei Liu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 42&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce Ego-R1, a novel framework for reasoning over ultra-long (i.e., in days and weeks) egocentric videos, which leverages a structured Chain-of-Tool-Thought (CoTT) process, orchestrated by an Ego-R1 Agent trained via reinforcement learning (RL). Inspired by human problem-solving strategies, CoTT decomposes complex reasoning into modular steps, with the RL agent invoking specific tools, one per step, to iteratively and collaboratively answer sub-questions tackling such tasks as temporal retrieval and multi-modal understanding. We design a two-stage training paradigm involving supervised finetuning (SFT) of a pretrained language model using CoTT data and RL to enable our agent to dynamically propose step-by-step tools for long-range reasoning. To facilitate training, we construct a dataset called Ego-R1 Data, which consists of Ego-CoTT-25K for SFT and Ego-QA-4.4K for RL. Furthermore, our Ego-R1 agent is evaluated on a newly curated week-long video QA benchmark, Ego-R1 Bench, which contains human-verified QA pairs from hybrid sources. Extensive results demonstrate that the dynamic, tool-augmented chain-of-thought reasoning by our Ego-R1 Agent can effectively tackle the unique challenges of understanding ultra-long egocentric videos, significantly extending the time coverage from few hours to a week.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.13654</guid>
      <pubDate>Mon, 16 Jun 2025 16:17:08 +0000</pubDate>
    </item>
    <item>
      <title>Discrete Diffusion in Large Language and Multimodal Models: A Survey</title>
      <link>https://arxiv.org/abs/2506.13759</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.13759.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Runpeng Yu, Qi Li, Xinchao Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 41&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; In this work, we provide a systematic survey of Discrete Diffusion Language Models (dLLMs) and Discrete Diffusion Multimodal Language Models (dMLLMs). Unlike autoregressive (AR) models, dLLMs and dMLLMs adopt a multi-token, parallel decoding paradigm using full attention and a denoising-based generation strategy. This paradigm naturally enables parallel generation, fine-grained output controllability, and dynamic, response-aware perception. These capabilities are previously difficult to achieve with AR models. Recently, a growing number of industrial-scale proprietary d(M)LLMs, as well as a large number of open-source academic d(M)LLMs, have demonstrated performance comparable to their autoregressive counterparts, while achieving up to 10x acceleration in inference speed.   The advancement of discrete diffusion LLMs and MLLMs has been largely driven by progress in two domains. The first is the development of autoregressive LLMs and MLLMs, which has accumulated vast amounts of data, benchmarks, and foundational infrastructure for training and inference. The second contributing domain is the evolution of the mathematical models underlying discrete diffusion. Together, these advancements have catalyzed a surge in dLLMs and dMLLMs research in early 2025.   In this work, we present a comprehensive overview of the research in the dLLM and dMLLM domains. We trace the historical development of dLLMs and dMLLMs, formalize the underlying mathematical frameworks, and categorize representative models. We further analyze key techniques for training and inference, and summarize emerging applications across language, vision-language, and biological domains. We conclude by discussing future directions for research and deployment.   Paper collection: https://github.com/LiQiiiii/DLLM-Survey&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.13759</guid>
      <pubDate>Mon, 16 Jun 2025 17:59:08 +0000</pubDate>
    </item>
    <item>
      <title>MultiFinBen: A Multilingual, Multimodal, and Difficulty-Aware Benchmark   for Financial LLM Evaluation</title>
      <link>https://arxiv.org/abs/2506.14028</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.14028.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Guojun Xiong, Zhiyang Deng, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 90&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in large language models (LLMs) have accelerated progress in financial NLP and applications, yet existing benchmarks remain limited to monolingual and unimodal settings, often over-relying on simple tasks and failing to reflect the complexity of real-world financial communication. We introduce MultiFinBen, the first multilingual and multimodal benchmark tailored to the global financial domain, evaluating LLMs across modalities (text, vision, audio) and linguistic settings (monolingual, bilingual, multilingual) on domain-specific tasks. We introduce two novel tasks, including PolyFiQA-Easy and PolyFiQA-Expert, the first multilingual financial benchmarks requiring models to perform complex reasoning over mixed-language inputs; and EnglishOCR and SpanishOCR, the first OCR-embedded financial QA tasks challenging models to extract and reason over information from visual-text financial documents. Moreover, we propose a dynamic, difficulty-aware selection mechanism and curate a compact, balanced benchmark rather than simple aggregation existing datasets. Extensive evaluation of 22 state-of-the-art models reveals that even the strongest models, despite their general multimodal and multilingual capabilities, struggle dramatically when faced with complex cross-lingual and multimodal tasks in financial domain. MultiFinBen is publicly released to foster transparent, reproducible, and inclusive progress in financial studies and applications.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.14028</guid>
      <pubDate>Mon, 16 Jun 2025 22:01:49 +0000</pubDate>
    </item>
    <item>
      <title>Essential-Web v1.0: 24T tokens of organized web data</title>
      <link>https://arxiv.org/abs/2506.14111</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.14111.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Essential AI, Andrew Hojel, Michael Pust, Tim Romanski, Yash Vanjani, Ritvik Kapila, Mohit Parmar, Adarsh Chaluvaraju, Alok Tripathy, Anil Thomas, Ashish Tanwer, Darsh J Shah, Ishaan Shah, Karl Stratos, Khoi Nguyen, Kurt Smith, Michael Callahan, Peter Rushton, Philip Monk, Platon Mazarakis, Saad Jamal, Saurabh Srivastava, Somanshu Singla, Ashish Vaswani&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 39&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Data plays the most prominent role in how language models acquire skills and knowledge. The lack of massive, well-organized pre-training datasets results in costly and inaccessible data pipelines. We present Essential-Web v1.0, a 24-trillion-token dataset in which every document is annotated with a twelve-category taxonomy covering topic, format, content complexity, and quality. Taxonomy labels are produced by EAI-Distill-0.5b, a fine-tuned 0.5b-parameter model that achieves an annotator agreement within 3% of Qwen2.5-32B-Instruct. With nothing more than SQL-style filters, we obtain competitive web-curated datasets in math (-8.0% relative to SOTA), web code (+14.3%), STEM (+24.5%) and medical (+8.6%). Essential-Web v1.0 is available on HuggingFace: https://huggingface.co/datasets/EssentialAI/essential-web-v1.0&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.14111</guid>
      <pubDate>Tue, 17 Jun 2025 02:03:36 +0000</pubDate>
    </item>
    <item>
      <title>Xolver: Multi-Agent Reasoning with Holistic Experience Learning Just   Like an Olympiad Team</title>
      <link>https://arxiv.org/abs/2506.14234</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.14234.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Md Tanzib Hosain, Salman Rahman, Md Kishor Morol, Md Rizwan Parvez&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 37&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Despite impressive progress on complex reasoning, current large language models (LLMs) typically operate in isolation - treating each problem as an independent attempt, without accumulating or integrating experiential knowledge. In contrast, expert problem solvers - such as Olympiad or programming contest teams - leverage a rich tapestry of experiences: absorbing mentorship from coaches, developing intuition from past problems, leveraging knowledge of tool usage and library functionality, adapting strategies based on the expertise and experiences of peers, continuously refining their reasoning through trial and error, and learning from other related problems even during competition. We introduce Xolver, a training-free multi-agent reasoning framework that equips a black-box LLM with a persistent, evolving memory of holistic experience. Xolver integrates diverse experience modalities, including external and self-retrieval, tool use, collaborative interactions, agent-driven evaluation, and iterative refinement. By learning from relevant strategies, code fragments, and abstract reasoning patterns at inference time, Xolver avoids generating solutions from scratch - marking a transition from isolated inference toward experience-aware language agents. Built on both open-weight and proprietary models, Xolver consistently outperforms specialized reasoning agents. Even with lightweight backbones (e.g., QWQ-32B), it often surpasses advanced models including Qwen3-235B, Gemini 2.5 Pro, o3, and o4-mini-high. With o3-mini-high, it achieves new best results on GSM8K (98.1%), AIME'24 (94.4%), AIME'25 (93.7%), Math-500 (99.8%), and LiveCodeBench-V5 (91.6%) - highlighting holistic experience learning as a key step toward generalist agents capable of expert-level reasoning. Code and data are available at https://kagnlp.github.io/xolver.github.io/.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.14234</guid>
      <pubDate>Tue, 17 Jun 2025 06:47:19 +0000</pubDate>
    </item>
    <item>
      <title>Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes   Correct Reasoning in Base LLMs</title>
      <link>https://arxiv.org/abs/2506.14245</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.14245.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Xumeng Wen, Zihan Liu, Shun Zheng, Zhijian Xu, Shengyu Ye, Zhirong Wu, Xiao Liang, Yang Wang, Junjie Li, Ziming Miao, Jiang Bian, Mao Yang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a promising paradigm for advancing the reasoning capabilities of Large Language Models (LLMs). However, a critical paradox clouds its efficacy: RLVR-tuned models often underperform their base models on the Pass@K metric for solution-finding, leading to the hypothesis that RLVR merely re-weights existing reasoning paths at the cost of reasoning diversity. In this work, we resolve this contradiction by identifying the source of the problem: the Pass@K metric itself is a flawed measure of reasoning, as it credits correct final answers that probably arise from inaccurate or incomplete chains of thought (CoTs). To address this, we introduce a more precise evaluation metric, CoT-Pass@K, which mandates that both the reasoning path and the final answer be correct. We provide a new theoretical foundation that formalizes how RLVR, unlike traditional RL, is uniquely structured to incentivize logical integrity. Our empirical results are supportive: using CoT-Pass@K, we observe that RLVR can incentivize the generalization of correct reasoning for all values of K. Furthermore, by analyzing the training dynamics, we find that this enhanced reasoning capability emerges early in the training process and smoothly generalizes. Our work provides a clear perspective on the role of RLVR, offers a more reliable method for its evaluation, and confirms its potential to genuinely advance machine reasoning.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.14245</guid>
      <pubDate>Tue, 17 Jun 2025 07:06:56 +0000</pubDate>
    </item>
    <item>
      <title>LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs</title>
      <link>https://arxiv.org/abs/2506.14429</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.14429.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Xiaoran Liu, Zhigeng Liu, Zengfeng Huang, Qipeng Guo, Ziwei He, Xipeng Qiu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 43&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Large Language Diffusion Models, or diffusion LLMs, have emerged as a significant focus in NLP research, with substantial effort directed toward understanding their scalability and downstream task performance. However, their long-context capabilities remain unexplored, lacking systematic analysis or methods for context extension. In this work, we present the first systematic investigation comparing the long-context performance of diffusion LLMs and traditional auto-regressive LLMs. We first identify a unique characteristic of diffusion LLMs, unlike auto-regressive LLMs, they maintain remarkably \textit{stable perplexity} during direct context extrapolation. Furthermore, where auto-regressive models fail outright during the Needle-In-A-Haystack task with context exceeding their pretrained length, we discover diffusion LLMs exhibit a distinct \textit{local perception} phenomenon, enabling successful retrieval from recent context segments. We explain both phenomena through the lens of Rotary Position Embedding (RoPE) scaling theory. Building on these observations, we propose LongLLaDA, a training-free method that integrates LLaDA with the NTK-based RoPE extrapolation. Our results validate that established extrapolation scaling laws remain effective for extending the context windows of diffusion LLMs. Furthermore, we identify long-context tasks where diffusion LLMs outperform auto-regressive LLMs and others where they fall short. Consequently, this study establishes the first context extrapolation method for diffusion LLMs while providing essential theoretical insights and empirical benchmarks critical for advancing future research on long-context diffusion LLMs.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.14429</guid>
      <pubDate>Tue, 17 Jun 2025 11:45:37 +0000</pubDate>
    </item>
    <item>
      <title>OAgents: An Empirical Study of Building Effective Agents</title>
      <link>https://arxiv.org/abs/2506.15741</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.15741.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; He Zhu, Tianrui Qin, King Zhu, Heyuan Huang, Yeyi Guan, Jinxiang Xia, Yi Yao, Hanhao Li, Ningning Wang, Pai Liu, Tianhao Peng, Xin Gui, Xiaowan Li, Yuhui Liu, Yuchen Eleanor Jiang, Jun Wang, Changwang Zhang, Xiangru Tang, Ge Zhang, Jian Yang, Minghao Liu, Xitong Gao, Jiaheng Liu, Wangchunshu Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 34&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recently, Agentic AI has become an increasingly popular research field. However, we argue that current agent research practices lack standardization and scientific rigor, making it hard to conduct fair comparisons among methods. As a result, it is still unclear how different design choices in agent frameworks affect effectiveness, and measuring their progress remains challenging. In this work, we conduct a systematic empirical study on GAIA benchmark and BrowseComp to examine the impact of popular design choices in key agent components in a fair and rigorous manner. We find that the lack of a standard evaluation protocol makes previous works, even open-sourced ones, non-reproducible, with significant variance between random runs. Therefore, we introduce a more robust evaluation protocol to stabilize comparisons. Our study reveals which components and designs are crucial for effective agents, while others are redundant, despite seeming logical. Based on our findings, we build and open-source OAgents, a new foundation agent framework that achieves state-of-the-art performance among open-source projects. OAgents offers a modular design for various agent components, promoting future research in Agentic AI.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.15741</guid>
      <pubDate>Tue, 17 Jun 2025 17:59:02 +0000</pubDate>
    </item>
    <item>
      <title>Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain   Perspective</title>
      <link>https://arxiv.org/abs/2506.14965</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.14965.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zhoujun Cheng, Shibo Hao, Tianyang Liu, Fan Zhou, Yutao Xie, Feng Yao, Yuexin Bian, Yonghao Zhuang, Nilabjo Dey, Yuheng Zha, Yi Gu, Kun Zhou, Yuqi Wang, Yuan Li, Richard Fan, Jianshu She, Chengqian Gao, Abulhair Saparov, Haonan Li, Taylor W. Killian, Mikhail Yurochkin, Zhengzhong Liu, Eric P. Xing, Zhiting Hu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 43&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Reinforcement learning (RL) has emerged as a promising approach to improve large language model (LLM) reasoning, yet most open efforts focus narrowly on math and code, limiting our understanding of its broader applicability to general reasoning. A key challenge lies in the lack of reliable, scalable RL reward signals across diverse reasoning domains. We introduce Guru, a curated RL reasoning corpus of 92K verifiable examples spanning six reasoning domains--Math, Code, Science, Logic, Simulation, and Tabular--each built through domain-specific reward design, deduplication, and filtering to ensure reliability and effectiveness for RL training. Based on Guru, we systematically revisit established findings in RL for LLM reasoning and observe significant variation across domains. For example, while prior work suggests that RL primarily elicits existing knowledge from pretrained models, our results reveal a more nuanced pattern: domains frequently seen during pretraining (Math, Code, Science) easily benefit from cross-domain RL training, while domains with limited pretraining exposure (Logic, Simulation, and Tabular) require in-domain training to achieve meaningful performance gains, suggesting that RL is likely to facilitate genuine skill acquisition. Finally, we present Guru-7B and Guru-32B, two models that achieve state-of-the-art performance among open models RL-trained with publicly available data, outperforming best baselines by 7.9% and 6.7% on our 17-task evaluation suite across six reasoning domains. We also show that our models effectively improve the Pass@k performance of their base models, particularly on complex tasks less likely to appear in pretraining data. We release data, models, training and evaluation code to facilitate general-purpose reasoning at: https://github.com/LLM360/Reasoning360&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.14965</guid>
      <pubDate>Tue, 17 Jun 2025 20:24:00 +0000</pubDate>
    </item>
    <item>
      <title>ProtoReasoning: Prototypes as the Foundation for Generalizable Reasoning   in LLMs</title>
      <link>https://arxiv.org/abs/2506.15211</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.15211.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Feng He, Zijun Chen, Xinnian Liang, Tingting Ma, Yunqi Qiu, Shuangzhi Wu, Junchi Yan&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 33&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in Large Reasoning Models (LRMs) trained with Long Chain-of-Thought (Long CoT) reasoning have demonstrated remarkable cross-domain generalization capabilities. However, the underlying mechanisms supporting such transfer remain poorly understood. We hypothesize that cross-domain generalization arises from shared abstract reasoning prototypes -- fundamental reasoning patterns that capture the essence of problems across domains. These prototypes minimize the nuances of the representation, revealing that seemingly diverse tasks are grounded in shared reasoning structures.Based on this hypothesis, we propose ProtoReasoning, a framework that enhances the reasoning ability of LLMs by leveraging scalable and verifiable prototypical representations (Prolog for logical reasoning, PDDL for planning).ProtoReasoning features: (1) an automated prototype construction pipeline that transforms problems into corresponding prototype representations; (2) a comprehensive verification system providing reliable feedback through Prolog/PDDL interpreters; (3) the scalability to synthesize problems arbitrarily within prototype space while ensuring correctness. Extensive experiments show that ProtoReasoning achieves 4.7% improvement over baseline models on logical reasoning (Enigmata-Eval), 6.3% improvement on planning tasks, 4.0% improvement on general reasoning (MMLU) and 1.0% on mathematics (AIME24). Significantly, our ablation studies confirm that learning in prototype space also demonstrates enhanced generalization to structurally similar problems compared to training solely on natural language representations, validating our hypothesis that reasoning prototypes serve as the foundation for generalizable reasoning in large language models.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.15211</guid>
      <pubDate>Wed, 18 Jun 2025 07:44:09 +0000</pubDate>
    </item>
    <item>
      <title>All is Not Lost: LLM Recovery without Checkpoints</title>
      <link>https://arxiv.org/abs/2506.15461</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.15461.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Nikolay Blagoev, Oğuzhan Ersoy, Lydia Yiyu Chen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Training LLMs on decentralized and wimpy computation nodes, e.g., multiple on-spot instances, lowers the training cost and enables model democratization. The inevitable challenge here is the churn of nodes due to failures and the operator's scheduling policies, leading to losing a stage - a part of the model. The conventional approaches to recover from failures are to either use checkpointing, where periodically a copy of the entire model is sent to an additional storage, or redundant computation. These approaches yield significant communication and/or computation overhead even in non-failure cases and scale poorly in settings with large models. In this paper, we propose, CheckFree, an efficient recovery method where a failing stage is substituted by a weighted average of the closest neighboring stages. In contrast to the state of the art, CheckFree requires no additional computation or storage. However, because of the nature of averaging neighbouring stages, it can only recover failures of intermediate stages. We further extend our method to CheckFree+ with out-of-order pipeline execution to tolerate crashes of the first and last stages. Thanks to out-of-order pipelining, behaviour of those stages is mimicked by their neighboring ones, which allows CheckFree+ to recover them by simply copying the weights from the immediate neighbour. To be able to recover the (de)embedding layers, CheckFree+ copies those layers to the neighboring stages, which requires relatively small storage overhead. We extensively evaluate our method on LLaMa models of model sizes from 124M to 1.5B with varying failure frequencies. In the case of low and medium failure rates (5-10%), CheckFree and CheckFree+ outperform both checkpointing and redundant computation in terms of convergence in wall-clock time by over 12%. Both of our proposals can be run via our code available at: https://github.com/gensyn-ai/CheckFree.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.15461</guid>
      <pubDate>Wed, 18 Jun 2025 13:48:33 +0000</pubDate>
    </item>
    <item>
      <title>Sekai: A Video Dataset towards World Exploration</title>
      <link>https://arxiv.org/abs/2506.15675</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.15675.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zhen Li, Chuanhao Li, Xiaofeng Mao, Shaoheng Lin, Ming Li, Shitian Zhao, Zhaopan Xu, Xinyue Li, Yukang Feng, Jianwen Sun, Zizhen Li, Fanrui Zhang, Jiaxin Ai, Zhixiang Wang, Yuwei Wu, Tong He, Jiangmiao Pang, Yu Qiao, Yunde Jia, Kaipeng Zhang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 61&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Video generation techniques have made remarkable progress, promising to be the foundation of interactive world exploration. However, existing video generation datasets are not well-suited for world exploration training as they suffer from some limitations: limited locations, short duration, static scenes, and a lack of annotations about exploration and the world. In this paper, we introduce Sekai (meaning ``world'' in Japanese), a high-quality first-person view worldwide video dataset with rich annotations for world exploration. It consists of over 5,000 hours of walking or drone view (FPV and UVA) videos from over 100 countries and regions across 750 cities. We develop an efficient and effective toolbox to collect, pre-process and annotate videos with location, scene, weather, crowd density, captions, and camera trajectories. Experiments demonstrate the quality of the dataset. And, we use a subset to train an interactive video world exploration model, named YUME (meaning ``dream'' in Japanese). We believe Sekai will benefit the area of video generation and world exploration, and motivate valuable applications.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.15675</guid>
      <pubDate>Wed, 18 Jun 2025 17:57:06 +0000</pubDate>
    </item>
    <item>
      <title>GenRecal: Generation after Recalibration from Large to Small   Vision-Language Models</title>
      <link>https://arxiv.org/abs/2506.15681</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.15681.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Byung-Kwan Lee, Ryo Hachiuma, Yong Man Ro, Yu-Chiang Frank Wang, Yueh-Hua Wu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 36&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advancements in vision-language models (VLMs) have leveraged large language models (LLMs) to achieve performance on par with closed-source systems like GPT-4V. However, deploying these models in real-world scenarios, particularly on resource-constrained devices, remains challenging due to their substantial computational demands. This has spurred interest in distilling knowledge from large VLMs into smaller, more efficient counterparts. A key challenge arises here from the diversity of VLM architectures, which are built on different LLMs and employ varying token types-differing in vocabulary size, token splits, and token index ordering. To address this challenge of limitation to a specific VLM type, we present Generation after Recalibration (GenRecal), a novel, general-purpose distillation framework for VLMs. GenRecal incorporates a Recalibrator that aligns and adapts feature representations between heterogeneous VLMs, enabling effective knowledge transfer across different types of VLMs. Through extensive experiments on multiple challenging benchmarks, we demonstrate that GenRecal significantly improves baseline performances, eventually outperforming large-scale open- and closed-source VLMs.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.15681</guid>
      <pubDate>Wed, 18 Jun 2025 17:59:49 +0000</pubDate>
    </item>
    <item>
      <title>Vision-Guided Chunking Is All You Need: Enhancing RAG with Multimodal   Document Understanding</title>
      <link>https://arxiv.org/abs/2506.16035</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.16035.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Vishesh Tripathi, Tanmay Odapally, Indraneel Das, Uday Allu, Biddwan Ahmed&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 82&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Retrieval-Augmented Generation (RAG) systems have revolutionized information retrieval and question answering, but traditional text-based chunking methods struggle with complex document structures, multi-page tables, embedded figures, and contextual dependencies across page boundaries. We present a novel multimodal document chunking approach that leverages Large Multimodal Models (LMMs) to process PDF documents in batches while maintaining semantic coherence and structural integrity. Our method processes documents in configurable page batches with cross-batch context preservation, enabling accurate handling of tables spanning multiple pages, embedded visual elements, and procedural content. We evaluate our approach on a curated dataset of PDF documents with manually crafted queries, demonstrating improvements in chunk quality and downstream RAG performance. Our vision-guided approach achieves better accuracy compared to traditional vanilla RAG systems, with qualitative analysis showing superior preservation of document structure and semantic coherence.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.16035</guid>
      <pubDate>Thu, 19 Jun 2025 05:11:43 +0000</pubDate>
    </item>
    <item>
      <title>PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and   Quantized Attention in Visual Generation Models</title>
      <link>https://arxiv.org/abs/2506.16054</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.16054.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Tianchen Zhao, Ke Hong, Xinhao Yang, Xuefeng Xiao, Huixia Li, Feng Ling, Ruiqi Xie, Siqi Chen, Hongyu Zhu, Yichong Zhang, Yu Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 57&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; In visual generation, the quadratic complexity of attention mechanisms results in high memory and computational costs, especially for longer token sequences required in high-resolution image or multi-frame video generation. To address this, prior research has explored techniques such as sparsification and quantization. However, these techniques face significant challenges under low density and reduced bitwidths. Through systematic analysis, we identify that the core difficulty stems from the dispersed and irregular characteristics of visual attention patterns. Therefore, instead of introducing specialized sparsification and quantization design to accommodate such patterns, we propose an alternative strategy: *reorganizing* the attention pattern to alleviate the challenges. Inspired by the local aggregation nature of visual feature extraction, we design a novel **Pattern-Aware token ReOrdering (PARO)** technique, which unifies the diverse attention patterns into a hardware-friendly block-wise pattern. This unification substantially simplifies and enhances both sparsification and quantization. We evaluate the performance-efficiency trade-offs of various design choices and finalize a methodology tailored for the unified pattern. Our approach, **PAROAttention**, achieves video and image generation with lossless metrics, and nearly identical results from full-precision (FP) baselines, while operating at notably lower density (~20%-30%) and bitwidth (**INT8/INT4**), achieving a **1.9x** to **2.7x** end-to-end latency speedup.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.16054</guid>
      <pubDate>Thu, 19 Jun 2025 06:25:02 +0000</pubDate>
    </item>
    <item>
      <title>Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights</title>
      <link>https://arxiv.org/abs/2506.16406</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.16406.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zhiyuan Liang, Dongwen Tang, Yuhao Zhou, Xuanlei Zhao, Mingjia Shi, Wangbo Zhao, Zekai Li, Peihao Wang, Konstantin Schürholt, Damian Borth, Michael M. Bronstein, Yang You, Zhangyang Wang, Kai Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 113&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Modern Parameter-Efficient Fine-Tuning (PEFT) methods such as low-rank adaptation (LoRA) reduce the cost of customizing large language models (LLMs), yet still require a separate optimization run for every downstream dataset. We introduce Drag-and-Drop LLMs (\textit{DnD)}, a prompt-conditioned parameter generator that eliminates per-task training by mapping a handful of unlabeled task prompts directly to LoRA weight updates. A lightweight text encoder distills each prompt batch into condition embeddings, which are then transformed by a cascaded hyper-convolutional decoder into the full set of LoRA matrices. Once trained in a diverse collection of prompt-checkpoint pairs, DnD produces task-specific parameters in seconds, yielding i) up to 12,000times lower overhead than full fine-tuning, ii) average gains up to 30\% in performance over the strongest training LoRAs on unseen common-sense reasoning, math, coding, and multimodal benchmarks, and iii) robust cross-domain generalization despite never seeing the target data or labels. Our results demonstrate that prompt-conditioned parameter generation is a viable alternative to gradient-based adaptation for rapidly specializing LLMs. Our project is available at https://jerryliang24.github.io/DnD{https://jerryliang24.github.io/DnD}.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.16406</guid>
      <pubDate>Thu, 19 Jun 2025 15:38:21 +0000</pubDate>
    </item>
    <item>
      <title>Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with   Hybrid History Condition</title>
      <link>https://arxiv.org/abs/2506.17201</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.17201.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jiaqi Li, Junshu Tang, Zhiyong Xu, Longhuang Wu, Yuan Zhou, Shuai Shao, Tianbao Yu, Zhiguo Cao, Qinglin Lu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 50&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in diffusion-based and controllable video generation have enabled high-quality and temporally coherent video synthesis, laying the groundwork for immersive interactive gaming experiences. However, current methods face limitations in dynamics, generality, long-term consistency, and efficiency, which limit the ability to create various gameplay videos. To address these gaps, we introduce Hunyuan-GameCraft, a novel framework for high-dynamic interactive video generation in game environments. To achieve fine-grained action control, we unify standard keyboard and mouse inputs into a shared camera representation space, facilitating smooth interpolation between various camera and movement operations. Then we propose a hybrid history-conditioned training strategy that extends video sequences autoregressively while preserving game scene information. Additionally, to enhance inference efficiency and playability, we achieve model distillation to reduce computational overhead while maintaining consistency across long temporal sequences, making it suitable for real-time deployment in complex interactive environments. The model is trained on a large-scale dataset comprising over one million gameplay recordings across over 100 AAA games, ensuring broad coverage and diversity, then fine-tuned on a carefully annotated synthetic dataset to enhance precision and control. The curated game scene data significantly improves the visual fidelity, realism and action controllability. Extensive experiments demonstrate that Hunyuan-GameCraft significantly outperforms existing models, advancing the realism and playability of interactive game video generation.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.17201</guid>
      <pubDate>Fri, 20 Jun 2025 17:50:37 +0000</pubDate>
    </item>
    <item>
      <title>BlenderFusion: 3D-Grounded Visual Editing and Generative Compositing</title>
      <link>https://arxiv.org/abs/2506.17450</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.17450.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jiacheng Chen, Ramin Mehran, Xuhui Jia, Saining Xie, Sanghyun Woo&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 45&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We present BlenderFusion, a generative visual compositing framework that synthesizes new scenes by recomposing objects, camera, and background. It follows a layering-editing-compositing pipeline: (i) segmenting and converting visual inputs into editable 3D entities (layering), (ii) editing them in Blender with 3D-grounded control (editing), and (iii) fusing them into a coherent scene using a generative compositor (compositing). Our generative compositor extends a pre-trained diffusion model to process both the original (source) and edited (target) scenes in parallel. It is fine-tuned on video frames with two key training strategies: (i) source masking, enabling flexible modifications like background replacement; (ii) simulated object jittering, facilitating disentangled control over objects and camera. BlenderFusion significantly outperforms prior methods in complex compositional scene editing tasks.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.17450</guid>
      <pubDate>Fri, 20 Jun 2025 19:38:34 +0000</pubDate>
    </item>
    <item>
      <title>JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo   Retouching Agent</title>
      <link>https://arxiv.org/abs/2506.17612</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.17612.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yunlong Lin, Zixu Lin, Kunjie Lin, Jinbin Bai, Panwang Pan, Chenxin Li, Haoyu Chen, Zhongdao Wang, Xinghao Ding, Wenbo Li, Shuicheng Yan&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 59&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Photo retouching has become integral to contemporary visual storytelling, enabling users to capture aesthetics and express creativity. While professional tools such as Adobe Lightroom offer powerful capabilities, they demand substantial expertise and manual effort. In contrast, existing AI-based solutions provide automation but often suffer from limited adjustability and poor generalization, failing to meet diverse and personalized editing needs. To bridge this gap, we introduce JarvisArt, a multi-modal large language model (MLLM)-driven agent that understands user intent, mimics the reasoning process of professional artists, and intelligently coordinates over 200 retouching tools within Lightroom. JarvisArt undergoes a two-stage training process: an initial Chain-of-Thought supervised fine-tuning to establish basic reasoning and tool-use skills, followed by Group Relative Policy Optimization for Retouching (GRPO-R) to further enhance its decision-making and tool proficiency. We also propose the Agent-to-Lightroom Protocol to facilitate seamless integration with Lightroom. To evaluate performance, we develop MMArt-Bench, a novel benchmark constructed from real-world user edits. JarvisArt demonstrates user-friendly interaction, superior generalization, and fine-grained control over both global and local adjustments, paving a new avenue for intelligent photo retouching. Notably, it outperforms GPT-4o with a 60% improvement in average pixel-level metrics on MMArt-Bench for content fidelity, while maintaining comparable instruction-following capabilities. Project Page: https://jarvisart.vercel.app/.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.17612</guid>
      <pubDate>Sat, 21 Jun 2025 06:36:00 +0000</pubDate>
    </item>
    <item>
      <title>ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image   Generation</title>
      <link>https://arxiv.org/abs/2506.18095</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18095.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Junying Chen, Zhenyang Cai, Pengcheng Chen, Shunian Chen, Ke Ji, Xidong Wang, Yunjin Yang, Benyou Wang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 61&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in multimodal generative models have unlocked photorealistic, instruction-aligned image generation, yet leading systems like GPT-4o-Image remain proprietary and inaccessible. To democratize these capabilities, we present ShareGPT-4o-Image, the first dataset comprising 45K text-to-image and 46K text-and-image-to-image data, all synthesized using GPT-4o's image generation capabilities for distilling its advanced image generation abilities. Leveraging this dataset, we develop Janus-4o, a multimodal large language model capable of both text-to-image and text-and-image-to-image generation. Janus-4o not only significantly improves text-to-image generation over its predecessor, Janus-Pro, but also newly supports text-and-image-to-image generation. Notably, it achieves impressive performance in text-and-image-to-image generation from scratch, using only 91K synthetic samples and 6 hours of training on an 8 A800-GPU machine. We hope the release of ShareGPT-4o-Image and Janus-4o will foster open research in photorealistic, instruction-aligned image generation.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.18095</guid>
      <pubDate>Sun, 22 Jun 2025 16:51:09 +0000</pubDate>
    </item>
    <item>
      <title>Chain-of-Experts: Unlocking the Communication Power of   Mixture-of-Experts Models</title>
      <link>https://arxiv.org/abs/2506.18945</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18945.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zihan Wang, Rui Pan, Jiarui Yao, Robert Csordas, Linjie Li, Lu Yin, Jiajun Wu, Tong Zhang, Manling Li, Shiwei Liu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 39&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We propose Chain-of-Experts (CoE), a new Mixture-of-Experts (MoE) architecture that introduces sequential expert communication within each layer. Unlike traditional MoE models, where experts operate independently in parallel, CoE processes tokens iteratively across a chain of experts inside a layer. To support dynamic expert selection across iterations, CoE employs a dedicated router at each iteration step within a layer. This design allows tokens to re-evaluate and select different experts during each iteration, rather than being statically assigned. As a result, CoE introduces a flexible routing mechanism that increases the diversity of expert combinations and enriches the model's representational capacity. CoE demonstrates improved performance under fixed compute: on math reasoning tasks, it reduces validation loss from 1.20 to 1.12 compared to a standard MoE. Beyond performance, CoE offers a new scaling axis: depth through expert iteration, which complements conventional width/depth scaling. For example, using 2x iterations matches the performance of 3x expert selections (in width), while reducing memory usage by 17.6-42% relative to other scaling strategies. Our analysis reveals that CoE's benefits stem from its iterative residual structure and enhanced expert specialization empowered by iterative routing, which together unlock more expressive representations. Code is available at https://github.com/ZihanWang314/coe.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.18945</guid>
      <pubDate>Mon, 23 Jun 2025 02:15:43 +0000</pubDate>
    </item>
    <item>
      <title>Matrix-Game: Interactive World Foundation Model</title>
      <link>https://arxiv.org/abs/2506.18701</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18701.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yifan Zhang, Chunli Peng, Boyang Wang, Puyi Wang, Qingcheng Zhu, Fei Kang, Biao Jiang, Zedong Gao, Eric Li, Yang Liu, Yahui Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 58&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce Matrix-Game, an interactive world foundation model for controllable game world generation. Matrix-Game is trained using a two-stage pipeline that first performs large-scale unlabeled pretraining for environment understanding, followed by action-labeled training for interactive video generation. To support this, we curate Matrix-Game-MC, a comprehensive Minecraft dataset comprising over 2,700 hours of unlabeled gameplay video clips and over 1,000 hours of high-quality labeled clips with fine-grained keyboard and mouse action annotations. Our model adopts a controllable image-to-world generation paradigm, conditioned on a reference image, motion context, and user actions. With over 17 billion parameters, Matrix-Game enables precise control over character actions and camera movements, while maintaining high visual quality and temporal coherence. To evaluate performance, we develop GameWorld Score, a unified benchmark measuring visual quality, temporal quality, action controllability, and physical rule understanding for Minecraft world generation. Extensive experiments show that Matrix-Game consistently outperforms prior open-source Minecraft world models (including Oasis and MineWorld) across all metrics, with particularly strong gains in controllability and physical consistency. Double-blind human evaluations further confirm the superiority of Matrix-Game, highlighting its ability to generate perceptually realistic and precisely controllable videos across diverse game scenarios. To facilitate future research on interactive image-to-world generation, we will open-source the Matrix-Game model weights and the GameWorld Score benchmark at https://github.com/SkyworkAI/Matrix-Game.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.18701</guid>
      <pubDate>Mon, 23 Jun 2025 14:40:49 +0000</pubDate>
    </item>
    <item>
      <title>LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement   Learning</title>
      <link>https://arxiv.org/abs/2506.18841</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18841.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuhao Wu, Yushi Bai, Zhiqiang Hu, Roy Ka-Wei Lee, Juanzi Li&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 51&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Ultra-long generation by large language models (LLMs) is a widely demanded scenario, yet it remains a significant challenge due to their maximum generation length limit and overall quality degradation as sequence length increases. Previous approaches, exemplified by LongWriter, typically rely on ''teaching'', which involves supervised fine-tuning (SFT) on synthetic long-form outputs. However, this strategy heavily depends on synthetic SFT data, which is difficult and costly to construct, often lacks coherence and consistency, and tends to be overly artificial and structurally monotonous. In this work, we propose an incentivization-based approach that, starting entirely from scratch and without relying on any annotated or synthetic data, leverages reinforcement learning (RL) to foster the emergence of ultra-long, high-quality text generation capabilities in LLMs. We perform RL training starting from a base model, similar to R1-Zero, guiding it to engage in reasoning that facilitates planning and refinement during the writing process. To support this, we employ specialized reward models that steer the LLM towards improved length control, writing quality, and structural formatting. Experimental evaluations show that our LongWriter-Zero model, trained from Qwen2.5-32B, consistently outperforms traditional SFT methods on long-form writing tasks, achieving state-of-the-art results across all metrics on WritingBench and Arena-Write, and even surpassing 100B+ models such as DeepSeek R1 and Qwen3-235B. We open-source our data and model checkpoints under https://huggingface.co/THU-KEG/LongWriter-Zero-32B&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.18841</guid>
      <pubDate>Mon, 23 Jun 2025 16:59:02 +0000</pubDate>
    </item>
    <item>
      <title>OmniGen2: Exploration to Advanced Multimodal Generation</title>
      <link>https://arxiv.org/abs/2506.18871</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18871.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, Ze Liu, Ziyi Xia, Chaofan Li, Haoge Deng, Jiahao Wang, Kun Luo, Bo Zhang, Defu Lian, Xinlong Wang, Zhongyuan Wang, Tiejun Huang, Zheng Liu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 68&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; In this work, we introduce OmniGen2, a versatile and open-source generative model designed to provide a unified solution for diverse generation tasks, including text-to-image, image editing, and in-context generation. Unlike OmniGen v1, OmniGen2 features two distinct decoding pathways for text and image modalities, utilizing unshared parameters and a decoupled image tokenizer. This design enables OmniGen2 to build upon existing multimodal understanding models without the need to re-adapt VAE inputs, thereby preserving the original text generation capabilities. To facilitate the training of OmniGen2, we developed comprehensive data construction pipelines, encompassing image editing and in-context generation data. Additionally, we introduce a reflection mechanism tailored for image generation tasks and curate a dedicated reflection dataset based on OmniGen2. Despite its relatively modest parameter size, OmniGen2 achieves competitive results on multiple task benchmarks, including text-to-image and image editing. To further evaluate in-context generation, also referred to as subject-driven tasks, we introduce a new benchmark named OmniContext. OmniGen2 achieves state-of-the-art performance among open-source models in terms of consistency. We will release our models, training code, datasets, and data construction pipeline to support future research in this field. Project Page: https://vectorspacelab.github.io/OmniGen2; GitHub Link: https://github.com/VectorSpaceLab/OmniGen2&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.18871</guid>
      <pubDate>Mon, 23 Jun 2025 17:38:54 +0000</pubDate>
    </item>
    <item>
      <title>Light of Normals: Unified Feature Representation for Universal   Photometric Stereo</title>
      <link>https://arxiv.org/abs/2506.18882</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18882.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Hong Li, Houyuan Chen, Chongjie Ye, Zhaoxi Chen, Bohan Li, Shaocong Xu, Xianda Guo, Xuhui Liu, Yikai Wang, Baochang Zhang, Satoshi Ikehata, Boxin Shi, Anyi Rao, Hao Zhao&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 80&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Universal photometric stereo (PS) aims to recover high-quality surface normals from objects under arbitrary lighting conditions without relying on specific illumination models. Despite recent advances such as SDM-UniPS and Uni MS-PS, two fundamental challenges persist: 1) the deep coupling between varying illumination and surface normal features, where ambiguity in observed intensity makes it difficult to determine whether brightness variations stem from lighting changes or surface orientation; and 2) the preservation of high-frequency geometric details in complex surfaces, where intricate geometries create self-shadowing, inter-reflections, and subtle normal variations that conventional feature processing operations struggle to capture accurately.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.18882</guid>
      <pubDate>Mon, 23 Jun 2025 17:53:11 +0000</pubDate>
    </item>
    <item>
      <title>Inverse-and-Edit: Effective and Fast Image Editing by Cycle Consistency   Models</title>
      <link>https://arxiv.org/abs/2506.19103</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19103.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Ilia Beletskii, Andrey Kuznetsov, Aibek Alanov&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 40&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in image editing with diffusion models have achieved impressive results, offering fine-grained control over the generation process. However, these methods are computationally intensive because of their iterative nature. While distilled diffusion models enable faster inference, their editing capabilities remain limited, primarily because of poor inversion quality. High-fidelity inversion and reconstruction are essential for precise image editing, as they preserve the structural and semantic integrity of the source image. In this work, we propose a novel framework that enhances image inversion using consistency models, enabling high-quality editing in just four steps. Our method introduces a cycle-consistency optimization strategy that significantly improves reconstruction accuracy and enables a controllable trade-off between editability and content preservation. We achieve state-of-the-art performance across various image editing tasks and datasets, demonstrating that our method matches or surpasses full-step diffusion models while being substantially more efficient. The code of our method is available on GitHub at https://github.com/ControlGenAI/Inverse-and-Edit.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.19103</guid>
      <pubDate>Mon, 23 Jun 2025 20:34:43 +0000</pubDate>
    </item>
    <item>
      <title>Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in   LLMs</title>
      <link>https://arxiv.org/abs/2506.19290</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19290.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Liang Zeng, Yongcong Li, Yuzhen Xiao, Changshi Li, Chris Yuhao Liu, Rui Yan, Tianwen Wei, Jujie He, Xuchen Song, Yang Liu, Yahui Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 48&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Software engineering (SWE) has recently emerged as a crucial testbed for next-generation LLM agents, demanding inherent capabilities in two critical dimensions: sustained iterative problem-solving (e.g., &gt;50 interaction rounds) and long-context dependency resolution (e.g., &gt;32k tokens). However, the data curation process in SWE remains notoriously time-consuming, as it heavily relies on manual annotation for code file filtering and the setup of dedicated runtime environments to execute and validate unit tests. Consequently, most existing datasets are limited to only a few thousand GitHub-sourced instances. To this end, we propose an incremental, automated data-curation pipeline that systematically scales both the volume and diversity of SWE datasets. Our dataset comprises 10,169 real-world Python task instances from 2,531 distinct GitHub repositories, each accompanied by a task specified in natural language and a dedicated runtime-environment image for automated unit-test validation. We have carefully curated over 8,000 successfully runtime-validated training trajectories from our proposed SWE dataset. When fine-tuning the Skywork-SWE model on these trajectories, we uncover a striking data scaling phenomenon: the trained model's performance for software engineering capabilities in LLMs continues to improve as the data size increases, showing no signs of saturation. Notably, our Skywork-SWE model achieves 38.0% pass@1 accuracy on the SWE-bench Verified benchmark without using verifiers or multiple rollouts, establishing a new state-of-the-art (SOTA) among the Qwen2.5-Coder-32B-based LLMs built on the OpenHands agent framework. Furthermore, with the incorporation of test-time scaling techniques, the performance further improves to 47.0% accuracy, surpassing the previous SOTA results for sub-32B parameter models. We release the Skywork-SWE-32B model checkpoint to accelerate future research.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.19290</guid>
      <pubDate>Tue, 24 Jun 2025 03:53:36 +0000</pubDate>
    </item>
    <item>
      <title>Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large   Language Models</title>
      <link>https://arxiv.org/abs/2506.19697</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19697.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jungwoo Park, Taewhoo Lee, Chanwoong Yoon, Hyeon Hwang, Jaewoo Kang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 43&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Extreme activation outliers in Large Language Models (LLMs) critically degrade quantization performance, hindering efficient on-device deployment. While channel-wise operations and adaptive gradient scaling are recognized causes, practical mitigation remains challenging. We introduce Outlier-Safe Pre-Training (OSP), a practical guideline that proactively prevents outlier formation rather than relying on post-hoc mitigation. OSP combines three key innovations: (1) the Muon optimizer, eliminating privileged bases while maintaining training efficiency; (2) Single-Scale RMSNorm, preventing channel-wise amplification; and (3) a learnable embedding projection, redistributing activation magnitudes originating from embedding matrices. We validate OSP by training a 1.4B-parameter model on 1 trillion tokens, which is the first production-scale LLM trained without such outliers. Under aggressive 4-bit quantization, our OSP model achieves a 35.7 average score across 10 benchmarks (compared to 26.5 for an Adam-trained model), with only a 2% training overhead. Remarkably, OSP models exhibit near-zero excess kurtosis (0.04) compared to extreme values (1818.56) in standard models, fundamentally altering LLM quantization behavior. Our work demonstrates that outliers are not inherent to LLMs but are consequences of training strategies, paving the way for more efficient LLM deployment. The source code and pretrained checkpoints are available at https://github.com/dmis-lab/Outlier-Safe-Pre-Training.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.19697</guid>
      <pubDate>Tue, 24 Jun 2025 15:03:57 +0000</pubDate>
    </item>
    <item>
      <title>AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion   Models</title>
      <link>https://arxiv.org/abs/2506.19851</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19851.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zehuan Huang, Haoran Feng, Yangtian Sun, Yuanchen Guo, Yanpei Cao, Lu Sheng&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 52&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We present AnimaX, a feed-forward 3D animation framework that bridges the motion priors of video diffusion models with the controllable structure of skeleton-based animation. Traditional motion synthesis methods are either restricted to fixed skeletal topologies or require costly optimization in high-dimensional deformation spaces. In contrast, AnimaX effectively transfers video-based motion knowledge to the 3D domain, supporting diverse articulated meshes with arbitrary skeletons. Our method represents 3D motion as multi-view, multi-frame 2D pose maps, and enables joint video-pose diffusion conditioned on template renderings and a textual motion prompt. We introduce shared positional encodings and modality-aware embeddings to ensure spatial-temporal alignment between video and pose sequences, effectively transferring video priors to motion generation task. The resulting multi-view pose sequences are triangulated into 3D joint positions and converted into mesh animation via inverse kinematics. Trained on a newly curated dataset of 160,000 rigged sequences, AnimaX achieves state-of-the-art results on VBench in generalization, motion fidelity, and efficiency, offering a scalable solution for category-agnostic 3D animation. Project page: https://anima-x.github.io/{https://anima-x.github.io/}.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.19851</guid>
      <pubDate>Tue, 24 Jun 2025 17:59:58 +0000</pubDate>
    </item>
    <item>
      <title>OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling</title>
      <link>https://arxiv.org/abs/2506.20512</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.20512.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zengzhi Wang, Fan Zhou, Xuefeng Li, Pengfei Liu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 38&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Different base language model families, such as Llama and Qwen, exhibit divergent behaviors during post-training with reinforcement learning (RL), especially on reasoning-intensive tasks. What makes a base language model suitable for reinforcement learning? Gaining deeper insight into this question is essential for developing RL-scalable foundation models of the next generation. In this work, we investigate how mid-training strategies shape RL dynamics, focusing on two representative model families: Qwen and Llama. Our study reveals that (1) high-quality mathematical corpora, such as MegaMath-Web-Pro, significantly improve both base model and RL performance, while existing alternatives (e.g., FineMath-4plus) fail to do so; (2) further adding QA-style data, particularly long chain-of-thought (CoT) reasoning examples, enhances RL outcomes, and instruction data further unlocks this effect; (3) while long-CoT improves reasoning depth, it can also induce verbosity of model responses and unstability of RL training, underscoring the importance of data formatting; (4) scaling mid-training consistently leads to stronger downstream RL performance. Building on these insights, we introduce a two-stage mid-training strategy, Stable-then-Decay, in which base models are first trained on 200B tokens with a constant learning rate, followed by 20B tokens across three CoT-focused branches with learning rate decay. This yields OctoThinker, a family of models demonstrating strong RL compatibility and closing the performance gap with more RL-friendly model families, i.e., Qwen. We hope our work will help shape pre-training strategies for foundation models in the RL era. To support further research, we release our open-source models along with a curated math reasoning-intensive corpus of over 70 billion tokens (i.e., MegaMath-Web-Pro-Max).&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.20512</guid>
      <pubDate>Wed, 25 Jun 2025 14:58:13 +0000</pubDate>
    </item>
    <item>
      <title>MMSearch-R1: Incentivizing LMMs to Search</title>
      <link>https://arxiv.org/abs/2506.20670</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.20670.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jinming Wu, Zihao Deng, Wei Li, Yiding Liu, Bo You, Bo Li, Zejun Ma, Ziwei Liu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 51&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Robust deployment of large multimodal models (LMMs) in real-world scenarios requires access to external knowledge sources, given the complexity and dynamic nature of real-world information. Existing approaches such as retrieval-augmented generation (RAG) and prompt engineered search agents rely on rigid pipelines, often leading to inefficient or excessive search behaviors. We present MMSearch-R1, the first end-to-end reinforcement learning framework that enables LMMs to perform on-demand, multi-turn search in real-world Internet environments. Our framework integrates both image and text search tools, allowing the model to reason about when and how to invoke them guided by an outcome-based reward with a search penalty. To support training, We collect a multimodal search VQA dataset through a semi-automated pipeline that covers diverse visual and textual knowledge needs and curate a search-balanced subset with both search-required and search-free samples, which proves essential for shaping efficient and on-demand search behavior. Extensive experiments on knowledge-intensive and info-seeking VQA tasks show that our model not only outperforms RAG-based baselines of the same model size, but also matches the performance of a larger RAG-based model while reducing search calls by over 30%. We further analyze key empirical findings to offer actionable insights for advancing research in multimodal search.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.20670</guid>
      <pubDate>Wed, 25 Jun 2025 17:59:42 +0000</pubDate>
    </item>
    <item>
      <title>FaSTA^*: Fast-Slow Toolpath Agent with Subroutine Mining for Efficient   Multi-turn Image Editing</title>
      <link>https://arxiv.org/abs/2506.20911</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.20911.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Advait Gupta, Rishie Raj, Dang Nguyen, Tianyi Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 38&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We develop a cost-efficient neurosymbolic agent to address challenging multi-turn image editing tasks such as "Detect the bench in the image while recoloring it to pink. Also, remove the cat for a clearer view and recolor the wall to yellow.'' It combines the fast, high-level subtask planning by large language models (LLMs) with the slow, accurate, tool-use, and local A^* search per subtask to find a cost-efficient toolpath -- a sequence of calls to AI tools. To save the cost of A^* on similar subtasks, we perform inductive reasoning on previously successful toolpaths via LLMs to continuously extract/refine frequently used subroutines and reuse them as new tools for future tasks in an adaptive fast-slow planning, where the higher-level subroutines are explored first, and only when they fail, the low-level A^* search is activated. The reusable symbolic subroutines considerably save exploration cost on the same types of subtasks applied to similar images, yielding a human-like fast-slow toolpath agent "FaSTA^*'': fast subtask planning followed by rule-based subroutine selection per subtask is attempted by LLMs at first, which is expected to cover most tasks, while slow A^* search is only triggered for novel and challenging subtasks. By comparing with recent image editing approaches, we demonstrate FaSTA^* is significantly more computationally efficient while remaining competitive with the state-of-the-art baseline in terms of success rate.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.20911</guid>
      <pubDate>Thu, 26 Jun 2025 00:33:43 +0000</pubDate>
    </item>
    <item>
      <title>FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data   Processing to Every Language</title>
      <link>https://arxiv.org/abs/2506.20920</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.20920.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Guilherme Penedo, Hynek Kydlíček, Vinko Sabolčec, Bettina Messmer, Negar Foroutan, Amir Hossein Kargaran, Colin Raffel, Martin Jaggi, Leandro Von Werra, Thomas Wolf&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 45&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Pre-training state-of-the-art large language models (LLMs) requires vast amounts of clean and diverse text data. While the open development of large high-quality English pre-training datasets has seen substantial recent progress, training performant multilingual LLMs remains a challenge, in large part due to the inherent difficulty of tailoring filtering and deduplication pipelines to a large number of languages. In this work, we introduce a new pre-training dataset curation pipeline based on FineWeb that can be automatically adapted to support any language. We extensively ablate our pipeline design choices on a set of nine diverse languages, guided by a set of meaningful and informative evaluation tasks that were chosen through a novel selection process based on measurable criteria. Ultimately, we show that our pipeline can be used to create non-English corpora that produce more performant models than prior datasets. We additionally introduce a straightforward and principled approach to rebalance datasets that takes into consideration both duplication count and quality, providing an additional performance uplift. Finally, we scale our pipeline to over 1000 languages using almost 100 Common Crawl snapshots to produce FineWeb2, a new 20 terabyte (5 billion document) multilingual dataset which we release along with our pipeline, training, and evaluation codebases.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.20920</guid>
      <pubDate>Thu, 26 Jun 2025 01:01:47 +0000</pubDate>
    </item>
    <item>
      <title>Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge</title>
      <link>https://arxiv.org/abs/2506.21506</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.21506.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Boyu Gou, Zanming Huang, Yuting Ning, Yu Gu, Michael Lin, Weijian Qi, Andrei Kopanev, Botao Yu, Bernal Jiménez Gutiérrez, Yiheng Shu, Chan Hee Song, Jiaman Wu, Shijie Chen, Hanane Nour Moussa, Tianshu Zhang, Jian Xie, Yifei Li, Tianci Xue, Zeyi Liao, Kai Zhang, Boyuan Zheng, Zhaowei Cai, Viktor Rozgic, Morteza Ziyadi, Huan Sun, Yu Su&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 41&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Agentic search such as Deep Research systems, where large language models autonomously browse the web, synthesize information, and return comprehensive citation-backed answers, represents a major shift in how users interact with web-scale information. While promising greater efficiency and cognitive offloading, the growing complexity and open-endedness of agentic search have outpaced existing evaluation benchmarks and methodologies, which largely assume short search horizons and static answers. In this paper, we introduce Mind2Web 2, a benchmark of 130 realistic, high-quality, and long-horizon tasks that require real-time web browsing and extensive information synthesis, constructed with over 1,000 hours of human labor. To address the challenge of evaluating time-varying and complex answers, we propose a novel Agent-as-a-Judge framework. Our method constructs task-specific judge agents based on a tree-structured rubric design to automatically assess both answer correctness and source attribution. We conduct a comprehensive evaluation of nine frontier agentic search systems and human performance, along with a detailed error analysis to draw insights for future development. The best-performing system, OpenAI Deep Research, can already achieve 50-70% of human performance while spending half the time, showing a great potential. Altogether, Mind2Web 2 provides a rigorous foundation for developing and benchmarking the next generation of agentic search systems.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.21506</guid>
      <pubDate>Thu, 26 Jun 2025 17:32:50 +0000</pubDate>
    </item>
    <item>
      <title>MADrive: Memory-Augmented Driving Scene Modeling</title>
      <link>https://arxiv.org/abs/2506.21520</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.21520.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Polina Karpikova, Daniil Selikhanovych, Kirill Struminsky, Ruslan Musaev, Maria Golitsyna, Dmitry Baranchuk&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 35&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in scene reconstruction have pushed toward highly realistic modeling of autonomous driving (AD) environments using 3D Gaussian splatting. However, the resulting reconstructions remain closely tied to the original observations and struggle to support photorealistic synthesis of significantly altered or novel driving scenarios. This work introduces MADrive, a memory-augmented reconstruction framework designed to extend the capabilities of existing scene reconstruction methods by replacing observed vehicles with visually similar 3D assets retrieved from a large-scale external memory bank. Specifically, we release MAD-Cars, a curated dataset of {sim}70K 360{\deg} car videos captured in the wild and present a retrieval module that finds the most similar car instances in the memory bank, reconstructs the corresponding 3D assets from video, and integrates them into the target scene through orientation alignment and relighting. The resulting replacements provide complete multi-view representations of vehicles in the scene, enabling photorealistic synthesis of substantially altered configurations, as demonstrated in our experiments. Project page: https://yandex-research.github.io/madrive/&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.21520</guid>
      <pubDate>Thu, 26 Jun 2025 17:41:07 +0000</pubDate>
    </item>
    <item>
      <title>WorldVLA: Towards Autoregressive Action World Model</title>
      <link>https://arxiv.org/abs/2506.21539</link>
      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.21539.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Jun Cen, Chaohui Yu, Hangjie Yuan, Yuming Jiang, Siteng Huang, Jiayan Guo, Xin Li, Yibing Song, Hao Luo, Fan Wang, Deli Zhao, Hao Chen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 34&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We present WorldVLA, an autoregressive action world model that unifies action and image understanding and generation. Our WorldVLA intergrates Vision-Language-Action (VLA) model and world model in one single framework. The world model predicts future images by leveraging both action and image understanding, with the purpose of learning the underlying physics of the environment to improve action generation. Meanwhile, the action model generates the subsequent actions based on image observations, aiding in visual understanding and in turn helps visual generation of the world model. We demonstrate that WorldVLA outperforms standalone action and world models, highlighting the mutual enhancement between the world model and the action model. In addition, we find that the performance of the action model deteriorates when generating sequences of actions in an autoregressive manner. This phenomenon can be attributed to the model's limited generalization capability for action prediction, leading to the propagation of errors from earlier actions to subsequent ones. To address this issue, we propose an attention mask strategy that selectively masks prior actions during the generation of the current action, which shows significant performance improvement in the action chunk generation task.&lt;/p&gt;</description>
      <guid isPermaLink="false">https://arxiv.org/abs/2506.21539</guid>
      <pubDate>Thu, 26 Jun 2025 17:55:40 +0000</pubDate>
    </item>
  </channel>
</rss>