bot: update RSS feed
This commit is contained in:
github-actions[bot]
154
feed.xml
154
feed.xml
@@ -7,125 +7,83 @@
|
||||
<docs>http://www.rssboard.org/rss-specification</docs>
|
||||
<generator>python-feedgen</generator>
|
||||
<language>en</language>
|
||||
<lastBuildDate>Thu, 11 Sep 2025 00:02:53 +0000</lastBuildDate>
|
||||
<lastBuildDate>Fri, 12 Sep 2025 00:02:36 +0000</lastBuildDate>
|
||||
<item>
|
||||
<title>Q-Sched: Pushing the Boundaries of Few-Step Diffusion Models with Quantization-Aware Scheduling</title>
|
||||
<link>https://arxiv.org/abs/2509.01624</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.01624.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Natalia Frumkin, Diana Marculescu</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> Text-to-image diffusion models are computationally intensive, often requiring dozens of forward passes through large transformer backbones. For instance, Stable Diffusion XL generates high-quality images with 50 evaluations of a 2.6B-parameter model, an expensive process even for a single batch. Few-step diffusion models reduce this cost to 2-8 denoising steps but still depend on large, uncompressed U-Net or diffusion transformer backbones, which are often too costly for full-precision inference without datacenter GPUs. These requirements also limit existing post-training quantization methods that rely on full-precision calibration. We introduce Q-Sched, a new paradigm for post-training quantization that modifies the diffusion model scheduler rather than model weights. By adjusting the few-step sampling trajectory, Q-Sched achieves full-precision accuracy with a 4x reduction in model size. To learn quantization-aware pre-conditioning coefficients, we propose the JAQ loss, which combines text-image compatibility with an image quality metric for fine-grained optimization. JAQ is reference-free and requires only a handful of calibration prompts, avoiding full-precision inference during calibration. Q-Sched delivers substantial gains: a 15.5% FID improvement over the FP16 4-step Latent Consistency Model and a 16.6% improvement over the FP16 8-step Phased Consistency Model, showing that quantization and few-step distillation are complementary for high-fidelity generation. A large-scale user study with more than 80,000 annotations further confirms Q-Sched's effectiveness on both FLUX.1[schnell] and SDXL-Turbo.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.01624</guid>
|
||||
<pubDate>Mon, 01 Sep 2025 17:09:22 +0000</pubDate>
|
||||
<title>3D and 4D World Modeling: A Survey</title>
|
||||
<link>https://arxiv.org/abs/2509.07996</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.07996.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Lingdong Kong, Wesley Yang, Jianbiao Mei, Youquan Liu, Ao Liang, Dekai Zhu, Dongyue Lu, Wei Yin, Xiaotao Hu, Mingkai Jia, Junyuan Deng, Kaiwen Zhang, Yang Wu, Tianyi Yan, Shenyuan Gao, Song Wang, Linfeng Li, Liang Pan, Yong Liu, Jianke Zhu, Wei Tsang Ooi, Steven C. H. Hoi, Ziwei Liu</p><p><b>Upvotes:</b> 38</p><p><b>Summary:</b> World modeling has become a cornerstone in AI research, enabling agents to understand, represent, and predict the dynamic environments they inhabit. While prior work largely emphasizes generative methods for 2D image and video data, they overlook the rapidly growing body of work that leverages native 3D and 4D representations such as RGB-D imagery, occupancy grids, and LiDAR point clouds for large-scale scene modeling. At the same time, the absence of a standardized definition and taxonomy for ``world models'' has led to fragmented and sometimes inconsistent claims in the literature. This survey addresses these gaps by presenting the first comprehensive review explicitly dedicated to 3D and 4D world modeling and generation. We establish precise definitions, introduce a structured taxonomy spanning video-based (VideoGen), occupancy-based (OccGen), and LiDAR-based (LiDARGen) approaches, and systematically summarize datasets and evaluation metrics tailored to 3D/4D settings. We further discuss practical applications, identify open challenges, and highlight promising research directions, aiming to provide a coherent and foundational reference for advancing the field. A systematic summary of existing literature is available at https://github.com/worldbench/survey</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.07996</guid>
|
||||
<pubDate>Thu, 04 Sep 2025 17:59:58 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>Emergent Hierarchical Reasoning in LLMs through Reinforcement Learning</title>
|
||||
<link>https://arxiv.org/abs/2509.03646</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.03646.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Haozhe Wang, Qixin Xu, Che Liu, Junhong Wu, Fangzhen Lin, Wenhu Chen</p><p><b>Upvotes:</b> 13</p><p><b>Summary:</b> Reinforcement Learning (RL) has proven highly effective at enhancing the complex reasoning abilities of Large Language Models (LLMs), yet underlying mechanisms driving this success remain largely opaque. Our analysis reveals that puzzling phenomena like ``aha moments", ``length-scaling'' and entropy dynamics are not disparate occurrences but hallmarks of an emergent reasoning hierarchy, akin to the separation of high-level strategic planning from low-level procedural execution in human cognition. We uncover a compelling two-phase dynamic: initially, a model is constrained by procedural correctness and must improve its low-level skills. The learning bottleneck then decisively shifts, with performance gains being driven by the exploration and mastery of high-level strategic planning. This insight exposes a core inefficiency in prevailing RL algorithms like GRPO, which apply optimization pressure agnostically and dilute the learning signal across all tokens. To address this, we propose HIerarchy-Aware Credit Assignment (HICRA), an algorithm that concentrates optimization efforts on high-impact planning tokens. HICRA significantly outperforms strong baselines, demonstrating that focusing on this strategic bottleneck is key to unlocking advanced reasoning. Furthermore, we validate semantic entropy as a superior compass for measuring strategic exploration over misleading metrics such as token-level entropy.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.03646</guid>
|
||||
<pubDate>Wed, 03 Sep 2025 18:52:49 +0000</pubDate>
|
||||
<title>Hunyuan-MT Technical Report</title>
|
||||
<link>https://arxiv.org/abs/2509.05209</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.05209.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Mao Zheng, Zheng Li, Bingxin Qu, Mingyang Song, Yang Du, Mingrui Sun, Di Wang</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> In this report, we introduce Hunyuan-MT-7B, our first open-source multilingual translation model, which supports bidirectional translation across 33 major languages and places a special emphasis on translation between Mandarin and several ethnic minority languages as well as dialects. Furthermore, to serve and address diverse translation scenarios and enhance model performance at test time, we introduce Hunyuan-MT-Chimera-7B, a translation model inspired by the slow thinking mode. This model integrates multiple outputs generated by the Hunyuan-MT-7B model under varying parameter settings, thereby achieving performance superior to that of conventional slow-thinking models based on Chain-of-Thought (CoT). The development of our models follows a holistic training process specifically engineered for multilingual translation, which begins with general and MT-oriented pre-training to build foundational capabilities, proceeds to Supervised Fine-Tuning (SFT) for task-specific adaptation, and culminates in advanced alignment through Reinforcement Learning (RL) and weak-to-strong RL. Through comprehensive experimentation, we demonstrate that both Hunyuan-MT-7B and Hunyuan-MT-Chimera-7B significantly outperform all translation-specific models of comparable parameter size and most of the SOTA large models, particularly on the task of translation between Mandarin and minority languages as well as dialects. In the WMT2025 shared task (General Machine Translation), our models demonstrate state-of-the-art performance, ranking first in 30 out of 31 language pairs. This result highlights the robustness of our models across a diverse linguistic spectrum, encompassing high-resource languages such as Chinese, English, and Japanese, as well as low-resource languages including Czech, Marathi, Estonian, and Icelandic.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.05209</guid>
|
||||
<pubDate>Fri, 05 Sep 2025 16:11:05 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Reward</title>
|
||||
<link>https://arxiv.org/abs/2509.06818</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.06818.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yufeng Cheng, Wenxu Wu, Shaojin Wu, Mengqi Huang, Fei Ding, Qian He</p><p><b>Upvotes:</b> 23</p><p><b>Summary:</b> Recent advancements in image customization exhibit a wide range of application prospects due to stronger customization capabilities. However, since we humans are more sensitive to faces, a significant challenge remains in preserving consistent identity while avoiding identity confusion with multi-reference images, limiting the identity scalability of customization models. To address this, we present UMO, a Unified Multi-identity Optimization framework, designed to maintain high-fidelity identity preservation and alleviate identity confusion with scalability. With "multi-to-multi matching" paradigm, UMO reformulates multi-identity generation as a global assignment optimization problem and unleashes multi-identity consistency for existing image customization methods generally through reinforcement learning on diffusion models. To facilitate the training of UMO, we develop a scalable customization dataset with multi-reference images, consisting of both synthesised and real parts. Additionally, we propose a new metric to measure identity confusion. Extensive experiments demonstrate that UMO not only improves identity consistency significantly, but also reduces identity confusion on several image customization methods, setting a new state-of-the-art among open-source methods along the dimension of identity preserving. Code and model: https://github.com/bytedance/UMO</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.06818</guid>
|
||||
<pubDate>Mon, 08 Sep 2025 15:54:55 +0000</pubDate>
|
||||
<title>P3-SAM: Native 3D Part Segmentation</title>
|
||||
<link>https://arxiv.org/abs/2509.06784</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.06784.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Changfeng Ma, Yang Li, Xinhao Yan, Jiachen Xu, Yunhan Yang, Chunshi Wang, Zibo Zhao, Yanwen Guo, Zhuo Chen, Chunchao Guo</p><p><b>Upvotes:</b> 13</p><p><b>Summary:</b> Segmenting 3D assets into their constituent parts is crucial for enhancing 3D understanding, facilitating model reuse, and supporting various applications such as part generation. However, current methods face limitations such as poor robustness when dealing with complex objects and cannot fully automate the process. In this paper, we propose a native 3D point-promptable part segmentation model termed P3-SAM, designed to fully automate the segmentation of any 3D objects into components. Inspired by SAM, P3-SAM consists of a feature extractor, multiple segmentation heads, and an IoU predictor, enabling interactive segmentation for users. We also propose an algorithm to automatically select and merge masks predicted by our model for part instance segmentation. Our model is trained on a newly built dataset containing nearly 3.7 million models with reasonable segmentation labels. Comparisons show that our method achieves precise segmentation results and strong robustness on any complex objects, attaining state-of-the-art performance. Our code will be released soon.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.06784</guid>
|
||||
<pubDate>Mon, 08 Sep 2025 15:12:17 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>Curia: A Multi-Modal Foundation Model for Radiology</title>
|
||||
<link>https://arxiv.org/abs/2509.06830</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.06830.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Corentin Dancette, Julien Khlaut, Antoine Saporta, Helene Philippe, Elodie Ferreres, Baptiste Callard, Théo Danielou, Léo Alberge, Léo Machado, Daniel Tordjman, Julie Dupuis, Korentin Le Floch, Jean Du Terrail, Mariam Moshiri, Laurent Dercle, Tom Boeken, Jules Gregory, Maxime Ronot, François Legou, Pascal Roux, Marc Sapoval, Pierre Manceron, Paul Hérent</p><p><b>Upvotes:</b> 17</p><p><b>Summary:</b> AI-assisted radiological interpretation is based on predominantly narrow, single-task models. This approach is impractical for covering the vast spectrum of imaging modalities, diseases, and radiological findings. Foundation models (FMs) hold the promise of broad generalization across modalities and in low-data settings. However, this potential has remained largely unrealized in radiology. We introduce Curia, a foundation model trained on the entire cross-sectional imaging output of a major hospital over several years, which to our knowledge is the largest such corpus of real-world data-encompassing 150,000 exams (130 TB). On a newly curated 19-task external validation benchmark, Curia accurately identifies organs, detects conditions like brain hemorrhages and myocardial infarctions, and predicts outcomes in tumor staging. Curia meets or surpasses the performance of radiologists and recent foundation models, and exhibits clinically significant emergent properties in cross-modality, and low-data regimes. To accelerate progress, we release our base model's weights at https://huggingface.co/raidium/curia.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.06830</guid>
|
||||
<pubDate>Mon, 08 Sep 2025 16:04:12 +0000</pubDate>
|
||||
<title>The Majority is not always right: RL training for solution aggregation</title>
|
||||
<link>https://arxiv.org/abs/2509.06870</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.06870.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Wenting Zhao, Pranjal Aggarwal, Swarnadeep Saha, Asli Celikyilmaz, Jason Weston, Ilia Kulikov</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> Scaling up test-time compute, by generating multiple independent solutions and selecting or aggregating among them, has become a central paradigm for improving large language models (LLMs) on challenging reasoning tasks. While most prior work relies on simple majority voting or reward model ranking to aggregate solutions, these approaches may only yield limited benefits. In this work, we propose to learn aggregation as an explicit reasoning skill: given a set of candidate solutions, we train an aggregator model to review, reconcile, and synthesize a final, correct answer using reinforcement learning from verifiable rewards. A key ingredient is careful balancing of easy and hard training examples, allowing the model to learn both to recover minority-but-correct answers as well as easy majority-correct answers. Empirically, we find our method, AggLM, outperforms both strong rule-based and reward-model baselines, across multiple benchmarks. Furthermore, it generalizes effectively to solutions from differing models, including stronger ones than contained in the training data, all while requiring substantially fewer tokens than majority voting with larger numbers of solutions.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.06870</guid>
|
||||
<pubDate>Mon, 08 Sep 2025 16:39:38 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>Staying in the Sweet Spot: Responsive Reasoning Evolution via Capability-Adaptive Hint Scaffolding</title>
|
||||
<link>https://arxiv.org/abs/2509.06923</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.06923.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Ziheng Li, Zexu Sun, Jinman Zhao, Erxue Min, Yongcheng Zeng, Hui Wu, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Xu Chen, Zhi-Hong Deng</p><p><b>Upvotes:</b> 17</p><p><b>Summary:</b> Reinforcement learning with verifiable rewards (RLVR) has achieved remarkable success in enhancing the reasoning capabilities of large language models (LLMs). However, existing RLVR methods often suffer from exploration inefficiency due to mismatches between the training data's difficulty and the model's capability. LLMs fail to discover viable reasoning paths when problems are overly difficult, while learning little new capability when problems are too simple. In this work, we formalize the impact of problem difficulty by quantifying the relationship between loss descent speed and rollout accuracy. Building on this analysis, we propose SEELE, a novel supervision-aided RLVR framework that dynamically adjusts problem difficulty to stay within the high-efficiency region. SEELE augments each training sample by appending a hint (part of a full solution) after the original problem. Unlike previous hint-based approaches, SEELE deliberately and adaptively adjusts the hint length for each problem to achieve an optimal difficulty. To determine the optimal hint length, SEELE employs a multi-round rollout sampling strategy. In each round, it fits an item response theory model to the accuracy-hint pairs collected in preceding rounds to predict the required hint length for the next round. This instance-level, real-time difficulty adjustment aligns problem difficulty with the evolving model capability, thereby improving exploration efficiency. Experimental results show that SEELE outperforms Group Relative Policy Optimization (GRPO) and Supervised Fine-tuning (SFT) by +11.8 and +10.5 points, respectively, and surpasses the best previous supervision-aided approach by +3.6 points on average across six math reasoning benchmarks.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.06923</guid>
|
||||
<pubDate>Mon, 08 Sep 2025 17:36:21 +0000</pubDate>
|
||||
<title>Statistical Methods in Generative AI</title>
|
||||
<link>https://arxiv.org/abs/2509.07054</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.07054.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Edgar Dobriban</p><p><b>Upvotes:</b> 2</p><p><b>Summary:</b> Generative Artificial Intelligence is emerging as an important technology, promising to be transformative in many areas. At the same time, generative AI techniques are based on sampling from probabilistic models, and by default, they come with no guarantees about correctness, safety, fairness, or other properties. Statistical methods offer a promising potential approach to improve the reliability of generative AI techniques. In addition, statistical methods are also promising for improving the quality and efficiency of AI evaluation, as well as for designing interventions and experiments in AI. In this paper, we review some of the existing work on these topics, explaining both the general statistical techniques used, as well as their applications to generative AI. We also discuss limitations and potential future directions.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.07054</guid>
|
||||
<pubDate>Mon, 08 Sep 2025 17:42:59 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>From Noise to Narrative: Tracing the Origins of Hallucinations in Transformers</title>
|
||||
<link>https://arxiv.org/abs/2509.06938</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.06938.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Praneet Suresh, Jack Stanley, Sonia Joseph, Luca Scimeca, Danilo Bzdok</p><p><b>Upvotes:</b> 1</p><p><b>Summary:</b> As generative AI systems become competent and democratized in science, business, and government, deeper insight into their failure modes now poses an acute need. The occasional volatility in their behavior, such as the propensity of transformer models to hallucinate, impedes trust and adoption of emerging AI solutions in high-stakes areas. In the present work, we establish how and when hallucinations arise in pre-trained transformer models through concept representations captured by sparse autoencoders, under scenarios with experimentally controlled uncertainty in the input space. Our systematic experiments reveal that the number of semantic concepts used by the transformer model grows as the input information becomes increasingly unstructured. In the face of growing uncertainty in the input space, the transformer model becomes prone to activate coherent yet input-insensitive semantic features, leading to hallucinated output. At its extreme, for pure-noise inputs, we identify a wide variety of robustly triggered and meaningful concepts in the intermediate activations of pre-trained transformer models, whose functional integrity we confirm through targeted steering. We also show that hallucinations in the output of a transformer model can be reliably predicted from the concept patterns embedded in transformer layer activations. This collection of insights on transformer internal processing mechanics has immediate consequences for aligning AI models with human values, AI safety, opening the attack surface for potential adversarial attacks, and providing a basis for automatic quantification of a model's hallucination risk.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.06938</guid>
|
||||
<pubDate>Mon, 08 Sep 2025 17:50:45 +0000</pubDate>
|
||||
<title>EnvX: Agentize Everything with Agentic AI</title>
|
||||
<link>https://arxiv.org/abs/2509.08088</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.08088.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Linyao Chen, Zimian Peng, Yingxuan Yang, Yikun Wang, Wenzheng Tom Tang, Hiroki H. Kobayashi, Weinan Zhang</p><p><b>Upvotes:</b> 2</p><p><b>Summary:</b> The widespread availability of open-source repositories has led to a vast collection of reusable software components, yet their utilization remains manual, error-prone, and disconnected. Developers must navigate documentation, understand APIs, and write integration code, creating significant barriers to efficient software reuse. To address this, we present EnvX, a framework that leverages Agentic AI to agentize GitHub repositories, transforming them into intelligent, autonomous agents capable of natural language interaction and inter-agent collaboration. Unlike existing approaches that treat repositories as static code resources, EnvX reimagines them as active agents through a three-phase process: (1) TODO-guided environment initialization, which sets up the necessary dependencies, data, and validation datasets; (2) human-aligned agentic automation, allowing repository-specific agents to autonomously perform real-world tasks; and (3) Agent-to-Agent (A2A) protocol, enabling multiple agents to collaborate. By combining large language model capabilities with structured tool integration, EnvX automates not just code generation, but the entire process of understanding, initializing, and operationalizing repository functionality. We evaluate EnvX on the GitTaskBench benchmark, using 18 repositories across domains such as image processing, speech recognition, document analysis, and video manipulation. Our results show that EnvX achieves a 74.07% execution completion rate and 51.85% task pass rate, outperforming existing frameworks. Case studies further demonstrate EnvX's ability to enable multi-repository collaboration via the A2A protocol. This work marks a shift from treating repositories as passive code resources to intelligent, interactive agents, fostering greater accessibility and collaboration within the open-source ecosystem.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.08088</guid>
|
||||
<pubDate>Tue, 09 Sep 2025 18:51:36 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>Directly Aligning the Full Diffusion Trajectory with Fine-Grained Human Preference</title>
|
||||
<link>https://arxiv.org/abs/2509.06942</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.06942.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Xiangwei Shen, Zhimin Li, Zhantao Yang, Shiyi Zhang, Yingfang Zhang, Donghao Li, Chunyu Wang, Qinglin Lu, Yansong Tang</p><p><b>Upvotes:</b> 6</p><p><b>Summary:</b> Recent studies have demonstrated the effectiveness of directly aligning diffusion models with human preferences using differentiable reward. However, they exhibit two primary challenges: (1) they rely on multistep denoising with gradient computation for reward scoring, which is computationally expensive, thus restricting optimization to only a few diffusion steps; (2) they often need continuous offline adaptation of reward models in order to achieve desired aesthetic quality, such as photorealism or precise lighting effects. To address the limitation of multistep denoising, we propose Direct-Align, a method that predefines a noise prior to effectively recover original images from any time steps via interpolation, leveraging the equation that diffusion states are interpolations between noise and target images, which effectively avoids over-optimization in late timesteps. Furthermore, we introduce Semantic Relative Preference Optimization (SRPO), in which rewards are formulated as text-conditioned signals. This approach enables online adjustment of rewards in response to positive and negative prompt augmentation, thereby reducing the reliance on offline reward fine-tuning. By fine-tuning the FLUX.1.dev model with optimized denoising and online reward adjustment, we improve its human-evaluated realism and aesthetic quality by over 3x.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.06942</guid>
|
||||
<pubDate>Mon, 08 Sep 2025 17:54:08 +0000</pubDate>
|
||||
<title><think> So let's replace this phrase with insult... </think> Lessons learned from generation of toxic texts with LLMs</title>
|
||||
<link>https://arxiv.org/abs/2509.08358</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.08358.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Sergey Pletenev, Daniil Moskovskiy, Alexander Panchenko</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> Modern Large Language Models (LLMs) are excellent at generating synthetic data. However, their performance in sensitive domains such as text detoxification has not received proper attention from the scientific community. This paper explores the possibility of using LLM-generated synthetic toxic data as an alternative to human-generated data for training models for detoxification. Using Llama 3 and Qwen activation-patched models, we generated synthetic toxic counterparts for neutral texts from ParaDetox and SST-2 datasets. Our experiments show that models fine-tuned on synthetic data consistently perform worse than those trained on human data, with a drop in performance of up to 30% in joint metrics. The root cause is identified as a critical lexical diversity gap: LLMs generate toxic content using a small, repetitive vocabulary of insults that fails to capture the nuances and variety of human toxicity. These findings highlight the limitations of current LLMs in this domain and emphasize the continued importance of diverse, human-annotated data for building robust detoxification systems.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.08358</guid>
|
||||
<pubDate>Wed, 10 Sep 2025 07:48:24 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actions</title>
|
||||
<link>https://arxiv.org/abs/2509.06951</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.06951.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Qi Lv, Weijie Kong, Hao Li, Jia Zeng, Zherui Qiu, Delin Qu, Haoming Song, Qizhi Chen, Xiang Deng, Jiangmiao Pang</p><p><b>Upvotes:</b> 19</p><p><b>Summary:</b> Executing language-conditioned tasks in dynamic visual environments remains a central challenge in embodied AI. Existing Vision-Language-Action (VLA) models predominantly adopt reactive state-to-action mappings, often leading to short-sighted behaviors and poor robustness in dynamic scenes. In this paper, we introduce F1, a pretrained VLA framework which integrates the visual foresight generation into decision-making pipeline. F1 adopts a Mixture-of-Transformer architecture with dedicated modules for perception, foresight generation, and control, thereby bridging understanding, generation, and actions. At its core, F1 employs a next-scale prediction mechanism to synthesize goal-conditioned visual foresight as explicit planning targets. By forecasting plausible future visual states, F1 reformulates action generation as a foresight-guided inverse dynamics problem, enabling actions that implicitly achieve visual goals. To endow F1 with robust and generalizable capabilities, we propose a three-stage training recipe on an extensive dataset comprising over 330k trajectories across 136 diverse tasks. This training scheme enhances modular reasoning and equips the model with transferable visual foresight, which is critical for complex and dynamic environments. Extensive evaluations on real-world tasks and simulation benchmarks demonstrate F1 consistently outperforms existing approaches, achieving substantial gains in both task success rate and generalization ability.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.06951</guid>
|
||||
<pubDate>Mon, 08 Sep 2025 17:58:30 +0000</pubDate>
|
||||
<title>HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants</title>
|
||||
<link>https://arxiv.org/abs/2509.08494</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.08494.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Benjamin Sturgeon, Daniel Samuelson, Jacob Haimes, Jacy Reese Anthis</p><p><b>Upvotes:</b> 0</p><p><b>Summary:</b> As humans delegate more tasks and decisions to artificial intelligence (AI), we risk losing control of our individual and collective futures. Relatively simple algorithmic systems already steer human decision-making, such as social media feed algorithms that lead people to unintentionally and absent-mindedly scroll through engagement-optimized content. In this paper, we develop the idea of human agency by integrating philosophical and scientific theories of agency with AI-assisted evaluation methods: using large language models (LLMs) to simulate and validate user queries and to evaluate AI responses. We develop HumanAgencyBench (HAB), a scalable and adaptive benchmark with six dimensions of human agency based on typical AI use cases. HAB measures the tendency of an AI assistant or agent to Ask Clarifying Questions, Avoid Value Manipulation, Correct Misinformation, Defer Important Decisions, Encourage Learning, and Maintain Social Boundaries. We find low-to-moderate agency support in contemporary LLM-based assistants and substantial variation across system developers and dimensions. For example, while Anthropic LLMs most support human agency overall, they are the least supportive LLMs in terms of Avoid Value Manipulation. Agency support does not appear to consistently result from increasing LLM capabilities or instruction-following behavior (e.g., RLHF), and we encourage a shift towards more robust safety and alignment targets.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.08494</guid>
|
||||
<pubDate>Wed, 10 Sep 2025 11:10:10 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>Benchmarking Information Retrieval Models on Complex Retrieval Tasks</title>
|
||||
<link>https://arxiv.org/abs/2509.07253</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.07253.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Julian Killingback, Hamed Zamani</p><p><b>Upvotes:</b> 2</p><p><b>Summary:</b> Large language models (LLMs) are incredible and versatile tools for text-based tasks that have enabled countless, previously unimaginable, applications. Retrieval models, in contrast, have not yet seen such capable general-purpose models emerge. To achieve this goal, retrieval models must be able to perform complex retrieval tasks, where queries contain multiple parts, constraints, or requirements in natural language. These tasks represent a natural progression from the simple, single-aspect queries that are used in the vast majority of existing, commonly used evaluation sets. Complex queries naturally arise as people expect search systems to handle more specific and often ambitious information requests, as is demonstrated by how people use LLM-based information systems. Despite the growing desire for retrieval models to expand their capabilities in complex retrieval tasks, there exist limited resources to assess the ability of retrieval models on a comprehensive set of diverse complex tasks. The few resources that do exist feature a limited scope and often lack realistic settings making it hard to know the true capabilities of retrieval models on complex real-world retrieval tasks. To address this shortcoming and spur innovation in next-generation retrieval models, we construct a diverse and realistic set of complex retrieval tasks and benchmark a representative set of state-of-the-art retrieval models. Additionally, we explore the impact of LLM-based query expansion and rewriting on retrieval quality. Our results show that even the best models struggle to produce high-quality retrieval results with the highest average nDCG@10 of only 0.346 and R@100 of only 0.587 across all tasks. Although LLM augmentation can help weaker models, the strongest model has decreased performance across all metrics with all rewriting techniques.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.07253</guid>
|
||||
<pubDate>Mon, 08 Sep 2025 22:11:10 +0000</pubDate>
|
||||
<title>AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning</title>
|
||||
<link>https://arxiv.org/abs/2509.08755</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.08755.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zhiheng Xi, Jixuan Huang, Chenyang Liao, Baodai Huang, Honglin Guo, Jiaqi Liu, Rui Zheng, Junjie Ye, Jiazheng Zhang, Wenxiang Chen, Wei He, Yiwen Ding, Guanyu Li, Zehui Chen, Zhengyin Du, Xuesong Yao, Yufei Xu, Jiecao Chen, Tao Gui, Zuxuan Wu, Qi Zhang, Xuanjing Huang, Yu-Gang Jiang</p><p><b>Upvotes:</b> 19</p><p><b>Summary:</b> Developing autonomous LLM agents capable of making a series of intelligent decisions to solve complex, real-world tasks is a fast-evolving frontier. Like human cognitive development, agents are expected to acquire knowledge and skills through exploration and interaction with the environment. Despite advances, the community still lacks a unified, interactive reinforcement learning (RL) framework that can effectively train such agents from scratch -- without relying on supervised fine-tuning (SFT) -- across diverse and realistic environments. To bridge this gap, we introduce AgentGym-RL, a new framework to train LLM agents for multi-turn interactive decision-making through RL. The framework features a modular and decoupled architecture, ensuring high flexibility and extensibility. It encompasses a wide variety of real-world scenarios, and supports mainstream RL algorithms. Furthermore, we propose ScalingInter-RL, a training approach designed for exploration-exploitation balance and stable RL optimization. In early stages, it emphasizes exploitation by restricting the number of interactions, and gradually shifts towards exploration with larger horizons to encourage diverse problem-solving strategies. In this way, the agent develops more diverse behaviors and is less prone to collapse under long horizons. We perform extensive experiments to validate the stability and effectiveness of both the AgentGym-RL framework and the ScalingInter-RL approach. Our agents match or surpass commercial models on 27 tasks across diverse environments. We offer key insights and will open-source the complete AgentGym-RL framework -- including code and datasets -- to empower the research community in developing the next generation of intelligent agents.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.08755</guid>
|
||||
<pubDate>Wed, 10 Sep 2025 16:46:11 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>Reconstruction Alignment Improves Unified Multimodal Models</title>
|
||||
<link>https://arxiv.org/abs/2509.07295</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.07295.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Ji Xie, Trevor Darrell, Luke Zettlemoyer, XuDong Wang</p><p><b>Upvotes:</b> 31</p><p><b>Summary:</b> Unified multimodal models (UMMs) unify visual understanding and generation within a single architecture. However, conventional training relies on image-text pairs (or sequences) whose captions are typically sparse and miss fine-grained visual details--even when they use hundreds of words to describe a simple image. We introduce Reconstruction Alignment (RecA), a resource-efficient post-training method that leverages visual understanding encoder embeddings as dense "text prompts," providing rich supervision without captions. Concretely, RecA conditions a UMM on its own visual understanding embeddings and optimizes it to reconstruct the input image with a self-supervised reconstruction loss, thereby realigning understanding and generation. Despite its simplicity, RecA is broadly applicable: across autoregressive, masked-autoregressive, and diffusion-based UMMs, it consistently improves generation and editing fidelity. With only 27 GPU-hours, post-training with RecA substantially improves image generation performance on GenEval (0.73rightarrow0.90) and DPGBench (80.93rightarrow88.15), while also boosting editing benchmarks (ImgEdit 3.38rightarrow3.75, GEdit 6.94rightarrow7.25). Notably, RecA surpasses much larger open-source models and applies broadly across diverse UMM architectures, establishing it as an efficient and general post-training alignment strategy for UMMs</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.07295</guid>
|
||||
<pubDate>Mon, 08 Sep 2025 23:59:32 +0000</pubDate>
|
||||
<title>RewardDance: Reward Scaling in Visual Generation</title>
|
||||
<link>https://arxiv.org/abs/2509.08826</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.08826.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jie Wu, Yu Gao, Zilyu Ye, Ming Li, Liang Li, Hanzhong Guo, Jie Liu, Zeyue Xue, Xiaoxia Hou, Wei Liu, Yan Zeng, Weilin Huang</p><p><b>Upvotes:</b> 49</p><p><b>Summary:</b> Reward Models (RMs) are critical for improving generation models via Reinforcement Learning (RL), yet the RM scaling paradigm in visual generation remains largely unexplored. It primarily due to fundamental limitations in existing approaches: CLIP-based RMs suffer from architectural and input modality constraints, while prevalent Bradley-Terry losses are fundamentally misaligned with the next-token prediction mechanism of Vision-Language Models (VLMs), hindering effective scaling. More critically, the RLHF optimization process is plagued by Reward Hacking issue, where models exploit flaws in the reward signal without improving true quality. To address these challenges, we introduce RewardDance, a scalable reward modeling framework that overcomes these barriers through a novel generative reward paradigm. By reformulating the reward score as the model's probability of predicting a "yes" token, indicating that the generated image outperforms a reference image according to specific criteria, RewardDance intrinsically aligns reward objectives with VLM architectures. This alignment unlocks scaling across two dimensions: (1) Model Scaling: Systematic scaling of RMs up to 26 billion parameters; (2) Context Scaling: Integration of task-specific instructions, reference examples, and chain-of-thought (CoT) reasoning. Extensive experiments demonstrate that RewardDance significantly surpasses state-of-the-art methods in text-to-image, text-to-video, and image-to-video generation. Crucially, we resolve the persistent challenge of "reward hacking": Our large-scale RMs exhibit and maintain high reward variance during RL fine-tuning, proving their resistance to hacking and ability to produce diverse, high-quality outputs. It greatly relieves the mode collapse problem that plagues smaller models.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.08826</guid>
|
||||
<pubDate>Wed, 10 Sep 2025 17:59:31 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>Causal Attention with Lookahead Keys</title>
|
||||
<link>https://arxiv.org/abs/2509.07301</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.07301.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zhuoqing Song, Peng Sun, Huizhuo Yuan, Quanquan Gu</p><p><b>Upvotes:</b> 10</p><p><b>Summary:</b> In standard causal attention, each token's query, key, and value (QKV) are static and encode only preceding context. We introduce CAuSal aTtention with Lookahead kEys (CASTLE), an attention mechanism that continually updates each token's keys as the context unfolds. We term these updated keys lookahead keys because they belong to earlier positions yet integrate information from tokens that appear later relative to those positions, while strictly preserving the autoregressive property. Although the mechanism appears sequential, we derive a mathematical equivalence that avoids explicitly materializing lookahead keys at each position and enables efficient parallel training. On language modeling benchmarks, CASTLE consistently outperforms standard causal attention across model scales, reducing validation perplexity and improving performance on a range of downstream tasks.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.07301</guid>
|
||||
<pubDate>Tue, 09 Sep 2025 00:15:23 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>Language Self-Play For Data-Free Training</title>
|
||||
<link>https://arxiv.org/abs/2509.07414</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.07414.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jakub Grudzien Kuba, Mengting Gu, Qi Ma, Yuandong Tian, Vijai Mohan</p><p><b>Upvotes:</b> 19</p><p><b>Summary:</b> Large language models (LLMs) have advanced rapidly in recent years, driven by scale, abundant high-quality training data, and reinforcement learning. Yet this progress faces a fundamental bottleneck: the need for ever more data from which models can continue to learn. In this work, we propose a reinforcement learning approach that removes this dependency by enabling models to improve without additional data. Our method leverages a game-theoretic framework of self-play, where a model's capabilities are cast as performance in a competitive game and stronger policies emerge by having the model play against itself - a process we call Language Self-Play (LSP). Experiments with Llama-3.2-3B-Instruct on instruction-following benchmarks show that pretrained models can not only enhance their performance on challenging tasks through self-play alone, but can also do so more effectively than data-driven baselines.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.07414</guid>
|
||||
<pubDate>Tue, 09 Sep 2025 05:51:34 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>ΔL Normalization: Rethink Loss Aggregation in RLVR</title>
|
||||
<link>https://arxiv.org/abs/2509.07558</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.07558.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zhiyuan He, Xufang Luo, Yike Zhang, Yuqing Yang, Lili Qiu</p><p><b>Upvotes:</b> 3</p><p><b>Summary:</b> We propose Delta L Normalization, a simple yet effective loss aggregation method tailored to the characteristic of dynamic generation lengths in Reinforcement Learning with Verifiable Rewards (RLVR). Recently, RLVR has demonstrated strong potential in improving the reasoning capabilities of large language models (LLMs), but a major challenge lies in the large variability of response lengths during training, which leads to high gradient variance and unstable optimization. Although previous methods such as GRPO, DAPO, and Dr. GRPO introduce different loss normalization terms to address this issue, they either produce biased estimates or still suffer from high gradient variance. By analyzing the effect of varying lengths on policy loss both theoretically and empirically, we reformulate the problem as finding a minimum-variance unbiased estimator. Our proposed Delta L Normalization not only provides an unbiased estimate of the true policy loss but also minimizes gradient variance in theory. Extensive experiments show that it consistently achieves superior results across different model sizes, maximum lengths, and tasks. Our code will be made public at https://github.com/zerolllin/Delta-L-Normalization.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.07558</guid>
|
||||
<pubDate>Tue, 09 Sep 2025 09:52:34 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge</title>
|
||||
<link>https://arxiv.org/abs/2509.07968</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.07968.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Lukas Haas, Gal Yona, Giovanni D'Antonio, Sasha Goldshtein, Dipanjan Das</p><p><b>Upvotes:</b> 7</p><p><b>Summary:</b> We introduce SimpleQA Verified, a 1,000-prompt benchmark for evaluating Large Language Model (LLM) short-form factuality based on OpenAI's SimpleQA. It addresses critical limitations in OpenAI's benchmark, including noisy and incorrect labels, topical biases, and question redundancy. SimpleQA Verified was created through a rigorous multi-stage filtering process involving de-duplication, topic balancing, and source reconciliation to produce a more reliable and challenging evaluation set, alongside improvements in the autorater prompt. On this new benchmark, Gemini 2.5 Pro achieves a state-of-the-art F1-score of 55.6, outperforming other frontier models, including GPT-5. This work provides the research community with a higher-fidelity tool to track genuine progress in parametric model factuality and to mitigate hallucinations. The benchmark dataset, evaluation code, and leaderboard are available at: https://www.kaggle.com/benchmarks/deepmind/simpleqa-verified.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.07968</guid>
|
||||
<pubDate>Tue, 09 Sep 2025 17:53:58 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search</title>
|
||||
<link>https://arxiv.org/abs/2509.07969</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.07969.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Xin Lai, Junyi Li, Wei Li, Tao Liu, Tianjian Li, Hengshuang Zhao</p><p><b>Upvotes:</b> 45</p><p><b>Summary:</b> Recent advances in large multimodal models have leveraged image-based tools with reinforcement learning to tackle visual problems. However, existing open-source approaches often exhibit monotonous reasoning patterns and allow only a limited number of interaction turns, making them inadequate for difficult tasks that require trial-and-error exploration. In this work, we address this limitation by scaling up tool-based interactions and introduce Mini-o3, a system that executes deep, multi-turn reasoning -- spanning tens of steps -- and achieves state-of-the-art performance on challenging visual search tasks. Our recipe for reproducing OpenAI o3-style behaviors comprises three key components. First, we construct the Visual Probe Dataset, a collection of thousands of challenging visual search problems designed for exploratory reasoning. Second, we develop an iterative data collection pipeline to obtain cold-start trajectories that exhibit diverse reasoning patterns, including depth-first search, trial-and-error, and goal maintenance. Third, we propose an over-turn masking strategy that prevents penalization of over-turn responses (those that hit the maximum number of turns) during reinforcement learning, thereby balancing training-time efficiency with test-time scalability. Despite training with an upper bound of only six interaction turns, our model generates trajectories that naturally scale to tens of turns at inference time, with accuracy improving as the number of turns increases. Extensive experiments demonstrate that Mini-o3 produces rich reasoning patterns and deep thinking paths, effectively solving challenging visual search problems.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.07969</guid>
|
||||
<pubDate>Tue, 09 Sep 2025 17:54:21 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>Visual Representation Alignment for Multimodal Large Language Models</title>
|
||||
<link>https://arxiv.org/abs/2509.07979</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.07979.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Heeji Yoon, Jaewoo Jung, Junwan Kim, Hyungyu Choi, Heeseong Shin, Sangbeom Lim, Honggyu An, Chaehyun Kim, Jisang Han, Donghyun Kim, Chanho Eom, Sunghwan Hong, Seungryong Kim</p><p><b>Upvotes:</b> 52</p><p><b>Summary:</b> Multimodal large language models (MLLMs) trained with visual instruction tuning have achieved strong performance across diverse tasks, yet they remain limited in vision-centric tasks such as object counting or spatial reasoning. We attribute this gap to the prevailing text-only supervision paradigm, which provides only indirect guidance for the visual pathway and often leads MLLMs to discard fine-grained visual details during training. In this paper, we present VIsual Representation ALignment (VIRAL), a simple yet effective regularization strategy that aligns the internal visual representations of MLLMs with those of pre-trained vision foundation models (VFMs). By explicitly enforcing this alignment, VIRAL enables the model not only to retain critical visual details from the input vision encoder but also to complement additional visual knowledge from VFMs, thereby enhancing its ability to reason over complex visual inputs. Our experiments demonstrate consistent improvements across all tasks on widely adopted multimodal benchmarks. Furthermore, we conduct comprehensive ablation studies to validate the key design choices underlying our framework. We believe this simple finding opens up an important direction for the effective integration of visual information in training MLLMs.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.07979</guid>
|
||||
<pubDate>Tue, 09 Sep 2025 17:59:14 +0000</pubDate>
|
||||
</item>
|
||||
<item>
|
||||
<title>Parallel-R1: Towards Parallel Thinking via Reinforcement Learning</title>
|
||||
<link>https://arxiv.org/abs/2509.07980</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.07980.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Tong Zheng, Hongming Zhang, Wenhao Yu, Xiaoyang Wang, Xinyu Yang, Runpeng Dai, Rui Liu, Huiwen Bao, Chengsong Huang, Heng Huang, Dong Yu</p><p><b>Upvotes:</b> 66</p><p><b>Summary:</b> Parallel thinking has emerged as a novel approach for enhancing the reasoning capabilities of large language models (LLMs) by exploring multiple reasoning paths concurrently. However, activating such capabilities through training remains challenging, as existing methods predominantly rely on supervised fine-tuning (SFT) over synthetic data, which encourages teacher-forced imitation rather than exploration and generalization. Different from them, we propose Parallel-R1, the first reinforcement learning (RL) framework that enables parallel thinking behaviors for complex real-world reasoning tasks. Our framework employs a progressive curriculum that explicitly addresses the cold-start problem in training parallel thinking with RL. We first use SFT on prompt-generated trajectories from easier tasks to instill the parallel thinking ability, then transition to RL to explore and generalize this skill on harder problems. Experiments on various math benchmarks, including MATH, AMC23, and AIME, show that Parallel-R1 successfully instills parallel thinking, leading to 8.4% accuracy improvements over the sequential thinking model trained directly on challenging tasks with RL. Further analysis reveals a clear shift in the model's thinking behavior: at an early stage, it uses parallel thinking as an exploration strategy, while in a later stage, it uses the same capability for multi-perspective verification. Most significantly, we validate parallel thinking as a mid-training exploration scaffold, where this temporary exploratory phase unlocks a higher performance ceiling after RL, yielding a 42.9% improvement over the baseline on AIME25. Our model, data, and code will be open-source at https://github.com/zhengkid/Parallel-R1.</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.07980</guid>
|
||||
<pubDate>Tue, 09 Sep 2025 17:59:35 +0000</pubDate>
|
||||
<title>A Survey of Reinforcement Learning for Large Reasoning Models</title>
|
||||
<link>https://arxiv.org/abs/2509.08827</link>
|
||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.08827.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Kaiyan Zhang, Yuxin Zuo, Bingxiang He, Youbang Sun, Runze Liu, Che Jiang, Yuchen Fan, Kai Tian, Guoli Jia, Pengfei Li, Yu Fu, Xingtai Lv, Yuchen Zhang, Sihang Zeng, Shang Qu, Haozhan Li, Shijie Wang, Yuru Wang, Xinwei Long, Fangfu Liu, Xiang Xu, Jiaze Ma, Xuekai Zhu, Ermo Hua, Yihao Liu, Zonglin Li, Huayu Chen, Xiaoye Qu, Yafu Li, Weize Chen, Zhenzhao Yuan, Junqi Gao, Dong Li, Zhiyuan Ma, Ganqu Cui, Zhiyuan Liu, Biqing Qi, Ning Ding, Bowen Zhou</p><p><b>Upvotes:</b> 92</p><p><b>Summary:</b> In this paper, we survey recent advances in Reinforcement Learning (RL) for reasoning with Large Language Models (LLMs). RL has achieved remarkable success in advancing the frontier of LLM capabilities, particularly in addressing complex logical tasks such as mathematics and coding. As a result, RL has emerged as a foundational methodology for transforming LLMs into LRMs. With the rapid progress of the field, further scaling of RL for LRMs now faces foundational challenges not only in computational resources but also in algorithm design, training data, and infrastructure. To this end, it is timely to revisit the development of this domain, reassess its trajectory, and explore strategies to enhance the scalability of RL toward Artificial SuperIntelligence (ASI). In particular, we examine research applying RL to LLMs and LRMs for reasoning abilities, especially since the release of DeepSeek-R1, including foundational components, core problems, training resources, and downstream applications, to identify future opportunities and directions for this rapidly evolving area. We hope this review will promote future research on RL for broader reasoning models. Github: https://github.com/TsinghuaC3I/Awesome-RL-for-LRMs</p></description>
|
||||
<guid isPermaLink="false">https://arxiv.org/abs/2509.08827</guid>
|
||||
<pubDate>Wed, 10 Sep 2025 17:59:43 +0000</pubDate>
|
||||
</item>
|
||||
</channel>
|
||||
</rss>
|
||||
|
||||
Reference in New Issue
Block a user