bot: update RSS feed

feed.xml

@@ -7,11 +7,11 @@
     <docs>http://www.rssboard.org/rss-specification</docs>
     <generator>python-feedgen</generator>
     <language>en</language>
-    <lastBuildDate>Sun, 12 Oct 2025 00:02:36 +0000</lastBuildDate>
+    <lastBuildDate>Mon, 13 Oct 2025 00:02:41 +0000</lastBuildDate>
     <item>
       <title>Meta-Awareness Enhances Reasoning Models: Self-Alignment Reinforcement   Learning</title>
       <link>https://arxiv.org/abs/2510.03259</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.03259.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yoonjeon Kim, Doohyuk Jang, Eunho Yang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 42&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent studies on reasoning models explore the meta-awareness of language models, the ability to know how to think by itself. We argue that large reasoning models lack this meta-awareness property by proving severe misalignment between true rollouts and predicted meta information. We posit that aligning meta-prediction with true rollouts will lead to significant performance gains. To verify this hypothesis, we design a training pipeline that boosts Meta-Awareness via Self-Alignment (MASA), and prove that enhanced meta-awareness directly translates to improved accuracy. Unlike existing meta-cognitive reasoning models, our method does not require external training sources but leverages self-generated signals to train meta-awareness. Moreover, our method enables efficient training by i) filtering out zero-variance prompts that are either trivial or unsolvable and ii) cutting off lengthy rollouts when they are unlikely to lead to correct answers. The results are inspiring: our strategy yields significant improvements in both accuracy and training efficiency on in-domain tasks and shows strong generalization to out-of-domain benchmarks. More specifically, our method can speed up GRPO training by over 1.28x to reach the same performance, and achieve a 19.3% gain in accuracy on AIME25, and a 6.2 % average gain over six mathematics benchmarks. Training with meta-cognitive guidance enhances out-of-domain generalization, giving a 3.87 % boost on GPQA-Diamond and a 2.08 % overall accuracy gain across 13 benchmarks spanning logical, scientific, and coding domains.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.03259.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yoonjeon Kim, Doohyuk Jang, Eunho Yang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 44&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent studies on reasoning models explore the meta-awareness of language models, the ability to know how to think by itself. We argue that large reasoning models lack this meta-awareness property by proving severe misalignment between true rollouts and predicted meta information. We posit that aligning meta-prediction with true rollouts will lead to significant performance gains. To verify this hypothesis, we design a training pipeline that boosts Meta-Awareness via Self-Alignment (MASA), and prove that enhanced meta-awareness directly translates to improved accuracy. Unlike existing meta-cognitive reasoning models, our method does not require external training sources but leverages self-generated signals to train meta-awareness. Moreover, our method enables efficient training by i) filtering out zero-variance prompts that are either trivial or unsolvable and ii) cutting off lengthy rollouts when they are unlikely to lead to correct answers. The results are inspiring: our strategy yields significant improvements in both accuracy and training efficiency on in-domain tasks and shows strong generalization to out-of-domain benchmarks. More specifically, our method can speed up GRPO training by over 1.28x to reach the same performance, and achieve a 19.3% gain in accuracy on AIME25, and a 6.2 % average gain over six mathematics benchmarks. Training with meta-cognitive guidance enhances out-of-domain generalization, giving a 3.87 % boost on GPQA-Diamond and a 2.08 % overall accuracy gain across 13 benchmarks spanning logical, scientific, and coding domains.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.03259</guid>
       <pubDate>Fri, 26 Sep 2025 14:05:48 +0000</pubDate>
     </item>
@@ -25,7 +25,7 @@
     <item>
       <title>From What to Why: A Multi-Agent System for Evidence-based Chemical   Reaction Condition Reasoning</title>
       <link>https://arxiv.org/abs/2509.23768</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.23768.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Cheng Yang, Jiaxuan Lu, Haiyuan Wan, Junchi Yu, Feiwei Qin&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 45&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; The chemical reaction recommendation is to select proper reaction condition parameters for chemical reactions, which is pivotal to accelerating chemical science. With the rapid development of large language models (LLMs), there is growing interest in leveraging their reasoning and planning capabilities for reaction condition recommendation. Despite their success, existing methods rarely explain the rationale behind the recommended reaction conditions, limiting their utility in high-stakes scientific workflows. In this work, we propose ChemMAS, a multi-agent system that reframes condition prediction as an evidence-based reasoning task. ChemMAS decomposes the task into mechanistic grounding, multi-channel recall, constraint-aware agentic debate, and rationale aggregation. Each decision is backed by interpretable justifications grounded in chemical knowledge and retrieved precedents. Experiments show that ChemMAS achieves 20-35% gains over domain-specific baselines and outperforms general-purpose LLMs by 10-15% in Top-1 accuracy, while offering falsifiable, human-trustable rationales, which establishes a new paradigm for explainable AI in scientific discovery.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2509.23768.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Cheng Yang, Jiaxuan Lu, Haiyuan Wan, Junchi Yu, Feiwei Qin&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 46&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; The chemical reaction recommendation is to select proper reaction condition parameters for chemical reactions, which is pivotal to accelerating chemical science. With the rapid development of large language models (LLMs), there is growing interest in leveraging their reasoning and planning capabilities for reaction condition recommendation. Despite their success, existing methods rarely explain the rationale behind the recommended reaction conditions, limiting their utility in high-stakes scientific workflows. In this work, we propose ChemMAS, a multi-agent system that reframes condition prediction as an evidence-based reasoning task. ChemMAS decomposes the task into mechanistic grounding, multi-channel recall, constraint-aware agentic debate, and rationale aggregation. Each decision is backed by interpretable justifications grounded in chemical knowledge and retrieved precedents. Experiments show that ChemMAS achieves 20-35% gains over domain-specific baselines and outperforms general-purpose LLMs by 10-15% in Top-1 accuracy, while offering falsifiable, human-trustable rationales, which establishes a new paradigm for explainable AI in scientific discovery.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2509.23768</guid>
       <pubDate>Sun, 28 Sep 2025 09:34:35 +0000</pubDate>
     </item>
@@ -81,7 +81,7 @@
     <item>
       <title>Low-probability Tokens Sustain Exploration in Reinforcement Learning   with Verifiable Reward</title>
       <link>https://arxiv.org/abs/2510.03222</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.03222.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Guanhua Huang, Tingqiang Xu, Mingze Wang, Qi Yi, Xue Gong, Siheng Li, Ruibin Xiong, Kejiao Li, Yuhao Jiang, Bo Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 34&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Reinforcement Learning with Verifiable Rewards (RLVR) has propelled Large Language Models in complex reasoning, yet its scalability is often hindered by a training bottleneck where performance plateaus as policy entropy collapses, signaling a loss of exploration. Previous methods typically address this by maintaining high policy entropy, yet the precise mechanisms that govern meaningful exploration have remained underexplored. Our analysis suggests that an unselective focus on entropy risks amplifying irrelevant tokens and destabilizing training. This paper investigates the exploration dynamics within RLVR and identifies a key issue: the gradual elimination of valuable low-probability exploratory tokens, which we term \textit{reasoning sparks}. We find that while abundant in pre-trained models, these sparks are systematically extinguished during RLVR due to over-penalization, leading to a degeneracy in exploration. To address this, we introduce Low-probability Regularization (Lp-Reg). Its core mechanism regularizes the policy towards a heuristic proxy distribution. This proxy is constructed by filtering out presumed noise tokens and re-normalizing the distribution over the remaining candidates. The result is a less-noisy proxy where the probability of reasoning sparks is amplified, which then serves as a soft regularization target to shield these valuable tokens from elimination via KL divergence. Experiments show that Lp-Reg enables stable on-policy training for around 1,000 steps, a regime where baseline entropy-control methods collapse. This sustained exploration leads to state-of-the-art performance, achieving a 60.17% average accuracy on five math benchmarks, an improvement of 2.66% over prior methods. Code is available at https://github.com/CarlanLark/Lp-Reg.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.03222.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Guanhua Huang, Tingqiang Xu, Mingze Wang, Qi Yi, Xue Gong, Siheng Li, Ruibin Xiong, Kejiao Li, Yuhao Jiang, Bo Zhou&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 37&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Reinforcement Learning with Verifiable Rewards (RLVR) has propelled Large Language Models in complex reasoning, yet its scalability is often hindered by a training bottleneck where performance plateaus as policy entropy collapses, signaling a loss of exploration. Previous methods typically address this by maintaining high policy entropy, yet the precise mechanisms that govern meaningful exploration have remained underexplored. Our analysis suggests that an unselective focus on entropy risks amplifying irrelevant tokens and destabilizing training. This paper investigates the exploration dynamics within RLVR and identifies a key issue: the gradual elimination of valuable low-probability exploratory tokens, which we term \textit{reasoning sparks}. We find that while abundant in pre-trained models, these sparks are systematically extinguished during RLVR due to over-penalization, leading to a degeneracy in exploration. To address this, we introduce Low-probability Regularization (Lp-Reg). Its core mechanism regularizes the policy towards a heuristic proxy distribution. This proxy is constructed by filtering out presumed noise tokens and re-normalizing the distribution over the remaining candidates. The result is a less-noisy proxy where the probability of reasoning sparks is amplified, which then serves as a soft regularization target to shield these valuable tokens from elimination via KL divergence. Experiments show that Lp-Reg enables stable on-policy training for around 1,000 steps, a regime where baseline entropy-control methods collapse. This sustained exploration leads to state-of-the-art performance, achieving a 60.17% average accuracy on five math benchmarks, an improvement of 2.66% over prior methods. Code is available at https://github.com/CarlanLark/Lp-Reg.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.03222</guid>
       <pubDate>Fri, 03 Oct 2025 17:56:13 +0000</pubDate>
     </item>
@@ -102,14 +102,14 @@
     <item>
       <title>DreamOmni2: Multimodal Instruction-based Editing and Generation</title>
       <link>https://arxiv.org/abs/2510.06679</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.06679.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Bin Xia, Bohao Peng, Yuechen Zhang, Junjia Huang, Jiyang Liu, Jingyao Li, Haoru Tan, Sitong Wu, Chengyao Wang, Yitong Wang, Xinglong Wu, Bei Yu, Jiaya Jia&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 42&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advancements in instruction-based image editing and subject-driven generation have garnered significant attention, yet both tasks still face limitations in meeting practical user needs. Instruction-based editing relies solely on language instructions, which often fail to capture specific editing details, making reference images necessary. Meanwhile, subject-driven generation is limited to combining concrete objects or people, overlooking broader, abstract concepts. To address these challenges, we propose two novel tasks: multimodal instruction-based editing and generation. These tasks support both text and image instructions and extend the scope to include both concrete and abstract concepts, greatly enhancing their practical applications. We introduce DreamOmni2, tackling two primary challenges: data creation and model framework design. Our data synthesis pipeline consists of three steps: (1) using a feature mixing method to create extraction data for both abstract and concrete concepts, (2) generating multimodal instruction-based editing training data using the editing and extraction models, and (3) further applying the extraction model to create training data for multimodal instruction-based editing. For the framework, to handle multi-image input, we propose an index encoding and position encoding shift scheme, which helps the model distinguish images and avoid pixel confusion. Additionally, we introduce joint training with the VLM and our generation/editing model to better process complex instructions. In addition, we have proposed comprehensive benchmarks for these two new tasks to drive their development. Experiments show that DreamOmni2 has achieved impressive results. Models and codes will be released.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.06679.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Bin Xia, Bohao Peng, Yuechen Zhang, Junjia Huang, Jiyang Liu, Jingyao Li, Haoru Tan, Sitong Wu, Chengyao Wang, Yitong Wang, Xinglong Wu, Bei Yu, Jiaya Jia&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 50&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advancements in instruction-based image editing and subject-driven generation have garnered significant attention, yet both tasks still face limitations in meeting practical user needs. Instruction-based editing relies solely on language instructions, which often fail to capture specific editing details, making reference images necessary. Meanwhile, subject-driven generation is limited to combining concrete objects or people, overlooking broader, abstract concepts. To address these challenges, we propose two novel tasks: multimodal instruction-based editing and generation. These tasks support both text and image instructions and extend the scope to include both concrete and abstract concepts, greatly enhancing their practical applications. We introduce DreamOmni2, tackling two primary challenges: data creation and model framework design. Our data synthesis pipeline consists of three steps: (1) using a feature mixing method to create extraction data for both abstract and concrete concepts, (2) generating multimodal instruction-based editing training data using the editing and extraction models, and (3) further applying the extraction model to create training data for multimodal instruction-based editing. For the framework, to handle multi-image input, we propose an index encoding and position encoding shift scheme, which helps the model distinguish images and avoid pixel confusion. Additionally, we introduce joint training with the VLM and our generation/editing model to better process complex instructions. In addition, we have proposed comprehensive benchmarks for these two new tasks to drive their development. Experiments show that DreamOmni2 has achieved impressive results. Models and codes will be released.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.06679</guid>
       <pubDate>Wed, 08 Oct 2025 06:07:14 +0000</pubDate>
     </item>
     <item>
       <title>LongRM: Revealing and Unlocking the Context Boundary of Reward Modeling</title>
       <link>https://arxiv.org/abs/2510.06915</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.06915.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zecheng Tang, Baibei Ji, Quantong Qiu, Haitian Wang, Xiaobo Liang, Juntao Li, Min Zhang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 13&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Reward model (RM) plays a pivotal role in aligning large language model (LLM) with human preferences. As real-world applications increasingly involve long history trajectories, e.g., LLM agent, it becomes indispensable to evaluate whether a model's responses are not only high-quality but also grounded in and consistent with the provided context. Yet, current RMs remain confined to short-context settings and primarily focus on response-level attributes (e.g., safety or helpfulness), while largely neglecting the critical dimension of long context-response consistency. In this work, we introduce Long-RewardBench, a benchmark specifically designed for long-context RM evaluation, featuring both Pairwise Comparison and Best-of-N tasks. Our preliminary study reveals that even state-of-the-art generative RMs exhibit significant fragility in long-context scenarios, failing to maintain context-aware preference judgments. Motivated by the analysis of failure patterns observed in model outputs, we propose a general multi-stage training strategy that effectively scales arbitrary models into robust Long-context RMs (LongRMs). Experiments show that our approach not only substantially improves performance on long-context evaluation but also preserves strong short-context capability. Notably, our 8B LongRM outperforms much larger 70B-scale baselines and matches the performance of the proprietary Gemini 2.5 Pro model.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.06915.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Zecheng Tang, Baibei Ji, Quantong Qiu, Haitian Wang, Xiaobo Liang, Juntao Li, Min Zhang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 14&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Reward model (RM) plays a pivotal role in aligning large language model (LLM) with human preferences. As real-world applications increasingly involve long history trajectories, e.g., LLM agent, it becomes indispensable to evaluate whether a model's responses are not only high-quality but also grounded in and consistent with the provided context. Yet, current RMs remain confined to short-context settings and primarily focus on response-level attributes (e.g., safety or helpfulness), while largely neglecting the critical dimension of long context-response consistency. In this work, we introduce Long-RewardBench, a benchmark specifically designed for long-context RM evaluation, featuring both Pairwise Comparison and Best-of-N tasks. Our preliminary study reveals that even state-of-the-art generative RMs exhibit significant fragility in long-context scenarios, failing to maintain context-aware preference judgments. Motivated by the analysis of failure patterns observed in model outputs, we propose a general multi-stage training strategy that effectively scales arbitrary models into robust Long-context RMs (LongRMs). Experiments show that our approach not only substantially improves performance on long-context evaluation but also preserves strong short-context capability. Notably, our 8B LongRM outperforms much larger 70B-scale baselines and matches the performance of the proprietary Gemini 2.5 Pro model.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.06915</guid>
       <pubDate>Wed, 08 Oct 2025 11:48:16 +0000</pubDate>
     </item>
@@ -158,7 +158,7 @@
     <item>
       <title>PickStyle: Video-to-Video Style Transfer with Context-Style Adapters</title>
       <link>https://arxiv.org/abs/2510.07546</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.07546.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Soroush Mehraban, Vida Adeli, Jacob Rommann, Babak Taati, Kyryl Truskovskyi&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 16&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We address the task of video style transfer with diffusion models, where the goal is to preserve the context of an input video while rendering it in a target style specified by a text prompt. A major challenge is the lack of paired video data for supervision. We propose PickStyle, a video-to-video style transfer framework that augments pretrained video diffusion backbones with style adapters and benefits from paired still image data with source-style correspondences for training. PickStyle inserts low-rank adapters into the self-attention layers of conditioning modules, enabling efficient specialization for motion-style transfer while maintaining strong alignment between video content and style. To bridge the gap between static image supervision and dynamic video, we construct synthetic training clips from paired images by applying shared augmentations that simulate camera motion, ensuring temporal priors are preserved. In addition, we introduce Context-Style Classifier-Free Guidance (CS-CFG), a novel factorization of classifier-free guidance into independent text (style) and video (context) directions. CS-CFG ensures that context is preserved in generated video while the style is effectively transferred. Experiments across benchmarks show that our approach achieves temporally coherent, style-faithful, and content-preserving video translations, outperforming existing baselines both qualitatively and quantitatively.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.07546.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Soroush Mehraban, Vida Adeli, Jacob Rommann, Babak Taati, Kyryl Truskovskyi&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 17&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We address the task of video style transfer with diffusion models, where the goal is to preserve the context of an input video while rendering it in a target style specified by a text prompt. A major challenge is the lack of paired video data for supervision. We propose PickStyle, a video-to-video style transfer framework that augments pretrained video diffusion backbones with style adapters and benefits from paired still image data with source-style correspondences for training. PickStyle inserts low-rank adapters into the self-attention layers of conditioning modules, enabling efficient specialization for motion-style transfer while maintaining strong alignment between video content and style. To bridge the gap between static image supervision and dynamic video, we construct synthetic training clips from paired images by applying shared augmentations that simulate camera motion, ensuring temporal priors are preserved. In addition, we introduce Context-Style Classifier-Free Guidance (CS-CFG), a novel factorization of classifier-free guidance into independent text (style) and video (context) directions. CS-CFG ensures that context is preserved in generated video while the style is effectively transferred. Experiments across benchmarks show that our approach achieves temporally coherent, style-faithful, and content-preserving video translations, outperforming existing baselines both qualitatively and quantitatively.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.07546</guid>
       <pubDate>Wed, 08 Oct 2025 21:02:55 +0000</pubDate>
     </item>
@@ -200,21 +200,21 @@
     <item>
       <title>UniMMVSR: A Unified Multi-Modal Framework for Cascaded Video   Super-Resolution</title>
       <link>https://arxiv.org/abs/2510.08143</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08143.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shian Du, Menghan Xia, Chang Liu, Quande Liu, Xintao Wang, Pengfei Wan, Xiangyang Ji&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 18&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Cascaded video super-resolution has emerged as a promising technique for decoupling the computational burden associated with generating high-resolution videos using large foundation models. Existing studies, however, are largely confined to text-to-video tasks and fail to leverage additional generative conditions beyond text, which are crucial for ensuring fidelity in multi-modal video generation. We address this limitation by presenting UniMMVSR, the first unified generative video super-resolution framework to incorporate hybrid-modal conditions, including text, images, and videos. We conduct a comprehensive exploration of condition injection strategies, training schemes, and data mixture techniques within a latent video diffusion model. A key challenge was designing distinct data construction and condition utilization methods to enable the model to precisely utilize all condition types, given their varied correlations with the target video. Our experiments demonstrate that UniMMVSR significantly outperforms existing methods, producing videos with superior detail and a higher degree of conformity to multi-modal conditions. We also validate the feasibility of combining UniMMVSR with a base model to achieve multi-modal guided generation of 4K video, a feat previously unattainable with existing techniques.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08143.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shian Du, Menghan Xia, Chang Liu, Quande Liu, Xintao Wang, Pengfei Wan, Xiangyang Ji&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 19&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Cascaded video super-resolution has emerged as a promising technique for decoupling the computational burden associated with generating high-resolution videos using large foundation models. Existing studies, however, are largely confined to text-to-video tasks and fail to leverage additional generative conditions beyond text, which are crucial for ensuring fidelity in multi-modal video generation. We address this limitation by presenting UniMMVSR, the first unified generative video super-resolution framework to incorporate hybrid-modal conditions, including text, images, and videos. We conduct a comprehensive exploration of condition injection strategies, training schemes, and data mixture techniques within a latent video diffusion model. A key challenge was designing distinct data construction and condition utilization methods to enable the model to precisely utilize all condition types, given their varied correlations with the target video. Our experiments demonstrate that UniMMVSR significantly outperforms existing methods, producing videos with superior detail and a higher degree of conformity to multi-modal conditions. We also validate the feasibility of combining UniMMVSR with a base model to achieve multi-modal guided generation of 4K video, a feat previously unattainable with existing techniques.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.08143</guid>
       <pubDate>Thu, 09 Oct 2025 12:25:16 +0000</pubDate>
     </item>
     <item>
       <title>Training-Free Group Relative Policy Optimization</title>
       <link>https://arxiv.org/abs/2510.08191</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08191.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuzheng Cai, Siqi Cai, Yuchen Shi, Zihan Xu, Lichao Chen, Yulei Qin, Xiaoyu Tan, Gang Li, Zongyi Li, Haojia Lin, Yong Mao, Ke Li, Xing Sun&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 30&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in Large Language Model (LLM) agents have demonstrated their promising general capabilities. However, their performance in specialized real-world domains often degrades due to challenges in effectively integrating external tools and specific prompting strategies. While methods like agentic reinforcement learning have been proposed to address this, they typically rely on costly parameter updates, for example, through a process that uses Supervised Fine-Tuning (SFT) followed by a Reinforcement Learning (RL) phase with Group Relative Policy Optimization (GRPO) to alter the output distribution. However, we argue that LLMs can achieve a similar effect on the output distribution by learning experiential knowledge as a token prior, which is a far more lightweight approach that not only addresses practical data scarcity but also avoids the common issue of overfitting. To this end, we propose Training-Free Group Relative Policy Optimization (Training-Free GRPO), a cost-effective solution that enhances LLM agent performance without any parameter updates. Our method leverages the group relative semantic advantage instead of numerical ones within each group of rollouts, iteratively distilling high-quality experiential knowledge during multi-epoch learning on a minimal ground-truth data. Such knowledge serves as the learned token prior, which is seamlessly integrated during LLM API calls to guide model behavior. Experiments on mathematical reasoning and web searching tasks demonstrate that Training-Free GRPO, when applied to DeepSeek-V3.1-Terminus, significantly improves out-of-domain performance. With just a few dozen training samples, Training-Free GRPO outperforms fine-tuned small LLMs with marginal training data and cost.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08191.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yuzheng Cai, Siqi Cai, Yuchen Shi, Zihan Xu, Lichao Chen, Yulei Qin, Xiaoyu Tan, Gang Li, Zongyi Li, Haojia Lin, Yong Mao, Ke Li, Xing Sun&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 32&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Recent advances in Large Language Model (LLM) agents have demonstrated their promising general capabilities. However, their performance in specialized real-world domains often degrades due to challenges in effectively integrating external tools and specific prompting strategies. While methods like agentic reinforcement learning have been proposed to address this, they typically rely on costly parameter updates, for example, through a process that uses Supervised Fine-Tuning (SFT) followed by a Reinforcement Learning (RL) phase with Group Relative Policy Optimization (GRPO) to alter the output distribution. However, we argue that LLMs can achieve a similar effect on the output distribution by learning experiential knowledge as a token prior, which is a far more lightweight approach that not only addresses practical data scarcity but also avoids the common issue of overfitting. To this end, we propose Training-Free Group Relative Policy Optimization (Training-Free GRPO), a cost-effective solution that enhances LLM agent performance without any parameter updates. Our method leverages the group relative semantic advantage instead of numerical ones within each group of rollouts, iteratively distilling high-quality experiential knowledge during multi-epoch learning on a minimal ground-truth data. Such knowledge serves as the learned token prior, which is seamlessly integrated during LLM API calls to guide model behavior. Experiments on mathematical reasoning and web searching tasks demonstrate that Training-Free GRPO, when applied to DeepSeek-V3.1-Terminus, significantly improves out-of-domain performance. With just a few dozen training samples, Training-Free GRPO outperforms fine-tuned small LLMs with marginal training data and cost.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.08191</guid>
       <pubDate>Thu, 09 Oct 2025 13:18:17 +0000</pubDate>
     </item>
     <item>
       <title>Memory Retrieval and Consolidation in Large Language Models through   Function Tokens</title>
       <link>https://arxiv.org/abs/2510.08203</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08203.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shaohua Zhang, Yuan Lin, Hang Li&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 5&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; The remarkable success of large language models (LLMs) stems from their ability to consolidate vast amounts of knowledge into the memory during pre-training and to retrieve it from the memory during inference, enabling advanced capabilities such as knowledge memorization, instruction-following and reasoning. However, the mechanisms of memory retrieval and consolidation in LLMs remain poorly understood. In this paper, we propose the function token hypothesis to explain the workings of LLMs: During inference, function tokens activate the most predictive features from context and govern next token prediction (memory retrieval). During pre-training, predicting the next tokens (usually content tokens) that follow function tokens increases the number of learned features of LLMs and updates the model parameters (memory consolidation). Function tokens here roughly correspond to function words in linguistics, including punctuation marks, articles, prepositions, and conjunctions, in contrast to content tokens. We provide extensive experimental evidence supporting this hypothesis. Using bipartite graph analysis, we show that a small number of function tokens activate the majority of features. Case studies further reveal how function tokens activate the most predictive features from context to direct next token prediction. We also find that during pre-training, the training loss is dominated by predicting the next content tokens following function tokens, which forces the function tokens to select the most predictive features from context.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08203.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shaohua Zhang, Yuan Lin, Hang Li&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 6&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; The remarkable success of large language models (LLMs) stems from their ability to consolidate vast amounts of knowledge into the memory during pre-training and to retrieve it from the memory during inference, enabling advanced capabilities such as knowledge memorization, instruction-following and reasoning. However, the mechanisms of memory retrieval and consolidation in LLMs remain poorly understood. In this paper, we propose the function token hypothesis to explain the workings of LLMs: During inference, function tokens activate the most predictive features from context and govern next token prediction (memory retrieval). During pre-training, predicting the next tokens (usually content tokens) that follow function tokens increases the number of learned features of LLMs and updates the model parameters (memory consolidation). Function tokens here roughly correspond to function words in linguistics, including punctuation marks, articles, prepositions, and conjunctions, in contrast to content tokens. We provide extensive experimental evidence supporting this hypothesis. Using bipartite graph analysis, we show that a small number of function tokens activate the majority of features. Case studies further reveal how function tokens activate the most predictive features from context to direct next token prediction. We also find that during pre-training, the training loss is dominated by predicting the next content tokens following function tokens, which forces the function tokens to select the most predictive features from context.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.08203</guid>
       <pubDate>Thu, 09 Oct 2025 13:31:20 +0000</pubDate>
     </item>
@@ -256,14 +256,14 @@
     <item>
       <title>UniVideo: Unified Understanding, Generation, and Editing for Videos</title>
       <link>https://arxiv.org/abs/2510.08377</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08377.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Cong Wei, Quande Liu, Zixuan Ye, Qiulin Wang, Xintao Wang, Pengfei Wan, Kun Gai, Wenhu Chen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 51&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Unified multimodal models have shown promising results in multimodal content generation and editing but remain largely limited to the image domain. In this work, we present UniVideo, a versatile framework that extends unified modeling to the video domain. UniVideo adopts a dual-stream design, combining a Multimodal Large Language Model (MLLM) for instruction understanding with a Multimodal DiT (MMDiT) for video generation. This design enables accurate interpretation of complex multimodal instructions while preserving visual consistency. Built on this architecture, UniVideo unifies diverse video generation and editing tasks under a single multimodal instruction paradigm and is jointly trained across them. Extensive experiments demonstrate that UniVideo matches or surpasses state-of-the-art task-specific baselines in text/image-to-video generation, in-context video generation and in-context video editing. Notably, the unified design of UniVideo enables two forms of generalization. First, UniVideo supports task composition, such as combining editing with style transfer, by integrating multiple capabilities within a single instruction. Second, even without explicit training on free-form video editing, UniVideo transfers its editing capability from large-scale image editing data to this setting, handling unseen instructions such as green-screening characters or changing materials within a video. Beyond these core capabilities, UniVideo also supports visual-prompt-based video generation, where the MLLM interprets visual prompts and guides the MMDiT during synthesis. To foster future research, we will release our model and code.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08377.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Cong Wei, Quande Liu, Zixuan Ye, Qiulin Wang, Xintao Wang, Pengfei Wan, Kun Gai, Wenhu Chen&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 56&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Unified multimodal models have shown promising results in multimodal content generation and editing but remain largely limited to the image domain. In this work, we present UniVideo, a versatile framework that extends unified modeling to the video domain. UniVideo adopts a dual-stream design, combining a Multimodal Large Language Model (MLLM) for instruction understanding with a Multimodal DiT (MMDiT) for video generation. This design enables accurate interpretation of complex multimodal instructions while preserving visual consistency. Built on this architecture, UniVideo unifies diverse video generation and editing tasks under a single multimodal instruction paradigm and is jointly trained across them. Extensive experiments demonstrate that UniVideo matches or surpasses state-of-the-art task-specific baselines in text/image-to-video generation, in-context video generation and in-context video editing. Notably, the unified design of UniVideo enables two forms of generalization. First, UniVideo supports task composition, such as combining editing with style transfer, by integrating multiple capabilities within a single instruction. Second, even without explicit training on free-form video editing, UniVideo transfers its editing capability from large-scale image editing data to this setting, handling unseen instructions such as green-screening characters or changing materials within a video. Beyond these core capabilities, UniVideo also supports visual-prompt-based video generation, where the MLLM interprets visual prompts and guides the MMDiT during synthesis. To foster future research, we will release our model and code.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.08377</guid>
       <pubDate>Thu, 09 Oct 2025 16:01:30 +0000</pubDate>
     </item>
     <item>
       <title>Reinforcing Diffusion Models by Direct Group Preference Optimization</title>
       <link>https://arxiv.org/abs/2510.08425</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08425.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yihong Luo, Tianyang Hu, Jing Tang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 8&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; While reinforcement learning methods such as Group Relative Preference Optimization (GRPO) have significantly enhanced Large Language Models, adapting them to diffusion models remains challenging. In particular, GRPO demands a stochastic policy, yet the most cost-effective diffusion samplers are based on deterministic ODEs. Recent work addresses this issue by using inefficient SDE-based samplers to induce stochasticity, but this reliance on model-agnostic Gaussian noise leads to slow convergence. To resolve this conflict, we propose Direct Group Preference Optimization (DGPO), a new online RL algorithm that dispenses with the policy-gradient framework entirely. DGPO learns directly from group-level preferences, which utilize relative information of samples within groups. This design eliminates the need for inefficient stochastic policies, unlocking the use of efficient deterministic ODE samplers and faster training. Extensive results show that DGPO trains around 20 times faster than existing state-of-the-art methods and achieves superior performance on both in-domain and out-of-domain reward metrics. Code is available at https://github.com/Luo-Yihong/DGPO.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08425.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Yihong Luo, Tianyang Hu, Jing Tang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 9&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; While reinforcement learning methods such as Group Relative Preference Optimization (GRPO) have significantly enhanced Large Language Models, adapting them to diffusion models remains challenging. In particular, GRPO demands a stochastic policy, yet the most cost-effective diffusion samplers are based on deterministic ODEs. Recent work addresses this issue by using inefficient SDE-based samplers to induce stochasticity, but this reliance on model-agnostic Gaussian noise leads to slow convergence. To resolve this conflict, we propose Direct Group Preference Optimization (DGPO), a new online RL algorithm that dispenses with the policy-gradient framework entirely. DGPO learns directly from group-level preferences, which utilize relative information of samples within groups. This design eliminates the need for inefficient stochastic policies, unlocking the use of efficient deterministic ODE samplers and faster training. Extensive results show that DGPO trains around 20 times faster than existing state-of-the-art methods and achieves superior performance on both in-domain and out-of-domain reward metrics. Code is available at https://github.com/Luo-Yihong/DGPO.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.08425</guid>
       <pubDate>Thu, 09 Oct 2025 16:40:43 +0000</pubDate>
     </item>
@@ -277,28 +277,28 @@
     <item>
       <title>DeepPrune: Parallel Scaling without Inter-trace Redundancy</title>
       <link>https://arxiv.org/abs/2510.08483</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08483.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shangqing Tu, Yaxuan Li, Yushi Bai, Lei Hou, Juanzi Li&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 22&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Parallel scaling has emerged as a powerful paradigm to enhance reasoning capabilities in large language models (LLMs) by generating multiple Chain-of-Thought (CoT) traces simultaneously. However, this approach introduces significant computational inefficiency due to inter-trace redundancy -- our analysis reveals that over 80% of parallel reasoning traces yield identical final answers, representing substantial wasted computation. To address this critical efficiency bottleneck, we propose DeepPrune, a novel framework that enables efficient parallel scaling through dynamic pruning. Our method features a specialized judge model trained with focal loss and oversampling techniques to accurately predict answer equivalence from partial reasoning traces which realizes 0.87 AUROC on equivalence prediction, combined with an online greedy clustering algorithm that dynamically prunes redundant paths while preserving answer diversity. Comprehensive evaluations across three challenging benchmarks (AIME 2024, AIME 2025, and GPQA) and multiple reasoning models demonstrate that DeepPrune achieves remarkable token reduction by over 80% compared to conventional consensus sampling on most cases, while maintaining competitive accuracy within 3 percentage points. Our work establishes a new standard for efficient parallel reasoning, making high-performance reasoning more efficient. Our code and data are here: https://deepprune.github.io/&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08483.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Shangqing Tu, Yaxuan Li, Yushi Bai, Lei Hou, Juanzi Li&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 23&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Parallel scaling has emerged as a powerful paradigm to enhance reasoning capabilities in large language models (LLMs) by generating multiple Chain-of-Thought (CoT) traces simultaneously. However, this approach introduces significant computational inefficiency due to inter-trace redundancy -- our analysis reveals that over 80% of parallel reasoning traces yield identical final answers, representing substantial wasted computation. To address this critical efficiency bottleneck, we propose DeepPrune, a novel framework that enables efficient parallel scaling through dynamic pruning. Our method features a specialized judge model trained with focal loss and oversampling techniques to accurately predict answer equivalence from partial reasoning traces which realizes 0.87 AUROC on equivalence prediction, combined with an online greedy clustering algorithm that dynamically prunes redundant paths while preserving answer diversity. Comprehensive evaluations across three challenging benchmarks (AIME 2024, AIME 2025, and GPQA) and multiple reasoning models demonstrate that DeepPrune achieves remarkable token reduction by over 80% compared to conventional consensus sampling on most cases, while maintaining competitive accuracy within 3 percentage points. Our work establishes a new standard for efficient parallel reasoning, making high-performance reasoning more efficient. Our code and data are here: https://deepprune.github.io/&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.08483</guid>
       <pubDate>Thu, 09 Oct 2025 17:24:54 +0000</pubDate>
     </item>
     <item>
       <title>InstructX: Towards Unified Visual Editing with MLLM Guidance</title>
       <link>https://arxiv.org/abs/2510.08485</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08485.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Chong Mou, Qichao Sun, Yanze Wu, Pengze Zhang, Xinghui Li, Fulong Ye, Songtao Zhao, Qian He&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 13&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; With recent advances in Multimodal Large Language Models (MLLMs) showing strong visual understanding and reasoning, interest is growing in using them to improve the editing performance of diffusion models. Despite rapid progress, most studies lack an in-depth analysis of MLLM design choices. Moreover, the integration of MLLMs and diffusion models remains an open challenge in some difficult tasks, such as video editing. In this paper, we present InstructX, a unified framework for image and video editing. Specifically, we conduct a comprehensive study on integrating MLLMs and diffusion models for instruction-driven editing across diverse tasks. Building on this study, we analyze the cooperation and distinction between images and videos in unified modeling. (1) We show that training on image data can lead to emergent video editing capabilities without explicit supervision, thereby alleviating the constraints imposed by scarce video training data. (2) By incorporating modality-specific MLLM features, our approach effectively unifies image and video editing tasks within a single model. Extensive experiments demonstrate that our method can handle a broad range of image and video editing tasks and achieves state-of-the-art performance.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08485.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Chong Mou, Qichao Sun, Yanze Wu, Pengze Zhang, Xinghui Li, Fulong Ye, Songtao Zhao, Qian He&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 14&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; With recent advances in Multimodal Large Language Models (MLLMs) showing strong visual understanding and reasoning, interest is growing in using them to improve the editing performance of diffusion models. Despite rapid progress, most studies lack an in-depth analysis of MLLM design choices. Moreover, the integration of MLLMs and diffusion models remains an open challenge in some difficult tasks, such as video editing. In this paper, we present InstructX, a unified framework for image and video editing. Specifically, we conduct a comprehensive study on integrating MLLMs and diffusion models for instruction-driven editing across diverse tasks. Building on this study, we analyze the cooperation and distinction between images and videos in unified modeling. (1) We show that training on image data can lead to emergent video editing capabilities without explicit supervision, thereby alleviating the constraints imposed by scarce video training data. (2) By incorporating modality-specific MLLM features, our approach effectively unifies image and video editing tasks within a single model. Extensive experiments demonstrate that our method can handle a broad range of image and video editing tasks and achieves state-of-the-art performance.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.08485</guid>
       <pubDate>Thu, 09 Oct 2025 17:26:09 +0000</pubDate>
     </item>
     <item>
       <title>CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards</title>
       <link>https://arxiv.org/abs/2510.08529</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08529.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Xiangyuan Xue, Yifan Zhou, Guibin Zhang, Zaibin Zhang, Yijiang Li, Chen Zhang, Zhenfei Yin, Philip Torr, Wanli Ouyang, Lei Bai&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 16&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Self-evolution is a central research topic in enabling large language model (LLM)-based agents to continually improve their capabilities after pretraining. Recent research has witnessed a transition from reinforcement learning (RL)-free to RL-based methods. Current RL-based methods either rely on dense external reward signals or extract intrinsic reward signals from LLMs themselves. However, these approaches diverge from the self-evolution mechanisms observed in human intelligence, where individuals learn and improve through mutual discussion and collaboration. In this work, we introduce Co-Evolving Multi-Agent Systems (CoMAS), a novel framework that enables agents to improve autonomously by learning from inter-agent interactions without external supervision. CoMAS generates intrinsic rewards from rich discussion dynamics, employs an LLM-as-a-judge mechanism to formulate these rewards, and optimizes each agent's policy through RL, thereby enabling decentralized and scalable co-evolution. Experimental results demonstrate that CoMAS consistently outperforms untrained agents and achieves state-of-the-art performance across most evaluation settings. Ablation studies confirm the necessity of interaction-based reward signals and reveal promising scalability as the number and diversity of agents increase. These findings establish CoMAS as a novel and effective paradigm for self-evolution in LLM-based agents.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08529.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Xiangyuan Xue, Yifan Zhou, Guibin Zhang, Zaibin Zhang, Yijiang Li, Chen Zhang, Zhenfei Yin, Philip Torr, Wanli Ouyang, Lei Bai&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 17&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; Self-evolution is a central research topic in enabling large language model (LLM)-based agents to continually improve their capabilities after pretraining. Recent research has witnessed a transition from reinforcement learning (RL)-free to RL-based methods. Current RL-based methods either rely on dense external reward signals or extract intrinsic reward signals from LLMs themselves. However, these approaches diverge from the self-evolution mechanisms observed in human intelligence, where individuals learn and improve through mutual discussion and collaboration. In this work, we introduce Co-Evolving Multi-Agent Systems (CoMAS), a novel framework that enables agents to improve autonomously by learning from inter-agent interactions without external supervision. CoMAS generates intrinsic rewards from rich discussion dynamics, employs an LLM-as-a-judge mechanism to formulate these rewards, and optimizes each agent's policy through RL, thereby enabling decentralized and scalable co-evolution. Experimental results demonstrate that CoMAS consistently outperforms untrained agents and achieves state-of-the-art performance across most evaluation settings. Ablation studies confirm the necessity of interaction-based reward signals and reveal promising scalability as the number and diversity of agents increase. These findings establish CoMAS as a novel and effective paradigm for self-evolution in LLM-based agents.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.08529</guid>
       <pubDate>Thu, 09 Oct 2025 17:50:26 +0000</pubDate>
     </item>
     <item>
       <title>MM-HELIX: Boosting Multimodal Long-Chain Reflective Reasoning with   Holistic Platform and Adaptive Hybrid Policy Optimization</title>
       <link>https://arxiv.org/abs/2510.08540</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08540.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Xiangyu Zhao, Junming Lin, Tianhao Liang, Yifan Zhou, Wenhao Chai, Yuzhe Gu, Weiyun Wang, Kai Chen, Gen Luo, Wenwei Zhang, Junchi Yan, Hua Yang, Haodong Duan, Xue Yang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 96&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; While current Multimodal Large Language Models (MLLMs) have demonstrated proficiency in reasoning tasks such as mathematics and logic, their capacity for long-chain reflective reasoning, a prerequisite for solving complex real-world problems, remains largely underexplored. In this work, we first conduct an extensive empirical investigation to evaluate this capability. Leveraging a carefully designed data synthesis engine, we construct MM-HELIX, a multimodal benchmark consisting 1,260 samples of 42 challenging synthetic tasks that require iterative thinking and backtracking. Empirical results on this benchmark reveal that existing MLLMs exhibit significant performance deficits in long-chain reflective reasoning. To address this limitation, we generate post-training data and further explore learning paradigms for exploiting such data. We first develop the Step-Elicited Response Generation pipeline to create MM-HELIX-100K, a large-scale dataset of 100k high-quality, reflective reasoning traces for instruction-tuning stage. Given that standard Reinforcement Learning fails on complex tasks due to sparse reward signals and catastrophic forgetting after Supervised Fine-Tuning, we propose Adaptive Hybrid Policy Optimization (AHPO), a novel training strategy that dynamically unifies offline supervision and online optimization into a single stage. This strategy enables the model to learn from expert data when rewards are sparse and conduct independent exploration once proficient. When applied to the Qwen2.5-VL-7B baseline, our method achieves a +18.6\% accuracy improvement on MM-HELIX benchmark and demonstrates strong generalization with a +5.7\% average performance gain on general mathematic and logic tasks. Our work demonstrate that reflective reasoning in MLLMs can be effectively learned and generalized, paving the way for developing more capable MLLMs.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08540.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Xiangyu Zhao, Junming Lin, Tianhao Liang, Yifan Zhou, Wenhao Chai, Yuzhe Gu, Weiyun Wang, Kai Chen, Gen Luo, Wenwei Zhang, Junchi Yan, Hua Yang, Haodong Duan, Xue Yang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 98&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; While current Multimodal Large Language Models (MLLMs) have demonstrated proficiency in reasoning tasks such as mathematics and logic, their capacity for long-chain reflective reasoning, a prerequisite for solving complex real-world problems, remains largely underexplored. In this work, we first conduct an extensive empirical investigation to evaluate this capability. Leveraging a carefully designed data synthesis engine, we construct MM-HELIX, a multimodal benchmark consisting 1,260 samples of 42 challenging synthetic tasks that require iterative thinking and backtracking. Empirical results on this benchmark reveal that existing MLLMs exhibit significant performance deficits in long-chain reflective reasoning. To address this limitation, we generate post-training data and further explore learning paradigms for exploiting such data. We first develop the Step-Elicited Response Generation pipeline to create MM-HELIX-100K, a large-scale dataset of 100k high-quality, reflective reasoning traces for instruction-tuning stage. Given that standard Reinforcement Learning fails on complex tasks due to sparse reward signals and catastrophic forgetting after Supervised Fine-Tuning, we propose Adaptive Hybrid Policy Optimization (AHPO), a novel training strategy that dynamically unifies offline supervision and online optimization into a single stage. This strategy enables the model to learn from expert data when rewards are sparse and conduct independent exploration once proficient. When applied to the Qwen2.5-VL-7B baseline, our method achieves a +18.6\% accuracy improvement on MM-HELIX benchmark and demonstrates strong generalization with a +5.7\% average performance gain on general mathematic and logic tasks. Our work demonstrate that reflective reasoning in MLLMs can be effectively learned and generalized, paving the way for developing more capable MLLMs.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.08540</guid>
       <pubDate>Thu, 09 Oct 2025 17:53:58 +0000</pubDate>
     </item>
@@ -319,14 +319,14 @@
     <item>
       <title>ARTDECO: Towards Efficient and High-Fidelity On-the-Fly 3D   Reconstruction with Structured Scene Representation</title>
       <link>https://arxiv.org/abs/2510.08551</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08551.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Guanghao Li, Kerui Ren, Linning Xu, Zhewen Zheng, Changjian Jiang, Xin Gao, Bo Dai, Jian Pu, Mulin Yu, Jiangmiao Pang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 25&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; On-the-fly 3D reconstruction from monocular image sequences is a long-standing challenge in computer vision, critical for applications such as real-to-sim, AR/VR, and robotics. Existing methods face a major tradeoff: per-scene optimization yields high fidelity but is computationally expensive, whereas feed-forward foundation models enable real-time inference but struggle with accuracy and robustness. In this work, we propose ARTDECO, a unified framework that combines the efficiency of feed-forward models with the reliability of SLAM-based pipelines. ARTDECO uses 3D foundation models for pose estimation and point prediction, coupled with a Gaussian decoder that transforms multi-scale features into structured 3D Gaussians. To sustain both fidelity and efficiency at scale, we design a hierarchical Gaussian representation with a LoD-aware rendering strategy, which improves rendering fidelity while reducing redundancy. Experiments on eight diverse indoor and outdoor benchmarks show that ARTDECO delivers interactive performance comparable to SLAM, robustness similar to feed-forward systems, and reconstruction quality close to per-scene optimization, providing a practical path toward on-the-fly digitization of real-world environments with both accurate geometry and high visual fidelity. Explore more demos on our project page: https://city-super.github.io/artdeco/.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08551.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Guanghao Li, Kerui Ren, Linning Xu, Zhewen Zheng, Changjian Jiang, Xin Gao, Bo Dai, Jian Pu, Mulin Yu, Jiangmiao Pang&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 26&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; On-the-fly 3D reconstruction from monocular image sequences is a long-standing challenge in computer vision, critical for applications such as real-to-sim, AR/VR, and robotics. Existing methods face a major tradeoff: per-scene optimization yields high fidelity but is computationally expensive, whereas feed-forward foundation models enable real-time inference but struggle with accuracy and robustness. In this work, we propose ARTDECO, a unified framework that combines the efficiency of feed-forward models with the reliability of SLAM-based pipelines. ARTDECO uses 3D foundation models for pose estimation and point prediction, coupled with a Gaussian decoder that transforms multi-scale features into structured 3D Gaussians. To sustain both fidelity and efficiency at scale, we design a hierarchical Gaussian representation with a LoD-aware rendering strategy, which improves rendering fidelity while reducing redundancy. Experiments on eight diverse indoor and outdoor benchmarks show that ARTDECO delivers interactive performance comparable to SLAM, robustness similar to feed-forward systems, and reconstruction quality close to per-scene optimization, providing a practical path toward on-the-fly digitization of real-world environments with both accurate geometry and high visual fidelity. Explore more demos on our project page: https://city-super.github.io/artdeco/.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.08551</guid>
       <pubDate>Thu, 09 Oct 2025 17:57:38 +0000</pubDate>
     </item>
     <item>
       <title>VideoCanvas: Unified Video Completion from Arbitrary Spatiotemporal   Patches via In-Context Conditioning</title>
       <link>https://arxiv.org/abs/2510.08555</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08555.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Minghong Cai, Qiulin Wang, Zongli Ye, Wenze Liu, Quande Liu, Weicai Ye, Xintao Wang, Pengfei Wan, Kun Gai, Xiangyu Yue&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 52&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce the task of arbitrary spatio-temporal video completion, where a video is generated from arbitrary, user-specified patches placed at any spatial location and timestamp, akin to painting on a video canvas. This flexible formulation naturally unifies many existing controllable video generation tasks--including first-frame image-to-video, inpainting, extension, and interpolation--under a single, cohesive paradigm. Realizing this vision, however, faces a fundamental obstacle in modern latent video diffusion models: the temporal ambiguity introduced by causal VAEs, where multiple pixel frames are compressed into a single latent representation, making precise frame-level conditioning structurally difficult. We address this challenge with VideoCanvas, a novel framework that adapts the In-Context Conditioning (ICC) paradigm to this fine-grained control task with zero new parameters. We propose a hybrid conditioning strategy that decouples spatial and temporal control: spatial placement is handled via zero-padding, while temporal alignment is achieved through Temporal RoPE Interpolation, which assigns each condition a continuous fractional position within the latent sequence. This resolves the VAE's temporal ambiguity and enables pixel-frame-aware control on a frozen backbone. To evaluate this new capability, we develop VideoCanvasBench, the first benchmark for arbitrary spatio-temporal video completion, covering both intra-scene fidelity and inter-scene creativity. Experiments demonstrate that VideoCanvas significantly outperforms existing conditioning paradigms, establishing a new state of the art in flexible and unified video generation.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08555.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Minghong Cai, Qiulin Wang, Zongli Ye, Wenze Liu, Quande Liu, Weicai Ye, Xintao Wang, Pengfei Wan, Kun Gai, Xiangyu Yue&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 56&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; We introduce the task of arbitrary spatio-temporal video completion, where a video is generated from arbitrary, user-specified patches placed at any spatial location and timestamp, akin to painting on a video canvas. This flexible formulation naturally unifies many existing controllable video generation tasks--including first-frame image-to-video, inpainting, extension, and interpolation--under a single, cohesive paradigm. Realizing this vision, however, faces a fundamental obstacle in modern latent video diffusion models: the temporal ambiguity introduced by causal VAEs, where multiple pixel frames are compressed into a single latent representation, making precise frame-level conditioning structurally difficult. We address this challenge with VideoCanvas, a novel framework that adapts the In-Context Conditioning (ICC) paradigm to this fine-grained control task with zero new parameters. We propose a hybrid conditioning strategy that decouples spatial and temporal control: spatial placement is handled via zero-padding, while temporal alignment is achieved through Temporal RoPE Interpolation, which assigns each condition a continuous fractional position within the latent sequence. This resolves the VAE's temporal ambiguity and enables pixel-frame-aware control on a frozen backbone. To evaluate this new capability, we develop VideoCanvasBench, the first benchmark for arbitrary spatio-temporal video completion, covering both intra-scene fidelity and inter-scene creativity. Experiments demonstrate that VideoCanvas significantly outperforms existing conditioning paradigms, establishing a new state of the art in flexible and unified video generation.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.08555</guid>
       <pubDate>Thu, 09 Oct 2025 17:58:59 +0000</pubDate>
     </item>
@@ -340,7 +340,7 @@
     <item>
       <title>Agent Learning via Early Experience</title>
       <link>https://arxiv.org/abs/2510.08558</link>
-      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08558.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Kai Zhang, Xiangchao Chen, Bo Liu, Tianci Xue, Zeyi Liao, Zhihan Liu, Xiyao Wang, Yuting Ning, Zhaorun Chen, Xiaohan Fu, Jian Xie, Yuxuan Sun, Boyu Gou, Qi Qi, Zihang Meng, Jianwei Yang, Ning Zhang, Xian Li, Ashish Shah, Dat Huynh, Hengduo Li, Zi Yang, Sara Cao, Lawrence Jang, Shuyan Zhou, Jiacheng Zhu, Huan Sun, Jason Weston, Yu Su, Yifan Wu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 153&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; A long-term goal of language agents is to learn and improve through their own experience, ultimately outperforming humans in complex, real-world tasks. However, training agents from experience data with reinforcement learning remains difficult in many environments, which either lack verifiable rewards (e.g., websites) or require inefficient long-horizon rollouts (e.g., multi-turn tool use). As a result, most current agents rely on supervised fine-tuning on expert data, which is challenging to scale and generalizes poorly. This limitation stems from the nature of expert demonstrations: they capture only a narrow range of scenarios and expose the agent to limited environment diversity. We address this limitation with a middle-ground paradigm we call early experience: interaction data generated by the agent's own actions, where the resulting future states serve as supervision without reward signals. Within this paradigm we study two strategies of using such data: (1) Implicit world modeling, which uses collected states to ground the policy in environment dynamics; and (2) Self-reflection, where the agent learns from its suboptimal actions to improve reasoning and decision-making. We evaluate across eight diverse environments and multiple model families. Our approaches consistently improve effectiveness and out-of-domain generalization, highlighting the value of early experience. Moreover, in environments with verifiable rewards, our results provide promising signals that early experience offers a strong foundation for subsequent reinforcement learning, positioning it as a practical bridge between imitation learning and fully experience-driven agents.&lt;/p&gt;</description>
+      <description>&lt;p&gt;&lt;img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.08558.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /&gt;&lt;/p&gt;&lt;p&gt;&lt;b&gt;Authors:&lt;/b&gt; Kai Zhang, Xiangchao Chen, Bo Liu, Tianci Xue, Zeyi Liao, Zhihan Liu, Xiyao Wang, Yuting Ning, Zhaorun Chen, Xiaohan Fu, Jian Xie, Yuxuan Sun, Boyu Gou, Qi Qi, Zihang Meng, Jianwei Yang, Ning Zhang, Xian Li, Ashish Shah, Dat Huynh, Hengduo Li, Zi Yang, Sara Cao, Lawrence Jang, Shuyan Zhou, Jiacheng Zhu, Huan Sun, Jason Weston, Yu Su, Yifan Wu&lt;/p&gt;&lt;p&gt;&lt;b&gt;Upvotes:&lt;/b&gt; 175&lt;/p&gt;&lt;p&gt;&lt;b&gt;Summary:&lt;/b&gt; A long-term goal of language agents is to learn and improve through their own experience, ultimately outperforming humans in complex, real-world tasks. However, training agents from experience data with reinforcement learning remains difficult in many environments, which either lack verifiable rewards (e.g., websites) or require inefficient long-horizon rollouts (e.g., multi-turn tool use). As a result, most current agents rely on supervised fine-tuning on expert data, which is challenging to scale and generalizes poorly. This limitation stems from the nature of expert demonstrations: they capture only a narrow range of scenarios and expose the agent to limited environment diversity. We address this limitation with a middle-ground paradigm we call early experience: interaction data generated by the agent's own actions, where the resulting future states serve as supervision without reward signals. Within this paradigm we study two strategies of using such data: (1) Implicit world modeling, which uses collected states to ground the policy in environment dynamics; and (2) Self-reflection, where the agent learns from its suboptimal actions to improve reasoning and decision-making. We evaluate across eight diverse environments and multiple model families. Our approaches consistently improve effectiveness and out-of-domain generalization, highlighting the value of early experience. Moreover, in environments with verifiable rewards, our results provide promising signals that early experience offers a strong foundation for subsequent reinforcement learning, positioning it as a practical bridge between imitation learning and fully experience-driven agents.&lt;/p&gt;</description>
       <guid isPermaLink="false">https://arxiv.org/abs/2510.08558</guid>
       <pubDate>Thu, 09 Oct 2025 17:59:17 +0000</pubDate>
     </item>