VideoWeaver: Evaluating and Evolving Skills for Agentic Long Video Generation

A team of researchers has introduced VideoWeaver, an agent harness and benchmark designed to evaluate and evolve skills for long video generation, where an agent builds its own workflow rather than following a handcrafted pipeline [1]. The benchmark comprises 16 task categories and 285 cases, with references spanning text, image, audio, video, and their combinations [1]. Unlike earlier video agents whose pipelines are manually designed, VideoWeaver allows agent frameworks such as Claude Code, Codex, and OpenClaw to construct and refine their own generation workflows from a single instruction [2]. A central challenge in evaluating such agents is that errors can occur at any stage of the generation process. A plan may be flawed, a tool call may fail, or an intermediate artifact may be corrupted in ways the final clip does not reveal [3]. Existing video benchmarks judge only the final output, leaving these process-level failures undiagnosed [4]. To address this gap, the researchers propose an agent-as-judge that inspects both the execution trace and the final video, grounding its scores in evidence such as metadata and intermediate files [5]. It returns a score and textual feedback for each of 6 process and 7 output metrics, with emphasis on cross-clip consistency [3]. Using this feedback as a refinement signal, the team designed a skill evolution algorithm that progressively refines category-level composition and creator skills and merges the resulting creator skills into a single one [4]. Across multiple frameworks and models, the researchers found that an explicit composition skill improves the generation process over using foundation skills alone, and that skill evolution further improves output quality [1]. Performance varied notably across harness and model choices [2]. The proposed agent-as-judge also aligned well with human judgments, particularly on process metrics [1]. The work arrives amid broader efforts to apply agentic frameworks to long-form video production. A separate framework, ViMax, tackles long-form video generation through coordinated multi-agent workflows, decomposing production into specialized agents for screenwriting, shot planning, character styling, and quality control [9]. That approach uses hierarchical story decomposition with retrieval-augmented generation and graph-based dependency tracking to maintain visual consistency across shots [9]. Both efforts reflect a shift toward systems that can manage the complexity of extended multimodal outputs without relying on fixed pipelines. Code and dataset for VideoWeaver are available on GitHub [1].