LVSPM: Long Sequence View Synthesis and Pose Estimation Model

Chen, Xi; Zhang, Yachi; Chen, Linghao; Liu, Minghua; Su, Hao; Xu, Zexiang; Zhang, Xiaoshuai

LVSPM: Long Sequence View Synthesis and Pose Estimation Model

Xi Chen¹, Yachi Zhang², Linghao Chen², Minghua Liu², Hao Su², Zexiang Xu², Xiaoshuai Zhang²

¹UC San Diego ²Sudo AI GmbH

ECCV 2026

▤Paper ▶Slides ⌘Code soon ⬡Model soon ✓Results

The exact scene from Figure 1: a clean step-5,000 rerun, with predicted cameras projected directly onto the input view and the yellow camera synchronized to the lossless spline frames. Provenance

Paper Overview

An evidence-first visual tour with synchronized LVSPM, AnySplat, and DepthSplat paths; canonical NVS results; all six DL3DV pose baselines; saved camera trajectories; measured inference speed; ablations; and an explicit failure case.

LVSPM in 5 minutes 59 seconds1080p · captions available · download slides

Camera-Path Novel View Synthesis

We rendered complete spline camera paths for all 135 scenes in the canonical DL3DV 64-view split, ranked them using scene-level PSNR, SSIM, and LPIPS, and then reviewed temporal continuity. These four top-eight scenes were chosen for both quality and visual diversity.

Product galleryrank #2 · 30.91 dB

Exterior monumentrank #3 · 29.92 dB

Hotel showroomrank #6 · 29.75 dB

Historic car exhibitrank #8 · 27.50 dB

Every displayed frame is model-generated. Translation uses cubic interpolation and orientation uses rotation splines through held-out target cameras. Selection is performed per scene—never by choosing isolated frames. Complete 135-scene ranking.

Matched Motion Comparisons

Static targets can hide temporal instability. These videos render LVSPM, AnySplat, and DepthSplat along the exact same physical camera path: cubic translation and rotation splines through the same sorted held-out cameras. LVSPM is shown largest; method and pose-input labels are embedded in every frame.

Historic car exhibit64 shared inputs · 18 held-out path cameras

Library interior64 shared inputs · 12 held-out path cameras

LVSPM pose-free contextAnySplat pose-free · shared path via one context Sim(3)DepthSplat ground-truth context posesSelection scene-level, never individual frames

Abstract

We present LVSPM, a generalizable model that jointly estimates camera poses and synthesizes novel views from uncalibrated image collections. LVSPM unifies camera recovery and target-view image prediction as a sequence-to-sequence task: learnable camera tokens and image tokens pass through a long-context backbone, after which lightweight heads predict camera parameters and target RGB images. Trained with RGB images, poses, and intrinsics, the model avoids dense point-map or depth supervision. Test-time-training layers allow LVSPM to process hundreds of input views, including sequences longer than those seen during training. We evaluate pose estimation and novel-view synthesis on RealEstate10K, CO3Dv2, and DL3DV with 16 to 256 input views.

Method

LVSPM represents a scene as a sequence of image and camera tokens. Local view attention exchanges information among nearby views, while large-chunk test-time-training updates carry information through long sequences.

LVSPM pipeline with patchification, view attention, test-time-training blocks, and image and camera outputs — LVSPM uses a shared LaCT backbone and lightweight heads to predict target RGB images and input camera parameters. The central prediction path contains no explicit 3D representation.

Joint prediction. Image synthesis and camera estimation train one backbone, allowing rendering supervision to provide a useful geometric signal.

Long context. TTT hidden-state updates extend the number of input views at inference without changing the architecture.

Accessible supervision. Training uses RGB, poses, and intrinsics rather than dense depth or point maps.

Novel View Synthesis Results

These examples were selected by scene-level PSNR rank over the complete 135-scene canonical DL3DV 64-view evaluation—not by selecting individual frames. Click an image to enlarge it.

Strong case 91st PSNR percentile · 27.51 dB

One of 64 unposed input views of a hotel room — Input view

Held-out ground-truth hotel room view — Ground truth

LVSPM prediction of the hotel room — LVSPM pose-free

Target-aligned AnySplat prediction of the hotel room — AnySplat pose-free

DepthSplat prediction of the hotel room — DepthSplat GT poses

LVSPM preserves sharper doorway, window, and furniture detail. AnySplat retains the room layout but is softer, while DepthSplat shows pronounced geometric warping.

Median case Exact dataset median · 24.19 dB

One of 64 unposed input views in an appliance showroom — Input view

Held-out ground-truth appliance showroom view — Ground truth

LVSPM prediction of the appliance showroom — LVSPM pose-free

Target-aligned AnySplat prediction of the appliance showroom — AnySplat pose-free

DepthSplat prediction of the appliance showroom — DepthSplat GT poses

The median case retains the room layout and appliance edges while repeated fine structures remain challenging.

LVSPM images above are from a fresh cleaned-code rerun. Matched AnySplat and DepthSplat outputs are existing benchmark artifacts. Rerun details and hashes.

Quantitative Results

The equi-temporal protocol keeps sampling density approximately fixed while expanding temporal coverage, so larger input counts represent increasingly large scenes.

Per-scene PSNR difference between LVSPM and AnySplat over 135 DL3DV scenes at 64 input views

134 / 135higher PSNR

129 / 135higher SSIM

124 / 135lower LPIPS

+3.73 dBmedian PSNR difference

Direct scene-ID join between the complete LVSPM and AnySplat 64-view output trees. Five extra stored scenes are excluded from the canonical 135-scene count.

Complete novel-view synthesis metrics on DL3DV

Same 135 canonical scenes for every reported cell

Method	Metric	16	64	128	256
LVSPM	PSNR ↑	25.84	24.36	23.32	22.15
LVSPM	SSIM ↑	.815	.757	.714	.661
LVSPM	LPIPS ↓	.150	.180	.206	.244
AnySplat	PSNR ↑	23.19	20.54	19.45	18.56
AnySplat	SSIM ↑	.759	.647	.597	.548
AnySplat	LPIPS ↓	.157	.247	.293	.338
DepthSplat^†	PSNR ↑	24.81	18.40	OOM	OOM
DepthSplat^†	SSIM ↑	.849	.618	OOM	OOM
DepthSplat^†	LPIPS ↓	.133	.362	OOM	OOM

^†DepthSplat receives ground-truth camera poses; LVSPM and AnySplat are pose-free.

Camera pose on 256-view DL3DV

Relative pose AUC@3 ↑

Fast3R	Cut3R	TTT3R	Flare	VGGT	DA3	LVSPM
8.0	19.2	36.5	71.1	83.6	83.8	90.1

AUC@3 requires relative rotation and translation errors to fall below the strict 3° threshold.

Camera trajectory recovery

On the freshly rerun strong case, LVSPM reaches 93.8 AUC@3 over 64 input cameras. The displayed trajectories are similarity-aligned for visualization; the relative-pose metric itself does not depend on this alignment.

99.3 AUC@3096.1 AUC@593.8 AUC@3

Similarity-aligned predicted and ground-truth camera trajectories for 64 input views

All seven pose methods on one 256-view DL3DV scene. Camera centers are similarity-aligned only for visualization; the displayed AUC values come from the saved scene metrics.

LVSPM pose accuracy across datasets

AUC@3 ↑ under the release evaluation protocol

Dataset	16 views	32 views	64 views	128 views	256 views
RealEstate10K	40.0	50.3	58.2	61.3	—
CO3Dv2	39.2	55.4	64.3	68.8	—
DL3DV	88.4	92.3	92.0	90.0	90.1

Release-authoritative values are shown. RealEstate10K and CO3Dv2 numbers supersede stale rows in the submission manuscript.

Analysis

Rendering supervision improves pose

On the ASE ablation with 32 input and target views, removing RGB supervision reduces strict camera-pose accuracy from 71.9 to 50.9 AUC@3. This supports the joint formulation: learning to render provides useful geometric supervision.

Full LVSPM71.9

Without RGB50.9

Reported efficiency

Parameters: 312M
256-view pose inference: 1.78 s
Novel-view rendering: 58.8 FPS

H100 measurements reported in the paper at 512×288. Training used sequences up to 128 views; 256-view evaluation measures length generalization.

Limitations

LVSPM uses multi-stage training and remains challenged by extreme lighting changes, minimally textured surfaces, reflections, and fine view-dependent structure. Geometry-specialized methods may remain preferable when precise reconstruction is the only objective.

BibTeX

@inproceedings{chen2026lvspm,
  title     = {LVSPM: Long Sequence View Synthesis and Pose Estimation Model},
  author    = {Chen, Xi and Zhang, Yachi and Chen, Linghao and Liu, Minghua
               and Su, Hao and Xu, Zexiang and Zhang, Xiaoshuai},
  booktitle = {European Conference on Computer Vision},
  year      = {2026}
}