VGGT Survey

VGGT for 3D Reconstruction and Beyond

A Survey of Geometric State Strengthening and Its Applications

    Paper Repository
    VGGT architecture producing multi-view geometry latent features and structured geometric outputs.
    VGGT architecture. Image tokens pass through alternating frame-wise and global attention. Prediction heads decode cameras, depth maps, point maps, and tracks.

    01 · Overview

    A geometric-state view of VGGT research

    VGGT jointly predicts cameras, point maps, depth maps, and tracks from multiple images in one feed-forward pass. Its geometric state contains two components: multi-view geometry latent features and structured geometric outputs.

    State strengthening improves how the geometric state is formed from diverse inputs, computed efficiently, made robust, retained across streams and long sequences, or extended to dynamic scenes. State reuse uses the features, outputs, or their combination in downstream applications.

    Zgeo
    Multi-view geometry latent features. Features formed by aggregating information within and across input views.
    Rgeo
    Structured geometric outputs. Cameras, point maps, depth maps, and tracks decoded from those features.
    Overview of the survey, covering feed-forward reconstruction, geometric-state strengthening and reuse, datasets and benchmarks, and open challenges.
    Overview of the survey taxonomy, datasets and evaluation targets, and open challenges. Parenthesized counts indicate reviewed works.
    Read the abstract

    Development of the VGGT lineage around geometric state

    Geometric-state trend tree with 5 strengthening branches above the center, 5 application branches below, and years and quarters on the left.
    Methods are arranged outward from the central geometric state by when they appeared. The side panels show combinations of strengthening directions and applications, together with potential directions. Open the figure to zoom in.

    02 · Taxonomy

    Strengthen the state. Reuse the state.

    State strengthening5 directions
      State reuse5 applications

        03 · Literature

        Explore the reviewed papers

        04 · Datasets and benchmarks

        Evaluation depends on the evidence a dataset provides

        Explore datasets for 3D reconstruction, novel-view synthesis, and other downstream tasks, together with the tasks, metrics, and comparison conditions used for evaluation.

        The overview shows how many datasets in each group provide observations or annotations that support each evaluation task. Percentages are calculated within each group of reviewed datasets.

        0%100%
        Each cell shows the number and percentage of datasets supporting an evaluation task; color encodes the percentage.

        Search dataset capabilities

        Dataset Dataset group #Scenes Type Source Supported evaluation tasks

        Dataset scale is reported as scene counts. For Oxford RobotCar, OmniWorld, and SPair-71k, a scene count was not identified in the cited source; frame or pair counts indicate scale instead.

        05 · Open challenges

        Open challenges

        01

        Reliable and persistent geometric state

        How can geometric state remain dependable as measurements, sequence length, and scene motion change?

        02

        Adaptive geometric representations for downstream applications

        How can one state expose the information needed by different applications without rebuilding a separate representation for each application?

        03

        Dataset design for benchmarking geometric state

        How can controlled comparisons determine whether application gains arise from changes in or reuse of geometric state?

        Citation

        Cite the survey