Robust Change Detection Based on Neural Descriptor Fields

Abstract

The ability to reason about changes in the environment is crucial for robots operating over extended periods of time. Agents are expected to capture changes during operation so that actions can be followed to ensure a smooth progression of the working session. However, varying viewing angles and accumulated localization errors make it easy for robots to falsely detect changes in the surrounding world due to low observation overlap and drifted object associations. In this paper, based on the recently proposed category-level Neural Descriptor Fields (NDFs), we develop an object-level online change detection approach that is robust to partially overlapping observations and noisy localization results. Utilizing the shape completion capability and SE(3)-equivariance of NDFs, we represent objects with compact shape codes encoding full object shapes from partial observations. The objects are then organized in a spatial tree structure based on object centers recovered from NDFs for fast queries of object neighborhoods. By associating objects via shape code similarity and comparing local object-neighbor spatial layout, our proposed approach demonstrates robustness to low observation overlap and localization noises. We conduct experiments on both synthetic and real-world sequences and achieve improved change detection results compared to multiple baseline methods.

Approach Overview

Given a sequence of observations as the source and the target, during streaming of the target sequence, the system takes in partial object point clouds from depth sensors and outputs changed objects on-the-fly in the target w.r.t the source.
(a): Given each partial object point cloud, Neural Descriptor Fields (NDFs) represent the object as a shape code encoding the full object shape and recover the object center from full shape reconstruction.
(b): Based on recovered object centers, observed objects are organized in a spatial object tree, which consists of two coordinate interval trees (T_x, T_y) and allows for fast query of neighboring objects.
(c)-(d): Corresponding neighborhoods of the current object are found from the source and the target object tree using object locations, where objects of similar shape codes are matched. For each matched object pair, object graphs of the neighborhood are constructed and compared to determine if the local object layouts are different, which implies a changed object.

Spplementary Video

Paper

Bibtex

@article{fu2022ndfchange, title={Robust Change Detection Based on Neural Descriptor Fields}, author={Fu, Jiahui and Du, Yilun and Singh, Kurran and Tenenbaum, Joshua B. and Leonard, John J.}, booktitle={IROS}, year={2022} }

Our Related Projects

Check out our related projects on neural fields!

Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation

We present Neural Descriptor Fields (NDFs), an SE3 equivariant object representation which enables manipulation of different categories of objects at arbitrary poses from a limited number (5-10) demonstrations.

Learning Signal-Agnostic Manifolds of Neural Fields

We present a method to capture the underlying structure of arbitrary data signals by representing each point of data as a neural field. This enables us to interpolate and generate new samples in image, shape, audio, and audiovisual domains all using the same identical architecture.

Neural Radiance Flow for 4D View Synthesis and Video Processing

We present a method to capture a dynamic scene utilizing a spatial-temporal radiance field. We enforce consistency in this field utilizing a continuous flow field. We show that such an approach enables us to synthesize novel views in dynamic scenes captured using as little as a single monocular video, and further show that our radiance field can be utilized to denoise and super-resolve input images.