Learning Representations, Hybrid Geometric Compression, and Where GNNs Actually Fit

The problem is not "put a GNN on BIM." The problem is to build the right multiscale representation boundary between raw geometry, object identity, constraints, and sequential decision-making. Earlier generations of spatial ML systems often had to choose between raw perception and hand-authored structure. They either stayed too low-level to plan well, or jumped too early into brittle symbolic abstractions. The opportunity now is to learn a multiscale bridge: geometric primitives at the bottom, stable object representations in the middle, typed relational reasoning on top, and deterministic validation around the loop.

Thesis

For this domain, the winning representation is probably: geometric at the bottom, object-centric in the middle, typed-relational at the top, compressed hierarchically across scales, and coupled to hard validators rather than asked to learn validity from scratch. Each object should have its own representation, while the global state is the synchronization of interrelated representations.

In other words: raw points, pixels, and splats are too low-level to plan over directly. A single global embedding is too lossy and too brittle. A pure symbolic BIM graph is too hand-authored and too disconnected from perception. The useful middle ground is a hybrid object-and-relation representation where local geometry is preserved, objects carry their own state, and global coherence emerges through synchronization across relations.

Why Many Teams Got Stuck

A lot of work in this space got trapped in a pre-bitter-lesson local optimum. The old failure pattern looked like this: small, over-curated datasets; brittle hand-made graph schemas; too much hope that "the right architecture" would compensate for weak data coverage; poor interfaces between perception and reasoning; and no serious workflow tooling to collect edit traces, corrections, constraint violations, or counterfactuals.

Teams hit both of the classic walls. The data diversity wall: not enough variation to force the model to learn the right decomposition. And the expressivity wall: representations either collapsed too much detail or stayed too raw to support higher-level reasoning.

The right interim move would have been workflow tooling: instrument the CAD/BIM stack, capture edits, accepts/rejects, revisions, constraints, comments, and issue-resolution traces, build synthetic and semi-synthetic generators, and wait for the underlying representation learners to improve.

Why Now is Different

The ingredients are better now than they were even 12–18 months ago. Recent work suggests a real convergence. The field is shifting from "choose one representation" to "learn a hierarchy of interoperable representations." That is exactly the right shift for this problem.

UniPre3D (CVPR 2025) demonstrates unified 3D point-cloud pretraining using cross-modal Gaussian splatting, with Gaussian primitive prediction as the pretraining task and 2D features injected to stabilize geometry-focused learning. PanoGS (CVPR 2025) builds open-vocabulary 3D scene understanding on Gaussian primitives, with graph cuts and clustering over learned super-primitives rather than over raw geometry. ArcPro (CVPR 2025) introduces architectural programs for structured 3D abstraction of sparse points—important because it treats buildings as structured programs, not just surfaces.

PlaceIt3D (ICCV 2025) demonstrates language-guided object placement in real 3D scenes, a signal that the field is moving from passive scene understanding toward action and placement. SAS (ICCV 2025) achieves 3D scene segmentation by integrating multiple 2D priors into a shared 3D feature space. LangScene-X (ICCV 2025) produces language-embedded 3D scenes with a language quantized compressor—another signal that compression and semantic structure are becoming first-class representation concerns.

What the Representation Should Be

Geometric Substrate

Start from a geometry-native substrate: point clouds, superpoints, Gaussian primitives, octree / hash-grid / tri-plane style spatial features. This layer should preserve pose, orientation, scale, local surface structure, uncertainty, and provenance from the upstream perception stack. Do not ask the graph layer to recover geometry that was already compressed away.

Object-Centric Abstraction

The next layer should convert geometry into persistent entities: walls, slabs, doors, beams, columns, ducts, fixtures, openings. Each object should carry canonical geometry parameters, local frame, material and type, uncertainty, editable state, and links to evidence in the raw perceptual layer.

This is where most systems are still weak. They jump from geometry to scene-level embeddings without creating stable object tokens.

Typed Relational Graph

Only after you have good object tokens should you construct the graph. The graph should not merely encode adjacency. It should encode different relation classes: support and load path, containment, alignment, clearance, connectivity, access and egress, code-relevant separation, "same intended assembly" or "same design motif," and uncertainty and provenance couplings.

This is where GNNs become useful: not as universal learners of geometry, but as constraint propagation and relational synchronization machines over object-centric state.

Hierarchical Compression

You do not want one embedding per building, and you do not want full-fidelity geometry everywhere all the time. You want a Matryoshka-like multiscale representation: primitive-level detail when local precision matters, component-level state for edits and identity, room / zone / floor aggregates for meso-scale reasoning, and building-level summaries for global objectives. The strongest pattern here is probably not one monolithic latent. It is a bundle: geometric latent, semantic latent, relational latent, constraint residuals, and uncertainty state.

The compression must be progressive (more detail when needed), localized (not everything promoted to global state), typed (structural, functional, geometric, and regulatory information should not all collapse into one undifferentiated vector), and reversible enough for planning and debugging.

Hard-Validity Loop

This domain has too many non-negotiable constraints to rely on learned implicit validity alone. Use deterministic checks for impossible intersections, egress minima, code-distance minima, anchoring requirements, obvious structural invalidity, and hard vendor/part compatibility. Then let the learned system handle plausibility, ranking, preference, tradeoffs, search guidance, and recovery from incomplete perception. That is the right division of labor.

Where GNNs Help, and Where They Do Not

GNNs help when the key question is: how does a local change propagate through a structured relational field? Which distant objects become relevant because of support, routing, containment, or regulation? What context should synchronize across nearby and nonlocal entities?

GNNs do not help much if you ask them to replace the perceptual stack, learn every hard rule implicitly, operate directly on raw massive geometry without compression, or stand in for search or planning when explicit optimization is the real problem.

So the best system is not "GNN-first." It is: perception, compression into stable entities, typed graph reasoning, hard validation, search / RL / planner loop.

A Preferred Hybrid Model

Stage A: Perceptual encoding. Multi-view 2D backbone plus 3D geometric backbone. Predict Gaussian / superpoint / octree primitives. Distill open-vocabulary and texture priors from strong 2D models.

Stage B: Object formation. Cluster and parse primitives into components. Infer canonicalized object parameters when possible. Attach uncertainty and evidence links.

Stage C: Typed graph construction. Heterogeneous graph over components, spaces, systems, and hierarchy levels. Message passing only over relations that matter. Explicit relation typing and edge features.

Stage D: Geometric compression. Maintain multiscale latent states. Coarsen graph for long-range reasoning. Preserve local detail through residual retrieval instead of global embedding inflation.

Stage E: Control and optimization. Deterministic validators for hard failures. Learned energy or scoring model for soft objectives. Planner / RL / search over graph edit actions.

This gives you a system that is both more scalable and more legible.

Data Strategy: The Real Moat

The best training data is not just scans, BIM files, and labels. It is also edit histories, reviewer comments, accepted vs rejected proposals, code-check failures, exception requests, vendor substitutions, geometry repair actions, sequence-of-work traces, designer disagreement, and time-to-fix and cost-to-fix.

That data teaches the system where the representation was insufficient, which constraints are common failure modes, which ambiguities matter operationally, and what "good enough to move forward" actually looks like. If you are sitting on or can instrument workflow exhaust, that may be more important than the exact model choice.

Build for Future Model Gains

Do not build only for what current models can do. Build the system so that when models get better at invariance or compositional generalization, the product gets simpler instead of needing a rewrite.

That means separating the stack into stable interfaces that should survive model progress and replaceable learned components that should improve with new backbones, pretraining, or post-training.

What should stay stable: object identity and edit history, a typed scene graph or scene program, deterministic validators and constraint checkers, action grammar for graph edits, evidence links back to raw geometry, workflow traces and repair history, and evaluation suites for symmetry shift, composition shift, and constraint satisfaction.

What should remain replaceable: the perceptual encoder, the geometric compression scheme, the graph and message-passing backbone, the planner or RL/search layer, the amount of hand-authored canonicalization, and the amount of explicit curriculum shaping required.

That is the future-proofing principle: freeze the world interface, not the current model trick.

The Design Response Matrix

If invariance improves but compositionality does not: the model gets better at robust perception and matching, but still weak at novel structural recombination. Simplify input normalization, keep strong validators, keep search and planning external, and use the model mainly for better grounding, grouping, and scoring.

If compositionality improves but invariance does not: the model can reason over structure well, but perception remains brittle. Invest in better perceptual front ends and evidence tracking, keep explicit object formation, and let the stronger model operate on object/graph/program representations rather than raw geometry.

If both improve: you can seriously tighten the loop. Perception, object formation, graph reasoning, and planning can share a more unified state. More of the repair and placement loop can become learned. Some heuristic search can be replaced with learned proposal plus validator loops. Even here, keep deterministic validators. In AEC-like domains, they are not scaffolding—they are part of the substrate.

The bottleneck is not whether we can run message passing on a building graph. The bottleneck is whether the system has learned the right compressions: enough locality to preserve geometry, enough object identity to persist edits, enough relational structure to propagate constraints, and enough workflow signal to know what counts as a meaningful correction.