Get started with building apps for spatial computing

Intro

By default apps launch into Shared Space, apps live side by side like a mac
Each app can have multiple windows which are SwiftUI scenes (2d and 3d in the window works)
Shared space apps can also have Volumes - SwiftUI scenes for 3d content inside a bounded space
Volumes can be viewed from multiple angles
Volumes use RealityKit
Full Space makes it so your app is the only app visible
Full Space uses windows & volumes
ARKit is available in Full Space
You can use Passthrough in Full Space
Fully immersive inside of Full Space gives you more capabilities for sound and "customizing the lining of virtual objects"
Windows, Volumes, and Spaces are the foundational elements of Spatial Computing

Eyes and Hands: look at something, "tap their fingers together"
Users can also reach out and "physically" tap the button in space
Taps, long presses, drags, rotations, zooms are available and the system automatically detects them
The same gestures are integrated with SwiftUI and RealityKit
ARKit gives you skeletal hand tracking
Input devices like keyboards and trackpads and with game controllers

No access to the sensors directly
System renders hover effect on view when it is looked at but does not communicate to the app where the person is looking
Permissions are required for scene understanding like detecting walls and furniture
Permissions are required for hand tracking data

SwiftUI Preview provider shows 3D for RealityKit
Previews have an object mode for quick previews of 3D layouts
Use previews for layouts
Use Simulator to test interaction with simulated system gestures
Runtime visualizations with plane estimations, collision shapes, etc. works in simulator and on device

Preview and prepare 3D content for your apps
New feature: particles
Can add 3D sound
RealityKit has "physically based material" that will interact with real world lighting conditions
Create custom materials using MaterialX
Reality Composer Pro can send scenes directly to device without an app
WWDC23 video reference: Meet Reality Composer Pro

New Spatial app template
Initial Scene: window or volume
Immersive Scene Type: Select Space and a second scene will be added, you will have to launch to that from the initial shared space scene
WWDC23 video reference: Develop your first immersive app
Destination Video: shared space app
Happy Beam: immersive game
Hello World: transition between spaces

Windows serve as a starting point in your app, built in SwiftUI
Add WindowGroup to SwiftUI Scene to make a Window
Model3D is a view similar to an Image but displays 3D content, RealityKit renders it
SwiftUI existing gestures work: tap, onHover, rotateGesture, etc.
New SwiftUI gestures: RotateGesture3D, SpatialTapGesture, etc.

RealityView is a new SwiftUI View that allows you to interact with RealityKit
RealityView provides coordinate space conversion and SwiftUI attachments to 3D objects
RealityView has 3 parameters: make closure, update closure, and attachments view builder
Make closure is where you create entities and attach them to the root entity
Update closure gets content and attachments passed in, you add your attachments to the content there; find your attachments using tag identifiers
Attachments view builder is where you create your SwiftUI views for attaching
WWDC23 video reference: Build spatial experiences with RealityKit
WWDC23 video reference: Enhance your spatial computing app with RealityKit

Full Space hides all other apps
"Place app content anywhere" (this implies that location of non-full-space windows are bounded?)
Interact with your surroundings and custom hand interactions (i.e. you only get ARKit in Full Space)
WWDC23 video reference: Meet ARKit for spatial computing
ImmersiveSpace is a SwiftUI scene
.immersiveStyle can be .mixed, .progressive (default), .full