avian interpolation issues when syncing position / rotation instead of transform

[Piefayth]

Interpolated entities jitter when syncing Position and Rotation instead of Transform.

For Interpolated entities, network interpolation applies to Position and Rotation in Update. Avian sync happens in FixedUpdate. So if you don't manually extract the Position/Rotation and inject it into the Transform during PostUpdate, you don't get interpolation every frame. There is this guidance in the book and this code, but that will never work without putting RigidBodies on your Interpolated entities. I don't actually think it works if you do that either, but not 100% sure.

This can be resolved in 3d with this user code, ripped straight from avian.

app.add_systems(
    PostUpdate,
    position_to_transform_for_corrections
        .before(TransformSystem::TransformPropagate)
        .after(InterpolationSet::VisualInterpolation),
);

type ParentComponents = (
    &'static GlobalTransform,
    Option<&'static Position>,
    Option<&'static Rotation>,
);

type PosToTransformComponents = (
    &'static mut Transform,
    &'static Position,
    &'static Rotation,
    Option<&'static Parent>,
);

// Copy the network-interpolated Position values to Transform
pub fn position_to_transform_for_interpolated(
    mut query: Query<PosToTransformComponents, With<Interpolated>>,
    parents: Query<ParentComponents, With<Children>>,
) {
    for (mut transform, pos, rot, parent) in &mut query {
        if let Some(parent) = parent {
            if let Ok((parent_transform, parent_pos, parent_rot)) = parents.get(**parent) {
                let parent_transform = parent_transform.compute_transform();
                let parent_pos = parent_pos.map_or(parent_transform.translation, |pos| pos.f32());
                let parent_rot = parent_rot.map_or(parent_transform.rotation, |rot| rot.f32());
                let parent_scale = parent_transform.scale;
                let parent_transform = Transform::from_translation(parent_pos)
                    .with_rotation(parent_rot)
                    .with_scale(parent_scale);

                let new_transform = GlobalTransform::from(
                    Transform::from_translation(pos.f32()).with_rotation(rot.f32()),
                )
                .reparented_to(&GlobalTransform::from(parent_transform));

                transform.translation = new_transform.translation;
                transform.rotation = new_transform.rotation;
            }
        } else {
            transform.translation = pos.f32();
            transform.rotation = rot.f32();
        }
    }
}

[Piefayth]

Doing the above is incomplete, because there is a similar problem with using VisualInterpolation::<Transform>. If a Correction<Position> happens, it must be extracted to the Transform without getting overwritten by VisualInterpolation.

So the signature of the above system could be updated to have a query more like

mut query: Query<PosToTransformComponents, Or<(With<Interpolated>, With<Correction<Position>)>>

But I think, as we discussed, you're right, that Correction should just happen in the fixed schedule and VisualInterpolation should be applied on top.

edit: After some further testing, the solution in this comment isn't even reliable. Seems like there is a system ordering inconsistency, where on some startups the correction gets overwritten by visinterp, but other startups it doesn't. Fun!