Access Violation in Avalonia.Skia.SurfaceRenderTarget.Blit

[jangernert]

Describe the bug

A few Systems running a 24/7 Kiosk UI crashing randomly with the following Exception:

Application: MetriX2.UI.Kiosk.exe
CoreCLR Version: 8.0.23.53103
.NET Version: 8.0.0
Description: The process was terminated due to an unhandled exception.
Exception Info: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Stack:
   at SkiaSharp.SkiaApi.sk_canvas_flush(IntPtr)
   at SkiaSharp.SkiaApi.sk_canvas_flush(IntPtr)
   at Avalonia.Skia.SurfaceRenderTarget.Blit(Avalonia.Platform.IDrawingContextImpl)
   at Avalonia.Rendering.Composition.Server.ServerCompositionTarget.Render()
   at Avalonia.Rendering.Composition.Server.ServerCompositor.RenderCore()
   at Avalonia.Rendering.Composition.Server.ServerCompositor.RenderReentrancySafe()
   at Avalonia.Rendering.Composition.Server.ServerCompositor.Render()
   at Avalonia.Rendering.RenderLoop.TimerTick(System.TimeSpan)
   at Avalonia.Win32.WinRT.Composition.WinUiCompositorConnection+RunLoopHandler.Invoke(Avalonia.Win32.WinRT.IAsyncAction, Avalonia.Win32.WinRT.AsyncStatus)
   at Avalonia.Win32.WinRT.Impl.__MicroComIAsyncActionCompletedHandlerVTable.Invoke(Void*, Void*, Avalonia.Win32.WinRT.AsyncStatus)
   at Avalonia.Win32.Interop.UnmanagedMethods.DispatchMessage(MSG ByRef)
   at Avalonia.Win32.Interop.UnmanagedMethods.DispatchMessage(MSG ByRef)
   at Avalonia.Win32.WinRT.Composition.WinUiCompositorConnection.RunLoop()
   at Avalonia.Win32.WinRT.Composition.WinUiCompositorConnection+<>c__DisplayClass7_0.<TryCreateAndRegisterCore>b__0()

To Reproduce

I've been looking at crash statistics and trying to reproduce the issue for quite some time. But I have found no clear pattern so far. I haven't even managed to reproduce the crash once on my development machine (which runs Linux).

Some users seem to be affected quite heavily. While others encounter the crash rarely or not at all. All while on very similar hardware.

"Affected heavily" in this case means once or twice a day. So not often enough to do some iterative testing with debug builds.

Expected behavior

Not crashing

Avalonia version

11.1.4

OS

Windows

Additional context

A relevant issue from the SkiaSharp tracker: mono/SkiaSharp#2125

It suggests the crash could be caused by not disposing the SKSurface on time after drawing. The avalonia rendering code with all its abstractions is not straight forward to follow. So I'm not sure if this is actually the issue here.

The issue started to appear after a big refactor and visual redesign of the UI code. So maybe how avalonia is used is part of the equation. But without being able to consistently trigger the crash this also just a guess.

I haven't had the time to update to avalonia 11.2, yet. But once I did and the update is deployed for a while I can provide an update if 11.2 fixed the issue.

[kekekeks]

If you suspect that some SKSurface is being finalized, you can try collecting GC and finalization events using dotnet-trace https://github.com/dotnet/diagnostics/blob/main/documentation/dotnet-trace-instructions.md#commonly-used-keywords-for-the-microsoft-windows-dotnetruntime-provider

[vadimart92]

My guess is that you are using some skia primitives to render UI (like SkPaint or SkShader), and forget to dispose some instances. If there is a lot of code you can collect memory dump during this crash (with procdump for example) and you will probably see one more thread that calls Dispose on this instance (from destructor), so you will be able to inspect this object and guess where it was created.

[jangernert]

@vadimart92 your guess seems to be correct

Sorry for the late update. It took some time to refactor and fix the problem. And afterward I deployed a build and monitored a system that was prone to crashing for about a month. No crashes with the new build.

I'll try to roughly describe what I changed in hope it is useful to anyone in the future.

The application in question interfaces with hardware and thus can completely reinitialize on unrecoverable hardware failures. Everything in the ViewModels is managed by Autofac DI. Views were registered via DataTemplates in Xaml. This meant DI was not available in Views. So CustomVisualHandler were created manually and not disposed by Autofac.

Now Views are created by a ViewLocator that has access to the ILifetimeScope and can resolve registered views. Which means they are also automatically disposed when the lifetime is closed.

private static int StartApplication(string[] args)
{
    ReinitializableApplicationLifetime lifetime = new ReinitializableApplicationLifetime();
    AppBuilder appBuilder = BuildAvaloniaApp().SetupWithLifetime(lifetime);
    

    int exitCode;
    do
    {
        using (ILifetimeScope childScope = _container.BeginLifetimeScope(Constants.MainScopeTag))
        {
            Common.ViewLocator viewLocator = childScope.Resolve<Common.ViewLocator>();
            appBuilder.Instance.DataTemplates.Add(viewLocator);

            lifetime.MainWindow = childScope.Resolve<MainWindow>();
            exitCode = lifetime.Start(args);

            // make sure that lifetime has no references to old instances
            lifetime.MainWindow = null;
            appBuilder.Instance.DataTemplates.Remove(viewLocator);
        }

    } while (exitCode == -1);

    return exitCode;
}

Bindings had to be moved from Xaml to C# for this to work.

public RectifiedImage(RectifiedImageCustomVisualHandler handler)
{
    InitializeComponent();
    AttachCustomVisual(handler, host);

    Bind(ImageDataProperty, new Binding(nameof(RectifiedImageModel.Data), BindingMode.OneWay));
}

In addition to the big change a few usings were sprinkled around SKPaint.

Thanks!