Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Sync progress status #260

Draft
wants to merge 15 commits into
base: main
Choose a base branch
from
Draft

WIP: Sync progress status #260

wants to merge 15 commits into from

Conversation

simolus3
Copy link
Contributor

@simolus3 simolus3 commented Mar 24, 2025

This extends SyncStatus to report download progress as it happens. Progress is reported as a (completed, total, fraction) triple (where fraction = completed / total) and available:

  1. For entire sync operations: db.currentStatus.downloadProgress.untilCompletion.
  2. For the progress until a specific priority: db.currentStatus.downloadProgress.untilPriority(myPrio).

The best way to get a look and feel for the API may be to look at this example usage:

final status = snapshot.requireData;
final (didSync, progress) = switch (priority) {
null => (
status.hasSynced ?? false,
status.downloadProgress?.untilCompletion
),
var priority? => (
status.statusForPriority(priority).hasSynced ?? false,
status.downloadProgress?.untilPriority(priority)
),
};
if (didSync) {
return child;
} else {
return Center(
child: Column(
children: [
const Text('Busy with sync...'),
LinearProgressIndicator(value: progress?.fraction),
if (progress case final progress?)
Text('${progress.completed} out of ${progress.total}')
],
),
);
}
},
);

In an actual app, it looks like this:

Bildschirmaufnahme.2025-03-31.um.13.27.20.mov

There is some delay in the sync_local commit not included in the progress bar now, but the amount of operations is updated as it should be.

Copy link
Contributor

@rkistner rkistner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with the overall structure & apis - just added some implementation comments.


/// A widget that shows [child] after a complete sync on the database has
/// completed and a progress bar before that.
class GuardBySync extends StatelessWidget {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a side-note, it could be useful to package some of these widgets in a separate lib. Doesn't have to cover all use cases - I'm thinking just making a couple of sync-related widgets available, to make it easier to get started.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good idea 👍 Given that we already have package:powersync as a Flutter-only library, I think it could reasonably live there (or at least be exported from there, we might want a package:powersync_widgets to share code between the main package and the SQLCipher one). It just can't be in package:powersync_core so that we can keep using the package without Flutter.

}).toList(),
);
} else {
return const CircularProgressIndicator();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not important for right now, but is there a good way to combine the GuardBySync initial-sync progress, and the query progress indicator? It would be nice if there could be a single common widget handling both, instead of having separate progress indicators.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think we could definitely have some kind of query widget that takes SQL + parameters and then shows:

  1. A linear progress indicator until the first sync.
  2. A circular progress indicator while the query is running.
  3. Then finally an inner builder rendering the actual data in all other cases.

This might need some more thought put into it to combine it with e.g. drift, but the basic pattern is always the same (guard for first sync, guard for data on stream, show results). I'm not sure if an abstraction is worth it here or if it'll end up complacting the use-sites further, but it's worth trying out.

Comment on lines 221 to 222
final Map<BucketPriority, int> downloaded;
final Map<BucketPriority, int> target;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be more accurate if we keep track of the counts for each specific bucket, rather than per priority.

In most cases it shouldn't make a difference, but there are some edge cases where it does:

  1. A checkpoint may be "interrupted" before all data is sent. The new checkpoint after the interruption may add buckets, remove buckets, or change the priority of buckets.
  2. The counts are not guaranteed to be 100% accurate in all cases. There are some cases such as compacting buckets, which may alter the count without affecting the integrity - potentially while the client is downloading data. To cater for this, we may want to enforce that:
    a. downloaded <= target for each bucket (never go over 100%).
    b. downloaded = target when we have all data for the bucket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants