Skip to content
This repository has been archived by the owner on Dec 9, 2022. It is now read-only.

Concurrent Downloads #22

Merged
merged 80 commits into from
Jan 29, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
f608fd5
Instancing RxPatcher + Added UpdateServerHandler for URLs
Jan 12, 2018
f2803fa
Should be checking if an object is of a class type, not checking for …
Jan 12, 2018
12812a9
More work towards Friendly Names
Jan 12, 2018
edd9cc9
Added Latency testing for patch mirrors
Jan 15, 2018
e25fd6c
Added a file & console logger project
Jan 16, 2018
cf4d7c0
Added some basic logging, much more to come soon
Jan 16, 2018
6296faf
Oops, was not using append...
Jan 16, 2018
6da4afb
Added more logging, update xdelta to new version
Jan 16, 2018
cb473a0
Included xdelta3.1.0 in project file
Jan 16, 2018
bded0fc
Updated logger to be thread-safe
Jan 17, 2018
219ed39
Applying xdelta stage is now multi-threaded
Jan 17, 2018
3ff7f57
Phase Two progress now shown to user
Jan 17, 2018
345218b
Removed UpdateServerSelector as it's no longer required
Jan 17, 2018
bb28c25
Changed UpdateServerHandler to have async latency testing
Jan 17, 2018
e44db00
Made UpdateServerHandler public
Jan 17, 2018
52ab2cc
Disposing of unused objects
Jan 17, 2018
9a94dc1
Commented out the Close and Pause buttons
Jan 17, 2018
0eab94c
Reverted back to Queue for server selection
Jan 17, 2018
2c172e2
New GitIgnore, this is wonderful
Jan 17, 2018
d5fb9ad
Renamed FriendlyName to Name
Jan 17, 2018
b07d85c
Missed these files for Name rename
Jan 17, 2018
9fd2eb6
Removed old xdelta versions
Jan 17, 2018
f7ca6c3
Added logging to the UpdateServerSelector
Jan 18, 2018
0eec46e
All variable names have been standardised as it's good practice.
Jan 18, 2018
b34295b
First pass of multi-threaded downloading
Jan 18, 2018
c750a6c
Concurrent Downloading actually working
Jan 19, 2018
beda8b9
Updated logging, added a new catch statement
Jan 19, 2018
a697252
Fixes & Added active threads to download window
Jan 19, 2018
f829d55
Fixed download window showing invalid statuses
Jan 20, 2018
a5f1246
Added debug window for downloads
Jan 20, 2018
6783c90
Upped the .NET HTTP Connection limit to 50
Jan 20, 2018
1237fe1
Built libraries again.
Jan 20, 2018
b843860
Downloading Debug shows with --log now
Jan 20, 2018
300ec2a
Work towards the Cancel button actually working & DLL again
Jan 20, 2018
8517eae
Added Installer background back in.
Jan 20, 2018
3181c46
More comments
Jan 20, 2018
27beb75
Fixed index out of array issue when first installing
Jan 20, 2018
da0c8a3
Fixes VisualStudio seeing these files as Components
Jan 20, 2018
f092558
Added variable for MaxDownloadThreads
Jan 24, 2018
b2e3041
Added Logging / Threaded xDelta / Friendly Server Names (#21)
Jan 24, 2018
be43688
Removed binary files that are not wanted here.
Jan 27, 2018
f42d3a3
Instancing RxPatcher + Added UpdateServerHandler for URLs
Jan 12, 2018
954bfcf
Merge Conflicts done
Jan 27, 2018
b6a9944
More work towards Friendly Names
Jan 12, 2018
a821b00
Added Latency testing for patch mirrors
Jan 15, 2018
8391d6f
Added some basic logging, much more to come soon
Jan 16, 2018
392e4f3
Added more logging, update xdelta to new version
Jan 16, 2018
cc38bdb
Included xdelta3.1.0 in project file
Jan 16, 2018
ba9efa8
Applying xdelta stage is now multi-threaded
Jan 17, 2018
3b5cc51
Changed UpdateServerHandler to have async latency testing
Jan 17, 2018
4493e35
Made UpdateServerHandler public
Jan 17, 2018
c5ac6c1
Commented out the Close and Pause buttons
Jan 17, 2018
970e44a
Reverted back to Queue for server selection
Jan 17, 2018
0f76d24
New GitIgnore, this is wonderful
Jan 17, 2018
71ecbf3
Missed these files for Name rename
Jan 17, 2018
aa60638
Removed old xdelta versions
Jan 17, 2018
d1231ff
All variable names have been standardised as it's good practice.
Jan 18, 2018
c1295fc
First pass of multi-threaded downloading
Jan 18, 2018
b673f92
Concurrent Downloading actually working
Jan 19, 2018
7de3ba5
Fixes & Added active threads to download window
Jan 19, 2018
722d4de
Fixed download window showing invalid statuses
Jan 20, 2018
28326b6
Added debug window for downloads
Jan 20, 2018
3a79fb9
Upped the .NET HTTP Connection limit to 50
Jan 20, 2018
7027b1c
Built libraries again.
Jan 20, 2018
6ceaca3
Downloading Debug shows with --log now
Jan 20, 2018
269b42c
Work towards the Cancel button actually working & DLL again
Jan 20, 2018
3f3b079
Added Installer background back in.
Jan 20, 2018
a9671d3
More comments
Jan 20, 2018
57e88d2
Fixed index out of array issue when first installing
Jan 20, 2018
a804d7a
Fixes VisualStudio seeing these files as Components
Jan 20, 2018
71edc8d
Added variable for MaxDownloadThreads
Jan 24, 2018
4a5b81b
Removed binary files that are not wanted here.
Jan 27, 2018
d3a00b4
Merge branch 'concurrent_downloads' of https://github.com/TotemArts/R…
Jan 27, 2018
029dbb8
Deleing the .user file
Jan 27, 2018
28bb3bb
Fixed an incorrect merge - Null exception
Jan 27, 2018
9ba88dd
Fixed Join Game button fading when clicking on a server
Jan 28, 2018
ab4c3da
Updated installer, urls, and fixed a small bug with --log
Jan 28, 2018
9b89d80
Deleted files not needed
Jan 28, 2018
750fef3
Updated release.json URL again...
Jan 28, 2018
8493dce
Version number updating
Jan 28, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Concurrent Downloading actually working
  • Loading branch information
AlienXAXS committed Jan 27, 2018
commit b673f92475d36acff9e1665e6614fb0efc9af460
47 changes: 47 additions & 0 deletions RXPatchLib/DirectoryPatcher.cs
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,52 @@ public InstructionModel(FilePatchInstruction x, long y)
}
}

internal async Task Analyze(CancellationToken cancellationToken, Action<IFilePatchAction> callback, Action<DirectoryPatchPhaseProgress> progressCallback, string instructions_hash)
{
// Download instructions
await PatchSource.Load("instructions.json", instructions_hash, cancellationToken, (done, total) => { });

// Open downloaded instructions.json and copy its contents to headerFileContents
string headerFileContents;
using (var file = File.Open(PatchSource.GetSystemPath("instructions.json"), FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
using (var streamReader = new StreamReader(file, Encoding.UTF8))
{
headerFileContents = streamReader.ReadToEnd();
}

// Deserialize JSON data from headerFileContents
List<FilePatchInstruction> instructions = JsonConvert.DeserializeObject<List<FilePatchInstruction>>(headerFileContents);

// Initialize progress-related variables
var progress = new DirectoryPatchPhaseProgress();
var paths = instructions.Select(i => Path.Combine(_targetPath, i.Path));
var sizes = paths.Select(p => !File.Exists(p) ? 0 : new FileInfo(p).Length);
progress.SetTotals(instructions.Count, sizes.Sum());
progress.State = DirectoryPatchPhaseProgress.States.Started;
progressCallback(progress);

// Process each instruction in instructions.json
foreach (var pair in instructions.Zip(sizes, (i, s) => new { Instruction = i, Size = s }))
{
var instruction = pair.Instruction;
cancellationToken.ThrowIfCancellationRequested();
string targetFilePath = Path.Combine(_targetPath, instruction.Path);

// Determine action(s) to take based on instruction; any new actions get passed to the callback
await BuildFilePatchAction(instruction, targetFilePath, callback);

// Update progress
progress.AdvanceItem(pair.Size);
progressCallback(progress);
}

// We're done here; update our State and update progress
progress.State = DirectoryPatchPhaseProgress.States.Finished;
progressCallback(progress);
}


/*
internal async Task Analyze(CancellationToken cancellationToken, Action<IFilePatchAction> callback, Action<DirectoryPatchPhaseProgress> progressCallback, string instructionsHash)
{
// Create our background workers
Expand Down Expand Up @@ -390,6 +436,7 @@ internal async Task Analyze(CancellationToken cancellationToken, Action<IFilePat
progress.State = DirectoryPatchPhaseProgress.States.Finished;
progressCallback(progress);
}
*/

private async Task BuildFilePatchAction(FilePatchInstruction instruction, string targetFilePath, Action<IFilePatchAction> callback)
{
Expand Down
1 change: 1 addition & 0 deletions RXPatchLib/RXPatcher.cs
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ public class RxPatcher
public static RxPatcher Instance => _instance ?? (_instance = new RxPatcher());

public readonly UpdateServerHandler UpdateServerHandler = new UpdateServerHandler();
public UpdateServerSelector UpdateServerSelector = new UpdateServerSelector();

public void AddNewUpdateServer(string url, string friendlyName)
{
Expand Down
22 changes: 19 additions & 3 deletions RXPatchLib/WebPatchSource.cs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ class WebPatchSource : IPatchSource, IDisposable
readonly string _downloadPath;
bool _isDownloading = false;
readonly object _isDownloadingLock = new object();
private byte _downloadsRunning = 0;

public WebPatchSource(RxPatcher patcher, string downloadPath)
{
Expand Down Expand Up @@ -51,21 +52,28 @@ private async Task LockDownload()
{
lock (_isDownloadingLock)
{
if (!_isDownloading)
//if (!_isDownloading)
if (_downloadsRunning < 8)
{
_isDownloading = true;
_downloadsRunning++;
Debug.Print($"{Thread.CurrentThread.ManagedThreadId} | DLRUN: {_downloadsRunning} | ACCEPTING DOWNLOAD");
break;
}
}

await Task.Delay(100);
await Task.Delay(1000);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self note: make a issue to replace this old hack with a proper mutex

}
}

private void UnlockDownload()
{
lock (_isDownloadingLock)
{
_downloadsRunning--;
Debug.Print($"{Thread.CurrentThread.ManagedThreadId} | DLRUN: {_downloadsRunning} | DOWNLOAD COMPLETE");
_isDownloading = false;
}
}

public async Task LoadNew(string subPath, string hash, CancellationToken cancellationToken, Action<long, long> progressCallback)
Expand Down Expand Up @@ -116,11 +124,19 @@ await retryStrategy.Run(async () =>
{
try
{
var thisPatchServer = _patcher.UpdateServerHandler.SelectBestPatchServer();
var rnd = new Random();
var xyz = _patcher.UpdateServerSelector.Hosts.ToArray();
var thisPatchServer = xyz[rnd.Next(0, 3)];
if (thisPatchServer == null)
throw new Exception("Unable to find a better update server");

thisPatchServer.IsUsed = true;

// Download file and wait until finished
RxLogger.Logger.Instance.Write($"Starting file transfer: {_patcher.UpdateServer.Uri.AbsoluteUri}/{_patcher.WebPatchPath}/{subPath}");
await webClient.DownloadFileTaskAsync(new Uri($"{_patcher.UpdateServer.Uri.AbsoluteUri}/{_patcher.WebPatchPath}/{subPath}"), filePath);
RxLogger.Logger.Instance.Write(" > File Transfer Complete");

thisPatchServer.IsUsed = false;

// File finished downoading successfully; allow next download to start and check hash
Expand Down
3 changes: 3 additions & 0 deletions Renegade X Launcher/Views/ApplyUpdateWindow.xaml.cs
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ public object Convert(object value, Type targetType, object parameter, CultureIn
{
if (!(value is DirectoryPatchPhaseProgress)) return DependencyProperty.UnsetValue;
var progress = (DirectoryPatchPhaseProgress)value;
//RxLogger.Logger.Instance.Write($"PhaseProgressPercentageConverter -> Value = {progress.Items.Done} / {progress.Items.Total} | {progress.State}");
if (progress.State == DirectoryPatchPhaseProgress.States.Unstarted)
return 0;
else if (progress.State == DirectoryPatchPhaseProgress.States.Indeterminate)
Expand All @@ -60,6 +61,8 @@ public object Convert(object value, Type targetType, object parameter, CultureIn
var progress = value as DirectoryPatchPhaseProgress;
if (progress == null)
return "unknown";

//RxLogger.Logger.Instance.Write($"PhaseProgressPercentageConverter -> Value = {progress.Items.Done} / {progress.Items.Total} | {progress.State}");
if (progress.State == DirectoryPatchPhaseProgress.States.Unstarted)
return "not started";
else if (progress.State == DirectoryPatchPhaseProgress.States.Finished)
Expand Down