Bug 1875859 - Refactor MemoryPool::PurgeExpired and PurgeByFrecency t…

…o reduce overhead and protect against races while sorting. r=necko-reviewers,valentin

A `CacheEntry`'s `mExpirationTime` and `mFrecency` can be updated asynchronously while sorting. This is at best undefined behavior and at worst it can derail `std::sort`'s algorithm out of bounds.
Looking a bit closer it turns out that sorting for `PurgeExpired` is actually not needed and we can have just a `mManagedEntries` linked list in a memory pool instead of two arrays. This way we can also achieve constant O(1) time for unregistering entries when purging.
`PurgeByFrecency` needs sorting and must ensure to not race with modifications of `mFrecency`. We ensure to make some progress on each run to compensate for the cost of sorting.
We also do `PurgeExpired` always, not only if under memory pressure, as it is cheap (no sorting) and interuptable., as well as an emergency handling if `PurgeByFrecency` failed to allocate the auxiliary sort array.

Differential Revision: https://phabricator.services.mozilla.com/D199955

modules/libpref/init/StaticPrefList.yaml

@@ -12695,6 +12695,24 @@
   value: false
   mirror: always
 
+# CacheStorageService::MemoryPool::PurgeByFrecency does a rather expensive
+# sort we cannot interrupt by an incoming cache operation, so we want the
+# purging to do some progress to not have just wasted cycles.
+# Initial default 0, see bug 1879814.
+- name: network.cache.purgebyfrecency_minprogress_disk
+  type: RelaxedAtomicUint32
+  value: 0
+  mirror: always
+
+# CacheStorageService::MemoryPool::PurgeByFrecency does a rather expensive
+# sort we cannot interrupt by an incoming cache operation, so we want the
+# purging to do at least some progress to not have just wasted cycles.
+# Initial default 0, see bug 1879814.
+- name: network.cache.purgebyfrecency_minprogress_memory
+  type: RelaxedAtomicUint32
+  value: 0
+  mirror: always
+
 # When true we will dispatch a background task (separate process) to
 # delete the cache folder at shutdown in order to avoid shutdown hangs.
 - name: network.cache.shutdown_purge_in_background_task

netwerk/cache2/CacheEntry.h

@@ -5,6 +5,7 @@
 #ifndef CacheEntry__h__
 #define CacheEntry__h__
 
+#include "mozilla/LinkedList.h"
 #include "nsICacheEntry.h"
 #include "CacheFile.h"
 
@@ -42,7 +43,10 @@ class CacheStorage;
 class CacheOutputCloseListener;
 class CacheEntryHandle;
 
-class CacheEntry final : public nsIRunnable, public CacheFileListener {
+class CacheEntry final : public nsIRunnable,
+                         public CacheFileListener,
+                         // Used by CacheStorageService::MemoryPool
+                         public LinkedListElement<RefPtr<CacheEntry>> {
  public:
   NS_DECL_THREADSAFE_ISUPPORTS
   NS_DECL_NSIRUNNABLE