Skip to content

Commit

Permalink
[WTF] Add WorkerPool
Browse files Browse the repository at this point in the history
https://bugs.webkit.org/show_bug.cgi?id=174569

Reviewed by Carlos Garcia Campos.

Source/WebCore:

We start using WorkerPool for NicosiaPaintingEngineThreaded instead of glib thread pool.
This makes NicosiaPaintingEngineThreaded platform-independent and usable for WinCairo.

* platform/graphics/nicosia/NicosiaPaintingEngineThreaded.cpp:
(Nicosia::PaintingEngineThreaded::PaintingEngineThreaded):
(Nicosia::PaintingEngineThreaded::~PaintingEngineThreaded):
(Nicosia::PaintingEngineThreaded::paint):
(Nicosia::s_threadFunc): Deleted.
* platform/graphics/nicosia/NicosiaPaintingEngineThreaded.h:

Source/WTF:

This patch adds WorkerPool, which is a thread pool that consists of AutomaticThread.
Since it is based on AutomaticThread, this WorkerPool can take `timeout`: once `timeout`
passes without any tasks, threads in WorkerPool will be destroyed.

We add shouldSleep handler to AutomaticThread to make destruction of threads in WorkerPool moderate.
Without this, all threads are destroyed at once after `timeout` passes.

* WTF.xcodeproj/project.pbxproj:
* wtf/AutomaticThread.cpp:
(WTF::AutomaticThread::AutomaticThread):
(WTF::AutomaticThread::start):
* wtf/AutomaticThread.h:
* wtf/CMakeLists.txt:
* wtf/WorkerPool.cpp: Added.
(WTF::WorkerPool::WorkerPool):
(WTF::WorkerPool::~WorkerPool):
(WTF::WorkerPool::shouldSleep):
(WTF::WorkerPool::postTask):
* wtf/WorkerPool.h: Added.
(WTF::WorkerPool::create):

Tools:

* TestWebKitAPI/CMakeLists.txt:
* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WTF/WorkerPool.cpp: Added.
(TestWebKitAPI::TEST):

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@232619 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
[email protected] committed Jun 8, 2018
1 parent 0caddbd commit c476d16
Show file tree
Hide file tree
Showing 14 changed files with 329 additions and 28 deletions.
28 changes: 28 additions & 0 deletions Source/WTF/ChangeLog
Original file line number Diff line number Diff line change
@@ -1,3 +1,31 @@
2018-06-07 Yusuke Suzuki <[email protected]>

[WTF] Add WorkerPool
https://bugs.webkit.org/show_bug.cgi?id=174569

Reviewed by Carlos Garcia Campos.

This patch adds WorkerPool, which is a thread pool that consists of AutomaticThread.
Since it is based on AutomaticThread, this WorkerPool can take `timeout`: once `timeout`
passes without any tasks, threads in WorkerPool will be destroyed.

We add shouldSleep handler to AutomaticThread to make destruction of threads in WorkerPool moderate.
Without this, all threads are destroyed at once after `timeout` passes.

* WTF.xcodeproj/project.pbxproj:
* wtf/AutomaticThread.cpp:
(WTF::AutomaticThread::AutomaticThread):
(WTF::AutomaticThread::start):
* wtf/AutomaticThread.h:
* wtf/CMakeLists.txt:
* wtf/WorkerPool.cpp: Added.
(WTF::WorkerPool::WorkerPool):
(WTF::WorkerPool::~WorkerPool):
(WTF::WorkerPool::shouldSleep):
(WTF::WorkerPool::postTask):
* wtf/WorkerPool.h: Added.
(WTF::WorkerPool::create):

2018-06-07 Chris Dumez <[email protected]>

Add base class to get WeakPtrFactory member and avoid some boilerplate code
Expand Down
6 changes: 6 additions & 0 deletions Source/WTF/WTF.xcodeproj/project.pbxproj
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,7 @@
DCEE22011CEA7551000C2396 /* BlockObjCExceptions.mm in Sources */ = {isa = PBXBuildFile; fileRef = DCEE21FD1CEA7551000C2396 /* BlockObjCExceptions.mm */; };
E15556F518A0CC18006F48FB /* CryptographicUtilities.cpp in Sources */ = {isa = PBXBuildFile; fileRef = E15556F318A0CC18006F48FB /* CryptographicUtilities.cpp */; };
E311FB171F0A568B003C08DE /* ThreadGroup.cpp in Sources */ = {isa = PBXBuildFile; fileRef = E311FB151F0A568B003C08DE /* ThreadGroup.cpp */; };
E388886F20C9095100E632BC /* WorkerPool.cpp in Sources */ = {isa = PBXBuildFile; fileRef = E388886D20C9095100E632BC /* WorkerPool.cpp */; };
E38C41251EB4E04C0042957D /* CPUTimeCocoa.mm in Sources */ = {isa = PBXBuildFile; fileRef = E38C41241EB4E04C0042957D /* CPUTimeCocoa.mm */; };
E38C41281EB4E0680042957D /* CPUTime.cpp in Sources */ = {isa = PBXBuildFile; fileRef = E38C41261EB4E0680042957D /* CPUTime.cpp */; };
E38D6E271F5522E300A75CC4 /* StringBuilderJSON.cpp in Sources */ = {isa = PBXBuildFile; fileRef = E38D6E261F5522E300A75CC4 /* StringBuilderJSON.cpp */; };
Expand Down Expand Up @@ -617,6 +618,8 @@
E311FB161F0A568B003C08DE /* ThreadGroup.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ThreadGroup.h; sourceTree = "<group>"; };
E3200AB41E9A536D003B59D2 /* PlatformRegisters.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PlatformRegisters.h; sourceTree = "<group>"; };
E33D5F871FBED66700BF625E /* RecursableLambda.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RecursableLambda.h; sourceTree = "<group>"; };
E388886D20C9095100E632BC /* WorkerPool.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WorkerPool.cpp; sourceTree = "<group>"; };
E388886E20C9095100E632BC /* WorkerPool.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WorkerPool.h; sourceTree = "<group>"; };
E38C41241EB4E04C0042957D /* CPUTimeCocoa.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = CPUTimeCocoa.mm; sourceTree = "<group>"; };
E38C41261EB4E0680042957D /* CPUTime.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = CPUTime.cpp; sourceTree = "<group>"; };
E38C41271EB4E0680042957D /* CPUTime.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CPUTime.h; sourceTree = "<group>"; };
Expand Down Expand Up @@ -1125,6 +1128,8 @@
0F3501631BB258C800F0A2A3 /* WeakRandom.h */,
0FE4479A1B7AAA03009498EB /* WordLock.cpp */,
0FE4479B1B7AAA03009498EB /* WordLock.h */,
E388886D20C9095100E632BC /* WorkerPool.cpp */,
E388886E20C9095100E632BC /* WorkerPool.h */,
E4A0AD371A96245500536DF6 /* WorkQueue.cpp */,
E4A0AD381A96245500536DF6 /* WorkQueue.h */,
FE05FAFE1FE5007500093230 /* WTFAssertions.cpp */,
Expand Down Expand Up @@ -1560,6 +1565,7 @@
0F66B2921DC97BAB004A1D3F /* WallTime.cpp in Sources */,
1FA47C8A152502DA00568D1B /* WebCoreThread.cpp in Sources */,
0FE4479C1B7AAA03009498EB /* WordLock.cpp in Sources */,
E388886F20C9095100E632BC /* WorkerPool.cpp in Sources */,
E4A0AD391A96245500536DF6 /* WorkQueue.cpp in Sources */,
E4A0AD3D1A96253C00536DF6 /* WorkQueueCocoa.cpp in Sources */,
FE05FAFF1FE5007500093230 /* WTFAssertions.cpp in Sources */,
Expand Down
7 changes: 4 additions & 3 deletions Source/WTF/wtf/AutomaticThread.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,10 @@ bool AutomaticThreadCondition::contains(const AbstractLocker&, AutomaticThread*
return m_threads.contains(thread);
}

AutomaticThread::AutomaticThread(const AbstractLocker& locker, Box<Lock> lock, RefPtr<AutomaticThreadCondition> condition)
AutomaticThread::AutomaticThread(const AbstractLocker& locker, Box<Lock> lock, RefPtr<AutomaticThreadCondition> condition, Seconds timeout)
: m_lock(lock)
, m_condition(condition)
, m_timeout(timeout)
{
if (verbose)
dataLog(RawPointer(this), ": Allocated AutomaticThread.\n");
Expand Down Expand Up @@ -204,10 +205,10 @@ void AutomaticThread::start(const AbstractLocker&)
// Shut the thread down after a timeout.
m_isWaiting = true;
bool awokenByNotify =
m_waitCondition.waitFor(*m_lock, 10_s);
m_waitCondition.waitFor(*m_lock, m_timeout);
if (verbose && !awokenByNotify && !m_isWaiting)
dataLog(RawPointer(this), ": waitFor timed out, but notified via m_isWaiting flag!\n");
if (m_isWaiting) {
if (m_isWaiting && shouldSleep(locker)) {
m_isWaiting = false;
if (verbose)
dataLog(RawPointer(this), ": Going to sleep!\n");
Expand Down
8 changes: 7 additions & 1 deletion Source/WTF/wtf/AutomaticThread.h
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ class WTF_EXPORT_PRIVATE AutomaticThread : public ThreadSafeRefCounted<Automatic
protected:
// This logically creates the thread, but in reality the thread won't be created until someone
// calls AutomaticThreadCondition::notifyOne() or notifyAll().
AutomaticThread(const AbstractLocker&, Box<Lock>, RefPtr<AutomaticThreadCondition>);
AutomaticThread(const AbstractLocker&, Box<Lock>, RefPtr<AutomaticThreadCondition>, Seconds timeout = 10_s);

// To understand PollResult and WorkResult, imagine that poll() and work() are being called like
// so:
Expand Down Expand Up @@ -168,6 +168,11 @@ class WTF_EXPORT_PRIVATE AutomaticThread : public ThreadSafeRefCounted<Automatic
// can be sure that the default ones don't do anything (so you don't need a super call).
virtual void threadDidStart();
virtual void threadIsStopping(const AbstractLocker&);

// Control whether this automatic thread should sleep when timeout happens.
// By overriding this function, we can customize how automatic threads will sleep.
// For example, when you have thread pool, you can decrease active threads moderately.
virtual bool shouldSleep(const AbstractLocker&) { return true; }

private:
friend class AutomaticThreadCondition;
Expand All @@ -176,6 +181,7 @@ class WTF_EXPORT_PRIVATE AutomaticThread : public ThreadSafeRefCounted<Automatic

Box<Lock> m_lock;
RefPtr<AutomaticThreadCondition> m_condition;
Seconds m_timeout;
bool m_isRunning { true };
bool m_isWaiting { false };
bool m_hasUnderlyingThread { false };
Expand Down
2 changes: 2 additions & 0 deletions Source/WTF/wtf/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,7 @@ set(WTF_PUBLIC_HEADERS
WindowsExtras.h
WordLock.h
WorkQueue.h
WorkerPool.h
dtoa.h

dtoa/bignum-dtoa.h
Expand Down Expand Up @@ -390,6 +391,7 @@ set(WTF_SOURCES
WallTime.cpp
WordLock.cpp
WorkQueue.cpp
WorkerPool.cpp
dtoa.cpp

dtoa/bignum-dtoa.cc
Expand Down
124 changes: 124 additions & 0 deletions Source/WTF/wtf/WorkerPool.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
/*
* Copyright (C) 2017 Yusuke Suzuki <[email protected]>.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

#include "config.h"
#include "WorkerPool.h"

#include <wtf/NeverDestroyed.h>

namespace WTF {

class WorkerPool::Worker final : public AutomaticThread {
public:
friend class WorkerPool;

Worker(const AbstractLocker& locker, WorkerPool& pool, Box<Lock> lock, RefPtr<AutomaticThreadCondition> condition, Seconds timeout)
: AutomaticThread(locker, lock, condition, timeout)
, m_pool(pool)
{
}

PollResult poll(const AbstractLocker&) final
{
if (m_pool.m_tasks.isEmpty())
return PollResult::Wait;
m_task = m_pool.m_tasks.takeFirst();
if (!m_task)
return PollResult::Stop;
return PollResult::Work;
}

WorkResult work() final
{
m_task();
m_task = nullptr;
return WorkResult::Continue;
}

void threadDidStart() final
{
LockHolder locker(*m_pool.m_lock);
m_pool.m_numberOfActiveWorkers++;
}

void threadIsStopping(const AbstractLocker&) final
{
m_pool.m_numberOfActiveWorkers--;
}

bool shouldSleep(const AbstractLocker& locker) final
{
return m_pool.shouldSleep(locker);
}

private:
WorkerPool& m_pool;
Function<void()> m_task;
};

WorkerPool::WorkerPool(unsigned numberOfWorkers, Seconds timeout)
: m_lock(Box<Lock>::create())
, m_condition(AutomaticThreadCondition::create())
, m_timeout(timeout)
{
LockHolder locker(*m_lock);
for (unsigned i = 0; i < numberOfWorkers; ++i)
m_workers.append(adoptRef(*new Worker(locker, *this, m_lock, m_condition, timeout)));
}

WorkerPool::~WorkerPool()
{
{
LockHolder locker(*m_lock);
for (unsigned i = m_workers.size(); i--;)
m_tasks.append(nullptr); // Use null task to indicate that we want the thread to terminate.
m_condition->notifyAll(locker);
}
for (auto& worker : m_workers)
worker->join();
ASSERT(!m_numberOfActiveWorkers);
}

bool WorkerPool::shouldSleep(const AbstractLocker&)
{
if (m_timeout > 0_s && std::isinf(m_timeout))
return false;

MonotonicTime currentTime = MonotonicTime::now();
if (std::isnan(m_lastTimeoutTime) || (currentTime >= (m_lastTimeoutTime + m_timeout))) {
m_lastTimeoutTime = currentTime;
return true;
}
return false;
}

void WorkerPool::postTask(Function<void()>&& task)
{
LockHolder locker(*m_lock);
m_tasks.append(WTFMove(task));
m_condition->notifyOne(locker);
}

}
68 changes: 68 additions & 0 deletions Source/WTF/wtf/WorkerPool.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/*
* Copyright (C) 2017 Yusuke Suzuki <[email protected]>.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

#pragma once

#include <wtf/AutomaticThread.h>
#include <wtf/Deque.h>
#include <wtf/Function.h>
#include <wtf/NumberOfCores.h>
#include <wtf/Vector.h>

namespace WTF {

class WorkerPool : public ThreadSafeRefCounted<WorkerPool> {
public:
WTF_EXPORT_PRIVATE void postTask(Function<void()>&&);

WTF_EXPORT_PRIVATE ~WorkerPool();

// If timeout is infinity, it means AutomaticThread will be never automatically destroyed.
static Ref<WorkerPool> create(unsigned numberOfWorkers = WTF::numberOfProcessorCores(), Seconds timeout = Seconds::infinity())
{
ASSERT(numberOfWorkers >= 1);
return adoptRef(*new WorkerPool(numberOfWorkers, timeout));
}

private:
class Worker;
friend class Worker;

WTF_EXPORT_PRIVATE WorkerPool(unsigned numberOfWorkers, Seconds timeout);

bool shouldSleep(const AbstractLocker&);

Box<Lock> m_lock;
RefPtr<AutomaticThreadCondition> m_condition;
Seconds m_timeout;
MonotonicTime m_lastTimeoutTime { MonotonicTime::nan() };
unsigned m_numberOfActiveWorkers { 0 };
Vector<Ref<Worker>> m_workers;
Deque<Function<void()>> m_tasks;
};

}

using WTF::WorkerPool;
17 changes: 17 additions & 0 deletions Source/WebCore/ChangeLog
Original file line number Diff line number Diff line change
@@ -1,3 +1,20 @@
2018-06-07 Yusuke Suzuki <[email protected]>

[WTF] Add WorkerPool
https://bugs.webkit.org/show_bug.cgi?id=174569

Reviewed by Carlos Garcia Campos.

We start using WorkerPool for NicosiaPaintingEngineThreaded instead of glib thread pool.
This makes NicosiaPaintingEngineThreaded platform-independent and usable for WinCairo.

* platform/graphics/nicosia/NicosiaPaintingEngineThreaded.cpp:
(Nicosia::PaintingEngineThreaded::PaintingEngineThreaded):
(Nicosia::PaintingEngineThreaded::~PaintingEngineThreaded):
(Nicosia::PaintingEngineThreaded::paint):
(Nicosia::s_threadFunc): Deleted.
* platform/graphics/nicosia/NicosiaPaintingEngineThreaded.h:

2018-06-08 Miguel Gomez <[email protected]>

[GTK][WPE] Wrong result when calling ImageBufferCairo's getImageData()
Expand Down
Loading

0 comments on commit c476d16

Please sign in to comment.