Xvfb display not terminated on cdp_driver stop, causing potential memory leak

[montovaneli]

When running the cdp driver with the xvfb=True option, the Xvfb display process is not terminated after calling driver.stop(). This results in an increasing number of Xvfb processes when running scrapers in an infinite loop, potentially leading to a memory leak.

Environment:
OS: Ubuntu 22.04
Python: 3.10.12
Latest SeleniumBase

Steps to Reproduce:

Run the following code example:

import asyncio
import subprocess

from seleniumbase.undetected import cdp_driver


async def main():
    driver = await cdp_driver.cdp_util.start_async(xvfb=True)
    page = await driver.get("about:blank")
    await page.sleep(2)
    driver.stop()

def count_xvfb_processes():
    try:
        cmd = "ps aux | grep Xvfb | grep -v grep | wc -l"
        result = subprocess.run(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            text=True,
            shell=True,
        )
        count = result.stdout.strip()
        print(f"Number of Xvfb processes running: {count}")
        return int(count)
    except Exception as e:
        print(f"Error counting Xvfb processes: {e}")
        return -1

if __name__ == "__main__":
    i = 0
    while i < 10:
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        loop.run_until_complete(main())

        # Count Xvfb processes
        count_xvfb_processes()
        i += 1

Notice that after each iteration, the number of running Xvfb processes increases.

Expected Behavior:
The Xvfb display process should be terminated when driver.stop() is called, ensuring that no residual Xvfb processes remain active.

Actual Behavior:
Xvfb processes continue to run after driver.stop(), leading to an accumulation of processes which may cause a memory leak over time, particularly in scrapers running in an infinite loop.

Thank you for your amazing work!

[mdmintz]

The Xvfb virtual display gets saved to sb_config._virtual_display:

SeleniumBase/seleniumbase/undetected/cdp_driver/cdp_util.py

Line 92 in 3353646

sb_config._virtual_display = _xvfb_display

So you can call quit(), (or equivalent), from that object.

Since multiple drivers running in parallel could be using the same virtual display, it doesn't make sense to quit the virtual display if the driver is quit (because another driver might be using it). Therefore, quit the display separately when it's no longer needed.

To access sb_config use:

from seleniumbase import config as sb_config