无法在 AWS Lambda 上运行 Playwright with Docker & xvfb



我已经试着用xvfb在AWS Lambda上运行Playwright好几天了。我能够构建一些在本地工作的Docker镜像变体,但当推送到AWS时总是会遇到错误。

Dockerfile:

FROM mcr.microsoft.com/playwright/python:v1.24.0-focal 
COPY index.py /
COPY start.sh /
RUN apt install xvfb
RUN chmod 755 /start.sh
CMD ["/bin/bash", "./start.sh"]

start.sh

#!/bin/bash
xvfb-run --auto-servernum --server-num 1 --server-args "-screen 0, 1920x1080x24" python index.py

index.py

from playwright.sync_api import sync_playwright
def lambda_handler(event, context):
with sync_playwright() as p:
browser = p.chromium.launch(headless=False)
page = browser.new_page()
page.set_default_timeout(0)
page.goto("http://example.com")
title = page.title()
browser.close()
return title

看起来由于某些与权限相关的原因,浏览器无法启动-任何可能导致它的想法&如何解决?

2022-08-30T09:50:28.053+02:00   Traceback (most recent call last):
2022-08-30T09:50:28.053+02:00   File "index.py", line 8, in lambda_handler
2022-08-30T09:50:28.053+02:00   page = browser.new_page()
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/sync_api/_generated.py", line 11188, in new_page
2022-08-30T09:50:28.053+02:00   self._sync(
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_sync_base.py", line 89, in _sync
2022-08-30T09:50:28.053+02:00   return task.result()
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_browser.py", line 174, in new_page
2022-08-30T09:50:28.053+02:00   page = await context.new_page()
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_browser_context.py", line 230, in new_page
2022-08-30T09:50:28.053+02:00   return from_channel(await self._channel.send("newPage"))
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_connection.py", line 43, in send
2022-08-30T09:50:28.053+02:00   return await self._connection.wrap_api_call(
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_connection.py", line 369, in wrap_api_call
2022-08-30T09:50:28.053+02:00   return await cb()
2022-08-30T09:50:28.053+02:00   File "/usr/local/lib/python3.8/dist-packages/playwright/_impl/_connection.py", line 78, in inner_send
2022-08-30T09:50:28.053+02:00   result = next(iter(done)).result()
2022-08-30T09:50:28.053+02:00   playwright._impl._api_types.Error: Browser closed.
2022-08-30T09:50:28.053+02:00   ==================== Browser output: ====================
2022-08-30T09:50:28.053+02:00    <launching> /ms-playwright/chromium-1015/chrome-linux/chrome --disable-field-trial-config --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --no-sandbox --user-data-dir=/tmp/playwright_chromiumdev_profile-gHIdYn --remote-debugging-pipe --no-startup-window
<launching> /ms-playwright/chromium-1015/chrome-linux/chrome --disable-field-trial-config --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-back-forward-cache --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,DialMediaRouteProvider,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater,AvoidUnnecessaryBeforeUnloadCheckSync --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --no-sandbox --user-data-dir=/tmp/playwright_chromiumdev_profile-gHIdYn --remote-debugging-pipe --no-startup-window
2022-08-30T09:50:28.053+02:00   <launched> pid=129
2022-08-30T09:50:28.053+02:00
[pid=129][err] [0830/075021.572233:WARNING:crashpad_client_linux.cc(362)] prctl: Operation not permitted (1)
[pid=129][err] [0830/075021.572233:WARNING:crashpad_client_linux.cc(362)] prctl: Operation not permitted (1)
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00
[pid=129][err] [129:147:0830/075023.537911:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
[pid=129][err] [129:147:0830/075023.537911:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:147:0830/075023.538017:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075023.774216:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075023.774254:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075024.183157:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075024.183204:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.449390:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.453692:ERROR:network_service_instance_impl.cc(461)] Network service crashed, restarting service.
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.510964:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.511033:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.511061:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.511109:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.512161:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:161:0830/075024.512820:ERROR:bus.cc(398)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.570748:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.632383:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.671409:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075024.674327:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075024.854785:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:146:0830/075024.854822:ERROR:bus.cc(398)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075025.701411:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075025.712194:ERROR:network_service_instance_impl.cc(461)] Network service crashed, restarting service.
2022-08-30T09:50:28.053+02:00   [pid=129][err] prctl(PR_SET_NO_NEW_PRIVS) failed
2022-08-30T09:50:28.053+02:00   [pid=129][err] [129:129:0830/075025.793276:ERROR:gpu_process_host.cc(959)] GPU process launch failed: error_code=1002
2022-08-30T09:50:28.053+02:00   [pid=129][err] [0830/075025.850673:ERROR:scoped_ptrace_attach.cc(27)] ptrace: Operation not permitted (1)
2022-08-30T09:50:28.053+02:00   [pid=129][err] Received signal 11 SEGV_MAPERR 000000000000
2022-08-30T09:50:28.053+02:00   [pid=129][err] #0 0x55fa8ab9efb9 base::debug::CollectStackTrace()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #1 0x55fa8ab11853 base::debug::StackTrace::StackTrace()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #2 0x55fa8ab9eac1 base::debug::(anonymous namespace)::StackDumpSignalHandler()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #3 0x7f93d4dc4420 (/usr/lib/x86_64-linux-gnu/libpthread-2.31.so+0x1441f)
2022-08-30T09:50:28.053+02:00   [pid=129][err] #4 0x55fa88ae4589 content::WebContentsImpl::SetDeviceEmulationSize()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #5 0x55fa8864714b content::protocol::EmulationHandler::SetDeviceMetricsOverride()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #6 0x55fa886476e8 content::protocol::EmulationHandler::SetDeviceMetricsOverride()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #7 0x55fa884a52af content::protocol::Emulation::DomainDispatcherImpl::setDeviceMetricsOverride()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #8 0x55fa89b0c0c3 v8_crdtp::UberDispatcher::DispatchResult::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #9 0x55fa886286fd content::DevToolsSession::HandleCommandInternal()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #10 0x55fa8862866b content::DevToolsSession::HandleCommand()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #11 0x55fa8862ac6e base::internal::Invoker<>::RunOnce()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #12 0x55fa8d741a18 ChromeDevToolsSession::HandleCommand()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #13 0x55fa8d740048 ChromeDevToolsManagerDelegate::HandleCommand()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #14 0x55fa8862858e content::DevToolsSession::DispatchProtocolMessageInternal()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #15 0x55fa8862823f content::DevToolsSession::DispatchProtocolMessage()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #16 0x55fa88624162 base::internal::FunctorTraits<>::Invoke<>()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #17 0x55fa8ab5b010 base::TaskAnnotator::RunTaskImpl()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #18 0x55fa8ab6ebfc base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWorkImpl()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #19 0x55fa8ab6e7ad base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #20 0x55fa8ab6f172 base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::DoWork()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #21 0x55fa8ab1834b base::(anonymous namespace)::WorkSourceDispatch()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #22 0x7f93d4c7917d g_main_context_dispatch
2022-08-30T09:50:28.053+02:00   [pid=129][err] #23 0x7f93d4c79400 (/usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.6400.6+0x523ff)
2022-08-30T09:50:28.053+02:00   [pid=129][err] #24 0x7f93d4c794a3 g_main_context_iteration
2022-08-30T09:50:28.053+02:00   [pid=129][err] #25 0x55fa8ab18177 base::MessagePumpGlib::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #26 0x55fa8ab6f3e3 base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #27 0x55fa8ab37aed base::RunLoop::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #28 0x55fa8859dcad content::BrowserMainLoop::RunMainMessageLoop()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #29 0x55fa8859f4f2 content::BrowserMainRunnerImpl::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #30 0x55fa8859b7dc content::BrowserMain()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #31 0x55fa8a67ab88 content::RunBrowserProcessMain()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #32 0x55fa8a67c0da content::ContentMainRunnerImpl::RunBrowser()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #33 0x55fa8a67bc70 content::ContentMainRunnerImpl::Run()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #34 0x55fa8a678e1e content::RunContentProcess()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #35 0x55fa8a6797de content::ContentMain()
2022-08-30T09:50:28.053+02:00   [pid=129][err] #36 0x55fa86f8f227 ChromeMain
2022-08-30T09:50:28.053+02:00   [pid=129][err] #37 0x7f93d3eef083 __libc_start_main
2022-08-30T09:50:28.053+02:00   [pid=129][err] #38 0x55fa86f8f02a _start
2022-08-30T09:50:28.053+02:00   [pid=129][err] r8: aaaaaaaaaaaaaaaa r9: 00007fff828b2460 r10: 0000000000000003 r11: 00000000000002d0
2022-08-30T09:50:28.053+02:00   [pid=129][err] r12: 00007fff828b2730 r13: 00007fff828b2460 r14: 0000000000000000 r15: 00000000000002d0
2022-08-30T09:50:28.053+02:00   [pid=129][err] di: 00003ac400a8e070 si: 00007fff828b2460 bp: 00007fff828b2280 bx: 00003ac40108cc00
2022-08-30T09:50:28.053+02:00   [pid=129][err] dx: aaaaaaaaaaaaaaaa ax: 0000000000000000 cx: 000055fa90552dd8 sp: 00007fff828b2230
2022-08-30T09:50:28.053+02:00   [pid=129][err] ip: 000055fa88ae4589 efl: 0000000000010206 cgf: 002b000000000033 erf: 0000000000000004
2022-08-30T09:50:28.053+02:00   [pid=129][err] trp: 000000000000000e msk: 0000000000000000 cr2: 0000000000000000
2022-08-30T09:50:28.053+02:00   [pid=129][err] [end of stack trace] 

关于您的lambda,我已经取得了一些进展。首先,我使用AWS SAM命令sam build && sam local invoke [fn-name]在本地测试了您的lambda,然后使用sam deploy --guided将堆栈部署到AWS。为此,我为您的lambda代码创建了一个SAM模板,我使用Playwright映像作为基础,并在多阶段构建的第二阶段添加了AWS lambda运行时接口。

当我在AWS上运行lambda时,我注意到如果在入口点脚本中使用xvfb-run,我需要添加--single-process启动参数。经过一些测试,我能够在AWS上使用运行lambda

browser = p.chromium.launch(headless=True, args=["--single-process"])

不幸的是,即使入口点脚本中有xvfb-run,我也无法切换到非无头模式。但是,我相信您希望使用xvfb-run来查看虚拟帧缓冲区中的浏览器窗口。在之前的一个项目中,我成功地将xvfbx11vnc结合使用,建立了与使用headless=False运行的Playwright docker容器的VNC连接。

值得注意的是,连接到Lambda上运行的帧缓冲区可能是不可能的,因为这将是传入流量,因此您可能需要在无头模式下运行Playwright。此外,Lambda最多只运行15分钟。

考虑到这些限制,我建议使用像Fargate这样的服务来运行带有xvfbx11vnc的Docker映像。这将为您的项目提供更多的控制和灵活性。此外,我认为允许传入的VNC通过SSH流量会更容易。

最新更新