无法使用Selenium(Python)检索脚本执行的html源代码

我正在尝试使用使用硒的python脚本在 http://www.digikey.it/products/en 网站上执行网络搜索，因为页面是动态生成的。我不明白为什么我无法检索源 html，如使用 Mozilla 上的检查器工具看到的那样。我正在使用以下代码：

from selenium import webdriver
# Browse to parts
driver = webdriver.Firefox()
driver.get('http://www.digikey.it/products/en')
#driver.get('http://www.python.org')
html = driver.page_source
print(html)

输出将是：

<html><head><script>var i10cdone =(function(){ function pingBeacon(msg){ var i10cimg = document.createElement('script'); i10cimg.src='/i10c@p1/botox/file/nv-loaded.js?status='+window.encodeURIComponent(msg); i10cimg.onload = function(){ (document.head || document.documentElement).removeChild(i10cimg) }; i10cimg.onerror = function(){ (document.head || document.documentElement).removeChild(i10cimg) }; ( document.head || document.documentElement).appendChild(i10cimg) }; pingBeacon('loaded'); if(String(document.cookie).indexOf('i10c.bdddb=c2-f0103ZLNqAeI3BH6yYOfG7TZlRtCrMwqUo')>=0) { document.cookie = 'i10c.bdddb=;path=/';}; var error=''; function errorHandler(e) { if (e && e.error && e.error.stack ) { error=e.error.stack; } else if( e && e.message ) { error = e.message; } else { error = 'unknown';}} if(window.addEventListener) { window.addEventListener('error',errorHandler, false); } else { if ( window.attachEvent ){ window.attachEvent('onerror',errorHandler); }} return function(){ if (window.removeEventListener) {window.removeEventListener('error',errorHandler); } else { if (window.detachEvent) { window.detachEvent('onerror',errorHandler); }} if(error) { pingBeacon('error-' + String(error).substring(0,500)); document.cookie='i10c.bdddb=c2-f0103ZLNqAeI3BH6yYOfG7TZlRtCrMwqUo;path=/'; }}; })();</script><script>typeof i10cdone === 'function' && i10cdone();</script></head><body><script>setTimeout(function(){document.cookie="i10c.eac23=1; secure";window.location.reload(true);},30);</script></body></html>

好像我错过了一些重要的东西...

在提取页面源代码时，您需要诱导WebDriverWait在 DOM 树中visibility_of_element_located()元素，您可以使用以下定位器策略：

代码块：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:UtilityBrowserDriverschromedriver.exe')
driver.get("https://www.digikey.it/products/en")
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[@id='matching-records-text']")))
print(driver.page_source)

控制台输出：

<html dir="ltr" lang="en-it"><head><script id="eGainOffers" src="https://digikey.egain.cloud/system/Offers.egain?command=GetRulesJS&amp;egofferpageurl=https%3A%2F%2Fwww.digikey.it%2Fproducts%2Fen&amp;egofferpagetitle=Electronic%20Components%20and%20Parts%20Search%20%7C%20DigiKey%20Electronics&amp;egofferpatternchecksum="></script><script id="_egainallowdockchat" src="https://digikey.egain.cloud/system/templates/chat/egain-chat.js"></script><script src="//tags.tiqcdn.com/utag/digikey/main/prod/utag.js" type="text/javascript" async=""></script><script>var i10cdone =(function(){ function pingBeacon(msg){ var i10cimg = document.createElement('script'); i10cimg.src='/i10c@p1/botox/file/nv-loaded.js?status='+window.encodeURIComponent(msg); i10cimg.onload = function(){ (document.head || document.documentElement).removeChild(i10cimg) }; i10cimg.onerror = function(){ (document.head || document.documentElement).removeChild(i10cimg) }; ( document.head || document.documentElement).appendChild(i10cimg) }; pingBeacon('loaded'); if(String(document.cookie).indexOf('i10c.bdddb=c2-f0103ZLNqAeI3BH6yYOfG7TZlRtCrMwqUo')>=0) { document.cookie = 'i10c.bdddb=;path=/';}; var error=''; function errorHandler(e) { if (e && e.error && e.error.stack ) { error=e.error.stack; } else if( e && e.message ) { error = e.message; } else { error = 'unknown';}} if(window.addEventListener) { window.addEventListener('error',errorHandler, false); } else { if ( window.attachEvent ){ window.attachEvent('onerror',errorHandler); }} return function(){ if (window.removeEventListener) {window.removeEventListener('error',errorHandler); } else { if (window.detachEvent) { window.detachEvent('onerror',errorHandler); }} if(error) { pingBeacon('error-' + String(error).substring(0,500)); document.cookie='i10c.bdddb=c2-f0103ZLNqAeI3BH6yYOfG7TZlRtCrMwqUo;path=/'; }}; })();</script><script>typeof i10cdone === 'function' && i10cdone();</script>
<title>Electronic Components and Parts Search | DigiKey Electronics</title>
<meta http-equiv="content-type" content="text/html" charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
<meta http-equiv="expires" content="gio, 26 dic 2019 14:09:12 GMT">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="description" content="Find electronic components from hundreds of suppliers. Immediate shipment available on thousands of components. Free shipping on orders over €50.00">
<meta name="keywords" content="Led, connector, power supply, components, distributor, capacitor, ic, resistor, sensors, transistor, relays, mosfet, electronic parts, opto, inductor, heatsink, potentiometer, diodes, semiconductors, integrated circuits, microcontrollers">
<meta name="generator" content="Digi-Key Search Engine">
<link rel="canonical" href="https://www.digikey.it/products/en">

<link rel="stylesheet" type="text/css" href="/-/media/designer/Global/fonts/fonts.css"><link rel="stylesheet" type="text/css" href="//www.digikey.com/-/media/Designer/Global/EnavHeaderMVC/CSS/global.css?la=en-IT&amp;ts=e11322fe-fe5e-4a84-8b56-9eb9a1a4ff42"><link rel="stylesheet" type="text/css" href="//www.digikey.it/-/media/Designer/Header/ENav%20Update/CSS/header.css?la=en-IT&amp;ts=3cf724b3-881c-4aec-ad13-7792a3aa5485"><link rel="stylesheet" type="text/css" href="//www.digikey.it/-/media/Designer/EnavGDPR/CSS/cookie-notice.css?la=en-IT&amp;ts=26b5b12a-f1e0-49c5-8573-03595c73c9f1"><link rel="stylesheet" type="text/css" href="//www.digikey.it/-/media/Designer/Footer/Footer%20Redesign/MVC/CSS/cobrowse.css?la=en-IT&amp;ts=80ff7116-4027-4909-91af-c930503d3006"><link rel="stylesheet" type="text/css" href="//www.digikey.it/-/media/Designer/Footer/Footer%20Redesign/MVC/CSS/footer.css?la=en-IT&amp;ts=793e36cf-81d3-4b58-877c-2d97006db077">
<link rel="stylesheet" type="text/css" href="//www.digikey.com/-/media/Designer/Global/CSS/jquery-ui.1.11.4.css">
<link rel="stylesheet" type="text/css" href="//www.digikey.com/-/media/Designer/Global/CSS/jquery-ui.dk.css">
<link rel="stylesheet" type="text/css" href="//www.digikey.com/-/media/designer/global/css/jquery.chosen.css">
<link href="/PartSearchBundles/searchCSS?v=GC8oaloR7C0V_BrleYZFs0zdsyMZmwRgpxeet7_7ujg1" rel="stylesheet">


<script type="text/javascript">window.__evergageinjected=true;</script><script src="//cdn.evgnet.com/beacon/digikey/international/scripts/evergage.min.js"></script>
<style type="text/css"></style><script type="text/javascript" async="" charset="utf-8" id="utag_digikey.main_304" src="//tags.tiqcdn.com/utag/digikey/main/prod/utag.304.js?utv=ut4.42.201805151502"></script><script type="text/javascript" async="" charset="utf-8" id="utag_digikey.main_269" src="//tags.tiqcdn.com/utag/digikey/main/prod/utag.269.js?utv=ut4.42.201912132237"></script><script type="text/javascript" async="" charset="utf-8" id="utag_digikey.main_279" src="//tags.tiqcdn.com/utag/digikey/main/prod/utag.279.js?utv=ut4.42.201806131506"></script><script type="text/javascript" async="" charset="utf-8" id="utag_digikey.main_288" src="//tags.tiqcdn.com/utag/digikey/main/prod/utag.288.js?utv=ut4.42.201801010024"></script><script type="text/javascript" async="" charset="utf-8" id="utag_digikey.main_351" src="//tags.tiqcdn.com/utag/digikey/main/prod/utag.351.js?utv=ut4.42.201911132039"></script><script type="text/javascript" async="" charset="utf-8" id="tiqapp" src="//tags.tiqcdn.com/utag/tiqapp/utag.v.js?a=digikey/main/201912182132&amp;cb=1577369143693"></script><link id="mktoForms2BaseStyle" rel="stylesheet" type="text/css" href="https://app-ab06.marketo.com//js/forms2/css/forms2.css"><link id="mktoForms2ThemeStyle" rel="stylesheet" type="text/css" href="https://app-ab06.marketo.com//js/forms2/css/forms2-theme-simple.css"></head>
<body class="ltr flymenu-body gdpr" itemscope="" itemtype="http://schema.org/WebPage">

<header class="header flymenu" id="header" cookie-tracking="ref_page_type=;ref_page_sub_type=;ref_page_id=;header_flag=link">
<img src="//www.digikey.com/mydigikey/home/setpflangcookie?lang=en-IT" style="display:none">
<div class="header__nav-bar">
<span class="icon-menu-offset flymenu__trigger"></span>
<a href="/en" class="header__logo" cookie-tracking="ref_page_event=Header Nav;ref_page_category=Logo">
<img src="//www.digikey.it/-/media/Images/ENavHeader/logo-sm.png?la=en-IT&amp;ts=91bafe1b-8c1d-4775-be5d-07dbb4a69651" alt="Digikey">
</a>
<a href="/mydigikey?site=IT&amp;lang=en">
<span class="icon-user"></span>
</a>
<a href="/ordering/shoppingcart?lang=en">
<span class="icon-cart"></span>
<span class="toggle--cart-total">0</span>
</a>
</div>
<div class="header__top">
<a href="/en" class="header__logo" cookie-tracking="ref_page_event=Header Nav;ref_page_category=Logo">
<img src="//www.digikey.it/-/media/Images/Header/logo_dk.png?la=en-IT&amp;ts=308e3134-d860-47ff-b12c-18d1d9ddc6bc">
</a>
<div class="header__search">
<div class="header__searchbar header-search-panel">
<div class="searchbox-mask"></div>
<div class="searchbox-inner">
<div class="searchbox-inner-categories">
<select class="header__searchtype-wrapper product-search-categories search-product-categories" id="header-search-type"><option value="/products/en?keywords={0}" data-name="Ntt">All Products</option><option value="/en/content-search?t={0}" data-name="Ntt">All Content</option></select>
</div>
<div class="searchbox-inner-searchtext" ref-page-event="Initiate Search">
<input name="keywords" class="header__searchinput dkdirchanger product-search-text search-textbox" type="text" autocomplete="off" style="direction: ltr;">
</div>
<div class="searchbox-inner-searchbutton">
<button class="search-button" type="button" value="" id="header-search-button" cookie-event="ref_page_event=Initiate%20Search;header_flag=link">?</button>
</div>
</div>
</div>
<script type="text/javascript">
window.addEventListener("DOMContentLoaded", function (event) { { searchBarModule() } });
</script>
</div>
<div class="header__clc">
<div class="header__clc-lang-curr">
<div id="lang-dropdown">
<div class="lang-dropdown">
<div>
<ul>
<li>
<span>English</span>
</li>
</ul>
</div>
<div class="lang-dropdown-list">
<ul>
<li>
<span data-lang="it">Italiano</span>
</li>
<li>
<span data-lang="en">English</span>
</li>
<li>
<span data-lang="de">Deutsch</span>
</li>
</ul>
</div>
</div>
</div>
<span id="disabled-lang" style="display: none">English</span>
<span class="header__text-separator"></span>
<div id="cur-dropdown">
<div class="cur-dropdown">
<ul>
<li>
<span>EUR</span>
</li>
</ul>
<div class="cur-dropdown-list">
<ul>
<li>
<span>EUR</span>
</li>
<li>
<span>USD</span>
</li>
</ul>
</div>
</div>
</div>
<span id="disabled-cur" style="display: none">EUR</span>
</div>
<div class="header__clc-locale">
<div id="country-dropdown">
<div class="country-dropdown">
<div>Italy</div>
<div class="country-drawer closed" id="country-drawer">
<input id="country-input" type="text" value="">
<div class="country-dropdown-list">
<span>Africa</span>
<ul>
<li><a href="//www.digikey.co.za">South Africa</a></li>
</ul>
</div>
<div class="country-dropdown-list">
<span>Asia</span>
<ul>
<li><a href="//www.digikey.cn">China</a></li><li><a href="//www.digikey.hk">Hong Kong</a></li><li><a href="//www.digikey.in">India</a></li><li><a href="//www.digikey.jp">Japan</a></li><li><a href="//www.digikey.my">Malaysia</a></li><li><a href="//www.digikey.ph">Philippines</a></li><li><a href="//www.digikey.sg">Singapore</a></li><li><a href="//www.digikey.kr">South Korea</a></li><li><a href="//www.digikey.tw">Taiwan</a></li><li><a href="//www.digikey.co.th">Thailand</a></li>
</ul>
</div>
<div class="country-dropdown-list">
<span>Australia</span>
<ul>
<li><a href="//www.digikey.com.au">Australia</a></li><li><a href="//www.digikey.co.nz">New Zealand</a></li>
</ul>
</div>
<div class="country-dropdown-list">
<span>Europe</span>
<ul>
<li><a href="//www.digikey.at">Austria</a></li><li><a href="//www.digikey.be">Belgium</a></li><li><a href="//www.digikey.bg">Bulgaria</a></li><li><a href="//www.digikey.cz">Czech Republic</a></li><li><a href="//www.digikey.dk">Denmark</a></li><li><a href="//www.digikey.ee">Estonia</a></li><li><a href="//www.digikey.fi">Finland</a></li><li><a href="//www.digikey.fr">France</a></li><li><a href="//www.digikey.de">Germany</a></li><li><a href="//www.digikey.gr">Greece</a></li><li><a href="//www.digikey.hu">Hungary</a></li><li><a href="//www.digikey.ie">Ireland</a></li><li><a href="//www.digikey.it">Italy</a></li><li><a href="//www.digikey.lv">Latvia</a></li><li><a href="//www.digikey.lt">Lithuania</a></li><li><a href="//www.digikey.lu">Luxembourg</a></li><li><a href="//www.digikey.nl">Netherlands</a></li><li><a href="//www.digikey.no">Norway</a></li><li><a href="//www.digikey.pl">Poland</a></li><li><a href="//www.digikey.pt">Portugal</a></li><li><a href="//www.digikey.ro">Romania</a></li><li><a href="//www.digikey.sk">Slovakia</a></li><li><a href="//www.digikey.si">Slovenia</a></li><li><a href="//www.digikey.es">Spain</a></li><li><a href="//www.digikey.se">Sweden</a></li><li><a href="//www.digikey.ch">Switzerland</a></li><li><a href="//www.digikey.co.uk">United Kingdom</a></li>
</ul>
</div>
<div class="country-dropdown-list">
<span>Middle East</span>
<ul>
<li><a href="//www.digikey.co.il">Israel</a></li>
</ul>
</div>
<div class="country-dropdown-list">
<span>North America</span>
<ul>
<li><a href="//www.digikey.ca">Canada</a></li><li><a href="//www.digikey.com.mx">Mexico</a></li><li><a href="//www.digikey.com">United States</a></li>
</ul>
</div>
</div>
</div></div>
</div>
</div>
</div>
<div class="menu-backdrop"></div></header>
<div class="flymenu" id="header__storage">
<div class="header__bottom">
<div class="flymenu__tray">
<a href="/en" class="header__logo" cookie-tracking="ref_page_event=Header Nav;ref_page_category=Logo">
<img src="//www.digikey.it/-/media/Images/Header/logo_dk.png?la=en-IT&amp;ts=308e3134-d860-47ff-b12c-18d1d9ddc6bc" alt="Digi-Key Electronics - Electronic Components Distributor">
</a>
<span class="flymenu__trigger icon-close"></span>
</div>
<div class="flymenu__nav-bar">
<div class="flymenu__container">
<ul class="flymenu__sections" cookie-event="ref_page_event=Header Nav">
<li class="flymenu__section " tabindex="0" data-index="0">
<a class="flymenu__section-title" data-level="1" href="/products/en">Products</a>
<div class="flymenu__section-contents">
<div class="flymenu__column">
<a class="flymenu__back">Back</a>
<ul class="flymenu__items">

相关内容

最新更新

热门标签：