小贝子编程

Goutte - Guzzle/DomCrawler语言 - 从HTML表中抓取-非常复杂



我开始使用Goutte从网站获取我需要的信息，它很聪明，它节省了我大量的时间和麻烦。但另一方面，有时会出现异常，我不知道是什么原因造成的。这是我现在从上面摘录的一页:http://lifecenter.sgst.cn/hit/search.jsp?key1=%27a%27& key2 = % 27 b % 27和key3 = % 27 c % 27, key4 = % d % 27日27日,key5 = C0305

我一直在浏览这个网站，跟随链接，网站上的每个页面都有自己的表格。这个表看起来很简单，下面是其中的一些HTML:

<table align="center" border="1" cellpadding="4" cellspacing="1" width="927">
<tbody><tr bgcolor="#3399FF">
<th align="center">ENTRY</th>
<th align="center">COMPOUND</th>
<th align="center">HERB</th>
<th align="center">TARGET ID</th>
<th align="center">TARGET NAME</th>
<th align="center">TARGET TYPE</th>
</tr>
</tbody><tbody id="table2">

<tr>
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0595">HIT001882</a></td>
<td align="center">linalool</td>
<td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>
<td align="center"><a href="Protein_ID.jsp?pid=T0595&amp;protein=Katanin p60 ATPase-containing subunit A1">T0595</a></td>
<td align="center">Katanin p60 ATPase-containing subunit A1</td>
<td align="center">Direct Target</td>
</tr>
<tr>
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0596">HIT001883</a></td>
<td align="center">linalool</td>
<td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>
<td align="center"><a href="Protein_ID.jsp?pid=T0596&amp;protein=Adenosine receptor A2a">T0596</a></td>
<td align="center">Adenosine receptor A2a</td>
<td align="center">Direct Target</td>
</tr>
<tr>
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0040">HIT001885</a></td>
<td align="center">linalool</td>
<td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>
<td align="center"><a href="Protein_ID.jsp?pid=T0040&amp;protein=Nitric oxide synthase, inducible">T0040</a></td>
<td align="center">Nitric oxide synthase, inducible</td>
<td align="center">Direct Target</td>
</tr>
<tr>
<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0054">HIT001884</a></td>
<td align="center">linalool</td>
<td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>
<td align="center"><a href="Protein_ID.jsp?pid=T0054&amp;protein=Prostaglandin G/H synthase 2">T0054</a></td>
<td align="center">Prostaglandin G/H synthase 2</td>
<td align="center">Indirect Target</td>
</tr>
</tbody>
</table>
所以我做的第一件事是选择<tbody>因为我需要的所有数据都在那里: 
    $tbody = $compoundPage->filter('tbody#table2');

一切看起来都很好，当我运行:

> exit(dump($compoundPage->filter('tbody#table2')->html()));

结果如下:

<tr>n
    <td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0595">HIT001882</a></td>rn
    <td align="center">linalool</td>rn
    <td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Benth(bai hua xia ku cao|白花夏枯草); Flos Micheliae albae(bai lan hua|白兰花); bai siu zi; Caulis Perillae frutescentis(bai su geng|白苏梗); Folium Perillae frutescentis(bai su ye|白苏叶); Magnolia denudata(bai yu lan|白玉兰); Fructus Litseae(bi chen jia|毕澄茄); Radix Bupleuri chinensis(chai hu|柴胡); Pericarpium Citri Reticulatae(chen pi|陈皮); Radix albiflorae(chou jie cao gen|臭节草根); Ligusticum brachylobum Franch(chuan fang feng|川防风); Radix chuanxiong; Rhizoma Chuanxiong(chuan xiong|川芎); chun sha hua; Abies nephrolepis(cou leng shan|臭冷杉); Basho(da ba jiao|大芭蕉); Herba Elsholtziae penduliflorae(da hei tou cao|大黑头草); Mosla dianthera (Ham.) Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>rn
    <td align="center"><a href="Protein_ID.jsp?pid=T0595&amp;protein=Katanin%20p60%20ATPase-containing%20subunit%20A1">T0595</a></td>rn
    <td align="center">Katanin p60 ATPase-containing subunit A1</td>rn
    <td align="center">Direct Target</td>rn
</tr>n
<tr>n
    <td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0596">HIT001883</a></td>rn
    <td align="center">linalool</td>rn
    <td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae Maxim(da ye xiang ru|大叶香薷); (di feng|地枫)</td>rn
    <td align="center"><a href="Protein_ID.jsp?pid=T0596&amp;protein=Adenosine%20receptor%20A2a">T0596</a></td>rn
    <td align="center">Adenosine receptor A2a</td>rn
    <td align="center">Direct Target</td>rn
</tr>n
<tr>n
    <td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0040">HIT001885</a></td>rn
    <td align="center">linalool</td>rn
    <td align="center">Ardisiae japonicae(ai di cha|矮地茶); Fructus Artemiiae argyi(ai shi|艾实); Folium Artemisiae Argyi(ai ye|艾叶); (bai cong|白葱); Dracoccephalum heterophyllum Radix Bupleuri chinensis(</td>rn
    <td align="center"><a href="Protein_ID.jsp?pid=T0040&amp;protein=Nitric%20oxide%20synthase,%20inducible">T0040</a></td>rn
    <td align="center">Nitric oxide synthase, inducible</td>rn
    <td align="center">Direct Target</td>rn
</tr>n
<tr>n
    <td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0054">HIT001884</a></td>rn
    <td align="center">linalool</td>rn
    <td align="center">Ardisiae japonicae</td>rn
    <td align="center"><a href="Protein_ID.jsp?pid=T0054&amp;protein=Prostaglandin%20G/H%20synthase%202">T0054</a></td>rn
    <td align="center">Prostaglandin G/H synthase 2</td>rn
    <td align="center">Indirect Target</td>rn
    </tr>

我看到4行

但是现在我需要遍历这些行，所以我必须更进一步:

$tbody = $compoundPage->filter('tbody#table2 > tr');

运行:exit(dump($compoundPage->filter('tbody#table2 > tr')->html()));

只输出一行:

<td align="center"><a href="detail.jsp?compoundid=C0305&amp;pid=T0595">HIT001882</a></td>rn
<td align="center">linalool</td>rn
<td align="center">Ardisiae japonicae(ai di cha|矮地茶);</td>rn
<td align="center"><a href="Protein_ID.jsp?pid=T0595&amp;protein=Katanin%20p60%20ATPase-containing%20subunit%20A1">T0595</a></td>rn
<td align="center">Katanin p60 ATPase-containing subunit A1</td>rn
<td align="center">Direct Target</td>rn

但是当我像这样计算行数时:

退出(转储(compoundPage ->过滤器(' tbody #表> tr")——> count ())),

显示有4行

当我遍历行时出现问题，如下所示:

    $data = $rows->each(function($row,$i) use ($client) {
      $tds = $row->filter('td');
      $keys = array('entry','compound','herb','target_id','target_name','target_type');
      foreach($keys as $td_i => $key) {
          $data[$key] = $tds->eq($i)->text();
      }
  });

首先，当我数td个元素时，它说只有1个。如果我试着对td节点做点什么，我会得到这个错误:https://gyazo.com/94042ba2e3101ba918a71cc9e8e3be5a

如果我这样做:

退出(转储(行->过滤器(td))),

在循环中，它告诉我有24个节点:

Crawler {#1121 ▼
  #uri: "http://lifecenter.sgst.cn/hit/search.jsp?key1='a'&key2='b'&key3='c'&key4='d'&key5=C0305"
   -defaultNamespacePrefix: "default"
   -namespaces: []
   -baseHref: "http://lifecenter.sgst.cn/hit/search.jsp?key1='a'&key2='b'&key3='c'&key4='d'&key5=C0305"
   -document: DOMDocument {#52 ▶}
   -nodes: array:24 [▼
     0 => DOMElement {#1123 ▶}
     1 => DOMElement {#1124 ▶}
     2 => DOMElement {#1125 ▶}
     3 => DOMElement {#1126 ▶}
     4 => DOMElement {#1127 ▶}
     5 => DOMElement {#1128 ▶}
     6 => DOMElement {#1129 ▶}
     7 => DOMElement {#1130 ▶}
     8 => DOMElement {#1131 ▶}
     9 => DOMElement {#1132 ▶}
     ... (omitted the rest cuz I have to do this indentation manually, its so time consuming.
  ]
  -isHtml: true

我让它工作，我添加了这个:

$ = $ compoundPage行->过滤器(tbody #表二)->过滤器(tr);

而不是:

$rows = $compoundPage->filter('tbody#table2> tr');

，现在它工作正常。我仍然不知道问题是什么，因为这两种选择方法应该是相同的，我很确定我以前尝试过这种方法。在循环遍历表的过程中，我遇到了很多问题，我发现在

$rows = $compoundPage->filter('tbody#table2> tr');

和

$ = $ compoundPage行->过滤器(tbody #表二)->过滤器(tr);

我看到CSS选择方法多次破坏脚本。

相关内容

最新更新


热门标签：

javascript python java c# php android html jquery c++ css ios sql mysql arrays asp.net json python-3.x ruby-on-rails .net sql-server django objective-c excel regex ruby linux ajax iphone xml vba spring asp.net-mvc database wordpress string postgresql wpf windows xcode bash git oracle list vb.net multithreading eclipse algorithm macos powershell visual-studio image forms numpy scala function api selenium