我正在尝试提取产品的分类(类别，子类别，子子类别)并在网站之间进行比较。对于其他网站，HMTL允许快速识别层次结构。然而，这个特定的正在使用似乎是内部值(class= cat_[一个对他们有意义的数字])。
有了这个，我得到了文本，但没有办法正确提取层次结构：
soup = BeautifulSoup(html, "lxml")
for i in soup.find_all("li", class_="cat_858"):
print(i.text)
季节性是类别，园艺 - 园林绿化和园林工具是子类别，其余都是子类别。但是，我无法找到一种方法来从 html 中提取它以导出为 csv。除此之外，整个块被视为 1 件事，因为它们都共享主要的"li"标签，仅通过不够"逻辑"的类猫编号来区分。
我想要的是能够输出：
SEASONAL Gardening - Landscaping Soil - Potting Mix SEASONAL Gardening - Landscaping Fertilizer: Trees - Flowers - Garden SEASONAL Gardening - Landscaping Fertilizer: Lawn SEASONAL Gardening - Landscaping Manure - Compost SEASONAL Gardening - Landscaping Bulbs - Seeds SEASONAL Gardening - Landscaping Insecticides - Pesticides SEASONAL Gardening - Landscaping Weed Killers - Pest Control SEASONAL Gardening - Landscaping Purchasing Restriction Pesticides SEASONAL Gardening - Landscaping Landscape Fabric - Ground Covers - Mulch SEASONAL Gardening - Landscaping Blocks - Borders - Concrete Slabs SEASONAL Gardening - Landscaping Asphalt Repair & Sealants SEASONAL Gardening - Landscaping Concrete Protectors & Sealants SEASONAL Garden Tools Shovels - Rakes SEASONAL Garden Tools Lawn Shears - Pruners - hedge Trimmers SEASONAL Garden Tools Small Tools & Accessories SEASONAL Garden Tools Trimmers - Lawn Mowers SEASONAL Garden Tools Blowers - Mulchers SEASONAL Garden Tools Wheelbarrows & Accessories SEASONAL Garden Tools Spreaders SEASONAL Garden Tools Sprayers SEASONAL Garden Tools Outdoor Storage
另一个烦人的事情是，似乎我必须手动为页面中的所有主要类别列出类别值(季节性为 858)并循环访问它......
例如：HTML =
"""<nav class="nav-menu-cat-web"><ul class='menu-cat-web clearfix' style='display:none;'>
<li class="cat_858"><a href="http://www.canac.ca/en/popular-seasonal.aspx">Seasonal</a>
<div class="subMenu"><ul>
<li class="cat_863 col-md-3 col-sm-3"><a href="http://www.canac.ca/en/gardening-landscaping/c863/seasonal.aspx">Gardening - Landscaping</a>
<div class="subSubMenu"><ul>
<li class="cat_908"><a href="http://www.canac.ca/en/gardening-landscaping/soil-potting-mix/c863c908/seasonal.aspx">Soil - Potting Mix</a></li>
<li class="cat_903"><a href="http://www.canac.ca/en/gardening-landscaping/fertilizer-trees-flowers-garden/c863c903/seasonal.aspx">Fertilizer: Trees - Flowers - Garden</a></li>
<li class="cat_904"><a href="http://www.canac.ca/en/gardening-landscaping/fertilizer-lawn/c863c904/seasonal.aspx">Fertilizer: Lawn</a></li>
<li class="cat_905"><a href="http://www.canac.ca/en/gardening-landscaping/manure-compost/c863c905/seasonal.aspx">Manure - Compost</a></li>
<li class="cat_898"><a href="http://www.canac.ca/en/gardening-landscaping/bulbs-seeds/c863c898/seasonal.aspx">Bulbs - Seeds</a></li>
<li class="cat_907"><a href="http://www.canac.ca/en/gardening-landscaping/insecticides-pesticides/c863c907/seasonal.aspx">Insecticides - Pesticides</a></li>
<li class="cat_906"><a href="http://www.canac.ca/en/gardening-landscaping/weed-killers-pest-control/c863c906/seasonal.aspx">Weed Killers - Pest Control</a></li>
<li class="cat_1597"><a href="http://www.canac.ca/en/gardening-landscaping/purchasing-restriction-pesticides/c863c1597/seasonal.aspx">Purchasing Restriction Pesticides</a></li>
<li class="cat_900"><a href="http://www.canac.ca/en/gardening-landscaping/landscape-fabric-ground-covers-mulch/c863c900/seasonal.aspx">Landscape Fabric - Ground Covers - Mulch</a></li>
<li class="cat_901"><a href="http://www.canac.ca/en/gardening-landscaping/blocks-borders-concrete-slabs/c863c901/seasonal.aspx">Blocks - Borders - Concrete Slabs</a></li>
<li class="cat_909"><a href="http://www.canac.ca/en/gardening-landscaping/asphalt-repair-sealants/c863c909/seasonal.aspx">Asphalt Repair & Sealants</a></li>
<li class="cat_1596"><a href="http://www.canac.ca/en/gardening-landscaping/concrete-protectors-sealants/c863c1596/seasonal.aspx">Concrete Protectors & Sealants</a></li>
</ul></div>
</li>
<li class="cat_865 col-md-3 col-sm-3"><a href="http://www.canac.ca/en/garden-tools/c865/seasonal.aspx">Garden Tools</a>
<div class="subSubMenu"><ul>
<li class="cat_918"><a href="http://www.canac.ca/en/garden-tools/shovels-rakes/c865c918/seasonal.aspx">Shovels - Rakes</a></li>
<li class="cat_914"><a href="http://www.canac.ca/en/garden-tools/lawn-shears-pruners-hedge-trimmers/c865c914/seasonal.aspx">Lawn Shears - Pruners - hedge Trimmers</a></li>
<li class="cat_917"><a href="http://www.canac.ca/en/garden-tools/small-tools-accessories/c865c917/seasonal.aspx">Small Tools & Accessories</a></li>
<li class="cat_915"><a href="http://www.canac.ca/en/garden-tools/trimmers-lawn-mowers/c865c915/seasonal.aspx">Trimmers - Lawn Mowers</a></li>
<li class="cat_921"><a href="http://www.canac.ca/en/garden-tools/blowers-mulchers/c865c921/seasonal.aspx">Blowers - Mulchers</a></li>
<li class="cat_913"><a href="http://www.canac.ca/en/garden-tools/wheelbarrows-accessories/c865c913/seasonal.aspx">Wheelbarrows & Accessories</a></li>
<li class="cat_916"><a href="http://www.canac.ca/en/garden-tools/spreaders/c865c916/seasonal.aspx">Spreaders</a></li>
<li class="cat_919"><a href="http://www.canac.ca/en/garden-tools/sprayers/c865c919/seasonal.aspx">Sprayers</a></li>
<li class="cat_920"><a href="http://www.canac.ca/en/garden-tools/outdoor-storage/c865c920/seasonal.aspx">Outdoor Storage</a></li>
</ul></div>"""
非常感谢 ！
我不会说我发现这很容易做到。首先，找到(first)大纲的顶部，即包含"季节性"的项目。现在找到包含此元素的ul元素以及所有感兴趣的项。然后找到主菜单项并将其用作主导航点。
from lxml import etree
parser = etree.HTMLParser()
tree = etree.parse('http://www.canac.ca/en/home.aspx', parser)
first = tree.xpath('.//li[@class="cat_858"]/a[@href="http://www.canac.ca/en/popular-seasonal.aspx"]')[0]
containing_ul = first.xpath('../..')[0]
menuEntries = containing_ul.xpath('li/a')
menuEntryTexts = containing_ul.xpath('li/a/text()')
for menuEntry, menuEntryText in zip(menuEntries, menuEntryTexts):
print (menuEntryText)
subHeadings = menuEntry.xpath('../div[@class="subMenu"]/ul/li/a/text()')
for subHeading in subHeadings:
print ('t', subHeading)
link = containing_ul.xpath('//a[text()="%s"]' % subHeading)[0]
for item in link.xpath('../div[@class="subSubMenu"]/ul/li/a/text()'):
print ('tt', item)
结果的初始部分如下：
Seasonal
Gardening - Landscaping
Soil - Potting Mix
Fertilizer: Trees - Flowers - Garden
Fertilizer: Lawn
Manure - Compost
Bulbs - Seeds
Insecticides - Pesticides
Weed Killers - Pest Control
Purchasing Restriction Pesticides
Landscape Fabric - Ground Covers - Mulch
Blocks - Borders - Concrete Slabs
Asphalt Repair & Sealants
Concrete Protectors & Sealants
Garden Tools
Shovels - Rakes
Lawn Shears - Pruners - hedge Trimmers
Small Tools & Accessories
Trimmers - Lawn Mowers
Blowers - Mulchers
Wheelbarrows & Accessories
Spreaders
Sprayers
Outdoor Storage
BBQs - Stoves - Fireplaces
BBQs - Portable Stoves
BBQs Utensils & Accessories
Patio Heaters
Outdoor Fireplaces
Logs - Wood Pellets
Fireplace Tools & Accessories
Wood Stoves - Black Stovepipes
Propane - Butane
Outdoor Furnishings
Furniture - Patio Sets
Swings - Benches
Sun Shelters - Umbrellas
Arches - Arbours
Yard Decor