使用Python从站点提取数据

Question

我正在制作一个程序，该程序将从http://www.gujarat.ngosindia.com/

提取数据

我写了以下代码：

def split_line(text):
    words = text.split()
    i = 0
    details = ''
    while ((words[i] !='Contact')) and (i<len(words)):
        i=i+1
        if(words[i] == 'Contact:'):
            break
    while ((words[i] !='Purpose')) and (i<len(words)):
        if (words[i] == 'Purpose:'):
            break
        details = details+words[i]+' '
        i=i+1
    print(details)
def get_ngo_detail(ngo_url):
        html=urlopen(ngo_url).read()
        soup = BeautifulSoup(html)
        table = soup.find('table', {'class': 'border3'})
        td = soup.find('td', {'class': 'border'})
        split_line(td.text)
def get_ngo_names(gujrat_url):
    html = urlopen(gujrat_url).read()
    soup = BeautifulSoup(html)
    for link in soup.findAll('div',{'id':'mainbox'}):
        for text in link.find_all('a'):
            print(text.get_text())
            ngo_link = 'http://www.gujarat.ngosindia.com/'+text.get('href')
            get_ngo_detail(ngo_link)
            #NGO_name = text2.get_text())
a = get_ngo_names(BASE_URL)
print a

但是，当我运行此脚本时，我只会得到非政府组织和联系人的名字。我想要电子邮件，电话号码，网站，目的和联系人。

Answer 1

您的 split_line可以改进。想象您有此文本：

s = """Add: 3rd Floor Khemha House
Drive in Road, Opp Drive in Cinema
Ahmedabad - 380 054
Gujarat
Tel: [phone]
Email: [e-mail]
Website: http://www.aavishkaar.org
Contact: Angha Mitra
Purpose: Economics and Finance, Micro-enterprises
Aim/Objective/Mission: To provide timely financing, management support and professional expertise ..."""

现在，我们可以使用s.split("n")（在每条新行上拆分）将其转换为行，并给出每个项目为行的列表：

lines = s.split("n")
lines == ['Add: 3rd Floor Khemha House', 
          'Drive in Road, Opp Drive in Cinema', 
          ...]

我们可以定义要提取的元素的列表，以及一个词典来保存结果：

targets = ["Contact", "Purpose", "Email"]
results = {}

并通过每一行进行工作，捕获我们想要的信息：

for line in lines:
    l = line.split(":")
    if l[0] in targets:
        results[l[0]] = l[1]

这给了我：

results == {'Contact': ' Angha Mitra', 
            'Purpose': ' Economics and Finance, Micro-enterprises', 
            'Email': ' [e-mail]'}

Answer 2

尝试更好地拆分非政府网站的内容，您可以给"拆分"方法作为正则表达式。例如" [联系] [电子邮件] [电话号码] [网站] [目的] [联系人]

我的正则表达方式可能是错误的，但这是您应该前进的方向。