我想自动化从UCSC基因组浏览器提取信息的过程,以节省大量手动输入。以下代码到达我认为是表单的正确部分,但我无法提取结果:
from mechanize import Browser
from bs4 import BeautifulSoup
import requests
import re
br = Browser()
url = 'http://genome-euro.ucsc.edu/cgi-bin/hgTables?hgsid=201790284_dkwVYFu7V6ISmTzFGlXzo23aUhXk'
br.set_handle_robots( False )
for form in br.forms():
if form['position']:
print form['position']
form['position'] = 'chr9:21802635-21865969'
print form['position']
break
for form in br.forms():
if form['hgta_doTopSubmit']:
br.submit()
第一部分工作正常,并更改要查询的基因组的位置。第二部分似乎没有提交任何东西。它返回此错误:
回溯(最近一次调用): 文件"C:\Documents and Settings\Silvia\Desktop\Copy\web_scraping\UCSC\table_browser_form.py",第 26 行,位于 br.submit() 文件 "C:\Python27\lib\site-packages\mechanize-0.2.5-py2.7.egg\mechanize_mechanize.py",第 541 行,提交中 return self.open(self.click(*args, **kwds)) 文件 "C:\Python27\lib\site-packages\mechanize-0.2.5-py2.7.egg\mechanize_mechanize.py",第 530 行,单击 request = self.form.click(*args, **kwds)属性错误:"NoneType"对象没有属性"单击"
手动,网站返回与有关此位置的详细信息相对应的文本屏幕,但我似乎无法在此处提取它。有人对如何解决这个问题有任何建议吗?我只需要存储输出的结果 - 相当于用户在网站上按"获取结果"。非常感谢。
您可以随时使用 requests
发出请求:
import requests
url = 'http://genome-euro.ucsc.edu/cgi-bin/hgTables?hgsid=201790284_dkwVYFu7V6ISmTzFGlXzo23aUhXk'
session = requests.Session()
params = {
'hgsid': '201790284_dkwVYFu7V6ISmTzFGlXzo23aUhXk',
'jsh_pageVertPos': '0',
'clade': 'mammal',
'org': 'Human',
'db': 'hg19',
'hgta_group': 'genes',
'hgta_track': 'refGene',
'hgta_table': 'refFlat',
'hgta_regionType': 'range',
'position': 'chr9:21802635-21865969',
'hgta_outputType': 'gff',
'boolshad.sendToGalaxy': '0',
'boolshad.sendToGreat': '0',
'boolshad.sendToGenomeSpace': '0',
'hgta_outFileName': '',
'hgta_compressType': 'none',
'hgta_doTopSubmit': 'get output'
}
response = session.post(url, data=params)
print response.content
指纹:
chr9 hg19_refFlat start_codon 21802748 21802750 0.000000 + . gene_id "MTAP"; transcript_id "MTAP";
chr9 hg19_refFlat CDS 21802748 21802780 0.000000 + 0 gene_id "MTAP"; transcript_id "MTAP";
chr9 hg19_refFlat exon 21802635 21802780 0.000000 + . gene_id "MTAP"; transcript_id "MTAP";
chr9 hg19_refFlat CDS 21815432 21815518 0.000000 + 0 gene_id "MTAP"; transcript_id "MTAP";
...
chr9 hg19_refFlat exon 21861975 21865969 0.000000 + . gene_id "MTAP"; transcript_id "MTAP";