我是网络抓取的新手，并试图提取一个特定景点的评论列表。https://www.tripadvisor.co.uk/Attraction_Review-g304141-d4782530-Reviews-Sigiriya_The_Ancient_Rock_Fortress-Sigiriya_Central_Province.html
我正在使用美丽的汤代码，我没有得到任何输出，请帮助我这个
import requests
from bs4 import BeautifulSoup
headers = {
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Methods': 'GET',
'Access-Control-Allow-Headers': 'Content-Type',
'accept': '*/*',
'accept-encoding': 'gzip, deflate',
'accept-language': 'en,mr;q=0.9',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36'}
url = "https://www.tripadvisor.co.uk/Attraction_Review-g304141-d4782530-Reviews-Sigiriya_The_Ancient_Rock_Fortress-Sigiriya_Central_Province.html"
req = requests.get(url,headers=headers,timeout=5,verify=False)
print (req.status_code)
soup = BeautifulSoup(req.content, 'html.parser')
#review = soup.find_all(class_="XllAv H4 _a").get_text()
#print(review)
for x in soup.body.find_all(class_="XllAv H4 _a"):
print(x.text)
很可能站点有API。从动态类中获取数据并不是一个好主意。我写了一个简单的例子，使用基本的注释数据，也许会对你有帮助。
import requests
from bs4 import BeautifulSoup
url = "https://www.tripadvisor.co.uk/Attraction_Review-g304141-d4782530-Reviews-Sigiriya_The_Ancient_Rock_Fortress-Sigiriya_Central_Province.html"
headers = {
'accept': '*/*',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36'
}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.text, 'lxml')
for review in soup.find_all('div', {'data-automation': 'reviewCard'}):
data = [data.getText() for data in review.find_all('span') if data.getText()]
user = data[0]
if 'contributions' in data[1]:
country = 'No Country'
contributions = data[1]
title = data[4]
description = data[5]
else:
country = data[1]
contributions = data[2]
title = data[5]
description = data[6]
bubbles = [bubble.get('aria-label') for bubble in review.find_all('svg') if bubble.has_attr('aria-label')][0]
print(user, bubbles)
print(country, contributions)
print(title)
print(description)
print('----------------------------------')
输出:
TravellingSLduo 5.0 of 5 bubbles
No Country 9 contributions
Enjoyable experience
Even though we've visited the area several times, we had not climbed the rock. The climb was not difficult and the surrounding views were very nice. It was nice to see the Sigirya Frescos as they've been drawn centuries ago. Must visit if you are a lover of cultural heritage.
----------------------------------
Marit 5.0 of 5 bubbles
Maastricht, The Netherlands 4 contributions
Amazing palace on top with an amazing view
Me and my brother climbed this rock, after the pidurangala rock. We climbed the Sigiriya rock around 2pm. It was not too hot, we brought a lot of water. The climb is really easy because it’s only stairs. The ancient palace on top is amazing to walk around. The view is also really incredible. We didn’t see any wasps. Only be careful with some dogs, because some can be a little aggressive.
----------------------------------
Navin 5.0 of 5 bubbles
Nugegoda, Sri Lanka 10 contributions
Sigiriya rock climb with guide
The historical explanation of a guide adds so much more value. Do get a good guide we went with Mr. Albert.
----------------------------------
sandinmyjandals 4.0 of 5 bubbles
Muscat, Oman 38 contributions
Must see on your Sri Lankan itinerary
Well worth a visit. Recommend that you take one of the numerous guides who are at the entrance waiting for tourists to roll up. They all wear a uniform polo shirt so you know that they are offical guides, however as like anywhere you need to be prepared to negotiate a price. We paid $20 US for a guide for 3 hours, our guide was excellent giving you a lot of background to the fortress which you would not get from reading the information pamphlet you get given with your ticket. There are toilets located just by the ticket office, very clean but no toilet paper so make sure you take tissues. There is also a water station before you enter with purified drinking water where you can fill up your water bottle. Be prepared to climb up narrow stairs to get to the top, the view is amazing and worth the climb. When we went in Feb there were no hornets which we had read about however we were told that it was the wrong season. Beware of monkeys everywhere which is all part of the Sri Lankan experience. When you exit the fortress there are numerous toilet blocks which were all clean and well maintained, again take your tissues. Make sure you wear comfortable walking clothes, shoes, hat and take water, there is nowhere to purchase water along the way until the exit. We arrived about 9.30ish which we found to be fine, the heat was bearable, the timing of your visit will all be dependent on the time of year you go.
----------------------------------
Pramod 3.0 of 5 bubbles
No Country 48 contributions
Climb worth the view
About 1200+2 steps on the top most point of the summit. This hill was developed by King Ravana and later occupied by Buddhists monks. These monks were then relocated by King Kashyapa and the hill was developed into a full fledged fort. This fort had a robust defence system containing concentric circles of water, quick sand, crocodiles and barracks of soldiers guarding the palaces.One gets to see remains of water fountains and swimming pools used by the kings of the past. Right next to the swimming pool is a huge elephant shaped stone. On the way up are soldiers barracks, a huge stone resting on a hill, paws of the lion and steps leading to the summit. Do note that the original lion head is no longer in place and only the paws can be seen. Once you reach the summit there is a series of ruins of walls, beautiful scenic view of the forests and a buddha statue in the distance which is the second largest standing statue in the world. (note: not on same this hill).On the way down, there is a cave with a few paintings and a mirror wall(a plastered wall) and resting rooms of the king. Followed by a cobra head shaped rock, an open conference area and another rock pass leading to the outward area where you can buy souvenirs.All in all, I would the say the place is worth visiting since it is a UNESCO heritage site and has a rich past. However things to keep in mind are:1. Guides are optional since every major check point has a bar code readable display and a placard of information.2. It can get extremely hot if you start after 8am. So plan to be there early and do not underestimate the number of steps.3. Steps are not even, damaged in some areas and extremely steep as well. Without proper posture one will definitely slip and injure themselves.4. After a tough climb do remember that all you will see there are ruins that do not speak anything about the history as such for a layman. Limited objects to see but the climb is definitely worth the view.5. Sellers need a bit of haggling and are very polite to tourists.6. Carry a bottle of water since it isn't sold until you reach the exit. Can be difficult on a humid day.7. Keep an eye on kids while you walk through the initial garden patches, since insects are not rare. But nothing to worry for a regular tourist.
----------------------------------
Lee W 5.0 of 5 bubbles
Gold Coast, Australia 51 contributions
Telling it how it is
My Husband and I were here 4 weeks ago and decided to go early in the morning at 7am as the heat and humidity was just horrid anyway and thought it would be less then, but it really wasn't. Anyway we went shortly after opening time which is at 7am. Now, we are both nearly 60 years old and totally unfit fatties and both of us have bad hearts, bad knees and ankles, plus me with an already sprained ankle from a collapsing bridge a week earlier, but hey, I wasn't going to come all this way from Australia and not climb this Sigiriya Rock right?. Anyway we climbed to the very top in 50 minutes from the very start to the very top, including a few stops in between to catch our breath and trying to stop out hearts from exploding, but just quietly were were so very proud of ourselves, so kudos to us. Now just a word of warning, for those of you who have really bad knees or ankles this climb is not for you, and if you have even the slightest fear of heights, this climb is definitely not for you. I'd say go as far as the Lions feet and that's it as it is really frightening after that especially if people are trying to get past you and your clinging to the edge of the rock for grim death while letting them pass. For those of you who are not scared of heights then go on up to the very top, the 360 degree view you will love and you get to see all the ruins also. I must say watch out for the scammers too who will approach you along the way up at different points, they will make conversation with you and try and help you get up rocks etc and they will start to tell you all about the rock and the history and then they will become your guide without you even knowing about it and it will cost you money, but if you want to use them then go ahead, these locals are desperate for money. Also when you get there, there is no signage, well in English anyway, to where you buy the entry tickets from, we found it a bit confusing where to go to get the tickets, and that's a walk in itself too. There are plenty of Tuk Tuks at the bottom when you come back down that will take you back to your Hotels at a reasonable price. Just be aware that where the Tuk Tuks will drop you off at the start where you buy your tickets from is not the same place where you finish up when you come back down from the rock, you seem to get rail roaded out a different exit to an other side of the rock so you end up having to go past all the souvenir stalls. Our rooky mistake was telling our Tuk Tuk driver to pick us up here where he dropped us off, and he failed to tell us this was not the spot you finish up at when you come down from the rock, so once we came down and realised this was not where we started from, we then had to figure out how to get back around to the other side of the rock where the entrance was to meet our tuk tuk driver who was waiting so patiently for us, it was a long walk in the heat. All in all, we were both glad we did it and didn't drop dead from heat exposure and blown heart valves. We never want to see another step again as long as we live, our knees and ankles were aching for days afterwards. But done it now and so a big TICK off our bucket list.
----------------------------------
Amilia F 1.0 of 5 bubbles
No Country 13 contributions
Waste of time and money
DON'T WASTE YOUR TIME, EFFORT AND MONEYThis is a over hyped place most probably by locals to attract tourists and extort money. There is nothing special about this place, other than steep climbing of a huge rock using steel staircase.Ticket for foreigners USD30 per person, USD15 for people from SAARC countries, for locals it is Sri Lankan Rupee 50/-.The steel staircase is very narrow and most of the part climbing up and down use the same narrow path, so it slow down the movement of the people. With so many people climbing up and down on the narrow staircase, there is a chance of stampede like situation. NOT AT ALL SAFE.I had been to many UNESCO sites, this one is a BIG disappointment. CLIMB PIDURANGALLA ROCK INSTEAD. BETTER VIEWS, MORE FUN AND MUCH CHEAPER.
----------------------------------
Andrew S 5.0 of 5 bubbles
Sheffield, UK 786 contributions
Easily one of the highlights of our holiday.
Wow… this is just spectacular, walking through the grounds, past the fountains the rock of Sigiriya looms large in front.You then reach the foot of the rock at the entrance of the Lions Feet you start to climb the steps, and there are many, if you don’t have a good head for heights this may unnerve you a little, when you eventually get to the top the 360 degree views are awe inspiring. As you start to make your way down you climb up some spiral stairs to the see the paintings which are 2500 years old, the colours still vibrant today.
----------------------------------
Ines S 5.0 of 5 bubbles
London, UK 7 contributions
Excellent awe inspiring
Incredible place, really well maintained, the highlight of the trip so far. Stunning views. Go early because of how hot it gets during the day, we arrived at 7, we’re back downstairs around 9am.
----------------------------------
Suraj_sanje 5.0 of 5 bubbles
Kegalle, Sri Lanka 1 contribution
Good
one of the best world heritage in sri lanka. It have royal look. Top on the mountain have beauty view.
----------------------------------