用requests.get去抓Dcard API的範例指令貌似失效
2022/07/13 下午 11:10
使用 DCard API 練習如何存取第三方網路資料
蔣淳斌
觀看數：29
回答數：0
收藏數：0
爬蟲
requests
dcard
Web Crawler
各位好，我照著爬蟲課程Day4的範例去抓取Dcard寵物版的資料，但是一直沒辦法成功。
不論是輸入
```
import requests
r = requests.get('https://www.dcard.tw/_api/forums/pet/posts?popular=true')
response = r.text
response
```
還是參照另外的答案輸入
```
import requests
headers = {
'User-Agent': 'Mozilla/5.0 (Linux; Android 10; SM-A205U) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.71 Mobile Safari/537.36.',
}
r = requests.get('https://www.dcard.tw/_api/forums/pet/posts?popular=true', headers=headers)
response = r.text
response
```
或是輸入
```
import requests
headers = {
'User-Agent': '',
}
r = requests.get('https://www.dcard.tw/_api/forums/pet/posts?popular=true', headers=headers)
response = r.text
response
```
都會得到以下的html訊息
```
'\n\n\n\n \n
\n\nAttention Required! | Cloudflare\n\n\n\n\n\n
(略)
```
無法透過python requests.get()獲得資料的同時，我卻同時可以直接連上
[https://www.dcard.tw/_api/forums/pet/posts?popular=true](https://www.dcard.tw/_api/forums/pet/posts?popular=true)
通過「我不是機器人」認證後看見json格式的資料。
```
[{"id":239422311,"title":"起不來鴨🐱🦆","excerpt":"","anonymousSchool":false,"anonymousDepartment":false,"pinned":false,"forumId":"7ee21581-1307-4ba9-b9dc-82028bdcca49","replyId":null,"createdAt":"2022-07-12T04:38:23.053Z","updatedAt":"2022-07-12T04:50:00.792Z","commentCount":47,"likeCount":1215,"collectionCount":398,"withNickname":true,"tags":[],"topics":["有趣","貓","鴨子","可愛","搞笑"],"meta":{"layout":"video"},"forumName":"寵物","forumAlias":"pet","nsfw":false,"gender":"F","school":"呱呱叫主-呱呱🦆","department":"ruuubyyy","replyTitle":null,"reactions":[{"id":"286f599c-f86a-4932-82f0-f5a06f1eca03","count":1125},{"id":"e8e6bc5d-41b0-4129-b134-97507523d7ff","count":86},{"id":"4b018f48-e184-445f-adf1-fc8e04ba09b9","count":3},{"id":"aa0d425f-d530-4478-9a77-fe3aedc79eea","count":1}],"hidden":false,"customStyle":null,"isSuspiciousAccount":false,"isModerator":false,"layout":"video","spoilerAlert":false,"withImages":false,"withVideos":false,"media":[],"reportReasonText":"","supportedReactions":null,"isSelectedPost":false,"unsafe":false,"leaderboardCategoryId":"29554c5d-229e-4439-a757-74c31ca8e6e7","leaderboardCategoryName":"有趣","leaderboardRank":{"id":"e018c06f-8679-4f43-aaca-055e698f7c40","name":"全站","rank":27,"leaderboardAll"
(略)
```
請問我該怎麼改寫我的requests.get()好讓我可以順利取得Dcard網站上的json格式資料？
