由于几天来我无法使用Powershell脚本使用Youtube中的任何信息，因此首先必须接受cookie才能查看视频。有什么想法可以用Powershell接受Youtube Cookies吗？已经有一个问题，但它不涉及Powershell接受Youtube 的cookie同意
$HTML=调用WebRequest-Urihttps://www.youtube.com/user/YouTube/videos
经过一番测试，我终于可以工作了。
但请记住，如果谷歌改变流程或回复的形式数据，目前的流程将来可能会失败。
背景
在我们第一次呼叫youtube后，我们被重定向到consent.google.com，并收到以下表单内容：
<form action="https://consent.youtube.com/s" method="POST" style="display:inline;">
<input type="hidden" name="gl" value="DE">
<input type="hidden" name="m" value="0">
<input type="hidden" name="pc" value="yt">
<input type="hidden" name="continue" value="https://www.youtube.com/user/YouTube/videos">
<input type="hidden" name="ca" value="r">
<input type="hidden" name="x" value="8">
<input type="hidden" name="v" value="cb.20210329-17-p2.de+FX+873">
<input type="hidden" name="t" value="ADw3F8i44JCpypLjx8SOx3tbsrxxS7ug:1617806186191">
<input type="hidden" name="hl" value="de">
<input type="hidden" name="src" value="1">
<input type="hidden" name="uxe" value="2398321372">
<input type="submit" value="Ich stimme zu" class="button" aria-label="In die Verwendung von Cookies und anderen Daten zu den beschriebenen Zwecken einwilligen" />
</form>
我假设只有两个属性continue和v是重要的，但我只是像浏览器一样在POST过程中使用所有属性v将是我们最终CONSENTcookie值的正确部分。它将有一个YES+前缀
例如，v中的cb.20210329-17-p2.de+FX+873在cookieCONSENT中变为YES+cb.20210329-17-p2.de+FX+873
不幸的是，我们对Invoke-WebRequest的调用没有为我们提供任何预定义的表单属性。属性(Invoke-WebRequest abc).Form仅为NULL。
因此，我们必须解析响应内容中的特定表单数据，构建key=value主体，并将主体POST到action属性中提到的URL。
请在代码中找到流程的其余部分作为注释。
代码
这是没有详细输出的干净代码。在下面找到具有详细输出的相同代码。
$youtubeUrl = 'https://www.youtube.com/user/YouTube/videos'
$consentDomain = 'consent.youtube.com'
$webUserAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.4976.0 Safari/537.36 Edg/102.0.1227.0'
# at first we disable the annoying (at least for this process) and in PS5.1- performance affecting progress bar for web requests
$currentProgressPreference = $ProgressPreference
$ProgressPreference = [System.Management.Automation.ActionPreference]::SilentlyContinue
try {
# in our first GET call we should get a response from consent.youtube.com.
# we save the session including all cookies in variable $youtubeSession.
$response = Invoke-WebRequest -Uri $youtubeUrl -UseBasicParsing -SessionVariable 'youtubeSession' -UserAgent $webUserAgent -ErrorAction Stop
# using BaseResponse to figure out which host has responded
if ($PSVersionTable.PSVersion.Major -gt 5) {
# PS 6+ has other properties than PS5.1 and below
$responseRequestUri = $response.BaseResponse.RequestMessage.RequestUri
} else {
$responseRequestUri = $response.BaseResponse.ResponseUri
}
if ($responseRequestUri.Host -eq $consentDomain) {
# check if got redirected to "consent.youtube.com"
# unfortunately the response object from "Invoke-WebRequest" does not provide any "Form" data as property,
# so we have to parse it from the content. There are two <form..> nodes, but we only need the one for method "POST".
$formContent = [regex]::Match(
$response.Content,
# we use lazy match, even if it's expensive when it comes to performance.
('{0}.+?(?:{1}.+?{2}|{2}.+?{1}).+?{3}' -f
[regex]::Escape('<form'),
[regex]::Escape("action=`"https://$consentDomain"),
[regex]::Escape('method="POST"'),
[regex]::Escape('</form>')
)
)
# getting the POST URL using our parsed form data. As of now it should parse: "https://consent.youtube.com/s"
$postUrl = [regex]::Match($formContent, '(?<=action=")[^"]+(?=")').Value
# build POST body as hashtable using our parsed form data.
# only elements with a "name" attribute are relevant and we only need the plain names and values
$postBody = @{}
[regex]::Matches($formContent -replace 'r?n', '<input[^>]+>').Value | ForEach-Object {
$name = [regex]::Match($_, '(?<=name=")[^"]+(?=")').Value
$value = [regex]::Match($_, '(?<=value=")[^"]+(?=")').Value
if (![string]::IsNullOrWhiteSpace($name)) {
$postBody[[string]$name] = [string]$value
}
}
# now let's try to get an accepted CONSENT cookie by POSTing our hashtable to the parsed URL and override the sessionVariable again.
# Using the previous session variable here would return a HTTP error 400 ("method not allowed")
$response = Invoke-WebRequest -Uri $postUrl -Method Post -UseBasicParsing -SessionVariable 'youtubeSession' -UserAgent $webUserAgent -Body $postBody -ErrorAction Stop
# get all the cookies for domain '.youtube.com'
$youtubeCookies = [object[]]$youtubeSession.Cookies.GetCookies('https://youtube.com')
# check if we got the relevant cookie "CONSENT" with a "yes+" prefix in its value
# if the value changes in future, we have to adapt the condition here accordingly
$consentCookie = [object[]]($youtubeCookies | Where-Object { $_.Name -eq 'CONSENT' })
if (!$consentCookie.Count) {
Write-Error -Message 'The cookie "CONSENT" is missing in our session after our POST! Please check.' -ErrorAction Stop
} elseif (!($consentCookie.Value -like 'YES+*').count) {
Write-Error -Message ("The value of cookie ""CONSENT"" (""$($consentCookie.Value -join '" OR "')"") does not start with ""YES+"", but maybe it's intended and the condition has to be adapted!") -ErrorAction Stop
}
}
} finally {
# set the progress preference back to the previous value
$ProgressPreference = $currentProgressPreference
}
# From here on use the argument '-WebSession $youtubeSession' with each 'Invoke-WebRequest'
# e.g.: Invoke-WebRequest $youtubeUrl -WebSession $youtubeSession -UseBasicParsing
与上面的代码相同，但有详细的输出语句
进程与上面的相同，但它包括详细输出。它只包括详细的输出语句，这样，如果发生更改，您或任何其他人都可以更轻松地调试它。
$youtubeUrl = 'https://www.youtube.com/user/YouTube/videos'
$consentDomain = 'consent.youtube.com'
$webUserAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.4976.0 Safari/537.36 Edg/102.0.1227.0'
# remove this verbose preference definition or set it to "SilentlyContinue" to suppress verbose output
$currentVerbosePreference = $VerbosePreference
$VerbosePreference = [System.Management.Automation.ActionPreference]::Continue
# at first we disable the annoying (at least for this process) and in PS5.1- performance affecting progress bar for web requests
$currentProgressPreference = $ProgressPreference
$ProgressPreference = [System.Management.Automation.ActionPreference]::SilentlyContinue
#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#
#region THIS REGION CAN BE REMOVED.
#------------------------------------------------------------------------------------------------------------------------
Write-Verbose "`r`n>> Let's start with a GET to:`r`n`t$youtubeUrl"
#------------------------------------------------------------------------------------------------------------------------
#endregion
#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#
try {
# in our first GET call we should get a response from consent.youtube.com.
# we save the session including all cookies in variable $youtubeSession.
$response = Invoke-WebRequest -Uri $youtubeUrl -UseBasicParsing -SessionVariable 'youtubeSession' -UserAgent $webUserAgent -ErrorAction Stop
# using BaseResponse to figure out which host has responded
if ($PSVersionTable.PSVersion.Major -gt 5) {
# PS 6+ has other properties than PS5.1 and below
$responseRequestUri = $response.BaseResponse.RequestMessage.RequestUri
} else {
$responseRequestUri = $response.BaseResponse.ResponseUri
}
#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#
#region THIS REGION CAN BE REMOVED.
#------------------------------------------------------------------------------------------------------------------------
Write-Verbose "`r`n>> We got a response from:`r`n`t$responseRequestUri"
Write-Verbose "`r`n>> Let''s check our cookies. We should see a cookie 'CONSENT' which is pending:"
Write-Verbose ($youtubeSession.Cookies.GetCookies('https://youtube.com') | Format-Table Domain, Name, Value | Out-String)
#------------------------------------------------------------------------------------------------------------------------
#endregion
#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#
# check if got redirected to consent domain
if ($responseRequestUri.Host -eq $consentDomain) {
#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#
#region THIS REGION CAN BE REMOVED.
#------------------------------------------------------------------------------------------------------------------------
Write-Verbose "`r`n>> Let's parse the required form data and post it to the correct URL"
#------------------------------------------------------------------------------------------------------------------------
#endregion
#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#
# unfortunately the response object from "Invoke-WebRequest" does not provide any "Form" data as property,
# so we have to parse it from the content. There are two <form..> nodes, but we only need the one for method "POST".
$formContent = [regex]::Match(
$response.Content,
# we use lazy match, even if it's expensive when it comes to performance.
('{0}.+?(?:{1}.+?{2}|{2}.+?{1}).+?{3}' -f
[regex]::Escape('<form'),
[regex]::Escape("action=`"https://$consentDomain"),
[regex]::Escape('method="POST"'),
[regex]::Escape('</form>')
)
)
# getting the POST URL using our parsed form data. As of now it should parse: "https://consent.youtube.com/s"
$postUrl = [regex]::Match($formContent, '(?<=action=")[^"]+(?=")').Value
# build POST body as hashtable using our parsed form data.
# only elements with a "name" attribute are relevant and we only need the plain names and values
$postBody = @{}
[regex]::Matches($formContent -replace 'r?n', '<input[^>]+>').Value | ForEach-Object {
$name = [regex]::Match($_, '(?<=name=")[^"]+(?=")').Value
$value = [regex]::Match($_, '(?<=value=")[^"]+(?=")').Value
if (![string]::IsNullOrWhiteSpace($name)) {
$postBody[[string]$name] = [string]$value
}
}
#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#
#region THIS REGION CAN BE REMOVED.
#------------------------------------------------------------------------------------------------------------------------
Write-Verbose "`r`n>> Now let's post the following body to:`r`n`t$postUrl"
Write-Verbose ($postBody | Out-String)
#------------------------------------------------------------------------------------------------------------------------
#endregion
#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#
# now let's try to get an accepted CONSENT cookie by POSTing our hashtable to the parsed URL and override the sessionVariable again.
# Using the previous session variable here would return a HTTP error 400 ("method not allowed")
$response = Invoke-WebRequest -Uri $postUrl -Method Post -UseBasicParsing -SessionVariable 'youtubeSession' -UserAgent $webUserAgent -Body $postBody -ErrorAction Stop
# get all the cookies for domain '.youtube.com'
$youtubeCookies = [object[]]$youtubeSession.Cookies.GetCookies('https://youtube.com')
# check if we got the relevant cookie "CONSENT" with a "yes+" prefix in its value
# if the value changes in future, we have to adapt the condition here accordingly
$consentCookie = [object[]]($youtubeCookies | Where-Object { $_.Name -eq 'CONSENT' })
if (!$consentCookie.Count) {
Write-Error -Message 'The cookie "CONSENT" is missing in our session after our POST! Please check.' -ErrorAction Stop
} elseif (!($consentCookie.Value -like 'YES+*').count) {
Write-Error -Message ("The value of cookie ""CONSENT"" (""$($consentCookie.Value -join '" OR "')"") does not start with ""YES+"", but maybe it's intended and the condition has to be adapted!") -ErrorAction Stop
}
#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#
#region THIS REGION CAN BE REMOVED. Even the $responseRequestUri part. Just for Verbose output
#------------------------------------------------------------------------------------------------------------------------
# using BaseResponse to figure out which host has responded
if ($PSVersionTable.PSVersion.Major -gt 5) {
# PS 6+ has other properties than PS5.1 and below
$responseRequestUri = $response.BaseResponse.RequestMessage.RequestUri
} else {
$responseRequestUri = $response.BaseResponse.ResponseUri
}
Write-Verbose "`r`n>> This time we got a response from:`r`n`t$responseRequestUri"
#------------------------------------------------------------------------------------------------------------------------
#endregion
#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#
}
#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#↓#
#region THIS REGION CAN BE REMOVED. JUST A TEST. Always use: -WebSession $youtubeSession
#------------------------------------------------------------------------------------------------------------------------
Write-Verbose "`r`n>> Let's check our cookies again:"
Write-Verbose ($youtubeSession.Cookies.GetCookies('https://youtube.com') | Format-Table Domain, Name, Value | Out-String)
Write-Verbose "`r`n>> Let''s check a video from github using our new session variable.`r`n`tVideo: https://www.youtube.com/watch?v=w3jLJU7DT5"
$test = Invoke-WebRequest 'https://www.youtube.com/watch?v=w3jLJU7DT5E' -UseBasicParsing -WebSession $youtubeSession
Write-Verbose "`r`n>> And again, let''s check our cookies:"
Write-Verbose ($youtubeSession.Cookies.GetCookies('https://youtube.com') | Format-Table Domain, Name, Value | Out-String)
Write-Verbose "`r`n>> And our content. But please press [Enter] first."
if ($VerbosePreference -eq [System.Management.Automation.ActionPreference]::Continue) {
Pause
$test.Content
}
#------------------------------------------------------------------------------------------------------------------------
#endregion
#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#↑#
} finally {
# set the progress preference back to the previous value
$ProgressPreference = $currentProgressPreference
# set the verbose preference back to the previous value, in case it was used in ths script
# can be removed if not used
if ($currentVerbosePreference) {
$VerbosePreference = $currentVerbosePreference
}
}
# From here on use the argument '-WebSession $youtubeSession' with each 'Invoke-WebRequest'
# e.g.: Invoke-WebRequest $youtubeUrl -WebSession $youtubeSession -UseBasicParsing
最后
在获得正确的CONSENTcookie后，您可以使用Invoke-WebRequest使用参数-WebSession $youtubeSession
获取任何站点，例如
Invoke-WebRequest 'https://www.youtube.com/watch?v=w3jLJU7DT5E' -WebSession $youtubeSession -UseBasicParsing