Perl REST::Client - 响应中的垃圾数据

Question

我在从 Perl REST::Client 针对 Red Hat Satellite REST API 获得有效响应时遇到问题。我收到以下编码响应：

$VAR1 = '���j�0��~
�g9��   ���#�9�`dIm�m�-���uJ
        �����f4U�@▒��
                     ���F��xګ X�;�'r��/���3R�s�C�u�*�2_N��٧�������f\�������WA0����نp��T͖�l�▒Pȣ}�x��8�&�d�n��ߦ`��.���Tƙ�V�c�&����a���%�ZH·�aJ�0�yT��q� �Jz��ճMO�\�����'

当我通过Encode::decode_utf8解码它时，它似乎同样是乱码：

$VAR1 = "x{fffdx{fffd}x{fffd}x{fffd}jx{fffd}0x{fffd}x{fffd}~
x{fffd}g9x{fffd}x{fffd}      x{fffd}x{fffd}x{fffd}#x{fffd}9x{fffd}`dImx{fffd}mx{fffd}-x{fffd}x{fffd}x{fffd}uJ
        x{fffd}x{fffd}x{fffd}x{fffd}x{fffd}f4Ux{fffd}@▒x{fffd}x{1a0220}
                                                                                x{fffd}x{fffd}x{fffd}Fx{fffd}x{fffd}xx{6ab} Xx{fffd};x{fffd}'rx{fffd}x{fffd}/x{fffd}x{fffd}x{fffd}3Rx{fffd}sx{fffd}Cx{fffd}ux{fffd}*x{fffd}2_Nx{fffd}x{fffd}x{667}x{fffd}x{fffd}x{fffd}x{fffd}x{fffd}x{fffd}x{fffd}f\x{fffd}x{fffd}x{fffd}x{fffd}x{fffd}x{fffd}x{fffd}WA0x{fffd}x{fffd}x{fffd}x{fffd}x{646}px{fffd}x{fffd}Tx{356}x{fffd}lx{fffd}▒Px{223}}x{fffd}xx{fffd}x{fffd}8x{fffd}&x{fffd}dx{fffd}nx{fffd}x{fffd}x{7e6}`x{fffd}x{fffd}.x{fffd}x{fffd}x{fffd}Tx{199}x{fffd}Vx{fffd}cx{fffd}&x{fffd}x{fffd}x{fffd}x{fffd}ax{fffd}x{fffd}x{fffd}%x{fffd}ZHx{b7}x{fffd}aJx{fffd}0x{fffd}yTx{fffd}x{fffd}qx{fffd} x{fffd}Jzx{fffd}x{fffd}x{573}MOx{fffd}\x{fffd}x{fffd}x{fffd}x{fffd}x{fffd}";

我用来测试的脚本：

#!/usr/bin/perl
use strict;
use warnings;
use 5.010;
use REST::Client;
use Data::Dumper;
require JSON;
require MIME::Base64;
require HTTP::Cookies;
my $host        = 'https://sat.example.com/api/v2/domains';
my $client      = REST::Client->new({useragent => LWP::UserAgent->new(cookie_jar => HTTP::Cookies->new)});
my $headers     = {
        'Authorization'         => 'Basic XXXXXXXXX',
        'Accept-Encoding'       => scalar HTTP::Message::decodable,
        #'Content-Type'         => 'application/json',
        'Content-Type'          => 'application/json;charset=utf8',
        'Connection'            => 'keep-alive',
        'Accept'                => 'application/json',
        'Host'                  => URI->new($host)->canonical->host_port,
        #'Charset'              => 'UTF-8'
};
$client->addHeader($_, $headers->{$_}) for keys %{$headers};
$client->setCa('/var/www/html/pub/katello-server-ca.crt');
print Dumper JSON::decode_json($client->GET($host)->responseContent);
#print Dumper $client->GET($host);
__END__

当我通过Data::Dumper查看REST::Client对象时，所有标题似乎都设置正确且可读。唯一不可读的是实际内容。

我能够使用来自同一服务器的 curl 命令查询 API：

$ curl -# -X GET -v -H 'Accept: application/json' -H 'Authorization: Basic XXXXXXXXX' --cacert katello-server-ca.crt https://sat.example.com/api/v2/domains -o /dev/null 
* About to connect() to sat.example.com port 443 (#0)
*   Trying 192.168.0.1...
* Connected to sat.example.com (192.168.0.1) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: katello-server-ca.crt
  CApath: none
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=sat.example.com,OU=SomeOrgUnit,O=Katello,ST=North Carolina,C=US
*       start date: Oct 11 20:24:12 2018 GMT
*       expire date: Jan 17 20:24:12 2038 GMT
*       common name: sat.example.com
*       issuer: CN=sat.example.com,OU=SomeOrgUnit,O=Katello,L=Raleigh,ST=North Carolina,C=US
> GET /api/v2/domains HTTP/1.1
> User-Agent: curl/7.29.0
> Host: sat.example.com
> Accept: application/json
> Authorization: Basic XXXXXXXXX
> 
< HTTP/1.1 200 OK
< Date: Wed, 24 Apr 2019 18:50:34 GMT
< Server: Apache/2.4.6 (Red Hat Enterprise Linux)
< Foreman_version: 1.15.6.48
< Foreman_api_version: 2
< Apipie-Checksum: 2a54cbc5a3f59fad6e7e697ec609cda8
< Cache-Control: max-age=0, private, must-revalidate
< X-Request-Id: 57672b22-6a4f-4b3a-83fb-6b1a54747d67
< X-Runtime: 0.036638
< Content-Security-Policy: default-src 'self'; child-src 'self'; connect-src 'self' ws: wss:; img-src 'self' data: *.gravatar.com; script-src 'unsafe-eval' 'unsafe-inline' 'self'; style-src 'unsafe-inline' 'self'
< Strict-Transport-Security: max-age=631152000; includeSubdomains
< X-Content-Type-Options: nosniff
< X-Download-Options: noopen
< X-Frame-Options: sameorigin
< X-Permitted-Cross-Domain-Policies: none
< X-XSS-Protection: 1; mode=block
< X-Powered-By: Phusion Passenger 4.0.18
< Set-Cookie: _session_id=d35c9df0bce80baeca85b0ad38298ee8; path=/; secure; HttpOnly
< ETag: W/"0eeb7a2c131e780c72202cf712c972e3"
< Status: 200 OK
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: application/json; charset=utf-8
< 
{ [data not shown]
######################################################################## 100.0%* Connection #0 to host sat.example.com left intact

我已经使用REST::Client来对付可能不同的 API，没有问题，这是第一个真正让我感到悲伤的 API。我不确定解码响应后从哪里开始故障排除。任何建议都会有所帮助。

print Dumper $client->GET($host)->{_res}->as_string;输出

$VAR1 = 'HTTP/1.1 200 OK
Cache-Control: max-age=0, private, must-revalidate
Connection: close
Date: Thu, 25 Apr 2019 19:45:48 GMT
ETag: W/"0eeb7a2c131e780c72202cf712c972e3-gzip"
Server: Apache/2.4.6 (Red Hat Enterprise Linux)
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 242
Content-Type: application/json; charset=utf-8
Apipie-Checksum: 2a54cbc5a3f59fad6e7e697ec609cda8
Client-Date: Thu, 25 Apr 2019 19:45:53 GMT
Client-Peer: 192.168.0.1:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /C=US/ST=North Carolina/L=Raleigh/O=Katello/OU=SomeOrgUnit/CN=sat.example.com
Client-SSL-Cert-Subject: /C=US/ST=North Carolina/O=Katello/OU=SomeOrgUnit/CN=sat.example.com
Client-SSL-Cipher: ECDHE-RSA-AES128-GCM-SHA256
Client-SSL-Socket-Class: IO::Socket::SSL
Content-Security-Policy: default-src 'self'; child-src 'self'; connect-src 'self' ws: wss:; img-src 'self' data: *.gravatar.com; script-src 'unsafe-eval' 'unsafe-inline' 'self'; style-src 'unsafe-inline' 'self'
Foreman_api_version: 2
Foreman_version: 1.15.6.48
Set-Cookie: _session_id=ea51c42b7a65478d27b21c2fb02e4c4a; path=/; secure; HttpOnly
Status: 200 OK
Strict-Transport-Security: max-age=631152000; includeSubdomains
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: sameorigin
X-Permitted-Cross-Domain-Policies: none
X-Powered-By: Phusion Passenger 4.0.18
X-Request-Id: 528535f3-c8cf-4d8b-91db-1572e94a974a
X-Runtime: 4.600748
X-XSS-Protection: 1; mode=block
???j?0??~
?g9??   ???#?9?`dIm?m?-???uJ
    ???f4U?@?????
                     ???F??xګ X?;?'r??/??3R?s?C?u?*?2_N??٧??????f\???????WA0????نp??T͖?l?Pȣ}?x??8?&?d?n??ߦ`??.???Tƙ?V?c?&????a???%?ZH·?aJ?0?yT?q? ?Jz??ճMO?\?????
';

Answer 1

根据附在您的问题中的注释，我们可以说问题是服务器使用 gzip 压缩数据进行响应，而 REST：：Client 包只是返回原始数据而不先对其进行解码。您可以使用 REST：：Client 对象上的 $client->{_res} 直接访问底层 HTTP：：Response 对象，并且可以使用 ->decoded_content 方法获取正确解压缩和字符集转换的数据。请注意，您需要对解码的内容使用JSON::from_json()而不是JSON::decode_json()，因为它已经从 UTF-8 字节序列转换为 Unicode 字符串作为解码的一部分。

简而言之，你想要的是以下内容。

print Dumper JSON::from_json($client->GET($host)->{_res}->decoded_content);

直接访问底层对象并不漂亮，但除非你想修补 REST：：Client，否则这是获得所需结果的最简单方法。