猪 - 加入不起作用

Question

我有一个问题，在猪加入。我先给大家讲一下背景。下面是我的代码:

-- START file loading
start_file = LOAD 'dir/start_file.csv' USING PigStorage(';') as (PARTRANGE:chararray,     COD_IPUSER:chararray);
-- trim
A = FOREACH start_file GENERATE TRIM(PARTRANGE) AS PARTRANGE, TRIM(COD_IPUSER) AS COD_IPUSER;
dump A;

输出:

(79.92.147.88,20140310)
(79.92.147.88,20140310)
(109.31.67.3,20140310)
(109.31.67.3,20140310)
(109.7.229.143,20140310)
(109.8.114.133,20140310)
(77.198.79.99,20140310)
(77.200.174.171,20140310)
(77.200.174.171,20140310)
(109.17.117.212,20140310)

加载其他文件:

-- Chargement du fichier recherche Hadopi
file2 = LOAD 'dir/file2.csv' USING PigStorage(';') as (IP_RECHERCHEE:chararray, DATE_HADO:chararray);
dump file2;

输出如下:

(2014/03/10 00:00:00,79.92.147.88)
(2014/03/10 00:00:01,79.92.147.88)
(2014/03/10 00:00:00,192.168.2.67)

现在，我想做一个左外连接。下面是代码:

result = JOIN file2 by IP_RECHERCHEE LEFT OUTER, A by COD_IPUSER;
dump result;

输出如下:

(2014/03/10 00:00:00,79.92.147.88,,)
(2014/03/10 00:00:00,192.168.2.67,,)
(2014/03/10 00:00:01,79.92.147.88,,)

"file2"的所有记录都在这里，这很好，但start_file的任何记录都在这里。这就像连接失败了。

你知道问题在哪里吗?

谢谢。

Answer 1

您在file2中错误标记了字段。您将第一个字段称为IP，将第二个字段称为日期，但是，正如您的dump所显示的那样，情况正好相反。尝试FOREACH file2 GENERATE IP_RECHERCHEE，您将看到您试图加入的字段

Answer 2

结果如预期。您正在调用左外连接，它查找file2中的IP_RECHERCHEE字段与a的COD_IPUSER的匹配。
由于没有匹配，它返回file2中的所有IP_RECHERCHEE字段，并将a中的字段替换为空。
显然2014/03/10 00:00:00 != 20140310

Answer 3

您的字段名称错误，并且您通过错误的字段加入。看来您想通过IP地址加入。

start_file = LOAD 'dir/start_file.csv' USING PigStorage(';') as (IP:chararray, PARTRANGE:chararray);
A = FOREACH start_file GENERATE TRIM(IP) AS IP, TRIM(PARTRANGE) AS PARTRANGE;
file2 = LOAD 'dir/file2.csv' USING PigStorage(';') as (DATE_HADO:chararray, IP:chararray);

我得到的是这个

(2014/03/10 00:00:00,192.168.2.67,,)
(2014/03/10 00:00:00,79.92.147.88,79.92.147.88,20140310)
(2014/03/10 00:00:00,79.92.147.88,79.92.147.88,20140310)
(2014/03/10 00:00:01,79.92.147.88,79.92.147.88,20140310)
(2014/03/10 00:00:01,79.92.147.88,79.92.147.88,20140310)