数据工厂polybase-零行复制到下沉

Question

数据工厂管道使用来自源数据仓库的复制活动 ->分阶段blob存储 ->接收器数据仓库。

从源到斑点的副本按预期工作(复制行(。从分阶段到接收器的副本失败 - 复制0行

禁用polybase，并使用大量插入作品。

{
    "name": "PI_TEST",
    "properties": {
        "activities": [
            {
                "name": "MaterializedEventIdFilter_Copy",
                "type": "Copy",
                "dependsOn": [],
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [
                    {
                        "name": "Destination",
                        "value": "[formigration].[MaterializedEventIdFilter]"
                    }
                ],
                "typeProperties": {
                    "source": {
                        "type": "SqlDWSource",
                        "sqlReaderStoredProcedureName": "[formigration].[proc_GetStgMaterializedEventIdFilter]"
                    },
                    "sink": {
                        "type": "SqlDWSink",
                        "allowPolyBase": true,
                        "writeBatchSize": 100000,
                        "polyBaseSettings": {
                            "rejectValue": 0,
                            "rejectType": "value",
                            "useTypeDefault": false
                        }
                    },
                    "enableStaging": true,
                    "stagingSettings": {
                        "linkedServiceName": {
                            "referenceName": "riskstoreprd",
                            "type": "LinkedServiceReference"
                        },
                        "enableCompression": true
                    }
                },
                "inputs": [
                    {
                        "referenceName": "ioPrePrdMaterializedEventIdFilter",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "CloudPrdMaterializedEventIdFilter",
                        "type": "DatasetReference"
                    }
                ]
            },
            {
                "name": "MaterialisedEvent",
                "type": "SqlServerStoredProcedure",
                "dependsOn": [
                    {
                        "activity": "MaterializedEventIdFilter_Copy",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 2,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "storedProcedureName": "[formigration].[proc_SetStgMaterializedEventIdFilter]"
                },
                "linkedServiceName": {
                    "referenceName": "cloud_prd",
                    "type": "LinkedServiceReference"
                }
            }
        ],
        "annotations": []
    },
    "type": "Microsoft.DataFactory/factories/pipelines"
}

我期望Blob的数据将其输入水槽，但没有复制行。

编辑1：检查了数据仓库(接收器(，建立了一个连接，我可以在一秒钟内观察到外部表格等。

INSERT INTO [formigration].[MaterializedEventIdFilter] SELECT * FROM [ADFCopyGeneratedExternalTable_307e2c7f-a56f-4b75-86fb-10ab0cb94548]

Answer 1

在polybase中，外部表只是对blob存储文件夹/文件的引用，它们没有任何行。如果您想将数据实际复制到仓库中，请创建一个常规表并将其用作复制活动中的水槽！

希望这有帮助！