Visual Studio Data Load from flat file to Postgres Database



我正在使用Visual Studio将数据从CSV文件传输到Postgres数据库。我的数据库安装在 Windows Server 2012 上,我正在使用本地计算机传输数据。我的进程成功运行而没有抛出任何错误,但不知何故它没有加载我的 CSV 文件的所有行。该文件包含 382,363 行,但在我检查数据库后只加载了 26000 行。

我直接从 Postgres 的导入向导加载了 CSV,它成功加载了所有 382,363 行,但是当我通过 Visual Studio 加载数据时,它只加载了 26000 行而不会引发任何错误。我只收到两个警告消息。有没有人遇到任何问题,如果是这样,我该如何解决?

粘贴到我的流程的整个输出下方

SSIS package "C:UsersShivam SARINsourcereposIntegration Services Project3Integration Services Project3Package.dtsx" starting.
Information: 0x4004300A at Data Flow Task, SSIS.Pipeline: Validation phase is beginning.
Information: 0x4004300A at Data Flow Task, SSIS.Pipeline: Validation phase is beginning.
Warning: 0x80049304 at Data Flow Task, SSIS.Pipeline: Warning: Could not open global shared memory to communicate with performance DLL; data flow performance counters are not available.  To resolve, run this package as an administrator, or on the system's console.
Warning: 0x80047076 at Data Flow Task, SSIS.Pipeline: The output column "T_CTRY_DESTINATION" (115) on output "Flat File Source Output" (6) and component "Flat File Source" (2) is not subsequently used in the Data Flow task. Removing this unused output column can increase Data Flow task performance.
Information: 0x40043006 at Data Flow Task, SSIS.Pipeline: Prepare for Execute phase is beginning.
Information: 0x40043007 at Data Flow Task, SSIS.Pipeline: Pre-Execute phase is beginning.
Information: 0x402090DC at Data Flow Task, Flat File Source [2]: The processing of file "C:UsersShivam SARINDocumentsExcel-csvMS 2018 Q3.csv" has started.
Information: 0x4004300C at Data Flow Task, SSIS.Pipeline: Execute phase is beginning.
Information: 0x402090DE at Data Flow Task, Flat File Source [2]: The total number of data rows processed for file "C:UsersShivam SARINDocumentsExcel-csvMS 2018 Q3.csv" is 382364.
Information: 0x402090DF at Data Flow Task, OLE DB Destination [275]: The final commit for the data insertion in "OLE DB Destination" has started.
Information: 0x402090E0 at Data Flow Task, OLE DB Destination [275]: The final commit for the data insertion in "OLE DB Destination" has ended.
Information: 0x40043008 at Data Flow Task, SSIS.Pipeline: Post Execute phase is beginning.
Information: 0x402090DD at Data Flow Task, Flat File Source [2]: The processing of file "C:UsersShivam SARINDocumentsExcel-csvMS 2018 Q3.csv" has ended.
Information: 0x4004300B at Data Flow Task, SSIS.Pipeline: "OLE DB Destination" wrote 382363 rows.
Information: 0x40043009 at Data Flow Task, SSIS.Pipeline: Cleanup phase is beginning.
SSIS package "C:UsersShivam SARINsourcereposIntegration Services Project3Integration Services Project3Package.dtsx" finished: Success.
The program '[21956] DtsDebugHost.exe: DTS' has exited with code 0 (0x0).

根据我对 postgres 的经验,我发现使用 oledb 目标插入行非常慢,而且您上面提到的奇怪事情似乎发生了。 它很慢,因为提供程序不支持批量插入操作。

我的建议是用Execute Process Task调用 psql。 这将更快,您将使用Postgres原生的工具。

更多关于这里的信息: https://www.postgresql.org/docs/9.6/app-psql.html

最新更新