Azure数据流无法连接到私有云中的托管实例



我在Express Route对等子网下有托管实例,打开了公共端点,并提供了端口3342以允许公共端点访问,链接服务按预期工作,复制活动也很好,但是,当使用数据流接收器时,会引发以下错误

StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: at Sink 'Group': java.lang.RuntimeException: Cannot connect to SQL database: 'jdbc:sqlserver://:3342;database=', 'User: '.[SQL Exception]Error Code:0, Error Message: The TCP/IP connection to the host .database.windows.net, port 3342 has failed. Error: "Connection timed out: no further information.. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall."., error stack:shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:228)nshaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerException.ConvertConnectExceptionToSQLServerException(SQLServerException.java:285)nshaded.msdataflow.com.microsoft.sq

我使用的是内存优化的Azure托管IR,据我所知,Spark集群无法查看数据库服务器/数据库。正如我上面提到的,链接服务测试连接是绿色的。

还尝试了以下选项:在设计器中打开数据流,然后单击";测试连接";在Sink中的数据流内部,并得到相同的错误。

Cannot connect to SQL database: 'jdbc:sqlserver://.database.windows.net:3342;database=', 'User: '.[SQL Exception]Error Code:0, Error Message: The TCP/IP connection to the host .database.windows.net, port 3342 has failed. Error: "Connection timed out: no further information.. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall."., error stack:shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:228)
shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerException.ConvertConnectExceptionToSQLServerException(SQLServerException.java:285)
shaded.msdataflow.com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2478)
shaded.msdataflow.com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:641)
shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:2245)
shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:1921)
shaded.msdataflow.com.microso`enter code here`ft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:1762)
shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1077)
shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:623)
com.microsoft.dataflow.store.mssql.CustomSqlServerDriver$$anonfun$3.apply(CustomSqlServerDriver.scala:48)
com.microsoft.dataflow.store.mssql.CustomSqlServerDriver$$anonfun$3.apply(CustomSqlServerDriver.scala:48)
scala.util.Try$.apply(Try.scala:192)
com.microsoft.dataflow.store.mssql.CustomSqlServerDriver.connect(CustomSqlServerDriver.scala:48)
org.apache.spark.sql.execution.datasources.jdbc.DriverWrapper.connect(DriverWrapper.scala:45)
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:64)
org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:55)
com.microsoft.dataflow.transformers.store.JDBCCharacteristic$class.newConnection(JDBCStore.scala:38)
com.microsoft.dataflow.store.mssql.MSSQLCharacteristic.newConnection(MSSQLStore.scala:78)
com.microsoft.dataflow.store.mssql.MSSQLStore$$anonfun$connect$1.apply$mcV$sp(MSSQLStore.scala:453)
com.microsoft.dataflow.store.mssql.MSSQLStore$$anonfun$connect$1.apply(MSSQLStore.scala:452)
com.microsoft.dataflow.store.mssql.MSSQLStore$$anonfun$connect$1.apply(MSSQLStore.scala:452)
scala.util.Try$.apply(Try.scala:192)
com.microsoft.dataflow.store.mssql.MSSQLStore.connect(MSSQLStore.scala:452)```code```
com.microsoft.dataflow.transformers.StoreDelegate.connect(StoreDefinition.scala:83)
com.microsoft.dataflow.DataflowJobFuture$$anonfun$connect$2.apply(DataflowJobFuture.scala:973)
com.microsoft.dataflow.DataflowJobFuture$$anonfun$connect$2.apply(DataflowJobFuture.scala:970)
scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)

请提供您的专家建议以克服这一挑战。

提前感谢

这个问题已经在Mirosoft问答中讨论过了;一个论坛和OP解决了这个问题,并分享了以下解决细节。以下是参考链接,以防有人遇到类似问题:https://learn.microsoft.com/answers/questions/343550/index.html

问题:如何打开并连接到SQL MI的公共端点,以及连接到用于复制/数据流活动的数据工厂。

原因:链接服务网络连接使用"数据工厂IP的/服务标签";而Data Flow在后端使用spark集群,并且它们具有不属于Data Factory Service Tag的动态IP。

建议的解决方案:由于此限制,在数据流中有两个使用SQL MI的选项:

  • 打开端口3342到MI子网NSG中的源Internet和目标虚拟网络
  • 将数据流中的数据暂存到Azure Blob/表存储复制活动到另一个源(SQL或Blob(,然后继续数据流

最新更新