如何在另一个 avro 架构中引用一个 avro 架构

Question

我需要从另一个架构中引用这个学生模式：

{
"type": "record",
"namespace": "data.add",
"name": "Student",
"fields": [
{
"name": "Name",
"type": "string"
},
{
"name": "Age",
"type": "int"
}
]
}

这是需要引用学生的父地址架构：

{
"type": "record",
"namespace": "data.add",
"name": "Address",
"fields": [
{
"name": "student",
"type": "Student"
}
]
}

当我使用 Gradle 和 Avro 插件进行构建时，上述内容会引发错误。两个架构位于同一文件夹中。

Answer 1

这是成功的：

{
"type" : "record",
"namespace" : "data.add",
"name" : "Address",
"fields" : [
{
"name": "student",
"type": "data.add.Student"
}
]
}

Answer 2

抱歉，如果我来不及参加聚会，但在我看来，maven avro 插件和 avro-tools 编译器在加载时都不会确定依赖顺序，但如果您在命令行上自己排序它们会成功。我有一个示例从标准 maven 目录结构中的示例文件中演示了这一点。

当我将没有依赖项的架构放在命令行中的第一个时，它成功了：

java -jar /path/to/avro-tools-1.11.0.jar 
compile schema  
src/main/avro/student.avsc 
src/main/avro/address.avsc 
target/generated-sources/avro
ls target/generated-sources/avro/data/add/*
target/generated-sources/avro/data/add/Address.java target/generated-sources/avro/data/add/Student.java

当我将带有依赖项的架构放在第一个命令行中时，它失败了：

java -jar /path/to/avro-tools-1.11.0.jar 
compile schema  
src/main/avro/address.avsc 
src/main/avro/student.avsc 
target/generated-sources/avro
Exception in thread "main" org.apache.avro.SchemaParseException: "data.add.Student" is not a defined name. The type of the "student" field must be a defined name or a {"type": ...} expression.
at org.apache.avro.Schema.parse(Schema.java:1676)
at org.apache.avro.Schema$Parser.parse(Schema.java:1433)
at org.apache.avro.Schema$Parser.parse(Schema.java:1396)
at org.apache.avro.tool.SpecificCompilerTool.run(SpecificCompilerTool.java:154)
at org.apache.avro.tool.Main.run(Main.java:67)
at org.apache.avro.tool.Main.main(Main.java:56)

Answer 3

对于Maven，可以使用导入配置。

在此示例中，在您的 pom 中包含以下内容.xml在project/build/plugins

<plugin>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>${avro.version}</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>schema</goal>
</goals>
<configuration>
<imports>
<import>${project.basedir}/path/to/student.avsc</import>
</imports>
<sourceDirectory>${project.basedir}/path/to/schema/files</sourceDirectory>
<includes>
<include>*.avsc</include>
</includes>
<outputDirectory>${project.basedir}/path/to/target/</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>

那么你应该能够做一个 Maven 构建...

查看此链接了解更多详情

https://feitam.es/use-of-avro-maven-plugin-with-complex-schemas-defined-in-several-files-to-be-reused-in-different-typed-messages/

Answer 4

我们有一个场景，我们需要在篮子中使用篮子项目数组

所以我们定义了篮子项目的模式，如下所示

{
"type": "record",
"name": "BasketItem",
"namespace": "com.demo",
"fields": [{
"name": "id",
"type": "string"
},
{
"name": "shortDesc",
"type": [
"null",
"string"
],
"default": null
},
{
"name": "longDesc",
"type": [
"null",
"string"
],
"default": null
},
{
"name": "requestedQuantity",
"type": [
"null",
"string"
],
"default": null
}
]
}

然后，此类型BasketItem在Basket架构中用于引用命名空间。

{
"name": "Basket",
"type": "record",
"namespace": "com.demo",
"fields": [{
"name": "id",
"type": "string"
},
{
"name": "creationTimeStamp",
"type": "string"
},
{
"name": "basketItems",
"type": {
"type": "array",
"items": {
"type": "com.demo.BasketItems",
"name": "basketItem"
}
}
}
]
}

这是定义可重用的 Avro 架构并根据需要引用它们的简单解决方案，以避免模型生成中的冲突。