尝试使用AVRO4定义序列化器，但会遇到丢失的隐式错误

Question

我正在使用flink(1.7(kafka客户端和avro4s(2.0.4(，我想序列化到字节数组：

class AvroSerializationSchema[IN : SchemaFor : FromRecord: ToRecord] extends SerializationSchema[IN] {
  override def serialize(element: IN): Array[Byte] = {
    val str = AvroSchema[IN]
    val schema: Schema = new Parser().parse(str.toString)
    val out = new ByteArrayOutputStream()
    val os = AvroOutputStream.data[IN].to(out).build(schema)
    os.write(element)
    out.close()
    out.flush()
    os.flush()
    os.close()
    out.toByteArray
  }
}

但是，我一直在得到这个例外：

Error:(15, 35) could not find implicit value for evidence parameter of type com.sksamuel.avro4s.Encoder[IN]
    val os = AvroOutputStream.data[IN].to(out).build(schema)

和

Error:(15, 35) not enough arguments for method data: (implicit evidence$3: com.sksamuel.avro4s.Encoder[IN])com.sksamuel.avro4s.AvroOutputStreamBuilder[IN].
Unspecified value parameter evidence$3.
    val os = AvroOutputStream.data[IN].to(out).build(schema)

Answer 1

根据代码IN必须为 Encoder类型：

object AvroOutputStream {
  /**
    * An [[AvroOutputStream]] that does not write the schema. Use this when
    * you want the smallest messages possible at the cost of not having the schema available
    * in the messages for downstream clients.
    */   def binary[T: Encoder] = new AvroOutputStreamBuilder[T](BinaryFormat)
  def json[T: Encoder] = new AvroOutputStreamBuilder[T](JsonFormat)
  def data[T: Encoder] = new AvroOutputStreamBuilder[T](DataFormat)
}

所以应该像：

class AvroSerializationSchema[IN : Encoder] ...

Answer 2

在写入输出流时，您不需要使用FromRecord。那是针对想要拥有GenericRecord使用的人。您需要使用Encoder。

class AvroSerializationSchema[IN : SchemaFor : Encoder] extends SerializationSchema[IN] {
  override def serialize(element: IN): Array[Byte] = {
    val str = AvroSchema[IN]
    val schema: Schema = new Parser().parse(str.toString)
    val out = new ByteArrayOutputStream()
    val os = AvroOutputStream.data[IN].to(out).build(schema)
    os.write(element)
    out.close()
    out.flush()
    os.flush()
    os.close()
    out.toByteArray
  }
}