Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to write union when creating Avro file in Java

Tags:

java

union

avro

I'm trying to create Avro file in Java (just testing code at the moment). Everything works fine, the code looks about like this:

GenericRecord record = new GenericData.Record(schema);

File file = new File("test.avro");
DatumWriter<GenericRecord> datumWriter = new GenericDatumWriter<GenericRecord>(schema);
DataFileWriter<GenericRecord> dataFileWriter = new DataFileWriter<GenericRecord>(datumWriter);
dataFileWriter.create(schema, file);
dataFileWriter.append(record);
dataFileWriter.close();

The problem I'm facing now is - what kind of Java object do I instantiate when I want to write Union? Not necessarily on the top level, possibly attach the union to a record being written. There are a few objects for complex types prepared, like GenericData.Record, GenericData.Array etc. For those that are not prepared, usually the right object is simply a standard Java object (java.util.Map implementing classes for "map" Avro type etc.).

But I cannot figure out what is the right object to instantiate for writing a Union.

This question refers to writing Avro file WITHOUT code generation. Any help is very much appreciated.

like image 351
Sethiel Avatar asked Oct 21 '25 11:10

Sethiel


1 Answers

Here's what I did:

Suppose the schema is defined like this:

record MyStructure {
  ...
  record MySubtype {
    int p1;
  }
  union {null, MySubtype} myField = null;
  ...
}

And this is the Java code:

Schema schema; // the schema of the main structure
// ....
GenericRecord rec = new GenericData.Record(schema);
int i = schema.getField("myField").schema().getIndexNamed("MySubtype");
GenericRecord myField = new GenericData.Record(schema.getField("myField").schema().getTypes().get(i));
myField.put("p1", 100);
rec.put("myField", myField);
like image 62
Bogdan Avatar answered Oct 23 '25 00:10

Bogdan



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!