Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

populating nested records with array in Avro using a GenericRecord

Tags:

java

scala

avro

I have the following schema:

{
    "name": "AgentRecommendationList",
    "type": "record",
    "fields": [
        {
            "name": "userid",
            "type": "string"
        },
        {
            "name": "friends",
            "type": {
                "type": "array",
                "items": {
                    "name": "SchoolFriends",
                    "type": "record",
                    "fields": [
                        {
                            "name": "Name",
                            "type": "string"
                        },
                        {
                            "name": "phoneNumber",
                            "type": "string"
                        },
                        {
                            "name": "email",
                            "type": "string"
                        }
                    ]
                }
            }
        }
    ]
}

I'm using GenericRecord, and I want to put in an array of arrays for the SchoolFriends.

val avschema = new RestService(URL).getLatestVersion(name)
val schema = new Schema.Parser().parse(avschema.getSchema)
val record = new GenericData.Record(schema)

I would like to do something like record.put(x)

like image 476
Brian Avatar asked Oct 20 '25 13:10

Brian


1 Answers

for that particular schema, you can do it in the following way. I would recommend put your record type SchoolFriends in a different schema, it would make easy to get the schema for the collection elements.

import org.apache.avro.Schema;
import org.apache.avro.generic.GenericData;
import org.apache.avro.generic.GenericRecord;

import java.util.ArrayList;
import java.util.List;

public class PopulateNestedAvroObjects {
    public static void main(String [] args ){
        String strSchema = "{\n" +
                "    \"name\": \"AgentRecommendationList\",\n" +
                "    \"type\": \"record\",\n" +
                "    \"fields\": [\n" +
                "        {\n" +
                "            \"name\": \"userid\",\n" +
                "            \"type\": \"string\"\n" +
                "        },\n" +
                "        {\n" +
                "            \"name\": \"friends\",\n" +
                "            \"type\": {\n" +
                "                \"type\": \"array\",\n" +
                "                \"items\": {\n" +
                "                    \"name\": \"SchoolFriends\",\n" +
                "                    \"type\": \"record\",\n" +
                "                    \"fields\": [\n" +
                "                        {\n" +
                "                            \"name\": \"Name\",\n" +
                "                            \"type\": \"string\"\n" +
                "                        },\n" +
                "                        {\n" +
                "                            \"name\": \"phoneNumber\",\n" +
                "                            \"type\": \"string\"\n" +
                "                        },\n" +
                "                        {\n" +
                "                            \"name\": \"email\",\n" +
                "                            \"type\": \"string\"\n" +
                "                        }\n" +
                "                    ]\n" +
                "                }\n" +
                "            }\n" +
                "        }\n" +
                "    ]\n" +
                "}";

        Schema schema = new Schema.Parser().parse(strSchema);
        GenericRecord record = new GenericData.Record(schema);
        record.put("userid", "test user");
        Schema childSchema = record.getSchema().getField("friends").schema().getElementType();
        List<GenericRecord> friendList = new ArrayList();
        GenericRecord friend1 = new GenericData.Record(childSchema);
        friend1.put("Name", "1");
        friend1.put("phoneNumber", "2");
        friend1.put("email", "3");
        friendList.add(friend1);
        record.put("friends", friendList);
        System.out.println(record);


    }
}
like image 118
hlagos Avatar answered Oct 22 '25 02:10

hlagos



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!