JSON Null Pointer Exception in Array with Jackson ObjectMapper

Question

I am getting an array size of zero for the "tweets" array when trying to access the arraylist that it should be stored in. I am using Jackon databinding to a class, which at this point has "worked" as in not gave any errors or exceptions. But, the problem is still that the array is returning empty.

Here is the method where I am accessing the class that the JSON is being binded to:

WeatherDatabaseCreator.java

    //Read in files to be parsed.
    for (int i = 0; i < twitterFiles.size(); i++) {

        File twitterFile = twitterFiles.get(i);
        WeatherTweetParser wtp = new WeatherTweetParser(twitterFile);
        wtp.mapJavaObject();

        tws = new Tweets();
        tweets = tws.getTweets();

        System.out.print(tweets.size());

        tweetIndex = 0;

        System.out.println("Parsing and inserting twitter file '" + twitterFile.getAbsolutePath() + "'...");

    //Step through all the tweets in each file, using the TweetParser class.
    while (true) {
        tweet = tweets.get(tweetIndex);
        user = tweet.new User();
        entity = tweet.new Entities();

        userMentions = entity.getUserMentions();
        hashtags = entity.getHashtags();

        if (tweet.getText() == null) {
            break;
        }
        numParsed++;

        Boolean success = wdbm.insertTweet(tweet, user);
        if (success == true) {
            numInserted++;
        }

        if (entity.getNumHashtags() > 0){

            for (int j = 0; j < entity.getNumHashtags(); j++) {
                wdbm.insertHashtag(tweet, hashtags.get(j)); 
            }   
        }

        if (entity.getNumUserMentions() > 0) {

            for (int k = 0; k < entity.getNumUserMentions(); k++) {
                wdbm.insertReference(tweet, userMentions.get(k));
            }
        }

        if (numParsed % UPDATE_INTERVAL == 0) {
            System.out.println(numParsed + " tweets parsed. " + numInserted + " tweets inserted into database...");
        }

        tweetIndex++;
    }

Here is part of the JSON File. The file is made up of an array of tweets, with some nested classes. Here I included two elements of the array:

2014-01-03-11-03.terms.json

{
"tweets": [
    {
        "filter_level": "medium",
        "contributors": null,
        "text": "U know your ice fishing at home when ur snacks are sandwiches,chips,dips,beef jerky and all the goodies instead of just a case of beer #mom",
        "geo": null,
        "retweeted": false,
        "in_reply_to_screen_name": null,
        "truncated": false,
        "lang": "en",
        "entities": {
            "symbols": [

            ],
            "urls": [

            ],
            "hashtags": [
                {
                    "text": "mom",
                    "indices": [
                        135,
                        139
                    ]
                }
            ],
            "user_mentions": [

            ]
        },
        "in_reply_to_status_id_str": null,
        "id": 419137025376145408,
        "source": "<a href=\"http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone<\/a>",
        "in_reply_to_user_id_str": null,
        "favorited": false,
        "in_reply_to_status_id": null,
        "retweet_count": 0,
        "created_at": "Fri Jan 03 16:03:48 +0000 2014",
        "in_reply_to_user_id": null,
        "favorite_count": 0,
        "id_str": "419137025376145408",
        "place": null,
        "user": {
            "location": "",
            "default_profile": false,
            "profile_background_tile": true,
            "statuses_count": 5411,
            "lang": "en",
            "profile_link_color": "17EBCF",
            "profile_banner_url": "https://pbs.twimg.com/profile_banners/408240224/1384838946",
            "id": 408240224,
            "following": null,
            "protected": false,
            "favourites_count": 4222,
            "profile_text_color": "333333",
            "description": "It is what it is, no more, no less. Trust in the Lord. BSU Softball #5.  My whole being is happy - Psalm 16:9",
            "verified": false,
            "contributors_enabled": false,
            "profile_sidebar_border_color": "FFFFFF",
            "name": "Nicole Anne",
            "profile_background_color": "C0DEED",
            "created_at": "Wed Nov 09 05:03:58 +0000 2011",
            "default_profile_image": false,
            "followers_count": 452,
            "profile_image_url_https": "https://pbs.twimg.com/profile_images/413270299925164033/mWiJBKP2_normal.jpeg",
            "geo_enabled": true,
            "profile_background_image_url": "http://a0.twimg.com/profile_background_images/698793822/14d9ce538d96b435ad4449d5385a3ec1.jpeg",
            "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/698793822/14d9ce538d96b435ad4449d5385a3ec1.jpeg",
            "follow_request_sent": null,
            "url": null,
            "utc_offset": -32400,
            "time_zone": "Alaska",
            "notifications": null,
            "profile_use_background_image": true,
            "friends_count": 634,
            "profile_sidebar_fill_color": "DDEEF6",
            "screen_name": "NDupay",
            "id_str": "408240224",
            "profile_image_url": "http://pbs.twimg.com/profile_images/413270299925164033/mWiJBKP2_normal.jpeg",
            "listed_count": 1,
            "is_translator": false
        },
        "coordinates": null
    },
    {
        "filter_level": "medium",
        "contributors": null,
        "text": "Just watched DeBlasio shovel snow for the cameras and I have to say, his form is terrible. Looks a bit inexperienced.",
        "geo": null,
        "retweeted": false,
        "in_reply_to_screen_name": null,
        "truncated": false,
        "lang": "en",
        "entities": {
            "symbols": [

            ],
            "urls": [

            ],
            "hashtags": [

            ],
            "user_mentions": [

            ]
        },
        "in_reply_to_status_id_str": null,
        "id": 419137026458673152,
        "source": "<a href=\"http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone<\/a>",
        "in_reply_to_user_id_str": null,
        "favorited": false,
        "in_reply_to_status_id": null,
        "retweet_count": 0,
        "created_at": "Fri Jan 03 16:03:48 +0000 2014",
        "in_reply_to_user_id": null,
        "favorite_count": 0,
        "id_str": "419137026458673152",
        "place": null,
        "user": {
            "location": "",
            "default_profile": false,
            "profile_background_tile": false,
            "statuses_count": 2609,
            "lang": "en",
            "profile_link_color": "FF8000",
            "id": 163686045,
            "following": null,
            "protected": false,
            "favourites_count": 204,
            "profile_text_color": "333333",
            "description": "Love learning, Hockey #NYR, music and photography. My family is my whole life.",
            "verified": false,
            "contributors_enabled": false,
            "profile_sidebar_border_color": "EEEEEE",
            "name": "Steven Marques",
            "profile_background_color": "ACDED6",
            "created_at": "Wed Jul 07 01:26:25 +0000 2010",
            "default_profile_image": false,
            "followers_count": 56,
            "profile_image_url_https": "https://pbs.twimg.com/profile_images/378800000795958214/5e3ff928282ad692a36e941e5c5ff628_normal.jpeg",
            "geo_enabled": false,
            "profile_background_image_url": "http://abs.twimg.com/images/themes/theme18/bg.gif",
            "profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme18/bg.gif",
            "follow_request_sent": null,
            "url": null,
            "utc_offset": -18000,
            "time_zone": "Eastern Time (US & Canada)",
            "notifications": null,
            "profile_use_background_image": true,
            "friends_count": 58,
            "profile_sidebar_fill_color": "F6F6F6",
            "screen_name": "SMGafanha",
            "id_str": "163686045",
            "profile_image_url": "http://pbs.twimg.com/profile_images/378800000795958214/5e3ff928282ad692a36e941e5c5ff628_normal.jpeg",
            "listed_count": 2,
            "is_translator": false
        },
        "coordinates": null
    },

Here is the Java class that the JSON file is mapped to, followed by the class mapping the JSON to the class:

Tweets.java

public class Tweets {

private ArrayList<Tweet> tweets;

public Tweets() {
    tweets = new ArrayList<Tweet>();
}

public ArrayList<Tweet> getTweets() {
    return tweets;
}

public void setTweets(ArrayList<Tweet> tweets) {
    this.tweets = tweets;
}

public static class Tweet {

    private String text;
    @JsonProperty("source")
    private String postMethod;
    @JsonProperty("created_at")
    private String time;
    @JsonProperty("id")
    private String ID;

    public Tweet() {

    }

    public String getText() {
        return text;
    }

    public void setText(String text) {
        this.text = text;
    }

    public String getID() {
        return ID;
    }

    public void setID(String iD) {
        ID = iD;
    }

    public String getPostMethod() {
        return postMethod;
    }

    public void setPostMethod(String postMethod) {
        this.postMethod = postMethod;
    }

    public String getTime() {
        return time;
    }

    public void setTime(String time) {
        this.time = time;
    }

    public class User {

        private String location;
        @JsonProperty("screen_name")
        private String user;
        @JsonProperty("time_zone")
        private String timeZone;
        @JsonProperty("description")
        private String accountDescription;

        public User(){

        }

        public String getLocation() {
            return location;
        }

        public void setLocation(String location) {
            this.location = location;
        }

        public String getUser() {
            return user;
        }

        public void setUser(String user) {
            this.user = user;
        }

        public String getTimeZone() {
            return timeZone;
        }

        public void setTimeZone(String timeZone) {
            this.timeZone = timeZone;
        }

        public String getAccountDescription() {
            return accountDescription;
        }

        public void setAccountDescription(String accountDescription) {
            this.accountDescription = accountDescription;
        }
    }
    public class Entities {

        @JsonProperty("user_mentions")
        ArrayList<UserMention> userMentions;
        @JsonProperty("hashtags")
        ArrayList<Hashtag> hashtags;

        public Entities() {

        }

        public ArrayList<UserMention> getUserMentions() {
            return userMentions;
        }

        public int getNumUserMentions() {
            return userMentions.size();
        }

        public UserMention getUserMention(int index) {
            return userMentions.get(index);
        }

        public void setUserMentions(ArrayList<UserMention> userMentions) {
            this.userMentions = userMentions;
        }

        public ArrayList<Hashtag> getHashtags() {
            return hashtags;
        }

        public int getNumHashtags() {
            return hashtags.size();
        }

        public Hashtag getHashtag(int index) {
            return hashtags.get(index);
        }

        public void setHashtags(ArrayList<Hashtag> hashtags) {
            this.hashtags = hashtags;
        }

        public class UserMention {

            @JsonProperty("screen_name")
            private String userRef;

            public UserMention() {

            }

            public String getUserRef() {
                return userRef;
            }

            public void setUserRef(String userRef) {
                this.userRef = userRef;
            }
        }
        public class Hashtag {

            @JsonProperty("text")
            private String hashText;

            public Hashtag() {

            }

            public String getHashText() {
                return hashText;
            }

            public void setHashText(String hashText) {
                this.hashText = hashText;
            }
        }
    }
}

}

WeatherTweetParser.java

public class WeatherTweetParser {

private static File twitterFile;

public WeatherTweetParser(File twitterFile) {
    WeatherTweetParser.twitterFile = twitterFile;
}

public void mapJavaObject() {

    String jsonFile = twitterFile.getName();

    System.out.println("Mapping JSON to WeatherTweet Class.");

    ObjectMapper mapper = new ObjectMapper();
    mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
    try {
        mapper.readValue(new File(jsonFile), Tweets.class);
    } catch (JsonParseException e) {
        e.printStackTrace();
    } catch (JsonMappingException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }

    System.out.println("Finished Mapping of JSON File.");
}

And here is the error:

Mapping JSON to WeatherTweet Class.
Finished Mapping of JSON File.
Tweets size: 0
Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
    at java.util.ArrayList.rangeCheck(Unknown Source)
    at java.util.ArrayList.get(Unknown Source)
    at database.WeatherDatabaseCreator.main(WeatherDatabaseCreator.java:145)
Parsing and inserting twitter file 'C:\Users\Jared\Documents\Eclipse               
WorkSpace\workspace\WeatherDatabaseCreator\2014-01-03-11-03.terms.json'...

On this line of WeatherDatabaseCreator.java:

tweet = tweets.get(tweetIndex);

TL:DR: I understand this error is because the index is out of range, because there is nothing obviously even in the tweets array. But, if the Jackson mapping of the JSON file to my Java Class went through without exceptions, why is the tweets array empty?

Answer 1

  while (true) {
    tweet = tweets.get(tweetIndex);
    user = tweet.new User();
    entity = tweet.new Entities();

    userMentions = entity.getUserMentions();
    hashtags = entity.getHashtags();

    if (tweet.getText() == null) {
        break;
    }

The only break in this loop will work only if getText is null. If getText is not null for all the tweets, you'll get index out of bound exception. Instead of while(true) you should probably use a check like tweetIndex < tweets.size()

The second problem. You create a Tweets object without loading any items there:

    wtp.mapJavaObject();

    tws = new Tweets();
    tweets = tws.getTweets();

Of course tws object will not have any tweets in this case. What you need to do is to make sure that your mapJavaObjects() method returns a Tweets object with the tweets loaded by ObjectMapper and after this is done your code should look like this:

tweets = wtp.mapJavaObject();

Something like this:

public Tweets mapJavaObject() {

String jsonFile = twitterFile.getName();
Tweets ret = null; 

System.out.println("Mapping JSON to WeatherTweet Class.");

ObjectMapper mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
try {
    ret = (Tweets) mapper.readValue(new File(jsonFile), Tweets.class);
} catch (JsonParseException e) {
    e.printStackTrace();
} catch (JsonMappingException e) {
    e.printStackTrace();
} catch (IOException e) {
    e.printStackTrace();
}

System.out.println("Finished Mapping of JSON File.");
retrun ret;
}