serde_json flattening object with indices as keys

Question

I have some json with from an external API that I would like to type. The shape of the data looks like:

{
 "summary": {
    "field1": "foo",
    "field2": "bar",
  },
 "0": {
   "fieldA": "123",
   "fieldB": "foobar"
 },
 "1": {
   "fieldA": "245",
   "fieldB": "foobar"
 },
 ...
}

There is an unknown number of indexed objects returned below the summary field, depending on the query that is run. These objects have the same shape, but a different shape than the "summary" object. I would like to use serde_json to type this response into something like:

#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct SearchResults {
    pub summary: Summary,
    pub results: Vec<IndexedFieldType>
}

Is it possible to do this with serde macros? Is there a "all other fields" catchall that I can flatten into a vec?

Answer 1

2 years later is a little too late I think, but if you still need it...

First, the data with static structure:

use serde::Deserializer;

#[derive(serde::Deserialize, Debug)]
#[serde(rename_all = "camelCase")]
struct Summary {
    field1: String,
    field2: String,
}

#[derive(serde::Deserialize, Debug)]
#[serde(rename_all = "camelCase")]
struct FieldType {
    field_a: String,
    field_b: String,
}

For parsing extra options from an object you could use the #[serde(flatten)] attribute, see https://serde.rs/attr-flatten.html.

#[derive(serde::Deserialize, Debug)]
#[serde(rename_all = "camelCase")]
struct SearchResults {
    pub summary: Summary,
    #[serde(flatten)]
    pub results: HashMap<String, FieldType>,
}

This way, any extra property of the object is going to be included inside the results HashMap. Of course, the problem is that not any string is a valid index.

Then, you'll need to create a custom type to properly parse the keys.

use std::Hash;

#[derive(Hash, PartialEq, Eq, Debug)]
struct Stringified(u32);

impl<'de> serde::Deserialize<'de> for Stringified {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: serde::Deserializer<'de>,
    {
        let s = String::deserialize(deserializer)?;

        match s.parse::<u32>() {
            Ok(n) => Ok(Stringified(n)),
            Err(_) => Err(serde::de::Error::custom(format!(
                r#"recieved "{s}", expected "{{u32}}""#
            ))),
        }
    }
}

So, you can try the behavior.

#[derive(serde::Deserialize, Debug)]
#[serde(rename_all = "camelCase")]
struct SearchResults {
    pub summary: Summary,
    #[serde(flatten)]
    pub results: HashMap<Stringified, FieldType>,
}

fn main() {
    let json = r#"
{
 "summary": {
    "field1": "foo",
    "field2": "bar"
  },
 "0": {
   "fieldA": "123",
   "fieldB": "foobar"
 },
 "1": {
   "fieldA": "245",
   "fieldB": "foobar"
 }
}"#;

    let results: SearchResults = serde_json::from_str(json).unwrap();
    println!("{:#?}", results);
}

Note that, u32 is only one of the possible numeric types and may not be the one you want, change it to whatever you need, you can even make it generic.

Good luck.

Answer 2

I ended up doing this manually like so:

let summary = &json.search_results["summary"];
let summary: Summary = serde_json::from_value(summary.to_owned()).unwrap();
let results: Vec<SearchResult> = json
        .search_results
        .as_object()
        .unwrap()
        .iter()
        .filter(|(key, _val)| key.parse::<i32>().is_ok())
        .map(|(_key, value)| serde_json::from_value(value.to_owned()).unwrap())
        .collect();
        
Ok(SearchResults {
            summary,
            results
        })