Merging two yaml documents while concatenating arrays

Question

I want to merge two yaml documents with the result containing

• all mapped values (with the last one taking precedence)
• concatenated arrays

e.g. given this file

# file1.yml
animals:
  - elephant
    donkey
flavours:
  sour:
    - lemon
  sweet:
    - chocolate
strange: true

and this file

#file2.yml
animals:
  - monkey
    pelican
flavours:
  sweet:
    - vanilla
strange: false

the result should contain all the nodes, with merged arrays and values that are not arrays from the last file

#result.yml
animals:
  - elephant
    donkey
    monkey
    pelican
flavours:
  sour:
    - lemon
  sweet:
    - chocolate
      vanilla
strange: false

Can yq do this, maybe?

Answer 1

Merging YAML files with yq, concatenating arrays.

Assumption

In both file1.yml and file2.yml, the animals array contains a single index with a multiline string.

elephant
donkey

for file1.yml, and

monkey
pelican

for file2.yml.

Since you asked about concatenated array, I am assuming file1.yml, file2.yml and results.yml should be like this:

# file1.yml
animals:
  - elephant
  - donkey
flavours:
  sour:
    - lemon
  sweet:
    - chocolate
strange: true

---

# file2.yml
animals:
  - monkey
  - pelican
flavours:
  sweet:
    - vanilla
strange: false

---

# result.yml
animals:
  - elephant
  - donkey
  - monkey
  - pelican
flavours:
  sour:
    - lemon
  sweet:
    - chocolate
    - vanilla
strange: false

Example

With yq 4.x, you can use the ireduce operator to merge two or more files:

https://mikefarah.gitbook.io/yq/operators/reduce#merge-all-yaml-files-together

$ yq4 eval-all '. as $item ireduce ({}; . * $item)' file1.yml file2.yml
animals:
  - monkey
  - pelican
flavours:
  sour:
    - lemon
  sweet:
    - vanilla
strange: false

Both files have been merged, but duplicated keys and arrays have been overwritten by the latest file.

To appends arrays instead of overriding them, simply add a + after the merge operator (*).

https://mikefarah.gitbook.io/yq/operators/multiply-merge#merge-appending-arrays

$ yq4 eval-all '. as $item ireduce ({}; . *+ $item)' file1.yml file2.yml
animals:
  - elephant
  - donkey
  - monkey
  - pelican
flavours:
  sour:
    - lemon
  sweet:
    - chocolate
    - vanilla
strange: false