Log4J2 dynamic appender doesn't work with maven-shade-plugin

Question

I add an appender programmatically. It was working until I added maven-shade-plugin. I wonder what makes my appender fail.

The appender works ✅ or not ❌ in these scenarios:

1. ✅ From IDE (IntelliJ IDEA)
2. ❌ With the shade (Uber/fat) jar <-- now works ✅ see answer
3. ✅ With separate jars (app jar, log4j jars)
4. ❌ With the app jar, and the log4j jars unzipped into a folder
5. ✅ With the app jar, and the log4j folder re-zipped

Sample project

• https://github.com/fmaylinch/log4j2-thread-context
• Here's the point where I add the custom appender: ThreadContextFileLogHandler.java#L79
• When the appender fails, it's because it doesn't receive the events here: ThreadContextFileLogHandler.java#L156

Reproduce scenarios with the sample project above

mvn clean compile
mkdir -p local/log4j-jars
unzip $HOME/.m2/repository/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.jar -d local/log4j-jars
unzip -o $HOME/.m2/repository/org/apache/logging/log4j/log4j-core/2.17.2/log4j-core-2.17.2.jar -d local/log4j-jars
cd local/log4j-jars
zip -r ../log4j-jars.zip .
cd ../..

# Scenario 2 ❌ uses fat jar
java -cp "target/log4j-test-1.0-SNAPSHOT.jar" org.example.Main

# Scenario 3 ✅ uses separate jars
java -cp "target/original-log4j-test-1.0-SNAPSHOT.jar:$HOME/.m2/repository/org/apache/logging/log4j/log4j-core/2.17.2/log4j-core-2.17.2.jar:$HOME/.m2/repository/org/apache/logging/log4j/log4j-api/2.17.2/log4j-api-2.17.2.jar" org.example.Main

# Scenario 4 ❌ uses log4j files unzipped
java -cp "target/original-log4j-test-1.0-SNAPSHOT.jar:local/log4j-jars" org.example.Main

# Scenario 5 ✅ uses log4j files re-zipped
java -cp "target/original-log4j-test-1.0-SNAPSHOT.jar:local/log4j-jars.zip" org.example.Main

Extra notes

In the scenario 5, I have noticed that I can remove some files in META-INF, but for my appender to work, I need to keep the following:

• META-INF
  • org (contains Log4j2Plugins.dat)
  • services (without this, the app even crashes)
  • versions
  • MANIFES.MF

Related questions

• log4j2 JsonTemplateLayout not working with maven shade plugin - this seems to be a problem with Log4j2Plugins.dat, but in my project there's only one file, in log4j2-core.
• is there any diff between run a java program with jar or with a package unpacked? - according to my problem, there IS a difference (maybe the META-INF folder is not processed like in a jar)

Answer 1

The problem with maven-shade-plugin is that it breaks the manifest of the original jars and overwrites important resources. I find the spring-boot-maven-plugin much more useful and it can be also used by applications that don't use Spring at all.

The maven-shade-plugin in the context of Log4j requires a minimal configuration as in this question:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-shade-plugin</artifactId>
  <dependencies>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-transform-maven-shade-plugin-extensions</artifactId>
      <version>0.1.0</version>
    </dependency>
  </dependencies>
  <configuration>
    <transformers>
      <transformer implementation="org.apache.logging.log4j.maven.plugins.shade.transformer.Log4j2PluginCacheFileTransformer"/>
      <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
      <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
        <manifestEntries>
          <Multi-Release>true</Multi-Release>
        </manifestEntries>
      </transformer>
    </transformers>
  </configuration>
</plugin>

This configuration takes care of:

• merging Log4j2Plugins.dat files. Without Log4j2PluginCacheFileTransformer you can not use additional component libraries except log4j-api and log4j-core,
• merging service files. Without ServiceResourceTransformer you'll lose additional component like property sources,
• marking the JAR as multi-release jar: some classes used to gather caller's information were replaced in JDK 11. Therefore some Log4j classes have two versions (JDK 8 and JDK 9+). If you don't mark the JAR as multi-release, it will not work on JDK 11+.

Edit: All these problems with the maven-shade-plugin sum up to one: every time two jars have a file with the same name, it must be somehow merged.

That is why I prefer the spring-boot-maven-plugin: instead of breaking multiple jars and adding their files into a single archive, it adds the original jars to the archive. The exact structure of the resulting jar is described in executable Jar format.

The usage is straightforward: just add the repackage goal to your build and remove maven-shade-plugin.

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <version>2.7.10</version>
    <executions>
        <execution>
            <goals>
                <goal>repackage</goal>
            </goals>
        </execution>
    </executions>
</plugin>

This will effectively add the small spring-boot-loader to your application. Version 2.x of the library requires Java 8, while version 3.x requires Java 17.