2020 IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA)
Research on Dynamic Detection of Java
Dependency Conflict
Kefeng Pan
School of Software & Microelectronics
Peking University
Beijing, China
pankefeng@pku.edu.cn
Abstract—Platform independence is one of the
characteristics of the Java language. The Java program is
compiled to generate bytecode files, then the Java runtime
environment translates the bytecode into machine instructions
for the corresponding platform and runs them. The JVM can
load the corresponding classes when needed. Incorrect class
loading is the root cause of dependency conflicts. Static code
detection can be performed on this type of dependency conflict
problem, and the function call graphs under different
dependency paths can be analyzed and compared to find
possible conflict problems. However, due to the dynamic
nature of the Java language, based on static analysis tools for
analysis, there will be some problems that cannot be handled.
This paper proposes a set of processes and a solution to solve
this kind of problem through bytecode analysis and enhanced
methods.
Keywords—dependency conflict, dynamic detection, bytecode
enhancement
I. INTRODUCTION
The problem of dependency conflicts has always troubled
developers and affected the stability and security of system
operation [1]. For a long time, developers spent a lot of time
and energy to troubleshoot problems. The positioning of this
type of problem often requires developers to have rich
experience. It is difficult for people who have not dealt with
related problems to realize that the problem stems from the
conflict of dependence. For the Java language, some
developers and organizations choose to use class isolation
mechanisms [2] to isolate the effects between components.
But this makes the system must introduce a new framework,
resulting in the system become bloated, must rely on the
class isolation framework to function properly, increasing the
complexity of the system. This approach usually requires
some mandatory development specifications, but also puts
forward higher requirements for developers. When a project
is used as a component, it often needs to be simple enough. If
the introduced component itself has a complicated
framework or because the newly introduced component
needs to change the existing project’s architecture or even
reconstruct some technologies, this is unrealistic. Therefore,
using a class isolation framework is not a general solution. In
most systems, there is a problem of dependency conflicts.
Therefore, it is of great value and significance to study such
problems.
The Java program is compiled to generate a bytecode file,
and the Java runtime environment translates the bytecode
into the machine instructions of the corresponding platform
978-1-7281-6521-9/20/$31.00 ©2020 IEEE
Authorized licensed use limited to: Corporacion Universitaria de la Costa. Downloaded on August 26,2021 at 01:22:22 UTC from IEEE Xplore. Restrictions apply.
Yongzhi Wang
School of Software & Microelectronics
Peking University
Beijing, China
wyz_@pku.edu.cn
and runs it [3]. Loading the incorrect class is the root cause
of dependency conflicts. Static code detection can be
performed on this type of dependency conflict problem, and
the function call graphs under different dependency paths
can be analyzed and compared to find possible conflict
problems [4].
However, due to the dynamic nature of the Java language
[5], based on static analysis tools for analysis, there will be
some problems that cannot be handled. JAVA reflection
mechanism is in the running state, for any class, you can
know all the properties and methods of this class; for any
object, you can call any of its methods and properties. The
class name of the reflection call may be passed in at runtime,
and it is only known at runtime which object is called, which
cannot be obtained by static analysis.
For this kind of situation, this paper proposes a set of
processes and an algorithm to solve this kind of problem
through bytecode analysis and enhanced methods.
II. PROCESS AND SOLUTION DESIGN
As mentioned earlier, the Java language has some
dynamic features, you can use the reflection mechanism to
get and call a class’s properties and methods at runtime. The
class name and the called method are sometimes obtained as
parameters from the outside, so it is impossible to analyze
the information of the class loaded by reflection during static
analysis. If you need to call a method of this class in
operation, and this class loads a conflicting version because
of the JVM’s class loading mechanism. The call to this
method will fail, resulting in an exception at runtime, which
in turn will cause the system to degrade or lose response. In
order to be able to report this kind of dependency conflict,
we need to handle it before the method is called and perform
a check function. If you find that the called method is not in
the actually loaded class method, you can immediately detect
the dependency conflict. To this end, we need to be able to
modify the class method at the time of the reflection call, that
is, to modify the source code or byte code.
The dynamic enhancement process of the dependency
conflict that may exist in the reflection call is shown in Fig. 1.
First, get the bytecode files involved in the system, parse
these bytecode files, and generate the corresponding
bytecode objects. Secondly, analyze whether there are
reflection method calls in the constant pool of these bytecode
objects. If it exists, further analyze the method content of the
bytecode object to find the specific location of the reflection
method call. Then, the bytecode file with the reflection call is
711 Dalian, China•August 25-27, 2020
subjected to bytecode enhancement, and the detection of the
reflection call method is added before the reflection call.
Finally replace the previous file with the modified bytecode
Bytecode file Bytecode file
acquisition& analysis
interpretation
Fig. 1. Bytecode dynamic detection design.
A. Bytecode File Acquisition and Interpretation
Maven is the most common package management tool in
the Java language. By parsing the pom.xml file of the Maven
project, we can obtain all the component jar packages used in
the system, and then get all the .class files. Since .class files
are closely arranged binary stream files, these .class files are
parsed according to certain rules to generate corresponding
class objects, so that the objects can be analyzed.
BCEL, SERP, ASM, Javaassit are some common
ClassReader Visitor
read
ClassVisitor
MethodVisitor
…
Bytecode
Fig. 2. ASM modified byte code flow chart.
B. Bytecode File Analysis
There is a method called getMethod() and a descriptor of
Ljava/lang/Class and a method descriptor beginning with
(Ljava/lang/String; and ending with
Ljava/lang/reflect/Method; At the same time, there is a
NameAndType definition of this method and a Methodref
structure of this method. Then by analyzing the inside of the
method body, we can find out the program location where
the Methodref is called.
C. Bytecode File Enhancement
In the foregoing, through the analysis of the bytecode, the
location of the reflection calls in the system is found. Since
the class name, method name, and parameters of the
reflection call may only be passed in at runtime, static
analysis cannot analyze the exact function call relationship.
If the runtime can perform some verification work before the
method is called by reflection and find and report the non-
712
Authorized licensed use limited to: Corporacion Universitaria de la Costa. Downloaded on August 26,2021 at 01:22:22 UTC from IEEE Xplore. Restrictions apply.
file. The modified bytecode file adds a check to the calling
method at runtime, and if there is a dependency conflict
problem, it can actively report it.
Bytecode file Bytecode file
enhancement replacement
bytecode modification tools. These tools have the function of
parsing bytecode files. But ASM has better performance and
faster speed, and its implementation and design are as small
and faster as possible.
The process of modifying the bytecode using the ASM
framework is shown in Fig. 2. Use ClassReader to read the
bytecode file first, and then use Visitors such as ClassVisitor,
MethodVisitor, etc. to modify the class and method through
the visitor mode, and finally use ClassWriter to modify The
bytecode file is converted back into a binary stream.
ClassWriter
generate
Bytecode
existing method in time, the detection purpose can be
achieved.
It can be seen that before the reflection method is called,
the first three elements on the top of the stack are the
reference of the java.lang.Class[] object array, the name of
the reflection method, and the java.lang.Class object that
calls the reflection method. If the top element of the stack is
popped in sequence, we can get the parameter type of the
reflective call method, the name of the reflective call method,
and the object of the reflective call. By checking these
parameters, we can find whether the class loaded in the
system has a method to be called. If it does not exist, we can
find the problem and report it in time. In summary, the
general steps of bytecode enhancement are given:
1) Add three local variables in the local variable table,
which are used to represent the object of the reflection call,
the name of the reflection call method and the parameter
type of the reflection call method. The indexes in the local
variable table are n, n+1, and n+2, respectively, where n
represents the number of local variables stored in the local
variable table. Their types are java.lang.Class,
java.lang.String, java.lang.Class[], and the corresponding
descriptors are Ljava.lang.Class, Ljava.lang.String,
[Ljava.lang.Class.
2) Bytecode modification performs a stack operation:
Pop the top element of the stack and store it in the local
variable represented by index n+2. The corresponding
bytecode instruction is astore n+2.
Pop the top element of the stack and store it in the local
variable represented by index n+1. The corresponding
bytecode instruction is astore n+1.
Pop the top element of the stack and store it in the local
variable represented by index n, and the corresponding
bytecode instruction is astore n.
3) Bytecode modification performs the new function
operation, and adds a method checkCheck() to the current
class to detect the previous variable. The parameters of the
method correspond to the parameter types at n, n+1, n+2 in
the index of the local variable table.
4) Bytecode modification is performed on the stack:
Push the local variable with index 0 in the local variable
table onto the stack (usually this), and the corresponding
bytecode instruction is aload 0.
Push the local variable with index n in the local variable
table onto the stack, and the corresponding bytecode
instruction is aload n.
Push the local variable with index n in the local variable
table onto the stack, and the corresponding bytecode
instruction is aload n+1.
Execute agent
Load Java
premain
agent
function
Instrumentation
addTransformer()
Monitor bytecode
reading
ClassLoader
Fig. 3. Dynamically enhanced bytecode replacement scheme
Based on Java instrumentation technology [6], we can
design a dynamic enhancement framework as shown in Fig.
3. Proceed as follows:
(1) Obtain the program entry that reflection may affect.
713
Authorized licensed use limited to: Corporacion Universitaria de la Costa. Downloaded on August 26,2021 at 01:22:22 UTC from IEEE Xplore. Restrictions apply.
Push the local variable with index n in the local variable
table onto the stack, and the corresponding bytecode
instruction is aload n+2.
5) Bytecode modification performs the function call
operation, the corresponding bytecode instruction shows
like INVOKEVIRTUAL com/wyz/aaa/A.invokeCheck
(Ljava/lang/Class;Ljava/lang/String;[Ljava/lang/Class;)V
6) Restore the previous execution environment, that is,
put the previously saved parameters back on the stack. The
corresponding bytecode instructions are aload n, aload n+1,
and aload n+2 in this order.
7) Increase the MAXLOCALS variable of the local
variable table by 3 and the MAXSTACK variable by 1. The
MAXLOCALS variable indicates the number of local
variables in the method, and MAXSTACK indicates the
maximum depth of the stack used. The bytecode operation
for the reflection call check adds the operation of pushing
this on the original basis, which may cause the maximum
stack depth to change.
D. Bytecode File Replacement
There are two alternatives for bytecode enhancements to
classes and methods of reflection calls that may have
dependency conflict issues. The first is to modify the original
jar package file and repackage the enhanced bytecode file.
The second is to modify it dynamically at runtime, trying to
replace it before the class loads. The first method will affect
the original jar package file, resulting in unpredictable
consequences when referenced by other projects, and the
files in the local warehouse are inconsistent with the files in
the remote warehouse. The problem is still not solved when
redeploying on a new machine, so this section mainly
discusses the second approach, byte code replacement at
runtime.
Execute
system main
function
Bytecode
enhancem
ent
Callback before
loading class
(2) Enhance the bytecode file and insert the context of
the reflection call into the newly added check function.
(3) Read the class that needs to be modified.
(4) Monitor the loading process of classes that need to be
modified in the system during operation.
(5) When the loading action is monitored, the byte code
of this class is replaced.
III. CONCLUSIONS
In this article we mainly discussed the design and
solution to the dynamic detection of Java language
dependency conflicts . For some problems that cannot be
solved by static analysis, the positioning-bytecode
enhancement-dynamic replacement scheme is adopted, and
the bytecode enhancement is performed at runtime based on
Java agent technology, so that when conflicts occur, they can
be actively reported.
There are two interesting future directions for this work.
First, The dependency conflict detection tool implemented in
this article is only applicable to the Java language, especially
the Java components under the management of Maven
management tools. But in the actual project, first of all, each
language has the problem of dependency conflict, not just
Java. Secondly, the management tool of each language is not
a kind. Java alone also has package management tools such
as Gradle. How to get rid of the limitation of management
tools and language and realize a universal detection
technology still needs further study and summary. Second,
The dynamic detection method in this article is closely
714
Authorized licensed use limited to: Corporacion Universitaria de la Costa. Downloaded on August 26,2021 at 01:22:22 UTC from IEEE Xplore. Restrictions apply.
related to the design of the Java program, especially when
the bytecode is enhanced, it is closely combined with the
structure of the Java file and the mechanism of the JVM.
When there is no such design for other languages, how to
perform dynamic detection is still a problem that needs to be
considered.
REFERENCES
[1] J. Businge, A. Serebrenik and M. V. D. Brand, “Compatibility
Prediction of Eclipse Third-Party Plug-ins in New Eclipse Releases,”
12th IEEE International Working Conference on Source Code
Analysis and Manipulation. IEEE, 2012.
[2] A. L. C. Tavares and M. T. Valente, “A gentle introduction to OSGi,”
ACM SIGSOFT Software Engineering Notes, vol. 33, no. 5, pp 8,
2008.
[3] F. Robert, S. Joachim and B. Egon, “Java and the Java virtual
machine: definition, verification, validation,” Springer Science &
Business Media, 2012.
[4] Y. Wang, et al., “Do the dependency conflicts in my project matter?”
Proceedings of the 2018 26th ACM Joint Meeting on European
Software Engineering Conference and Symposium on the
Foundations of Software Engineering, 2018.
[5] T. Jensen, M. D. Le, T. Thorn, “Security and dynamic class loading in
Java: A formalisation,” 1998 International Conference on Computer
Languages, Proceedings. pp. 4-15, 1998.
[6] W. Binder, J. Hulaas and P. Moret, “Advanced Java bytecode
instrumentation”, Proceedings of the 5th international symposium on
Principles and practice of programming in Java, pp. 135-144, 2007.