Static Java classpath hacks

A few days ago I encountered a situation where I needed to incorporate a JAR file into Matlab’s static Java classpath. I came across several relevant hacks that I thought could be useful for others:

• The documented approach
• Fixing javaclasspath.txt in run-time
• Preloading Java class files in javaclasspath.txt
• Loading Java class files into the static classpath in run-time (!)
• Analyzing loaded Java classes in run-time

The documented approach

In most use-cases, adding Java classes (or ZIP/JAR files) to the dynamic Java classpath is good enough. This can easily be done in run-time using the javaaddpath function. In certain cases, for example where the Java code uses asynchronous events, the Java files need to be added to the static, rather than the dynamic, Java classpath. In other cases, the Java code misbehaves when loaded using Matlab’s dynamic Java classloader, rather than the system classloader (which is used for the static classpath (some additional info). One example of this are the JAR/ZIP files used to connect to various databases (each database has its own Java JDBC connector, but they all need to reside in the static Java classpath to work properly).

Adding class-file folders or ZIP/JAR files to the static Java classpath can be done in several manners: we can update the Matlab installation’s classpath.txt file, or (starting in R2012b) a user-prepared javaclasspath.txt file. Either of these files can be placed in the Matlab (or deployed application’s) startup folder, or the user’s Matlab preferences folder (prefdir). This is all documented.

There are several important drawbacks to this approach:

1. Both classpath.txt and javaclasspath.txt do not accept relative paths, only full path-names. In other words, we cannot specify ../../libs/mylib.jar but only C:\Yair\libs\mylib.jar or similar variants. We can use the $matlabroot, $arch, and $jre_home place-holder macros, but not user folders. We can easily do this in our personal computer where the full path is known, but this is difficult to ensure if we deploy the application onto other computers.
   Matlab’s compiler only supports adding Java classes and JARs to the dynamic classpath, but not to the static one – we need to manually add a classpath.txt or javaclasspath.txt to the deployment folder, complete with the deployment target’s correct full path. Naturally this is problematic when we wish to grant the user the flexibility of installing the deployed application anywhere on their computer disk.
2. The javaclasspath.txt file is only supported on R2012b onward. The classpath.txt file is supported in both old and new releases, but (1) modifying it in Matlab’s installation folder may require system privileges, and (2) require update upon each and every Matlab reinstallation or upgrade. A classpath.txt file from one Matlab release will NOT be compatible with another Matlab release – if we try using an incorrect file Matlab will crash or hang. If you are not using just a single matlab release, this quickly turns into a maintenance nightmare.
3. The javaclasspath.txt file loads the user-specified classes after those loaded in Matlab’s pre-defined classpath.txt. If we edit The classpath.txt file, we can place our classes at the top of the file, but this again entails the maintenance nightmare described above

Luckily, there are a few hacks that can alleviate some of the pain when dealing with deployed applications that use static Java classes:

Fixing javaclasspath.txt in run-time

At the very top of our main function, we could fix (or create) the javaclasspath.txt file to include the current folder’s actual full path, and then relaunch the application and exit the current session. The newly-launched application will use this javaclasspath.txt file and everything should work well from that moment onward. Here’s a bare-bones implementation example:

function mainProgram()
   if ~exist('javaclasspath.txt','file')
      try
         fid = fopen('javaclasspath.txt', 'wt');
         fprintf(fid, [pwd '/classesFolder/']);  % add folder of *.class files
         fprintf(fid, [pwd '/subFolder/classes.jar']);  % add jar/zip file
         fclose(fid);
         system(['./' mfilename ' &']);
      catch err
         msg = sprintf('Could not create %s/javaclasspath.txt - error was: %s', pwd, err.message);
         uiwait(msgbox(msg, 'Error', 'warn'));  % need uiwait since otherwise exit below will delete the msgbox
      end
      exit
   end
 
   % If we reached this point, then everything is ok - proceed normally
   ...

A related solution is to recreate the classpath.txt file using the build-generation script, as described by Andrew Janke.

Preloading Java class files in javaclasspath.txt

Class folders and JAR/ZIP files can be loaded before those in classpath.txt, without having to modify the system’s classpath.txt, as exposed by Christopher Barber. All we need to do is to add a line containing the <before> string as a separate line in our javaclasspath.txt user-file. All entries beneath this line will be loaded into the front (rather than the end) of the static Java classpath. For example:

# This will be placed at the end of the static Java classpath
C:\placed\at\the\end\of\the\static\classpath\mylib1.jar
 
<before>
 
# This will be placed at the beginning of the static Java classpath
C:\placed\at\the\top\of\the\static\classpath\mylib2.jar

Loading Java class files into the static classpath in run-time (!)

The crown jewels in Java static classpath hacking belong without a doubt to power-users Andrew Janke and Amro. Based on Amro’s tip (based on a Java forum post from back in 2002), Andrew created a Matlab function that loads a Matlab class into the static classpath at run-time (i.e., without needing to modify/create either classpath.txt or javaclasspath.txt), slightly modified by me to include an optional classname input arg. Here’s the resulting javaaddpathstatic function:

function javaaddpathstatic(file, classname)
%JAVAADDPATHSTATIC Add an entry to the static classpath at run time
%
% javaaddpathstatic(file, classname)
%
% Adds the given file to the static classpath. This is in contrast to the
% regular javaaddpath, which adds a file to the dynamic classpath.
%
% Files added to the path will not show up in the output of
% javaclasspath(), but they will still actually be on there, and classes
% from it will be picked up.
%
% Caveats:
%  * This is a HACK and bound to be unsupported.
%  * You need to call this before attempting to reference any class in it,
%    or Matlab may "remember" that the symbols could not be resolved. Use
%    the optional classname input arg to let Matlab know this class exists.
%  * There is no way to remove the new path entry once it is added.
 
% Andrew Janke 20/3/2014 http://stackoverflow.com/questions/19625073/how-to-run-clojure-from-matlab/22524112#22524112
 
parms = javaArray('java.lang.Class', 1);
parms(1) = java.lang.Class.forName('java.net.URL');
loaderClass = java.lang.Class.forName('java.net.URLClassLoader');
addUrlMeth = loaderClass.getDeclaredMethod('addURL', parms);
addUrlMeth.setAccessible(1);
 
sysClassLoader = java.lang.ClassLoader.getSystemClassLoader();
 
argArray = javaArray('java.lang.Object', 1);
jFile = java.io.File(file);
argArray(1) = jFile.toURI().toURL();
addUrlMeth.invoke(sysClassLoader, argArray);
 
if nargin > 1
    % load the class into Matlab's memory (a no-args public constructor is expected for classname)
    eval(classname);
end

The usage is super simple: instead of using javaaddpath to load a class-files folder (or ZIP/JAR file) into Matlab’s dynamic Java classpath, we’d use javaaddpathstatic to load the same input into the static classpath – in runtime.

As Andrew notes, there are important caveats to using the new javaaddpathstatic function. One of the important caveats is that the class needs to be loaded before Matlab gets around to “discover” that it does not exist, because from then on even if we load the class into the static classpath, Matlab will report the class as missing. Therefore, we need to call javaaddpathstatic at the very top of the program (possibly even in our startup.m file). Moreover, if we directly call the class anywhere in the same m-file as our call to javaaddpathstatic, it will fail. The reason is that Matlab’s built-in interpreter processes the direct Java call when it pre-parses the file, before the javaaddpathstatic had a chance to run:

function this_Will_Fail()
   % mylib.jar contains the com.app.MyClass class
   javaaddpathstatic([pwd '/mylib.jar']);
   obj = com.app.MyClass;  % this will fail!

Instead, we need to prevent the interpreter from processing the class before run-time, by loading the class at run-time:

function this_Should_Work()
   % mylib.jar contains the com.app.MyClass class
   javaaddpathstatic([pwd '/mylib.jar']);
   obj = javaObject('com.app.MyClass');  % this should work ok

At least in one case, for a recent client project that required deploying a compiled Matlab application that used JDBC to connect to a database (and therefore relied on the static classpath), this saved the day for me.

I’m sad I wasn’t aware of this years ago, but now that I have posted it, we have less of an excuse to complain about static classpath files in Matlab. Hopefully MathWorks will incorporate this idea into Matlab so that we can more easily load classes into the static classpath in runtime, and remove the need for ugly deployment hacks (and yes, this includes MathWorks’ own toolboxes), hopefully removing the caveats listed above along the way.

Analyzing loaded Java classes in run-time

A related Matlab function, whereisjavaclassloadingfrom, was posted by Andrew Janke to report information about loaded classes, including whether they were loaded by the static (=standard Java) classloader, or the dynamic (=MathWorks’ custom) classloader, and from which location. For example:

% This is loaded into the static classpath as part of Matlab's standard classpath.txt file:
>> whereisjavaclassloadingfrom('java.util.HashMap')
Version: 8.6.0.232648 (R2015b)
Class:       java.util.HashMap
ClassLoader: sun.misc.Launcher$AppClassLoader@69365360
URL:         jar:file:/C:/Program%20Files/Matlab/R2015b/sys/java/jre/win64/jre/lib/rt.jar!/java/util/HashMap.class
 
% This is loaded into the dynamic classpath using MathWorks' custom classloader
>> javaaddpath('C:\Yair\Work\MultiClassTableModel')
>> whereisjavaclassloadingfrom('MultiClassTableModel')
Version: 8.6.0.232648 (R2015b)
Class:       MultiClassTableModel
ClassLoader: com.mathworks.jmi.CustomURLClassLoader@5270d6ad
URL:         file:/C:/Yair/Work/MultiClassTableModel/MultiClassTableModel.class
 
% This is an example of a class that is not loaded
>> whereisjavaclassloadingfrom('no.such.Class')
Error using javaArray
No class no.such.Class can be located on the Java class path
Error in whereisjavaclassloadingfrom (line 25)

Parting words

How risky is all this in terms of future compatibility? well, I’m not a prophet, but I’d say that using these hacks is pretty risky in this regard. Matlab’s custom classloader is way-deep in undocumented territory. This does not mean that the functionality will change tomorrow, but don’t be surprised if and when it does.

Have you ever used any additional undocumented hack relating to using the Java classpaths in Matlab? If so then please post a comment below.

Users who have endured my post this far, should probably also read my related article about Java class access pitfalls.

1. Matlab-Java interface using a static control The switchyard function design pattern can be very useful when setting Matlab callbacks to Java GUI controls. This article explains why and how....
2. Java class access pitfalls There are several potential pitfalls when accessing Matlab classes from Matlab. ...
3. Matlab callbacks for Java events Events raised in Java code can be caught and handled in Matlab callback functions - this article explains how...
4. JMI – Java-to-Matlab Interface JMI enables calling Matlab functions from within Java. This article explains JMI's core functionality....