Thursday, December 13, 2007

Java Native Interface (JNI) Tutorial - Hell on Stage

Intro

There are so many tutorials about using JNI to access C/C++ code from Java so I don't want to write the next ... but to share some tips & tricks with a very short overview.

Java and C together! Why?

  1. Implement time-critical code
  2. Access legacy code or code libraries from Java programs
  3. Need of platform-dependent features not supported in Java (not covered)
Calling C/C++ code from Java programs

Six steps to do this with Java Native Interface:

  1. Write Java code
    public class Sample{
    public native String stringMethod(String text);
    public static void main(String[] args) {
    System.loadLibrary("sample");
    Sample sample = new Sample();
    String text = sample.stringMethod("java");
    System.out.println("stringMethod: " + text)
    }
  2. Compile the Java code
    javac Sample.java
  3. Generate C/C++ header file
    Generate Sample.h from the java class file: $javah -jni Sample. The result uses a pointer to a table of function pointers (JNIEnv parameter) and a parameter refers to the current invoking Java object or this pointer (jobject) :
    #include
    extern "C" {
    JNIEXPORT jstring JNICALL Java_Sample_stringMethod
    (JNIEnv *, jobject, jstring);
    }

  4. Write C/C++ code
    Implement the methods from the generated header files:
    JNIEXPORT jstring JNICALL Java_Sample_stringMethod(JNIEnv *env, jobject obj, jstring string) {
    const char *str = env->GetStringUTFChars(string, 0);
    char cap[128];
    strcpy(cap, str);
    env->ReleaseStringUTFChars(string, str);
    return env->NewStringUTF(strupr(cap));
    }

  5. Create shared library file
    The naming convension for the output shared library is different depending on the OS and must be followed: sample.dll (Windows) or libsample.so (*NIX).
    $g++ -I /usr/lib/jvm/java-1.5.0-sun/include sample.cpp -L ~/resources/verimatrix -shared -o libsample.so -lcrypto
  6. Run the Java application
    java Sample

Access shared libraries

Prerequisites
Usually you have some shared libraries that provide some functionality. For example two dynamic linux shared libraries (libvpr.so and libringdll.so).
Problems
  1. Difference between the provided method names in the libraries and the naming convension used by JNI - you cannot invoke one of the methods directly unless it is designed with JNI in mind. JNI requires some prefixes as shown above with the method name "Java_Sample_stringMethod"
  2. Not all JNI data types can be mapped to C/C++ data types and vice versa:
    jint == int, jbyte == jbyte
    jbyteArray != byte *, jstring != string, jclass != class
  3. Freeing Native Resources - some resources cannot be freed by the GC like strings and global references.

Access C/C++ methods provided by the libraries
The solution that handles the above problems is to implement a proxy library that follows the JNI naming convention and proxies all method calls to the functional shared library:

  1. Implement Java code with all required native methods:
    private native boolean VprEncryptMovie(EncryptMovieSettings encryptionSettings);
  2. Design C/C++ shared library that implement the interface of the methods as proxy:
    jboolean Java_jni_VerimatrixClient_VprEncryptMovie(EncryptMovieSettings jobject);
  3. Implement data conversion from the JNI data types to standard C/C++ data types
  4. Proxy all method calls to the corresponding methods in functional libraries
  5. Implement exception handling
    jclass newExc=env->FindClass("java/lang/IllegalArgumentException");
    env->ThrowNew(newExc, "thrown from C code");

Tips and tricks

  1. Use Apache Harmony VM for trouble shooting or the patched version as described here - it generates a JNI stack trace instead of just crashing the VM as JDK5 or JDM6 eg:

    SIGSEGV in VM code.
    Stack trace:
    0: memcpy (??:-1)
    1: VprRpcEncryptMovieWithKey(rpc_handle_s_t*, void*, unsigned char*, long, long, long, long, long, long, long, long, long, long, long, long, long, long, long, long, long, unsigned char*, long, long, long*, long*) (??:-1)
    2: VprEncryptMovieWithKey (??:-1)
    3: Java_com_minerva_edc_vig_verimatrix_vcas_VprClient_VprEncryptMovieWithKey (/home/emo/perforce/3ML/SB/VideoIngest/src/com/minerva/edc/vig/verimatrix/proxy/VprProxy.cpp:100)
    4: 0xA591A50B
    5: com/minerva/edc/vig/verimatrix/vcas/VprClient.createEncryptSession(Lcom/minerva/edc/vig/verimatrix/vcas/model/VprServerRegisterResponse;Lcom/minerva/edc/vig/verimatrix/vcas/model/EncryptMovieSettings;Lcom/minerva/edc/vig/verimatrix/vcas/model/MovieInfo;Lcom/minerva/edc/vig/core/data/DTO;)J (VprClient.java:73)
    ...
    23: _start (/usr/src/packages/BUILD/glibc-2.3/csu/../sysdeps/i386/elf/start.S:105)
  2. Eclipse plugin for C/C++ development - CDT
  3. Do not store local variables in the C++ classes if more that one simultaneous client to the proxy is planned - only one instance of this class from the library is loaded by the JVM.
  4. You can implement a Java class to pass multiple parameters to a JNI function and extract them with reflection but remmember that every reflection call has big performance hit. Also if this input parameter is "converted" into a C/C++ structure or class, the constructed objects cannot be cached (see the previos tip). If the parametes are not so many - pass them all.
  5. A Java class can be returned as out parameter by reference. It is better to create the instance in the Java and just fill the properties in the C++ proxy. This eliminates the need to call the constructor of the response object and to hardcode its package and class name in the proxy class. Only the method names are required.

  6. A setter method can be invoked in such a way:

    // Create the encryption settings class
    jclass vprResponseClass = env->GetObjectClass(vprResponseObj);
    // Allocates memory for an array of one long long output parameter
    jvalue* args = (jvalue*)malloc(sizeof(jlong)); way:
    // Create the encryption settings class
    jclass vprResponseClass = env->GetObjectClass(vprResponseObj);
    // Allocates memory for an array of one long long output parameter
    jvalue* args = (jvalue*)malloc(sizeof(jlong));

    args[0].j=input_file_size;
    jmethodID setInputFileSizeID =env->GetMethodID(vprResponseClass,"setInputFileSize", "(J)V");
    env->CallLongMethodA(vprResponseObj, setInputFileSizeID, args);
    // Cleanup
    free((jvalue*) args);

  7. A thrown exception in a C++ method do not stops its execution so all allocated resources in the method can be freed first before the return clause.

  8. Passing Invalid Arguments to JNI Functions - most often JVM crashes. Use JVM option -Xcheck:jni to detect errors like passing passing NULL or (jobject)0xFFFFFFFF in place of reference. This option degrates the performance.

Troubleshooting

  • java.lang.UnsatisfiedLinkError: no sample in java.library.path
    The library name in Java (System.loadLibrary)is wrong - chech the prefix or extension
    Could not find the library - set the directory locations as a JVM parameter: -Djava.library.path=/home/lib:/opt/lib/

  • java.lang.UnsatisfiedLinkError: /libverimatrixproxy.so: libvpr.so: cannot open shared object file: No such file or directory
    A library dependency cannot be found. Add the location of the missing library e.g. ~/lib/ in "/etc/ld.so.conf". Then reconfigure linker bindings $ldconfig

  • java.lang.UnsatisfiedLinkError: /libverimatrix.so: /libverimatrix.so: undefined symbol: VprDestroyContext
    Check the shared library for all undefined symbols in the list of its dependences
    $ldd -d ~/lib/libverimatrix.so
    Place the missing library that provides those methods, structres, enumerations or ... in the libraries dependency search path.

  • java.lang.UnsatisfiedLinkError: /libverimatrix.so: Can't load IA 32-bit .so on a IA 32-bit platform
    Bug in JDK5 - wrong error message. Use another VM (JDK6, Harmony) to get the correct stack trace.

  • [Too many errors, abort]
    Infinite error message dump:
    Many times: [error occurred during error reporting, step 270, id 0xb]
    Then infinite: [Too many errors, abort]
    This occurs when the JDK detects an error in called C++ code but does not crash imediate. See the first tip about the Apache Harmony JVM.

  • error: redefinition of ‘struct EncryptMovieSettings’
    This type of C++ compile error may happpen if a header file is called or included more than once. Try to precede and end it like that to avoid that:
    #ifndef __yourheader_h
    #define __yourheader_h (1)
    // Put here the body of your ".h" file including the class
    #endif

    Debug JNI

    Integrated Debugger for JNI Environments
    Eclipse version 3.2 (not a newer) can be used to debug both the Java and C++ code of JNI. Apache Harmony JVM provides interface for agent which could manage and handle events in debug session.
    Synchronized Sollution
    Solution with two debuggers that attach to a running Java or C++ process and cooperate -  a Java debugger eg. Eclipse and a C/C++ debugger eg. Insight - GUI frondend to GDB

    References

    Java Native Interface 1999.pdf
    Java programming with JNI.pdf
  • 5 comments:

    Anil Philip said...

    typos

    Sample.java:8: ';' expected
    }
    ^
    Sample.java:9: '}' expected
    ^
    2 errors

    Anil Philip said...

    $ g++ -I /usr/lib/jvm/java-1.5.0-sun/include learn_Sample.cpp -L . -shared -o libsample.so
    In file included from learn_Sample.h:2,
    from learn_Sample.cpp:1:
    /usr/lib/jvm/java-1.5.0-sun/include/jni.h:27: fatal error: jni_md.h: No such file or directory

    Anonymous said...

    -I/usr/lib64/jvm/java/include/linux

    Anonymous said...

    Sorry but this tutorial is terrible.
    Serious noob errors and ambiguities.
    Please revise.

    Divya Besant said...

    Very Nice Thoughts. Thanks For Sharing with us. I got More information about Java from Besant Technologies. If anyone wants to get Java Training in Chennai visit Besant Technologies.