转自: https://dzone.com/articles/understanding-sunmiscunsafe

The biggest competitor to the Java virtual machine might be Microsoft's CLR that hosts languages such as C#. The CLR allows to write unsafe code as an entry gate for low level programming, something that is hard to achieve on the JVM. If you need such advanced functionality in Java, you might be forced to use the JNIwhich requires you to know some C and will quickly lead to code that is tightly coupled to a specific platform. With sun.misc.Unsafe, there is however another alternative to low-level programming on the Java plarform using a Java API, even though this alternative is discouraged. Nevertheless, several applications rely on sun.misc.Unsafe such for example objenesis and therewith all libraries that build on the latter such for example kryo which is again used in for example Twitter's Storm. Therefore, it is time to have a look, especially since the functionality of sun.misc.Unsafe is considered to become part of Java's public API in Java 9.

Getting hold of an instance of sun.misc.Unsafe

The sun.misc.Unsafe class is intended to be only used by core Java classes which is why its authors made its only constructor private and only added an equally private singleton instance. The public getter for this instances performs a security check in order to avoid its public use:

public static Unsafe getUnsafe() {
Class cc = sun.reflect.Reflection.getCallerClass(2);
if (cc.getClassLoader() != null)
throw new SecurityException("Unsafe");
return theUnsafe;
}

This method first looks up the calling Class from the current thread’s method stack. This lookup is implemented by another internal class named sun.reflection.Reflection which is basically browsing down the given number of call stack frames and then returns this method’s defining class. This security check is however likely to change in future version. When browsing the stack, the first found class (index 0) will obviously be the Reflection class itself, and the second (index 1) class will be the Unsafe class such that index 2 will hold your application class that was calling Unsafe#getUnsafe().

This looked-up class is then checked for its ClassLoader where a null reference is used to represent the bootstrap class loader on a HotSpot virtual machine. (This is documented in Class#getClassLoader() where it says that “some implementations may use null to represent the bootstrap class loader”.) Since no non-core Java class is normally ever loaded with this class loader, you will therefore never be able to call this method directly but receive a thrown SecurityException as an answer. (Technically, you could force the VM to load your application classes using the bootstrap class loader by adding it to the –Xbootclasspath, but this would require some setup outside of your application code which you might want to avoid.) Thus, the following test will succeed:

@Test(expected = SecurityException.class)
public void testSingletonGetter() throws Exception {
Unsafe.getUnsafe();
}

However, the security check is poorly designed and should be seen as a warning against the singleton anti-pattern. As long as the use of reflection is not prohibited (which is hard since it is so widely used in many frameworks), you can always get hold of an instance by inspecting the private members of the class. From theUnsafe class's source code, you can learn that the singleton instance is stored in a private static field calledtheUnsafe. This is at least true for the HotSpot virtual machine. Unfortunately for us, other virtual machine implementations sometimes use other names for this field. Android’s Unsafe class is for example storing its singleton instance in a field called THE_ONE. This makes it hard to provide a “compatible” way of receiving the instance. However, since we already left the save territory of compatibility by using the Unsafe class, we should not worry about this more than we should worry about using the class at all. For getting hold of the singleton instance, you simply read the singleton field's value:

Field theUnsafe = Unsafe.class.getDeclaredField("theUnsafe");
theUnsafe.setAccessible(true);
Unsafe unsafe = (Unsafe) theUnsafe.get(null);

Alternatively, you can invoke the private instructor. I do personally prefer this way since it works for example with Android while extracting the field does not:

Constructor<Unsafe> unsafeConstructor = Unsafe.class.getDeclaredConstructor();
unsafeConstructor.setAccessible(true);
Unsafe unsafe = unsafeConstructor.newInstance();

The price you pay for this minor compatibility advantage is a minimal amount of heap space. The security checks performed when using reflection on fields or constructors are however similar.

Create an Instance of a Class Without Calling a Constructor

The first time I made use of the Unsafe class was for creating an instance of a class without calling any of the class's constructors. I needed to proxy an entire class which only had a rather noisy constructor but I only wanted to delegate all method invocations to a real instance which I did however not know at the time of construction. Creating a subclass was easy and if the class had been represented by an interface, creating a proxy would have been a straight-forward task. With the expensive constructor, I was however stuck. By using the Unsafe class, I was however able to work my way around it. Consider a class with an artificially expensive constructor:

class ClassWithExpensiveConstructor {

  private final int value;

  private ClassWithExpensiveConstructor() {
value = doExpensiveLookup();
} private int doExpensiveLookup() {
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
return 1;
} public int getValue() {
return value;
}
}

Using the Unsafe, we can create an instance of ClassWithExpensiveConstructor (or any of its subclasses) without having to invoke the above constructor, simply by allocating an instance directly on the heap:

@Test
public void testObjectCreation() throws Exception {
ClassWithExpensiveConstructor instance = (ClassWithExpensiveConstructor)
unsafe.allocateInstance(ClassWithExpensiveConstructor.class);
assertEquals(0, instance.getValue());
}

Note that final field remained uninitialized by the constructor but is set with its type's default value. Other than that, the constructed instance behaves like a normal Java object. It will for example be garbage collected when it becomes unreachable.

The Java run time itself creates objects without calling a constructor when for example creating objects for deserialization. Therefore, the ReflectionFactory offers even more access to individual object creation:

@Test
public void testReflectionFactory() throws Exception {
@SuppressWarnings("unchecked")
Constructor<ClassWithExpensiveConstructor> silentConstructor = ReflectionFactory.getReflectionFactory()
.newConstructorForSerialization(ClassWithExpensiveConstructor.class, Object.class.getConstructor());
silentConstructor.setAccessible(true);
assertEquals(10, silentConstructor.newInstance().getValue());
}

Note that the ReflectionFactory class only requires a RuntimePermission called reflectionFactoryAccess for receiving its singleton instance and no reflection is therefore required here. The received instance of ReflectionFactory allows you to define any constructor to become a constructor for the given type. In the example above, I used the default constructor of java.lang.Object for this purpose. You can however use any constructor:

class OtherClass {

  private final int value;
private final int unknownValue; private OtherClass() {
System.out.println("test");
this.value = 10;
this.unknownValue = 20;
}
} @Test
public void testStrangeReflectionFactory() throws Exception {
@SuppressWarnings("unchecked")
Constructor<ClassWithExpensiveConstructor> silentConstructor = ReflectionFactory.getReflectionFactory()
.newConstructorForSerialization(ClassWithExpensiveConstructor.class,
OtherClass.class.getDeclaredConstructor());
silentConstructor.setAccessible(true);
ClassWithExpensiveConstructor instance = silentConstructor.newInstance();
assertEquals(10, instance.getValue());
assertEquals(ClassWithExpensiveConstructor.class, instance.getClass());
assertEquals(Object.class, instance.getClass().getSuperclass());
}

Note that value was set in this constructor even though the constructor of a completely different class was invoked. Non-existing fields in the target class are however ignored as also obvious from the above example. Note that OtherClass does not become part of the constructed instances type hierarchy, the OtherClass's constructor is simply borrowed for the "serialized" type.

Not mentioned in this blog entry are other methods such as Unsafe#defineClassUnsafe#defineAnonymousClass orUnsafe#ensureClassInitialized. Similar functionality is however also defined in the public API's ClassLoader.

Native Memory Allocation

Did you ever want to allocate an array in Java that should have had more than Integer.MAX_VALUE entries? Probably not because this is not a common task, but if you once need this functionality, it is possible. You can create such an array by allocating native memory. Native memory allocation is used by for example direct byte buffers that are offered in Java's NIO packages. Other than heap memory, native memory is not part of the heap area and can be used non-exclusively for example for communicating with other processes. As a result, Java's heap space is in competition with the native space: the more memory you assign to the JVM, the less native memory is left.

Let us look at an example for using native (off-heap) memory in Java with creating the mentioned oversized array:

class DirectIntArray {

  private final static long INT_SIZE_IN_BYTES = 4;

  private final long startIndex;

  public DirectIntArray(long size) {
startIndex = unsafe.allocateMemory(size * INT_SIZE_IN_BYTES);
unsafe.setMemory(startIndex, size * INT_SIZE_IN_BYTES, (byte) 0);
}
} public void setValue(long index, int value) {
unsafe.putInt(index(index), value);
} public int getValue(long index) {
return unsafe.getInt(index(index));
} private long index(long offset) {
return startIndex + offset * INT_SIZE_IN_BYTES;
} public void destroy() {
unsafe.freeMemory(startIndex);
}
} @Test
public void testDirectIntArray() throws Exception {
long maximum = Integer.MAX_VALUE + 1L;
DirectIntArray directIntArray = new DirectIntArray(maximum);
directIntArray.setValue(0L, 10);
directIntArray.setValue(maximum, 20);
assertEquals(10, directIntArray.getValue(0L));
assertEquals(20, directIntArray.getValue(maximum));
directIntArray.destroy();
}

First, make sure that your machine has sufficient memory for running this example! You need at least(2147483647 + 1) * 4 byte = 8192 MB of native memory for running the code. If you have worked with other programming languages as for example C, direct memory allocation is something you do every day. By calling Unsafe#allocateMemory(long), the virtual machine allocates the requested amount of native memory for you. After that, it will be your responsibility to handle this memory correctly.

The amount of memory that is required for storing a specific value is dependent on the type's size. In the above example, I used an int type which represents a 32-bit integer. Consequently a single int value consumes 4 byte. For primitive types, size is well-documented. It is however more complex to compute the size of object types since they are dependent on the number of non-static fields that are declared anywhere in the type hierarchy. The most canonical way of computing an object's size is using the Instrumented class from Java's attach API which offers a dedicated method for this purpose called getObjectSize. I will however evaluate another (hacky) way of dealing with objects in the end of this section.

Be aware that directly allocated memory is always native memory and therefore not garbage collected. You therefore have to free memory explicitly as demonstrated in the above example by a call toUnsafe#freeMemory(long). Otherwise you reserved some memory that can never be used for something else as long as the JVM instance is running what is a memory leak and a common problem in non-garbage collected languages. Alternatively, you can also directly reallocate memory at a certain address by callingUnsafe#reallocateMemory(long, long) where the second argument describes the new amount of bytes to be reserved by the JVM at the given address.

Also, note that the directly allocated memory is not initialized with a certain value. In general, you will find garbage from old usages of this memory area such that you have to explicitly initialize your allocated memory if you require a default value. This is something that is normally done for you when you let the Java run time allocate the memory for you. In the above example, the entire area is overriden with zeros with help of theUnsafe#setMemory method.

When using directly allocated memory, the JVM will neither do range checks for you. It is therefore possible to corrupt your memory as this example shows:

@Test
public void testMallaciousAllocation() throws Exception {
long address = unsafe.allocateMemory(2L * 4);
unsafe.setMemory(address, 8L, (byte) 0);
assertEquals(0, unsafe.getInt(address));
assertEquals(0, unsafe.getInt(address + 4));
unsafe.putInt(address + 1, 0xffffffff);
assertEquals(0xffffff00, unsafe.getInt(address));
assertEquals(0x000000ff, unsafe.getInt(address + 4));
}

Note that we wrote a value into the space that was each partly reserved for the first and for the second number. This picture might clear things up. Be aware that the values in the memory run from the "right to the left" (but this might be machine dependent).

The first row shows the initial state after writing zeros to the entire allocated native memory area. Then we override 4 byte with an offset of a single byte using 32 ones. The last row shows the result after this writing operation.

Finally, we want to write an entire object into native memory. As mentioned above, this is a difficult task since we first need to compute the size of the object in order to know the amount of size we need to reserve. The Unsafe class does however not offer such functionality. At least not directly since we can at least use theUnsafe class to find the offset of an instance's field which is used by the JVM when itself allocates objects on the heap. This allows us to find the approximate size of an object:

public long sizeOf(Class<?> clazz)
long maximumOffset = 0;
do {
for (Field f : clazz.getDeclaredFields()) {
if (!Modifier.isStatic(f.getModifiers())) {
maximumOffset = Math.max(maximumOffset, unsafe.objectFieldOffset(f));
}
}
} while ((clazz = clazz.getSuperclass()) != null);
return maximumOffset + 8;
}

This might at first look cryptic, but there is no big secret behind this code. We simply iterate over all non-static fields that are declared in the class itself or in any of its super classes. We do not have to worry about interfaces since those cannot define fields and will therefore never alter an object's memory layout. Any of these fields has an offset which represents the first byte that is occupied by this field's value when the JVM stores an instance of this type in memory, relative to a first byte that is used for this object. We simply have to find the maximum offset in order to find the space that is required for all fields but the last field. Since a field will never occupy more than 64 bit (8 byte) for a long or double value or for an object reference when run on a 64 bit machine, we have at least found an upper bound for the space that is used to store an object. Therefore, we simply add these 8 byte to the maximum index and we will not run into danger of having reserved to little space. This idea is of course wasting some byte and a better algorithm should be used for production code.

In this context, it is best to think of a class definition as a form of heterogeneous array. Note that the minimum field offset is not 0 but a positive value. The first few byte contain meta information. The graphic below visualizes this principle for an example object with an int and a long field where both fields have an offset. Note that we do not normally write meta information when writing a copy of an object into native memory so we could further reduce the amount of used native memoy. Also note that this memory layout might be highly dependent on an implementation of the Java virtual machine.

With this overly careful estimate, we can now implement some stub methods for writing shallow copies of objects directly into native memory. Note that native memory does not really know the concept of an object. We are basically just setting a given amount of byte to values that reflect an object's current values. As long as we remember the memory layout for this type, these byte contain however enough information to reconstruct this object.

public void place(Object o, long address) throws Exception {
Class clazz = o.getClass();
do {
for (Field f : clazz.getDeclaredFields()) {
if (!Modifier.isStatic(f.getModifiers())) {
long offset = unsafe.objectFieldOffset(f);
if (f.getType() == long.class) {
unsafe.putLong(address + offset, unsafe.getLong(o, offset));
} else if (f.getType() == int.class) {
unsafe.putInt(address + offset, unsafe.getInt(o, offset));
} else {
throw new UnsupportedOperationException();
}
}
}
} while ((clazz = clazz.getSuperclass()) != null);
} public Object read(Class clazz, long address) throws Exception {
Object instance = unsafe.allocateInstance(clazz);
do {
for (Field f : clazz.getDeclaredFields()) {
if (!Modifier.isStatic(f.getModifiers())) {
long offset = unsafe.objectFieldOffset(f);
if (f.getType() == long.class) {
unsafe.putLong(instance, offset, unsafe.getLong(address + offset));
} else if (f.getType() == int.class) {
unsafe.putLong(instance, offset, unsafe.getInt(address + offset));
} else {
throw new UnsupportedOperationException();
}
}
}
} while ((clazz = clazz.getSuperclass()) != null);
return instance;
} @Test
public void testObjectAllocation() throws Exception {
long containerSize = sizeOf(Container.class);
long address = unsafe.allocateMemory(containerSize);
Container c1 = new Container(10, 1000L);
Container c2 = new Container(5, -10L);
place(c1, address);
place(c2, address + containerSize);
Container newC1 = (Container) read(Container.class, address);
Container newC2 = (Container) read(Container.class, address + containerSize);
assertEquals(c1, newC1);
assertEquals(c2, newC2);
}

Note that these stub methods for writing and reading objects in native memory only support int and long field values. Of course, Unsafe supports all primitive values and can even write values without hitting thread-local caches by using the volatile forms of the methods. The stubs were only used to keep the examples concise. Be aware that these "instances" would never get garbage collected since their memory was allocated directly. (But maybe this is what you want.) Also, be careful when precalculating size since an object's memory layout might be VM dependent and also alter if a 64-bit machine runs your code compared to a 32-bit machine. The offsets might even change between JVM restarts.

For reading and writing primitives or object references, Unsafe provides the following type-dependent methods:

  • getXXX(Object target, long offset): Will read a value of type XXX from target's address at the specified offset.
  • putXXX(Object target, long offset, XXX value): Will place value at target's address at the specified offset.
  • getXXXVolatile(Object target, long offset): Will read a value of type XXX from target's address at the specified offset and not hit any thread local caches.
  • putXXXVolatile(Object target, long offset, XXX value): Will place value at target's address at the specified offset and not hit any thread local caches.
  • putOrderedXXX(Object target, long offset, XXX value): Will place value at target's address at the specified offet and might not hit all thread local caches.
  • putXXX(long address, XXX value): Will place the specified value of type XXX directly at the specified address.
  • getXXX(long address): Will read a value of type XXX from the specified address.
  • compareAndSwapXXX(Object target, long offset, long expectedValue, long value): Will atomicly read a value of type XXX from target's address at the specified offset and set the given value if the current value at this offset equals the expected value.

Be aware that you are copying references when writing or reading object copies in native memory by using the getObject(Object, long) method family. You are therefore only creating shallow copies of instances when applying the above method. You could however always read object sizes and offsets recursively and create deep copies. Pay however attention for cyclic object references which would cause infinitive loops when applying this principle carelessly.

Not mentioned here are existing utilities in the Unsafe class that allow manipulation of static field values sucht as staticFieldOffset and for handling array types. Finally, both methods named Unsafe#copyMemory allow to instruct a direct copy of memory, either relative to a specific object offset or at an absolute address as the following example shows:

@Test
public void testCopy() throws Exception {
long address = unsafe.allocateMemory(4L);
unsafe.putInt(address, 100);
long otherAddress = unsafe.allocateMemory(4L);
unsafe.copyMemory(address, otherAddress, 4L);
assertEquals(100, unsafe.getInt(otherAddress));
}

Throwing Checked Exceptions Without Declaration

There are some other interesting methods to find in Unsafe. Did you ever want to throw a specific exception to be handled in a lower layer but you high layer interface type did not declare this checked exception?Unsafe#throwException allows to do so:

@Test(expected = Exception.class)
public void testThrowChecked() throws Exception {
throwChecked();
} public void throwChecked() {
unsafe.throwException(new Exception());
}

Native Concurrency

The park and unpark methods allow you to pause a thread for a certain amount of time and to resume it:

@Test
public void testPark() throws Exception {
final boolean[] run = new boolean[1];
Thread thread = new Thread() {
@Override
public void run() {
unsafe.park(true, 100000L);
run[0] = true;
}
};
thread.start();
unsafe.unpark(thread);
thread.join(100L);
assertTrue(run[0]);
}

Also, monitors can be acquired directly by using Unsafe using monitorEnter(Object)monitorExit(Object) andtryMonitorEnter(Object)
A file containing all the examples of this blog entry is available as a gist.

Understanding sun.misc.Unsafe的更多相关文章

  1. sun.misc.Unsafe的理解

    以下sun.misc.Unsafe源码和demo基于jdk1.7: 最近在看J.U.C里的源码,很多都用到了sun.misc.Unsafe这个类,一知半解,看起来总感觉有点不尽兴,所以打算对Unsaf ...

  2. Java--如何使用sun.misc.Unsafe完成compareAndSwapObject原子操作

    package com; import sun.misc.Unsafe; import java.lang.reflect.Field; /** * Created by yangyu on 16/1 ...

  3. Java sun.misc.Unsafe类的学习笔记

    Java未开源的Unsafe类 Unsafe类可以为我们提供高效并且线程安全方式操作变量,直接和内存数据打交道. 获取Unsafe实体的方法 private static Unsafe getUnsa ...

  4. java对象的内存布局(二):利用sun.misc.Unsafe获取类字段的偏移地址和读取字段的值

    在上一篇文章中.我们列出了计算java对象大小的几个结论以及jol工具的使用,jol工具的源代码有兴趣的能够去看下.如今我们利用JDK中的sun.misc.Unsafe来计算下字段的偏移地址,一则验证 ...

  5. sun.misc.Unsafe 详解

    原文地址 译者:许巧辉 校对:梁海舰 Java是一门安全的编程语言,防止程序员犯很多愚蠢的错误,它们大部分是基于内存管理的.但是,有一种方式可以有意的执行一些不安全.容易犯错的操作,那就是使用Unsa ...

  6. JDK 1.8 sun.misc.Unsafe类CAS底层实现

    在java.util.concurrent包下面的很多类为了追求性能都采用了sun.misc.Unsafe类中的CAS操作,从而避免使用synchronized等加锁方式带来性能上的不足. 在sun. ...

  7. Java中的sun.misc.Unsafe包

    chronicle项目:https://github.com/peter-lawrey/Java-Chronicle 这个项目是利用mmap机制来实现高效的读写数据,号称每秒写入5到20百万条数据. ...

  8. Java sun.misc.unsafe类

    Java是一个安全的开发工具,它阻止开发人员犯很多低级的错误,而大部份的错误都是基于内存管理方面的.如果你想搞破坏,可以使用Unsafe这个类.这个类是属于sun.*API中的类,并且它不是J2SE中 ...

  9. Java Magic. Part 4: sun.misc.Unsafe

    Java Magic. Part 4: sun.misc.Unsafe @(Base)[JDK, Unsafe, magic, 黑魔法] 转载请写明:原文地址 系列文章: -Java Magic. P ...

随机推荐

  1. Vue语法

    第一个Vue示例: <!DOCTYPE html> <html lang="en"> <head> <meta charset=" ...

  2. 【CentOS 6.5】 安装VNCServer及配置,注销处理

    如果没有安装VNCServer,只有在机器上登录进桌面后才可以通过VNC连接,否则连不上... 安装: yum install tigervnc-server   运行并设置密码: vncserver ...

  3. zedgraph绘图(修改)

    转自原文 zedgraph绘图(修改) 首先先下载 zedgraph.dll和zedgraph.web.DLL两个文件 添加项目并引用 首先添加一个用户控件 WebUserDrawGrap.ascx ...

  4. leetcode498

    public class Solution { public int[] FindDiagonalOrder(int[,] matrix) { ); ); + col - ; var ary = ne ...

  5. ubuntu 下安装nanomsg和nnpy

    nanomsg nanomsg是ZeroMQ作者用C语言重写的一个Socket库,其用法和模式和ZeroMQ差不多,但是具有更好的性能和更完善的接口. 首先下载源码 wget https://gith ...

  6. 前端开发之JavaScript基础篇三

    主要内容: 1.创建对象的几种方式 2.JavaScript内置对象 3.JavaScript错误--Throw.Try 和 Catch 4.JavaScript 表单验证 一.创建对象的几种方式 1 ...

  7. 99. Recover Binary Search Tree (Tree; DFS)

    Two elements of a binary search tree (BST) are swapped by mistake. Recover the tree without changing ...

  8. Tp3.1 文件上传到七牛云

    TP3.1 中不支持Composer 就无法用composer 安装 下载历史的SDK https://github.com/qiniu/php-sdk/releases/tag/v7.0.8 下载下 ...

  9. mybatis+oracle如何批量执行多条update

    接口 public void setStatus(List<Columns> columnsList); mapping xmlmapping 中使用foreach,关于标签的使用,资料非 ...

  10. -other linker flags - 详解

    • 值:-objC,-all_load,-force_load

 • -objC: 在iOS 中,使用-all_load时,如果静态库中有类别时会出问题,使用其他两个值则不会有问题.

 • -al ...