NEO VM原理及其实现(转载)

NEO Vm原理及其实现

简介及与evm主要区别

neo vm和evm类似。底层都实现了一套opcode以及对应的执行器，opcode设计差距蛮大的，总体上来说evm的更加简洁，neo vm的功能更加丰富强大。

1. neo底层原生支持类型系统，所有栈上的内容都是有类型的，而在evm中所有的栈上内容都是无类型的，依赖于运行时转换。实质上在于空间和时间的取舍。

2. neo在opcode级别支持类型及其操作，在内部函数调用方面evm通过jumpdest解决，neo vm的jump类指令则仅用于循环，函数调用则通过call和systemcall解决（这样很好的支持了调用函数栈）。总的来说neo vm一些高级操作可以下沉到code级别执行，方便了代码的转换。而evm则更加考验编译器的能力。

3. 隔离执行器和外部对象的具体实现，vm提供了一些外部接口用于执行一些和外部对象和存储相关的操作。eth的外部对象和指令揉合在一起。

4. 理论上来说eth也可以支持多种语言，实际上来说solidity一家独大，其中未尝没有eth的opcode转换难度大的原因，而neo vm的opcode则相对友好，支持多种语言开发。当然evm也并非没有好处，solidity支持内联编程，如果熟悉evm的opcode能写出效率非常高的合约。neo在这方面尚未提供支持。

5. neo vm支持debug，支持step into，step over，break point operation。

如需了解evm的详细实现可以参考另一篇博文 https://my.oschina.net/hunjixin/blog/1805306

代码结构

.
├── ExecutionContext.cs   //执行上下文 引擎，代码，断点
├── ExecutionEngine.cs    //执行引擎
├── Helper.cs             //编码及long型
├── ICrypto.cs            //加密解密
├── IInteropInterface.cs  //外部对象相关
├── InteropService.cs     //存储相关
├── IScriptContainer.cs   //脚本
├── IScriptTable.cs       //脚本读取
├── neo-vm.csproj
├── OpCode.cs             //操作码
├── RandomAccessStack.cs  //快速访问栈
├── ScriptBuilder.cs      //脚本构建
├── StackItem.cs          //栈
├── Types                 //类型
│   ├── Array.cs          //数组
│   ├── Boolean.cs        //bool
│   ├── ByteArray.cs      //比特数组
│   ├── Integer.cs        //整数
│   ├── InteropInterface.cs  //操作对象接口
│   ├── Map.cs            //映射
│   └── Struct.cs         //结构体
└── VMState.cs            //执行状态

类型系统

vm支持七种类型，分别是数组，bool，byte数组，整数，外部对象，映射，结构体。所有的结构体都是继承子StackItem，同时具有自己的数据字段。

stackItem

 public abstract class StackItem : IEquatable<StackItem>
    {
        public abstract bool Equals(StackItem other);

        public sealed override bool Equals(object obj)
        {
            if (obj == null) return false;
            if (obj == this) return true;
            if (obj is StackItem other)
                return Equals(other);
            return false;
        }

        public static StackItem FromInterface(IInteropInterface value)
        {
            return new InteropInterface(value);
        }

        public virtual BigInteger GetBigInteger()
        {
            return new BigInteger(GetByteArray());
        }

        public virtual bool GetBoolean()
        {
            return GetByteArray().Any(p => p != );
        }

        public abstract byte[] GetByteArray();

        public override int GetHashCode()
        {
            unchecked
            {
                int hash = ;
                foreach (byte element in GetByteArray())
                    hash = hash *  + element;
                return hash;
            }
        }

        public virtual string GetString()
        {
            return Encoding.UTF8.GetString(GetByteArray());
        }

        public static implicit operator StackItem(int value)
        {
            return (BigInteger)value;
        }

        public static implicit operator StackItem(uint value)
        {
            return (BigInteger)value;
        }

        public static implicit operator StackItem(long value)
        {
            return (BigInteger)value;
        }

        public static implicit operator StackItem(ulong value)
        {
            return (BigInteger)value;
        }

        public static implicit operator StackItem(BigInteger value)
        {
            return new Integer(value);
        }

        public static implicit operator StackItem(bool value)
        {
            return new Boolean(value);
        }

        public static implicit operator StackItem(byte[] value)
        {
            return new ByteArray(value);
        }

        public static implicit operator StackItem(StackItem[] value)
        {
            return new Array(value);
        }

        public static implicit operator StackItem(List<StackItem> value)
        {
            return new Array(value);
        }
    }

bool 为例子

 public class Boolean : StackItem
    {
        private static readonly byte[] TRUE = {  };
        private static readonly byte[] FALSE = new byte[];

        private bool value;

        public Boolean(bool value)
        {
            this.value = value;
        }

        public override bool Equals(StackItem other)
        {
            if (ReferenceEquals(this, other)) return true;
            if (ReferenceEquals(null, other)) return false;
            if (other is Boolean b) return value == b.value;
            byte[] bytes_other;
            try
            {
                bytes_other = other.GetByteArray();
            }
            catch (NotSupportedException)
            {
                return false;
            }
            return GetByteArray().SequenceEqual(bytes_other);
        }

        public override BigInteger GetBigInteger()
        {
            return value ? BigInteger.One : BigInteger.Zero;
        }

        public override bool GetBoolean()
        {
            return value;
        }

        public override byte[] GetByteArray()
        {
            return value ? TRUE : FALSE;
        }
    }

opecode

贴一些上来感受下

数值常量

PUSH2 = 0x52, // The number 2 is pushed onto the stack.
    PUSH3 = 0x53, // The number 3 is pushed onto the stack.
    PUSH4 = 0x54, // The number 4 is pushed onto the stack.
    PUSH5 = 0x55, // The number 5 is pushed onto the stack.

跳转

    JMP = 0x62,
    JMPIF = 0x63,
    JMPIFNOT = 0x64,

调用

    CALL = 0x65,
    RET = 0x66,
    APPCALL = 0x67,
    SYSCALL = 0x68,
    TAILCALL = 0x69,

栈操作，neo再着一块的相对比较丰富（这里并不是全部）

    DROP = 0x75, // Removes the top stack item.
    DUP = 0x76, // Duplicates the top stack item.
    PICK = 0x79, // The item n back in the stack is copied to the top.
    ROLL = 0x7A, // The item n back in the stack is moved to the top.
    SWAP = 0x7C, // The top two items on the stack are swapped.

运算，仅贴了些代表性的上来

    INC = 0x8B, // 1 is added to the input.
    SIGN = 0x8D,
    ABS = 0x90, // The input is made positive.
    NZ = 0x92, // Returns 0 if the input is 0. 1 otherwise.
    DIV = 0x96, // a is divided by b.
    MOD = 0x97, // Returns the remainder after dividing a by b.
    SHR = 0x99, // Shifts a right b bits, preserving sign.
    BOOLAND = 0x9A, // If both a and b are not 0, the output is 1. Otherwise 0.
    GTE = 0xA2, // Returns 1 if a is greater than or equal to b, 0 otherwise.
    MAX = 0xA4, // Returns the larger of a and b.
    WITHIN = 0xA5, // Returns 1 if x is within the specified range (left-inclusive), 0 otherwise.

加密验证

    SHA1 = 0xA7, // The input is hashed using SHA-1.
    SHA256 = 0xA8, // The input is hashed using SHA-256.
    HASH160 = 0xA9,
    HASH256 = 0xAA,
    CHECKSIG = 0xAC,
    VERIFY = 0xAD,
    CHECKMULTISIG = 0xAE,

数组，结构体及相关操作

    ARRAYSIZE = 0xC0,
    PACK = 0xC1,
    UNPACK = 0xC2,
    PICKITEM = 0xC3,
    SETITEM = 0xC4,
    NEWARRAY = 0xC5, //用作引用類型
    NEWSTRUCT = 0xC6, //用作值類型
    NEWMAP = 0xC7,
    APPEND = 0xC8,
    REVERSE = 0xC9,
    REMOVE = 0xCA,
    HASKEY = 0xCB,
    KEYS = 0xCC,
    VALUES = 0xCD,

异常

 THROW = 0xF0,
    THROWIFNOT = 0xF1

外部接口

脚本容器，保存当前执行脚本

 public interface IScriptContainer : IInteropInterface
    {
        byte[] GetMessage();
    }

合约脚本查找

    public interface IScriptTable
    {
        byte[] GetScript(byte[] script_hash);
    }

加密

    public interface ICrypto
    {
        byte[] Hash160(byte[] message);

        byte[] Hash256(byte[] message);

        bool VerifySignature(byte[] message, byte[] signature, byte[] pubkey);
    }

外部服务调用接口

 public class InteropService
    {
        private Dictionary<string, Func<ExecutionEngine, bool>> dictionary = new Dictionary<string, Func<ExecutionEngine, bool>>();

        public InteropService()
        {
            Register("System.ExecutionEngine.GetScriptContainer", GetScriptContainer);
            Register("System.ExecutionEngine.GetExecutingScriptHash", GetExecutingScriptHash);
            Register("System.ExecutionEngine.GetCallingScriptHash", GetCallingScriptHash);
            Register("System.ExecutionEngine.GetEntryScriptHash", GetEntryScriptHash);
        }

        protected void Register(string method, Func<ExecutionEngine, bool> handler)
        {
            dictionary[method] = handler;
        }

        internal bool Invoke(string method, ExecutionEngine engine)
        {
            if (!dictionary.TryGetValue(method, out Func<ExecutionEngine, bool> func)) return false;
            return func(engine);
        }

        private static bool GetScriptContainer(ExecutionEngine engine)
        {
            engine.EvaluationStack.Push(StackItem.FromInterface(engine.ScriptContainer));
            return true;
        }

        private static bool GetExecutingScriptHash(ExecutionEngine engine)
        {
            engine.EvaluationStack.Push(engine.CurrentContext.ScriptHash);
            return true;
        }

        private static bool GetCallingScriptHash(ExecutionEngine engine)
        {
            engine.EvaluationStack.Push(engine.CallingContext.ScriptHash);
            return true;
        }

        private static bool GetEntryScriptHash(ExecutionEngine engine)
        {
            engine.EvaluationStack.Push(engine.EntryContext.ScriptHash);
            return true;
        }
    }

外部对象接口

    public interface IInteropInterface
    {
    }

执行器

 public class ExecutionEngine : IDisposable
    {
        //调用栈
        public RandomAccessStack<ExecutionContext> InvocationStack { get; } = new RandomAccessStack<ExecutionContext>();
        //执行栈
        public RandomAccessStack<StackItem> EvaluationStack { get; } = new RandomAccessStack<StackItem>();
        //参数栈
        public RandomAccessStack<StackItem> AltStack { get; } = new RandomAccessStack<StackItem>();
        public ExecutionContext CurrentContext => InvocationStack.Peek();
        public ExecutionContext CallingContext => InvocationStack.Count >  ? InvocationStack.Peek() : null;
        public ExecutionContext EntryContext => InvocationStack.Peek(InvocationStack.Count - );
        //执行状态
        public VMState State { get; protected set; } = VMState.BREAK;

        //载入执行脚本
        void LoadScript(byte[] script, bool push_only = false){}
        //添加断点
        void AddBreakPoint(uint position){}
        //删除断点
        bool RemoveBreakPoint(uint position){}

        //执行脚本
        void Execute(){}
        //执行opcode
        void ExecuteOp(OpCode opcode, ExecutionContext context){}

        //执行下一步
        void StepInto(){}
        //当前call执行完成
        void StepOut(){}
        //全部执行
        void StepOver(){}
    }

操作符号执行过程

     private void ExecuteOp(OpCode opcode, ExecutionContext context)
        {
            if (opcode > OpCode.PUSH16 && opcode != OpCode.RET && context.PushOnly)
            {
                State |= VMState.FAULT;
                return;
            }
            if (opcode >= OpCode.PUSHBYTES1 && opcode <= OpCode.PUSHBYTES75)
                EvaluationStack.Push(context.OpReader.ReadBytes((byte)opcode));
            else
                switch (opcode)
                {
                    //常量push
                    case OpCode.PUSH1:
                    case OpCode.PUSH16:
                        EvaluationStack.Push((int)opcode - (int)OpCode.PUSH1 + );
                        break;
                    case OpCode.JMP: //跳转
                        {
                            int offset = context.OpReader.ReadInt16();
                            offset = context.InstructionPointer + offset - ;
                            if (offset <  || offset > context.Script.Length)
                            {
                                State |= VMState.FAULT;
                                return;
                            }
                            bool fValue = true;
                            if (opcode > OpCode.JMP)
                            {
                                fValue = EvaluationStack.Pop().GetBoolean();
                                if (opcode == OpCode.JMPIFNOT)
                                    fValue = !fValue;
                            }
                            if (fValue)
                                context.InstructionPointer = offset;
                        }
                        break;
                    case OpCode.CALL:  //和systemcall差不多， 区别在于system是系统预先注册的函数 call调用的是用户自己写的函数
                        InvocationStack.Push(context.Clone());
                        context.InstructionPointer += ;
                        ExecuteOp(OpCode.JMP, CurrentContext);
                        break;
                    case OpCode.RET:   //退出当前函数栈
                        InvocationStack.Pop().Dispose();
                        if (InvocationStack.Count == )
                            State |= VMState.HALT;
                        break;
                    case OpCode.APPCALL:  //调用外部合约
                    case OpCode.TAILCALL:
                        {
                            if (table == null)
                            {
                                State |= VMState.FAULT;
                                return;
                            }

                            byte[] script_hash = context.OpReader.ReadBytes();
                            if (script_hash.All(p => p == ))
                            {
                                script_hash = EvaluationStack.Pop().GetByteArray();
                            }

                            byte[] script = table.GetScript(script_hash);
                            if (script == null)
                            {
                                State |= VMState.FAULT;
                                return;
                            }
                            if (opcode == OpCode.TAILCALL)
                                InvocationStack.Pop().Dispose();
                            LoadScript(script);
                        }
                        break;
                    case OpCode.SYSCALL:  //内部合约函数调用
                        if (!service.Invoke(Encoding.ASCII.GetString(context.OpReader.ReadVarBytes()), this))
                            State |= VMState.FAULT;
                        break;
                    case OpCode.DROP:  //移除栈顶
                        EvaluationStack.Pop();
                        break;
                    case OpCode.DUP:   //赋值栈顶  有对应按位置复制的指令
                        EvaluationStack.Push(EvaluationStack.Peek());
                        break;
                    case OpCode.EQUAL: //判等
                        {
                            StackItem x2 = EvaluationStack.Pop();
                            StackItem x1 = EvaluationStack.Pop();
                            EvaluationStack.Push(x1.Equals(x2));
                        }
                        break;

                    // Numeric
                    case OpCode.ABS: //运算 加减乘除 最大值最小值等等
                        {
                            BigInteger x = EvaluationStack.Pop().GetBigInteger();
                            EvaluationStack.Push(BigInteger.Abs(x));
                        }
                        break;
                    // Crypto
                    case OpCode.SHA256: //加密
                        using (SHA256 sha = SHA256.Create())
                        {
                            byte[] x = EvaluationStack.Pop().GetByteArray();
                            EvaluationStack.Push(sha.ComputeHash(x));
                        }
                        break;
                    case OpCode.CHECKSIG:  //验证
                        {
                            byte[] pubkey = EvaluationStack.Pop().GetByteArray();
                            byte[] signature = EvaluationStack.Pop().GetByteArray();
                            try
                            {
                                EvaluationStack.Push(Crypto.VerifySignature(ScriptContainer.GetMessage(), signature, pubkey));
                            }
                            catch (ArgumentException)
                            {
                                EvaluationStack.Push(false);
                            }
                        }
                        break;
                    // Array
                    case OpCode.PICKITEM:   //数组映射取值
                        {
                            StackItem key = EvaluationStack.Pop();
                            if (key is ICollection)
                            {
                                State |= VMState.FAULT;
                                return;
                            }
                            switch (EvaluationStack.Pop())
                            {
                                case VMArray array:
                                    int index = (int)key.GetBigInteger();
                                    if (index <  || index >= array.Count)
                                    {
                                        State |= VMState.FAULT;
                                        return;
                                    }
                                    EvaluationStack.Push(array[index]);
                                    break;
                                case Map map:
                                    if (map.TryGetValue(key, out StackItem value))
                                    {
                                        EvaluationStack.Push(value);
                                    }
                                    else
                                    {
                                        State |= VMState.FAULT;
                                        return;
                                    }
                                    break;
                                default:
                                    State |= VMState.FAULT;
                                    return;
                            }
                        }
                        break;
                    case OpCode.SETITEM:  //数组 映射赋值
                        {
                            StackItem value = EvaluationStack.Pop();
                            if (value is Struct s) value = s.Clone();
                            StackItem key = EvaluationStack.Pop();
                            if (key is ICollection)
                            {
                                State |= VMState.FAULT;
                                return;
                            }
                            switch (EvaluationStack.Pop())
                            {
                                case VMArray array:
                                    int index = (int)key.GetBigInteger();
                                    if (index <  || index >= array.Count)
                                    {
                                        State |= VMState.FAULT;
                                        return;
                                    }
                                    array[index] = value;
                                    break;
                                case Map map:
                                    map[key] = value;
                                    break;
                                default:
                                    State |= VMState.FAULT;
                                    return;
                            }
                        }
                        break;
                    case OpCode.NEWARRAY: //创建数组
                        {
                            int count = (int)EvaluationStack.Pop().GetBigInteger();
                            List<StackItem> items = new List<StackItem>(count);
                            for (var i = ; i < count; i++)
                            {
                                items.Add(false);
                            }
                            EvaluationStack.Push(new Types.Array(items));
                        }
                        break;
                    case OpCode.NEWSTRUCT: //创建结构体
                        {
                            int count = (int)EvaluationStack.Pop().GetBigInteger();
                            List<StackItem> items = new List<StackItem>(count);
                            for (var i = ; i < count; i++)
                            {
                                items.Add(false);
                            }
                            EvaluationStack.Push(new VM.Types.Struct(items));
                        }
                        break;
                    case OpCode.NEWMAP: //创建映射
                        EvaluationStack.Push(new Map());
                        break;
                    case OpCode.APPEND:  //追加元素
                        {
                            StackItem newItem = EvaluationStack.Pop();
                            if (newItem is Types.Struct s)
                            {
                                newItem = s.Clone();
                            }
                            StackItem arrItem = EvaluationStack.Pop();
                            if (arrItem is VMArray array)
                            {
                                array.Add(newItem);
                            }
                            else
                            {
                                State |= VMState.FAULT;
                                return;
                            }
                        }
                        break;
                    case OpCode.REMOVE:  //移除元素
                        {
                            StackItem key = EvaluationStack.Pop();
                            if (key is ICollection)
                            {
                                State |= VMState.FAULT;
                                return;
                            }
                            switch (EvaluationStack.Pop())
                            {
                                case VMArray array:
                                    int index = (int)key.GetBigInteger();
                                    if (index <  || index >= array.Count)
                                    {
                                        State |= VMState.FAULT;
                                        return;
                                    }
                                    array.RemoveAt(index);
                                    break;
                                case Map map:
                                    map.Remove(key);
                                    break;
                                default:
                                    State |= VMState.FAULT;
                                    return;
                            }
                        }
                        break;
                    case OpCode.KEYS:  //获取映射键集合，对应的还有获取值集合 haskey
                        switch (EvaluationStack.Pop())
                        {
                            case Map map:
                                EvaluationStack.Push(new VMArray(map.Keys));
                                break;
                            default:
                                State |= VMState.FAULT;
                                return;
                        }
                        break;

                    // Exceptions
                    case OpCode.THROW: //异常中止
                        State |= VMState.FAULT;
                        return;
                    case OpCode.THROWIFNOT:
                        if (!EvaluationStack.Pop().GetBoolean())
                        {
                            State |= VMState.FAULT;
                            return;
                        }
                        break;

                    default:
                        State |= VMState.FAULT;
                        return;
                }
            if (!State.HasFlag(VMState.FAULT) && InvocationStack.Count > )
            {
                //断点起效的位置
                if (CurrentContext.BreakPoints.Contains((uint)CurrentContext.InstructionPointer))
                    State |= VMState.BREAK;
            }
        }

转载自 https://my.oschina.net/hunjixin/blog/1812516