花了不到一周的时间看完了一本reids设计与实现的书,感觉整体的设计有些地方的确很巧妙,各个结构之间联系的非常紧密,但是很简单,逻辑性的没有太多,但是学到了一个bitcount计数1的方法比较巧妙,记录下来

看了一个老外的介绍的很详细

转载过来

OK, let's go through the code line by line:

Line 1:

i = i - ((i >> 1) & 0x55555555);

First of all, the significance of the constant 0x55555555 is that, written using the Java / GCC style binary literal notation),

0x55555555 = 0b01010101010101010101010101010101

That is, all its odd-numbered bits (counting the lowest bit as bit 1 = odd) are 1, and all the even-numbered bits are 0.

The expression ((i >> 1) & 0x55555555) thus shifts the bits of i right by one, and then sets all the even-numbered bits to zero. (Equivalently, we could've first set all the odd-numbered bits of i to zero with & 0xAAAAAAAA and then shifted the result right by one bit.) For convenience, let's call this intermediate value j.

What happens when we subtract this j from the original i? Well, let's see what would happen if ihad only two bits:

    i           j         i - j
----------------------------------
0 = 0b00 0 = 0b00 0 = 0b00
1 = 0b01 0 = 0b00 1 = 0b01
2 = 0b10 1 = 0b01 1 = 0b01
3 = 0b11 1 = 0b01 2 = 0b10

Hey! We've managed to count the bits of our two-bit number!

OK, but what if i has more than two bits set? In fact, it's pretty easy to check that the lowest two bits of i - j will still be given by the table above, and so will the third and fourth bits, and the fifth and sixth bits, and so and. In particular:

  • despite the >> 1, the lowest two bits of i - j are not affected by the third or higher bits of i, since they'll be masked out of j by the & 0x55555555; and

  • since the lowest two bits of j can never have a greater numerical value than those of i, the subtraction will never borrow from the third bit of i: thus, the lowest two bits of i also cannot affect the third or higher bits of i - j.

In fact, by repeating the same argument, we can see that the calculation on this line, in effect, applies the table above to each of the 16 two-bit blocks in i in parallel. That is, after executing this line, the lowest two bits of the new value of i will now contain the number of bits set among the corresponding bits in the original value of i, and so will the next two bits, and so on.

Line 2:

i = (i & 0x33333333) + ((i >> 2) & 0x33333333);

Compared to the first line, this one's quite simple. First, note that

0x33333333 = 0b00110011001100110011001100110011

Thus, i & 0x33333333 takes the two-bit counts calculated above and throws away every second one of them, while (i >> 2) & 0x33333333 does the same after shifting i right by two bits. Then we add the results together.

Thus, in effect, what this line does is take the bitcounts of the lowest two and the second-lowest two bits of the original input, computed on the previous line, and add them together to give the bitcount of the lowest four bits of the input. And, again, it does this in parallel for all the 8 four-bit blocks (= hex digits) of the input.

Line 3:

return (((i + (i >> 4)) & 0x0F0F0F0F) * 0x01010101) >> 24;

OK, what's going on here?

Well, first of all, (i + (i >> 4)) & 0x0F0F0F0F does exactly the same as the previous line, except it adds the adjacent four-bit bitcounts together to give the bitcounts of each eight-bit block (i.e. byte) of the input. (Here, unlike on the previous line, we can get away with moving the & outside the addition, since we know that the eight-bit bitcount can never exceed 8, and therefore will fit inside four bits without overflowing.)

Now we have a 32-bit number consisting of four 8-bit bytes, each byte holding the number of 1-bit in that byte of the original input. (Let's call these bytes ABC and D.) So what happens when we multiply this value (let's call it k) by 0x01010101?

Well, since 0x01010101 = (1 << 24) + (1 << 16) + (1 << 8) + 1, we have:

k * 0x01010101 = (k << 24) + (k << 16) + (k << 8) + k

Thus, the highest byte of the result ends up being the sum of:

  • its original value, due to the k term, plus
  • the value of the next lower byte, due to the k << 8 term, plus
  • the value of the second lower byte, due to the k << 16 term, plus
  • the value of the fourth and lowest byte, due to the k << 24 term.

(In general, there could also be carries from lower bytes, but since we know the value of each byte is at most 8, we know the addition will never overflow and create a carry.)

That is, the highest byte of k * 0x01010101 ends up being the sum of the bitcounts of all the bytes of the input, i.e. the total bitcount of the 32-bit input number. The final >> 24 then simply shifts this value down from the highest byte to the lowest.

Ps. This code could easily be extended to 64-bit integers, simply by changing the 0x01010101 to 0x0101010101010101 and the >> 24 to >> 56. Indeed, the same method would even work for 128-bit integers; 256 bits would require adding one extra shift / add / mask step, however, since the number 256 no longer quite fits into an 8-bit byte.

其重要就是对于32位的数据看做一个整体,首先以其中2位为一组计算出1的个数,再以4位为一组,计算出1的个数,再以8位为一组,计算出1的个数

其实到这里1的个数就是4个8位的之和,可以通过计算求解

比如0xbbbbbbbb *0x01010101= 0xbbbbbbbb *(1<<24+1<<16+1<<8+1)

最后32位中1的个数的总数就被保存在了高8位中,此时只需要>>24就可以求出来了

当然在实际中,我们可以一次计算出多个32位,比如计算出4*32 那样的时间效率相对于遍历就节约了128倍,相对于查表法也快了4*4倍

redis bitcount variable-precision swar算法的更多相关文章

  1. variable precision SWAR算法

    计算二进制形式中1的数量这种问题,在各种刷题网站上比较常见,以往都是选择最笨的遍历方法“蒙混”过关.在了解Redis的过程中接触到了variable precision SWAR算法(以下简称VP-S ...

  2. variable-precision SWAR算法介绍

    BITCOUNT命令是统计一个位数组中非0进制位的数量,数学上称作:”Hanmming Weight“ 目前效率最好的为variable-precision SWAR算法,可以常数时间内计算出多个字节 ...

  3. variable-precision SWAR算法:计算Hamming Weight

    variable-precision SWAR算法:计算Hamming Weight 转自我的Github 最近看书看到了一个计算Hamming Weight的算法,觉得挺巧妙的,纪录一下. Hamm ...

  4. [算法]从一道题引出variable-precision SWAR算法

    苏君君出了一道题,是牛客网上面的: 输入一个int型整数,输出该数二进制表示中1的个数.其中负数用补码表示. 其实这道题并不难,大家很容易想到的解法是转成字符串的思路,即如下所示: public st ...

  5. 11.redis cluster的hash slot算法和一致性 hash 算法、普通hash算法的介绍

    分布式寻址算法 hash 算法(大量缓存重建) 一致性 hash 算法(自动缓存迁移)+ 虚拟节点(自动负载均衡) redis cluster 的 hash slot 算法 一.hash 算法 来了一 ...

  6. CYQ.Data V5 分布式缓存Redis应用开发及实现算法原理介绍

    前言: 自从CYQ.Data框架出了数据库读写分离.分布式缓存MemCache.自动缓存等大功能之后,就进入了频繁的细节打磨优化阶段. 从以下的更新列表就可以看出来了,3个月更新了100条次功能: 3 ...

  7. Redis的一致性哈希算法

    一.节点取余 根据redis的键或者ID,再根据节点数量进行取余. key:value如下 name:1 zhangsna:18:北京 对name:1 进行hash操作,得出来得值是242342345 ...

  8. Redis 为何使用近似 LRU 算法淘汰数据,而不是真实 LRU?

    在<Redis 数据缓存满了怎么办?>我们知道 Redis 缓存满了之后能通过淘汰策略删除数据腾出空间给新数据. 淘汰策略如下所示: 设置过期时间的 key volatile-ttl.vo ...

  9. Redis rdb文件CRC64校验算法 Java实现

    查看RDB文件结构,发现最后的8字节是CRC64校验算得,从文件头开始直到8字节校验码前的FF结束码(含),经过CRC64校验计算发现,貌似最后的8字节是小端模式实现的. 参考redis的crc64实 ...

随机推荐

  1. Javascript中变量作用域

    <script type="text/javascript"> var a = 10; var Bar = (function () { console.log(a); ...

  2. [Git]06 如何提交空目录

     git和 svn不同,仅仅跟踪文件的变动,不跟踪目录.所以,一个空目录,如果里面没有文件,即便 git add 这个目录,另外在别处 check out 的时候,是没有这个空目录的. 只跟踪文件 ...

  3. PHP学习笔记-3

    PHP 数据类型: 字符串.整数.浮点数.逻辑.数组.对象.NULL. JavaScript数据类型: 字符串.数字.布尔.数组.对象.Null.Undefined. 从上面可以看出来,数据类型都是7 ...

  4. Hibernate配置文件中配置各种数据库链接

    hibernate.properties ###################### ### Query Language ### ###################### ## define ...

  5. 虚幻UE4中移动端水材质的设置

    内容: *概述 *纹理文件 *基本颜色 *法线的设置 *标量参数和材质属性 *场景设置 *最终效果 概述 本教程由52VR翻译自unrealengine官方,在本教程中,我们将教您如何创建可以在移动设 ...

  6. 对游览器遭到劫持的处理方案(RemoveAds Not By This Site)

    近期游览器一直被劫持 :火狐用不了,ie出毛病了,谷歌游览器也不好使了,已经快被逼疯!!! 但是事情总是向着美好的方向发展的么! 接下来就和大家一起分享一下我的解决经验! 首先用360杀毒(虽然大多数 ...

  7. 每天一道Java题[4]

    问题 怎么将字符串转换为int? 解答 此题看似简单,但经常出现在笔试等地方,由于大家习惯了用IDE,有什么还真未必能写出来.通常都是parseInt()方法进行转换,如下: Int n = Inte ...

  8. 如何用php实现简单的文件上传功能?(带图解)

    如图所示:点击浏览出现选择文件的对话框,将所选文件上传到保存文件的文件.  关键点:文件上传的图解: 代码: <!DOCTYPE html> <html> <head&g ...

  9. Java学习笔记——Linux下安装配置MySQL

    山重水复疑无路,柳暗花明又一村 --游山西村 系统:Ubuntu 16.04LTS 1\官网下载mysql-5.7.18-linux-glibc2.5-x86_64.tar.gz2\建立工作组:$su ...

  10. 关于Java解压文件的一些坑及经验分享(MALFORMED异常)

    文章也已经同步到我的csdn博客: http://blog.csdn.net/u012881584/article/details/72615481 关于Java解压文件的一些坑及经验分享 就在本周, ...