利用浏览器favicon的缓存机制(F-Cache)生成客户端浏览器唯一指纹

首先介绍下：

这个技术出自

UIC论文：https://www.cs.uic.edu/~polakis/papers/solomos-ndss21.pdf

源码：https://github.com/jonasstrehle/supercookie

原理图解：

下面这个图是解释了让浏览器的favicon的请求缓存机制缓存我们想要缓存的路由

下面这个图是解释了针对客户端浏览器的请求缓存机制反推到唯一指纹

本篇文章主要分析源码层面是如何实现的

初始化参数和favicon的路由

{

  "index": 1,

  "cacheID": "eb60b0a3"

}

favicon的路由设置的是32个，那么理论上最多可以支持创建40亿个唯一指纹

[

  "eb60b0a3:yQqmEg2rcV4hX6FFrr5khA",

  "eb60b0a3:rxK3EqtBI2GdYI58UTQKsg",

  "eb60b0a3:2GaUtZwg3fFUFMg4Eirrcg",

  "eb60b0a3:49C0Fec66xaJ3yQhz0WCcw",

  "eb60b0a3:hmMDHBUG9DB4CW02clFxRw",

  "eb60b0a3:klFWlGuzbS3k49qoMu4YmQ",

  "eb60b0a3:rkb7uew0g6ZfQ1qzIr0n3A",

  "eb60b0a3:szhFZEttZK7HCAD9V8encQ",

  "eb60b0a3:Gi37CDcH90FPlb2P257xew",

  "eb60b0a3:WwLR0GW7s9VfbZc75SglxQ",

  "eb60b0a3:gR5KV6MPacNombv0ssbcGg",

  "eb60b0a3:6HLr0YwczyJkgd5a0imA0A",

  "eb60b0a3:lt3NSeEE9OjY4PnUMKQ3Kg",

  "eb60b0a3:uYx443BpkfANUbVbEiS6iQ",

  "eb60b0a3:tv0SDGvMbZWHK4siR3J9rg",

  "eb60b0a3:iM7UdU8h6I0tN35ykaGmgA",

  "eb60b0a3:6HVrMyGR0130jJq00hqv3A",

  "eb60b0a3:dasC4zubTWlxExsb6dUmig",

  "eb60b0a3:ubdfIPtJAcF3u4z7HLU1WQ",

  "eb60b0a3:rtSm3AMgDCN9ibvYRa5dAQ",

  "eb60b0a3:g1EMlXuNH0WpKlDQ8ECpXQ",

  "eb60b0a3:rL0WLoKRICrAycO8bQ0TZA",

  "eb60b0a3:UlGw3nwB0PfZgPqhnYHjRQ",

  "eb60b0a3:YIkljO2Ta2fxePjWVbUhaA",

  "eb60b0a3:buNyF0aeM5q6HBBgEMhemA",

  "eb60b0a3:vsyOlIR3mFlk5eE4DVTd4A",

  "eb60b0a3:LpM4qTHHpEXdngwBxuIrvA",

  "eb60b0a3:Xh0eIiJxG9KyW6F9JLkdYg",

  "eb60b0a3:1krk2hqGZsWZP99mVVNJSA",

  "eb60b0a3:R1Ks5T4HliO6JZZhioeIMA",

  "eb60b0a3:Tbl0S6dn7QuIcO3w0HScZw",

  "eb60b0a3:dK9174FYAVotz9hz0gLGcQ"

]

打开 http://localhost:10081/ 进入服务端逻辑：

webserver_2.get('/', (_req, res) => {

    Webserver.setCookie(res, "rid", true);

    res.clearCookie("mid");

    res.redirect(`/eb60b0a3`);

});

eb60b0a3 这里的这个是在上面随机配置的

设置一个cookie 叫 rid：true

清除cookie mid (下面会说到)

重定向到路由/eb60b03



webserver_2.get(`/eb60b0a3`, (req, res) => {

    const rid = !!req.cookies.rid;

    res.clearCookie("rid");

    if (!rid)

        //不支持

        Webserver.sendFile(res, path.join(path.resolve(), "www/redirect.html"), {

            url_demo: WEBSERVER_DOMAIN_2

        });

    else

        Webserver.sendFile(res, path.join(path.resolve(), "www/launch.html"), {

            favicon: CACHE_IDENTIFIER

        });

});

下面是 www/launch.html的内容

<!DOCTYPE html>

<html>

    <head>

        <link rel="shortcut icon" href="/l/{{favicon}}" type="image/x-icon"/>

    </head>

    <body>

        <h1>...</h1>

        <script type="module">

            window.onload = async () => {

                await new Promise((resolve) => setTimeout(resolve, 500));

                const mid = (document.cookie.match(new RegExp(`(^| )mid=([^;]+)`)) || [])[2];

                const route = !!mid ? `/write/${mid}` : "/read";

                document.cookie.split(";").forEach((c) => document.cookie = c.replace(/^ +/, "").replace(/=.*/, `=;expires=${new Date().toUTCString()};path=/`));

                window.location.href = route;

            }

        </script>

    </body>

</html>

如上面的页面，注意有一个

<link rel="shortcut icon" href="/l/{{favicon}}" type="image/x-icon"/>

页面加载过程会触发加载上面的icon，对应会进入服务端代码：

webserver_2.get("/l/:ref", (_req, res) => {

    console.info(`supercookie | Unknown visitor detected.`);

    Webserver.setCookie(res, "mid", generateWriteToken());

    const data = Buffer.from(FILE, "base64");

    res.writeHead(200, {

        "Cache-Control": "public, max-age=31536000",

        "Expires": new Date(Date.now() + 31536000000).toUTCString(),

        "Content-Type": "image/png",

        "Content-Length": data.length

    });

    res.end(data);

});

第一次进入站点则会先访问 /l/:ref 路由

1.如果是第一次进入网站那么浏览器的F-Cache会没有对favicon：/l/eb60b0a3,则会触发进入上面的代码，然后创建一个mid到cookie 返回favicon

2.window.onload事件里的timout走完后，肯定能拿到cookie里面的mid，客户端改变路由到/write/xxx

webserver_2.get("/write/:mid", (req, res) => {

    const mid = req.params.mid;

    if (!hasWriteToken(mid))

        return res.redirect('/');

    res.clearCookie("mid");

    deleteWriteToken(mid);

    const uid = generateUUID();

    console.info(`supercookie | Visitor uid='${uid}' is unknown • Write`, STORAGE.index);

    const profile = Profile.from(uid, STORAGE.index);

    if (profile === null)

        return res.redirect('/');

    STORAGE.index++;//这里++的目的是留给下一个

    Webserver.setCookie(res, "uid", uid);

    res.redirect(`/t/${Webserver.getRouteByIndex(0)}`);

});

const profile = Profile.from(uid, STORAGE.index);

注意这个代码：

比如当前的index=29，29的二进制11101，我们上面设置的32个路由，前面补0凑足32个：

0000000000000000000000000011101

然后进行reverse变成

1011100000000000000000000000000

以上就是唯一数：29

1对应哪些favicon的路由需要进入客户端浏览器缓存，0的话会舍弃请求不让客户端浏览器缓存

创建uid
index++ （唯一的编号，最上面配置的index，服务端会更新配置）
转到 route /t/第一个路由（下面会重点介绍）

非第一次进入站点会先走到 /read 路由

如果已经浏览器的F-Cache已对favicon：/l/eb60b0a3 做过缓存的话，客户端的代码会跳转到 /read

webserver_2.get("/read", (_req, res) => {

    const uid = generateUUID();

    console.info(`supercookie | Visitor uid='${uid}' is known • Read`);

    const profile = Profile.from(uid);

    if (profile === null)

        return res.redirect("/read");

    //设置要遍历的总次数

    profile._setStorageSize(Math.floor(Math.log2(STORAGE.index ?? 1)) + 1);

    Webserver.setCookie(res, "uid", uid);

    res.redirect(`/t/${Webserver.getRouteByIndex(0)}?f=${generateUUID()}`);

});

创建一个随机 uid
从Profile读取一个uid，如果不存在创建一个，如果存在的话为null
如果为null 重新路由到/read （这目的是防止generateUUID()重复）
转到 route /t/第一个路由

下面重点的是 /t/ 路由

webserver_2.get("/t/:ref", (req, res) => {

    const referrer = req.params.ref;

    const uid = req.cookies.uid;

    const profile = Profile.get(uid);

    if (!Webserver.hasRoute(referrer) || profile === null)

        return res.redirect('/');

    const route = Webserver.getNextRoute(referrer);

    if (profile._isReading() && profile.visited.has(referrer))

        return res.redirect('/');

    let nextReferrer = null;

    const redirectCount = profile._isReading() ?

        profile.storageSize :

        Math.floor(Math.log2(profile.identifier)) + 1;

    if (route)

        nextReferrer = `t/${route}?f=${generateUUID()}`;

    if (!profile._isReading()) {

        if (Webserver.getIndexByRoute(referrer) >= redirectCount - 1)

            nextReferrer = "read";

    }

    else if (Webserver.getIndexByRoute(referrer) >= redirectCount - 1 || nextReferrer === null)

        nextReferrer = "identity";

    console.log(nextReferrer)

    const bit = !profile._isReading() ? profile.vector.includes(referrer) : "{}";

    Webserver.sendFile(res, path.join(path.resolve(), "www/referrer.html"), {

        delay: profile._isReading() ? 500 : 800,

        referrer: nextReferrer,

        favicon: referrer,

        bit: bit,

        index: `${Webserver.getIndexByRoute(referrer) + 1} / ${redirectCount}`

    });

});

最上面的路由数组挨个的遍历，遍历的次数为 Math.floor(Math.log2(index)) + 1，(2的对数去掉小数点+1)，总次数是和唯一数相对的一个算法，比如说当前已经生成了到100万个唯一数需要20次,16亿个唯一数，那么要遍历的总次数为31次！40亿就是32次到头了！
会返回客户端www/referrer.html

这个www/referrer.html 的html内容里面有一个

<link rel="shortcut icon" href="/f/{{favicon}}" type="image/x-icon"/>

对应服务端：

webserver_2.get("/f/:ref", (req, res) => {

    const referrer = req.params.ref;

    const uid = req.cookies.uid;

    console.log(referrer);

    if (!Profile.has(uid) || !Webserver.hasRoute(referrer))

        return res.status(404), res.end();

    const profile = Profile.get(uid);

    if (profile._isReading()) {

        profile._visitRoute(referrer);

        console.info(`supercookie | Favicon requested by uid='${uid}' • Read `, Webserver.getIndexByRoute(referrer), "•", Array.from(profile.visited).map(route => Webserver.getIndexByRoute(route)));

        return;

    }

    if (!profile.vector.includes(referrer)) {

        //第一次进入站点会进入写的逻辑

        console.info(`supercookie | Favicon requested by uid='${uid}' • Write`, Webserver.getIndexByRoute(referrer), "•", Array.from(profile.vector).map(route => Webserver.getIndexByRoute(route)));

        return;

    }

    const data = Buffer.from(FILE, "base64");

    res.writeHead(200, {

        "Cache-Control": "public, max-age=31536000",

        "Expires": new Date(Date.now() + 31536000000).toUTCString(),

        "Content-Type": "image/png",

        "Content-Length": data.length

    });

    res.end(data);

});

如果浏览器有F-cache存在的话不会走到上面的代码
走进去了代表没有该icon，那么服务端会把这个路由记录下来

路由 /identity

/t/ 路由走完后，会走到 /identity

webserver_2.get("/identity", (req, res) => {

    const uid = req.cookies.uid;

    const profile = Profile.get(uid);

    if (profile === null)

        return res.redirect('/');

    res.clearCookie("uid");

    res.clearCookie("vid");

    const identifier = profile._calcIdentifier();

    if (identifier === maxN || profile.visited.size === 0 || identifier === 0)

        return res.redirect(`/write/${generateWriteToken()}`);

    if (identifier !== 0) {

        const identifierHash = hashNumber(identifier);

        console.info(`supercookie | Visitor successfully identified as '${identifierHash}' • (#${identifier}).`);

        Webserver.sendFile(res, path.join(path.resolve(), "www/identity.html"), {

            hash: identifierHash,

            identifier: `#${identifier}`,

            url_workwise: `${WEBSERVER_DOMAIN_1}/workwise`,

            url_main: WEBSERVER_DOMAIN_1

        });

    }

    else

        Webserver.sendFile(res, path.join(path.resolve(), "www/identity.html"), {

            hash: "AN ON YM US",

            identifier: "browser not vulnerable",

            url_workwise: `${WEBSERVER_DOMAIN_1}/workwise`,

            url_main: WEBSERVER_DOMAIN_1

        });

});

走到这里基本上能确定了客户端走了哪些favicon请求
根据记录了哪些favicon路由请求就可以确定是哪个index(一个index代表一个唯一的用户)

For Mac, delete: ${user.home}/Library/Application Support/Google/Chrome/Default/Favicons

For Windows: go to %LocalAppData%\Google\Chrome\User Data\Default and delete favicons and favicons-journal files

以下是index=29的路由日志：

可以看出来第一次进入，将29对应的favicon路由写进客户端浏览器的F-Cache缓存，

然后在触发read 读取哪些缓存哪些没缓存会可以还原得到29

supercookie | Unknown visitor detected.

supercookie | Visitor uid='8ec7-m226-b3d9-k5yc-i5yh' is known • Read

t/eb60b0a3:rxK3EqtBI2GdYI58UTQKsg?f=3inj-ac2u-9wwg-jh96-dww7

supercookie | Favicon requested by uid='8ec7-m226-b3d9-k5yc-i5yh' • Read  0 • [ 0 ]

t/eb60b0a3:2GaUtZwg3fFUFMg4Eirrcg?f=jwoo-x1u9-uu4g-8u1y-32i2

supercookie | Favicon requested by uid='8ec7-m226-b3d9-k5yc-i5yh' • Read  1 • [ 0, 1 ]

t/eb60b0a3:49C0Fec66xaJ3yQhz0WCcw?f=q9i4-hopc-lcq2-j0pv-h1hx

supercookie | Favicon requested by uid='8ec7-m226-b3d9-k5yc-i5yh' • Read  2 • [ 0, 1, 2 ]

t/eb60b0a3:hmMDHBUG9DB4CW02clFxRw?f=81lb-gevm-3tdj-ycg4-ap8q

supercookie | Favicon requested by uid='8ec7-m226-b3d9-k5yc-i5yh' • Read  3 • [ 0, 1, 2, 3 ]

identity

supercookie | Favicon requested by uid='8ec7-m226-b3d9-k5yc-i5yh' • Read  4 • [ 0, 1, 2, 3, 4 ]

supercookie | Visitor uid='gr8o-kmh5-j2fo-rh6m-uj8z' is unknown • Write 29

t/eb60b0a3:rxK3EqtBI2GdYI58UTQKsg?f=y451-fhdp-5asc-qqww-tvan

t/eb60b0a3:2GaUtZwg3fFUFMg4Eirrcg?f=hpf9-w33z-ank8-oesm-hmmn

supercookie | Favicon requested by uid='gr8o-kmh5-j2fo-rh6m-uj8z' • Write 1 • [ 0, 2, 3, 4 ]

t/eb60b0a3:49C0Fec66xaJ3yQhz0WCcw?f=evwy-vcni-sxlg-ncc8-mr51

t/eb60b0a3:hmMDHBUG9DB4CW02clFxRw?f=ropd-egna-2q0c-3f8k-svew

read

supercookie | Visitor uid='ai0j-6xbm-9ikb-mcs3-23ty' is known • Read

t/eb60b0a3:rxK3EqtBI2GdYI58UTQKsg?f=f5mh-8rgh-zrtp-ao7a-fgie

t/eb60b0a3:2GaUtZwg3fFUFMg4Eirrcg?f=xusf-lse6-fssp-1pgb-4w1c

supercookie | Favicon requested by uid='ai0j-6xbm-9ikb-mcs3-23ty' • Read  1 • [ 1 ]

t/eb60b0a3:49C0Fec66xaJ3yQhz0WCcw?f=qqsk-36vu-a333-5fnq-rmdp

t/eb60b0a3:hmMDHBUG9DB4CW02clFxRw?f=geij-0sqf-6hw1-dkax-6kj3

identity

supercookie | Visitor successfully identified as '44 40 C3 17 E2 1B' • (#29).