利用Redlock演算法实现自己的分布式锁

前言

我之前有写透过 lock or CAS 来防治，Racing condition 问题，但如果这个问题延深到多台服务器甚至是 micor-services 架构我们要怎麽处理资料问题呢?

下面程序在单体服务或应用程序不会出问题，但如果服务器有多台问题可就大了，因为下面的 lock 只限於单体 Server 上

private readonly static object _lock = new object();

[HttpGet()]
public string Get()
{
    int remainCount;
    lock (_lock)
    {
        
        remainCount = (int)RedisConnection.GetDatabase().StringGet(_productId);
        if (remainCount > 0)
        {
            RedisConnection.GetDatabase().StringSet(_productId, --remainCount);
        }
    }

    string productMsg = remainCount <= 0 ? "没货了卖完了" : $"还剩下 {remainCount} 商品";
    string result = $"{Dns.GetHostName()} 处理货物状态!! {productMsg}";
    _logger.LogInformation(result);
    return result;
}

如果有听过 Redis 可能就会听过 RedLock.net 来处理，但您知道 RedLock.net 底层大致上怎麽实作的吗?

本篇文章会带领大家透过 distlock 算法时做出，自己的Redlock

本篇只介绍核心概念，细部防错我没有写出来，所以建议本篇程序不要用在 prod 环境

我使用 docker-compose 建立问题架构，架构图如下

原始码位置 Redlock-Csharp

How to Run

在根目录使用 docker-compose up -d 跑起来後应该会有下面五个组件

後面我们进入 Redis Server 利用命令建立一个商品和数量 set pid:1 1000

pid=1 有 1000 个

$ docker exec -it b489eb20ab74 bash
root@b489eb20ab74:/data# redis-cli
127.0.0.1:6379> set pid:1 1000
OK

在查询 http://localhost:8080/WebStore 可以获得下图代表组件建立完毕

Redlock 演算法

其实 distlock 算法说明在 Redis 官网有一篇专门来说明，

测试档案已经准备好了 /test/shopping.jmx，我们使用 jmeter 来压测，使用压力测试程序，建议 Jmeter 使用 Thread 最好小於或等於 cpu core count (最好设定2的倍数)

ex: 我的电脑 8 core 我可以设定 8 thread concurrent

这边有一个情境假设，有一个商品秒杀只有 200 个商品

这边我们要怎麽防止卖超呢?

我们把 lock 部分替换成我们自己写的 RedLock 物件．我有透过 Redis 算法实现一个简单的分布式锁

[HttpGet()]
public string Get()
{
    RedLock redlock = new RedLock(RedisConnection);
    int remainCount;
    try
    {
        redlock.LockInstance("redkey", new TimeSpan(00, 00, 10), out var lockObject);
        remainCount = (int)RedisConnection.GetDatabase().StringGet(_productId);
        if (remainCount > 0)
        {
            RedisConnection.GetDatabase().StringSet(_productId, --remainCount);
        }

    }
    finally
    {
        redlock.UnlockInstance("redkey");
    }
    string productMsg = remainCount <= 0 ? "没货了卖完了" : $"还剩下 {remainCount} 商品";
    string result = $"{Dns.GetHostName()} 处理货物状态!! {productMsg}";
    _logger.LogInformation(result);
    return result;
}

透过 jmeter 压力测试

$ .\jmeter -n -t .\shopping.jmx -l .\shopping.jtl
Creating summariser <summary>
Starting standalone 
Waiting for possible Shutdown/StopTestNow/HeapDump/ThreadDump message on port 4445
summary +      1 in 00:00:01 =    1.0/s Avg:    91 Min:    91 Max:    91 Err:     0 (0.00%) Active: 15 Started: 15 Finished: 0
summary +   1599 in 00:00:12 =  137.1/s Avg:    77 Min:     3 Max:  8921 Err:     0 (0.00%) Active: 0 Started: 16 Finished: 16
summary =   1600 in 00:00:13 =  126.8/s Avg:    77 Min:     3 Max:  8921 Err:     0 (0.00%)
Tidying up ...    @ Mon Oct 25 11:37:15 CST 2021 (1635133035562)
... end of run

结果如下：

在多台 server 上并没有出现卖超

RedLock 解说程序

里面最核心程序是 RedLock 类别．建构子会传入我们使用 Redis Connection

其中核心方式是

LockInstance
UnlockInstance

LockInstance

在 Lock 需要注意 Atomic 并且需要给一个 TTL 不然假如机器突然当机或跳电，会造成锁不会解放其他人会无限期 blocking

下面这段话是来是官网

since this is actually a viable solution in applications where a race condition from time to time is acceptable, and because locking into a single instance is the foundation we’ll use for the distributed algorithm described here.

SET resource_name my_random_value NX PX 30000

NX -- Only set the key if it does not already exist.

Redis 操作命令是一个 single Thread 执行所以命令如果可以包在一包或一条来执行就可以保证 Atomic，SET NX 命令不存在 key 建立数值 (具有 Atomic )

另外我为了避免一直在空转，我会判断假如目前有人占有锁我会自旋等待一个时间( lock TTL )，到了後在尝试访问（可以有更优的算法但我懒得写了）避免浪费资源空转

public class RedLock
{
    private IConnectionMultiplexer _connection;
    private string _keyId;

    public RedLock(IConnectionMultiplexer connection)
    {
        _connection = connection;
        _keyId = $"{Guid.NewGuid().ToString()}-Tid:{Thread.CurrentThread.ManagedThreadId}";
    }

    public bool LockInstance(string resource, TimeSpan ttl, out LockObject lockObject)
    {   
        bool result;
        lockObject = new LockObject(resource, _keyId, ttl);
        try
        {
        
            do
            {
            result = _connection.GetDatabase().StringSet(resource, _keyId, ttl, When.NotExists);
                
                if(!result)
                    WaitForLock(resource);

            } while (!result);
        }
        catch (Exception)
        {
            result = false;
        }

        return result;
    }

    private void WaitForLock(string resource)
    {
        var waitTime = _connection.GetDatabase().KeyTimeToLive(resource);

        if(waitTime.HasValue){
            SpinWait.SpinUntil(()=>true,waitTime.Value.Milliseconds);
        }
    }
}

UnlockInstance

我们必须让查询跟删除 keyid 有 Atomic 所以有两种做法

透过 Lua 脚本让命令连续
使用 Redis transaction 模式

本次程序解锁程序是靠 Lua 脚本来完成( Redis 官网推荐)

下面是官网的说明

This is important in order to avoid removing a lock that was created by another client. For example a client may acquire the lock, get blocked in some operation for longer than the lock validity time (the time at which the key will expire), and later remove the lock, that was already acquired by some other client. Using just DEL is not safe as a client may remove the lock of another client. With the above script instead every lock is “signed” with a random string, so the lock will be removed only if it is still the one that was set by the client trying to remove it.

建议在 Keyid 那边可以标示是由你产生的锁避免删除到别人 lock，所以我的程序 keyid 使用 GUID + ThreadId 来保证不会有人跟我产生一样的 keyId

public class RedLock
{
    private IConnectionMultiplexer _connection;
    private string _keyId;
    const string UNLOCK_SCRIPT = @"
        if redis.call(""get"",KEYS[1]) == ARGV[1] then
            return redis.call(""del"",KEYS[1])
        else
            return 0
        end";

    public RedLock(IConnectionMultiplexer connection)
    {
        _connection = connection;
        _keyId = $"{Guid.NewGuid().ToString()}-Tid:{Thread.CurrentThread.ManagedThreadId}";
    }

    public void UnlockInstance(string resource)
    {
        RedisKey[] keys = { resource };
        RedisValue[] values = { _keyId };
        _connection.GetDatabase().ScriptEvaluate(
            UNLOCK_SCRIPT,
            keys,
            values
            );
    }
}