简介

目前的.net 生态中,最终一致性组件的选择一直是一个问题。本地事务表(cap需要在每个服务的数据库中插入消息表,而且做不了此类事务 比如:创建订单需要 余额满足+库存满足,库存和余额处于两个服务中。masstransit 是我目前主要用的方案。以往一般都用 masstransit 中的 sagas 来实现 最终一致性,但是随着并发的增加必定会对sagas 持久化的数据库造成很大的压力,根据stackoverflow 中的一个回答 我发现了 一个用  Request/Response 与 Courier 功能 实现最终一致性的方案Demo地址

Masstransit 中 Resquest/Response 功能

消息DTO

    public class SampleMessageCommand
{
}

消费者

    public class SampleMessageCommandHandler : IConsumer<SampleMessageCommand>
{
public async Task Consume(ConsumeContext<SampleMessageCommand> context)
{
await context.RespondAsync(new SampleMessageCommandResult() { Data = "Sample" });
}
}

返回结果DTO

    public class SampleMessageCommandResult
{
public string Data { get; set; }
}

调用方式与注册方式略过,详情请看 官方文档

  

本质上使用消息队列实现 Resquest/Response,客户端(生产者)将请求消息发送至指定消息队列并赋予RequestId和ResponseAddress(临时队列 rabbitmq),服务端(消费者)消费消息并把 需要返回的消息放入指定ResponseAddress,客户端收到 Response message  通过匹配 RequestId 找到 指定Request,最后返回信息。

Masstransit 中 Courier  功能

通过有序组合一系列的Activity,得到一个routing slip。每个 activity(忽略 Execute Activities) 都有 Execute 和 Compensate 两个方法。Compensate 用来执撤销 Execute 方法产生的影响(就是回退 Execute 方法)。每个 Activity Execute 最后都会 调用 Completed 方法把 回退所需要的的信息记录在message中,最后持久化到消息队列的某一个消息中。

余额扣减的Activity ,这里的 DeductBalanceModel 是请求扣减的数据模型,DeductBalanceLog 是回退时需要用到的信息。

public class DeductBalanceActivity : IActivity<DeductBalanceModel, DeductBalanceLog>
{
private readonly ILogger<DeductBalanceActivity> logger;
public DeductBalanceActivity(ILogger<DeductBalanceActivity> logger)
{
this.logger = logger;
}
public async Task<CompensationResult> Compensate(CompensateContext<DeductBalanceLog> context)
{
logger.LogInformation("还原余额");
var log = context.Log; //可以获取 所有execute 完成时保存的信息
//throw new ArgumentException("some things were wrong");
return context.Compensated();
} public async Task<ExecutionResult> Execute(ExecuteContext<DeductBalanceModel> context)
{ logger.LogInformation("扣减余额");
await Task.Delay(100);
return context.Completed(new DeductBalanceLog() { Price = 100 });
}
}

扣减库存 Activity

    public class DeductStockActivity : IActivity<DeductStockModel, DeductStockLog>
{
private readonly ILogger<DeductStockActivity> logger;
public DeductStockActivity(ILogger<DeductStockActivity> logger)
{
this.logger = logger;
}
public async Task<CompensationResult> Compensate(CompensateContext<DeductStockLog> context)
{
var log = context.Log;
logger.LogInformation("还原库存");
return context.Compensated();
} public async Task<ExecutionResult> Execute(ExecuteContext<DeductStockModel> context)
{
var argument = context.Arguments;
logger.LogInformation("扣减库存");
await Task.Delay(100);
return context.Completed(new DeductStockLog() { ProductId = argument.ProductId, Amount = 1 });
}
}

生成订单 Execute Activity

    public class CreateOrderActivity : IExecuteActivity<CreateOrderModel>
{
private readonly ILogger<CreateOrderActivity> logger;
public CreateOrderActivity(ILogger<CreateOrderActivity> logger)
{
this.logger = logger;
}
public async Task<ExecutionResult> Execute(ExecuteContext<CreateOrderModel> context)
{
logger.LogInformation("创建订单");
await Task.Delay(100);
//throw new CommonActivityExecuteFaildException("当日订单已达到上限");
return context.CompletedWithVariables(new CreateOrderResult { OrderId="111122",Message="创建订单成功" });
}
}

  组装 以上 Activity 生成一个 Routing Slip,这是一个有序的组合,扣减库存=》扣减余额=》生成订单

            var builder = new RoutingSlipBuilder(NewId.NextGuid());
builder.AddActivity("DeductStock", new Uri($"{configuration["RabbitmqConfig:HostUri"]}/DeductStock_execute"), new DeductStockModel { ProductId = request.Message.ProductId }); builder.AddActivity("DeductBalance", new Uri($"{configuration["RabbitmqConfig:HostUri"]}/DeductBalance_execute"), new DeductBalanceModel { CustomerId = request.Message.CustomerId, Price = request.Message.Price }); builder.AddActivity("CreateOrder", new Uri($"{configuration["RabbitmqConfig:HostUri"]}/CreateOrder_execute"), new CreateOrderModel { Price = request.Message.Price, CustomerId = request.Message.CustomerId, ProductId = request.Message.ProductId });
var routingSlip = builder.Build();

  执行 Routing Slip

await bus.Execute(routingSlip);

 

这里是没有任何返回值的,所有activity都是 异步执行,虽然所有的activity可以执行完成或者由于某个Activity执行出错而全部回退。(其实这里有一种更坏的情况就是 Compensate 出错,默认情况下 Masstransit 只会发送一个回退错误的消息,后面讲到创建订单的时候我会把它塞到错误队列里,这样我们可以通过修改 Compensate bug后重新导入到正常队列来修正数据),这个功能完全满足不了 创建订单这个需求,执行 await bus.Execute(routingSlip) 后我们完全不知道订单到底创建成功,还是由于库存或余额不足而失败了(异步)。

还好 routing slip 在执行过程中产生很多消息,比如 RoutingSlipCompleted ,RoutingSlipCompensationFailed ,RoutingSlipActivityCompleted,RoutingSlipActivityFaulted 等,具体文档,我们可以订阅这些事件,再结合Request/Response 实现 创建订单的功能。

实现创建订单(库存满足+余额满足)长流程

创建订单 command

    /// <summary>
/// 长流程 分布式事务
/// </summary>
public class CreateOrderCommand
{
public string ProductId { get; set; }
public string CustomerId { get; set; }
public int Price { get; set; }
}

事务第一步,扣减库存相关 代码

  public class DeductStockActivity : IActivity<DeductStockModel, DeductStockLog>
{
private readonly ILogger<DeductStockActivity> logger;
public DeductStockActivity(ILogger<DeductStockActivity> logger)
{
this.logger = logger;
}
public async Task<CompensationResult> Compensate(CompensateContext<DeductStockLog> context)
{
var log = context.Log;
logger.LogInformation("还原库存");
return context.Compensated();
} public async Task<ExecutionResult> Execute(ExecuteContext<DeductStockModel> context)
{
var argument = context.Arguments;
logger.LogInformation("扣减库存");
await Task.Delay();
return context.Completed(new DeductStockLog() { ProductId = argument.ProductId, Amount = });
}
}
public class DeductStockModel
{
public string ProductId { get; set; }
}
public class DeductStockLog
{
public string ProductId { get; set; }
public int Amount { get; set; }
}

事务第二步,扣减余额相关代码

public class DeductBalanceActivity : IActivity<DeductBalanceModel, DeductBalanceLog>
{
private readonly ILogger<DeductBalanceActivity> logger;
public DeductBalanceActivity(ILogger<DeductBalanceActivity> logger)
{
this.logger = logger;
}
public async Task<CompensationResult> Compensate(CompensateContext<DeductBalanceLog> context)
{
logger.LogInformation("还原余额");
var log = context.Log;
//throw new ArgumentException("some things were wrong");
return context.Compensated();
} public async Task<ExecutionResult> Execute(ExecuteContext<DeductBalanceModel> context)
{ logger.LogInformation("扣减余额");
await Task.Delay();
return context.Completed(new DeductBalanceLog() { Price = });
}
}
public class DeductBalanceModel
{
public string CustomerId { get; set; }
public int Price { get; set; }
}
public class DeductBalanceLog
{
public int Price { get; set; }
}

事务第三步,创建订单相关代码

 public class CreateOrderActivity : IExecuteActivity<CreateOrderModel>
{
private readonly ILogger<CreateOrderActivity> logger;
public CreateOrderActivity(ILogger<CreateOrderActivity> logger)
{
this.logger = logger;
}
public async Task<ExecutionResult> Execute(ExecuteContext<CreateOrderModel> context)
{
logger.LogInformation("创建订单");
await Task.Delay();
//throw new CommonActivityExecuteFaildException("当日订单已达到上限");
return context.CompletedWithVariables(new CreateOrderResult { OrderId="",Message="创建订单成功" });
}
}
public class CreateOrderModel
{
public string ProductId { get; set; }
public string CustomerId { get; set; }
public int Price { get; set; }
}
public class CreateOrderResult
{
public string OrderId { get; set; }
public string Message { get; set; }
}

我通过 消费 创建订单 request,获取 request 的 response 地址与 RequestId,这两个值 返回 response 时需要用到,我把这些信息存到 RoutingSlip中,并且订阅 RoutingSlipEvents.Completed | RoutingSlipEvents.Faulted | RoutingSlipEvents.CompensationFailed 三种事件,当这三种消息出现时 我会根据 事件类别 和RoutingSlip中 之前加入的 (response 地址与 RequestId)生成 Response ,整个过程大概就是这么个意思,没理解可以看demo。这里由于每一个事物所需要用到的 RoutingSlip + Request/Response 步骤都类似 可以抽象一下(模板方法),把Activity 的组装 延迟到派生类去解决,这个代理类Masstransit有 ,但是官方没有顾及到 CompensationFailed 的情况,所以我干脆自己再写一个。

    public abstract class RoutingSlipDefaultRequestProxy<TRequest> :
IConsumer<TRequest>
where TRequest : class
{
public async Task Consume(ConsumeContext<TRequest> context)
{
var builder = new RoutingSlipBuilder(NewId.NextGuid()); builder.AddSubscription(context.ReceiveContext.InputAddress, RoutingSlipEvents.Completed | RoutingSlipEvents.Faulted | RoutingSlipEvents.CompensationFailed); builder.AddVariable("RequestId", context.RequestId);
builder.AddVariable("ResponseAddress", context.ResponseAddress);
builder.AddVariable("FaultAddress", context.FaultAddress);
builder.AddVariable("Request", context.Message); await BuildRoutingSlip(builder, context); var routingSlip = builder.Build(); await context.Execute(routingSlip).ConfigureAwait(false);
} protected abstract Task BuildRoutingSlip(RoutingSlipBuilder builder, ConsumeContext<TRequest> request);
}

这个 是派生类 Routing slip 的拼装过程

    public class CreateOrderRequestProxy : RoutingSlipDefaultRequestProxy<CreateOrderCommand>

    {
private readonly IConfiguration configuration;
public CreateOrderRequestProxy(IConfiguration configuration)
{
this.configuration = configuration;
}
protected override Task BuildRoutingSlip(RoutingSlipBuilder builder, ConsumeContext<CreateOrderCommand> request)
{
builder.AddActivity("DeductStock", new Uri($"{configuration["RabbitmqConfig:HostUri"]}/DeductStock_execute"), new DeductStockModel { ProductId = request.Message.ProductId }); builder.AddActivity("DeductBalance", new Uri($"{configuration["RabbitmqConfig:HostUri"]}/DeductBalance_execute"), new DeductBalanceModel { CustomerId = request.Message.CustomerId, Price = request.Message.Price }); builder.AddActivity("CreateOrder", new Uri($"{configuration["RabbitmqConfig:HostUri"]}/CreateOrder_execute"), new CreateOrderModel { Price = request.Message.Price, CustomerId = request.Message.CustomerId, ProductId = request.Message.ProductId }); return Task.CompletedTask;
}
}

构造response 基类,主要是对三种情况做处理。

    public abstract class RoutingSlipDefaultResponseProxy<TRequest, TResponse, TFaultResponse> : IConsumer<RoutingSlipCompensationFailed>, IConsumer<RoutingSlipCompleted>,
IConsumer<RoutingSlipFaulted>
where TRequest : class
where TResponse : class
where TFaultResponse : class
{
public async Task Consume(ConsumeContext<RoutingSlipCompleted> context)
{
var request = context.Message.GetVariable<TRequest>("Request");
var requestId = context.Message.GetVariable<Guid>("RequestId"); Uri responseAddress = null;
if (context.Message.Variables.ContainsKey("ResponseAddress"))
responseAddress = context.Message.GetVariable<Uri>("ResponseAddress"); if (responseAddress == null)
throw new ArgumentException($"The response address could not be found for the faulted routing slip: {context.Message.TrackingNumber}"); var endpoint = await context.GetResponseEndpoint<TResponse>(responseAddress, requestId).ConfigureAwait(false); var response = await CreateResponseMessage(context, request); await endpoint.Send(response).ConfigureAwait(false);
} public async Task Consume(ConsumeContext<RoutingSlipFaulted> context)
{
var request = context.Message.GetVariable<TRequest>("Request");
var requestId = context.Message.GetVariable<Guid>("RequestId"); Uri faultAddress = null;
if (context.Message.Variables.ContainsKey("FaultAddress"))
faultAddress = context.Message.GetVariable<Uri>("FaultAddress");
if (faultAddress == null && context.Message.Variables.ContainsKey("ResponseAddress"))
faultAddress = context.Message.GetVariable<Uri>("ResponseAddress"); if (faultAddress == null)
throw new ArgumentException($"The fault/response address could not be found for the faulted routing slip: {context.Message.TrackingNumber}"); var endpoint = await context.GetFaultEndpoint<TResponse>(faultAddress, requestId).ConfigureAwait(false); var response = await CreateFaultedResponseMessage(context, request, requestId); await endpoint.Send(response).ConfigureAwait(false);
}
public async Task Consume(ConsumeContext<RoutingSlipCompensationFailed> context)
{
var request = context.Message.GetVariable<TRequest>("Request");
var requestId = context.Message.GetVariable<Guid>("RequestId"); Uri faultAddress = null;
if (context.Message.Variables.ContainsKey("FaultAddress"))
faultAddress = context.Message.GetVariable<Uri>("FaultAddress");
if (faultAddress == null && context.Message.Variables.ContainsKey("ResponseAddress"))
faultAddress = context.Message.GetVariable<Uri>("ResponseAddress"); if (faultAddress == null)
throw new ArgumentException($"The fault/response address could not be found for the faulted routing slip: {context.Message.TrackingNumber}"); var endpoint = await context.GetFaultEndpoint<TResponse>(faultAddress, requestId).ConfigureAwait(false); var response = await CreateCompensationFaultedResponseMessage(context, request, requestId); await endpoint.Send(response).ConfigureAwait(false);
}
protected abstract Task<TResponse> CreateResponseMessage(ConsumeContext<RoutingSlipCompleted> context, TRequest request); protected abstract Task<TFaultResponse> CreateFaultedResponseMessage(ConsumeContext<RoutingSlipFaulted> context, TRequest request, Guid requestId);
protected abstract Task<TFaultResponse> CreateCompensationFaultedResponseMessage(ConsumeContext<RoutingSlipCompensationFailed> context, TRequest request, Guid requestId);
}

Response 派生类 ,这里逻辑可以随自己定义,我也是随便写了个 CommonResponse和一个业务错误抛错(牺牲了一点性能)。

    public class CreateOrderResponseProxy :
RoutingSlipDefaultResponseProxy<CreateOrderCommand, CommonCommandResponse<CreateOrderResult>, CommonCommandResponse<CreateOrderResult>>
{ protected override Task<CommonCommandResponse<CreateOrderResult>> CreateResponseMessage(ConsumeContext<RoutingSlipCompleted> context, CreateOrderCommand request)
{ return Task.FromResult(new CommonCommandResponse<CreateOrderResult>
{
Status = ,
Result = new CreateOrderResult
{
Message = context.Message.Variables.TryGetAndReturn(nameof(CreateOrderResult.Message))?.ToString(),
OrderId = context.Message.Variables.TryGetAndReturn(nameof(CreateOrderResult.OrderId))?.ToString(),
}
});
}
protected override Task<CommonCommandResponse<CreateOrderResult>> CreateFaultedResponseMessage(ConsumeContext<RoutingSlipFaulted> context, CreateOrderCommand request, Guid requestId)
{
var commonActivityExecuteFaildException = context.Message.ActivityExceptions.FirstOrDefault(m => m.ExceptionInfo.ExceptionType == typeof(CommonActivityExecuteFaildException).FullName);
if (commonActivityExecuteFaildException != null)
{
return Task.FromResult(new CommonCommandResponse<CreateOrderResult>
{
Status = ,
Message = commonActivityExecuteFaildException.ExceptionInfo.Message
});
}
// system error log here
return Task.FromResult(new CommonCommandResponse<CreateOrderResult>
{
Status = ,
Message = "System error"
});
} protected override Task<CommonCommandResponse<CreateOrderResult>> CreateCompensationFaultedResponseMessage(ConsumeContext<RoutingSlipCompensationFailed> context, CreateOrderCommand request, Guid requestId)
{
var exception = context.Message.ExceptionInfo;
// lg here context.Message.ExceptionInfo
return Task.FromResult(new CommonCommandResponse<CreateOrderResult>
{
Status = ,
Message = "System error"
});
}
}

对于  CompensationFailed 的处理 通过 ActivityCompensateErrorTransportFilter 实现 发送到错误消息队列,后续通过prometheus + rabbitmq-exporter + alertmanager 触发告警 通知相关人员处理。

  public class ActivityCompensateErrorTransportFilter<TActivity, TLog> : IFilter<CompensateActivityContext<TActivity, TLog>>
where TActivity : class, ICompensateActivity<TLog>
where TLog : class
{
public void Probe(ProbeContext context)
{
context.CreateFilterScope("moveFault");
} public async Task Send(CompensateActivityContext<TActivity, TLog> context, IPipe<CompensateActivityContext<TActivity, TLog>> next)
{
try
{
await next.Send(context).ConfigureAwait(false);
}
catch(Exception ex)
{
if (!context.TryGetPayload(out IErrorTransport transport))
throw new TransportException(context.ReceiveContext.InputAddress, $"The {nameof(IErrorTransport)} was not available on the {nameof(ReceiveContext)}.");
var exceptionReceiveContext = new RescueExceptionReceiveContext(context.ReceiveContext, ex);
await transport.Send(exceptionReceiveContext);
}
}
}

注册 filter

    public class RoutingSlipCompensateErrorSpecification<TActivity, TLog> : IPipeSpecification<CompensateActivityContext<TActivity, TLog>>
where TActivity : class, ICompensateActivity<TLog>
where TLog : class
{
public void Apply(IPipeBuilder<CompensateActivityContext<TActivity, TLog>> builder)
{
builder.AddFilter(new ActivityCompensateErrorTransportFilter<TActivity, TLog>());
} public IEnumerable<ValidationResult> Validate()
{
yield return this.Success("success");
}
} cfg.ReceiveEndpoint("DeductStock_compensate", ep =>
{
ep.PrefetchCount = ;
ep.CompensateActivityHost<DeductStockActivity, DeductStockLog>(context.Container, conf =>
{
conf.AddPipeSpecification(new RoutingSlipCompensateErrorSpecification<DeductStockActivity, DeductStockLog>());
}); });

实现创建产品(创建完成+添加库存)

实现了 创建订单的功能,整个流程其实是同步的,我在想能不能实现最为简单的最终一致性 比如 创建一个产品 ,然后异步生成它的库存 ,我发现是可以的,因为我们可以监听到每一个Execute Activity 的完成事件,并且把出错时的信息通过 filter 塞到 错误队列中。

这里的代码就不贴了,详情请看 demo

使用 Masstransit中的 Request/Response 与 Courier 功能实现最终一致性的更多相关文章

  1. python-django_rest_framework中的request/Response

    rest_framework中的request是被rest_framework再次封装过的,并在原request上添加了许多别的属性: (原Django中的request可用request._requ ...

  2. 在Struts2的Action中获得request response session几种方法

    转载自~ 在Struts2中,从Action中取得request,session的对象进行应用是开发中的必需步骤,那么如何从Action中取得这些对象呢?Struts2为我们提供了四种方式.分别为se ...

  3. struts2的action中获得request response session 对象

    在struts2中有两种方式可以得到这些对象 1.非IoC方式 要获得上述对象,关键Struts 2中com.opensymphony.xwork2.ActionContext类.我们可以通过它的静态 ...

  4. 过滤器中的chain.doFilter(request,response)

    Servlet中的过滤器Filter是实现了javax.servlet.Filter接口的服务器端程序,主要的用途是过滤字符编码.做一些业务逻辑判断等.其工作原理是,只要你在web.xml文件配置好要 ...

  5. Java 中的 request 和response 理解

    request和response(请求和响应)  1.当Web容器收到客户端的发送过来http请求,会针对每一次请求,分别创建一个用于代表此次请求的HttpServletRequest对象(reque ...

  6. LoadRunner中取Request、Response

    LoadRunner中取Request.Response LoadRunner两个“内置变量”: 1.REQUEST,用于提取完整的请求头信息. 2.RESPONSE,用于提取完整的响应头信息. 响应 ...

  7. ASP.NET中的Request、Response、Server对象

    Request对象 Response.Write(Request.ApplicationPath) //应用根路径 Request.AppRelativeCurrentExecutionFilePat ...

  8. struts2中获取request、response,与android客户端进行交互(文件传递给客户端)

    用struts2作为服务器框架,与android客户端进行交互需要得到request.response对象. struts2中获取request.response有两种方法. 第一种:利用Servle ...

  9. 【转】Django中的request与response对象

    关于request与response 前面几个 Sections 介绍了关于 Django 请求(Request)处理的流程分析,我们也了解到,Django 是围绕着 Request 与 Respon ...

随机推荐

  1. Linux、Android系统调用从上层到底层的调用路径浅析

    参考: https://blog.csdn.net/liuhangtiant/article/details/85149369 http://blog.sina.com.cn/s/blog_79433 ...

  2. JavaScript实现登录滑动验证

    来自于GitHub, 如何快速访问GitHub 先附上效果图 划到一半停止回自己回去的 PS: 附上代码,有需要自己更改, <!DOCTYPE html> <html lang=&q ...

  3. Java实现 LeetCode 221 最大正方形

    221. 最大正方形 在一个由 0 和 1 组成的二维矩阵内,找到只包含 1 的最大正方形,并返回其面积. 示例: 输入: 1 0 1 0 0 1 0 1 1 1 1 1 1 1 1 1 0 0 1 ...

  4. Linux 最大有效权限与删除ACL

    最大有效权限mask mask是用来指定最大有效权限的.如果给用户赋予了acl权限,则需要与mask权限”相与“才能得到用户的真正权限 setfacl -m m:rx 文件名,指定最大有效权限.例如: ...

  5. Linux RPM命令查询

    查看包是否安装 rpm -q 包名,其中,-q 表示查询 rpm -qa 表示查询所有已经安装的rpm包,a 表示所有 查询软件包详细信息 rpm -qi 包名,其中,-i 表示查询软件信息,-p 表 ...

  6. Vue封装公共组件TarBar

    github:https://github.com/zwnsyw/TabBar 一.实现简单思路 1.在页面底部有一个单独的TabBar组件1.1自定义TabBar组件,在APP中使用1.2让TabB ...

  7. 提高网站访问性能之Tomcat优化

    一.前言 tomcat 服务器在JavaEE项目中使用率非常高,所以在生产环境对tomcat的优化也变得非常重要了,对于tomcat的优化,主要是从2个方面入手,一是tomcat本身的配置,另一个是t ...

  8. multimap遍历与查找

    std::multimap<int, std::string> m; m.insert(std::make_pair(0, "w0")); m.insert(std:: ...

  9. flutter-web利用dart js 库发起http request

    初学flutter,初学前端,尝试在dart中直接使用HttpClient时,直接报出Platform not supported,查资料发现他还不支持浏览器. 通过查阅资料发现可以借助axios 与 ...

  10. c常用函数-strlen

    strlen 返回字符串长度 Action() { char test[] = "yyyHHHJJJoo"; unsigned int a; a = strlen(test); l ...