netflix turbine概述
1.turbine是什么?它的作用是什么?
Turbine is a tool for aggregating streams of Server-Sent Event (SSE) JSON data into a single stream. The targeted use case is metrics streams from instances in an SOA being aggregated for dashboards.
For example, Netflix uses Hystrix which has a realtime dashboard that uses Turbine to aggregate data from 100s or 1000s of machines.
2.eureka启动turbine类StartEurekaTurbine
public static void main(String[] args) {
OptionParser optionParser = new OptionParser();
optionParser.accepts("port").withRequiredArg();
optionParser.accepts("app").withRequiredArg();
optionParser.accepts("urlTemplate").withRequiredArg(); OptionSet options = optionParser.parse(args);
int port = -1;
if (!options.has("port")) {
System.err.println("Argument -port required for SSE HTTP server to start on. Eg. -port 8888");
System.exit(-1);
} else {
try {
port = Integer.parseInt(String.valueOf(options.valueOf("port")));
} catch (NumberFormatException e) {
System.err.println("Value of port must be an integer but was: " + options.valueOf("port"));
}
} String app = null;
if (!options.has("app")) {
System.err.println("Argument -app required for Eureka instance discovery. Eg. -app api");
System.exit(-1);
} else {
app = String.valueOf(options.valueOf("app"));
} String template = null;
if (!options.has("urlTemplate")) {
System.err.println("Argument -urlTemplate required. Eg. http://" + EurekaStreamDiscovery.HOSTNAME + "/metrics.stream");
System.exit(-1);
} else {
template = String.valueOf(options.valueOf("urlTemplate"));
if (!template.contains(EurekaStreamDiscovery.HOSTNAME)) {
System.err.println("Argument -urlTemplate must contain " + EurekaStreamDiscovery.HOSTNAME + " marker. Eg. http://" + EurekaStreamDiscovery.HOSTNAME + "/metrics.stream");
System.exit(-1);
}
} logger.info("Turbine => Eureka App: " + app);
logger.info("Turbine => Eureka URL Template: " + template); try {
Turbine.startServerSentEventServer(port, EurekaStreamDiscovery.create(app, template));
} catch (Throwable e) {
e.printStackTrace();
}
}
执行类如下;
startServerSentEventServer(port, aggregateHttpSSE(discovery));
首先,聚合http
/**
* Aggregate multiple HTTP Server-Sent Event streams into one stream with the values summed.
* <p>
* The returned data must be JSON data that contains the following keys:
* <p>
* instanceId => Unique instance representing each stream to be merged, such as the instanceId of the server the stream is from.
* type => The type of data such as HystrixCommand or HystrixThreadPool if aggregating Hystrix metrics.
* name => Name of a group of metrics to be aggregated, such as a HystrixCommand name if aggregating Hystrix metrics.
*
* @param uri
* @return
*/
public static Observable<GroupedObservable<TypeAndNameKey, Map<String, Object>>> aggregateHttpSSE(StreamDiscovery discovery) {
Observable<StreamAction> streamActions = discovery.getInstanceList().publish().refCount();
Observable<StreamAction> streamAdds = streamActions.filter(a -> a.getType() == ActionType.ADD);
Observable<StreamAction> streamRemoves = streamActions.filter(a -> a.getType() == ActionType.REMOVE); Observable<GroupedObservable<InstanceKey, Map<String, Object>>> streamPerInstance =
streamAdds.map(streamAction -> {
URI uri = streamAction.getUri(); Observable<Map<String, Object>> io = Observable.defer(() -> {
Observable<Map<String, Object>> flatMap = RxNetty.createHttpClient(uri.getHost(), uri.getPort(), PipelineConfigurators.<ByteBuf>sseClientConfigurator())
.submit(createRequest(uri))
.flatMap(response -> {
if (response.getStatus().code() != 200) {
return Observable.error(new RuntimeException("Failed to connect: " + response.getStatus()));
}
return response.getContent()
.doOnSubscribe(() -> logger.info("Turbine => Aggregate Stream from URI: " + uri.toASCIIString()))
.doOnUnsubscribe(() -> logger.info("Turbine => Unsubscribing Stream: " + uri))
.takeUntil(streamRemoves.filter(a -> a.getUri().equals(streamAction.getUri()))) // unsubscribe when we receive a remove event
.map(sse -> JsonUtility.jsonToMap(sse.getEventData()));
});
// eclipse is having issues with type inference so breaking up
return flatMap.retryWhen(attempts -> {
return attempts.flatMap(e -> {
return Observable.timer(1, TimeUnit.SECONDS)
.doOnEach(n -> logger.info("Turbine => Retrying connection to: " + uri));
});
}); }); return GroupedObservable.from(InstanceKey.create(uri.toASCIIString()), io);
}); return StreamAggregator.aggregateGroupedStreams(streamPerInstance);
}
然后启动聚合
public static void startServerSentEventServer(int port, Observable<GroupedObservable<TypeAndNameKey, Map<String, Object>>> streams) {
logger.info("Turbine => Starting server on " + port); // multicast so multiple concurrent subscribers get the same stream
Observable<Map<String, Object>> publishedStreams = streams
.doOnUnsubscribe(() -> logger.info("Turbine => Unsubscribing aggregation."))
.doOnSubscribe(() -> logger.info("Turbine => Starting aggregation"))
.flatMap(o -> o).publish().refCount(); RxNetty.createHttpServer(port, (request, response) -> {
logger.info("Turbine => SSE Request Received");
response.getHeaders().setHeader("Content-Type", "text/event-stream");
return publishedStreams
.doOnUnsubscribe(() -> logger.info("Turbine => Unsubscribing RxNetty server connection"))
.flatMap(data -> {
return response.writeAndFlush(new ServerSentEvent(null, null, JsonUtility.mapToJson(data)));
});
}, PipelineConfigurators.<ByteBuf>sseServerConfigurator()).startAndWait();
}
3.单独启动turbine的过程和上面类似StartTurbine
public static void main(String[] args) {
OptionParser optionParser = new OptionParser();
optionParser.accepts("port").withRequiredArg();
optionParser.accepts("streams").withRequiredArg(); OptionSet options = optionParser.parse(args);
int port = -1;
if (!options.has("port")) {
System.err.println("Argument -port required for SSE HTTP server to start on.");
System.exit(-1);
} else {
try {
port = Integer.parseInt(String.valueOf(options.valueOf("port")));
} catch (NumberFormatException e) {
System.err.println("Value of port must be an integer but was: " + options.valueOf("port"));
}
} URI[] streams = null;
if (!options.hasArgument("streams")) {
System.err.println("Argument -streams required with URIs to connect to. Eg. -streams \"http://host1/metrics.stream http://host2/metrics.stream\"");
System.exit(-1);
} else {
String streamsArg = String.valueOf(options.valueOf("streams"));
String[] ss = streamsArg.split(" ");
streams = new URI[ss.length];
for (int i = 0; i < ss.length; i++) {
try {
streams[i] = new URI(ss[i]);
} catch (URISyntaxException e) {
System.err.println("ERROR: Could not parse stream into URI: " + ss[i]);
System.exit(-1);
}
}
} if (streams == null || streams.length == 0) {
System.err.println("There must be at least 1 valid stream URI.");
System.exit(-1);
} try {
Turbine.startServerSentEventServer(port, Turbine.aggregateHttpSSE(streams));
} catch (Throwable e) {
e.printStackTrace();
}
}
netflix turbine概述的更多相关文章
- netflix ribbon概述
LB方案分类 目前主流的LB方案可分成两类:一种是集中式LB, 即在服务的消费方和提供方之间使用独立的LB设施(可以是硬件,如F5, 也可以是软件,如nginx), 由该设施负责把访问请求通过某种策略 ...
- netflix feign概述
1.什么是feign?feign的作用是什么? Feign is a java to http client binder inspired by Retrofit, JAXRS-2.0, and W ...
- 第二十六章 hystrix-dashboard + turbine
一.使用turbine的意义 引入多个hystrix stream: 1.使用hystrix-dashboard的可以添加多个stream的功能 图中添加的两个stream会在真正monitor的时候 ...
- 附7 turbine
一.作用 聚集同一个微服务的相同的commandKey.Threadpool.commandGroupKey数据进行聚合 二.配置 1.集群(cluster)(turbine聚集数据的粒度) turb ...
- Building microservices with Spring Cloud and Netflix OSS, part 2
In Part 1 we used core components in Spring Cloud and Netflix OSS, i.e. Eureka, Ribbon and Zuul, to ...
- 基于Spring Cloud和Netflix OSS构建微服务,Part 2
在上一篇文章中,我们已使用Spring Cloud和Netflix OSS中的核心组件,如Eureka.Ribbon和Zuul,部分实现了操作模型(operations model),允许单独部署的微 ...
- spring cloud熔断监控Hystrix Dashboard和Turbine
参考: http://blog.csdn.net/ityouknow/article/details/72625646 完整pom <?xml version="1.0" e ...
- SpringCloud系列七:Hystrix 熔断机制(Hystrix基本配置、服务降级、HystrixDashboard服务监控、Turbine聚合监控)
1.概念:Hystrix 熔断机制 2.具体内容 所谓的熔断机制和日常生活中见到电路保险丝是非常相似的,当出现了问题之后,保险丝会自动烧断,以保护我们的电器, 那么如果换到了程序之中呢? 当现在服务的 ...
- Spring Cloud 入门教程(八): 断路器指标数据监控Hystrix Dashboard 和 Turbine
1. Hystrix Dashboard (断路器:hystrix 仪表盘) Hystrix一个很重要的功能是,可以通过HystrixCommand收集相关数据指标. Hystrix Dashboa ...
随机推荐
- Linux bash常用快捷键
移动光标 ctrl-a 光标移动到行首 ctrl-e 光标移动到行尾 ctrl+xx 在行首和光标位置直接切换 ctrl-b 光标左移一位 ctrl-f 光标右移一位 alt-b 光标左移一词 alt ...
- Python 语言中经常有疑惑的地方
*)关于for循环中range(2),i到底是从0还是1开始.特别是在用数组的长度作为range的参数的时候经常会犯糊涂 #首先 >>> for i in range(5): ... ...
- 紫书 例题11-10 UVa 1349 (二分图最小权完美匹配)
二分图网络流做法 (1)最大基数匹配.源点到每一个X节点连一条容量为1的弧, 每一个Y节点连一条容量为1的弧, 然后每条有向 边连一条弧, 容量为1, 然后跑一遍最大流即可, 最大流即是最大匹配对数 ...
- MySQL中将数据库表名修改成大写的存储过程
原文:MySQL中将数据库表名修改成大写的存储过程 MySQL中将数据库表名修改成大写的存储过程 创建存储过程的代码: DROP PROCEDURE IF EXISTS uppercaseTablen ...
- C语言编译和链接
编译链接是使用高级语言编程所必须的操作,一个源程序只有经过编译.链接操作以后才可以变成计算机可以理解并执行的二进制可执行文件. 编译是指根据用户写的源程序代码,经过词法和语法分析,将高级语言编写的代码 ...
- [terry笔记]data guard基础知识
如下介绍了data guard的基础知识,整理自网络: Data Gurad 通过冗余数据来提供数据保护,Data Gurad 通过日志同步机制保证冗余数据和主数据之前的同步,这种同步可以是实时,延时 ...
- 在Windows Server 2008 R2中删除网桥
How to remove a network bridge in Windows Server 2008 R2 症状: 删除网桥的时候,按理说应该在“网络连接”中选择要被删除的网桥,右键点击,然后选 ...
- ie11 .pac代理脚本无法使用的问题
参考: http://blogs.msdn.com/b/ieinternals/archive/2013/10/11/web-proxy-configuration-and-ie11-changes. ...
- ActiveMQ maven
http://outofmemory.cn/java/mq/apache-activemq-demo
- hpuoj--校赛--爬楼梯(模拟)
问题 E: 感恩节KK专场--爬楼梯 时间限制: 1 Sec 内存限制: 1000 MB 提交: 382 解决: 89 [提交][状态][讨论版] 题目描述 来机房比赛的时候大家都会爬楼梯,但是每 ...