netflix turbine概述
1.turbine是什么?它的作用是什么?
Turbine is a tool for aggregating streams of Server-Sent Event (SSE) JSON data into a single stream. The targeted use case is metrics streams from instances in an SOA being aggregated for dashboards.
For example, Netflix uses Hystrix which has a realtime dashboard that uses Turbine to aggregate data from 100s or 1000s of machines.
2.eureka启动turbine类StartEurekaTurbine
public static void main(String[] args) {
OptionParser optionParser = new OptionParser();
optionParser.accepts("port").withRequiredArg();
optionParser.accepts("app").withRequiredArg();
optionParser.accepts("urlTemplate").withRequiredArg(); OptionSet options = optionParser.parse(args);
int port = -1;
if (!options.has("port")) {
System.err.println("Argument -port required for SSE HTTP server to start on. Eg. -port 8888");
System.exit(-1);
} else {
try {
port = Integer.parseInt(String.valueOf(options.valueOf("port")));
} catch (NumberFormatException e) {
System.err.println("Value of port must be an integer but was: " + options.valueOf("port"));
}
} String app = null;
if (!options.has("app")) {
System.err.println("Argument -app required for Eureka instance discovery. Eg. -app api");
System.exit(-1);
} else {
app = String.valueOf(options.valueOf("app"));
} String template = null;
if (!options.has("urlTemplate")) {
System.err.println("Argument -urlTemplate required. Eg. http://" + EurekaStreamDiscovery.HOSTNAME + "/metrics.stream");
System.exit(-1);
} else {
template = String.valueOf(options.valueOf("urlTemplate"));
if (!template.contains(EurekaStreamDiscovery.HOSTNAME)) {
System.err.println("Argument -urlTemplate must contain " + EurekaStreamDiscovery.HOSTNAME + " marker. Eg. http://" + EurekaStreamDiscovery.HOSTNAME + "/metrics.stream");
System.exit(-1);
}
} logger.info("Turbine => Eureka App: " + app);
logger.info("Turbine => Eureka URL Template: " + template); try {
Turbine.startServerSentEventServer(port, EurekaStreamDiscovery.create(app, template));
} catch (Throwable e) {
e.printStackTrace();
}
}
执行类如下;
startServerSentEventServer(port, aggregateHttpSSE(discovery));
首先,聚合http
/**
* Aggregate multiple HTTP Server-Sent Event streams into one stream with the values summed.
* <p>
* The returned data must be JSON data that contains the following keys:
* <p>
* instanceId => Unique instance representing each stream to be merged, such as the instanceId of the server the stream is from.
* type => The type of data such as HystrixCommand or HystrixThreadPool if aggregating Hystrix metrics.
* name => Name of a group of metrics to be aggregated, such as a HystrixCommand name if aggregating Hystrix metrics.
*
* @param uri
* @return
*/
public static Observable<GroupedObservable<TypeAndNameKey, Map<String, Object>>> aggregateHttpSSE(StreamDiscovery discovery) {
Observable<StreamAction> streamActions = discovery.getInstanceList().publish().refCount();
Observable<StreamAction> streamAdds = streamActions.filter(a -> a.getType() == ActionType.ADD);
Observable<StreamAction> streamRemoves = streamActions.filter(a -> a.getType() == ActionType.REMOVE); Observable<GroupedObservable<InstanceKey, Map<String, Object>>> streamPerInstance =
streamAdds.map(streamAction -> {
URI uri = streamAction.getUri(); Observable<Map<String, Object>> io = Observable.defer(() -> {
Observable<Map<String, Object>> flatMap = RxNetty.createHttpClient(uri.getHost(), uri.getPort(), PipelineConfigurators.<ByteBuf>sseClientConfigurator())
.submit(createRequest(uri))
.flatMap(response -> {
if (response.getStatus().code() != 200) {
return Observable.error(new RuntimeException("Failed to connect: " + response.getStatus()));
}
return response.getContent()
.doOnSubscribe(() -> logger.info("Turbine => Aggregate Stream from URI: " + uri.toASCIIString()))
.doOnUnsubscribe(() -> logger.info("Turbine => Unsubscribing Stream: " + uri))
.takeUntil(streamRemoves.filter(a -> a.getUri().equals(streamAction.getUri()))) // unsubscribe when we receive a remove event
.map(sse -> JsonUtility.jsonToMap(sse.getEventData()));
});
// eclipse is having issues with type inference so breaking up
return flatMap.retryWhen(attempts -> {
return attempts.flatMap(e -> {
return Observable.timer(1, TimeUnit.SECONDS)
.doOnEach(n -> logger.info("Turbine => Retrying connection to: " + uri));
});
}); }); return GroupedObservable.from(InstanceKey.create(uri.toASCIIString()), io);
}); return StreamAggregator.aggregateGroupedStreams(streamPerInstance);
}
然后启动聚合
public static void startServerSentEventServer(int port, Observable<GroupedObservable<TypeAndNameKey, Map<String, Object>>> streams) {
logger.info("Turbine => Starting server on " + port); // multicast so multiple concurrent subscribers get the same stream
Observable<Map<String, Object>> publishedStreams = streams
.doOnUnsubscribe(() -> logger.info("Turbine => Unsubscribing aggregation."))
.doOnSubscribe(() -> logger.info("Turbine => Starting aggregation"))
.flatMap(o -> o).publish().refCount(); RxNetty.createHttpServer(port, (request, response) -> {
logger.info("Turbine => SSE Request Received");
response.getHeaders().setHeader("Content-Type", "text/event-stream");
return publishedStreams
.doOnUnsubscribe(() -> logger.info("Turbine => Unsubscribing RxNetty server connection"))
.flatMap(data -> {
return response.writeAndFlush(new ServerSentEvent(null, null, JsonUtility.mapToJson(data)));
});
}, PipelineConfigurators.<ByteBuf>sseServerConfigurator()).startAndWait();
}
3.单独启动turbine的过程和上面类似StartTurbine
public static void main(String[] args) {
OptionParser optionParser = new OptionParser();
optionParser.accepts("port").withRequiredArg();
optionParser.accepts("streams").withRequiredArg(); OptionSet options = optionParser.parse(args);
int port = -1;
if (!options.has("port")) {
System.err.println("Argument -port required for SSE HTTP server to start on.");
System.exit(-1);
} else {
try {
port = Integer.parseInt(String.valueOf(options.valueOf("port")));
} catch (NumberFormatException e) {
System.err.println("Value of port must be an integer but was: " + options.valueOf("port"));
}
} URI[] streams = null;
if (!options.hasArgument("streams")) {
System.err.println("Argument -streams required with URIs to connect to. Eg. -streams \"http://host1/metrics.stream http://host2/metrics.stream\"");
System.exit(-1);
} else {
String streamsArg = String.valueOf(options.valueOf("streams"));
String[] ss = streamsArg.split(" ");
streams = new URI[ss.length];
for (int i = 0; i < ss.length; i++) {
try {
streams[i] = new URI(ss[i]);
} catch (URISyntaxException e) {
System.err.println("ERROR: Could not parse stream into URI: " + ss[i]);
System.exit(-1);
}
}
} if (streams == null || streams.length == 0) {
System.err.println("There must be at least 1 valid stream URI.");
System.exit(-1);
} try {
Turbine.startServerSentEventServer(port, Turbine.aggregateHttpSSE(streams));
} catch (Throwable e) {
e.printStackTrace();
}
}
netflix turbine概述的更多相关文章
- netflix ribbon概述
LB方案分类 目前主流的LB方案可分成两类:一种是集中式LB, 即在服务的消费方和提供方之间使用独立的LB设施(可以是硬件,如F5, 也可以是软件,如nginx), 由该设施负责把访问请求通过某种策略 ...
- netflix feign概述
1.什么是feign?feign的作用是什么? Feign is a java to http client binder inspired by Retrofit, JAXRS-2.0, and W ...
- 第二十六章 hystrix-dashboard + turbine
一.使用turbine的意义 引入多个hystrix stream: 1.使用hystrix-dashboard的可以添加多个stream的功能 图中添加的两个stream会在真正monitor的时候 ...
- 附7 turbine
一.作用 聚集同一个微服务的相同的commandKey.Threadpool.commandGroupKey数据进行聚合 二.配置 1.集群(cluster)(turbine聚集数据的粒度) turb ...
- Building microservices with Spring Cloud and Netflix OSS, part 2
In Part 1 we used core components in Spring Cloud and Netflix OSS, i.e. Eureka, Ribbon and Zuul, to ...
- 基于Spring Cloud和Netflix OSS构建微服务,Part 2
在上一篇文章中,我们已使用Spring Cloud和Netflix OSS中的核心组件,如Eureka.Ribbon和Zuul,部分实现了操作模型(operations model),允许单独部署的微 ...
- spring cloud熔断监控Hystrix Dashboard和Turbine
参考: http://blog.csdn.net/ityouknow/article/details/72625646 完整pom <?xml version="1.0" e ...
- SpringCloud系列七:Hystrix 熔断机制(Hystrix基本配置、服务降级、HystrixDashboard服务监控、Turbine聚合监控)
1.概念:Hystrix 熔断机制 2.具体内容 所谓的熔断机制和日常生活中见到电路保险丝是非常相似的,当出现了问题之后,保险丝会自动烧断,以保护我们的电器, 那么如果换到了程序之中呢? 当现在服务的 ...
- Spring Cloud 入门教程(八): 断路器指标数据监控Hystrix Dashboard 和 Turbine
1. Hystrix Dashboard (断路器:hystrix 仪表盘) Hystrix一个很重要的功能是,可以通过HystrixCommand收集相关数据指标. Hystrix Dashboa ...
随机推荐
- 企业级URL检测脚本
#!/bin/bash #检查web url 是否正常 . /etc/init.d/functions #引用系统函数库 #判断传参个数是否为一个 function usage(){ echo $&q ...
- C# winform压缩文件夹带进度条
注意:用了开源的CL.IO.Zip库 pbYSJD是进度条的控件名 btnImport是按钮控件名,当压缩结束之后,使按钮处于激活状态,否则无法点击按钮. /// <summary> // ...
- Qt之QSS(动态属性)
简述 QSS可以定制应用程序的外观,无需关注Qt样式背后的魔力.从非常轻微到极其复杂的调整,样式表都可以做到.对于一个完全定制和独特的用户体验,QtQuick和QGraphicsView是更好的选择. ...
- 【RQNOJ】460 诺诺的队列
[题目大意] 求全部数对(i,j)满足随意a[k]<=a[i]且a[k]<=a[j]. 形象地说,就是有一群人站成一列.每一个人有一定的身高,然后问有多少对人能够互相看得到. 把数对(i, ...
- angularjs ng-app
<!DOCTYPE HTML> <html ng-app> //ng-app是初始化指令,整个页面都会被angularjs解析,写在div或者其他标签上表示只是局部的div和标 ...
- PHP 二维数组去掉重复值并保持原结构
PHP 二维数组去掉重复值并保持原结构 直接上代码,解释很详细 //二维数组去掉重复值 function arrunique($a){ foreach($a[0] as $k => $v){ / ...
- ARM嵌入式复习
第一章 1.嵌入式系统 “以应用为中心,以计算机技术为基础,软硬件可裁剪,适用于应用系统对功能,可靠性,成本,体积,功耗要求严格的专用计算机系统” 根据应用的要求,沿着“体积小,低功耗,高可靠”方向发 ...
- python 3.x 写 shell sed 替换功能
shell sed 替换功能 f1 = open('yesterday','r',encoding='utf-8') f2 = open('yesterday2','w',encoding='utf- ...
- jQuery新浪微博表情插件教程
1.引入css文件 <link rel="stylesheet" type="text/css" href="jquery.sinaEmotio ...
- js parseFloat 精度问题
<script type="text/javascript"> //parseFloat function actionoftext(){var price = 10. ...