SSIS 阻塞,半阻塞和全阻塞 (Non-blocking, semi-blocking and Fully-blocking) transformations清单
三种Blocking类型,这里跟数据流的Buff关系很大:
■■ non-blocking transformations,每一行直接转换输出,没有等待.
■■ partial-blocking transformation,直到存储一定数量的好才输出。
■■ blocking transformation, 在输出前必须先读入所有行。
Logical Row-Level Transformations
Data flow transformation | purpose | blocking type |
Audit | Adds additional columns to each row based on sys- tem package variables such as ExecutionStartTime and PackageName. | N |
Cache Transform | Allows you to write data to a cache with the Cache con- nection manager. The data can then be used by the Lookup transformation. This is useful if you are using multiple Lookup transformations against the same data, because SSIS will cache the needed data only once and not for each Lookup component. | N |
Character Map | Performs common text operations such as Uppercase and allows advanced linguistic bit-conversion operations. | N |
Copy Column | Duplicates column values in each row to a new named column. | N |
Data Conversion | Creates a new column in each row based on a new data type converted from the existing column. An example is converting text to numeric data or text to Unicode text. | N |
Data flow transformation | purpose | blocking type |
Derived Column | Creates or replaces a column for each row based on a specified SSIS expression. This is the most often used logical row-level transformation because it enables the replacement of column values or the creation of new columns based on existing columns, variables, and parameters. |
N |
Export Column | Exports binary large objects (BLOB) columns, one row at a time, to a file. | N |
Import Column | Loads binary files such as images into the pipeline; intend- ed for a BLOB data type destination. | N |
Row Count | Tracks the number of rows that flow through the transfor- mation and stores the number in a package variable after the final row. | N |
Multi-Input and Multi-Output Transformations
Data flow transformation | purpose | blocking type |
CDC Splitter | Splits a single flow of changed rows from the CDC source component into multiple data flows based on the type of the source data change (that is, whether it is an insert, update, or delete operation). CDC Splitter routes the data based on the __$operation column into three possible outputs. this transformation is like a specific version of the Conditional Split transformation that automatically handles the standard values of the __$operation column. |
N |
Conditional Split | Routes or filters data based on a Boolean expression to one or more outputs, from which each row can be sent out only one output path. | N |
Lookup | Performs a lookup operation between a current row and an external dataset on one or more columns. Additional columns can be added to the data flow from the external dataset. | N |
Merge | Combines the rows of two similar sorted inputs, one on top of the other, based on a defined sort key. | P |
Merge Join | Joins the rows of two sorted inputs based on a defined join column or columns, adding columns from each source. | P |
Multicast | Generates one or mode identical outputs, from which every row is sent out every output. This transformation creates a logical copy of the data. | N |
Union All | Combines one or more similar inputs, stacking rows one on top of another, based on matching columns. The number of rows in the output of Union All is the combined row counts of all the inputs. | P |
Multi-Row Transformations
Data flow transformation | purpose | blocking type |
Aggregate | Associates rows based on defined grouping and generates aggregations such as SUM, MAX, MIN, and COUNT. | B |
Percent Sampling | Filters the input rows by allowing only a defined percent to be passed to the output path. | N |
Pivot | Takes multiple input rows and pivots the rows to generate an output with more columns based on the original row values. | P |
Row Sampling | Generates a fixed number of rows, sampling the data from the entire input, no matter how much larger than the defined output the input is. | B |
Sort | Orders the input based on defined sort columns and sort directions. The Sort transformation also allows the removal of duplicates across the sort columns. | B |
Unpivot | Takes a single row and generates multiple rows, moving column values to the new row based on defined columns. | P |
Advanced Data-Preparation Transformations
Data flow transformation | purpose | blocking type |
DQS Cleansing | Validates rows by automatically per- forming data cleansing using an exist- ing knowledge base in Data Quality Services (DQS). | P |
OLE DB Command | Performs database operations such as updates or deletions, one row at a time, based on mapped parameters from input rows. | N |
Slowly Changing Dimension | Generates transformations necessary to support loading dimension tables in data warehouse scenarios. This transformation handles SCD (Slowly Changing Dimension) Type 1 and Type 2 and also has support for inferred members. Chapter 7 focuses on this transformation. | N |
Data Mining Query | Applies input rows against a data min- ing model for prediction. | P |
Fuzzy Grouping | Performs de-duplication based on similarity of string values in selected columns. | B |
Fuzzy Lookup | Joins a data flow input to a reference table based on column similarity. The Similarity Threshold setting specifies the closeness of allowed matches—a high setting means that matching val- ues are close in similarity. | B |
Script Component | Applies custom .NET scripting capabilities against rows, columns, inputs, and outputs in the data flow pipeline. This is the most powerful component. Chapter 19, “Implementing Custom Code in SSIS Packages” looks at some of its possibilities. | N |
Term Extraction | Analyzes text input columns for English-language nouns and noun phrases. | P |
Term Lookup | Analyzes text input columns against a user-defined set of words for associa- tion. | P |
SSIS 阻塞,半阻塞和全阻塞 (Non-blocking, semi-blocking and Fully-blocking) transformations清单的更多相关文章
- 微软BI 之SSIS 系列 - 理解Data Flow Task 中的同步与异步, 阻塞,半阻塞和全阻塞以及Buffer 缓存概念
开篇介绍 在 SSIS Dataflow 数据流中的组件可以分为 Synchronous 同步和 Asynchronous 异步这两种类型. 同步与异步 Synchronous and Asynchr ...
- php使用flock阻塞写入文件和非阻塞写入文件的实例讲解
php使用flock阻塞写入文件和非阻塞写入文件的实例讲解: 阻塞写入代码:(所有程序会等待上次程序执行结束才会执行,30秒会超时) <?php $file = fopen("test ...
- 同步I/O、异步I/O与阻塞I/O、非阻塞I/O的区别
一.I/O I/O (Input/Output,输入/输出)即数据的读取(接收)或写入(发送)操作. 通常用户进程中的一个完整I/O分为两阶段:用户进程空间<-->内核空间.内核空间< ...
- (原创)JAVA阻塞队列LinkedBlockingQueue 以及非阻塞队列ConcurrentLinkedQueue 的区别
阻塞队列:线程安全 按 FIFO(先进先出)排序元素.队列的头部 是在队列中时间最长的元素.队列的尾部 是在队列中时间最短的元素.新元素插入到队列的尾部,并且队列检索操作会获得位于队列头部的元素.链接 ...
- 深入理解非阻塞同步IO和非阻塞异步IO
这两篇文章分析了Linux下的5种IO模型 http://blog.csdn.net/historyasamirror/article/details/5778378 http://blog.csdn ...
- java 多线程阻塞队列 与 阻塞方法与和非阻塞方法
Queue是什么 队列,是一种数据结构.除了优先级队列和LIFO队列外,队列都是以FIFO(先进先出)的方式对各个元素进行排序的.无论使用哪种排序方式,队列的头都是调用remove()或poll()移 ...
- 定时器详解和应用、js加载阻塞、css加载阻塞
1.setTimeout().setInterval()详解和应用 1.1 详解: setTimeout.setInterval执行时机 1.2 存在问题: setInterval重复定时器可能存在的 ...
- Python之阻塞IO模型与非阻塞IO模型
Python之阻塞IO模型与非阻塞IO模型 IO模型 1 阻塞IO: 全程阻塞 2 非阻塞IO: 发送多次系统调用: 优点:wait for data时无阻塞 缺点:1 系统调用太多 2 数据不是实时 ...
- IO阻塞模型、IO非阻塞模型、多路复用IO模型
IO操作主要包括两类: 本地IO 网络IO 本地IO:本地IO是指本地的文件读取等操作,本地IO的优化主要是在操作系统中进行,我们对于本地IO的优化作用十分有限 网络IO:网络IO指的是在进行网络操作 ...
- 阻塞I/O、非阻塞I/O和I/O多路复用、怎样理解阻塞非阻塞与同步异步的区别?
“阻塞”与"非阻塞"与"同步"与“异步"不能简单的从字面理解,提供一个从分布式系统角度的回答.1.同步与异步 同步和异步关注的是消息通信机制 (syn ...
随机推荐
- Web颜色搭配 - 收集
颜色1 颜色一 背景 字 RGB 43,41,46 92,187,207 HEX #2B292E #5CBBCF HSB 264,11,18 190,56,81 CMYK 7,11,0,82 5 ...
- 一周学会Mootools 1.4中文教程:(6)动画
先看一下动画的参数设置: 参数: fps - (number:默认是50) 每秒的帧数. unit - (string:默认是 false) 单位,可为 'px','em',或 '%'. link - ...
- FileInputStream(字节流)与fileReader(字符流) 的区别
FileInputStream 类 1 ) FileInputStream 类介绍: 以字节为单位的流处理.字节序列:二进制数据.与编码无关,不存在乱码问题. FileInputStream 类的主要 ...
- codeforces 603C. Lieges of Legendre sg函数
题目链接 n堆石子, 可以拿走一堆中的一颗, 或者将一堆数量为2*x的石子分为k堆x个的石子.k由题目给出. k分奇偶讨论. k为偶数时,k堆x个的石子异或结果为0: k为奇数时, k堆x个石子异或结 ...
- Web开发者需具备的8个好习惯
优秀的Web开发人员工作效率更高,因为他们拥有丰富的经验和良好的习惯.作者Gregor Dorfbauer分享了用于Web开发中的8个好习惯,这些良好的工作习惯不仅能提高效率,还能让您创建更加优秀的应 ...
- jquery 小插件,完成“输入字段预期值的提示信息”,防html5 placeholder属性
前言:在很多时候,我们需要文本框中显示默认值,获取焦点时,文字框中就会清空给的值,当失去焦点时,如果没有值,继续显示默认的文字,如果有输入值,就显示输入的值.现在项目中需要用到这个地方的功能比较多,于 ...
- java基础-修饰符
下列哪个修饰符可以使在一个类中定义的成员变量只能被同一包中的类访问?(B) A:private B:无修饰符 C:public D:procted 名称 说明 备注 public 可以被 ...
- 用rsync从Linux到Windows远程备份
论 rsync是Linux系统下的数据镜像备份工具,从软件的命名上就可以看出来 了——remote sync.rsync支持大多数的类Unix系统,无论是Linux.Solaris还是BSD上都经过了 ...
- UVA-Matrix Chain Multiplication(栈)
Matrix Chain Multiplication Suppose you have to evaluate an expression like A*B*C*D*E where A,B,C, ...
- SQL Server索引进阶:第三级,聚集索引
原文地址: Stairway to SQL Server Indexes: Level 3, Clustered Indexes 本文是SQL Server索引进阶系列(Stairway to SQL ...