Question: Should I use reads with good quality but failed-vendor flag?--biostart for vendor quality
https://www.biostars.org/p/198405/
Quick question is: I have some mapped reads in bam file which have good read quality, but they have sam flag 0x200 which means they didn't pass the vendor check. Should I include them or not in downstream analysis?
Long question is: what' s the relationship between read quality score and Chastity score?
First, everybody may know read quality score:
Reads quality score(phred score) is calculated by -10*log(P(error_base)), P(error_base) represents the probability that the base is incorrect.
Second, I want to talk about Chastity score during the vendor check:
For reads in fastq format, there is a header field 'Y/N' which indicates whether the read pass filtering step. And the corresponding sam flag is 0x200, indicating "not passing filters, such as platform/vendor quality controls". How does Illumina set the filtering criteria?
As far as I know, read filtering by Illumina Real Time Analysis (RTA) happens during the run, and filtering is determined by Chastity score. Chastity Score is calculated by “the ratio of the highest of the four (base type) intensities to the sum of highest two”. Illumina described the vendor check as follows:
"To remove the least reliable data from the analysis, the raw data can be filtered to remove any clusters that have “too much” intensity corresponding to bases other than the called base. By default, the purity of the signal from each cluster is examined over the first 25 cycles and calculated as Chastity = Highest_Intensity / (Highest_Intensity + Next_Highest_Intensity) for each cycle. The new default filtering implemented at the base calling stage allows at most one cycle that is less than the Chastity threshold. The higher the value, the better. This value is very dependent on cluster density, since the major cause of an impure signal in the early cycles is the presence of another cluster within a few micrometers."
So, to my understanding, every cycle the Sequencer scan a cluster, there would be 4 kinds of signals from 4 bases(am I right?) the most significant base would the final choice. The bigger the signal intensity divergence is the better for base calling. For the first 25 cycles, Illumina allow at most one base with smaller signal intensity divergence, otherwise, Illumina would set the read as vendor failed. Is my understanding right so far?
But what is the relationship between the Phred score and the Chastity score? if they really have. Can I still use vendor failed reads if they have high phred score?
Thanks! Tao
modified 10 months ago by ablanchetcohen • 1.1k • written 10 months ago by Tao • 110
Curious. Why are the vendor failed reads in your dataset?
I downloaded the bam file from GTEx (dbGaP). The bam file contains all the reads, including mapped, unmapped, vendor failed reads. For a sample with ~100M reads, ~12M are labeled as vendor failed including both mapped and unmapped reads. Part of the vendor failed reads have read good quality. So, I'm not sure if I should include them.
I second this comment. You should contact your vendor. I have never seen reads failing the filtering step indicated in the header field of a FASTQ file being given to a client. Why include these reads? They just take up storage space, and are likely to induce errors in the downstream analysis. There was either an error in the setting of the flag, or a mistake in giving you the reads.
I checked the GTEx Project FAQ. The alignment was probably done in 2012, since TopHat v1.4.1 was used. This was the very dawn of RNA-Seq. The analyses dating back to this period are often suspicious since bioinformaticians were not yet familiar with RNA-Seq, and the software programs contained bugs more often than not. My recommendation is always to treat with suspicion any analysis results dating back to this period. Most likely, those preparing the data were not aware yet that these reads should be filtered out.
I would filter out all the "vendor failed reads", and redo the alignment using a more recent aligner, genome, and annotation. At least, that would be my recommendation based on my knowledge. To get a definitive answer, you could contact the staff at the GTex project.
thanks for your comments. The sample is downloaded from a public project GTEx. I'm also confused why they deposit so many(10M vendor failed for a 100M sample) vendor-failed reads on dbGaP. In my study, I didn't realize this problem at first, which causing a big problem now. In your opinion, such reads should be removed without considering reads quality?
Short answer yes.
They were "failed" by Illumina pre-processing software for a reason (e.g. mixed sequence from one cluster, phasing issues etc).
Question: Should I use reads with good quality but failed-vendor flag?--biostart for vendor quality的更多相关文章
- 去除外显子低质量reads时弹出错误“Invalid quality score value (char '#' ord 35 quality value -29) on line 4”和“Invalid quality score value (char '.' ord 46 quality value -18) on line 12”的解决方法
楼主跑以下程序时分别弹出了“fastq_quality_filter: Invalid quality score value (char '.' ord 46 quality value -18) ...
- Quality assessment and quality control of NGS data
http://www.molecularevolution.org/resources/activities/QC_of_NGS_data_activity_new table of contents ...
- XSplit Quality, VBV-Buffer, VBV-Maxrate and Preset Settings
XSplit uses the x264 encoder, so let's start off by saying that parameters mentioned in the title, w ...
- Quality 是什么?
Quality 是什么? 通常,我们谈及 Quality(质量)时,最常见的问题就是:Quality 是什么? 有很多业界先驱和研究人员已经回答了这个问题,我在这里并不会再给出一个新的答案.在学习总结 ...
- 视频主观质量评价工具:MSU Perceptual Video Quality tool
MSU Perceptual Video Quality tool是莫斯科国立大学(Moscow State University)的Graphics and Media Lab制作的一款视频主观评价 ...
- software quality assurance 常见问题收录
1. What is Quality? Quality means, “meeting requirements.” ..Whether or not the product or service d ...
- ITU-T G.1080 IPTV的体验质量(QoE)要求 (Quality of experience requirements for IPTV services)
IPTV的服务质量(QoE)要求 Quality of experience requirements for IPTV services Summary This Recommendation de ...
- unity 质量设置 Quality Settings
Unity allows you to set the level of graphical quality it will attempt to render. Generally speaking ...
- samtools常用命令详解
samtools的说明文档:http://samtools.sourceforge.net/samtools.shtmlsamtools是一个用于操作sam和bam文件的工具合集.包含有许多命令.以下 ...
随机推荐
- intellij IDEA 报 非法字符 \65279 原因及解决方法
用eclipse创建的项目导入到 intellij IDEA 之后 编译时包 非法字符 '\65279' 该问题产生的原因是 IDEA对以UTF8编码的文件保存时自动加上了BOM(UTF-8文件签名) ...
- 使用log4net记录日志到数据库(含有自定义属性)
记录日志是管理系统中对用户行为的一种监控与审核,asp.net中记录日志的方式有很多种,这里我只介绍一下最近用到的log4net,关于他的具体介绍网上有很多,我讲一下他的用法. 第一步:在配置文件中的 ...
- 三进制状态压缩DP(旅行商问题TSP)HDU3001
http://acm.hdu.edu.cn/showproblem.php?pid=3001 Travelling Time Limit: 6000/3000 MS (Java/Others) ...
- Linux系统下tomcat安装配置
Linux系统中Tomcat的安装配置. 前提JDK已经安装好. 安装 下载tomcatwget http://mirrors.cnnic.cn/apache/tomcat/tomcat-8/v8.0 ...
- php 常用的系统函数
字符串函数 strlen:获取字符串长度,字节长度 substr_count 某字符串出现的次数 substr:字符串截取,获取字符串(按照字节进行截取) mb_strlenmb_substr str ...
- linux 统计文件数量
查找当前目录下compose文件的数量 ls -lr | grep "compose" | wc -l
- MS-SQL数据库定时自动备份
在SQL Server中出于数据安全的考虑,所以需要定期的备份数据库.而备份数据库一般又是在凌晨时间基本没有数据库操作的时候进行,所以我们不可能要求管理员每天守到晚上1点去备份数据库.要实现数据库的定 ...
- Cisco设备开启telnet登录
思科设备怎么开启telnet登录 configuration line vty 0 4SW1(config-line)#transp input telne
- Python开发【前端】:Ajax(一)
Ajax Ajax即“Asynchronous Javascript And XML”(异步JavaScript和XML),是指一种创建交互式网页应用的网页开发技术,AJAX = 异步 JavaScr ...
- Mysql覆盖索引与延迟关联
延迟关联:通过使用覆盖索引查询返回需要的主键,再根据主键关联原表获得需要的数据. 为什innodb的索引叶子节点存的是主键,而不是像myisam一样存数据的物理地址指针? 如果存的是物理地址指针不 ...