How does database indexing work?
When data is stored on disk based storage devices, it is stored as blocks of data. These blocks are accessed in their entirety, making them the atomic disk access operation. Disk blocks are structured in much the same way as linked lists; both contain a section for data, a pointer to the location of the next node (or block), and both need not be stored contiguously.
Due to the fact that a number of records can only be sorted on one field, we can state that searching on a field that isn’t sorted requires a Linear Search which requires N/2
block accesses (on average), where N is the number of blocks that the table spans. If that field is a non-key field (i.e. doesn’t contain unique entries) then the entire table space must be searched at N block accesses.
Whereas with a sorted field, a Binary Search may be used, this has log2 N
block accesses. Also since the data is sorted given a non-key field, the rest of the table doesn’t need to be searched for duplicate values, once a higher value is found. Thus the performance increase is substantial.
Cluster & Non-cluster Intex:
cluster : used for easy retrival data from DB by altering the way that records are stored.
non-cluster : not alter the way it stores, but create a complete seperate object within the table. It point back to the original table rows after searching.
What is indexing?
Indexing is a way of sorting a number of records on multiple fields. Creating an index on a field in a table creates another data structure which holds the field value, and pointer to the record it relates to. This index structure is then sorted, allowing Binary Searches to be performed on it.
The downside to indexing is that these indexes require additional space on the disk, (No-cluster Index)since the indexes are stored together in a table using the MyISAM engine, this file can quickly reach the size limits of the underlying file system if many fields within the same table are indexed.
How does it work?
Firstly, let’s outline a sample database table schema;
Field name Data type Size on disk
id (Primary key) Unsigned INT 4 bytes
firstName Char(50) 50 bytes
lastName Char(50) 50 bytes
emailAddress Char(100) 100 bytes
Note: char was used in place of varchar to allow for an accurate size on disk value. This sample database contains five million rows, and is unindexed. The performance of several queries will now be analyzed. These are a query using the id (a sorted key field) and one using the firstName (a non-key unsorted field).
Example 1
Given our sample database of r = 5,000,000
records of a fixed size giving a record length of R = 204
bytes and they are stored in a table using the MyISAM engine which is using the default block sizeB = 1,024
bytes. The blocking factor of the table would be bfr = (B/R) = 1024/204 = 5
records per disk block. The total number of blocks required to hold the table is N = (r/bfr) = 5000000/5 = 1,000,000
blocks.
A linear search on the id field would require an average of N/2 = 500,000
block accesses to find a value given that the id field is a key field. But since the id field is also sorted a binary search can be conducted requiring an average of log2 1000000 = 19.93 = 20
block accesses. Instantly we can see this is a drastic improvement.
Now the firstName field is neither sorted, so a binary search is impossible, nor are the values unique, and thus the table will require searching to the end for an exact N = 1,000,000
block accesses. It is this situation that indexing aims to correct.
Given that an index record contains only the indexed field and a pointer to the original record, it stands to reason that it will be smaller than the multi-field record that it points to. So the index itself requires fewer disk blocks that the original table, which therefore requires fewer block accesses to iterate through. The schema for an index on the firstName field is outlined below;
Field name Data type Size on disk
firstName Char(50) 50 bytes
(record pointer) Special 4 bytes
Note: Pointers in MySQL are 2, 3, 4 or 5 bytes in length depending on the size of the table.
Example 2
Given our sample database of r = 5,000,000 records with an index record length of R = 54 bytes and using the default block size B = 1,024 bytes. The blocking factor of the index would be bfr = (B/R) = 1024/54 = 18 records per disk block. The total number of blocks required to hold the table is N = (r/bfr) = 5000000/18 = 277,778 blocks.
Now a search using the firstName field can utilise the index to increase performance. This allows for a binary search of the index with an average of log2 277778 = 18.08 = 19 block accesses. To find the address of the actual record, which requires a further block access to read, bringing the total to 19 + 1 = 20 block accesses, a far cry from the 277,778 block accesses required by the non-indexed table.
When should it be used?
Given that creating an index requires additional disk space (277,778 blocks extra from the above example), and that too many indexes can cause issues arising from the file systems size limits, careful thought must be used to select the correct fields to index.
Since indexes are only used to speed up the searching for a matching field within the records, it stands to reason that indexing fields used only for output would be simply a waste of disk space and processing time when doing an insert or delete operation, and thus should be avoided. Also given the nature of a binary search, the cardinality or uniqueness of the data is important. Indexing on a field with a cardinality of 2 would split the data in half, whereas a cardinality of 1,000 would return approximately 1,000 records. With such a low cardinality the effectiveness is reduced to a linear sort, and the query optimizer will avoid using the index if the cardinality is less than 30% of the record number, effectively making the index a waste of space.
How does database indexing work?的更多相关文章
- What is the difference between a binary tree, a binary search tree, a B tree and a B+ tree?
Binary Tree : It is a tree data structure in which each node has at most two children. As such there ...
- 「2014-2-6」TokuMX and MongoDB related materials collection
简介参考 TokuMX 和 MongoDB 各自的官方站点. ## Tokutek 最重要的特点和 marketing word 是所谓 fractal tree indexing te ...
- EPiServer 简单项目总结
国内用到的EPiServer应该不多,所以记录点用到过的东西,以便分享 1.EPiServer office site http://www.episerver.com/ 2.EPiServer CM ...
- 【Professional English】Words Summary
01.数据库管理系统(Database Management Systems,DBMS) A database management system (DBMS) is a computer softw ...
- Exploring Micro-frameworks: Spring Boot--转载
原文地址:http://www.infoq.com/articles/microframeworks1-spring-boot Spring Boot is a brand new framework ...
- sql index改怎么建
https://stackoverflow.com/questions/11299217/how-can-i-optimize-this-sql-query-using-indexes ------- ...
- 对HashMap的一次记录
HashMap的具体学习,认识了解. 前言 也是最近开始面试才发现,HashMap是问的真多.以前听学长或自己在网上看到过一些面试资料都在说集合.线程这块比较重要,面试的重点.自己也是有那抵触情绪,所 ...
- Importing/Indexing database (MySQL or SQL Server) in Solr using Data Import Handler--转载
原文地址:https://gist.github.com/maxivak/3e3ee1fca32f3949f052 Install Solr download and install Solr fro ...
- [转]Amazon DynamoDB – a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications
This article is from blog of Amazon CTO Werner Vogels. -------------------- Today is a very exciting ...
随机推荐
- 分享10款功能强大的HTML5/CSS3应用插件
1.纯CSS3美化Checkbox和Radiobox按钮 外观很时尚 利用CSS3我们可以打造非常具有个性化的用户表单,今天我们就利用CSS3美化Checkbox复选框和Radiobox单选框.CSS ...
- The New Debugger
在debug下有一个中文叫做杂项的选项卡下有配置的内容 里面可以配置debug的模式 有的时候一些莫名其妙的问题需要 调整里面的设置 <<SAP debug的几种方式.pdf>> ...
- 从基础知识到重写Spring的Bean工厂中学习java的工厂模式
1.静态工厂模式其他对象不能直接通过new得到某个类,而是通过调用getInstance()方法得到该类的对象这样,就可以控制类的产生过程.顺带提一下单例模式和多例模式: 单例模式是指控制其他对象获 ...
- ADO.NET笔记——执行事务
相关知识: 处于同一事务(Transaction)内的一组操作,要么都成功执行,最后完全提交:但如果只要有任何一个操作失败或者出问题,所有值钱执行的操作也都取消并恢复到初始状态(即回滚) SqlTra ...
- multiple backgrounds 多重背景
语法缩写如下: background : [background-color] | [background-image] | [background-position][/background-siz ...
- react-native-vector-icons 安装
react-native-vector-icons 是可以直接使用图片名就能加载图片的第三方,类似于web的iconfont矢量图,使用很方便, 你不需要在工程文件夹里塞各种图片, 节省很多空间,下面 ...
- PHP:parse_str()字符串函数
parse_str()-把字符串解析成多个变量. 描述:void parse_str(sring $str [, array $arr]) 如果str是URL传递入的查询字符串(query stri ...
- Linq--扩展方法
如果现在有一个这样的需求,求筛选出来的大于20MB的进程的和,常用的方法是写一个静态方法传进去一个ProcessData列表 比如: public static Int64 TotalMemory( ...
- ORACLE 11G R2 修改"用户名"
SQL> create pfile from spfile; 修改pfile文件,添加隐含参数 *._enable_rename_user='TRUE',将数据库以restrict方式启动 1. ...
- ERROR 1062 (23000): Duplicate entry '0' for key 'PRIMARY'
OS: centos 6.3DB: 5.5.14 测试创建yoon测试表,没有主键,没有索引,基础数据内容如下: mysql> select * from yoon;+----+-------- ...