基于Python的大数据的分页模型代码

最近在写一个cmdb系统的分页，尽管Django本身有分页的模块儿，但是还是想实现一下自己心中想的分页的一种逻辑

因为，在我们工作当中，当我们的数据量超级大的时候，其实我们每次分页查询都不必将所有的数据查询出来，而是可以按阶段的查询，举个例子

每次查询5页数据，当需要第六页的时候，再次进行加载,为了更加明了，做了举例

首先我们规定好每次查询5页，每页4条数据　　

 db_data = list(range(1,100))
 
 final_page = len(db_data)
 
 one_page_num = 5 #每页4条
 one_time_num = 5 #每次查5页

1、查询索引的逻辑 设定findIndex查询的索引

     # page = 1  int(1/5) + 1  findIndex = 1
     # page = 2  int(2/5) + 1  findIndex = 1
     # page = 3  int(3/5) + 1  findIndex = 1
     # page = 4  int(4/5) + 1  findIndex = 1
     # page = 5  int(5/5)  findIndex = 1
     # page = 6  int(6/5) + 1  findIndex = 2
     # page = 7  int(7/5) + 1  findIndex = 2
     # page = 8  int(8/5) + 1  findIndex = 2
     # page = 9  int(9/5) + 1  findIndex = 2
     # page = 10  int(10/5)  findIndex = 2
     # page = 11  int(11/5) + 1  findIndex = 3
     # page = 12  int(12/5) + 1  findIndex = 3
     # page = 13  int(13/5) + 1  findIndex = 3
     # page = 14  int(14/5) + 1  findIndex = 3
     # page = 15  int(15/5)  findIndex = 3

所以我们可以写如下代码

 if page/one_time_num > int(page/one_time_num):
     findIndex = int(page / one_time_num) + 1
 else:
     findIndex = int(page / one_time_num)
 #进行第一次查询，查询每次5页，每页4条的数据
 select_num = one_time_num * one_page_num #这里决一次定查询20条
 select_start = (findIndex - 1) * select_num  # 开始查询的索引
 select_down =  findIndex * select_num  # 结束查询的索引
 select_data = db_data[select_start: select_down]  #这里查询出了一次20条数据

这个时候是查询出20条数据了，但是，我们想要的具体的数据和页码对不上，所以做了以下的数据

 #下面对20条数据进行截取
     #findIndex 1
         # page = 1  1-(0*5)  gnPage = 1
         # page = 2  2-(0*5)  gnPage = 2
         # page = 3  3-(0*5)  gnPage = 3
         # page = 4  4-(0*5)  gnPage = 4
         # page = 5  5-(0*5)  gnPage = 5
     #findIndex 2
         # page = 6  6-(1*5)  gnPage = 1
         # page = 7  7-(1*5)  gnPage = 2
         # page = 8  8-(1*5)  gnPage = 3
         # page = 9  9-(1*5)  gnPage = 4
         # page = 10  10-(1*5)  gnPage = 5
     #findIndex 3
         # page = 11  11-(2*5)  gnPage = 1
         # page = 12  12-(2*5)  gnPage = 2
         # page = 13  13-(2*5)  gnPage = 3
         # page = 14  14-(2*5)  gnPage = 4
         # page = 15  15-(2*5)  gnPage = 5

所以我们可以得到以下代码来获取具体的一页数据

#设定 截取的索引位now_index
now_index = page - (findIndex-1)*one_time_num
#设定单页的截取起始
page_start = (now_index - 1) * one_page_num
page_end = now_index * one_page_num
#开始截取
page_data = select_data[page_start:page_end]

最后进行代码结合

 #coding:utf-8
 
 db_data = list(range(1,100))
 
 one_page_num = 5 #每页4条
 one_time_num = 5 #每次查5页
 
 while True:
     page = int(input("page >>>"))  #输入页码
 
     #查询索引的逻辑 设定findIndex查询的索引
     # page = 1  int(1/5) + 1  findIndex = 1
     # page = 2  int(2/5) + 1  findIndex = 1
     # page = 3  int(3/5) + 1  findIndex = 1
     # page = 4  int(4/5) + 1  findIndex = 1
     # page = 5  int(5/5)  findIndex = 1
     # page = 6  int(6/5) + 1  findIndex = 2
     # page = 7  int(7/5) + 1  findIndex = 2
     # page = 8  int(8/5) + 1  findIndex = 2
     # page = 9  int(9/5) + 1  findIndex = 2
     # page = 10  int(10/5)  findIndex = 2
     # page = 11  int(11/5) + 1  findIndex = 3
     # page = 12  int(12/5) + 1  findIndex = 3
     # page = 13  int(13/5) + 1  findIndex = 3
     # page = 14  int(14/5) + 1  findIndex = 3
     # page = 15  int(15/5)  findIndex = 3
 
     if page/one_time_num > int(page/one_time_num):
         findIndex = int(page / one_time_num) + 1
     else:
         findIndex = int(page / one_time_num)
     #进行第一次查询，查询每次5页，每页4条的数据
     select_num = one_time_num * one_page_num #这里决一次定查询20条
     select_start = (findIndex - 1) * select_num  # 开始查询的索引
     select_down =  findIndex * select_num  # 结束查询的索引
 
     select_data = db_data[select_start: select_down]  #这里查询出了一次20条数据
 
     #下面对20条数据进行截取
     #findIndex 1
         # page = 1  1-(0*5)  gnPage = 1
         # page = 2  2-(0*5)  gnPage = 2
         # page = 3  3-(0*5)  gnPage = 3
         # page = 4  4-(0*5)  gnPage = 4
         # page = 5  5-(0*5)  gnPage = 5
     #findIndex 2
         # page = 6  6-(1*5)  gnPage = 1
         # page = 7  7-(1*5)  gnPage = 2
         # page = 8  8-(1*5)  gnPage = 3
         # page = 9  9-(1*5)  gnPage = 4
         # page = 10  10-(1*5)  gnPage = 5
     #findIndex 3
         # page = 11  11-(2*5)  gnPage = 1
         # page = 12  12-(2*5)  gnPage = 2
         # page = 13  13-(2*5)  gnPage = 3
         # page = 14  14-(2*5)  gnPage = 4
         # page = 15  15-(2*5)  gnPage = 5
     #设定 截取的索引位now_index
     now_index = page - (findIndex-1)*one_time_num
     #设定单页的截取起始
     page_start = (now_index - 1) * one_page_num
     page_end = now_index * one_page_num
     #开始截取
     page_data = select_data[page_start:page_end]
     #输出效果
     print("+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")
     print("当前是第%s次查询"%findIndex)
     print("当前查询到的所有数据是 \n %s" % select_data)
     print("当前是 %s 页" % page)
     print("当前的数据是 \n %s " % page_data)
     print("+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")

大家试一下吧。