Improving the Efficiency of Bitmap Indexes for Data Warehouses.pdf

1、The progresses of the computer technology have enabled the development of new kinds ofvery large database applications such as data warehouses and scientific databases. Unlike thetraditional databases systems, these data

2、 sets are characterized by the high dimensionality, andare mostly read-only and append-only systems. A data warehouse is large repository of information containing data collected fromdifferent data sources over a lon

3、g period of time. The warehouse data are used for analyticalpurposes by the knowledge workers (executive, manager, analyst) of an organization to makebetter and faster decisions. The query performance in such an environm

4、ent is a criticalchallenge as most of the associated queries are complex and adhoc in nature and require hugevolumes of data to be processed. A promise way to speed up the query performance in datawarehouses in the bitma

5、p index. Because the bitwise logical operations are well supported bycomputer hardware, the bitmaps can easily and effciently be combined to answer queries.However, as the database grow in size, the bitmap index size inc

6、reases as well. It is thereforeimperative to compress the index in order to improve the query performance. One of the mostnotable bitmap index compression schemes is the word-aligned hybrid code (WAH). This thesis ai

7、ms to present an efficient bitmap indexing technique to improving theperformance of the Word-Aligned Hybrid for high-cardinality attributes in data warehousingapplications. WAH compression is based on the run-length enco

8、ding, and its performancedepends on the presence of long runs of identical bits. For large data sets with high-cardinalityattributes, in most cases, the bitmap index of the indexed attribute (column) would be sparse;henc

9、e the compression process would be ineffective. To make the bitmap compression efficient,it's therefore necessary to reorder the data warehouse tuples to achieve higher compressionrates. To reorder the tuples of base

10、 data, our approach consists to simply sort the indexedattribute before generating the bitmap index. Our strategy is therefore used as a preprocessingstep before compression, only to improve the performance, without affe

11、cting algorithms usedfor compression and query processing. Sorting of the indexed attribute ensures long runs ofones and zeros. These long runs of ones and zeros are desirable for any compression schemethat is based on r

12、un length encoding, such as WAH. To further improve the performance, wepropose to bin and cluster the indexed attributed. Our experiments, conducted on five data setswith table cardinalities varying from 100,000 to 2,000

13、,000 records, showed that our strategysignificantly improves the performance of the word-aligned hybrid code. We observed that thesizes of the bitmap indices considerably decreased for various attributes cardinalities. M