Indexing and Hashing.ppt

Indexing and hashing in dbms

1. ©Silberschatz, Korth and Sudarshan12.1Database System Concepts
Chapter 12: Indexing and Hashing
nBasic Concepts
nOrdered Indices
nB+-Tree Index Files
nStatic Hashing
nDynamic Hashing
nComparison of Ordered Indexing and Hashing
nIndex Definition in SQL

2. ©Silberschatz, Korth and Sudarshan12.2Database System Concepts
Basic Concepts
nIndexing mechanisms used to speed up access to desired data.
E.g., author catalog in library
nSearch Key-attribute to set of attributes used to look up
records in a file.
nAn index fileconsists of records (called index entries) of the
form
nIndex files are typically much smaller than the original file
nTwo basic kinds of indices:
Ordered indices: search keys are stored in sorted order
Hash indices:search keys are distributed uniformly across
“buckets” using a “hash function”.
search-keypointer

3. ©Silberschatz, Korth and Sudarshan12.3Database System Concepts
Index Evaluation Metrics
nAccess types supported efficiently. E.g.,
records with a specified value in the attribute
or records with an attribute value falling in a specified range of
values.
nAccess time
nInsertion time
nDeletion time
nSpace overhead

4. ©Silberschatz, Korth and Sudarshan12.4Database System Concepts
Ordered Indices
nIn an ordered index, index entries are stored sorted on the
search key value. E.g., author catalog in library.
nPrimary index: in a sequentially ordered file, the index whose
search key specifies the sequential order of the file.
Also called clustering index
The search key of a primary index is usually but not necessarily the
primary key.
nSecondary index:an index whose search key specifies an order
different from the sequential order of the file. Also called
non-clustering index.
nIndex-sequential file:ordered sequential file with a primary index.
Indexing techniques evaluated on basis of:

5. ©Silberschatz, Korth and Sudarshan12.5Database System Concepts
Dense Index Files
nDense index—Index record appears for every search-key value
in the file.

6. ©Silberschatz, Korth and Sudarshan12.6Database System Concepts
Sparse Index Files
nSparse Index: contains index records for only some search-key
values.
Applicable when records are sequentially ordered on search-key
nTo locate a record with search-key value Kwe:
Find index record with largest search-key value < K
Search file sequentially starting at the record to which the index
record points
nLess space and less maintenance overhead for insertions and
deletions.
nGenerally slower than dense index for locating records.
nGood tradeoff: sparse index with an index entry for every block in
file, corresponding to least search-key value in the block.

7. ©Silberschatz, Korth and Sudarshan12.7Database System Concepts
Example of Sparse Index Files

8. ©Silberschatz, Korth and Sudarshan12.8Database System Concepts
Multilevel Index
nIf primary index does not fit in memory, access becomes
expensive.
nTo reduce number of disk accesses to index records, treat
primary index kept on disk as a sequential file and construct a
sparse index on it.
outer index –a sparse index of primary index
inner index –the primary index file
nIf even outer index is too large to fit in main memory, yet another
level of index can be created, and so on.
nIndices at all levels must be updated on insertion or deletion from
the file.

9. ©Silberschatz, Korth and Sudarshan12.9Database System Concepts
Multilevel Index (Cont.)

10. ©Silberschatz, Korth and Sudarshan12.10Database System Concepts
Index Update: Deletion
nIf deleted record was the only record in the file with its particular
search-key value, the search-key is deleted from the index also.
nSingle-level index deletion:
Dense indices –deletion of search-key is similar to file record
deletion.
Sparse indices –if an entry for the search key exists in the index, it
is deleted by replacing the entry in the index with the next search-
key value in the file (in search-key order). If the next search-key
value already has an index entry, the entry is deleted instead of
being replaced.

11. ©Silberschatz, Korth and Sudarshan12.11Database System Concepts
Index Update: Insertion
nSingle-level index insertion:
Perform a lookup using the search-key value appearing in the
record to be inserted.
Dense indices –if the search-key value does not appear in the
index, insert it.
Sparse indices –if index stores an entry for each block of the file, no
change needs to be made to the index unless a new block is
created. In this case, the first search-key value appearing in the
new block is inserted into the index.
nMultilevel insertion (as well as deletion) algorithms are simple
extensions of the single-level algorithms

12. ©Silberschatz, Korth and Sudarshan12.12Database System Concepts
Secondary Indices
nFrequently, one wants to find all the records whose
values in a certain field (which is not the search-key of
the primary index satisfy some condition.
Example 1: In the accountdatabase stored sequentially
by account number, we may want to find all accounts in a
particular branch
Example 2: as above, but where we want to find all
accounts with a specified balance or range of balances
nWe can have a secondary index with an index record
for each search-key value; index record points to a
bucket that contains pointers to all the actual records
with that particular search-key value.

13. ©Silberschatz, Korth and Sudarshan12.13Database System Concepts
Secondary Index on balancefield of
account

14. ©Silberschatz, Korth and Sudarshan12.14Database System Concepts
Primary and Secondary Indices
nSecondary indices have to be dense.
nClustering index may be sparse.
nA Secondary index on a candidate key looks just like a dense
clustering index, except records are not stored sequentially.
nIndices offer substantial benefits when searching for records.
nWhen a file is modified, every index on the file must be updated,
Updating indices imposes overhead on database modification.
nSequential scan using primary index is efficient, but a sequential
scan using a secondary index is expensive
each record access may fetch a new block from disk

15. ©Silberschatz, Korth and Sudarshan12.15Database System Concepts
B
+
-Tree Index Files
nDisadvantageofindexed-sequentialfiles:performancedegradesas
filegrows,sincemanyoverflowblocksgetcreated.Periodic
reorganizationofentirefileisrequired.
nAdvantageofB
+
-treeindexfiles:automaticallyreorganizesitselfwith
small,local,changes,inthefaceofinsertionsanddeletions.
Reorganizationofentirefileisnotrequiredtomaintainperformance.
nB+treeindextakestheformofbalancedtree.
nDisadvantageofB
+
-trees:extrainsertionanddeletionoverhead,
spaceoverhead.
nAdvantagesofB
+
-treesoutweighdisadvantages,andtheyareused
extensively.
B
+
-tree indices are an alternative to indexed-sequential files.

16. ©Silberschatz, Korth and Sudarshan12.16Database System Concepts
Example of B
+
-Tree

17. ©Silberschatz, Korth and Sudarshan12.17Database System Concepts
B
+
-Tree Index Files (Cont.)
nAll paths from root to leaf are of the same length
nEach node that is not a root or a leaf has between Γn/2˥and
nchildren.
nA leaf node has between Γ(n–1)/2˥ and n–1 values
nSpecial cases:
If the root is not a leaf, it has at least 2 children.
If the root is a leaf (that is, there are no other nodes in the
tree), it can have between 0 and (n–1) values.
nB+treeimposesperformanceoverheadoninsertionand
deletionandaddsspaceoverhead.Theoverheadis
acceptableevenforfrequentlymodifiedfiles,sincethecost
ofreorganizationisavoided.
A B
+
-tree is a rooted tree satisfying the following properties:

18. ©Silberschatz, Korth and Sudarshan12.18Database System Concepts
B
+
-Tree Node Structure
nB+ tree index is a multilevel index, but it has a structure different
from that of multilevel index sequential index file.
nTypical node
K
iare the search-key values (n-1 search key values)
P
iare pointers to children (for non-leaf nodes) or pointers to records
or buckets of records (for leaf nodes). (n pointers)
nThe search-keys in a node are ordered
K
1 < K
2 < K
3 < . . .< K
n–1

19. ©Silberschatz, Korth and Sudarshan12.19Database System Concepts
Leaf Nodes in B
+
-Trees
For i= 1, 2, . . ., n–1, pointer P
ieither points to a file record with search-
key value K
i, or to a bucket of pointers to file records, each record
having search-key value K
i. Only need bucket structure if search-key
does not form a primary key.
Each leaf can hold up to n-1 values. Leaf can hold as few as [ (n-1)/2]
values. The ranges of values in each leaf do not overlap.
If L
i, L
jare leaf nodes and i < j, L
i’s search-key values are less than L
j’s
search-key values
P
npoints to next leaf node in search-key order
Properties of a leaf node:

20. ©Silberschatz, Korth and Sudarshan12.20Database System Concepts
Non-Leaf Nodes in B
+
-Trees
Non leaf nodes form a multi-level sparse index on the leaf nodes.
For a non-leaf node with mpointers:
A non leaf node may hold up to n pointers and must hold at least
Γn/2˥ pointers.
The number of pointers in a node is called fanoutof the node.
All the search-keys in the subtreeto which P
1points are less than
K
1

21. ©Silberschatz, Korth and Sudarshan12.21Database System Concepts
Non-Leaf Nodes in B
+
-Trees
Non leaf nodes form a multi-level sparse index on the leaf nodes.
For a non-leaf node with mpointers:
Let us consider a node containing m pointers. For i= 2, 3, . . .,m
− 1,
pointerPipointstothesubtreethatcontainssearch-key
valueslessthanKiandgreaterthanorequaltoKi−1.Pointer
Pmpointstothepartofthesubtreethatcontainsthosekey
valuesgreaterthanorequaltoKm−1,andpointerP1points
tothepartofthesubtreethatcontainsthosesearch-key
valueslessthanK1.

22. ©Silberschatz, Korth and Sudarshan12.22Database System Concepts
Example of a B
+
-tree
B
+
-tree for accountfile (n = 3)

23. ©Silberschatz, Korth and Sudarshan12.23Database System Concepts
Example of B
+
-tree
Leaf nodes must have between 2 and 4 values
((n–1)/2and n –1, with n= 5).
Non-leaf nodes other than root must have between 3
and 5 children ((n/2and n with n=5).
Root must have at least 2 children fewer than [n/2]
pointers.
B
+
-tree for account file (n= 5)

24. ©Silberschatz, Korth and Sudarshan12.24Database System Concepts
Observations about B
+
-trees
Since the inter-node connections are done by pointers, “logically”
close blocks need not be “physically” close.
The non-leaf levels of the B
+
-tree form a hierarchy of sparse
indices.
The B
+
-tree contains a relatively small number of levels
(logarithmic in the size of the main file), thus searches can be
conducted efficiently.
Insertions and deletions to the main file can be handled
efficiently, as the index can be restructured in logarithmic time
(as we shall see).

25. ©Silberschatz, Korth and Sudarshan12.25Database System Concepts
Updates on B
+
-Trees: Insertion
1.Find the leaf node in which the search-key value would appear
2.If the search-key value is already present in the leaf node
1.Add record to the file
2.If necessary add a pointer to the bucket.
3.If the search-key value is not present, then
1.add the record to the main file (and create a bucket if
necessary)
2.If there is room in the leaf node, insert (key-value, pointer)
pair in the leaf node
3.Otherwise, split the node (along with the new (key-value,
pointer) entry) as discussed in the next slide.

26. ©Silberschatz, Korth and Sudarshan12.26Database System Concepts
Updates on B
+
-Trees: Insertion (Cont.)
Splitting a node:
take the n(search-key value, pointer) pairs (including the one being
inserted) in sorted order. Place the first n/2in the original node,
and the rest in a new node.
let the new node be p,and let kbe the least key value in p. Insert
(k,p) in the parent of the node being split. If the parent is full, split it
and propagate the split further up.
The splitting of nodes proceeds upwards till a node that is not full
is found. In the worst case the root node may be split increasing
the height of the tree by 1.
Result of splitting node containing Brighton and Downtown on
inserting Clearview

27. ©Silberschatz, Korth and Sudarshan12.27Database System Concepts
Updates on B
+
-Trees: Insertion (Cont.)
B
+
-Tree before and after insertion of “Clearview”

28. ©Silberschatz, Korth and Sudarshan12.28Database System Concepts
B
+
-Tree Insertion
B
+
-Tree before and after insertion of “Adams”

29. ©Silberschatz, Korth and Sudarshan12.29Database System Concepts
B
+
-Tree Insertion
B
+
-Tree before and after insertion of “Lamport”

30. ©Silberschatz, Korth and Sudarshan12.30Database System Concepts
B
+
-Tree File Organization
IndexfiledegradationproblemissolvedbyusingB
+
-Treeindices.
DatafiledegradationproblemissolvedbyusingB
+
-TreeFile
Organization.
TheleafnodesinaB
+
-treefileorganizationstorerecords,insteadof
pointers.
Sincerecordsarelargerthanpointers,themaximumnumberof
recordsthatcanbestoredinaleafnodeislessthanthenumberof
pointersinanonleafnode.
Leafnodesarestillrequiredtobehalffull.
Insertionanddeletionarehandledinthesamewayasinsertionand
deletionofentriesinaB
+
-treeindex.

31. ©Silberschatz, Korth and Sudarshan12.31Database System Concepts
Static Hashing
Disadvantageofsequentialfileorganization:accessanindex
structuretolocatedata,orusebinarysearch:toomanyI/O
operations.
Abucketisaunitofstoragecontainingoneormorerecords(a
bucketistypicallyadiskblock).
Inahashfileorganizationweobtainthebucketofarecord
directlyfromitssearch-keyvalueusingahashfunction.
Hashfunctionhisafunctionfromthesetofallsearch-key
valuesKtothesetofallbucketaddressesB.
Hashfunctionisusedtolocaterecordsforaccess,insertionas
wellasdeletion.
Recordswithdifferentsearch-keyvaluesmaybemappedto
thesamebucket;thusentirebuckethastobesearched
sequentiallytolocatearecord.

32. ©Silberschatz, Korth and Sudarshan12.32Database System Concepts
Example of Hash File Organization (Cont.)
There are 10 buckets,
The binary representation of the ithcharacter is assumed to be the
integer i.
The hash function returns the sum of the binary representations of the
characters modulo 10
E.g. h(Perryridge) = 5 h(Round Hill) = 3 h(Brighton) = 3
E.g.
Total : 15
Total mod 10 = 15 mod 10 = 5
Hash file organization of accountfile, using branch-name as key
(See figure in next slide.)
P E R R Y R I D G E
0 1 2 3 4 5 6 7 8 9
0 000
1
001
0
001
1
010
0
010
1
011
0
011
1
100
0
100
1
0 1 1 2 1 2 2 3 1 2
Binary
representation

33. ©Silberschatz, Korth and Sudarshan12.33Database System Concepts
Example of Hash File Organization
Hash file organization of accountfile, using branch-name as key
(see previous slide for details).

34. ©Silberschatz, Korth and Sudarshan12.34Database System Concepts
Hash Functions
Worst has function maps all search-key values to the same
bucket; this makes access time proportional to the number of
search-key values in the file.
An ideal hash function is uniform,i.e., each bucket is assigned
the same number of search-key values from the set of all
possible values.
Ideal hash function is random, so each bucket will have the
same number of records assigned to it irrespective of the actual
distributionof search-key values in the file.
Typical hash functions perform computation on the internal
binary representation of the search-key.
For example, for a string search-key, the binary representations of
all the characters in the string could be added and the sum modulo
the number of buckets could be returned. .

35. ©Silberschatz, Korth and Sudarshan12.35Database System Concepts
Handling of Bucket Overflows
If bucket do not have enough space, a bucket overflow is said
to occur.
Bucket overflow can occur because of
Insufficient buckets : The number of buckets, which we denote n
b,
must be chosen such that n
b> n
r/f
r, where nr denotes the total
number of records that will be stored and f
rdenotes the number of
records that will fit in a bucket.
Skew in distribution of records. Some buckets are assigned more
records than are others, so a bucket may oerflow oven when other
buckets still have space. This situation is called skew. This can
occur due to two reasons:
multiple records have same search-key value
chosen hash function produces non-uniform distribution of key
values
Although the probability of bucket overflow can be reduced, it
cannot be eliminated; it is handled by using overflow buckets.

36. ©Silberschatz, Korth and Sudarshan12.36Database System Concepts
Handling of Bucket Overflows (Cont.)
Overflow chaining–the overflow buckets of a given bucket are
chained together in a linked list.
Above scheme is called closed hashing.
An alternative, called open hashing, which does not use overflow
buckets, is not suitable for database applications.
The set of buckets is fixed, and there are no overflow chains. If a
bucket is full, the system inserts records in some other bucket : in
cyclic order, in next bucket if it has space. It is known as linear
probing.

37. ©Silberschatz, Korth and Sudarshan12.37Database System Concepts
Hash Indices
Hashingcanbeusednotonlyforfileorganization,butalsofor
index-structurecreation.
Ahashindexorganizesthesearchkeys,withtheirassociated
recordpointers,intoahashfilestructure.
Weapplyahashfunctiononasearchkeytoidentifyabucket,
andstorethekeyanditsassociatedpointersinthebucket(orin
overflowbucket)
Strictlyspeaking,hashindicesarealwayssecondaryindices
ifthefileitselfisorganizedusinghashing,aseparateprimaryhash
indexonitusingthesamesearch-keyisunnecessary.
However,weusethetermhashindextorefertobothsecondary
indexstructuresandhashorganizedfiles.

38. ©Silberschatz, Korth and Sudarshan12.38Database System Concepts
Example of Hash Index

39. ©Silberschatz, Korth and Sudarshan12.39Database System Concepts
Deficiencies of Static Hashing
In static hashing, function hmaps search-key values to a fixed
set of Bof bucket addresses.
Databases grow with time. If initial number of buckets is too small,
performance will degrade due to too much overflows.
If file size at some point in the future is anticipated and number of
buckets allocated accordingly, significant amount of space will be
wasted initially.
If database shrinks, again space will be wasted.
One option is periodic re-organization of the file with a new hash
function, but it is very expensive.
These problems can be avoided by using techniques that allow
the number of buckets to be modified dynamically.

40. ©Silberschatz, Korth and Sudarshan12.40Database System Concepts
Dynamic Hashing
Goodfordatabasethatgrowsandshrinksinsize
Allowsthehashfunctiontobemodifieddynamically
Extendablehashing–Copeswithchangesindatabasesizeby
splittingancoalescingbucketsasthedatabasegrowsand
shrinks.
Asaresultspaceefficiencyisretained.
Sincethereorganizationisperformedononlyononebucketatatime,
theresultingperformanceoverheadisacceptablylow.

41. ©Silberschatz, Korth and Sudarshan12.41Database System Concepts
Extendable Hashing
Extendable hashing–one form of dynamic hashing
Here buckets are created on demand.
Hash function generates values over a large range —typically b-bit
integers, with b= 32. 2
32
is 4 billion.
Use i bits, where 0<=i<=b.
These i bits are used as an offset into an additional table of bucket
address.
The value of i grows and shrinks with the size of database.
Bucket address table size = 2
i.
Initially i= 0
Value of igrows and shrinks as the size of the database grows and
shrinks.
Multiple entries in the bucket address table may point to a bucket.
Thus, actual number of buckets is < 2
i
The number of buckets also changes dynamically due to coalescing
and splitting of buckets.

42. ©Silberschatz, Korth and Sudarshan12.42Database System Concepts
General Extendable Hash Structure
In this structure, i
2= i
3= i, whereas i
1= i –1 (see
next slide for details)

43. ©Silberschatz, Korth and Sudarshan12.43Database System Concepts
Use of Extendable Hash Structure
Each bucket jstores a value i
j; all the entries that point to the
same bucket have the same values on the first i
jbits.
To locate the bucket containing search-key K
j:
1.Compute h(K
j) = X
2.Use the first ihigh order bits of Xas a displacement into bucket
address table, and follow the pointer to appropriate bucket
To insert a record with search-key value K
j
follow same procedure as look-up and locate the bucket, say j.
If there is room in the bucket jinsert record in the bucket.
Else the bucket must be split and insertion re-attempted (next slide.)
Overflow buckets used instead in some cases (will see shortly)

44. ©Silberschatz, Korth and Sudarshan12.44Database System Concepts
Updates in Extendable Hash Structure
If i> i
j(more than one pointer to bucket j)
allocate a new bucket z, and set i
jand i
zto the old i
j-+ 1.
make the second half of the bucket address table entries pointing
to jto point to z
remove and reinsert each record in bucket j.
recompute new bucket for K
jand insert record in the bucket (further
splitting is required if the bucket is still full)
If i = i
j(only one pointer to bucket j)
increment iand double the size of the bucket address table.
replace each entry in the table by two entries that point to the same
bucket.
recompute new bucket address table entry for K
j
Now i > i
jso use the first case above.
To split a bucket jwhen inserting record with search-key value K
j:

45. ©Silberschatz, Korth and Sudarshan12.45Database System Concepts
Updates in Extendable Hash Structure
(Cont.)
When inserting a value, if the bucket is full after several splits
(that is, ireaches some limit b) create an overflow bucket instead
of splitting bucket entry table further.
To delete a key value,
locate it in its bucket and remove it.
The bucket itself can be removed if it becomes empty (with
appropriate updates to the bucket address table).
Coalescing of buckets can be done (can coalesce only with a
“buddy” bucket having same value of i
j
and same i
j
–1 prefix, if it is
present)
Decreasing bucket address table size is also possible
Note: decreasing bucket address table size is an expensive
operation and should be done only if number of buckets becomes
much smaller than the size of the table

46. ©Silberschatz, Korth and Sudarshan12.46Database System Concepts
Use of Extendable Hash Structure:
Example
Initial Hash structure, bucket size = 2

47. ©Silberschatz, Korth and Sudarshan12.47Database System Concepts
Use of Extendable Hash Structure:
Example
Initial Hash structure, bucket size = 2
0 A-217 Brighton 750
0
Bucket address table Bucket 1

48. ©Silberschatz, Korth and Sudarshan12.48Database System Concepts
Use of Extendable Hash Structure:
Example
Initial Hash structure, bucket size = 2
0 A-217 Brighton 750
A-101 Downtown 500
0
Bucket address table Bucket 1

49. ©Silberschatz, Korth and Sudarshan12.49Database System Concepts
Example (Cont.)
Hash structure after insertion of one Brighton and two
Downtown records
0
1

50. ©Silberschatz, Korth and Sudarshan12.50Database System Concepts
Example (Cont.)
Hash structure after insertion of Mianus record
00
01
10
11

51. ©Silberschatz, Korth and Sudarshan12.51Database System Concepts
Comparison of Ordered Indexing and Hashing
Cost of periodic re-organization
Relative frequency of insertions and deletions
Is it desirable to optimize average access time at the expense of
worst-case access time?
Expected type of queries:
Hashing is generally better at retrieving records having a specified
value of the key.
If range queries are common, ordered indices are to be preferred
Queries of form select a1,a2,….., an from r where ai = c : hashing is
better
Queries of form select a1, a2, a3…., an from r where ai<=c2 and
ai>=c3; indexing is better.

52. ©Silberschatz, Korth and Sudarshan12.52Database System Concepts
Index Definition in SQL
Create an index
create index<index-name> on<relation-name>
(<attribute-list>)
E.g.: create index b-index onbranch(branch-name)
Use create unique indexto indirectly specify and enforce the
condition that the search key is a candidate key is a candidate
key.
Not really required if SQL uniqueintegrity constraint is supported
To drop an index
drop index <index-name>