RelationalDatabaseManagemetSystem-perfmancebenchmark.ppt

performance benchmarks

1. RDBMS PERFORMANCE BENCHMARKS

2. Suites of tasks used to quantify performance of software systems. Single task not enough for complex systems. Performance measures: - Throughput (transactions per second or TPS) - Response time - Availability Average throughput of different transaction types. Suppose that a system A runs transaction type T1 at 99 tps and transaction type T2 at 1 tps and other system runs B both T1 and T2 at 50 tps - Simple averaging TPS is wrong! - If we ran 50 transactions of each type

3. System A : T1 (0.01s) T2 ( 1s)  need 50.5 sec System B : T1 (0.02s) T2 (0.02s)  need 2 sec Instead, HARMONIC MEAN of n throghputs t1,t2…. tn : harmonic mean = [ n / ( 1/t1 + 1/t2 + …. + 1/ tn )] The harmonic mean for System B is 25 times faster than System A Interference ( eg : Lock Contention) makes even this incorrect if different transaction types run concurrently.

4. Database Application Classes Online Transaction Processing (OLTP) - requires high concurrency and clever techniques to speed up commit processing, to support a high rate of update transactions. Decision Support Applications - including Online Analytical Processing - require good query evaluation algorithms and query optimization Architecture of some database systems tuned to one of the two classes - eg : Teradata DBMS is tuned to decision support Others try to balance the two requirements - eg : Oracle, with snapshot support for long read-only transaction

5. Transaction Benchmarks Suites The transaction processing council (TPC) benchmark suites are widely used. TPC – A and TPC – B : used in bank teller applications with and without application TPC – C : used in inventory systems TPC – D : complex decision support applications TPC – H : (H and ad hoc) with some extra queries, total number of 22 queries - prohibits materializes views - permits indices only on primary and foreign keys TPC – R : (R for reporting) same as TPC-H, but without any restrictions on materialized views and indices TPC – W : (W for web) End to End web service benchmark modelling a web bookstore

6. TPC Performance Measures transactions - per - second with specified constraints on response time transactions - per – second per dollar accounts for cost of owning system TPC benchmark requires database sizes to be scaled up with increasing transactions-per-second Reflects real world applications where more customers means more database size and more transactions-per-second External audit of TPC performance numbers is mandatory TPC performance claims can be trusted

7. Sample TPC performance measures Two types of tests of TPC-H (ad hoc) and TPC-R (report) - power metric test takes mean to find queries per hour - throughput metric test multiple streams running in parallel, each stream generates queries, with one parallel update stream Composite “ Query per hour ” metric : Square root of (power metric * throughput metric) Composite “ price vs. performance ” metric : System price / composite metric

8. THANK YOU !!