Evaluation summary report

Main-memory database management systems

 

 

Mauritz Sundell

Upsala Systemkonsult, UPSYS AB

2003-06-19

Version 1


 

1.1 Application

Our target application is a subscriber database for a mediation device. The database will be accessed be I single thread on a single machine.

 

There will be a main table with 20000000 entries about 500 bytes each. And the application must be able to do 10000 operations a second ther an operation is modify some fields for a subscriber and get some other fields for the same subscriber. Beside this table there will be more tables of auxiallary nature.

 

1.2 Environment

The test machine is a Sun Enterprise 220R 1 cpu UltraSparc 450MHz 1 GByte RAM with Sun Solaris 2.8 operating system.

 

The target machine is probably a Sun Fire V280, 2 cpu UltraSparct 1GHz 8Gbyte RAM, there 1 CPU will be dedicated for the database application and the other for communication and other tasks.

 

1.3 Criteria

The necessary criteria for a successful evaluation is that it can perform 5000 operations per second on our test enviroment.

 

1.4 Products

Four products have been evaluated

 

A = FastDB (FreeWare)

B = Commercial MMDBs

C = Commercial MMDBs

D = Commercial disk based RDBMs

 

1.5 Test

First we tested how much transaction size did on performance. For all product where was a big difference between 1 operation/transaction and 10, but the difference up to 100 was less. For all test reported we are using a average transaction size of 144 operations per transaction.

 

The reported results are from three tests.


1.6 Results

The winner in each test is marked with bold font.

1.6.1 Relaxed disk usage

 

A

B

C

D

Operations

20000000

100000

100000

n/a

Ops/transaction

144

144

144

Entries

32768

31177

31177

SEL+INS/UPD

10'759 /s

7'732 /s

4'485 /s

SEL+UPD

10'509 /s

7241 /s

4429 /s

DB-size

20471808

7585896

8224768

DB-size/entry

625

243

264

1.6.2 Moderate disk usage

 

A

B

C

D

Operations

 

100000

100000

n/a

Ops/transaction

 

144

144

Entries

 

31177

31177

SEL+INS/UPD

 

4581 /s

3800 /s

SEL+UPD

 

4357 /s

3790 /s

DB-size

 

7610864

8224768

DB-size/entry

 

244

264

1.6.3 Intensive disk usage

 

A

B

C

D

Operations

100000

100000

100000

20000000

Ops/transaction

144

144

144

144

Entries

31177

31177

31177

32768

SEL+INS/UPD

1785 /s

3'284 /s

2'947 /s

1208 /s

SEL+UPD

1824 /s

1965 /s

2508 /s

1208 /s

DB-size

13320192

7610864

8224768

8192000

DB-size/entry

427

244

264

250

 

1.7 Conclusion

Only product A and B achieved our performance goal, and then only if one turned off disk logging and durable transactions. So for the time being we will probably not depend on any of the products. Initially we will build something using product A, but do it in such a way that it is easy to replace it in the future with a more suitable software when application is finished and the actual performance needs are known.