1.1 Application

Our target application is a subscriber database for a mediation device. The database will be accessed be I single thread on a single machine.

There will be a main table with 20’000’000 entries about 500 bytes each. And the application must be able to do 10’000 operations a second ther an operation is modify some fields for a subscriber and get some other fields for the same subscriber. Beside this table there will be more tables of auxiallary nature.

1.2 Environment

The test machine is a Sun Enterprise 220R 1 cpu UltraSparc 450MHz 1 GByte RAM with Sun Solaris 2.8 operating system.

The target machine is probably a Sun Fire V280, 2 cpu UltraSparct 1GHz 8Gbyte RAM, there 1 CPU will be dedicated for the database application and the other for communication and other tasks.

1.3 Criteria

The necessary criteria for a successful evaluation is that it can perform 5’000 operations per second on our test enviroment.

1.4 Products

Four products have been evaluated

A = FastDB (FreeWare)

B = Commercial MMDBs

C = Commercial MMDBs

D = Commercial disk based RDBMs

1.5 Test

First we tested how much transaction size did on performance. For all product where was a big difference between 1 operation/transaction and 10, but the difference up to 100 was less. For all test reported we are using a average transaction size of 144 operations per transaction.

The reported results are from three tests.

Relaxed disk usage: here we configured the database to work as little as possible with disk, ie we turned off logging and also allowed transaction to be commited but not syncronized on disk.
Moderate disk usage: here we turned on disk logging
Intensive disk usage: here both logging and syncronized commits are on.

1.6 Results

The winner in each test is marked with bold font.

1.6.1 Relaxed disk usage

	A	B	C	D
Operations	20’000’000	100’000	100’000	n/a
Ops/transaction	144	144	144
Entries	32’768	31’177	31’177
SEL+INS/UPD	10'759 /s	7'732 /s	4'485 /s
SEL+UPD	10'509 /s	7’241 /s	4’429 /s
DB-size	20’471’808	7’585’896	8’224’768
DB-size/entry	625	243	264

1.6.2 Moderate disk usage

	A	B	C	D
Operations		100’000	100’000	n/a
Ops/transaction		144	144
Entries		31’177	31’177
SEL+INS/UPD		4’581 /s	3’800 /s
SEL+UPD		4’357 /s	3’790 /s
DB-size		7’610’864	8’224’768
DB-size/entry		244	264

1.6.3 Intensive disk usage

	A	B	C	D
Operations	100’000	100’000	100’000	20’000’000
Ops/transaction	144	144	144	144
Entries	31’177	31’177	31’177	32’768
SEL+INS/UPD	1’785 /s	3'284 /s	2'947 /s	1’208 /s
SEL+UPD	1’824 /s	1’965 /s	2’508 /s	1’208 /s
DB-size	13’320’192	7’610’864	8’224’768	8’192’000
DB-size/entry	427	244	264	250

1.7 Conclusion

Only product A and B achieved our performance goal, and then only if one turned off disk logging and durable transactions. So for the time being we will probably not depend on any of the products. Initially we will build something using product A, but do it in such a way that it is easy to replace it in the future with a more suitable software when application is finished and the actual performance needs are known.