• Benchmark
• Results
• Comparison

JPA Performance Benchmark (JPAB)

Comparison of EclipseLink with SQLite embedded vs ObjectDB server

Each of the following tables focuses on a specific database operation, where the last table presents average results comparison.

Speed comparison of JPA database persistence operations (normalized score, higher is better)

Transaction Size =>	Few Entities		Many Entities		Average Score
	EclipseLink SQLite embedded	ObjectDB server	EclipseLink SQLite embedded	ObjectDB server	EclipseLink SQLite embedded	ObjectDB server
Basic Person Test	0.074	35.2	0.90	74.1	0.49	54.7
Element Collection Test	0.061	38.2	0.65	57.8	0.36	48.0
Inheritance Test	0.083	32.7	0.82	72.8	0.45	52.7
Indexing Test	0.11	49.9	1.6	90.1	0.87	70.0
Graph (Binary Tree) Test	0.24	100	0.91	89.2	0.57	94.6
Multithreading Test	0.10	76.6	2.1	99.4	1.1	88.0
All Tests	0.11	55.4	1.2	80.6	0.64	68.0

The results above show that in general ObjectDB server is much more efficient than EclipseLink with SQLite embedded in persisting JPA entity objects to the database. Comparing the normalized speed of EclipseLink with SQLite embedded database (0.64) to the normalized speed of ObjectDB database server (68.0) reveals that in these tests, ObjectDB server is 106 times faster than EclipseLink with SQLite embedded.

A huge performance gap has been detected when using multithreading with small transaction size. Comparing the normalized speed of EclipseLink with SQLite embedded database (0.10) to the normalized speed of ObjectDB database server (76.6) reveals that in that case, ObjectDB server is 766 times faster than EclipseLink with SQLite embedded.

Speed comparison of JPA database retrieval operations (normalized score, higher is better)

Retrieval Size =>	Few Entities		Many Entities		Average Score
	EclipseLink SQLite embedded	ObjectDB server	EclipseLink SQLite embedded	ObjectDB server	EclipseLink SQLite embedded	ObjectDB server
Basic Person Test	failed	15.7	failed	45.4	failed	30.5
Element Collection Test	0.0007	19.9	failed	49.2	0.0007	34.6
Inheritance Test	failed	13.2	failed	49.4	failed	31.3
Indexing Test	failed	14.0	failed	59.5	failed	36.8
Graph (Binary Tree) Test	failed	19.9	failed	24.7	failed	22.3
Multithreading Test	failed	19.7	failed	60.2	failed	40.0
All Tests	0.0007	17.1	failed	48.1	0.0007	32.6

EclipseLink with SQLite embedded has failed in 11 tests (see exceptions).

Speed comparison of JPA database query operations (normalized score, higher is better)

Retrieval Size =>	Few Entities		Many Entities		Average Score
	EclipseLink SQLite embedded	ObjectDB server	EclipseLink SQLite embedded	ObjectDB server	EclipseLink SQLite embedded	ObjectDB server
Basic Person Test	1.2	38.8	failed	25.9	1.2	32.4
Element Collection Test	0.29	39.5	failed	33.1	0.29	36.3
Inheritance Test	0.69	34.8	failed	34.0	0.69	34.4
Indexing Test	0.0008	14.5	failed	53.3	0.0008	33.9
Multithreading Test	failed	50.8	failed	52.1	failed	51.5
All Tests	0.55	35.7	failed	39.7	0.55	37.7

EclipseLink with SQLite embedded has failed in 6 tests (see exceptions).

Speed comparison of JPA database update operations (normalized score, higher is better)

Transaction Size =>	Few Entities		Many Entities		Average Score
	EclipseLink SQLite embedded	ObjectDB server	EclipseLink SQLite embedded	ObjectDB server	EclipseLink SQLite embedded	ObjectDB server
Basic Person Test	failed	18.3	failed	40.9	failed	29.6
Element Collection Test	0.0016	21.1	failed	53.7	0.0016	37.4
Inheritance Test	failed	19.9	failed	51.9	failed	35.9
Indexing Test	failed	17.9	failed	52.1	failed	35.0
Graph (Binary Tree) Test	failed	42.6	failed	24.1	failed	33.3
Multithreading Test	failed	68.6	failed	68.7	failed	68.6
All Tests	0.0016	31.4	failed	48.6	0.0016	40.0

EclipseLink with SQLite embedded has failed in 11 tests (see exceptions).

Speed comparison of JPA database removal operations (normalized score, higher is better)

Transaction Size =>	Few Entities		Many Entities		Average Score
	EclipseLink SQLite embedded	ObjectDB server	EclipseLink SQLite embedded	ObjectDB server	EclipseLink SQLite embedded	ObjectDB server
Basic Person Test	failed	20.2	failed	67.7	failed	44.0
Element Collection Test	0.0006	22.9	failed	57.6	0.0006	40.2
Inheritance Test	failed	23.1	failed	61.6	failed	42.3
Indexing Test	failed	34.9	failed	55.9	failed	45.4
Graph (Binary Tree) Test	failed	19.3	failed	27.9	failed	23.6
Multithreading Test	failed	38.5	failed	50.5	failed	44.5
All Tests	0.0006	26.5	failed	53.5	0.0006	40.0

EclipseLink with SQLite embedded has failed in 11 tests (see exceptions).

Comparison of JPA/Database speed - the averages (normalized score, higher is better)

Transaction/Retrieval Size	Few Entities		Many Entities		Average Score
	EclipseLink SQLite embedded	ObjectDB server	EclipseLink SQLite embedded	ObjectDB server	EclipseLink SQLite embedded	ObjectDB server
Basic Person Test	0.66	25.6	0.90	50.8	0.74	38.2
Element Collection Test	0.070	28.3	0.65	50.3	0.17	39.3
Inheritance Test	0.38	24.7	0.82	53.9	0.53	39.3
Indexing Test	0.058	26.2	1.6	62.2	0.58	44.2
Graph (Binary Tree) Test	0.24	45.4	0.91	41.5	0.57	43.5
Multithreading Test	0.10	50.8	2.1	66.2	1.1	58.5
All Tests	0.22	33.1	1.2	54.6	0.52	43.9

The results above show that in general ObjectDB server is much more efficient than EclipseLink with SQLite embedded in performing JPA database operations. Comparing the normalized speed of EclipseLink with SQLite embedded database (0.52) to the normalized speed of ObjectDB database server (43.9) reveals that in these tests, ObjectDB server is 84.4 times faster than EclipseLink with SQLite embedded.

A huge performance gap has been detected when using multithreading with small transaction/retrieval size. Comparing the normalized speed of EclipseLink with SQLite embedded database (0.10) to the normalized speed of ObjectDB database server (50.8) reveals that in that case, ObjectDB server is 508 times faster than EclipseLink with SQLite embedded.

Other Head to Head DBMS/JPA Comparisons

JPA Performance Benchmark - Copyright © 2010, 2011, 2012 ObjectDB Software Ltd. - All rights reserved.
The benchmark program is released as open source under the GPL license.