devzero2000 wrote:
On Thu, Mar 21, 2013 at 9:58 PM, Howard Chu <hyc@symas.com
<mailto:hyc@symas.com>> wrote:
Tobias Oberstein wrote:
Hello,
I have read the - very interesting - performance comparison
http://symas.com/mdb/__microbench/ <http://symas.com/mdb/microbench/>
I'd like to ask if someone did benchmark LMDB (and/or the others)
against http://www.garret.ru/fastdb.__html
<http://www.garret.ru/fastdb.html>
FastDB is an in-memory ACID database that works via shadow paging, and
without a transaction log.
OK, like LMDB it uses shadow root pages. I think the similarity ends there.
It is a relational database with an ASCII query language, while LMDB is
strictly a key/value store. That automatically means for simple get/put
operations LMDB will be orders of magnitude faster (just as it is so much
faster than SQLite3 and SQLite4).
I Not being so young, these considerations remind me of the same arguments
that there was about hierarchical DBMS and the relational one. I do not mean
that there are not still real, but just to tell that we are talking about
topic 20 years old. Isn't it ?
In some ways yes. Because it seems today's crop of NoSQL programmers haven't studied and learned from the lessons of 30+ years ago.
Sure there's a time and a place for a relational database. And LMDB actually supports them - there's an LMDB backend for SQLite3, there will be one for SQLite4 and MariaDB as well.
One of the other important lessons of software design that a lot of the NoSQL folks seem to have missed is having a good abstraction layer between your frontend query handler and your backend storage engine. SQLite4 and MariaDB don't make this mistake. SQLite3 does, and porting LMDB to it was a royal pain. Also the SQLite3 test suite doesn't pass because many of the tests assume they know the actual binary format of the on-disk files. Tightly marrying your data storage engine like this is always a mistake. It's the same mistake MongoDB and CouchBase makes. OpenLDAP slapd has been so flexible precisely because we've always maintained a distinction between the frontend and the backend, which has allowed us to easily experiment with multiple backend technologies over the years.
Another mistake most of these folks make is using high level languages and slow APIs and ASCII query languages. They believe the BS that Intel has been shoveling for the past couple decades, that CPUs are always faster so there's no point in worrying about the CPU cost of an individual function. But reality is that CPU speeds have maxed out, and CPU power is only increasing by virtue of multi-core expansion. Also they assume that network speeds are so fast today that inefficient codings don't matter. Reality is that the fastest growing segment of data consumption today is mobile computing, and most devices are connected to networks delivering 2G speed. Even the deployed 3G and 4G networks seldom deliver their rated speeds, nor can they hope to do so in congested/heavily populated locations.
Efficiency always matters. Bloat is always wrong. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/