DB::put function is getting hanged

Post by r***@gmail.com
Am a fresher in BerkeleyDB. If i open a database file(using
transaction) for put(adding to DB) and if i tried to open the same
file in different transaction the put is hanging. It seems a deadlock
is occuring. Is it not allowed, the same database file to be opened in
more than one transactions?

It is not allowed.

Florian Weimer

2007-06-11 19:33:59 UTC

It's allowed, but only if the threads or processes executing the open
and the put are ordered in some way unknown to Berkeley DB. If you've
got a single process that tries to do these things, they are trivially
ordered, so it's not allowed. Another source of orderings is IPC.

r***@gmail.com

2007-06-12 01:20:41 UTC

Thanks. My concern is in a single process.

The number of database files in not defined in my work.
So i think I need to use a single transaction in my process. Will it
be OK to open more than 1 database files in a single transaction?

Don Anderson

2007-06-18 11:58:22 UTC

Thanks. My concern is in a single process.
The number of database files in not defined in my work.
So i think I need to use a single transaction in my process. Will it
be OK to open more than 1 database files in a single transaction?

It's okay, but probably not your best bet. Why not open each database
in
its own transaction (use autocommit flag so you don't even have to
manage
the transaction). The only reasons I know to keep a transaction going
past the end of the open would be 1) you are creating your set of
databases atomically and you want the behavior where either all your
databases were created or none. or 2) performance - you are opening
1000 databases, I'd guess you'd see an open using a single transaction
to be faster then separate transactions. Same advice goes for your
original question if I understand it. Open the database in its own
separate transaction (use autocommit), and then do puts in their own
transaction. That's the typical approach anyway.

r***@gmail.com

2007-06-21 09:13:42 UTC

Thanks. My concern is in a single process.
The number of database files in not defined in my work.
So i think I need to use a single transaction in my process. Will it
be OK to open more than 1 database files in a single transaction?

So you are saying that explicit transaction is not necessary, simply
auto-commit may be needed. So the implicit transaction which we get by
auto-commit will work, am I right?
I have one more doubt. Hope if the database count is less, then there
is no problem(even for performance) in creating multiple transaction
under the same environment, am i right?

Don Anderson

2007-06-23 02:34:03 UTC

Thanks. My concern is in a single process.
The number of database files in not defined in my work.
So i think I need to use a single transaction in my process. Will it
be OK to open more than 1 database files in a single transaction?

So you are saying that explicit transaction is not necessary, simply
auto-commit may be needed. So the implicit transaction which we get by
auto-commit will work, am I right?

Using the autocommit flag for an operation is roughly equivalent to:

begin transaction
ret = db.operation.....
if (ret is error)
abort transaction
else
commit transaction

It's a convenience, you can do it either way.
The important point in avoiding the hang is that you finish
(commit) the transaction that you used in the open,
and start a new transaction for any put operation.
Always doing an autocommit on an open is an easy way to
remember to avoid that mistake.

A transaction holds locks on various pages in the database
during its lifetime. Each operation for a transaction may acquire
new locks (for example, a Db::get will acquire locks on the page
containing the data). The only way for the transaction to give up
locks is to commit or abort the transaction. This is way we recommend
to keep
transactions short, and minimize the number of actions within a
given transaction.

The common idiom is to open a database autocommit and keep
the handle open, often for a long time, like the lifetime of the
application.
Since the transaction is done, no locks are held.
Then when you are doing a put, you can either do autocommit if it's
one operation, or if you have multiple puts/gets that need to be
isolated
or atomic, you group them all in a transaction. There is still the
potential for deadlock (depending on what you are doing with various
threads), but you are much better off than keeping the open within
your transaction. (....and there are ways around deadlocks.)

Post by r***@gmail.com
I have one more doubt. Hope if the database count is less, then there
is no problem(even for performance) in creating multiple transaction
under the same environment, am i right?

If I understand the question, you can have multiple transactions under
the same environment. If you keep your transactions short, you have
better chance at parallelism, which may increase your overall
performance.
For example, thread 1 with transaction T1 may be writing key "ABC" in
the database -- the thread may be blocked waiting on the read of a
disk block containing
the data to be updated. But meanwhile, thread 2 with transaction T2
can proceed
reading or writing key "GHI", which might already be in the cache or
it can
make its own disk request. The fewer locks you hold (i.e. smaller
your transaction),
the more chance of this kind of parallelism.

- Don

r***@gmail.com

2007-06-28 03:40:57 UTC

Thanks. My concern is in a single process.
The number of database files in not defined in my work.
So i think I need to use a single transaction in my process. Will it
be OK to open more than 1 database files in a single transaction?

So you are saying that explicit transaction is not necessary, simply
auto-commit may be needed. So the implicit transaction which we get by
auto-commit will work, am I right?

begin transaction
ret = db.operation.....
if (ret is error)
abort transaction
else
commit transaction
It's a convenience, you can do it either way.
The important point in avoiding the hang is that you finish
(commit) the transaction that you used in the open,
and start a new transaction for any put operation.
Always doing an autocommit on an open is an easy way to
remember to avoid that mistake.
A transaction holds locks on various pages in the database
during its lifetime. Each operation for a transaction may acquire
new locks (for example, a Db::get will acquire locks on the page
containing the data). The only way for the transaction to give up
locks is to commit or abort the transaction. This is way we recommend
to keep
transactions short, and minimize the number of actions within a
given transaction.
The common idiom is to open a database autocommit and keep
the handle open, often for a long time, like the lifetime of the
application.
Since the transaction is done, no locks are held.
Then when you are doing a put, you can either do autocommit if it's
one operation, or if you have multiple puts/gets that need to be
isolated
or atomic, you group them all in a transaction. There is still the
potential for deadlock (depending on what you are doing with various
threads), but you are much better off than keeping the open within
your transaction. (....and there are ways around deadlocks.)

Thanks for the detailed information.

r***@gmail.com

2007-06-28 03:45:37 UTC