BDB 4.7.25 not deleting log files

[z***@gmail.com]

Hi all,

I am having a bit of an issue with the log files - they just keep
growing and dont ever get deleted. Even if I manually run db_archive
( even after I close the app down ).

I have a a bdb cache using 4.7.25 with the following flags:

DbEnv Flags:
DB_CREATE | DB_INIT_MPOOL | DB_INIT_TXN | DB_INIT_LOG | DB_INIT_LOCK
| DB_THREAD

Db Flags:
DB_CREATE | DB_AUTO_COMMIT | DB_DIRTY_READ

My requirements are:
Multi threaded writing into the db and a replication thread to copy
data over to another instance where the reads would happen.

In the code, I commit after every put ( unless it fails due to a
locking conflict , in which case I retry a few times (in a loop) and
if it still fails, I abort)

It sounds like the log files are just not freed of some lock or
something and hence they can't be deleted, but I can't put my finger
on it (not very familiar with bdb).


_dbEnv.set_timeout(100000, DB_SET_TXN_TIMEOUT);
_dbEnv.set_tx_max(40);
_dbEnv.set_lk_detect(DB_LOCK_MINWRITE);
_dbEnv.log_set_config(DB_LOG_AUTO_REMOVE, 1);
_dbEnv.open(env_name.c_str(), env_flags, 0);

_db = new Db(&_dbEnv, 0);
_db->open(NULL, "mydb", NULL, DB_HASH, db_flags,0);


Then its simple put / commit or abort ( ).

By the way, I dont have any checkpoiint calls , could that be my
problem?


Please help!

-Zer0Frequency

[Florian Weimer]

Post by z***@gmail.com
By the way, I dont have any checkpoiint calls , could that be my
problem?

Yes, log files are only removed after a checkpoint, after the data has
actually hit the disk.

[z***@gmail.com]

Post by Florian Weimer

Post by z***@gmail.com
By the way, I dont have any checkpoiint calls , could that be my
problem?

Yes, log files are only removed after a checkpoint, after the data has
actually hit the disk.

Thanks for your reply, but I thought checkpoints just flush the
current log file and uncommitted stuff. In my situation, new log files
are constantly being created, for exmaple, when the files have reached
upto log.1000, I would imagine, log.0001 can be safely removed?

But even db_archive -d doesn't remove a single log file.

Thanks again

[Florian Weimer]

Post by z***@gmail.com

Post by Florian Weimer

Post by z***@gmail.com
By the way, I dont have any checkpoiint calls , could that be my
problem?

Yes, log files are only removed after a checkpoint, after the data has
actually hit the disk.

Thanks for your reply, but I thought checkpoints just flush the
current log file and uncommitted stuff. In my situation, new log files
are constantly being created, for exmaple, when the files have reached
upto log.1000, I would imagine, log.0001 can be safely removed?

Not necessarily. This depends on factors such as log file and cache
file size. It's only permitted to remove those log files if the
transactions recorded within them have completed, and their updates
are reflected in the on-disk database files. A checkpoint ensures
this.