Secondary Indices

[Mike]

I'm starting to use Berkeley-DB on a Linux system.

Is there a way of permanently associating a secondary index with a
primary rather than having to do it explicitly in each application?

I'm looking for the ability to add indices to a database, and
maintaining them, without have to recompile my applications. I can
define the keys in terms of integer pairs (location and length of
field in the record) so I can write a generic "callback" function to
extract the key. I could maintain a list of indices in a file and
write an "open database" function to open the primary and associate
the secondaries at run time.

Ideally, I'd like to get something similar to the system of "physical"
and "logical" files used on the AS400 (for anyone familiar with such
machines).

Any suggestions (other than "get yourself an AS400 then" ;-) ) would
be much appreciated.

[Florian Weimer]

Post by Mike
Is there a way of permanently associating a secondary index with a
primary rather than having to do it explicitly in each application?

No, I don't think so.

Post by Mike
I'm looking for the ability to add indices to a database, and
maintaining them, without have to recompile my applications.

You could add this information to a configuration file, or a
designated database.

Post by Mike
I can define the keys in terms of integer pairs (location and length
of field in the record) so I can write a generic "callback" function
to extract the key.

Sounds useful, but in some cases (most cases for me), column offsets
vary with column widths.

[Mike]

Post by Florian Weimer

Post by Mike
Is there a way of permanently associating a secondary index with a
primary rather than having to do it explicitly in each application?

No, I don't think so.

OK, it seemed like that to me. I was just hoping against hope. :-(

Post by Florian Weimer

Post by Mike
I'm looking for the ability to add indices to a database, and
maintaining them, without have to recompile my applications.

You could add this information to a configuration file, or a
designated database.

Yes, that's the sort of thing I was thinking of - a database keyed on
"filename" holding a list of secondary indices. The application would
be written to read through that database and open all the secondaries,
together with the primary, and automatically associate them. My
difficulty is in extracting the key fields from the data record. The
"callback" function prototype doesn't allow for suitable parameters to
be passed so, it seems, I need a separate "callback" function for each
secondary.

Post by Florian Weimer

Post by Mike
I can define the keys in terms of integer pairs (location and length
of field in the record) so I can write a generic "callback" function
to extract the key.

Sounds useful, but in some cases (most cases for me), column offsets
vary with column widths.

Yes, the method is only really suitable for fixed length records/fields.

Thanks for your reply.

[Patrick Schaaf]

[...] My difficulty is in extracting the key fields from the data record.
The "callback" function prototype doesn't allow for suitable parameters to
be passed so, it seems, I need a separate "callback" function for each
secondary.

The callback function gets passed a DB* (of the secondary database),
and each of these has an app_private void* field that you can use any
way you like. You could point it to a suitable const key extraction
info struct when you open the secondary database.

best regards
Patrick

[Mike]

Post by Patrick Schaaf

[...] My difficulty is in extracting the key fields from the data record.
The "callback" function prototype doesn't allow for suitable parameters to
be passed so, it seems, I need a separate "callback" function for each
secondary.

The callback function gets passed a DB* (of the secondary database),
and each of these has an app_private void* field that you can use any
way you like. You could point it to a suitable const key extraction
info struct when you open the secondary database.

Thanks, Patrick. That looks like just the thing I need. :-)

[Florian Weimer]

Post by Mike

Post by Florian Weimer

Post by Mike
Is there a way of permanently associating a secondary index with a
primary rather than having to do it explicitly in each application?

No, I don't think so.

OK, it seemed like that to me. I was just hoping against hope. :-(

Sometimes, people joke that the library you are looking for is called
"MySQL".

Post by Mike
Yes, that's the sort of thing I was thinking of - a database keyed on
"filename" holding a list of secondary indices. The application would
be written to read through that database and open all the secondaries,
together with the primary, and automatically associate them. My
difficulty is in extracting the key fields from the data record. The
"callback" function prototype doesn't allow for suitable parameters to
be passed so, it seems, I need a separate "callback" function for each
secondary.

In addition to what Patrick wrote, it might be possible to use
subclassing in the C++ version of the library (I haven't checked).
You could also keep a hash table of DB pointers. My concern is that
once you use the app_private field, you lose generality (because some
application might already be using it).

[Mike]

Post by Florian Weimer
In addition to what Patrick wrote, it might be possible to use
subclassing in the C++ version of the library (I haven't checked).
You could also keep a hash table of DB pointers. My concern is that
once you use the app_private field, you lose generality (because some
application might already be using it).

Using the app_private field seems to be the answer to my problem. I
intend to write some functions "on top of" BerkeleyDB for my
application so losing generality shouldn't be a problem: I shan't
affect any existing application. If nobody used app_private, for fear
of some other application using it, there would be no point in having
it. ;-)