Art Cancro [Thu, 31 Aug 2023 04:03:53 +0000 (00:03 -0400)]
Rewrite of the indexer is complete!
No more search hooks. Modules can call each other now and that's a lot simpler.
We also eliminated the index cache, because it made things a lot more complex, and
a modern database is going to mmap most of the index as it gets used anyway.
Art Cancro [Sun, 27 Aug 2023 00:29:07 +0000 (20:29 -0400)]
loadtest.c: when deleting, always make sure we deleted 1 message.
If there are multiple loadtest processes running -- and that's really
the whole idea -- we will often try to delete a message that was already
deleted by another process. That's not an error condition, but it would
cause posts to outnumber deletes and the rooms would grow endlessly. So
if we didn't successfully delete something, we go back and try deleting
another one.
Fun fact: the algorithm used to decide which message to delete is the same
algorithm used in the 'fortune' program.
The use of the DB_RECOVER flag to dbenv->open() appears to do more than just try to
recover the environment upon startup. It also creates __db.00X files, which are
perhaps holding the environment??? In any case it seems to be more solid this way.
I am also removing the use of any "private" transactions, since they are no longer
needed. Operations which write to the database check to see if the thread's dedicated
transaction pointer is in effect, maintaining that transaction if it is, and creating
a transaction in the same pointer otherwise.
There is a "double free" bug somewhere in here and I'm still trying to figure out what
conditions cause it to happen.
Art Cancro [Thu, 24 Aug 2023 18:44:37 +0000 (09:44 -0900)]
ctdldump: use the backend API instead of BDB calls
This makes ctdldump backend agnostic, or at least as backend
agnostic as citserver itself is at this point. We will probably
add some calling syntax across the board when we add more
backends, but at least this utility doesn't have to be rewritten
now. This makes me a happy ig. ctdlload will be next.
Art Cancro [Thu, 24 Aug 2023 13:55:33 +0000 (04:55 -0900)]
Makefile: restructured so that backends can be compiled into utilities
database.c has been moved from server/ into server/backends/common.
Makefile now separates server objects from backend objects.
This is all in preparation for ctdldump and ctdlload to link the backend
code instead of calling BDB directly.
Before now, the Berkeley DB back end was supplied the DB_DBT_MALLOC flag
and it became the caller's responsibility to handle any memory allocated
and returned by cdb_fetch() and cdb_next_item(). These heuristics are
not expected to be compatible with new back ends currently in planning.
cdb_fetch() and cdb_next_item() now return data that is to be considered
immutable (don't write to it or you'll segfault). The caller must still
call cdb_free() to free the `struct cdbdata` itself, but the pointers
inside it point to buffers that the back end will reuse.
After a call to cdb_fetch() or cdb_next_item(), the buffers are
guaranteed to be readable until the next call to one of those functions
on the same table.
Art Cancro [Thu, 17 Aug 2023 22:32:56 +0000 (13:32 -0900)]
CtdlFetchMsgList() - treat no-record and zero-messages identically
In both cases, we return 0 to the caller and set the array pointer to NULL.
This simplifies the calling convention and eliminates extra code to check for both conditions
Art Cancro [Wed, 16 Aug 2023 03:14:24 +0000 (18:14 -0900)]
DRY fetch of msglists.
A lot of the calls to cdb_fetch(CDB_MSGLISTS...) follow the same design pattern
and many of them are in need of rework to handle how they free their memory. So
I created a new function CtdlGetMsgList() to do it in a consistent way. Three
callers have been migrated to the new syntax.
Art Cancro [Fri, 11 Aug 2023 15:46:34 +0000 (06:46 -0900)]
Moved cdb_chmod_data() out of the backend API
This function only does file and directory level operations on
the data/ directory and files in it. These operations will be
common across all backends.
Art Cancro [Thu, 10 Aug 2023 14:49:50 +0000 (05:49 -0900)]
Prepared for upcoming change to cdb_next_item() API.
cdb_next_item() is always preceded by a call to cdb_rewind() before the
iteration loop. We're going to remove the cursor from thread-specific
data, and replace it with a void pointer that the caller is responsible
for passing back. This will make the backend drivers easier to build.