-
- Using Citadel/UX with Sleepycat (Berkeley) DB
-
-Abstract
-
- Citadel/UX can now use the robust and scalable Berkeley DB from
- Sleepycat Software as its data store, for increased scalability,
- reliability, and recoverability.
-
-History and introduction
-
- From its inception in 1987 until versions 5.1x in 1998,
- Citadel/UX utilized a built-in data store loosely modeled after Jeff
- Prothero's original Citadel-CP/M design. But as Citadel systems
- scaled upwards, supporting Internet-connected systems with heavy
- concurrent use, and aspirations of becoming a world-class
- messaging/groupware platform someday, the developers made the decision
- to switch to an embedded database. The Free Software Foundation's
- GDBM product was chosen for its simple API and its free license
- (the LGPL).
-
- Somewhat less than trouble-free operation from 1998 through 2000,
- however, proved that GDBM was not the best choice. Heavily utilized
- systems experienced occasional database corruption, often resulting in
- repeated crashes of the Citadel server. As a result, we made the
- decision to switch to Berkeley DB.
-
- Berkeley DB offers numerous features which help Citadel/UX to meet
- its goals as a high-end messaging platform:
- * Database sizes can scale to hundreds of terabytes
- * A transaction-based logging system
- * Recovery utilities
-
- It is clear that Berkeley DB is a better choice than GDBM for a
- high-utilization database that requires crash recovery. Citadel/UX can
- currently be built with either DB or GDBM as the data store; however,
- THE USE OF GDBM IS DEPRECATED AND STRONGLY DISCOURAGED. If you are
- bringing up a new site you should use Berkeley DB, period. If you are
- maintaining an existing site using GDBM you should migrate it to Berkeley
- DB as soon as possible, because support for it will be dropped in a future
- release.
-
-
-Building Citadel/UX with DB support
-
- If you are running Citadel/UX on a Red Hat Linux system, you don't have
- to build Berekeley DB from source. Simply install the RPM's for "db3"
- and "db3-devel" (the latter may be on the second CD) and you're ready to
- run Citadel.
-
- Otherwise, here are the steps required to get Citadel running with
- Berkeley DB as its back end data store.
-
- 1. First, you must download and build Berkeley DB itself. Citadel
- has been developed and tested with DB 3.1.17, which can be
- downloaded from www.sleepycat.com. Follow the "Building for
- UNIX" instructions. Make sure that you run the test suite, and
- perhaps test with some of the sample applications, before moving
- on.
-
- Note that as of DB 3.1.17, DB's configure script can't check for
- OpenBSD systems. You will need to configure DB as follows:
-
- env CC='gcc -pthread -Di386' ../dist/configure [flags]
-
- DB 3.3.x's configure script should not need the -Di386 flag.
-
- 2. Start with a clean source tree. Either unpack a fresh copy of the
- source or do a "make distclean" before continuing.
- 3. Run the configure script: ./configure
- (Optionally specify --with-db=DIR for whatever prefix
- you've actually installed DB into, if you overrode DB's defaults.
- Also specify any other configure options you need at this time.)
- 4. Run "make" and "make install-exec." Continue installing Citadel
- as per the instructions supplied with the system.
-
-Migrating an existing GDBM-based Citadel to a DB-based Citadel
-
- If you have an existing system, you must export your databases,
- rebuild Citadel with DB support, and then re-import your databases
- into the new system. Please refer to the document "How to use the
- importer/exporter" for detailed instructions on this.
-
- After you export your database, but before you re-import it, you must
- perform the following steps:
- 1. Re-build Citadel with DB support, as described above
- 2. Remove all of the *.gdbm files from your data directory.
-
-Care and feeding of your DB-powered Citadel
-
- Citadel uses the transaction-based logging facility of Berkeley DB.
- Therefore you will notice log files accumulating in your data
- directory. These are required for automatic recovery in the event of
- a catastrophic system failure. Log files have filenames that look
- like "log.0000000001" whereas the normal database files have names
- like "cdb.05".
-
- So do you have to keep these log files around forever? No, but there
- are some rules you should follow:
- * Don't remove a log file if it's the only log file there.
- * If it's not listed in the output of the db_archive command, it's
- not safe to remove.
- * After a successful backup (see below) log files listed by the
- db_archive command may be removed to conserve disk space, if
- those log files were backed up.
-
- You may think that it's going to keep writing to that one log file
- forever, but don't panic; when the log file gets sufficiently large it
- will switch over to another one. As a general rule of thumb, your
- archival procedure should be to back up to tape every day. Berkeley DB
- supports "hot" backups; in other words, you are permitted to back up your
- Citadel data without having to first shut down the Citadel server, as long
- as you copy the data files before the log files.
-
- And don't worry about your system filling up with log files; the Citadel
- server will automatically remove them when they're no longer needed.
-
-
projects / citadel.git / commitdiff