The entire package is open source; you can +

The entire package is open source software. You may redistribute and/or modify it under the terms of the GNU General Public -License as published by the Free Software Foundation; either version 2 -of the License, or (at your option) any later version.
+License, version 2, which is included in this manual.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

For more information, visit either of these locations on @@ -136,7 +148,7 @@ the web:

UNCENSORED! BBS, the home of Citadel: http://uncensored.citadel.org

License
Database maintenance

Introduction
Backing up your +Citadel database
+
Database repair
Importing/Exporting your Citadel database
@@ -241,7 +256,7 @@ LDAP Connector for Citadel

GNU General Public License

@@ -546,7 +561,7 @@ DAMAGES.

END OF TERMS AND CONDITIONS

Installation

@@ -573,7 +588,7 @@ processing) the system. It is mandatory that the sysop have "root" access to the operating system. The following are required to install Citadel:

A Unix operating system (Linux, BSD, Solaris, Tru64, HP/UX)
A unix-like operating system (Linux, FreeBSD, Solaris, etc.)
C compiler (GCC with gmake is the recommended build environment)

Enough disk space to hold all of the programs and data

If you are running Citadel on a Linux system, it is STRONGLY -recommended that you run it on a recent distribution (such as Red Hat 7.3 or newer). A new-ish +recommended that you run it on a recent distribution (such as Fedora +Core 3 or newer). A new-ish distribution will have most or all of the prerequisite tools and libraries already integrated for you.

Now available:

Other pieces which complete the Citadel system:

"WebCit", a gateway program to allow full access to Citadel via the World Wide Web. Interactive access through any Web browser.
Access to Citadel via any standards-compliant e-mail program, thanks to Citadel's built-in SMTP, POP, and IMAP services. You can use Mozilla, Netscape, Evolution, Eudora, Pine, Outlook, etc. -with Citadel.
Access to Citadel's calendar and address book functions using any +GroupDAV-enabled PIM client (requires WebCit).
+

Coming soon:

Newer and better GUI-based clients.
More integration with third-party software.
+

Everything in its place...

Hopefully you've unpacked the distribution archive into its own @@ -633,11 +652,11 @@ will use /usr/local/citadel) and the shell should be either "citadel" in that directory, or a script that will start up the citadel client. Example:

bbs::100:1:Citadel Login:/usr/local/citadel:/usr/local/citadel/citadel

citadel::100:1:Citadel Login:/usr/local/citadel:/usr/local/citadel/citadel

When you run setup later, you will be required to tell it the username or user ID of the account you created is, so it knows what -user to run as. If you create an account called bbs, guest, -or citadel, the setup program will automatically pick up the +user to run as. If you create an account called citadel, bbs, +or guest, the setup program will automatically pick up the user ID by default.

For all other users in /etc/passwd (or in some other name service such as NIS), Citadel will automatically set up @@ -756,7 +775,9 @@ your favorite text editor to write messages. To do this you simply put a line in your citadel.rc file like this:

editor=/usr/bin/vi

The above example would make Citadel call the vi editor when using -the .Enter Editor command. You can also make +the .Enter Editor command, or when a user +selects the "Always compose messages with the full-screen +editor" option. You can also make it the default editor for the Enter command by editing the citadel.rc file. But be warned: external editors on public systems can @@ -776,8 +797,9 @@ else in your system. The variable PRINTCMD in citadel.rc specifies what command you use to print. Text is sent to the standard input (stdin) of the print command.

So if you did this:

printcmd="nl|pr|lpr -Plocal"

...that would add line numbers, then paginate, then print on the +

printcmd="a2ps -o - |lpr -Plocal"

...that would convert the printed text to PostScript, then print on +the printer named "local". There's tons of stuff you can do with this feature. For example, you could use a command like cat <<$HOME/archive to save copies of important messages in a @@ -982,7 +1004,7 @@ install ??

To report a problem, you can log on to UNCENSORED! or any other BBS -on the Citadel network which carries the Citadel> room. +on the Citadel network which carries the Citadel/UX> room. Please DO NOT e-mail the developers directly. Post a request for help on the BBS, with all of the following information:

@@ -998,7 +1020,7 @@ command).

@@ -1298,6 +1320,12 @@ messages around forever (or until they are manually deleted), until they become a certain number of days old, or until a certain number of additional messages are posted in the room, at which time the oldest ones will scroll out.

When a new Citadel system is first installed, the default +system-wide +expire policy is set to 'manual' -- no automatic purging of messages +takes place anywhere. For public message boards, you will probably want +to set some sort of automatic expire policy, in order to prevent your +message base from growing forever.

You will notice that you can also fall back to the default expire policy for the floor upon which the room resides. This is the default setting. You can change the floor's default with the ;Aide Edit @@ -1561,7 +1589,8 @@ rooms, because the review of content is considered one of their roles. If you wish to change these policies, the next two options allow you to. You may 'Allow Aides to Zap (forget) rooms', in which case they may use the Zap command just like any other user. -Aides may also .Goto any private mailbox belonging to any +Aides may also .Goto any private mailbox belonging to +any user, using a special room name format.

If your local security and/or privacy policy dictates that you keep a @@ -1574,7 +1603,7 @@ only the sender and recipient of each individual message will receive a copy. The next set of options deals with the tuning of your system. It is usually safe to leave these untouched. -Server connection idle timeout (in seconds) [900]: Maximum concurrent sessions [20]: Maximum message length [2147483647]: Minimum number of worker threads [5]: Maximum number of worker threads [256]: +Server connection idle timeout (in seconds) [900]: Maximum concurrent sessions [20]: Maximum message length [10000000]: Minimum number of worker threads [5]: Maximum number of worker threads [256]: Automatically delete committed database logs [Yes]: The 'Server connection idle timeout' is for the connection between client and server software. It is not an idle timer for the user interface. 900 seconds (15 minutes) is the default and a sane @@ -1598,17 +1627,25 @@ worker thread model, Citadel can handle a large number of concurrent sessions with a much smaller thread pool. If you don't know the programming theory -behind multithreaded servers, you should leave these parameters alone. +behind multithreaded servers, you should leave these parameters alone. + +'Automatically delete committed database logs' is a crucial setting which affects your +system's disk utilization and backup recoverability. Please refer +to the database maintenance +section of this document to learn how the presence or absence of +database logs affect your ability to reliably backup your Citadel +system. + The next set of options affect how Citadel behaves on a network. -How often to run network jobs (in seconds) [3600]: POP3 server port (-1 to disable) [110]: IMAP server port (-1 to disable) [143]: SMTP MTA server port (-1 to disable) [25]: SMTP MSA server port (-1 to disable) [587]: Correct forged From: lines during authenticated SMTP [Yes]: -"How often to run network jobs" refers to the sharing of content on -a -Citadel network. If your system is on a Citadel network, this -configuration -item dictates how often the Citadel server will contact other Citadel -servers to send and receive messages. In reality, this will happen more -frequently than you specify, because other Citadel servers will be -contacting yours at regular intervals as well. +Server IP address (0.0.0.0 for 'any') [0.0.0.0]: POP3 server port (-1 to disable) [110]: POP3S server port (-1 to disable) [995]: IMAP server port (-1 to disable) [143]: IMAPS server port (-1 to disable) [993]: SMTP MTA server port (-1 to disable) [25]: SMTP MSA server port (-1 to disable) [587]: SMTPS server port (-1 to disable) [465]: Correct forged From: lines during authenticated SMTP [Yes]: Allow unauthenticated SMTP clients to spoof my domains [No]: No Instantly expunge deleted IMAP messages [No]: Yes +"Server IP address" refers to the IP address on your server to which Citadel's +protocol services should be bound. Normally you will leave this +set to 0.0.0.0, which will cause Citadel to listen on all of your +server's interfaces. However, if you are running multiple +Citadels on a server with multiple IP addresses, this is where you +would specify which one to bind this instance of Citadel to. Then you can specify TCP port numbers for the SMTP, POP3, and IMAP services. For a system being used primarily for Internet e-mail, these are essential, so you'll want to specify the standard port numbers: 25, @@ -1616,16 +1653,39 @@ are essential, so you'll want to specify the standard port numbers: 25, though, then you might want to choose other, unused port numbers, or enter -1 for any protocol to disable it entirely. -You'll also notice that you can specify two port numbers for SMTP: one -for MTA (Mail Transport Agent) and one for MSA (Mail Submission Agent). The -traditional ports to use for these purposes are 25 and 587. If you are -running an external MTA, such as Postfix (which submits mail to Citadel using + You'll also notice that you can specify two port numbers for SMTP: +one +for MTA (Mail Transport Agent) and one for MSA (Mail Submission Agent). +The +traditional ports to use for these purposes are 25 and 587. If you are +running an external MTA, such as Postfix (which submits mail to Citadel +using LMTP) or Sendmail (which submits mail to Citadel using the 'citmail' -delivery agent), that external MTA will be running on port 25, and you should -specify "-1" for the Citadel MTA port to disable it. The MSA port (again, -usually 587) would be the port used by end-user mail client programs such as -Aethera, Thunderbird, Eudora, or Outlook, to submit mail into the system. -All connections to the MSA port must use Authenticated SMTP. +delivery agent), that external MTA will be running on port 25, and you +should +specify "-1" for the Citadel MTA port to disable it. The MSA port +(again, +usually 587) would be the port used by end-user mail client programs +such as +Aethera, Thunderbird, Eudora, or Outlook, to submit mail into the +system. +All connections to the MSA port must use Authenticated SMTP. + +The protocols ending in "S" (POP3S, IMAPS, and SMTPS) are +SSL-encrypted. Although all of these protocols support the +STARTTLS command, older client software sometimes requires connecting +to "always encrypted" server ports. Usually when you are looking +at a client program that gives you a choice of "SSL or TLS," the SSL +option will connect to one of these dedicated ports, while the TLS +option will connect to the unencrypted port and then issue a STARTTLS +command to begin encryption. (It is worth noting that this is not the proper use of the acronyms +SSL and TLS, but that's how they're usually used in many client +programs.) + +All of the default port numbers, including the encrypted ones, are +the standard ones. + The question about correcting forged From: lines affects how Citadel behaves with authenticated SMTP clients. Citadel does not ever allow third-party SMTP relaying from unauthenticated clients -- any incoming @@ -1648,9 +1708,36 @@ suppress this behavior, answer 'No' at the prompt (the default is 'Yes') and the headers will never be altered. +"Instant expunge" affects what happens when IMAP users delete +messages. As you may already know, messages are not truly deleted +when an IMAP client sends a delete command; they are only marked for +deletion. The IMAP client must also send an "expunge" command +to actually delete the message. The Citadel server automatically expunges +messages when the client logs out or selects a different folder, but if you +select the Instant Expunge option, an expunge operation will automatically +follow any delete operation (and the client will be notified, preventing any +mailbox state problems). This is a good option to select, for example, if you +have users who leave their IMAP client software open all the time and are +wondering why their deleted messages show up again when they log in from a +different location (such as WebCit). +"Allow spoofing" refers to the security level applied to +non-authenticated SMTP clients. Normally, when another host connects to +Citadel via SMTP to deliver mail, Citadel will reject any attempt to send +mail whose sender (From) address matches one of your host's own domains. This +forces your legitimate users to authenticate properly, and prevents foreign +hosts (such as spammers) from forging mail from your domains. If, however, +this behavior is creating a problem for you, you can select this option to +bypass this particular security check. + +Connect this Citadel to an LDAP directory [No]: No + +The LDAP configuration options are discussed elsewhere in this +document. + The final set of options configures system-wide defaults for the auto-purger: -Default user purge time (days) [120]: Default room purge time (days) [30]: System default message expire policy (? for list) [2]: Keep how many messages online? [150]: Mailbox default message expire policy (? for list) [1]: +Default user purge time (days) [120]: Default room purge time (days) [30]: System default message expire policy (? for list) [0]: Keep how many messages online? [150]: Mailbox default message expire policy (? for list) [0]: How often to run network jobs (in seconds) [1800]: Enable full text search index (warning: resource intensive) [Yes]: Yes Hour to run purges (0-23) [4]: +Perform journaling of email messages [No]: Perform journaling of non-email messages [No]: Email destination of journalized messages [example@example.com]: Any user who does not log in for the period specified in 'Default user purge time' will be deleted the next time a purge is run. This setting may be modified on a per-user basis. @@ -1670,12 +1757,51 @@ want to. This can allow you, for example, to set a policy under which old messages scroll out of public rooms, but private mail stays online indefinitely -until deleted by the mailbox owners. -Save this configuration? No +until deleted by the mailbox owners. + +"How often to run network jobs" refers to the sharing of content on +a +Citadel network. If your system is on a Citadel network, this +configuration +item dictates how often the Citadel server will contact other Citadel +servers to send and receive messages. In reality, this will happen more +frequently than you specify, because other Citadel servers will be +contacting yours at regular intervals as well. + +"Hour to run purges" determines when expired and/or deleted objects +are purged from the database. These purge operations are +typically run overnight and automatically, sometime during whatever +hour you specify. If your site is much busier at night than +during the day, you may choose to have the auto-purger run during the +day. +"Enable full text search index," if enabled, instructs the server to +build and maintain a searchable index of all messages on the +system. This is a time and resource intensive process -- it could +take days to build the index if you enable it on a large +database. It is also fairly memory intensive; we do not recommend +that you enable the index unless your host system has at least 512 MB +of memory. Once enabled, however, it will be updated +incrementally +and will not have any noticeable impact on the interactive response +time of your system. The full text index is currently only +searchable when using IMAP clients; other search facilities will be +made available in the near future. +The "Perform journaling..." options allow you to configure +your Citadel server to send an extra copy of every message, along with +recipient information if applicable, to the email address of your choice. +The journaling destination address may be an account on the local Citadel +server, an account on another Citadel server on your network, or an Internet +email address. These options, used in conjunction with an archiving service, +allow you to build an archive of all messages which flow through your Citadel +system. This is typically used for regulatory compliance in industries which +require such things. Please refer to the journaling +guide for more details on this subject. +Save this configuration? No + When you're done, enter 'Yes' to confirm the changes, or 'No' to discard the changes.

@@ -1756,6 +1882,12 @@ whatever) that all outbound mail be sent to an SMTP relay or forwarder. To configure this functionality, simply enter the domain name or IP address of your relay as a 'smart-host' entry.

If your relay server is running on a port other +than the standard SMTP port 25, you can also specify the port number +using "host:port" syntax; i.e. relay99.myisp.com:2525

Furthermore, if your relay server requires authentication, you can +specify it using username:password@host or username:password@host:port +syntax; i.e. jsmith:pass123@relay99.myisp.com:25

directory: a domain for which you are participating in directory services across any number of Citadel nodes. For example, if users who have addresses in the domain citadel.org are spread @@ -1877,8 +2009,22 @@ server, while keeping the existing Unix mailboxes intact. However, it is beyond the scope of this document to detail the finer points of the configuration of Postfix or any other mailer, so refer to the documentation to those programs and keep in mind that Citadel has -LMTP support.
-

+LMTP support.

There are actually two LMTP sockets. One is called +lmtp.socket and the other is called lmtp-unfiltered.socket +(both are found in your Citadel directory). The difference should be +obvious: messages submitted via lmtp.socket are subject to +any +spam filtering you may have configured (such as SpamAssassin), while +messages +submitted via lmtp-unfiltered.socket will bypass the filters. +You +would use the filtered socket when receiving mail from an external MTA +such +as Postfix, but you might want to use the unfiltered socket with +utilities +such as fetchmail.

For outbound mail, you can either allow Citadel to perform deliveries directly @@ -1971,7 +2117,7 @@ you simply click on (or paste into your browser if you can't click on URL's in your e-mail software) and the confirmation is automatically completed.

Building or joining a Citadel network

@@ -2124,19 +2270,77 @@ record manager. It is robust, high-performance, and transactional.
A few small data files are kept in your main Citadel directory, but the databases are in the data/ subdirectory. The files with names that begin with "cdb" are the databases themselves; the files -with names that begin with "log" are the journals. Journal files -will come and go as you use your system; when the database engine has -determined that a particular log file is no longer needed, the file -will automatically be deleted. Nevertheless, you should always -ensure that there is ample disk space for the files to grow.
+with names that begin with "log" are the logs (sometimes referred to as +"journals"). Log files will continue to appear as you use your +system; each will grow to approximately 10 megabytes in size before a +new one is started. There is a system configuration setting +(found in .Aide System-configuration General in the text mode +client, or in Administration +--> Edit site-wide configuration --> Tuning in the WebCit +client) which specifies "Automatically delete committed database +logs." If you have this option enabled, Citadel will +automatically delete any log files whose contents have been fully +committed to the database files.
+
+For more insight into how the database and log files work, you may wish +to read the Berkeley +DB documentation on this subject.
+
+

Backing up your +Citadel database

+Please read this section carefully.
+
+There are two backup strategies you can use, depending on your site's +availability requirements and disk space availability.
+

Strategy #1: Standard backup

+The standard (or "offline") backup is used when your Citadel server is +configured to automatically delete committed database logs. The +backup procedure is as follows:
+

Shut down the Citadel server.
Back up all files (database files, log files, etc.) to tape or +some other backup media.
Start the Citadel server.

+Advantage: very little disk +space is consumed by the logs.
+Disadvantage: Citadel is not +available during backups.
+
+

Strategy #2: "Hot" backup

+The "hot backup" procedure is used when your Citadel server is +configured not to +automatically delete committed database logs. The backup +procedure is as follows:
+

Back up all files. Make sure the database files (cdb.*) are backed up before the log files (log.*). This will usually +be the case, because the database files tend to appear first in both +alphabetical and on-disk ordering of the data/ directory.
After verifying that your backup completed successfully, delete +the committed log files with a command like this:

+/usr/local/citadel/sendcommand +"CULL"

-There is no need to shut down Citadel during backups. The data -store may be backed up "hot." The makers of Berkeley DB suggest -that you should back up the data files first and the log files second. - This is the only method that will guarantee that a database which -is being changed while you back it up will still be usable when you -restore it -from the tape later.
+Advantage: Citadel continues +to run normally during backups.
+Disadvantage: Much disk space is consumed by the log files, +particularly if the full text indexer is turned on.
+
+
+It is up to you to decide which backup strategy to use. Warning: if you configure Citadel to +automatically delete committed database logs, and do not shut the +Citadel service down during backups, there is no guarantee that your +backups will be usable!

Database repair

Although Citadel's data store is quite reliable, database corruption @@ -2305,6 +2509,12 @@ Authority when you order the certificate.

When the certificate is received, simply save it as citadel.cer and restart the Citadel server.

If your certificate authority delivers a 'chained' certificate +(one +with intermediate certificate authorities), simply append the +intermediate +certificate after your server's own certificate in the citadel.cer +file.

@@ -2516,10 +2726,7 @@ require textual input will read it from stdin. Commands which generate textual output will be sent to stdout.

This utility is intended to be used to enable Citadel server commands to -be executed from shell scripts. Review the script called weekly -which ships with the Citadel distribution for an example of how this -can -be used.

+be executed from shell scripts.

NOTE: be sure that this utility is not world-executable. It connects to the server in privileged mode, and therefore could present a security hole

Clint Adams @@ -36,8 +35,7 @@ others
Art Cancro	overall system design and lead -developer +	overall system design and lead developer
Michael Hampton +	David Given	client software development +	IMAP and build patches
Urs Jannsen +	Wilfried Goesgens	text -search algorithm +	build system patches +
Michael Hampton	client software development +
Andru Luvisi @@ -80,8 +83,13 @@ search algorithm
Stu Mark	additional client features, IGnet protocol -design +	additional client features, IGnet protocol design +
Edward S. Marshall +	RBL checking function design
assistance with project management
Trey Van Riper +	QA and portability enhancements +
John Walker	author of public domain base64 -encoder/decoder +	author of public domain base64 encoder/decoder

C I T A D E L

a messaging and collaboration platform for BBS and groupware -applications

an open source messaging and collaboration platform

Table of Contents