- hack.doc for Citadel/UX
- written by Art Cancro (ajc@uncnsrd.mt-kisco.ny.us)
+ hack.txt for Citadel/UX
+ (possibly a little out of date)
Much of this document is borrowed from the original hack.doc from
Citadel-CP/M and Citadel-86, because many of the concepts are the same. Hats
storage and retrieval of large binary objects (blobs) indexed by unique keys.
Please see database.c for more information on data store primitives.
- The message base (msgmain) is a big file of messages indexed by the message
+ The message base (MSGMAIN) is a big file of messages indexed by the message
number. Messages are numbered consecutively and start with an FF (hex)
byte. Except for this FF start-of-message byte, all bytes in the message
file have the high bit set to 0. This means that in principle it is
try for efficiency...)
Each room is basically just a list of message numbers. Each time
-we enter a new message in a room, we slide all the old message-numbers
-down a slot, and probably the oldest one falls off the bottom (in which case
-we must delete it from the message base). Reading a rooms is just a matter
-of looking up the messages one by one and sending them to the client for
-display, printing, or whatever.
+we enter a new message in a room, its message number is appended to the end
+of the list. If an old message is to be expired, we must delete it from the
+message base. Reading a room is just a matter of looking up the messages
+one by one and sending them to the client for display, printing, or whatever.
Implementing the "new message" function is also trivial in principle:
we just keep track, for each caller in the userlog, of the highest-numbered
-message which existed on the >last< call. (Remember, message numbers are
+message which existed on the *last* call. (Remember, message numbers are
simply assigned sequentially each time a message is created. This
sequence is global to the entire system, not local within a room.) If
we ignore all message-numbers in the room less than this, only new messages
will be printed. Voila!
+
+
+ Message format on disk (MSGMAIN)
- message format on disk (msgmain)
-
- Each message begins with an FF byte. The next byte will then be MES_NORMAL,
-MES_ANON, or MES_ANON2, depending on whether the message in anonymous or not.
-The next byte is either a 0 or 1. If it is 0, the message will be printed
-with the Citadel formatter. If it is a 1, the
-message is printed directly to the screen, as is. External editors generate
-this type of message. After these three opening bytes, the remainder of
+ As discussed above, each message begins with an FF byte.
+
+ The next byte denotes whether this is an anonymous message. The codes
+available are MES_NORMAL, MES_ANON, or MES_AN2 (defined in citadel.h).
+
+ The third byte is a "message type" code. The following codes are defined:
+ 0 - "Traditional" Citadel format. Message is to be displayed "formatted."
+ 1 - Plain pre-formatted ASCII text (otherwise known as text/plain)
+ 4 - MIME formatted message. The text of the message which follows is
+ expected to begin with a "Content-type:" header.
+
+ After these three opening bytes, the remainder of
the message consists of a sequence of character strings. Each string
begins with a type byte indicating the meaning of the string and is
ended with a null. All strings are printable ASCII: in particular,
BYTE Mnemonic Comments
-T Date/Time A 32-bit integer containing the date and time of
- the message in standard UNIX format (the number
- of seconds since January 1, 1970 GMT).
-P Path Complete path of message, as in the UseNet news
- standard. A user should be able to send UUCP mail to
- this path. (Note that your system name will not be
- tacked onto this until you're sending the message to
- someone else)
-I ID on orig A 32-bit integer containing the message ID on the
- system the message *originated* on.
-# ID on local A 32-bit integer containing the message ID on the
- system the message is *currently* on (obviously this
- is meaningless for a message being transmitted over
- a network).
A Author Name of originator of message.
-E EmailUserID Only present in messages originating from the Internet.
- It contains the username portion of the author's
- e-mail address, thereby allowing replies and bounces
- to be properly formulated.
-R Recipient Only present in Mail messages.
-O Room Room of origin.
-N Nodename Contains node name of system message originated on.
-H HumanNodeName Contains human name of system message originated on.
-D Destination Contains name of the system this message should
- be sent to, for mail routing (private mail only).
-U Subject Optional. Developers may choose whether they wish to
- generate or display subject fields. Citadel/UX does
- not generate them, but it does print them when found.
B Phone number The dialup number of the system this message
originated on. This is optional, and is only
defined for helping implement C86Net gateways.
+D Destination Contains name of the system this message should
+ be sent to, for mail routing (private mail only).
+E Extended ID A persistent alphanumeric Message ID used for
+ network replication. When a message arrives that
+ contains an Extended ID, any existing messages which
+ contain the same Extended ID and are *older* than this
+ message should be deleted. If there exist any messages
+ with the same Extended ID that are *newer*, then this
+ message should be dropped.
G Gateway domain This field is provided solely for the implementation
of C86Net gateways, and holds the C86Net domain of
the system this message originated on. Unless you're
implementing such a gateway, there's no need to even
bother with this field.
+H HumanNodeName Human-readable name of system message originated on.
+I Original ID A 32-bit integer containing the message ID on the
+ system the message *originated* on.
+M Message Text Normal ASCII, newlines seperated by CR's or LF's,
+ null terminated as always.
+N Nodename Contains node name of system message originated on.
+O Room Room of origin.
+P Path Complete path of message, as in the UseNet news
+ standard. A user should be able to send Internet mail
+ to this path. (Note that your system name will not be
+ tacked onto this until you're sending the message to
+ someone else)
+R Recipient Only present in Mail messages.
S Special field Only meaningful for messages being spooled over a
network. Usually means that the message isn't really
a message, but rather some other network function:
-> "S" followed by "FILE" (followed by a null, of
course) means that the message text is actually an
IGnet/Open file transfer.
-M Message Text Normal ASCII, newlines seperated by CR's or LF's,
- null terminated as always.
+T Date/Time A 32-bit integer containing the date and time of
+ the message in standard UNIX format (the number
+ of seconds since January 1, 1970 GMT).
+U Subject Optional. Developers may choose whether they wish to
+ generate or display subject fields. Citadel/UX does
+ not generate them, but it does print them when found.
EXAMPLE
distribution includes netproc.c, which is basically a database replicator;
please see network.txt on its operation and functionality (if any).
- portability problems
+ Portability issues
- At this point, most hardware-dependent stuff has been removed from the
+ At this point, all hardware-dependent stuff has been removed from the
system. On the server side, most of the OS-dependent stuff has been isolated
into the sysdep.c source module. The server should compile on any POSIX
compliant system with a full pthreads implementation and TCP/IP support. In
Win32 stuff).
- "Room" records (quickroom/fullroom)
+ "Room" records (quickroom)
The rooms are basically indices into msgmain, the message database.
As noted in the overview, each is essentially an array of pointers into
That should be enough background to tackle a full-scale room. From citadel.h:
-STRUCT QUickroom {
+struct quickroom {
char QRname[20]; /* Max. len is 19, plus null term */
char QRpasswd[10]; /* Only valid if it's a private rm */
long QRroomaide; /* User number of room aide */
long QRhighest; /* Highest message NUMBER in room */
- char QRgen; /* Generation number of room */
+ long QRgen; /* Generation number of room */
unsigned QRflags; /* See flag values below */
char QRdirname[15]; /* Directory name, if applicable */
char QRfloor; /* (not yet implemented) */
#define QR_NETWORK 2048 /* Shared network room */
#define QR_PREFONLY 4096 /* Preferred users only */
-struct fullroom {
- long FRnum[MSGSPERRM]; /* Message NUMBERS */
- };
-
[Note that all components start with "QR" for quickroom, to make sure we
don't accidentally use an offset in the wrong structure. Be very careful
also to get a meaningful sequence of components --
most recent message in the room. When we are searching for rooms with
messages a given caller hasn't seen, we can check this number
and avoid a whole lot of extra disk accesses.
+
+ There used to also be a structure called "fullroom" which resided in one
+file for each room on the system. This has been abandoned in favour of
+"message lists" which are variable sized and simply contain zero or more
+message numbers. The message numbers, in turn, point to messages on disk.
- The fullroom is the array of pointers into the message file. We keep one
-file for each fullroom array to keep the quickroom file small (and access time
-efficient). FRnum are the message numbers on disk of
-each message in the room. (For NIL, we stick zeroes in both fields.)
-
- user records (usersupp)
+ User records (usersupp)
This is the fun one. Get some fresh air and plug in your thinking cap
-first. (Time, space and complexity are the usernum software rivals.
+first. (Time, space and complexity are the eternal software rivals.
We've got lots of log entries times lots of messages spread over up to nnn
rooms to worry about, and with multitasking, disk access time is important...
so perforce, we opt for complexity to keep time and space in bounds.)
struct usersupp { /* User record */
int USuid; /* uid account is logged in under */
- char password[20]; /* password (for BBS-only users) */
+ char password[20]; /* password */
long lastseen[MAXROOMS]; /* Last message seen in each room */
char generation[MAXROOMS]; /* Generation # (for private rooms) */
char forget[MAXROOMS]; /* Forgotten generation number */
- long mailnum[MAILSLOTS]; /* Message #'s of each mail message */
- long mailpos[MAILSLOTS]; /* Disk positions of each mail */
unsigned flags; /* See US_ flags below */
int screenwidth; /* For formatting messages */
int timescalled; /* Total number of logins */
int posted; /* Number of messages posted (ever) */
char fullname[26]; /* Bulletin Board name for messages */
char axlevel; /* Access level */
- char spare[3]; /* spare bytes for future use */
long usernum; /* Eternal user number */
long lastcall; /* Last time the user called */
};
SUPPORTING PRIVATE MAIL
-Can one have an elegant kludge? This must come pretty close.
-
-Private mail is sent and recieved in the Mail> room, which otherwise
-behaves pretty much as any other room. To make this work, we store
-the actual message pointers in mailnum[] and mailpos[] in the caller's
-log record, and then copy them into the Mail> room array whenever we
-enter the room. This requires a little fiddling to get things just
-right. We have to update quickroom[1].QRhighest at login
-to reflect the presence or absence of new messages, for example. And
+ Can one have an elegant kludge? This must come pretty close.
+
+ Private mail is sent and recieved in the Mail> room, which otherwise
+behaves pretty much as any other room. To make this work, we have a
+separate Mail> room for each user behind the scenes. The actual room name
+in the database looks like "0000001234.Mail" (where '1234' is the user
+number) and it's flagged with the QR_MAILBOX flag. The user number is
+stripped off by the server before the name is presented to the client.
+
+ This requires a little fiddling to get things just right. For example,
make_message() has to be kludged to ask for the name of the recipient
of the message whenever a message is entered in Mail>. But basically
it works pretty well, keeping the code and user interface simple and
projects / citadel.git / blobdiff