Art Cancro [Mon, 22 Aug 2011 18:18:37 +0000 (14:18 -0400)]
Set all version numbers to 8.00 -- FEATURE FREEZE FOR 8.00
Wilfried Goesgens [Wed, 24 Aug 2011 13:15:43 +0000 (15:15 +0200)]
Cleanup Pop3Aggregator
- Fix varous Memleaks
- pop3client_scan_room() Combine old pop3collector syntax into a parseable URL.
- use just one URL pointer
- POP3C_GetOneMessageIDFromUseTable(): duplicate; obsolete; deleted.
- add username to pop3 usetable entries
- use our own Context
Wilfried Goesgens [Wed, 24 Aug 2011 13:10:34 +0000 (15:10 +0200)]
Fix Shutdown; work on IODebug
- StopClientWatchers(): stop watchers regardless whether the fd is 0 or not.
- streamline IODebugging
Wilfried Goesgens [Wed, 24 Aug 2011 13:09:45 +0000 (15:09 +0200)]
Add IODebugging to CtdlReadMessageBodyAsync
Wilfried Goesgens [Wed, 24 Aug 2011 09:43:56 +0000 (11:43 +0200)]
When creating system contexts, set the Context name.
Wilfried Goesgens [Wed, 24 Aug 2011 09:38:12 +0000 (11:38 +0200)]
rename our context so its more filename friendly.
Wilfried Goesgens [Sun, 21 Aug 2011 20:33:04 +0000 (22:33 +0200)]
add missing newline in logging.
Wilfried Goesgens [Sun, 21 Aug 2011 20:28:36 +0000 (22:28 +0200)]
in some rare conditions, we seem to need to check the pointer first before accessing it.
Wilfried Goesgens [Sat, 20 Aug 2011 17:38:17 +0000 (17:38 +0000)]
one time 'room' var is enough.
Wilfried Goesgens [Sat, 20 Aug 2011 17:37:45 +0000 (17:37 +0000)]
protect access to the counter by mutex, else we might get race conditions.
Wilfried Goesgens [Sat, 20 Aug 2011 11:58:14 +0000 (11:58 +0000)]
QP-Encode questionmarks.
Wilfried Goesgens [Sat, 20 Aug 2011 09:50:26 +0000 (09:50 +0000)]
When QP encoding, we need to convert blanks to underscores
- Spamassasin test 'BAD_ENC_HEADER' matches if a subject looks like that:
Subject: =?UTF-8?Q?=5Btestml=5D l=C3=B6sch misch ey.?=
it needs to look like that:
Subject: =?UTF-8?Q?=5Btestml=5D_l=C3=B6sch_misch_ey.?=
Wilfried Goesgens [Fri, 19 Aug 2011 17:17:31 +0000 (17:17 +0000)]
add missing var definition *ups*
Wilfried Goesgens [Fri, 19 Aug 2011 17:12:32 +0000 (17:12 +0000)]
Fix detecting of UTF8 Sequences
- we need to exactly check for both highest bits to be set, not one of them.
Wilfried Goesgens [Fri, 19 Aug 2011 16:43:30 +0000 (16:43 +0000)]
fix counting of UTF8-charwidth
- Ctdl_GetUtf8SequenceLength(): testbyte needs to be unsigned char, else >> will shift us new bits in from the left
- we need to shift 'test' to the right
- start counting at 0
Wilfried Goesgens [Thu, 18 Aug 2011 18:18:38 +0000 (18:18 +0000)]
fix StrHtmlEcmaEscAppend, StrECMAEscAppend
- StrHtmlEcmaEscAppend(): reimplement 'nbsp' - handling
- StrECMAEscAppend(): copy over required stuff from StrHtmlEcmaEscAppend()
Wilfried Goesgens [Thu, 18 Aug 2011 17:53:18 +0000 (17:53 +0000)]
Fix StrHtmlEcmaEscAppend() more edge cases in utf8 handling
- Ctdl_GetUtf8SequenceLength(): return 0 for broken, 1 for non-utf8 sequences
- StrHtmlEcmaEscAppend(): another attempt to fix json appending:
- use switch instead of Rumpelstilskin if sequence
- use HKEY for our static sequences
- don't check for buffer expansion again while utf8-handling; we have 11 bytes reserved, utf8 will at most become 6 bytes
- Simply handle ASCII chars in the utf8 loop
- since aptr is moved implicitely, we mustn't move it for the last byte
- add missing handling for some escapers
- let unicode escapers pass
Conflicts:
libcitadel/lib/stringbuf.c
Wilfried Goesgens [Thu, 18 Aug 2011 09:53:01 +0000 (09:53 +0000)]
fix QP encoding
- once we decided to QP encode we also need to encode = (forgotten in last commit)
Wilfried Goesgens [Thu, 18 Aug 2011 09:14:48 +0000 (09:14 +0000)]
fix QP encoding
- once we decided to QP encode we also need to encode =[]
Wilfried Goesgens [Wed, 17 Aug 2011 17:43:15 +0000 (17:43 +0000)]
Work on event'ing the pop3 aggregator
- POP3_C_ReAttachToFetchMessages(): we manage the transition DB-IO -> Network I/O
- POP3C_GetListCommandState(): after receiving the list-OK we have to continue reading the list items
- POP3C_GetListOneLine(): the server sends us the size of the message; save it for later use.
- POP3C_GetOneMessagID(): after reading all list-items, start DB lookup whether we knew them already.
- POP3C_GetOneMessageIDState(): if more items are in the list, call us again with the next message ID.
- POP3_FetchNetworkUsetableEntry(): fetch one UT-entry per call; forward to POP3_C_ReAttachToFetchMessages() once we saw all items in the list.
- POP3C_SendGetOneMsg(): cycle over the list till we find the next Mail we didn't know already & fetch it; if no more items in list -> goodbuye.
- POP3C_ReadMessageBody() after the I/O layer read the message from the socket, parse it; move our context over to the DB-Queue.
- POP3C_ReadMessageBodyFollowing(): On success initialize the async message reader; let the I/O layer read the message, then continue parsing & saving it.
- POP3C_StoreMsgRead(): parse & save the message to the DB. Continue with remembering it in the UT table.
- POP3C_SendDelete(): check whether we want to delete the mails on the server, if do; else continue with POP3C_SendGetOneMsg() to fetch next in list.
- POP3_C_ReadServerStatus(): switch between line based read operation and reading whole messages.
we now can successfully aggregate one pop3 account.
Wilfried Goesgens [Wed, 17 Aug 2011 17:18:07 +0000 (17:18 +0000)]
add infrastructure to cleanly swap inbetween the DB and the IO queue; add infrastructure needed for async message reading.
- QueueDBOperation(): use our own vars on the AsyncIO struct, so we don't overwrite handlers & events from the file I/O stuff.
- DB_PerformNext(): don't use default: so we get warnings if new states are added and not handled here.
- DB_PerformNext(): if we don't get a DB query or Terminate, the db operations are done, remove our handlers, this AsyncIO context is handed over into another Queue by the application logic.
- NextDBOperation(): use our own vars on the AsyncIO struct, so we don't overwrite handlers & events from the file I/O stuff.
- HandleInbound(): handle more read-states; needed for message reading.
- HandleInbound(): move Rumpelstilskin lookup if to switch()
- IO_send_callback(): fix/add IO debugging to /tmp/foo
- IO_send_callback(): Adjust switch to some edge cases:
- eSendReply: check whether we're done with sending, if, fall forward to writing.
- eReadMore: was missing, continue reading (if)
- eReadPayload: new, read smtp messages for example.
- eDBQuery: the application logic want to perform database I/O with this context, clean up hooks and remove it from this queue.
- set_start_callback(): properly hanle the new cases
- IO_recv_callback(): fix/add IO debugging to /tmp/foo
- ReAttachIO(): use me if you want to continue with file I/O after i.e. DB-I/O
Wilfried Goesgens [Wed, 17 Aug 2011 17:13:59 +0000 (17:13 +0000)]
Add async message reading
- CtdlSubmitMsg(): make param const, since its just read
- add NewAsyncMsg(): preprocess the parameters provided by application logic, return the worker struct for message reading
- add CtdlReadMessageBodyAsync(): call me from your IO handler to read a message from the socket asynchroneous.
- add DeleteAsyncMsg(): call me to delete the worker vars.
Wilfried Goesgens [Wed, 17 Aug 2011 17:09:57 +0000 (17:09 +0000)]
Rename function to cary the module prefix.
Wilfried Goesgens [Wed, 10 Aug 2011 14:02:00 +0000 (14:02 +0000)]
work on the pop3 aggregator
- rewrite the config parser similar to the rss-configparser
- copy over handler logic from async smtp connector
- chop worker into handy bits so the handler logic can control it.
Wilfried Goesgens [Wed, 10 Aug 2011 14:00:34 +0000 (14:00 +0000)]
fix naming of functions, so they live in the RSS-Namespace
Wilfried Goesgens [Wed, 10 Aug 2011 13:59:20 +0000 (13:59 +0000)]
tiny fixes
- log failed connection attempts
- add sample to set 127.0.0.1
Wilfried Goesgens [Wed, 10 Aug 2011 13:55:42 +0000 (13:55 +0000)]
fix parsing of IP-Urls
- need to point inet_pton() to the rightly casted pointer in case of non-IPv6
- set the port using hton()
- set the protocol depending on our findings
Wilfried Goesgens [Sun, 24 Jul 2011 13:35:59 +0000 (13:35 +0000)]
StrBufSipLine(): make input params const; fix several warnings about unused code.
Conflicts:
libcitadel/lib/libcitadel.h
Art Cancro [Mon, 18 Jul 2011 20:45:06 +0000 (16:45 -0400)]
DBG_QR[2] now a configurable loggable
Art Cancro [Mon, 18 Jul 2011 20:21:29 +0000 (16:21 -0400)]
Wrap all calls to pthread_mutex_lock() and pthread_mutex_unlock() within CtdlLogResult()
Art Cancro [Mon, 18 Jul 2011 20:19:45 +0000 (16:19 -0400)]
Convenience function CtdlLogResult()
Wilfried Goesgens [Sun, 10 Jul 2011 13:40:38 +0000 (13:40 +0000)]
finalize allocation & freeing of rss stuff
Wilfried Goesgens [Sat, 9 Jul 2011 20:50:48 +0000 (20:50 +0000)]
make shure invalid session fd's don't get set into the central select.
Wilfried Goesgens [Sat, 9 Jul 2011 20:50:33 +0000 (20:50 +0000)]
use right struct for memset.
Wilfried Goesgens [Thu, 7 Jul 2011 22:37:22 +0000 (22:37 +0000)]
work on making RSS aggregator instances and roomlists consistant.
Wilfried Goesgens [Thu, 7 Jul 2011 19:50:26 +0000 (19:50 +0000)]
Fix documentation
- fix bug about escaping in strbuf
- link Hashlist Componets to groups so they can be easily accessed.
Wilfried Goesgens [Sun, 3 Jul 2011 22:35:27 +0000 (22:35 +0000)]
fix conditions where several rooms read the same RSS feed
Wilfried Goesgens [Sun, 3 Jul 2011 22:34:03 +0000 (22:34 +0000)]
merge in euid generation from master
Wilfried Goesgens [Sun, 3 Jul 2011 13:49:59 +0000 (13:49 +0000)]
first working RSS collection with async DB-Saves
- for now, no shorter URL expansion
- replace handling of configs, old algorythm wasn't thread-safe
- DB i/o is done in its own libev Queue in its own thread now; so we don't block network IO with database IO.
Wilfried Goesgens [Sun, 3 Jul 2011 13:47:55 +0000 (13:47 +0000)]
Add ability to hex encode binary strings
- when encoding plain md5 binary buffers we have to pass the length into the hex encoder, add parameter
- add wrapper providing old schematic
- fix documentation of StrBufSipLine()
Wilfried Goesgens [Sat, 25 Jun 2011 07:12:46 +0000 (07:12 +0000)]
more work on libevĀ“ing the rss-reader.
Wilfried Goesgens [Mon, 6 Jun 2011 19:33:58 +0000 (19:33 +0000)]
fix loop / handling error when connecting fails immediately
- add new ev_idle to unwind the stack on connecting
- add a new handler that processes the disconnect
Wilfried Goesgens [Sun, 5 Jun 2011 21:22:37 +0000 (21:22 +0000)]
fix pointer indirections so we now can successfully fetch RSS feeds again.
Wilfried Goesgens [Sun, 5 Jun 2011 21:20:17 +0000 (21:20 +0000)]
when not forking, we fail to drop rood privileges in advance before writing netconfig files, so we loose later access to them. Sadly we don't know our GID...
Wilfried Goesgens [Sun, 5 Jun 2011 16:48:06 +0000 (16:48 +0000)]
move stuff out into own files from here...
Wilfried Goesgens [Sun, 5 Jun 2011 16:46:41 +0000 (16:46 +0000)]
Split RSS Parser...
- URL-De-Shortener module
- RSS/ATOM Handlers
- start migrating to event/curl requests
Wilfried Goesgens [Sun, 5 Jun 2011 16:44:59 +0000 (16:44 +0000)]
if specified, use our bind-ip for the http clients.
Wilfried Goesgens [Sun, 5 Jun 2011 16:42:29 +0000 (16:42 +0000)]
Give us an own Context per notification client.
Wilfried Goesgens [Sun, 5 Jun 2011 16:41:39 +0000 (16:41 +0000)]
rebuilt from .y
Wilfried Goesgens [Sun, 5 Jun 2011 15:28:24 +0000 (15:28 +0000)]
* make it const baby.
Wilfried Goesgens [Sun, 5 Jun 2011 13:49:09 +0000 (13:49 +0000)]
Free all our allocated resources after notifying / sending a http request.
Wilfried Goesgens [Mon, 30 May 2011 21:52:00 +0000 (21:52 +0000)]
finish rewriting of http client code
- we now have a context
- we now handle replies
- still some uncertain cleanup condition.
Wilfried Goesgens [Sat, 21 May 2011 20:09:39 +0000 (20:09 +0000)]
Migrate http context struct into the AsyncIO struct; streamline other structs.
Wilfried Goesgens [Sat, 21 May 2011 13:08:23 +0000 (13:08 +0000)]
unlink global struct and request structs
Wilfried Goesgens [Thu, 19 May 2011 21:19:25 +0000 (21:19 +0000)]
Fix unfolding of RFC822 lines so we don't loose blanks.
Wilfried Goesgens [Wed, 18 May 2011 21:35:30 +0000 (21:35 +0000)]
Don't overwrite the 'TO' when sending mailinglist mails; we put ourselves into 'reply to:'
Wilfried Goesgens [Sun, 15 May 2011 21:11:03 +0000 (21:11 +0000)]
Rewrite loading of notify configurations
Wilfried Goesgens [Sun, 15 May 2011 14:12:15 +0000 (14:12 +0000)]
sending http requests instant now.
Wilfried Goesgens [Sun, 15 May 2011 10:43:11 +0000 (10:43 +0000)]
work on libev <-> libcurl integration
- move libcurl handlers into the libev file; they need to be initialized to close to separate
- add some logging;
First successfull CURL notification via libev!
Wilfried Goesgens [Sun, 15 May 2011 10:42:26 +0000 (10:42 +0000)]
Fix parsing and rearaning of URLs
Wilfried Goesgens [Sat, 14 May 2011 14:28:50 +0000 (14:28 +0000)]
don't copy filedescriptors into the clone.
Wilfried Goesgens [Sat, 14 May 2011 10:23:48 +0000 (10:23 +0000)]
more work on async http
Wilfried Goesgens [Sat, 14 May 2011 09:49:43 +0000 (09:49 +0000)]
Memleak: free the vector of URLs to try to connect to.
Wilfried Goesgens [Tue, 10 May 2011 22:05:46 +0000 (22:05 +0000)]
just log the close if we're realy going to close.
Wilfried Goesgens [Tue, 10 May 2011 22:03:20 +0000 (22:03 +0000)]
Don't close 0-filedescriptors on session killing, its our indicator for not here, since 0 equals STDIN which at best is redirected to /dev/null
Wilfried Goesgens [Mon, 9 May 2011 21:44:48 +0000 (21:44 +0000)]
use long hash function, if we intend to use long values as keys, else we won't find anything
Wilfried Goesgens [Sun, 8 May 2011 17:43:46 +0000 (17:43 +0000)]
Stop watchers / don't close 0
Wilfried Goesgens [Sun, 8 May 2011 12:00:04 +0000 (12:00 +0000)]
work on errorhandling for the smtp event client
- add states for the mx lookup phases
- return the propper states
- close filedescriptors if we fail one attempt
- say which connection type we have: immediate, e_again, fail
- say which NS-lookup whe do (a/aaaa)
Wilfried Goesgens [Sun, 8 May 2011 11:58:27 +0000 (11:58 +0000)]
Add backtrace debugging facility that doesn't add linebreaks
Wilfried Goesgens [Sun, 8 May 2011 11:53:34 +0000 (11:53 +0000)]
libevent needs to link the mathlib, so we add -lm
Wilfried Goesgens [Tue, 3 May 2011 20:11:46 +0000 (20:11 +0000)]
ups, .lo not .o
Wilfried Goesgens [Tue, 3 May 2011 19:55:11 +0000 (19:55 +0000)]
add prerelease so we don't get deinstalled all the time..
Wilfried Goesgens [Tue, 3 May 2011 19:53:26 +0000 (19:53 +0000)]
fix problem with po missing \n
Wilfried Goesgens [Tue, 3 May 2011 19:52:05 +0000 (19:52 +0000)]
Work on integrating libcurl <-> libev
Wilfried Goesgens [Tue, 3 May 2011 19:49:23 +0000 (19:49 +0000)]
Fix possible endless loop conditions
Wilfried Goesgens [Thu, 28 Apr 2011 19:20:52 +0000 (19:20 +0000)]
Various cleanups of abort conditions
- if the timeout/connectionfail/... calls tell us to abort, shutdown the client
- unregister all event structs
Wilfried Goesgens [Thu, 28 Apr 2011 19:10:56 +0000 (19:10 +0000)]
Name of MX in error messages
- set it after successfull MX lookup
- put in that we don't have an mx record if its NULL instead of 'no name'
Wilfried Goesgens [Thu, 28 Apr 2011 19:09:53 +0000 (19:09 +0000)]
fix gcc warning about it not knowing which pointer indirection is const...
Wilfried Goesgens [Tue, 19 Apr 2011 19:55:16 +0000 (19:55 +0000)]
Include ourselves??
Wilfried Goesgens [Tue, 19 Apr 2011 19:50:55 +0000 (19:50 +0000)]
Fix Crash
* check pointer before accessing it
* fix probably uninitialized case
Wilfried Goesgens [Sat, 2 Apr 2011 15:56:18 +0000 (17:56 +0200)]
Give event clients a citadel session context.
Wilfried Goesgens [Sat, 2 Apr 2011 15:24:17 +0000 (17:24 +0200)]
Cleanup the event client struct, remove unused vars
Wilfried Goesgens [Sat, 2 Apr 2011 10:18:49 +0000 (12:18 +0200)]
just use the ParsedURL to bear our destination address & port
Wilfried Goesgens [Sat, 2 Apr 2011 09:26:07 +0000 (11:26 +0200)]
Use IOBuffer with its StrBuf + const char* inside instead of having two of them
Wilfried Goesgens [Fri, 1 Apr 2011 21:48:57 +0000 (23:48 +0200)]
implement abort handling
Wilfried Goesgens [Fri, 1 Apr 2011 19:56:20 +0000 (21:56 +0200)]
Switch handling to have a pointer to the target address
- change URL struct to contain sockaddr_in6
- change URL-Parsing to add that indirection
- change all pointer indirections so the new way fits
Wilfried Goesgens [Fri, 1 Apr 2011 14:51:48 +0000 (16:51 +0200)]
move URL parsing / struct over into libcitadel
Wilfried Goesgens [Thu, 31 Mar 2011 22:50:43 +0000 (00:50 +0200)]
Fix searching of ']'
Wilfried Goesgens [Thu, 31 Mar 2011 21:21:10 +0000 (23:21 +0200)]
Add busines logic for toggling multiple mx records etc.
- move handlers into their own file
- chain up stuff via URL-structs
(not yet all done)
Wilfried Goesgens [Thu, 31 Mar 2011 21:11:14 +0000 (23:11 +0200)]
Mailinglist client: don't add roomname to subject if its already there
Wilfried Goesgens [Mon, 21 Feb 2011 22:13:33 +0000 (23:13 +0100)]
fixing html2ascii
- add possibility to scan 4 digit entities
- don't scan over the end
- add tests
Wilfried Goesgens [Thu, 3 Feb 2011 21:34:38 +0000 (22:34 +0100)]
libev migration: start out with creating support for 'fallbackhost'
Wilfried Goesgens [Wed, 2 Feb 2011 20:34:49 +0000 (21:34 +0100)]
libev/c-ares migration; fix lookup chains
- use idle-watchers for unwinding the c-ares stack (thanks to Marc Lehmann for the hint)
- use the generic c-ares query & parse logic instead of the integrated call; we need to first do the a then fall back to the aaaa request which the c-ares default one doesn't implement
- stop the shutdown watchers when finalizing one client
Wilfried Goesgens [Mon, 31 Jan 2011 00:35:40 +0000 (01:35 +0100)]
libev migration - add shutdown handlers
- this ties us to libev > 4.0
- clean shutdown and storing of the new state in our quue
Wilfried Goesgens [Sun, 30 Jan 2011 23:10:11 +0000 (00:10 +0100)]
libev migration: use async methods to schedule jobs. thanks to Marc Lehmann for the friendly help.
Wilfried Goesgens [Sun, 30 Jan 2011 20:28:35 +0000 (21:28 +0100)]
libev migration - copy valus we use for loop control before we hand it off into the libevent queue
- this fixes a race condition causing a double free with the to be copied mailpayload
Wilfried Goesgens [Sun, 30 Jan 2011 14:29:59 +0000 (15:29 +0100)]
Merge branch 'libevent' of ssh://git.citadel.org/appl/gitroot/citadel into libevent
Wilfried Goesgens [Sun, 30 Jan 2011 14:27:29 +0000 (15:27 +0100)]
libev/c-ares migration: unstack out ouf c-ares before querying new requests
- we use the c-ares callback to parse the DNS reply and put the reply to our callback
- we then set a zero-timer with the call to continue our flow
- we return / unstack c-ares
- we then do the a/aaaa record lookup without having another ares lookup in our stack
this circumvents double dns lookups.
Wilfried Goesgens [Sun, 30 Jan 2011 12:14:05 +0000 (13:14 +0100)]
QP-Decoding: Ignore case of letter indicating _B_inary or _Q_uoted printeable, there seem to be solutions around sending q,b instead of Q,B
Wilfried Goesgens [Sun, 30 Jan 2011 11:12:44 +0000 (12:12 +0100)]
by jimcuclamath: use defines instead of numbers for logging; Change some loglevels to DEBUG