Wilfried Goesgens [Thu, 1 Sep 2011 16:42:07 +0000 (16:42 +0000)]
Multiple cleanup
- DeletePOP3Aggregator() free the read message struct
- comment out table revalidation
- remove duplicate return statement
- correctly handle queueing of dns query. We don't care about the result.
Wilfried Goesgens [Thu, 1 Sep 2011 16:40:27 +0000 (16:40 +0000)]
Queue functions have to return eNextState in order for the caller to i.e. swap successfully between queus.
Wilfried Goesgens [Thu, 1 Sep 2011 16:37:33 +0000 (16:37 +0000)]
Silence the very noisy 'Interrupted CtdlThreadSelect'
Wilfried Goesgens [Thu, 1 Sep 2011 16:34:44 +0000 (16:34 +0000)]
FB->ReadWritePointer has to point to FB->Buf->Buf if the buffer is nonempty; logic in other functions depends on this.
Wilfried Goesgens [Wed, 31 Aug 2011 09:45:58 +0000 (09:45 +0000)]
When sending chunks in a loop, terminate imemdiately if the remote party goes away.
Wilfried Goesgens [Thu, 4 Aug 2011 20:08:30 +0000 (20:08 +0000)]
add .silent and some echo logic to our makefiles; thanks to sECuRE for the hints.
Wilfried Goesgens [Wed, 31 Aug 2011 08:03:52 +0000 (08:03 +0000)]
Add testhash function, since we experienced troubles here.
Wilfried Goesgens [Wed, 31 Aug 2011 08:02:42 +0000 (08:02 +0000)]
Add function to do some revalidations on hashlists; Attention; this can be cpu intense; don't use for production code.
Wilfried Goesgens [Wed, 31 Aug 2011 08:01:45 +0000 (08:01 +0000)]
StrBufChunkSipLine(): compact buffer if we have a remaining snippet at the end of the buffer
Wilfried Goesgens [Wed, 31 Aug 2011 08:00:52 +0000 (08:00 +0000)]
Continue reading if the IO Grammer layer tells us so.
Wilfried Goesgens [Wed, 31 Aug 2011 08:00:23 +0000 (08:00 +0000)]
Correct logging expressions
Art Cancro [Mon, 22 Aug 2011 18:18:37 +0000 (14:18 -0400)]
Set all version numbers to 8.00 -- FEATURE FREEZE FOR 8.00
Wilfried Goesgens [Wed, 24 Aug 2011 13:15:43 +0000 (15:15 +0200)]
Cleanup Pop3Aggregator
- Fix varous Memleaks
- pop3client_scan_room() Combine old pop3collector syntax into a parseable URL.
- use just one URL pointer
- POP3C_GetOneMessageIDFromUseTable(): duplicate; obsolete; deleted.
- add username to pop3 usetable entries
- use our own Context
Wilfried Goesgens [Wed, 24 Aug 2011 13:10:34 +0000 (15:10 +0200)]
Fix Shutdown; work on IODebug
- StopClientWatchers(): stop watchers regardless whether the fd is 0 or not.
- streamline IODebugging
Wilfried Goesgens [Wed, 24 Aug 2011 13:09:45 +0000 (15:09 +0200)]
Add IODebugging to CtdlReadMessageBodyAsync
Wilfried Goesgens [Wed, 24 Aug 2011 09:43:56 +0000 (11:43 +0200)]
When creating system contexts, set the Context name.
Wilfried Goesgens [Wed, 24 Aug 2011 09:38:12 +0000 (11:38 +0200)]
rename our context so its more filename friendly.
Wilfried Goesgens [Sun, 21 Aug 2011 20:33:04 +0000 (22:33 +0200)]
add missing newline in logging.
Wilfried Goesgens [Sun, 21 Aug 2011 20:28:36 +0000 (22:28 +0200)]
in some rare conditions, we seem to need to check the pointer first before accessing it.
Wilfried Goesgens [Sat, 20 Aug 2011 17:38:17 +0000 (17:38 +0000)]
one time 'room' var is enough.
Wilfried Goesgens [Sat, 20 Aug 2011 17:37:45 +0000 (17:37 +0000)]
protect access to the counter by mutex, else we might get race conditions.
Wilfried Goesgens [Sat, 20 Aug 2011 11:58:14 +0000 (11:58 +0000)]
QP-Encode questionmarks.
Wilfried Goesgens [Sat, 20 Aug 2011 09:50:26 +0000 (09:50 +0000)]
When QP encoding, we need to convert blanks to underscores
- Spamassasin test 'BAD_ENC_HEADER' matches if a subject looks like that:
Subject: =?UTF-8?Q?=5Btestml=5D l=C3=B6sch misch ey.?=
it needs to look like that:
Subject: =?UTF-8?Q?=5Btestml=5D_l=C3=B6sch_misch_ey.?=
Wilfried Goesgens [Fri, 19 Aug 2011 17:17:31 +0000 (17:17 +0000)]
add missing var definition *ups*
Wilfried Goesgens [Fri, 19 Aug 2011 17:12:32 +0000 (17:12 +0000)]
Fix detecting of UTF8 Sequences
- we need to exactly check for both highest bits to be set, not one of them.
Wilfried Goesgens [Fri, 19 Aug 2011 16:43:30 +0000 (16:43 +0000)]
fix counting of UTF8-charwidth
- Ctdl_GetUtf8SequenceLength(): testbyte needs to be unsigned char, else >> will shift us new bits in from the left
- we need to shift 'test' to the right
- start counting at 0
Wilfried Goesgens [Thu, 18 Aug 2011 18:18:38 +0000 (18:18 +0000)]
fix StrHtmlEcmaEscAppend, StrECMAEscAppend
- StrHtmlEcmaEscAppend(): reimplement 'nbsp' - handling
- StrECMAEscAppend(): copy over required stuff from StrHtmlEcmaEscAppend()
Wilfried Goesgens [Thu, 18 Aug 2011 17:53:18 +0000 (17:53 +0000)]
Fix StrHtmlEcmaEscAppend() more edge cases in utf8 handling
- Ctdl_GetUtf8SequenceLength(): return 0 for broken, 1 for non-utf8 sequences
- StrHtmlEcmaEscAppend(): another attempt to fix json appending:
- use switch instead of Rumpelstilskin if sequence
- use HKEY for our static sequences
- don't check for buffer expansion again while utf8-handling; we have 11 bytes reserved, utf8 will at most become 6 bytes
- Simply handle ASCII chars in the utf8 loop
- since aptr is moved implicitely, we mustn't move it for the last byte
- add missing handling for some escapers
- let unicode escapers pass
Conflicts:
libcitadel/lib/stringbuf.c
Wilfried Goesgens [Thu, 18 Aug 2011 09:53:01 +0000 (09:53 +0000)]
fix QP encoding
- once we decided to QP encode we also need to encode = (forgotten in last commit)
Wilfried Goesgens [Thu, 18 Aug 2011 09:14:48 +0000 (09:14 +0000)]
fix QP encoding
- once we decided to QP encode we also need to encode =[]
Wilfried Goesgens [Wed, 17 Aug 2011 17:43:15 +0000 (17:43 +0000)]
Work on event'ing the pop3 aggregator
- POP3_C_ReAttachToFetchMessages(): we manage the transition DB-IO -> Network I/O
- POP3C_GetListCommandState(): after receiving the list-OK we have to continue reading the list items
- POP3C_GetListOneLine(): the server sends us the size of the message; save it for later use.
- POP3C_GetOneMessagID(): after reading all list-items, start DB lookup whether we knew them already.
- POP3C_GetOneMessageIDState(): if more items are in the list, call us again with the next message ID.
- POP3_FetchNetworkUsetableEntry(): fetch one UT-entry per call; forward to POP3_C_ReAttachToFetchMessages() once we saw all items in the list.
- POP3C_SendGetOneMsg(): cycle over the list till we find the next Mail we didn't know already & fetch it; if no more items in list -> goodbuye.
- POP3C_ReadMessageBody() after the I/O layer read the message from the socket, parse it; move our context over to the DB-Queue.
- POP3C_ReadMessageBodyFollowing(): On success initialize the async message reader; let the I/O layer read the message, then continue parsing & saving it.
- POP3C_StoreMsgRead(): parse & save the message to the DB. Continue with remembering it in the UT table.
- POP3C_SendDelete(): check whether we want to delete the mails on the server, if do; else continue with POP3C_SendGetOneMsg() to fetch next in list.
- POP3_C_ReadServerStatus(): switch between line based read operation and reading whole messages.
we now can successfully aggregate one pop3 account.
Wilfried Goesgens [Wed, 17 Aug 2011 17:18:07 +0000 (17:18 +0000)]
add infrastructure to cleanly swap inbetween the DB and the IO queue; add infrastructure needed for async message reading.
- QueueDBOperation(): use our own vars on the AsyncIO struct, so we don't overwrite handlers & events from the file I/O stuff.
- DB_PerformNext(): don't use default: so we get warnings if new states are added and not handled here.
- DB_PerformNext(): if we don't get a DB query or Terminate, the db operations are done, remove our handlers, this AsyncIO context is handed over into another Queue by the application logic.
- NextDBOperation(): use our own vars on the AsyncIO struct, so we don't overwrite handlers & events from the file I/O stuff.
- HandleInbound(): handle more read-states; needed for message reading.
- HandleInbound(): move Rumpelstilskin lookup if to switch()
- IO_send_callback(): fix/add IO debugging to /tmp/foo
- IO_send_callback(): Adjust switch to some edge cases:
- eSendReply: check whether we're done with sending, if, fall forward to writing.
- eReadMore: was missing, continue reading (if)
- eReadPayload: new, read smtp messages for example.
- eDBQuery: the application logic want to perform database I/O with this context, clean up hooks and remove it from this queue.
- set_start_callback(): properly hanle the new cases
- IO_recv_callback(): fix/add IO debugging to /tmp/foo
- ReAttachIO(): use me if you want to continue with file I/O after i.e. DB-I/O
Wilfried Goesgens [Wed, 17 Aug 2011 17:13:59 +0000 (17:13 +0000)]
Add async message reading
- CtdlSubmitMsg(): make param const, since its just read
- add NewAsyncMsg(): preprocess the parameters provided by application logic, return the worker struct for message reading
- add CtdlReadMessageBodyAsync(): call me from your IO handler to read a message from the socket asynchroneous.
- add DeleteAsyncMsg(): call me to delete the worker vars.
Wilfried Goesgens [Wed, 17 Aug 2011 17:09:57 +0000 (17:09 +0000)]
Rename function to cary the module prefix.
Wilfried Goesgens [Wed, 10 Aug 2011 14:02:00 +0000 (14:02 +0000)]
work on the pop3 aggregator
- rewrite the config parser similar to the rss-configparser
- copy over handler logic from async smtp connector
- chop worker into handy bits so the handler logic can control it.
Wilfried Goesgens [Wed, 10 Aug 2011 14:00:34 +0000 (14:00 +0000)]
fix naming of functions, so they live in the RSS-Namespace
Wilfried Goesgens [Wed, 10 Aug 2011 13:59:20 +0000 (13:59 +0000)]
tiny fixes
- log failed connection attempts
- add sample to set 127.0.0.1
Wilfried Goesgens [Wed, 10 Aug 2011 13:55:42 +0000 (13:55 +0000)]
fix parsing of IP-Urls
- need to point inet_pton() to the rightly casted pointer in case of non-IPv6
- set the port using hton()
- set the protocol depending on our findings
Wilfried Goesgens [Sun, 24 Jul 2011 13:35:59 +0000 (13:35 +0000)]
StrBufSipLine(): make input params const; fix several warnings about unused code.
Conflicts:
libcitadel/lib/libcitadel.h
Art Cancro [Mon, 18 Jul 2011 20:45:06 +0000 (16:45 -0400)]
DBG_QR[2] now a configurable loggable
Art Cancro [Mon, 18 Jul 2011 20:21:29 +0000 (16:21 -0400)]
Wrap all calls to pthread_mutex_lock() and pthread_mutex_unlock() within CtdlLogResult()
Art Cancro [Mon, 18 Jul 2011 20:19:45 +0000 (16:19 -0400)]
Convenience function CtdlLogResult()
Wilfried Goesgens [Sun, 10 Jul 2011 13:40:38 +0000 (13:40 +0000)]
finalize allocation & freeing of rss stuff
Wilfried Goesgens [Sat, 9 Jul 2011 20:50:48 +0000 (20:50 +0000)]
make shure invalid session fd's don't get set into the central select.
Wilfried Goesgens [Sat, 9 Jul 2011 20:50:33 +0000 (20:50 +0000)]
use right struct for memset.
Wilfried Goesgens [Thu, 7 Jul 2011 22:37:22 +0000 (22:37 +0000)]
work on making RSS aggregator instances and roomlists consistant.
Wilfried Goesgens [Thu, 7 Jul 2011 19:50:26 +0000 (19:50 +0000)]
Fix documentation
- fix bug about escaping in strbuf
- link Hashlist Componets to groups so they can be easily accessed.
Wilfried Goesgens [Sun, 3 Jul 2011 22:35:27 +0000 (22:35 +0000)]
fix conditions where several rooms read the same RSS feed
Wilfried Goesgens [Sun, 3 Jul 2011 22:34:03 +0000 (22:34 +0000)]
merge in euid generation from master
Wilfried Goesgens [Sun, 3 Jul 2011 13:49:59 +0000 (13:49 +0000)]
first working RSS collection with async DB-Saves
- for now, no shorter URL expansion
- replace handling of configs, old algorythm wasn't thread-safe
- DB i/o is done in its own libev Queue in its own thread now; so we don't block network IO with database IO.
Wilfried Goesgens [Sun, 3 Jul 2011 13:47:55 +0000 (13:47 +0000)]
Add ability to hex encode binary strings
- when encoding plain md5 binary buffers we have to pass the length into the hex encoder, add parameter
- add wrapper providing old schematic
- fix documentation of StrBufSipLine()
Wilfried Goesgens [Sat, 25 Jun 2011 07:12:46 +0000 (07:12 +0000)]
more work on libevĀ“ing the rss-reader.
Wilfried Goesgens [Mon, 6 Jun 2011 19:33:58 +0000 (19:33 +0000)]
fix loop / handling error when connecting fails immediately
- add new ev_idle to unwind the stack on connecting
- add a new handler that processes the disconnect
Wilfried Goesgens [Sun, 5 Jun 2011 21:22:37 +0000 (21:22 +0000)]
fix pointer indirections so we now can successfully fetch RSS feeds again.
Wilfried Goesgens [Sun, 5 Jun 2011 21:20:17 +0000 (21:20 +0000)]
when not forking, we fail to drop rood privileges in advance before writing netconfig files, so we loose later access to them. Sadly we don't know our GID...
Wilfried Goesgens [Sun, 5 Jun 2011 16:48:06 +0000 (16:48 +0000)]
move stuff out into own files from here...
Wilfried Goesgens [Sun, 5 Jun 2011 16:46:41 +0000 (16:46 +0000)]
Split RSS Parser...
- URL-De-Shortener module
- RSS/ATOM Handlers
- start migrating to event/curl requests
Wilfried Goesgens [Sun, 5 Jun 2011 16:44:59 +0000 (16:44 +0000)]
if specified, use our bind-ip for the http clients.
Wilfried Goesgens [Sun, 5 Jun 2011 16:42:29 +0000 (16:42 +0000)]
Give us an own Context per notification client.
Wilfried Goesgens [Sun, 5 Jun 2011 16:41:39 +0000 (16:41 +0000)]
rebuilt from .y
Wilfried Goesgens [Sun, 5 Jun 2011 15:28:24 +0000 (15:28 +0000)]
* make it const baby.
Wilfried Goesgens [Sun, 5 Jun 2011 13:49:09 +0000 (13:49 +0000)]
Free all our allocated resources after notifying / sending a http request.
Wilfried Goesgens [Mon, 30 May 2011 21:52:00 +0000 (21:52 +0000)]
finish rewriting of http client code
- we now have a context
- we now handle replies
- still some uncertain cleanup condition.
Wilfried Goesgens [Sat, 21 May 2011 20:09:39 +0000 (20:09 +0000)]
Migrate http context struct into the AsyncIO struct; streamline other structs.
Wilfried Goesgens [Sat, 21 May 2011 13:08:23 +0000 (13:08 +0000)]
unlink global struct and request structs
Wilfried Goesgens [Thu, 19 May 2011 21:19:25 +0000 (21:19 +0000)]
Fix unfolding of RFC822 lines so we don't loose blanks.
Wilfried Goesgens [Wed, 18 May 2011 21:35:30 +0000 (21:35 +0000)]
Don't overwrite the 'TO' when sending mailinglist mails; we put ourselves into 'reply to:'
Wilfried Goesgens [Sun, 15 May 2011 21:11:03 +0000 (21:11 +0000)]
Rewrite loading of notify configurations
Wilfried Goesgens [Sun, 15 May 2011 14:12:15 +0000 (14:12 +0000)]
sending http requests instant now.
Wilfried Goesgens [Sun, 15 May 2011 10:43:11 +0000 (10:43 +0000)]
work on libev <-> libcurl integration
- move libcurl handlers into the libev file; they need to be initialized to close to separate
- add some logging;
First successfull CURL notification via libev!
Wilfried Goesgens [Sun, 15 May 2011 10:42:26 +0000 (10:42 +0000)]
Fix parsing and rearaning of URLs
Wilfried Goesgens [Sat, 14 May 2011 14:28:50 +0000 (14:28 +0000)]
don't copy filedescriptors into the clone.
Wilfried Goesgens [Sat, 14 May 2011 10:23:48 +0000 (10:23 +0000)]
more work on async http
Wilfried Goesgens [Sat, 14 May 2011 09:49:43 +0000 (09:49 +0000)]
Memleak: free the vector of URLs to try to connect to.
Wilfried Goesgens [Tue, 10 May 2011 22:05:46 +0000 (22:05 +0000)]
just log the close if we're realy going to close.
Wilfried Goesgens [Tue, 10 May 2011 22:03:20 +0000 (22:03 +0000)]
Don't close 0-filedescriptors on session killing, its our indicator for not here, since 0 equals STDIN which at best is redirected to /dev/null
Wilfried Goesgens [Mon, 9 May 2011 21:44:48 +0000 (21:44 +0000)]
use long hash function, if we intend to use long values as keys, else we won't find anything
Wilfried Goesgens [Sun, 8 May 2011 17:43:46 +0000 (17:43 +0000)]
Stop watchers / don't close 0
Wilfried Goesgens [Sun, 8 May 2011 12:00:04 +0000 (12:00 +0000)]
work on errorhandling for the smtp event client
- add states for the mx lookup phases
- return the propper states
- close filedescriptors if we fail one attempt
- say which connection type we have: immediate, e_again, fail
- say which NS-lookup whe do (a/aaaa)
Wilfried Goesgens [Sun, 8 May 2011 11:58:27 +0000 (11:58 +0000)]
Add backtrace debugging facility that doesn't add linebreaks
Wilfried Goesgens [Sun, 8 May 2011 11:53:34 +0000 (11:53 +0000)]
libevent needs to link the mathlib, so we add -lm
Wilfried Goesgens [Tue, 3 May 2011 20:11:46 +0000 (20:11 +0000)]
ups, .lo not .o
Wilfried Goesgens [Tue, 3 May 2011 19:55:11 +0000 (19:55 +0000)]
add prerelease so we don't get deinstalled all the time..
Wilfried Goesgens [Tue, 3 May 2011 19:53:26 +0000 (19:53 +0000)]
fix problem with po missing \n
Wilfried Goesgens [Tue, 3 May 2011 19:52:05 +0000 (19:52 +0000)]
Work on integrating libcurl <-> libev
Wilfried Goesgens [Tue, 3 May 2011 19:49:23 +0000 (19:49 +0000)]
Fix possible endless loop conditions
Wilfried Goesgens [Thu, 28 Apr 2011 19:20:52 +0000 (19:20 +0000)]
Various cleanups of abort conditions
- if the timeout/connectionfail/... calls tell us to abort, shutdown the client
- unregister all event structs
Wilfried Goesgens [Thu, 28 Apr 2011 19:10:56 +0000 (19:10 +0000)]
Name of MX in error messages
- set it after successfull MX lookup
- put in that we don't have an mx record if its NULL instead of 'no name'
Wilfried Goesgens [Thu, 28 Apr 2011 19:09:53 +0000 (19:09 +0000)]
fix gcc warning about it not knowing which pointer indirection is const...
Wilfried Goesgens [Tue, 19 Apr 2011 19:55:16 +0000 (19:55 +0000)]
Include ourselves??
Wilfried Goesgens [Tue, 19 Apr 2011 19:50:55 +0000 (19:50 +0000)]
Fix Crash
* check pointer before accessing it
* fix probably uninitialized case
Wilfried Goesgens [Sat, 2 Apr 2011 15:56:18 +0000 (17:56 +0200)]
Give event clients a citadel session context.
Wilfried Goesgens [Sat, 2 Apr 2011 15:24:17 +0000 (17:24 +0200)]
Cleanup the event client struct, remove unused vars
Wilfried Goesgens [Sat, 2 Apr 2011 10:18:49 +0000 (12:18 +0200)]
just use the ParsedURL to bear our destination address & port
Wilfried Goesgens [Sat, 2 Apr 2011 09:26:07 +0000 (11:26 +0200)]
Use IOBuffer with its StrBuf + const char* inside instead of having two of them
Wilfried Goesgens [Fri, 1 Apr 2011 21:48:57 +0000 (23:48 +0200)]
implement abort handling
Wilfried Goesgens [Fri, 1 Apr 2011 19:56:20 +0000 (21:56 +0200)]
Switch handling to have a pointer to the target address
- change URL struct to contain sockaddr_in6
- change URL-Parsing to add that indirection
- change all pointer indirections so the new way fits
Wilfried Goesgens [Fri, 1 Apr 2011 14:51:48 +0000 (16:51 +0200)]
move URL parsing / struct over into libcitadel
Wilfried Goesgens [Thu, 31 Mar 2011 22:50:43 +0000 (00:50 +0200)]
Fix searching of ']'
Wilfried Goesgens [Thu, 31 Mar 2011 21:21:10 +0000 (23:21 +0200)]
Add busines logic for toggling multiple mx records etc.
- move handlers into their own file
- chain up stuff via URL-structs
(not yet all done)