> The bad news is that elogd is still crashing even after disabling Image Magick.  The good news is that this time it was reproducible and I did obtain a
> stack trace using gdb.

The seg fault reported above seems to be related to a specific elog entry combined with sending email notifications.  When trying to submit changes to that entry while sending email notifications, elogd seg faults.  We can edit the entry before it and create and edit a new entry after 
it.  Trying to edit that one entry and send notifications, however, reliably crashes elogd.  If I check "supress email notification", elogd does not seem to crash.  I have replicated this on separate elog servers running both SL4 and on SL5.  To compare with SL4, here is the trace I see on SL5 (Scientific Linux 5).

Once I receive the OK, I will send you the actual file that stores the problematic entry.  Please let me know if there is anything else I can send you.

Many thanks,
Devin

------
[root@lnx767 ~]# gdb /usr/local/sbin/elogd 13328
GNU gdb Red Hat Linux (6.5-37.el5_2.2rh)
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".

Attaching to program: /usr/local/sbin/elogd, process 13328
Reading symbols from /lib/libssl.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libssl.so.6
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /usr/lib/libgssapi_krb5.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libgssapi_krb5.so.2
Reading symbols from /usr/lib/libkrb5.so.3...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libkrb5.so.3
Reading symbols from /lib/libcom_err.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libcom_err.so.2
Reading symbols from /usr/lib/libk5crypto.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libk5crypto.so.3
Reading symbols from /lib/libresolv.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /lib/libcrypto.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypto.so.6
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/lib/libz.so.1...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /usr/lib/libkrb5support.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libkrb5support.so.0
Reading symbols from /lib/libkeyutils.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/libkeyutils.so.1
Reading symbols from /lib/libselinux.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libselinux.so.1
Reading symbols from /lib/libsepol.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libsepol.so.1
Reading symbols from /lib/libnss_files.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_nis.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_nis.so.2
Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libnsl.so.1

(no debugging symbols found)
0x008db402 in __kernel_vsyscall ()
(gdb) c
Continuing.
(no debugging symbols found)

Program received signal SIGSEGV, Segmentation fault.
0x0804a125 in strieq ()
(gdb) where
#0  0x0804a125 in strieq ()
#1  0x73207365 in ?? ()
#2  0x00000000 in ?? ()

Hi Stefan,

I hope I'm not bombarding you, but we seem to be seeing crashes in two separate scenarios.  In addition to the crashes I previously reported (editing a problematic entry and sending notifications), we 
have a (seemingly) separate means of crashing elogd.

One of our users connects from home using a satellite network (bursty and high latency).  Any time he attempts to upload an image from this connection, elogd crashes.  He has not yet seen any 
problems when using the same computer on site.  Here are the stack traces I've obtained, first on SL4, then on SL5.  Again, please let me know if there is any other information I can provide.

Thank you very much for your time and effort.

Sincerely,
Devin

------
[root@lnx100 ~]# gdb /usr/local/sbin/elogd 3728
GNU gdb Red Hat Linux (6.3.0.0-1.143.el4rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...(no debugging symbols found)
Using host libthread_db library "/lib/tls/libthread_db.so.1".

Attaching to program: /usr/local/sbin/elogd, process 3728
Reading symbols from /lib/libssl.so.4...(no debugging symbols found)...done.
Loaded symbols for /lib/libssl.so.4
Reading symbols from /lib/tls/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/tls/libc.so.6
Reading symbols from /usr/lib/libgssapi_krb5.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libgssapi_krb5.so.2
Reading symbols from /usr/lib/libkrb5.so.3...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libkrb5.so.3
Reading symbols from /lib/libcom_err.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libcom_err.so.2
Reading symbols from /usr/lib/libk5crypto.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libk5crypto.so.3
Reading symbols from /lib/libresolv.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /lib/libcrypto.so.4...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypto.so.4
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/lib/libz.so.1...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_nis.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_nis.so.2
Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libnsl.so.1
0x008c17a2 in _dl_sysinfo_int80 ()
   from /lib/ld-linux.so.2
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0x080af727 in decode_post ()
(gdb) where
#0  0x080af727 in decode_post ()
#1  0x00000000 in ?? ()
(gdb) quit
The program is running.  Quit anyway (and detach it)? (y or n) y
Detaching from program: /usr/local/sbin/elogd, process 3728

----

[root@lnx767 ~]# gdb /usr/local/sbin/elogd 13111
GNU gdb Red Hat Linux (6.5-37.el5_2.2rh)
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".

Attaching to program: /usr/local/sbin/elogd, process 13111
Reading symbols from /lib/libssl.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libssl.so.6
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /usr/lib/libgssapi_krb5.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libgssapi_krb5.so.2
Reading symbols from /usr/lib/libkrb5.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libkrb5.so.3
Reading symbols from /lib/libcom_err.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libcom_err.so.2
Reading symbols from /usr/lib/libk5crypto.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libk5crypto.so.3
Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /lib/libcrypto.so.6...
(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypto.so.6
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /usr/lib/libkrb5support.so.0...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libkrb5support.so.0
Reading symbols from /lib/libkeyutils.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libkeyutils.so.1
Reading symbols from /lib/libselinux.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libselinux.so.1
Reading symbols from /lib/libsepol.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/libsepol.so.1
Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_nis.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_nis.so.2
Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libnsl.so.1
(no debugging symbols found)
0x00308402 in __kernel_vsyscall ()
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0x080b2f8a in decode_post ()
(gdb) where
#0  0x080b2f8a in decode_post ()
#1  0x00000100 in ?? ()
#2  0x00000000 in ?? ()

Hi Stefan,

Just incase it helps, I am attaching the file for an entry in our demo logbook.  If I edit this entry and click on submit (without checking "suppress email notification"), I get the seg fault shown below.

If I delete any single character from the entry, I do not see any problems.  If I re-insert a character anywhere, the problem returns.

I hope this is useful.  

Thanks again,
Devin

------
[root@lnx100 ~]# gdb /usr/local/sbin/elogd 6040
GNU gdb Red Hat Linux (6.3.0.0-1.143.el4rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...(no debugging symbols found)
Using host libthread_db library "/lib/tls/libthread_db.so.1".

Attaching to program: /usr/local/sbin/elogd, process 6040
Reading symbols from /lib/libssl.so.4...(no debugging symbols found)...done.
Loaded symbols for /lib/libssl.so.4
Reading symbols from /lib/tls/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/tls/libc.so.6
Reading symbols from /usr/lib/libgssapi_krb5.so.2...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libgssapi_krb5.so.2
Reading symbols from /usr/lib/libkrb5.so.3...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libkrb5.so.3
Reading symbols from /lib/libcom_err.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libcom_err.so.2
Reading symbols from /usr/lib/libk5crypto.so.3...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libk5crypto.so.3
Reading symbols from /lib/libresolv.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /lib/libcrypto.so.4...(no debugging symbols found)...done.
Loaded symbols for /lib/libcrypto.so.4
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/lib/libz.so.1...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /lib/libnss_nis.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_nis.so.2
Reading symbols from /lib/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libnsl.so.1
0x008c17a2 in _dl_sysinfo_int80 ()
   from /lib/ld-linux.so.2
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0x080ad900 in interprete ()
(gdb) where
#0  0x080ad900 in interprete ()
#1  0x00000031 in ?? ()
#2  0x000005dc in ?? ()
#3  0x00000000 in ?? ()

 Sorry, it took me so long to respond. I didn't notice your request.

Attributes = Instrument, Notify Lead Tech

Options Instrument = Olympus, Beckman

Options Notify Lead Tech = Yes{a}, No {b}

{a}Email Instrument Olympus = John@lab.net         (i.e. if it is "Yes" and  "Olympus" - John gets an email)

{a}Email Instrument Beckman = Mary@lab.net         (i.e. if it is "Yes" and  "Beckman" - Mary gets an email)

 Sorry, it took me so long to respond. I didn't notice your request.

Attributes = Instrument, Notify Lead Tech

Options Instrument = Olympus, Beckman

Options Notify Lead Tech = Yes{a}, No {b}

{a}Email Instrument Olympus = John@lab.net         (i.e. if it is "Yes" and  "Olympus" - John gets an email)

{a}Email Instrument Beckman = Mary@lab.net         (i.e. if it is "Yes" and  "Beckman" - Mary gets an email)

No, no idea. I usually put several MB attachments into elogs without problems. Can you try with the demo logbook https://midas.psi.ch/elogs/Linux+Demo/. Maybe you have an old version of elog? 

You have an old version of elog, this bug has been fixed some time ago. Have a look for example at https://midas.psi.ch/elogs/Linux+Demo/.

If you click on a peperclip there, the attachment is shown correctly. 

Hi Devin,

first of all, your stack traces are only of limited use for me. This typically happens 
if you attach gdb to a running process, then you get something like

#0  0x080b2f8a in decode_post ()
#1  0x00000100 in ?? ()
#2  0x00000000 in ?? ()

(note the ??). If you run elogd directly from gdb, the stack trace contains much more information:

[meg@megon elog]# gdb elogd
...

(gdb) run
...
Server listening on port 8080 ...

Program received signal SIGINT, Interrupt.
0x0000003cb48c78d3 in __select_nocancel () from /lib64/libc.so.6
(gdb) where
#0  0x0000003cb48c78d3 in __select_nocancel () from /lib64/libc.so.6
#1  0x000000000046ea51 in server_loop () at src/elogd.c:27688
#2  0x0000000000471de8 in main (argc=1, argv=0x7fffe2b9bf18) at src/elogd.c:29018
(gdb) 

including the line numbers, arguments etc. So please try to start elogd from inside gdb 
and then reproduce your crash.

Your first problem seems to be related to some contents of your elogd.cfg, since in 
one stack dump I saw a 

strlen()
...
getcfg()

Here, the getcfg() function is called to retrieve some configuration from elogd.cfg. 
Maybe you have a very long line, or the file is otherwise corrupt. Please check that
carfully and send me your elogd.cfg so that I can have a look myself. Usually it helps
to remove one line after the other and check when the problem disappears.

Your other problem which has the decode_post() in the stack dump seems to be related
to the case when you upload an entry (or attachment), and the TCP link breaks in 
the middle. Probably the error handling in such a case is not correct. I will try
to reproduce this, although I don't have a satellite network.

Best regards,

  Stefan