Unwanted faux DST date shifts to time stamps in mirror transactions, posted by Andreas Warburton on Tue May 3 20:05:29 2011
|
Hi,
I have a MacOSX 10.6.7 laptop that periodically synchronizes my logbook with that on a Debian linux web server. Both instances of ELOG are 2.9.0-2396.
When I create a new entry on the laptop, the indicated time stamp is correct. When I click Synchronize on the laptop, the entry that gets mirrored to the Debian server has a time stamp exactly one hour earlier (in the past) on the Debian side.
Likewise, when I create a new entry on the Debian server, the indicated time stamp is correct. When I click Synchronize on the laptop, the entry that gets mirrored from the Debian machine to the laptop has a time stamp that is exactly one hour in the future on the laptop.
This appears to be due to an incorrect handling of times with regard to "summer" (daylight savings) and "standard" time when mirroring between these two different machines.
Is there a straightforward workaround to immunize against this sensitivity?
Thanks!
Andreas
|
elogd crashes during Find and Mirror operations under MacOS, posted by Andreas Warburton on Sat Dec 24 10:40:00 2016
|
When installing ELOG 3.1.2 on my Mac Sierra (10.12.2) with the latest XCode 8.2.1, it seemed to install fine but for one warning when building elogd:
cc -O3 -funroll-loops -fomit-frame-pointer -W -Wall -Wno-deprecated-declarations -DHAVE_SSL -I /opt/local/include -o elogd
src/elogd.c crypt.o auth.o regex.o mxml.o -lssl
src/elogd.c:13840:13: warning: '__builtin___strlcpy_chk' will always overflow
destination buffer [-Wbuiltin-memcpy-chk-size]
strlcpy(user_email, user_list[i], NAME_LENGTH);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/secure/_string.h:105:3: note: expanded from macro 'strlcpy'
__builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
My installation was done with "sudo make install", as the instructions state. I have also been using the prescribed "launchctl" approach.
1. Is the above warning something that needs attention/correcting?
2. My elogd crashes now every time that I attempt any kind of "Find" operation.
3. I synchronize my MacOS version of elog with a linux-based version on my university's web server. I have installed 3.1.2 on the linux server, and it is running there without problems. When I now try to synchronize my 3.1.2 ELOG on MacOS laptop with the same version on my linux server, it appears to do the first mirroring operation correctly, but elogd crashes when I try a second synchronization at a later time. (For now, I have "Mirror simulate = 1" to prevent inadvertent damage.)
Can anyone help me with the above points?
Thank you,
Andreas
|
Re: elogd crashes during Find and Mirror operations under MacOS, posted by Andreas Warburton on Tue Jan 3 12:43:56 2017
|
Stefan Ritt wrote: |
The current GIT revision (c017d07) fixes the problem with the strlcpy() function (macOS 10.12.2, XCode 8.2). Can you try if that fixes your other problems, too?
Plese remember to use the recursive clone to get that version:
git clone https://bitbucket.org/ritt/elog --recursive
Stefan
Andreas Warburton wrote: |
When installing ELOG 3.1.2 on my Mac Sierra (10.12.2) with the latest XCode 8.2.1, it seemed to install fine but for one warning when building elogd:
cc -O3 -funroll-loops -fomit-frame-pointer -W -Wall -Wno-deprecated-declarations -DHAVE_SSL -I /opt/local/include -o elogd
src/elogd.c crypt.o auth.o regex.o mxml.o -lssl
src/elogd.c:13840:13: warning: '__builtin___strlcpy_chk' will always overflow
destination buffer [-Wbuiltin-memcpy-chk-size]
strlcpy(user_email, user_list[i], NAME_LENGTH);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/secure/_string.h:105:3: note: expanded from macro 'strlcpy'
__builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
My installation was done with "sudo make install", as the instructions state. I have also been using the prescribed "launchctl" approach.
1. Is the above warning something that needs attention/correcting?
2. My elogd crashes now every time that I attempt any kind of "Find" operation.
3. I synchronize my MacOS version of elog with a linux-based version on my university's web server. I have installed 3.1.2 on the linux server, and it is running there without problems. When I now try to synchronize my 3.1.2 ELOG on MacOS laptop with the same version on my linux server, it appears to do the first mirroring operation correctly, but elogd crashes when I try a second synchronization at a later time. (For now, I have "Mirror simulate = 1" to prevent inadvertent damage.)
Can anyone help me with the above points?
Thank you,
Andreas
|
|
|
Re: elogd crashes during Find and Mirror operations under MacOS, posted by Andreas Warburton on Tue Jan 3 12:45:13 2017
|
Thank you very much (both Andreas and Stefan)! The build now runs without warnings, and I am now able to do Find operations without crashes. I still am having some issues with the synchronization, but I suspect that this is elog-content related, so will in the coming days do some more testing from CERN (where I have higher bandwidth with my home institution in Canada). Incidentally, I had to add a line to Makefile in order to be able to build on my Mac. See below, in case you want to update the Mac Makefile for others.
Many thanks again,
Andreas
ifdef USE_SSL
ifneq ($(USE_SSL),0)
CFLAGS += -DHAVE_SSL
CFLAGS += -I /opt/local/include
LIBS += -lssl
endif
endif
(You may wish to update the distribution if you think it may benefit other Mac users.)
Stefan Ritt wrote: |
The current GIT revision (c017d07) fixes the problem with the strlcpy() function (macOS 10.12.2, XCode 8.2). Can you try if that fixes your other problems, too?
Plese remember to use the recursive clone to get that version:
git clone https://bitbucket.org/ritt/elog --recursive
Stefan
Andreas Warburton wrote: |
When installing ELOG 3.1.2 on my Mac Sierra (10.12.2) with the latest XCode 8.2.1, it seemed to install fine but for one warning when building elogd:
cc -O3 -funroll-loops -fomit-frame-pointer -W -Wall -Wno-deprecated-declarations -DHAVE_SSL -I /opt/local/include -o elogd
src/elogd.c crypt.o auth.o regex.o mxml.o -lssl
src/elogd.c:13840:13: warning: '__builtin___strlcpy_chk' will always overflow
destination buffer [-Wbuiltin-memcpy-chk-size]
strlcpy(user_email, user_list[i], NAME_LENGTH);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/secure/_string.h:105:3: note: expanded from macro 'strlcpy'
__builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
My installation was done with "sudo make install", as the instructions state. I have also been using the prescribed "launchctl" approach.
1. Is the above warning something that needs attention/correcting?
2. My elogd crashes now every time that I attempt any kind of "Find" operation.
3. I synchronize my MacOS version of elog with a linux-based version on my university's web server. I have installed 3.1.2 on the linux server, and it is running there without problems. When I now try to synchronize my 3.1.2 ELOG on MacOS laptop with the same version on my linux server, it appears to do the first mirroring operation correctly, but elogd crashes when I try a second synchronization at a later time. (For now, I have "Mirror simulate = 1" to prevent inadvertent damage.)
Can anyone help me with the above points?
Thank you,
Andreas
|
|
|
elogd crashes during SSL Mirror operations involving attachments, posted by Andreas Warburton on Mon Jan 9 17:15:56 2017
|
My MacOS (10.12.2) elogd version V3.1.2 is a recent git commit (edc5e85), due to the fix to my earlier-described issue solved in the thread here: https://midas.psi.ch/elogs/Forum/68519.
I am trying to (re-)set up Mirror functionality with a linux server running the standard public (V3.1.2-bd75964). I had initially updated the linux server so that it also had the latest git commit (edc5e85), but could then not even add new logbook entries that involved attachments to it. I therefore rolled the linux server back to the standard public 3.1.2 version.
On the remote Mac, synchronizations usually look like they are going to work fine, with Mirror simulate = 1 switched on. After I set Mirror simulate = 0, and if the server and remote logbook are already identical, I *occasionally* get the proper "All Entries Identical" synchronization result. Unfortunately, this is very rare, and usually there is a failure whereby the remote (Mac) logbook decides that a significant fraction of its entries (usually sequential, from some seemingly random entry all the way up to the last entry) are missing on the linux server and need to be submitted back to the server from the remote Mac.
When the local and remote logbooks are not identical, and a record in need of synchronization contains an attachment, there is again destructive behaviour similar to that described above, except that the Mac elogd executable usually crashes. (As in the case of the already-identical synchronizations described above, I only tested this after observing the correct expected behaviour first with Mirror simulate = 1.)
I'd be grateful for some help/suggestions. My current testing suggests that my problems are likely not elog-content dependent. (The logbook now undergoing synching has less than 10 entries in it.)
More generally, the issue of having things behave fine with Mirror simulate = 1, but then experiencing corruption/damage when switching to Mirror simulate = 0 seems serious to me.
Many thanks, Andreas
|
Re: elogd crashes during SSL Mirror operations involving attachments, posted by Andreas Warburton on Fri Jan 13 08:27:15 2017
|
The attached screenshot shows the behaviour after doing a synchronization (with Mirror simulate = 1) following first having ensured that the local (Mac) and remote (linux) ELOGs initially showed "All entries identical" when doing a simulated synchronization, and then having edited local entries 9707 and 9709 by uploading (different) attachments to them.
The fact that the synchronization is suggesting to renumber two different entry IDs to the same number looks like a bug.
Best regards,
Andreas W.
Andreas Warburton wrote: |
My MacOS (10.12.2) elogd version V3.1.2 is a recent git commit (edc5e85), due to the fix to my earlier-described issue solved in the thread here: https://midas.psi.ch/elogs/Forum/68519.
I am trying to (re-)set up Mirror functionality with a linux server running the standard public (V3.1.2-bd75964). I had initially updated the linux server so that it also had the latest git commit (edc5e85), but could then not even add new logbook entries that involved attachments to it. I therefore rolled the linux server back to the standard public 3.1.2 version.
On the remote Mac, synchronizations usually look like they are going to work fine, with Mirror simulate = 1 switched on. After I set Mirror simulate = 0, and if the server and remote logbook are already identical, I *occasionally* get the proper "All Entries Identical" synchronization result. Unfortunately, this is very rare, and usually there is a failure whereby the remote (Mac) logbook decides that a significant fraction of its entries (usually sequential, from some seemingly random entry all the way up to the last entry) are missing on the linux server and need to be submitted back to the server from the remote Mac.
When the local and remote logbooks are not identical, and a record in need of synchronization contains an attachment, there is again destructive behaviour similar to that described above, except that the Mac elogd executable usually crashes. (As in the case of the already-identical synchronizations described above, I only tested this after observing the correct expected behaviour first with Mirror simulate = 1.)
I'd be grateful for some help/suggestions. My current testing suggests that my problems are likely not elog-content dependent. (The logbook now undergoing synching has less than 10 entries in it.)
More generally, the issue of having things behave fine with Mirror simulate = 1, but then experiencing corruption/damage when switching to Mirror simulate = 0 seems serious to me.
Many thanks, Andreas
|
|
Re: elogd crashes during SSL Mirror operations involving attachments, posted by Andreas Warburton on Fri Jan 13 21:05:49 2017
|
When I switch from SSL = 1 to SSL = 0 and I use http:// instead of https://, the ability to upload attachments to logbook entries returns. With both Chrome and Safari browsers, with SSL = 1 the file upload hangs after only a small percentage of the file has been uploaded. I ran the following openssl diagnostic on my elogd port. Would anyone have advice on what might be causing such errors?
tapajo [/usr/local/elog/elog-latest] openssl s_client -connect elog.hep.xxx.xxx.xx:80xx -state -nbio | grep "^SSL"
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:error in SSLv2/v3 read server hello A
SSL_connect:unknown state
depth=0 C = EU, ST = SomeState, L = SomeCity, O = SomeOranization, OU = SomeOrganizationUnit, CN = localhost
verify error:num=18:self signed certificate
verify return:1
depth=0 C = EU, ST = SomeState, L = SomeCity, O = SomeOranization, OU = SomeOrganizationUnit, CN = localhost
verify return:1
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:error in unknown state
SSL_connect:error in unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL handshake has read 1733 bytes and written 871 bytes
SSL-Session:
SSL3 alert read:warning:close notify
SSL3 alert write:warning:close notify
Andreas Warburton wrote: |
The attached screenshot shows the behaviour after doing a synchronization (with Mirror simulate = 1) following first having ensured that the local (Mac) and remote (linux) ELOGs initially showed "All entries identical" when doing a simulated synchronization, and then having edited local entries 9707 and 9709 by uploading (different) attachments to them.
The fact that the synchronization is suggesting to renumber two different entry IDs to the same number looks like a bug.
Best regards,
Andreas W.
Andreas Warburton wrote: |
My MacOS (10.12.2) elogd version V3.1.2 is a recent git commit (edc5e85), due to the fix to my earlier-described issue solved in the thread here: https://midas.psi.ch/elogs/Forum/68519.
I am trying to (re-)set up Mirror functionality with a linux server running the standard public (V3.1.2-bd75964). I had initially updated the linux server so that it also had the latest git commit (edc5e85), but could then not even add new logbook entries that involved attachments to it. I therefore rolled the linux server back to the standard public 3.1.2 version.
On the remote Mac, synchronizations usually look like they are going to work fine, with Mirror simulate = 1 switched on. After I set Mirror simulate = 0, and if the server and remote logbook are already identical, I *occasionally* get the proper "All Entries Identical" synchronization result. Unfortunately, this is very rare, and usually there is a failure whereby the remote (Mac) logbook decides that a significant fraction of its entries (usually sequential, from some seemingly random entry all the way up to the last entry) are missing on the linux server and need to be submitted back to the server from the remote Mac.
When the local and remote logbooks are not identical, and a record in need of synchronization contains an attachment, there is again destructive behaviour similar to that described above, except that the Mac elogd executable usually crashes. (As in the case of the already-identical synchronizations described above, I only tested this after observing the correct expected behaviour first with Mirror simulate = 1.)
I'd be grateful for some help/suggestions. My current testing suggests that my problems are likely not elog-content dependent. (The logbook now undergoing synching has less than 10 entries in it.)
More generally, the issue of having things behave fine with Mirror simulate = 1, but then experiencing corruption/damage when switching to Mirror simulate = 0 seems serious to me.
Many thanks, Andreas
|
|
|
Re: elogd crashes during SSL Mirror operations involving attachments, posted by Andreas Warburton on Sat Jan 14 08:27:42 2017
|
For the time being, I am deeming ELOG 3.1.2 unusable with https (SSL = 1) functionality on my "Debian GNU/Linux 7 (wheezy)" server with "OpenSSL 1.0.1t", due to the described apparent issues with SSL. Reverting to http (SSL = 0) brings back my ability to upload attachments and synchronize with a remote elogd running on a MacOS laptop.
Interestingly, my records indicate that I suffered a problem with very similar symptoms back in 2011, with version 2.8.0. The problem at that time, which is acknowledged in the Changelog as "Fixed bug with SSL connection shutdown", got fixed in version 2.9.0. Perhaps, when the "Replaced insecure SSLv23 with TLSv1 method" change was implemented for version 3.0.0, a similar issue was (re-)introduced?
It would of course be best if this issue were resolvable soon, due to the security vulnerabilities of http versus https. Thank you in advance for any efforts!
Best regards,
Andreas Warburton
Andreas Warburton wrote: |
When I switch from SSL = 1 to SSL = 0 and I use http:// instead of https://, the ability to upload attachments to logbook entries returns. With both Chrome and Safari browsers, with SSL = 1 the file upload hangs after only a small percentage of the file has been uploaded. I ran the following openssl diagnostic on my elogd port. Would anyone have advice on what might be causing such errors?
tapajo [/usr/local/elog/elog-latest] openssl s_client -connect elog.hep.xxx.xx:80xx -state -nbio | grep "^SSL"
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:error in SSLv2/v3 read server hello A
SSL_connect:unknown state
depth=0 C = EU, ST = SomeState, L = SomeCity, O = SomeOranization, OU = SomeOrganizationUnit, CN = localhost
verify error:num=18:self signed certificate
verify return:1
depth=0 C = EU, ST = SomeState, L = SomeCity, O = SomeOranization, OU = SomeOrganizationUnit, CN = localhost
verify return:1
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:error in unknown state
SSL_connect:error in unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL handshake has read 1733 bytes and written 871 bytes
SSL-Session:
SSL3 alert read:warning:close notify
SSL3 alert write:warning:close notify
Andreas Warburton wrote: |
The attached screenshot shows the behaviour after doing a synchronization (with Mirror simulate = 1) following first having ensured that the local (Mac) and remote (linux) ELOGs initially showed "All entries identical" when doing a simulated synchronization, and then having edited local entries 9707 and 9709 by uploading (different) attachments to them.
The fact that the synchronization is suggesting to renumber two different entry IDs to the same number looks like a bug.
Best regards,
Andreas W.
Andreas Warburton wrote: |
My MacOS (10.12.2) elogd version V3.1.2 is a recent git commit (edc5e85), due to the fix to my earlier-described issue solved in the thread here: https://midas.psi.ch/elogs/Forum/68519.
I am trying to (re-)set up Mirror functionality with a linux server running the standard public (V3.1.2-bd75964). I had initially updated the linux server so that it also had the latest git commit (edc5e85), but could then not even add new logbook entries that involved attachments to it. I therefore rolled the linux server back to the standard public 3.1.2 version.
On the remote Mac, synchronizations usually look like they are going to work fine, with Mirror simulate = 1 switched on. After I set Mirror simulate = 0, and if the server and remote logbook are already identical, I *occasionally* get the proper "All Entries Identical" synchronization result. Unfortunately, this is very rare, and usually there is a failure whereby the remote (Mac) logbook decides that a significant fraction of its entries (usually sequential, from some seemingly random entry all the way up to the last entry) are missing on the linux server and need to be submitted back to the server from the remote Mac.
When the local and remote logbooks are not identical, and a record in need of synchronization contains an attachment, there is again destructive behaviour similar to that described above, except that the Mac elogd executable usually crashes. (As in the case of the already-identical synchronizations described above, I only tested this after observing the correct expected behaviour first with Mirror simulate = 1.)
I'd be grateful for some help/suggestions. My current testing suggests that my problems are likely not elog-content dependent. (The logbook now undergoing synching has less than 10 entries in it.)
More generally, the issue of having things behave fine with Mirror simulate = 1, but then experiencing corruption/damage when switching to Mirror simulate = 0 seems serious to me.
Many thanks, Andreas
|
|
|
|
|