Demo Discussion
Forum Config Examples Contributions Vulnerabilities
  Discussion forum about ELOG, Page 18 of 238  Not logged in ELOG logo
icon5.gif   While mirroring, data fields not preserved, posted by Rob Calkins on Tue Oct 26 01:21:01 2021 

While running two e-log books that were mirrored, I ended up with the situation of two entries with the same number/id.  The mirroring did what it said it would, increment the local logbook entry and grab the entry from the remote logbook. However, when it did, it did not preserve the fields in the log book that are specified in the config file such as "Author", "Priority", "Subject" ect.  I ended up with a very minimal log:

icon5.gif   Too many open files - issue?, posted by Rob Calkins on Fri Oct 15 23:57:38 2021 

Has anyone had issues with having too many files open? I'll setup my server and let it go but after a while, I end up with a lot of "cannot create socket: Too many open files" errors being reported.  I have a sync to another e-log going which I suspect is part of the cause since that e-log server hasn't had this issue. I suspect that there are files being opened, going into some return loop code and then never getting closed. I'm not a C programmer but I see lines like :

fh = open(tmp_filename, O_RDONLY);
      if (fh > 0) {
         read(fh, result, size - 1);
         close(fh);
      }

      /* remove temporary file */
      remove(tmp_filename);

This looks like it opens the file but unless the remove function closes the file, it will remain open even through the file has been deleted. Maybe this isn't the correct behaviour of 'remove' and I am mistaken?

There are also parts like :

 fh = open(textfile, O_RDONLY | O_BINARY);
      if (fh < 0) {
         printf("Message file \"%s\" does not exist.\n", textfile);
         return 1;
      }

      size = (INT) lseek(fh, 0, SEEK_END);
      lseek(fh, 0, SEEK_SET);

      if (size > (INT) (sizeof(text) - 1)) {
         printf("Message file \"%s\" is too long (%zd bytes max).\n", textfile, sizeof(text));
         return 1;
      }

This looks like for the second error, it will complain that the file is too long, return an error message but not close the file and would leave it open. Is this a reasonable avenue to pursue or am I mis-reading the code?   Thanks.

    icon2.gif   Re: Too many open files - issue?, posted by Stefan Ritt on Mon Oct 25 13:34:06 2021 

The code segements you show are from the command line tool elog.c, not the server elogd.c. The tool is called to submit a new message from the command line. Even if there would be a file not properly closed, it will be closed by the operating system once the program finishes. So no problem of too many open files there.

Rob Calkins wrote:

Has anyone had issues with having too many files open? I'll setup my server and let it go but after a while, I end up with a lot of "cannot create socket: Too many open files" errors being reported.  I have a sync to another e-log going which I suspect is part of the cause since that e-log server hasn't had this issue. I suspect that there are files being opened, going into some return loop code and then never getting closed. I'm not a C programmer but I see lines like :

fh = open(tmp_filename, O_RDONLY);
      if (fh > 0) {
         read(fh, result, size - 1);
         close(fh);
      }

      /* remove temporary file */
      remove(tmp_filename);

This looks like it opens the file but unless the remove function closes the file, it will remain open even through the file has been deleted. Maybe this isn't the correct behaviour of 'remove' and I am mistaken?

There are also parts like :

 fh = open(textfile, O_RDONLY | O_BINARY);
      if (fh < 0) {
         printf("Message file \"%s\" does not exist.\n", textfile);
         return 1;
      }

      size = (INT) lseek(fh, 0, SEEK_END);
      lseek(fh, 0, SEEK_SET);

      if (size > (INT) (sizeof(text) - 1)) {
         printf("Message file \"%s\" is too long (%zd bytes max).\n", textfile, sizeof(text));
         return 1;
      }

This looks like for the second error, it will complain that the file is too long, return an error message but not close the file and would leave it open. Is this a reasonable avenue to pursue or am I mis-reading the code?   Thanks.

 

Entry   wrong server HTTP status code when login failed, posted by Chris Körner on Thu Oct 21 14:57:14 2021 

Hi,

I am trying to access elog through a python client (https://github.com/paulscherrerinstitute/py_elog) and found a strage strange behavior which may be related server side problem. The python script generates get/post messages via the python requests library. This works fine so far and I can view and post messages. However, if a wrong user/password is provided, the server still returns HTTP status code "200 OK", although login failed. Instead, it should return something like "401 Unauthorized". This behavior later causes problems since the python client thinks login was successful. After experimenting around I think this could be caused by a server side misconfiguration. Any ideas?

I am not sure if this imformation is important: We use LDAP as user/password provider for elog.

    icon2.gif   Re: wrong server HTTP status code when login failed, posted by Chris Körner on Thu Oct 21 15:19:16 2021 

Seems like I've discovered another bug here related to umlauts in my name. :D 

I was submitting this post and forgot to put an icon. Elog seems to have saved a copy of my message, which I could not edit since my username does not match the bugged name saved for this message.

Chris Körner wrote:

Hi,

I am trying to access elog through a python client (https://github.com/paulscherrerinstitute/py_elog) and found a strage strange behavior which may be related server side problem. The python script generates get/post messages via the python requests library. This works fine so far and I can view and post messages. However, if a wrong user/password is provided, the server still returns HTTP status code "200 OK", although login failed. Instead, it should return something like "401 Unauthorized". This behavior later causes problems since the python client thinks login was successful. After experimenting around I think this could be caused by a server side misconfiguration. Any ideas?

I am not sure if this imformation is important: We use LDAP as user/password provider for elog.

 

icon5.gif   wrong server HTTP status code when login failed, posted by Chris Körner on Thu Oct 21 15:17:52 2021 

Hi,

I am trying to access elog through a python client (https://github.com/paulscherrerinstitute/py_elog) and found a strage strange behavior which may be related server side problem. The python script generates get/post messages via the python requests library. This works fine so far and I can view and post messages. However, if a wrong user/password is provided, the server still returns HTTP status code "200 OK", although login failed. Instead, it should return something like "401 Unauthorized". This behavior later causes problems since the python client thinks login was successful. After experimenting around I think this could be caused by a server side misconfiguration. Any ideas?

I am not sure if this imformation is important: We use LDAP as user/password provider for elog.

icon3.gif   How about string constants in config files?, posted by Andreas Luedeke on Thu Oct 21 11:15:20 2021 

My config files are often large and use some shell scripting. In these shell scripting part I interact with files and execute scripts in specific folder.
It would be nice to have the ability to define for example a file path once and reuse it in different parts of the config file. It would help to make config files cleaner. For example:

Constant c_cid = "/usr/local/elog/logbooks/elog-campaignID.default"
Execute new = if ! [ -z "$CampaignID" ] ; then echo "$CampaignID" > $c_cid; fi

Preset CampaignID = $shell( if [ -r $c_cid ] ; then cat $c_cid; else echo None > $c_cid; echo None; fi )

icon4.gif   Redirect in Execute new needs space after ">", posted by Andreas Luedeke on Thu Oct 21 11:00:46 2021 

EDIT: forget the tip below. Instead just call script files: inline scripting in the ELOG config shows very strange behavior. Doing the same in external scripts works reliable.

 

I just spend an hour searching for a problem. To avoid others to spend the hour again, here's a little "special behaviour" of shell execution in ELOG you should know about:

If you want to do redirect to a file in a shell execution, put a space before and after the redirecting. The following does not work:

Execute new = if ! [ -z "$CampaignID" ] ; then echo "$CampaignID" >/usr/local/elog/logbooks/elog-campaign.default ; fi

You will not get an error message, but the file is not created. But if you add a space it will work as expected:

Execute new = if ! [ -z "$CampaignID" ] ; then echo "$CampaignID" > /usr/local/elog/logbooks/elog-campaign.default ; fi

It is not really a bug; if you know about it, then it is not a big deal: hence this entry here. I saw this behavior on a Linux RHEL7 system.

In case you are wondering: I use this to create a default for the field CampaignID, to be used for new entries in combination with a Preset:

Preset CampaignID = $shell( if [ -r /usr/local/elog/logbooks/elog-campaign.default ] ; then cat /usr/local/elog/logbooks/elog-campaign.default;fi )
icon5.gif   How to access PSI Elog data from other web clients , posted by Lin Wang on Wed Oct 13 02:38:34 2021 

We want to develop separate mobile web pages for the web applications deployed at CSNS accelerator, including the PSI Elog.

In Elog, is there RESTful API or HTTP/JSON or HTTP/XML interface for other web clients to access?

Or is there any workaround?

    icon2.gif   Re: How to access PSI Elog data from other web clients , posted by Stefan Ritt on Wed Oct 13 08:17:23 2021 

When elog has been developed, REST did not yet exist. The closest you can get is the RSS API. Just try https://elog.psi.ch/elogs/Forum/elog.rdf and you see the result for this forum. To write to elog, you can use teh HTTP/HTML interface and mimic a browser. See for example elog:69209

Stefan

Lin Wang wrote:

We want to develop separate mobile web pages for the web applications deployed at CSNS accelerator, including the PSI Elog.

In Elog, is there RESTful API or HTTP/JSON or HTTP/XML interface for other web clients to access?

Or is there any workaround?

 

    icon2.gif   Re: How to access PSI Elog data from other web clients , posted by Andreas Luedeke on Thu Oct 21 00:42:42 2021 

There is a python API to access ELOG via HTTP: https://github.com/paulscherrerinstitute/py_elog

Lin Wang wrote:

We want to develop separate mobile web pages for the web applications deployed at CSNS accelerator, including the PSI Elog.

In Elog, is there RESTful API or HTTP/JSON or HTTP/XML interface for other web clients to access?

Or is there any workaround?

 

icon5.gif   Reverse proxy of Elog using Docker and Nginx?, posted by Andrew Wade on Mon Aug 13 21:09:30 2018 

I've been trying to configured a Synology NAS to run my personal elog with a reverse proxy to the outside world.  The best way seems to be running Elog in a Docker instance and then running a separate connected Docker running a nginx-proxy (in this case jwilder/nginx-proxy). This second container manages the certificates to letsencrypt and mapping URL requests to relevant containers so that connection is secured properly.  

It worked great in the initial test. However, I have an issue with authentication.  When I password protect the elog it goes to a login page.  When I give an correct password it loops back to the login page (incidentally when I give an incorrect password it gives an 'Invalid user name or password!' warning).  So I know that its getting the correct password but there is some issue that is resetting or ignoring the authentication.  I am never able to actually get to the protected content. 

Does anyone have any experience in using Nginx to setup a secure reverse proxy? Any insights into why this would mess with the authentication of elog?

 

Side note: I have tried using Apache to do the same and authentication worked fine.  But the pre-canned jwilder/nginx-proxy docker manages all the certificates automatically and seamlessly and allows me to have multiple services running on the same outward facing port on my router.  There is no equivalent (as far as I know) that uses Apache for proxying with letsencrypt​.

    icon2.gif   Re: Reverse proxy of Elog using Docker and Nginx?, posted by Stefan Ritt on Tue Aug 14 06:04:53 2018 

Have you tried the "URL = ..." statement? This determines you elog redirects if you log in. If you reach elog through a proxy, the URL is a different one that if you access it directly. In your case the proxy URL might be necessary.

Stefan

Andrew Wade wrote:

I've been trying to configured a Synology NAS to run my personal elog with a reverse proxy to the outside world.  The best way seems to be running Elog in a Docker instance and then running a separate connected Docker running a nginx-proxy (in this case jwilder/nginx-proxy). This second container manages the certificates to letsencrypt and mapping URL requests to relevant containers so that connection is secured properly.  

It worked great in the initial test. However, I have an issue with authentication.  When I password protect the elog it goes to a login page.  When I give an correct password it loops back to the login page (incidentally when I give an incorrect password it gives an 'Invalid user name or password!' warning).  So I know that its getting the correct password but there is some issue that is resetting or ignoring the authentication.  I am never able to actually get to the protected content. 

Does anyone have any experience in using Nginx to setup a secure reverse proxy? Any insights into why this would mess with the authentication of elog?

 

Side note: I have tried using Apache to do the same and authentication worked fine.  But the pre-canned jwilder/nginx-proxy docker manages all the certificates automatically and seamlessly and allows me to have multiple services running on the same outward facing port on my router.  There is no equivalent (as far as I know) that uses Apache for proxying with letsencrypt​.

 

       icon2.gif   Re: Reverse proxy of Elog using Docker and Nginx?, posted by Andrew Wade on Fri Aug 17 22:07:41 2018 

Yes, I tried setting the URL parameter to the url used by the proxy.  It goes to the correct address but that landing is the login page.

Andrew

Stefan Ritt wrote:

Have you tried the "URL = ..." statement? This determines you elog redirects if you log in. If you reach elog through a proxy, the URL is a different one that if you access it directly. In your case the proxy URL might be necessary.

Stefan

Andrew Wade wrote:

I've been trying to configured a Synology NAS to run my personal elog with a reverse proxy to the outside world.  The best way seems to be running Elog in a Docker instance and then running a separate connected Docker running a nginx-proxy (in this case jwilder/nginx-proxy). This second container manages the certificates to letsencrypt and mapping URL requests to relevant containers so that connection is secured properly.  

It worked great in the initial test. However, I have an issue with authentication.  When I password protect the elog it goes to a login page.  When I give an correct password it loops back to the login page (incidentally when I give an incorrect password it gives an 'Invalid user name or password!' warning).  So I know that its getting the correct password but there is some issue that is resetting or ignoring the authentication.  I am never able to actually get to the protected content. 

Does anyone have any experience in using Nginx to setup a secure reverse proxy? Any insights into why this would mess with the authentication of elog?

 

Side note: I have tried using Apache to do the same and authentication worked fine.  But the pre-canned jwilder/nginx-proxy docker manages all the certificates automatically and seamlessly and allows me to have multiple services running on the same outward facing port on my router.  There is no equivalent (as far as I know) that uses Apache for proxying with letsencrypt​.

 

 

          icon2.gif   Re: Reverse proxy of Elog using Docker and Nginx?, posted by Stefan Ritt on Mon Aug 20 12:42:24 2018 

Actually this forum works through an Apache reverse proxy with authentication and it works, so I suspect that the problem has to do with jwilder/nginx-proxy. Since we don't have this here, all I can propose is that you do debugging yourself. Run elogd with the -v flag so that you see all requests coming from the user through the proxy. Compare the requests through Apache and Nginx to see if any argumets are stripped or mangled. Upon successful login, elog sets a cookie with a unique session-ID (the cookie name is "sid") to the browser. If you proxy strips that cookie, you would land on the login page. Maybe look in that direction.

Stefan

Andrew Wade wrote:

Yes, I tried setting the URL parameter to the url used by the proxy.  It goes to the correct address but that landing is the login page.

Andrew

Stefan Ritt wrote:

Have you tried the "URL = ..." statement? This determines you elog redirects if you log in. If you reach elog through a proxy, the URL is a different one that if you access it directly. In your case the proxy URL might be necessary.

Stefan

Andrew Wade wrote:

I've been trying to configured a Synology NAS to run my personal elog with a reverse proxy to the outside world.  The best way seems to be running Elog in a Docker instance and then running a separate connected Docker running a nginx-proxy (in this case jwilder/nginx-proxy). This second container manages the certificates to letsencrypt and mapping URL requests to relevant containers so that connection is secured properly.  

It worked great in the initial test. However, I have an issue with authentication.  When I password protect the elog it goes to a login page.  When I give an correct password it loops back to the login page (incidentally when I give an incorrect password it gives an 'Invalid user name or password!' warning).  So I know that its getting the correct password but there is some issue that is resetting or ignoring the authentication.  I am never able to actually get to the protected content. 

Does anyone have any experience in using Nginx to setup a secure reverse proxy? Any insights into why this would mess with the authentication of elog?

 

Side note: I have tried using Apache to do the same and authentication worked fine.  But the pre-canned jwilder/nginx-proxy docker manages all the certificates automatically and seamlessly and allows me to have multiple services running on the same outward facing port on my router.  There is no equivalent (as far as I know) that uses Apache for proxying with letsencrypt​.

 

 

 

             icon2.gif   Re: Reverse proxy of Elog using Docker and Nginx?, posted by Andrew Wade on Tue Aug 28 23:38:55 2018 

It does indeed seem to be a cookie stripping issue.  I just need to figure out how to get Nginx to forward these properly.

Thanks for the help.

Stefan Ritt wrote:

Actually this forum works through an Apache reverse proxy with authentication and it works, so I suspect that the problem has to do with jwilder/nginx-proxy. Since we don't have this here, all I can propose is that you do debugging yourself. Run elogd with the -v flag so that you see all requests coming from the user through the proxy. Compare the requests through Apache and Nginx to see if any argumets are stripped or mangled. Upon successful login, elog sets a cookie with a unique session-ID (the cookie name is "sid") to the browser. If you proxy strips that cookie, you would land on the login page. Maybe look in that direction.

Stefan

Andrew Wade wrote:

Yes, I tried setting the URL parameter to the url used by the proxy.  It goes to the correct address but that landing is the login page.

Andrew

Stefan Ritt wrote:

Have you tried the "URL = ..." statement? This determines you elog redirects if you log in. If you reach elog through a proxy, the URL is a different one that if you access it directly. In your case the proxy URL might be necessary.

Stefan

Andrew Wade wrote:

I've been trying to configured a Synology NAS to run my personal elog with a reverse proxy to the outside world.  The best way seems to be running Elog in a Docker instance and then running a separate connected Docker running a nginx-proxy (in this case jwilder/nginx-proxy). This second container manages the certificates to letsencrypt and mapping URL requests to relevant containers so that connection is secured properly.  

It worked great in the initial test. However, I have an issue with authentication.  When I password protect the elog it goes to a login page.  When I give an correct password it loops back to the login page (incidentally when I give an incorrect password it gives an 'Invalid user name or password!' warning).  So I know that its getting the correct password but there is some issue that is resetting or ignoring the authentication.  I am never able to actually get to the protected content. 

Does anyone have any experience in using Nginx to setup a secure reverse proxy? Any insights into why this would mess with the authentication of elog?

 

Side note: I have tried using Apache to do the same and authentication worked fine.  But the pre-canned jwilder/nginx-proxy docker manages all the certificates automatically and seamlessly and allows me to have multiple services running on the same outward facing port on my router.  There is no equivalent (as far as I know) that uses Apache for proxying with letsencrypt​.

 

 

 

 

                icon2.gif   Re: Reverse proxy of Elog using Docker and Nginx?, posted by Bolko Beutner on Wed Sep 15 13:52:59 2021 

I have the same problem -- did you find a solution in using the nginx revese proxy with user login?

Andrew Wade wrote:

It does indeed seem to be a cookie stripping issue.  I just need to figure out how to get Nginx to forward these properly.

Thanks for the help.

Stefan Ritt wrote:

Actually this forum works through an Apache reverse proxy with authentication and it works, so I suspect that the problem has to do with jwilder/nginx-proxy. Since we don't have this here, all I can propose is that you do debugging yourself. Run elogd with the -v flag so that you see all requests coming from the user through the proxy. Compare the requests through Apache and Nginx to see if any argumets are stripped or mangled. Upon successful login, elog sets a cookie with a unique session-ID (the cookie name is "sid") to the browser. If you proxy strips that cookie, you would land on the login page. Maybe look in that direction.

Stefan

Andrew Wade wrote:

Yes, I tried setting the URL parameter to the url used by the proxy.  It goes to the correct address but that landing is the login page.

Andrew

Stefan Ritt wrote:

Have you tried the "URL = ..." statement? This determines you elog redirects if you log in. If you reach elog through a proxy, the URL is a different one that if you access it directly. In your case the proxy URL might be necessary.

Stefan

Andrew Wade wrote:

I've been trying to configured a Synology NAS to run my personal elog with a reverse proxy to the outside world.  The best way seems to be running Elog in a Docker instance and then running a separate connected Docker running a nginx-proxy (in this case jwilder/nginx-proxy). This second container manages the certificates to letsencrypt and mapping URL requests to relevant containers so that connection is secured properly.  

It worked great in the initial test. However, I have an issue with authentication.  When I password protect the elog it goes to a login page.  When I give an correct password it loops back to the login page (incidentally when I give an incorrect password it gives an 'Invalid user name or password!' warning).  So I know that its getting the correct password but there is some issue that is resetting or ignoring the authentication.  I am never able to actually get to the protected content. 

Does anyone have any experience in using Nginx to setup a secure reverse proxy? Any insights into why this would mess with the authentication of elog?

 

Side note: I have tried using Apache to do the same and authentication worked fine.  But the pre-canned jwilder/nginx-proxy docker manages all the certificates automatically and seamlessly and allows me to have multiple services running on the same outward facing port on my router.  There is no equivalent (as far as I know) that uses Apache for proxying with letsencrypt​.

 

 

 

 

 

ELOG V3.1.5-3fb85fa6