Demo Discussion
Forum Config Examples Contributions Vulnerabilities
  Discussion forum about ELOG, Page 173 of 807  Not logged in ELOG logo
ID Date Icon Author Author Email Category OS ELOG Versiondown Subject
  69629   Wed Jan 25 18:41:27 2023 Question Tamas Galtgal@km3net.deQuestionLinux3.1.4-3Upload via command line through Apache reverse proxy and load balancer

After fiddling around I managed to get ELOG working behind the load balancer HAProxy by stacking ELOG together with an Apache reverse proxy in a Docker stack. I am currently pretty convinced that something with the HTTP communication is somehow faulty in ELOG and Apache is more forgiving than HAProxy, since the configuration is the same as without Apache. So putting ELOG behind an Apache and then Apache behind the HAProxy is working.

For the sake of completeness, here is the HAProxy configuration:

backend be_elog.km3net.de
    mode http
    server-template km3net-elog- 1 km3net-elog_apache:80 check resolvers docker init-addr libc,none

and here is the Apache httpd.conf:

Listen 80

LoadModule mpm_event_module modules/mod_mpm_event.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule reqtimeout_module modules/mod_reqtimeout.so
LoadModule filter_module modules/mod_filter.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule headers_module modules/mod_headers.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
<IfModule !mpm_prefork_module>
        #LoadModule cgid_module modules/mod_cgid.so
</IfModule>
<IfModule mpm_prefork_module>
        #LoadModule cgi_module modules/mod_cgi.so
</IfModule>
LoadModule dir_module modules/mod_dir.so
LoadModule alias_module modules/mod_alias.so
LoadModule rewrite_module modules/mod_rewrite.so


ServerAdmin email
ServerName elog.test.km3net.de

ErrorLog /proc/self/fd/2

LogLevel warn

<IfModule log_config_module>
    #
    # The following directives define some format nicknames for use with
    # a CustomLog directive (see below).
    #
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common

    <IfModule logio_module>
      # You need to enable mod_logio.c to use %I and %O
      LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
    </IfModule>

    #
    # The location and format of the access logfile (Common Logfile Format).
    # If you do not define any access logfiles within a <VirtualHost>
    # container, they will be logged here.  Contrariwise, if you *do*
    # define per-<VirtualHost> access logfiles, transactions will be
    # logged therein and *not* in this file.
    #
    CustomLog /proc/self/fd/1 common

    #
    # If you prefer a logfile with access, agent, and referer information
    # (Combined Logfile Format) you can use the following directive.
    #
    #CustomLog "logs/access_log" combined
</IfModule>

<IfModule headers_module>
    #
    # Avoid passing HTTP_PROXY environment to CGI's on this or any proxied
    # backend servers which have lingering "httpoxy" defects.
    # 'Proxy' request header is undefined by the IETF, not listed by IANA
    #
    RequestHeader unset Proxy early
</IfModule>

 

<VirtualHost *:80>
    ServerName elog.test.km3net.de
    #ProxyPreserveHost On
    ProxyPass / http://elog:8080/
    ProxyPassReverse / http://elog:8080/

    RewriteEngine On
    RewriteCond %{HTTP:Upgrade} =websocket [NC]
    RewriteRule /(.*)           ws://elog:8080/$1 [P,L]
    RewriteCond %{HTTP:Upgrade} !=websocket [NC]
    RewriteRule /(.*)           http://elog:8080/$1 [P,L]

    ErrorLog /apache/error.log
    CustomLog /apache/access.log combined
    TransferLog /apache/transfer.log
</VirtualHost>

Long story short: I am still not able to upload anything from the command line. So something like

elog -v -h elog.test.km3net.de -p 443 -l "Individual Logbooks" -v -m elog_test.txt  -n 0 -a author="Whoever" -a Subject="Upload Test" -u USER PWD -s

gives this:

root@b9db27a421e1:/# elog -v -h elog.test.km3net.de -p 443 -l "Individual Logbooks" -v -m elog_test.txt  -n 0 -a author="Whoever" -a Subject="Upload Test" -u USER PWD -s
Successfully connected to host elog.test.km3net.de, port 443
Possibly invalid certificate, continue on your own risk!
Request sent to host:
POST /Individual+Logbooks/ HTTP/1.0
Content-Type: multipart/form-data; boundary=---------------------------66D92EF0673838014927FA6E
Host: elog.test.km3net.de:443
User-Agent: ELOG
Content-Length: 977


Content sent to host:
---------------------------66D92EF0673838014927FA6E
Content-Disposition: form-data; name="cmd"

Submit
---------------------------66D92EF0673838014927FA6E
Content-Disposition: form-data; name="unm"

USER
---------------------------66D92EF0673838014927FA6E
Content-Disposition: form-data; name="upwd"

PWD_HASH
---------------------------66D92EF0673838014927FA6E
Content-Disposition: form-data; name="exp"

Individual Logbooks
---------------------------66D92EF0673838014927FA6E
Content-Disposition: form-data; name="encoding"

ELCode
---------------------------66D92EF0673838014927FA6E
Content-Disposition: form-data; name="author"

Whoever
---------------------------66D92EF0673838014927FA6E
Content-Disposition: form-data; name="Subject"

Upload Test
---------------------------66D92EF0673838014927FA6E
Content-Disposition: form-data; name="Text"

foo

---------------------------66D92EF0673838014927FA6E

Response received:
HTTP/1.1 503 Service Unavailable
content-length: 107
cache-control: no-cache
content-type: text/html
connection: close

<html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
</body></html>

Error transmitting message

Is this command line interface even able to communicate through a(n Apache) reverse proxy or does it need to communicate with elogd directly?

  69630   Wed Jan 25 19:51:29 2023 Reply Tamas Galtgal@km3net.deBug reportLinux3.1.4-3Re: Invalid Content-Length in header when running behind a load balancer

I put the ELOG service behind an Apache reverse proxy which is now sitting behind the HAProxy. It works like this, but it's just a workaround. I am pretty sure that ELOG has problems to communicate with HAProxy correctly and it seems that Apache is more forgiving. So that the chain HAProxy -> Apache -> ELOG and vice versa is working.

If anyone manages to figure out what's wrong, I am happy to get rid of the extra reverse proxy layer!

Tamas Gal wrote:

I am still struggling to get ELOG running behind a load balancer and hope to get some advice here. As already reported in https://elog.psi.ch/elogs/Forum/69542 I observed an infinite loop of redirects when prompted to log in and using a non-empty password file. Without a password, the service worked as expected. This was with version 3.1.3.

With the new version 3.1.4-3, I get another error: "Invalid Content-Length in header" when I click on "submit" of a new post. Viewing the logbooks works fine. The instance is currently live and running here: https://elog.test.km3net.de but I might change it anytime due to debugging etc.

This is a kind of difficult thing to debug (I spent the whole day and no progress). The only thing I've found was this post: https://techcommunity.microsoft.com/t5/iis-support-blog/invalid-content-length/ba-p/3038724 where it seems that some responses are not RFC conform and were rejected in the load-balancer.

The load balancer I use is HAProxy, the same as in my old setup where I got the infinite redirects, and I can't find any setting which would work. To my understanding, the most basic setup should work just fine. The SSL termination is on the load-balancer side so ELOG doesn't even have to know anything about it. The configuration is below. I am running a single instance, so there is not even replication with session keep-alive via cookies or anything fancy.

I also want to mention that I am runnin around 30 different services behind the load balancer and none of them are having any issues with the SSL termination or the load-balancing itself, that's why assume that something in ELOG is either non-conform or buggy.

Any thoughts? I'd really like to use the same infrastructure for the ELOG service as for every other service (automatic certificate renewal via letsencrypt, load-balancing, easy movement to other nodes, SSL termination etc.), to minimise the complexity of our Docker Swarm system.

backend be_elog.km3net.de
    mode http
    server-template km3net-elog- 1 km3net-elog_elog:8080 check resolvers docker init-addr libc,none

 

Btw. I am running ELOG with -v but I don't see any error whatsoever in the logs:

km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET /demo/ HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "ios_specific_templates_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_anonymous_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_group_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_group_trait"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_trait"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_user_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "logged_out_marketing_header_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 3437 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET / HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 120 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET /demo/ HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 3518 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET / HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 120 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET /demo/ HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 3518 bytes

 

  69808   Mon Jul 22 16:30:04 2024 Reply Andréandre.althaus@tu-dortmund.deBug reportLinux3.1.4-3Re: Invalid Content-Length in header when running behind a load balancer

I have the same problem. I found a temporary solution: https://elog.psi.ch/elogs/Forum/69807

Tamas Gal wrote:

I put the ELOG service behind an Apache reverse proxy which is now sitting behind the HAProxy. It works like this, but it's just a workaround. I am pretty sure that ELOG has problems to communicate with HAProxy correctly and it seems that Apache is more forgiving. So that the chain HAProxy -> Apache -> ELOG and vice versa is working.

If anyone manages to figure out what's wrong, I am happy to get rid of the extra reverse proxy layer!

Tamas Gal wrote:

I am still struggling to get ELOG running behind a load balancer and hope to get some advice here. As already reported in https://elog.psi.ch/elogs/Forum/69542 I observed an infinite loop of redirects when prompted to log in and using a non-empty password file. Without a password, the service worked as expected. This was with version 3.1.3.

With the new version 3.1.4-3, I get another error: "Invalid Content-Length in header" when I click on "submit" of a new post. Viewing the logbooks works fine. The instance is currently live and running here: https://elog.test.km3net.de but I might change it anytime due to debugging etc.

This is a kind of difficult thing to debug (I spent the whole day and no progress). The only thing I've found was this post: https://techcommunity.microsoft.com/t5/iis-support-blog/invalid-content-length/ba-p/3038724 where it seems that some responses are not RFC conform and were rejected in the load-balancer.

The load balancer I use is HAProxy, the same as in my old setup where I got the infinite redirects, and I can't find any setting which would work. To my understanding, the most basic setup should work just fine. The SSL termination is on the load-balancer side so ELOG doesn't even have to know anything about it. The configuration is below. I am running a single instance, so there is not even replication with session keep-alive via cookies or anything fancy.

I also want to mention that I am runnin around 30 different services behind the load balancer and none of them are having any issues with the SSL termination or the load-balancing itself, that's why assume that something in ELOG is either non-conform or buggy.

Any thoughts? I'd really like to use the same infrastructure for the ELOG service as for every other service (automatic certificate renewal via letsencrypt, load-balancing, easy movement to other nodes, SSL termination etc.), to minimise the complexity of our Docker Swarm system.

backend be_elog.km3net.de
    mode http
    server-template km3net-elog- 1 km3net-elog_elog:8080 check resolvers docker init-addr libc,none

 

Btw. I am running ELOG with -v but I don't see any error whatsoever in the logs:

km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET /demo/ HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "ios_specific_templates_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_anonymous_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_group_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_group_trait"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_trait"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_user_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "logged_out_marketing_header_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 3437 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET / HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 120 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET /demo/ HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 3518 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET / HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 120 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET /demo/ HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 3518 bytes

 

 

  69818   Wed Jul 31 14:22:52 2024 Reply Stefan Rittstefan.ritt@psi.chBug reportLinux3.1.4-3Re: Invalid Content-Length in header when running behind a load balancer

I changed elog to interprete the content-length header case in-sensitive and committed the change. Can you try again?

Stefan

André wrote:

I have the same problem. I found a temporary solution: https://elog.psi.ch/elogs/Forum/69807

Tamas Gal wrote:

I put the ELOG service behind an Apache reverse proxy which is now sitting behind the HAProxy. It works like this, but it's just a workaround. I am pretty sure that ELOG has problems to communicate with HAProxy correctly and it seems that Apache is more forgiving. So that the chain HAProxy -> Apache -> ELOG and vice versa is working.

If anyone manages to figure out what's wrong, I am happy to get rid of the extra reverse proxy layer!

Tamas Gal wrote:

I am still struggling to get ELOG running behind a load balancer and hope to get some advice here. As already reported in https://elog.psi.ch/elogs/Forum/69542 I observed an infinite loop of redirects when prompted to log in and using a non-empty password file. Without a password, the service worked as expected. This was with version 3.1.3.

With the new version 3.1.4-3, I get another error: "Invalid Content-Length in header" when I click on "submit" of a new post. Viewing the logbooks works fine. The instance is currently live and running here: https://elog.test.km3net.de but I might change it anytime due to debugging etc.

This is a kind of difficult thing to debug (I spent the whole day and no progress). The only thing I've found was this post: https://techcommunity.microsoft.com/t5/iis-support-blog/invalid-content-length/ba-p/3038724 where it seems that some responses are not RFC conform and were rejected in the load-balancer.

The load balancer I use is HAProxy, the same as in my old setup where I got the infinite redirects, and I can't find any setting which would work. To my understanding, the most basic setup should work just fine. The SSL termination is on the load-balancer side so ELOG doesn't even have to know anything about it. The configuration is below. I am running a single instance, so there is not even replication with session keep-alive via cookies or anything fancy.

I also want to mention that I am runnin around 30 different services behind the load balancer and none of them are having any issues with the SSL termination or the load-balancing itself, that's why assume that something in ELOG is either non-conform or buggy.

Any thoughts? I'd really like to use the same infrastructure for the ELOG service as for every other service (automatic certificate renewal via letsencrypt, load-balancing, easy movement to other nodes, SSL termination etc.), to minimise the complexity of our Docker Swarm system.

backend be_elog.km3net.de
    mode http
    server-template km3net-elog- 1 km3net-elog_elog:8080 check resolvers docker init-addr libc,none

 

Btw. I am running ELOG with -v but I don't see any error whatsoever in the logs:

km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET /demo/ HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "ios_specific_templates_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_anonymous_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_group_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_group_trait"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_trait"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_user_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "logged_out_marketing_header_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 3437 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET / HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 120 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET /demo/ HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 3518 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET / HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 120 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET /demo/ HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 3518 bytes

 

 

 

  69399   Thu Oct 21 11:00:46 2021 Warning Andreas Luedekeandreas.luedeke@psi.chBug reportLinux3.1.4-2e1708bRedirect in Execute new needs space after ">"

EDIT: forget the tip below. Instead just call script files: inline scripting in the ELOG config shows very strange behavior. Doing the same in external scripts works reliable.

 

I just spend an hour searching for a problem. To avoid others to spend the hour again, here's a little "special behaviour" of shell execution in ELOG you should know about:

If you want to do redirect to a file in a shell execution, put a space before and after the redirecting. The following does not work:

Execute new = if ! [ -z "$CampaignID" ] ; then echo "$CampaignID" >/usr/local/elog/logbooks/elog-campaign.default ; fi

You will not get an error message, but the file is not created. But if you add a space it will work as expected:

Execute new = if ! [ -z "$CampaignID" ] ; then echo "$CampaignID" > /usr/local/elog/logbooks/elog-campaign.default ; fi

It is not really a bug; if you know about it, then it is not a big deal: hence this entry here. I saw this behavior on a Linux RHEL7 system.

In case you are wondering: I use this to create a default for the field CampaignID, to be used for new entries in combination with a Preset:

Preset CampaignID = $shell( if [ -r /usr/local/elog/logbooks/elog-campaign.default ] ; then cat /usr/local/elog/logbooks/elog-campaign.default;fi )
  69153   Fri May 29 09:27:32 2020 Entry Jan Just Keijserjanjust@nikhef.nlBug reportLinux3.1.4-2"New User" option does not work when Authentication=Webserver

Our setup uses "Authentication=Webserver" + no automatic user registration. Thus, logbook admins should add a user by clicking "Config"  and then "New user". However, no matter what they fill in in the "new user " dialog, as soon as they hit "Save" an error pops up saying that their username (the admin one, not the new one) already exists. I found the following code:

int save_user_config(LOGBOOK * lbs, char *user, BOOL new_user)
{
   char file_name[256], str[256], *pl, user_enc[256], new_pwd[80], new_pwd2[80], smtp_host[256],
       email_addr[256], mail_from[256], mail_from_name[256], subject[256], mail_text[2000], str2[256],
       admin_user[80], url[256], error[2000], sid[32];
   int i, self_register, code, first_user;
   PMXML_NODE node, subnode, npwd; 

   /* if we outsourced the authentication, use external username */
   getcfg(lbs->name, "Authentication", str, sizeof(str));
   if (stristr(str, "Webserver")) {
      /* do not allow HTML in user name */
      strencode2(user_enc, http_user, sizeof(user_enc));
   } else {
      strencode2(user_enc, user, sizeof(user_enc));
   }

 

which seems to be the culprit:  the admin user is logged using his/her Webserver (http_user) credentials and this overrides anything that he/she might fill in.  If I remove the "Authentication" check then I can create a new user without problems.  So, how to fix this? should the "Authentication=Webserver" check be extended with a self/auto registration check?

 

  69154   Fri Jun 5 03:49:20 2020 Entry Hisataka YOSHIDAhisataka@rcnp.osaka-u.ac.jpBug reportLinux3.1.4-2SSL does not work

Hello.

I installed the latest elog (3.1.4-2) in CentOS 7, and it is working well without SSL.
When I enalbled SSL option (SSL = 1) in the "elogd.cfg", and tried to start the elogd, the message below was shown and failed to run.

SSL support not compiled into elogd

If I switched the elog to older one (3.1.4-1), I could successeed to run the elogd with SSL option.
Is there any other option required in the latest elog to run with SSL? Or is this bug in the latest version?

Thank you,
Hisataka YOSHIDA

  69155   Fri Jun 5 13:24:01 2020 Reply Stefan Rittstefan.ritt@psi.chBug reportLinux3.1.4-2Re: SSL does not work

When you compile elog from the soruces, you need the OpenSSL library to be installed. The CMake build process will then find it and include it in the compile process. When you use the "make" build process, you have to make sure that SSL is enabled there:

USE_SSL = 1

To install the OpenSSL library, you can do on most systems something like "sudo yum install openssl-dev" or "sudo apt-get install openssl-dev"

/Stefan

 

Hisataka YOSHIDA wrote:

Hello.

I installed the latest elog (3.1.4-2) in CentOS 7, and it is working well without SSL.
When I enalbled SSL option (SSL = 1) in the "elogd.cfg", and tried to start the elogd, the message below was shown and failed to run.

SSL support not compiled into elogd

If I switched the elog to older one (3.1.4-1), I could successeed to run the elogd with SSL option.
Is there any other option required in the latest elog to run with SSL? Or is this bug in the latest version?

Thank you,
Hisataka YOSHIDA

 

ELOG V3.1.5-3fb85fa6