Demo Discussion
Forum Config Examples Contributions Vulnerabilities
  Discussion forum about ELOG, Page 30 of 805  Not logged in ELOG logo
ID Date Icon Author Author Email Category OS ELOG Version Subject
  69627   Wed Jan 25 14:36:33 2023 Warning Tamas Galtgal@km3net.deBug reportLinux3.1.4-3Invalid Content-Length in header when running behind a load balancer

I am still struggling to get ELOG running behind a load balancer and hope to get some advice here. As already reported in https://elog.psi.ch/elogs/Forum/69542 I observed an infinite loop of redirects when prompted to log in and using a non-empty password file. Without a password, the service worked as expected. This was with version 3.1.3.

With the new version 3.1.4-3, I get another error: "Invalid Content-Length in header" when I click on "submit" of a new post. Viewing the logbooks works fine. The instance is currently live and running here: https://elog.test.km3net.de but I might change it anytime due to debugging etc.

This is a kind of difficult thing to debug (I spent the whole day and no progress). The only thing I've found was this post: https://techcommunity.microsoft.com/t5/iis-support-blog/invalid-content-length/ba-p/3038724 where it seems that some responses are not RFC conform and were rejected in the load-balancer.

The load balancer I use is HAProxy, the same as in my old setup where I got the infinite redirects, and I can't find any setting which would work. To my understanding, the most basic setup should work just fine. The SSL termination is on the load-balancer side so ELOG doesn't even have to know anything about it. The configuration is below. I am running a single instance, so there is not even replication with session keep-alive via cookies or anything fancy.

I also want to mention that I am runnin around 30 different services behind the load balancer and none of them are having any issues with the SSL termination or the load-balancing itself, that's why assume that something in ELOG is either non-conform or buggy.

Any thoughts? I'd really like to use the same infrastructure for the ELOG service as for every other service (automatic certificate renewal via letsencrypt, load-balancing, easy movement to other nodes, SSL termination etc.), to minimise the complexity of our Docker Swarm system.

backend be_elog.km3net.de
    mode http
    server-template km3net-elog- 1 km3net-elog_elog:8080 check resolvers docker init-addr libc,none

 

Btw. I am running ELOG with -v but I don't see any error whatsoever in the logs:

km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET /demo/ HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "ios_specific_templates_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_anonymous_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_group_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_group_trait"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_trait"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "rl_user_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Received unknown cookie "logged_out_marketing_header_id"
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 3437 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET / HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 120 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET /demo/ HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 3518 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET / HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 120 bytes
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | GET /demo/ HTTP/1.1
km3net-elog_elog.1.fm8i1eia9l9t@ecap-s021    | Returned 3518 bytes

Attachment 1: Screenshot_2023-01-25_at_14.46.05.png
Screenshot_2023-01-25_at_14.46.05.png
  69626   Tue Jan 24 11:31:59 2023 Reply Edmund Blomleyedmund.blomley@kit.eduRequestAll-Re: Wikipedia Article deleted

If I understand it correctly I think it has to be submitted for review with the blue button on that page, just not sure if that should come from your side or someone else

Stefan Ritt wrote:

I added some more references, that's about all I can do. Not sure if that is enough.

Stefan

Edmund Blomley wrote:

It was now moved to the Draft space (which I did not even now existed so far): https://en.wikipedia.org/wiki/Draft:ELOG

Sebastian Schenk wrote:

I have requested an undeletion of the article. The article was deleted  "PROD", which means that someone tagged it. And if noone removes the tag, it could be deleted.

I could revive the article. So in the future, One should have an eye on it and maybe update the current version of the software.

If there iy an paper on the elog, maybe it could be cited for more creditability.

Andreas Luedeke wrote:

It looks to me like only an author of an article can contradict a deletion. I did not find a single method to even comment on the deletion.
I am not an Wikipedia expert, can anyone suggest on how to push for the article to be restored? Or do we just write it again, until people stop deleting it?

Stefan Ritt wrote:

I agree. I ahead ;-) I think it is not a good idea if the ELOG author pushes on that, but better someone else.

Best,
Stefan

Sebastian Schenk wrote:

Hello,

I noticed the wikipedia article of the ELOG got deleted in November 2021.
With the reason: "Poorly sourced article, and I was not able to find good sources myself."

I could access the old article through web.archive.org, but for the project it would be good, if the article got revived.

 

 

 

 

 

 

  69625   Mon Jan 23 22:24:23 2023 Reply Stefan Rittstefan.ritt@psi.chRequestAll-Re: Wikipedia Article deleted

I added some more references, that's about all I can do. Not sure if that is enough.

Stefan

Edmund Blomley wrote:

It was now moved to the Draft space (which I did not even now existed so far): https://en.wikipedia.org/wiki/Draft:ELOG

Sebastian Schenk wrote:

I have requested an undeletion of the article. The article was deleted  "PROD", which means that someone tagged it. And if noone removes the tag, it could be deleted.

I could revive the article. So in the future, One should have an eye on it and maybe update the current version of the software.

If there iy an paper on the elog, maybe it could be cited for more creditability.

Andreas Luedeke wrote:

It looks to me like only an author of an article can contradict a deletion. I did not find a single method to even comment on the deletion.
I am not an Wikipedia expert, can anyone suggest on how to push for the article to be restored? Or do we just write it again, until people stop deleting it?

Stefan Ritt wrote:

I agree. I ahead ;-) I think it is not a good idea if the ELOG author pushes on that, but better someone else.

Best,
Stefan

Sebastian Schenk wrote:

Hello,

I noticed the wikipedia article of the ELOG got deleted in November 2021.
With the reason: "Poorly sourced article, and I was not able to find good sources myself."

I could access the old article through web.archive.org, but for the project it would be good, if the article got revived.

 

 

 

 

 

  69624   Mon Jan 23 21:21:56 2023 Reply Edmund Blomleyedmund.blomley@kit.eduRequestAll-Re: Wikipedia Article deleted

It was now moved to the Draft space (which I did not even now existed so far): https://en.wikipedia.org/wiki/Draft:ELOG

Sebastian Schenk wrote:

I have requested an undeletion of the article. The article was deleted  "PROD", which means that someone tagged it. And if noone removes the tag, it could be deleted.

I could revive the article. So in the future, One should have an eye on it and maybe update the current version of the software.

If there iy an paper on the elog, maybe it could be cited for more creditability.

Andreas Luedeke wrote:

It looks to me like only an author of an article can contradict a deletion. I did not find a single method to even comment on the deletion.
I am not an Wikipedia expert, can anyone suggest on how to push for the article to be restored? Or do we just write it again, until people stop deleting it?

Stefan Ritt wrote:

I agree. I ahead ;-) I think it is not a good idea if the ELOG author pushes on that, but better someone else.

Best,
Stefan

Sebastian Schenk wrote:

Hello,

I noticed the wikipedia article of the ELOG got deleted in November 2021.
With the reason: "Poorly sourced article, and I was not able to find good sources myself."

I could access the old article through web.archive.org, but for the project it would be good, if the article got revived.

 

 

 

 

  69623   Fri Jan 20 14:11:52 2023 Question Tamas Galtgal@km3net.deQuestionLinux3.1.3Re: Too many redirects when running behind load balancer?

The issue is still present and now it's quite urgent to move this last service into the Swarm. Does anyone maybe have an idea what's wrong? To sum up: if there is a non-empty password file, the login page chokes in an infinite loop of redirects. I am using the same HAProxy load balancer configuration as for all the other services (running Apache, NGINX, GitLab, XWiki, etc.):

backend be_elog.km3net.de
    mode http

    option forwardfor except 127.0.0.1
    http-request add-header X-Forwarded-Proto https if { ssl_fc }

    server-template km3net-elog- 1 km3net-elog_elog:8080 check resolvers docker init-addr libc,none

Tamas Gal wrote:

Yes, I used the empty `passwd` file from example. When I then click on one of the logbooks, I get to the page where I can register a user (see attached screenshot). After clickin on "Save" for the user registration, I again get the redirect error. Once there is a registered user (i.e. a non-empty password file) the redirect issue is persistent. Any idea where the problem might be? I just emptied the password file again, so you can have a one-shot, if you like.

Btw. I have SSL termination in the load balancer, so ELOG does not need to do any SSL related things (the swarm is in a locally isolated network, so all internal communication between the load balancer and the swarm machines are safe). Maybe that's the issue? On the other hand, the main page loads fine and uses SSL termination too, so I don't know, maybe there is logic behind the authentication which collides with the SSL termination.

Stefan Ritt wrote:

Yes I see the redirects. You say with the example logbook it works, right? Is it the password protection which triggers the problem or anything else? Does it work if you take out the password protection? The key is to identify which setting in your config file triggers the problem, so you can bracket the problem down between the example logbook and your logbook definition.

 

 

  Draft   Fri Jan 20 14:08:25 2023 Question Tamas Galhe iQuestionLinux3.1.3Re: Too many redirects when running behind load balancer?

The issue is still present and now it's quite urgent to move this last service into the Swarm. Does anyone maybe have an idea what's wrong? To sum up: if there is a non-empty password file, the login page chokes in an infinite loop of redirects. I am using the same HAProxy load balancer configuration as for all the other services (running Apache, NGINX, GitLab, XWiki, etc.):

backend be_elog.km3net.de
    mode http

    option forwardfor except 127.0.0.1
    http-request add-header X-Forwarded-Proto https if { ssl_fc }

    server-template km3net-elog- 1 km3net-elog_elog:8080 check resolvers docker init-addr libc,none

Tamas Gal wrote:

Yes, I used the empty `passwd` file from example. When I then click on one of the logbooks, I get to the page where I can register a user (see attached screenshot). After clickin on "Save" for the user registration, I again get the redirect error. Once there is a registered user (i.e. a non-empty password file) the redirect issue is persistent. Any idea where the problem might be? I just emptied the password file again, so you can have a one-shot, if you like.

Btw. I have SSL termination in the load balancer, so ELOG does not need to do any SSL related things (the swarm is in a locally isolated network, so all internal communication between the load balancer and the swarm machines are safe). Maybe that's the issue? On the other hand, the main page loads fine and uses SSL termination too, so I don't know, maybe there is logic behind the authentication which collides with the SSL termination.

Stefan Ritt wrote:

Yes I see the redirects. You say with the example logbook it works, right? Is it the password protection which triggers the problem or anything else? Does it work if you take out the password protection? The key is to identify which setting in your config file triggers the problem, so you can bracket the problem down between the example logbook and your logbook definition.

 

 

  69621   Fri Jan 20 13:12:48 2023 Reply Sebastian Schenksebastian.schenk@physik.uni-halle.deRequestAll-Re: Wikipedia Article deleted

I have requested an undeletion of the article. The article was deleted  "PROD", which means that someone tagged it. And if noone removes the tag, it could be deleted.

I could revive the article. So in the future, One should have an eye on it and maybe update the current version of the software.

If there iy an paper on the elog, maybe it could be cited for more creditability.

Andreas Luedeke wrote:

It looks to me like only an author of an article can contradict a deletion. I did not find a single method to even comment on the deletion.
I am not an Wikipedia expert, can anyone suggest on how to push for the article to be restored? Or do we just write it again, until people stop deleting it?

Stefan Ritt wrote:

I agree. I ahead ;-) I think it is not a good idea if the ELOG author pushes on that, but better someone else.

Best,
Stefan

Sebastian Schenk wrote:

Hello,

I noticed the wikipedia article of the ELOG got deleted in November 2021.
With the reason: "Poorly sourced article, and I was not able to find good sources myself."

I could access the old article through web.archive.org, but for the project it would be good, if the article got revived.

 

 

 

  69620   Fri Jan 20 10:25:23 2023 Reply Andreas Luedekeandreas.luedeke@psi.chRequestAll-Re: Wikipedia Article deleted

It looks to me like only an author of an article can contradict a deletion. I did not find a single method to even comment on the deletion.
I am not an Wikipedia expert, can anyone suggest on how to push for the article to be restored? Or do we just write it again, until people stop deleting it?

Stefan Ritt wrote:

I agree. I ahead ;-) I think it is not a good idea if the ELOG author pushes on that, but better someone else.

Best,
Stefan

Sebastian Schenk wrote:

Hello,

I noticed the wikipedia article of the ELOG got deleted in November 2021.
With the reason: "Poorly sourced article, and I was not able to find good sources myself."

I could access the old article through web.archive.org, but for the project it would be good, if the article got revived.

 

 

ELOG V3.1.5-3fb85fa6