Hello all,
I’m testing HTTP PUTs against the eos-nginx proxy, and I’m using the internal redirect functionality to hide the redirect from the client, the obvious issues with webdav and other clients following redirects. I’ve checked against the EOS source, uboxed and of course asked Google.
Another set of eyes to point out my error would be appreciated, as I’m totally missing where the fault lies.
I know the eos-nginx instance has a patch for redirecting on PUT, and it works if I use curl with follow redirects, and talk to the MGM directly.
If I talk through the Nginx instance, I’m getting 502 bad gateways, as shown below. GETs, HEADs, and PROPFINDs work properly, as do writes against the MGM directly using the same method (with -L turned on). A curl output is below:
curl -v -T testfile http://mgmproxy:10000/eos/test/testfile
* About to connect() to mgmproxy port 10000 (#0)
* Trying 10.42.4.16...
* Connected to mgmproxy (10.42.4.16) port 10000 (#0)
> PUT /eos/test/testfile HTTP/1.1
> User-Agent: curl/7.29.0
> Host: mgmproxy:10000
> Accept: */*
> Content-Length: 352
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 502 Bad Gateway
< Server: nginx/1.12.2
< Date: Wed, 12 Aug 2020 04:32:08 GMT
< Content-Type: text/html
< Content-Length: 173
< Connection: keep-alive
<
<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.12.2</center>
</body>
</html>
* Connection #0 to host mgmproxy left intact
Nginx’s internal_redirect.error.log reports:
2020/08/12 05:50:56 [error] 5613#0: *39 upstream prematurely closed connection while reading response header from upstream, client: 10.42.4.16, server: mgmproxy, request: "PUT /eos/test/testfile HTTP/1.1", upstream: "http://10.10.0.174:8001/eos/test/testfile?encURI=....mgm.logid%3Dca72a478-dc5f-11ea-b7cb-b6e839c4465e%26mgm.replicaindex%3D0%26mgm.replicahead%3D0%26mgm.etag%3D%223861444034560:00000000%22%26mgm.id%3D00003831", host: "mgmproxy:10000"
The MGM xrdlog.mgm and the Http.log report no errors, as they all accept the file. The error is in the Nginx redirected PUT.
The running nginx config is relatively simple, and taken from the eos-nginx template.
user daemon;
worker_processes 1;
error_log /var/log/nginx/error.log info;
pid /var/run/nginx.pid;
env OPENSSL_ALLOW_PROXY_CERTS;
events {
worker_connections 1024;
}
http {
default_type application/octet-stream;
upstream eos_mgm_backend {
ip_hash;
server eos-mgm-0.mgm.test.svc.cluster.local:8000;
keepalive 128;
}
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log
main;
sendfile on;
keepalive_timeout 65s;
client_max_body_size 1000000m;
server {
server_name mgmproxy;
listen 10000;
location / {
add_header content-type "text/html;charset=UTF-8";
proxy_pass http://eos_mgm_backend;
proxy_set_header Host $host;
proxy_set_header Auth-Type "unix";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Remote-User "testuser";
client_body_buffer_size 128k;
proxy_connect_timeout 90s;
proxy_send_timeout 90s;
proxy_read_timeout 90s;
proxy_buffer_size 256k;
proxy_buffers 8 256k;
proxy_busy_buffers_size 512k;
proxy_temp_file_write_size 512k;
}
location ~* ^/internal_redirect/(.*):(.*?)/(.*) {
internal;
resolver 10.43.0.10;
access_log /var/log/nginx/internal_redirect.access.log main;
error_log /var/log/nginx/internal_redirect.error.log;
set $download_host $1;
set $download_port $2;
set $download_uri $3;
set $download_url http://$download_host:$download_port/$download_uri?$args;
proxy_set_header Host $download_host:$download_port;
proxy_set_header Authorization '';
proxy_max_temp_file_size 0;
proxy_pass $download_url;
}
}
}
Is there something I’m missing, as PUTs certainly don’t work with OC sync presently, and it’s not working with webdav clients or redirect-less curl PUTs?