Post AOnZNCHAxVVppaiY2y by smortex@mamot.fr
(DIR) More posts by smortex@mamot.fr
(DIR) Post #AOnQv9PbDNfPCrlznM by feld@bikeshed.party
2022-10-21T15:35:05.110050Z
0 likes, 0 repeats
I spent all day yesterday trying to figure out why this application is bugged on deploy but works locally1/ (pretend twitter mode)
(DIR) Post #AOnQyIEgrbo3RVAXMe by feld@bikeshed.party
2022-10-21T15:35:42.252130Z
0 likes, 0 repeats
I even setup the dev environment on the server, ran from my local user account -- no problem. But it wouldn't run correctly in docker.2/
(DIR) Post #AOnR0qtayEPJW7WR3g by feld@bikeshed.party
2022-10-21T15:36:34.701623Z
0 likes, 0 repeats
What I was passing through was an ENV from docker/docker-compose which gives a list of URLs. The software will split them on commas and handle a list of endpoints if you want failover3/
(DIR) Post #AOnR56M3R6RScXrNQm by feld@bikeshed.party
2022-10-21T15:37:20.095380Z
0 likes, 0 repeats
The logs showed the request come in, and then printed an error connecting to the defined endpoint. But the URL was 100% correct in the logs. BUT IT WASNT INITIATING ANY NETWORK TRAFFIC4/
(DIR) Post #AOnRCpsCb7lpoxaJXc by feld@bikeshed.party
2022-10-21T15:38:36.664916Z
0 likes, 0 repeats
I pounded my head on the wall for hours. I tried so many things. I thought I was going crazy. I straced the process (python/gunicorn) and only found these calls. Perplexing![pid 58825] recvfrom(12, "GET /some;path"..., 8192, 0, NULL, NULL) = 486[pid 58825] getsockname(5, {sa_family=AF_INET, sin_port=htons(5000), sin_addr=inet_addr("0.0.0.0")}, [16]) = 0[pid 58825] stat("/root/.netrc", 0x7fff12cd4430) = -1 ENOENT (No such file or directory)[pid 58825] stat("/root/_netrc", 0x7fff12cd4430) = -1 ENOENT (No such file or directory)[pid 58825] getpid() = 7[pid 58825] stat("/tmp/app.log", {st_mode=S_IFREG|0644, st_size=2544, ...}) = 0[pid 58825] stat("/tmp/app.log", {st_mode=S_IFREG|0644, st_size=2544, ...}) = 0[pid 58825] lseek(11, 0, SEEK_END) = 2544[pid 58825] lseek(11, 0, SEEK_CUR) = 2544[pid 58825] write(11, "2022-10-20 23:08:16.048 WARNING "..., 193) = 193[pid 58825] write(1, "2022-10-20 23:08:16.048 WARNING "..., 193) = 193[pid 58825] sendto(12, "HTTP/1.1 500 INTERNAL SERVER ERR"..., 174, 0, NULL, 0) = 1745/
(DIR) Post #AOnRGh2POCwbZpCL8C by feld@bikeshed.party
2022-10-21T15:38:51.509423Z
0 likes, 0 repeats
I pounded my head on the wall for hours. I tried so many things. I thought I was going crazy. I straced the process (python/gunicorn) and only found these calls. Perplexing![pid 58825] recvfrom(12, "GET /some/path"..., 8192, 0, NULL, NULL) = 486[pid 58825] getsockname(5, {sa_family=AF_INET, sin_port=htons(5000), sin_addr=inet_addr("0.0.0.0")}, [16]) = 0[pid 58825] stat("/root/.netrc", 0x7fff12cd4430) = -1 ENOENT (No such file or directory)[pid 58825] stat("/root/_netrc", 0x7fff12cd4430) = -1 ENOENT (No such file or directory)[pid 58825] getpid() = 7[pid 58825] stat("/tmp/app.log", {st_mode=S_IFREG|0644, st_size=2544, ...}) = 0[pid 58825] stat("/tmp/app.log", {st_mode=S_IFREG|0644, st_size=2544, ...}) = 0[pid 58825] lseek(11, 0, SEEK_END) = 2544[pid 58825] lseek(11, 0, SEEK_CUR) = 2544[pid 58825] write(11, "2022-10-20 23:08:16.048 WARNING "..., 193) = 193[pid 58825] write(1, "2022-10-20 23:08:16.048 WARNING "..., 193) = 193[pid 58825] sendto(12, "HTTP/1.1 500 INTERNAL SERVER ERR"..., 174, 0, NULL, 0) = 1745/
(DIR) Post #AOnRLLBw2bgnGls0xs by feld@bikeshed.party
2022-10-21T15:40:18.731858Z
0 likes, 0 repeats
Why was it immediately returning a 500 error? And why can't I peer deeper into this python app? It seems to be skipping a lot of logic. I swear I get better data out of truss on FreeBSD than strace on Linux.So I tried many things. Eventually entering the container manually this morning, setting up the environment, and launching the app.IT WORKED6/
(DIR) Post #AOnRWPnZFjOuk8kkrY by feld@bikeshed.party
2022-10-21T15:42:06.863705Z
0 likes, 0 repeats
What was different? The ENVs were identical. Or seemed to be.And then I had an idea for a change to the docker-compose fileenvironment: - FOO="https://long_complex_url..."toenvironment: - FOO=https://long_complex_url...Let's unquote it for fun? The others are quoted. I've NEVER had an issue with quoted ENVs in docker-compose. I generally prefer to do that.7/
(DIR) Post #AOnRmUVHWkApy8rjHM by feld@bikeshed.party
2022-10-21T15:44:40.954595Z
0 likes, 0 repeats
and holy fuck, it workedIt appears when docker/docker-compose passes ENVs with quoted values it literally keeps the fucking quotes somehow!!!! WHAT THE FUCK?!!!!!THATS NOT HOW ENVs ARE SUPPOSED TO WORKAND WHY IS PYTHON NOT HANDLING THIS QUOTED STRING CORRECTLY?:fuckoff: :fuckoff: :fuckoff: :fuckoff: :fuckoff: :fuckoff: 8/8
(DIR) Post #AOnTz8xD9uQeo75f1M by feld@bikeshed.party
2022-10-21T16:09:37.358570Z
0 likes, 0 repeats
lookee what I foundhttps://faultbucket.ca/2020/08/docker-compose-environment-variables-and-quotes/
(DIR) Post #AOnUequocH4J1XrGJU by feld@bikeshed.party
2022-10-21T16:17:14.882330Z
0 likes, 0 repeats
This docker-compose bug has existed since 2016https://github.com/docker/compose/issues/2854
(DIR) Post #AOnZNCHAxVVppaiY2y by smortex@mamot.fr
2022-10-21T17:08:59Z
0 likes, 0 repeats
@feld All this look like expected YAML behavior 😜.The real question IMO is: why do people rely on YAML to store complex and error-prone data structures?This look like a bad choice from the developers of docker-compose: why would you want to pass environment variables as a sequence of strings (where splitting the name and not handling quotation from the value is the responsibility of the code that process the YAML file) when you can use a simple mapping of variable name / variable value?
(DIR) Post #AOnZNCvaXFq3qvcp2O by feld@bikeshed.party
2022-10-21T17:10:03.038390Z
0 likes, 0 repeats
I know, it's crazy....