In one of my Linux node, I was not able to use curl command if executed by a particular user(“root”). Strange enough, it was working when executed using some other user. If you face similar issue try to follow the following strategy to debug the issue:
When executed with non-root User:
curl google.com -I
HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Tue, 15 Mar 2022 20:22:52 GMT
Expires: Thu, 14 Apr 2022 20:22:52 GMT
Cache-Control: public, max-age=2592000
Server: gws
Content-Length: 219
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
When executed by “root” User:
I was wondering why curl is trying to connect to loopback interface? Why 127.0.0.1 , why port 8001 ??
root@cloudvm:~# curl google.com
curl: (7) Failed to connect to 127.0.0.1 port 8001: Connection refused
strace to rescue
I executed curl command by root and non-root user with strace. Everything was clear after checking the output.
ps@cloudvm#strace -e trace=connect,read,openat curl google.com
openat(AT_FDCWD, "/root/.curlrc", O_RDONLY) = 3
read(3, "proxy = 127.0.0.1:8001\n", 4096) = 23
read(3, "", 4096) = 0
connect(5, {sa_family=AF_INET, sin_port=htons(8001), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
openat(AT_FDCWD, "/usr/share/locale/C.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/C.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/C/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/C.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/C.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale-langpack/C/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
curl: (7) Failed to connect to 127.0.0.1 port 8001: Connection refused
+++ exited with 7 +++
Culprit:
Based on the above strace result, there was a rogue .curlrc file present in root user’s home directory. This file was getting loaded everytime when root user was running root user.
root@cloudvm:~# ls -lrt .curlrc
-rw-r--r-- 1 root root 23 Mar 15 20:00 .curlrc
root@cloudvm:~# cat .curlrc
proxy = 127.0.0.1:8001
root@cloudvm:~#
The idea for this post is to save time of anyone like me who faces this issue. Note that the this post is applicable to any user having .curlrc. Additionally strace command give a very good idea about what syscalls are being made by curl.