Ticket #1713 (assigned defect)

Opened 3 months ago

Last modified 2 weeks ago

No access to Toolbox when server.socket_host not set

Reported by: Chris Arndt Assigned to: Chris Arndt (accepted)
Priority: high Milestone: 1.0.x bugfix
Component: Toolbox Version: 1.0.4.2
Severity: major Keywords:
Cc:

Description

When you run tg-admin tooolbox and server.socket_host is not set in the app's config, the browser will display the following error message on opening the toolbox page: 

No access for ::ffff:127.0.0.1

By default only localhost (127.0.0.1) has access to the Toolbox

You can provide access to your client by passing your host address to Toolbox as a parameter. Ex:

                        tg-admin toolbox -c ::ffff:127.0.0.1

The error is exactly the same regardless of whether you access http://localhost:7654/, http://127.0.0.1:7654/ or http://0.0.0.0:7654/.

If you set server.socket_host="localhost" in dev.cfg everything works normal.

Attachments

toolbox-ipv6-workaround.diff (0.5 kB) - added by Chris Arndt on 04/27/08 18:39:06.
Simple patch implementing workaround solution a. from comment no. 10
toolbox-no-ipv6-default.diff (2.9 kB) - added by Chris Arndt on 04/27/08 22:33:40.
Patch implementing solution described in comment no. 11 (patch above included)
test_socket_server.py (1.7 kB) - added by Chris Arndt on 04/28/08 08:27:24.
Minimal socket test server that emulates cherrypy.WSGIServer behaviour
match-ipv6.patch (6.7 kB) - added by chrisz on 05/05/08 14:04:22.
This patch makes the match_ip function IPv6-aware

Change History

02/05/08 06:32:41 changed by faide

  • milestone changed from 1.1 to 1.0.4.

There is no such problem when using the toolbox on Windows. CP reports it is listening on 0.0.0.0 and it just works as expected otherwise.

We could force the socket_host value in the command line starter but I'd like someone to test and confirm the fix on thoses plateforms: Linux, Mac

02/05/08 06:34:26 changed by faide

I changed the target to 1.0.4 since we need to release a some fixes to 1.0.4 still and I feel the toolbox problem is big enough to be included in the 1.0 branch.

Moreover, this problem was reported after the security fix which forced an urgent upgrade to CP 2.3.0 and as such represents an regression and should be fixed.

02/05/08 08:37:55 changed by Chris Arndt

We could also consider to set server.socket_host="localhost" by default in dev.cfg of quickstarted projects. IMHO an app running in development mode should not be reachable from other hosts by default anyway.

Of course we should still make sure that the toolbox runs regardless of the setting dev.cfg.

02/05/08 10:40:06 changed by khorn

Looking through the mailing list, this problem seems to have only been reported when running under Ubuntu. Not sure if it's a Ubuntu-specific problem, or if its more general (Linux-only).

Perhaps Ubuntu has a IPv6 setup which is not being accounted for?

Possible workaround until this gets fixed: run tg-admin toolbox with "-c ::ffff:127.0.0.1"

Can anyone verify this?

(follow-up: ↓ 6 ) 02/12/08 06:57:10 changed by Chris Arndt

The workaround with the -c option works for me but the problem persists. Tested on two computers, both with Ubuntu Feisty 7.04, TG 1.0.4.

(in reply to: ↑ 5 ) 03/04/08 23:47:09 changed by alexandre

persists for me too (ubuntu 7.10) any version of tg with new cherrypy bugfix. cherrypy "remoteAddr" is also returning this address (seems to be an ipv6 like addr), I've been checked this on some tries to get the client IP for logging.

(follow-up: ↓ 8 ) 03/05/08 05:01:11 changed by faide

This seems to be happening on machines with IPV6 addresses (Linux boxes mainly).

The solution could be to force the socket host to be 127.0.0.1 by default in the config file...

Could someone who encounters the problem try this and report successes/failures?

(in reply to: ↑ 7 ) 03/05/08 06:32:48 changed by alexandre

The config workarround works for me.

04/23/08 13:36:37 changed by khorn

  • milestone changed from 1.0.4 to 1.0.x bugfix.

version 1.0.4 has already been released and this ticket hasn't been updated since the last 1.0.x release

moving to 1.0.x bugfix milestone

04/27/08 18:29:05 changed by Chris Arndt

  • owner changed from anonymous to Chris Arndt.
  • status changed from new to assigned.

Some investigation into this matter, but no solution yet:

  • The tg-admin toolbox command starts a CherryPy server with toolbox.Toolbox as the root controller, wrapped into a identity.SecureObject.
  • The predicate for the SecureObject is identity.from_any_host with the list of allowed hosts set to ['127.0.0.1', '::1'].
  • Ubuntu has a mixed IPV4/IPV6 setup, where the loopback device has an IPV6 address of '::1/128' (the /128 part is an alternative notation for ::1 , see Wikipedia: IPV6 - Network notation).
  • identity.from_any_host resp. identity.from_host uses identity.conditions._match_ip to check if the IP address of the requesting host is in the list of allowed addresses and networks. It passes the hostname returned by identity.conditions._remoteHost, which reads the remote hostname from the X-Forwarded-For or Remote-Addr request headers.
  • identity.conditions._remoteHost reports the request coming from localhost with an IPV6 address ::ffff:127.0.0.1, which is an IPV4 mapped address, equivalent to ::1.
  • identity.conditions._match_ip does not support IP addresses* :-( and so will return False, causing access to be rejected by identity.from_any_host.

* It will somehow work for non-network IPV6 addresses (with a "/"), because it does a simple equality test for what it believes are host addresss, but if the address to check against contains a "/", it will fail for IPV6 addresses, because it uses socket.inet_aton to resolve network masks, which does not support IPV6 addresses.

Open questions:

- Is the IPV6 address in the Remote-Addr request header set by CherryPy or the Linux IP stack? The fact that this phenomena only appeared after the switch to CherryPy 2.3 would indicate the former. Will investigate this.

Possible solutions:

  1. Add ::ffff:127.0.0.1 to self.hostlist in command.ToolboxCommand.
  2. Make identity.conditions._match_ip support IPV6 addresses (possibly by using the IPy module?).

04/27/08 18:39:06 changed by Chris Arndt

  • attachment toolbox-ipv6-workaround.diff added.

Simple patch implementing workaround solution a. from comment no. 10

04/27/08 22:11:20 changed by Chris Arndt

I nailed this down to the way CherryPy <3.0 handles the server listen socket. Apparently, CherryPy changeset 1665 has not been backported to CherryPy 2.x, which means, that the default for socket.server_host is the empty string ('') and cherrypy.WSGIServer (which is the default HTTP server class used by CherryPy) will then determine the IP address family to use for the server listen socket via socket.getaddrinfo() with the host argument set to None.

This will return [(10, 1, 6, '', ('::', 7654, 0, 0)), (2, 1, 6, '', ('0.0.0.0', 7654))] on Ubuntu. Note the 10 in the first tuple, which means socket.AF_INET6. So the server listen socket will use IPV6 and the address of connecting clients as returned by server.socket.accept() and consequently stored in in cherrypy.request.headers('Remote-Addr')will be a IPV6 address.

Solution: set server.socket_hostto '0.0.0.0' in command.base.ToolboxCommand.run. Or better, provide an option for tg-admin toolbox, to set the server listen address. This will cause CherryPy to use an IPV4 socket, unless the server is IPV6 only.

For the latter reason, identity.conditions._match_ip still needs to be fixed to support IPV6 networks (see previous comment). The workaround with extending ToolboxCommand.hostlist (see first patch) will still be needed for pure IPV6 setups until then.

04/27/08 22:33:40 changed by Chris Arndt

  • attachment toolbox-no-ipv6-default.diff added.

Patch implementing solution described in comment no. 11 (patch above included)

04/28/08 08:27:24 changed by Chris Arndt

  • attachment test_socket_server.py added.

Minimal socket test server that emulates cherrypy.WSGIServer behaviour

05/01/08 16:59:37 changed by chrisz

Solution a. works for me, but solution b. throws errors on Win XP and OpenSUSE 10.3.

05/02/08 06:36:54 changed by Chris Arndt

What do you mean? I have not implemented solution b), i.e. support for IPV6 addresses in identity.conditions._match_ip.

The second patch is a new solution c), combining a) and setting server.socket_host to 0.0.0.0. What is the error you get?

05/02/08 08:34:22 changed by chrisz

I just tried both patches, i.e. solution c) then. Error with OpenSUSE: 

Failed to add service: Invalid host name
01/May/2008:23:54:53 HTTP INFO Port 7654 not free on '0.0.0.0'
Traceback (most recent call last):
  File "/usr/local/bin/tg-admin", line 8, in <module>
  ...
  File "/usr/local/lib64/python2.5/site-packages/CherryPy-2.3.0-py2.5.egg/cherrypy/_cpserver.py", line 252, in wait_for_free_port
    raise cherrypy.NotReady("Port not free.")
cherrypy._cperror.NotReady: Port not free.

Error with WinXP: 

01/May/2008:23:55:54 HTTP INFO Port 7654 not bound on '0.0.0.0'
Exception in thread CPServer Callback Thread-1:
Traceback (most recent call last):
  File "C:\Programme\Python25\lib\threading.py", line 486, in __bootstrap_inner
    self.run()
  ...
  File "C:\Programme\Python25\lib\site-packages\cherrypy-2.3.0-py2.5.egg\cherrypy\_cpserver.py", line 162, in wait_for_http_ready
    raise cherrypy.NotReady("Port not bound.")
NotReady: Port not bound.

I will see if I can make _match_ip cope with IPV6; should not be too difficult.

05/02/08 10:11:06 changed by Chris Arndt

I tried my patch on OS X 10.4 (works) and Win 2000 (fails). The error on Windows is the same you get. The error is caused by cherrypy._cpserver.Server.check_port trying to connect to the given server.socket_host and server.socket_post. It seems that connecting to 0.0.0.0 does not work on all machines (maybe not on those without a IPV6 stack), only listening on it.

This seems like a CherryPy quirk to me. I have to investigate how CP 3 solves this.

05/02/08 12:21:15 changed by chrisz

Yes, 0.0.0.0 is an unspecified address that can be used if you want to listen on all interfaces, but of course you cannot connect to it.

I now have a working _match_ip that can deal with IPv6 addresses, plus unit test. It turned out that it was somewhat difficult, since the standard lib functions for converting IPv6 addresses is only available under Unix. Luckily some code from Twisted could be reused here.

However, I feel this auxiliary function (about 80 lines of code) does not belong in identity.conditions, but should go into a separate module, probably not even under identity. Shall I put it in turbogears.util? Any better idea?

05/04/08 15:46:49 changed by Chris Arndt

I suggest:

  • In TG 1.0 we simply add ::ffff:127.0.0.1and ::1/128 to self.hosts in ToolboxCommand (remember, critical bug-fixes only, we need to concentrate our efforts on TG 1.1).
  • In TG 1.1 we:
    • Do the same as above.
    • Add an option to tg-admin toolbox to set the interface address, but set no default.
    • Add the IPV6 address / network checking code to turbogears.util and use it in identity.conditions. The code may be useful in the validators module as well.

turbogears.util is such a mixed bag already, that we might consider turning this into a sub-package for TG 1.1.

05/05/08 01:37:59 changed by chrisz

Just to make that clear, if we compare the IP addresses with the IPv6 aware _match_ip then we don't need to add ::ffff:127.0.0.1, because it is considered equal with 127.0.0.1.

Yes, splitting util would make sense.

05/05/08 05:27:56 changed by Chris Arndt

Ok, I guessed so, but without seeing the code, I wasn't sure. Then scratch the first item from the changes for TG 1.1.

Will you prepare a patch?

05/05/08 07:46:19 changed by chrisz

Ok, I'll create a patch. Just wasn't sure whether we want to fix this in 1.0 as well, and where the code should go. I'll put it in tg.util for now. The refactoring of tg.util should become a ticket of its own. We may want to put some more of the toplevel modules in that package, not only the tg.util module itself.

05/05/08 14:04:22 changed by chrisz

  • attachment match-ipv6.patch added.

This patch makes the match_ip function IPv6-aware