Philippe Verdy
2004-12-28 23:44:22 UTC
----- Original Message -----
From: "Zlatin Balevsky" <***@limepeer.com>
To: "Philippe Verdy" <***@wanadoo.fr>
Cc: <***@limewire.org>
Sent: Tuesday, December 28, 2004 7:25 PM
Subject: Re: [codepatch] [core] UPnPManager.java
original source file, which was apparently created initially as a binary
file on a Mac and restored later as a text file, but with the Mac still
keeping its previous format.
That's why I had to filter CR+CRLF on my PC: this is not possible with
NotePad which does not allow editing the sequences for end-of-lines, and
preserves them unless they are deleted and recreated manually. This is not
possible too with WordPad because it changes silently CR+CRLF into
CRLF+CRLF, and there's no more indicator of which CR was substituted at
end-of-lines. But it's possible with MSDOS Edit.com which displays isolated
CR as a musical note symbol, which can be selected and replaced throughout
the file.
Under Linux, this should be done by getting the source from CVS, and
removing all CR characters that remain (CR+end-of-line in CVS text files
become CR+LF on local files, and only LF is needed to recreate end-of-lines
into CVS).
transport protocol to perform requests, plus a complex discovery machanism
using broadcasts on physical links to detect the IP addresses of UPNP
devices which are directly accessible or that are acting as gateways for
other UPnP devices.
The most common "bugs" or difficulties to resolve is that for now, LimeWire
still has no support to handle multiple local IP interfaces with separate
routing strategies. In fact, LimeWire now uses information from the local OS
with network interfaces (using APIs that are not part of UPnP), which would
allow it to handle several routing tables, each with their own firewalling
rules, their own filters, their own bandwidth management, their own
connection slots.
Unfortunately, Limewire makes various tricks in the code to handle all as if
there was only 1 networking interface plus a single loopback address, and it
fails when handling more complex cases like multicasting on a LAN. I do
think that the OS network interfaces support in Java.net should be better
used, so that Limewire would determine more precisely from which interface a
message was received, and how it can be transfered to another interface,
possibly after internal masquerading. This would allow LimeWire to act
transparently as a gateway between all physical links on which it is
connected, and through which there can be other nodes that are otherwise not
mutually accessible (except through Internet). LimeWire would be less
Internet-centric, and could be deployed within private LANs as well.
Today, we find many devices for home users that allow them to interconnect
various systems through distinct LANs or WANs: DSL routers, cable routers,
Ethernet hub, Wifi access points, BlueTooth, electrical wave adapters,
infrared, Wi-Max accesses (I have it available since one month in my area:
50 Megabits/second for a introduction price only 30% higher than 1Mb/s ADSL,
and the same price as 8Mb/s accesses, but the small city where I live now
cannot benefit of new ADSL2+ accesses at 8Mb/s to 15Mb/s offered only in
large French cities!), private Wi-Max networks (802.11a, authorized recently
within a 3km distance, and which is starting to expand across large cities
as private alternative bandwidth providers).
So users have now plenty of connections available, including for the
Internet access, and the only way to use these connections is to use
external transparent routers or local routing services exploiting more than
1 local interface. This is also a requirement for later access to IPv6
networks that also require multiple logical local interfaces, each one
managed by separate autodiscovery and autoconfiguration protocols, that
LimeWire should be able to live with.
Unfortunately, LimeWire has removed the possibility to force the IP address
for the local interface to listen, and it attempts to determine which local
interface to use and it sometimes fails. Instead Limewire could listen on
the 0.0.0.0 (any) pseudo-interface to listen for all local interfaces
simultaneously, but this causes various problems because port numbers
assigned on one interface are not necessarily the same through all
interfaces, when they are managed externally by distinct NAT routers.
Limewire should better use UPnP devices by allowing them to use PAT if
possible, so that all local interfaces will be listened on the same local
port, even if they are routed externally with distinct ports.
Also LimeWire does not work correctly for now in some enterprise networks,
that use Multi-NAT routers: 1 internal address for the local host connected
to the LAN, and several external IP interfaces to the Internet managed by
the MultiNAT router. The MultiNAT router is seen on the network as if it was
several (virtual) NAT routers each with their own local IP address on the
same physical link. But it's impossible to determine by a host in the
internal LAN, which of these virtual routers, Internet connections will be
coming from, or which one will be actually used for outgoing connections.
Here again, with UPnP, we could list of these external devices and get info
about their respective routes.
Some users do complain today that LimeWire was working in previous versions
from their Enterprise or University private LAN, but not today with the
recent introduction of the preliminary UPnP support: it certainly comes the
fact that this code failed to determine correctly which local IP interface
to listen or advertize, and this causes connection attempts to fail.
Another problem with multihomed hosts: even if UPnP is not supported in
external devices, there's a support for the local routing tables managed in
Windows or Unix IP kernels and in java.net. But LimeWire attempts to
determine the "public" IP address as if it was unique and the same for all
remote clients. However, LimeWire forgets to determine at least from which
local interface a message was received, and so makes false determination of
the "echoed back" public IP addresses. This unique setting can even become
unstable, always switching from one to another, and causing further
transfers of QueryHit data or push requests to fail as they contain the
wrong IP for the effectively used local interface.
If Limewire was separating each interface (including the loopback interface
at 127.0.0.1), things would be simpler and more stable. The configuration
panel could list each interface according to the addresse types they can
route. An interface that can route loopback addresses would have limited
configuration options and no need to specify IP addresses: it would be
configured directly under the name "local host loopback interface", and the
user could enable/disable its support separately (disabling the loopback
would have the effect of closing the support for external Magnet handlers,
so Magnet options should only be present in the configuration pane for the
local host interface).
Many hosts would have an additional interface active only when the Internet
connection is active: it can be detected by the fact that it routes the
0.0.0.0 address via the address of a gateway which can connect to the
Internet. Such interface routing 0.0.0.0 can be qualified of "Internet
connection", and the interface specified in the routing table is clear about
this.
See for exampel what my PC indicates in its routing table:
D:\CVSHOME\limewire\gui>netstat -rn
Table de routage
===========================================================================
Liste d'Interfaces
0x1 ........................... MS TCP Loopback interface
0x2 ...00 0e a6 c4 a7 5b ...... Broadcom NetXtreme Gigabit Ethernet -
Miniport d'ordonnancement de paquets
===========================================================================
===========================================================================
Itinéraires actifs :
Destination réseau Masque réseau Adr. passerelle Adr. interface
Métrique
0.0.0.0 0.0.0.0 10.0.0.138 10.0.0.3 20
10.0.0.0 255.255.255.0 10.0.0.3 10.0.0.3 20
10.0.0.3 255.255.255.255 127.0.0.1 127.0.0.1 20
10.255.255.255 255.255.255.255 10.0.0.3 10.0.0.3 20
127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1
224.0.0.0 240.0.0.0 10.0.0.3 10.0.0.3 20
255.255.255.255 255.255.255.255 10.0.0.3 10.0.0.3 1
Passerelle par défaut : 10.0.0.138
===========================================================================
Itinéraires persistants :
Aucun
The interesting column in this result is "Adr. interface" which above can be
either "10.0.0.3" (interface 0x2) or "127.0.0.1" (interface 0x1): this is
the discriminating information for our routing management data, which then
determines the LAN segment used by this interface (above, dest.
reseau=10.0.0.0, netmask=255.255.255.0), and whever this segment is
connected to the Internet by an intermediate router (this is the case here,
as the interface 10.0.0.3 routes 0.0.0.0 via the 10.0.0.138 gateway).
All this is directly accessible in java.net without UPnP. Before using UPnP
(which will be useful only to get similar routing tables for other remote
hosts), one should first handle correctly the local routing table as seen by
the "netstat -rn" command line or by the java.net.* classes (both use the
SNMP protocol interface with the standard MIBv2 schema to get this
information).
Now when the LAN interface is disconnected or disabled, the routing table
becomes:
Table de routage
===========================================================================
Liste d'Interfaces
0x1 ........................... MS TCP Loopback interface
0x2 ...00 0e a6 c4 a7 5b ...... Broadcom NetXtreme Gigabit Ethernet -
Miniport d'ordonnancement de paquets
===========================================================================
===========================================================================
Itinéraires actifs :
Destination réseau Masque réseau Adr. passerelle Adr. interface
Métrique
127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1
255.255.255.255 255.255.255.255 255.255.255.255 2 1
===========================================================================
Itinéraires persistants :
Aucun
where the interface address 10.0.0.3 has disappeared and been replaced by
the pseudo-interface address 2 (displayed simply as "2" or "0x2"): it does
not route 0.0.0.0 and so it is not connected to the Internet. But the
interface still routes 255.255.255.255 (ANY_ROUTER), and so is visible as a
router which can be reenabled and even queried by UPnP, to detect if and
when it will be able to connect to the Internet.
Using UPnP will display information about this interface 0x2: you can see
its firewall routing rules, traffic policy, allowed/filtered ports, ICMP
port/features opened, traffic counters, and allowed applications... Possibly
more information if you use SNMP and specific MIB schemas, instead of just
the XML schemas standardized for UPnP devices.
From: "Zlatin Balevsky" <***@limepeer.com>
To: "Philippe Verdy" <***@wanadoo.fr>
Cc: <***@limewire.org>
Sent: Tuesday, December 28, 2004 7:25 PM
Subject: Re: [codepatch] [core] UPnPManager.java
Thanks a lot for the patch Phillipe. Unfortunately the patch -p0 command
rejected it, so we applied the changes manually.
Most probably this did not work because of incoherent end-of-lines in therejected it, so we applied the changes manually.
original source file, which was apparently created initially as a binary
file on a Mac and restored later as a text file, but with the Mac still
keeping its previous format.
That's why I had to filter CR+CRLF on my PC: this is not possible with
NotePad which does not allow editing the sequences for end-of-lines, and
preserves them unless they are deleted and recreated manually. This is not
possible too with WordPad because it changes silently CR+CRLF into
CRLF+CRLF, and there's no more indicator of which CR was substituted at
end-of-lines. But it's possible with MSDOS Edit.com which displays isolated
CR as a musical note symbol, which can be selected and replaced throughout
the file.
Under Linux, this should be done by getting the source from CVS, and
removing all CR characters that remain (CR+end-of-line in CVS text files
become CR+LF on local files, and only LF is needed to recreate end-of-lines
into CVS).
Feel free to poke around the UPnP code more - its a very interesting
standard which allows the developers to do a lot of cool stuff with
routers. The third-party we used was not exactly flawless so more review
and comments are always welcome.
UPnP is a protocol based on a XML syntax and XSD schemas, and HTTP as thestandard which allows the developers to do a lot of cool stuff with
routers. The third-party we used was not exactly flawless so more review
and comments are always welcome.
transport protocol to perform requests, plus a complex discovery machanism
using broadcasts on physical links to detect the IP addresses of UPNP
devices which are directly accessible or that are acting as gateways for
other UPnP devices.
The most common "bugs" or difficulties to resolve is that for now, LimeWire
still has no support to handle multiple local IP interfaces with separate
routing strategies. In fact, LimeWire now uses information from the local OS
with network interfaces (using APIs that are not part of UPnP), which would
allow it to handle several routing tables, each with their own firewalling
rules, their own filters, their own bandwidth management, their own
connection slots.
Unfortunately, Limewire makes various tricks in the code to handle all as if
there was only 1 networking interface plus a single loopback address, and it
fails when handling more complex cases like multicasting on a LAN. I do
think that the OS network interfaces support in Java.net should be better
used, so that Limewire would determine more precisely from which interface a
message was received, and how it can be transfered to another interface,
possibly after internal masquerading. This would allow LimeWire to act
transparently as a gateway between all physical links on which it is
connected, and through which there can be other nodes that are otherwise not
mutually accessible (except through Internet). LimeWire would be less
Internet-centric, and could be deployed within private LANs as well.
Today, we find many devices for home users that allow them to interconnect
various systems through distinct LANs or WANs: DSL routers, cable routers,
Ethernet hub, Wifi access points, BlueTooth, electrical wave adapters,
infrared, Wi-Max accesses (I have it available since one month in my area:
50 Megabits/second for a introduction price only 30% higher than 1Mb/s ADSL,
and the same price as 8Mb/s accesses, but the small city where I live now
cannot benefit of new ADSL2+ accesses at 8Mb/s to 15Mb/s offered only in
large French cities!), private Wi-Max networks (802.11a, authorized recently
within a 3km distance, and which is starting to expand across large cities
as private alternative bandwidth providers).
So users have now plenty of connections available, including for the
Internet access, and the only way to use these connections is to use
external transparent routers or local routing services exploiting more than
1 local interface. This is also a requirement for later access to IPv6
networks that also require multiple logical local interfaces, each one
managed by separate autodiscovery and autoconfiguration protocols, that
LimeWire should be able to live with.
Unfortunately, LimeWire has removed the possibility to force the IP address
for the local interface to listen, and it attempts to determine which local
interface to use and it sometimes fails. Instead Limewire could listen on
the 0.0.0.0 (any) pseudo-interface to listen for all local interfaces
simultaneously, but this causes various problems because port numbers
assigned on one interface are not necessarily the same through all
interfaces, when they are managed externally by distinct NAT routers.
Limewire should better use UPnP devices by allowing them to use PAT if
possible, so that all local interfaces will be listened on the same local
port, even if they are routed externally with distinct ports.
Also LimeWire does not work correctly for now in some enterprise networks,
that use Multi-NAT routers: 1 internal address for the local host connected
to the LAN, and several external IP interfaces to the Internet managed by
the MultiNAT router. The MultiNAT router is seen on the network as if it was
several (virtual) NAT routers each with their own local IP address on the
same physical link. But it's impossible to determine by a host in the
internal LAN, which of these virtual routers, Internet connections will be
coming from, or which one will be actually used for outgoing connections.
Here again, with UPnP, we could list of these external devices and get info
about their respective routes.
Some users do complain today that LimeWire was working in previous versions
from their Enterprise or University private LAN, but not today with the
recent introduction of the preliminary UPnP support: it certainly comes the
fact that this code failed to determine correctly which local IP interface
to listen or advertize, and this causes connection attempts to fail.
Another problem with multihomed hosts: even if UPnP is not supported in
external devices, there's a support for the local routing tables managed in
Windows or Unix IP kernels and in java.net. But LimeWire attempts to
determine the "public" IP address as if it was unique and the same for all
remote clients. However, LimeWire forgets to determine at least from which
local interface a message was received, and so makes false determination of
the "echoed back" public IP addresses. This unique setting can even become
unstable, always switching from one to another, and causing further
transfers of QueryHit data or push requests to fail as they contain the
wrong IP for the effectively used local interface.
If Limewire was separating each interface (including the loopback interface
at 127.0.0.1), things would be simpler and more stable. The configuration
panel could list each interface according to the addresse types they can
route. An interface that can route loopback addresses would have limited
configuration options and no need to specify IP addresses: it would be
configured directly under the name "local host loopback interface", and the
user could enable/disable its support separately (disabling the loopback
would have the effect of closing the support for external Magnet handlers,
so Magnet options should only be present in the configuration pane for the
local host interface).
Many hosts would have an additional interface active only when the Internet
connection is active: it can be detected by the fact that it routes the
0.0.0.0 address via the address of a gateway which can connect to the
Internet. Such interface routing 0.0.0.0 can be qualified of "Internet
connection", and the interface specified in the routing table is clear about
this.
See for exampel what my PC indicates in its routing table:
D:\CVSHOME\limewire\gui>netstat -rn
Table de routage
===========================================================================
Liste d'Interfaces
0x1 ........................... MS TCP Loopback interface
0x2 ...00 0e a6 c4 a7 5b ...... Broadcom NetXtreme Gigabit Ethernet -
Miniport d'ordonnancement de paquets
===========================================================================
===========================================================================
Itinéraires actifs :
Destination réseau Masque réseau Adr. passerelle Adr. interface
Métrique
0.0.0.0 0.0.0.0 10.0.0.138 10.0.0.3 20
10.0.0.0 255.255.255.0 10.0.0.3 10.0.0.3 20
10.0.0.3 255.255.255.255 127.0.0.1 127.0.0.1 20
10.255.255.255 255.255.255.255 10.0.0.3 10.0.0.3 20
127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1
224.0.0.0 240.0.0.0 10.0.0.3 10.0.0.3 20
255.255.255.255 255.255.255.255 10.0.0.3 10.0.0.3 1
Passerelle par défaut : 10.0.0.138
===========================================================================
Itinéraires persistants :
Aucun
The interesting column in this result is "Adr. interface" which above can be
either "10.0.0.3" (interface 0x2) or "127.0.0.1" (interface 0x1): this is
the discriminating information for our routing management data, which then
determines the LAN segment used by this interface (above, dest.
reseau=10.0.0.0, netmask=255.255.255.0), and whever this segment is
connected to the Internet by an intermediate router (this is the case here,
as the interface 10.0.0.3 routes 0.0.0.0 via the 10.0.0.138 gateway).
All this is directly accessible in java.net without UPnP. Before using UPnP
(which will be useful only to get similar routing tables for other remote
hosts), one should first handle correctly the local routing table as seen by
the "netstat -rn" command line or by the java.net.* classes (both use the
SNMP protocol interface with the standard MIBv2 schema to get this
information).
Now when the LAN interface is disconnected or disabled, the routing table
becomes:
Table de routage
===========================================================================
Liste d'Interfaces
0x1 ........................... MS TCP Loopback interface
0x2 ...00 0e a6 c4 a7 5b ...... Broadcom NetXtreme Gigabit Ethernet -
Miniport d'ordonnancement de paquets
===========================================================================
===========================================================================
Itinéraires actifs :
Destination réseau Masque réseau Adr. passerelle Adr. interface
Métrique
127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1
255.255.255.255 255.255.255.255 255.255.255.255 2 1
===========================================================================
Itinéraires persistants :
Aucun
where the interface address 10.0.0.3 has disappeared and been replaced by
the pseudo-interface address 2 (displayed simply as "2" or "0x2"): it does
not route 0.0.0.0 and so it is not connected to the Internet. But the
interface still routes 255.255.255.255 (ANY_ROUTER), and so is visible as a
router which can be reenabled and even queried by UPnP, to detect if and
when it will be able to connect to the Internet.
Using UPnP will display information about this interface 0x2: you can see
its firewall routing rules, traffic policy, allowed/filtered ports, ICMP
port/features opened, traffic counters, and allowed applications... Possibly
more information if you use SNMP and specific MIB schemas, instead of just
the XML schemas standardized for UPnP devices.