Broadcast as fall-back mechanism when no HIT->IP

[Andrius Bentkus]

If the hip daemon can't resolve an address with hip_map_id_to_addr, it tries to broadcast an I1 message using standard interfaces.

This fallback mechanism is in my and Renes opinion useless and should be stripped out.

The cause of failing in my particular case was that I mounted /etc/hip/hosts with fuse and the fopen function failed to open it, even though hipd had root rights.

[Miika Komu (miika-iki)]

I disagree with it being useless. For beginners, it's easy to try HIP in a local network without any hosts configuration by pinging a HIT. Also, I think it's better fail by trying to do something than just to fail and do nothing. IPsec doesn't return error messages that fast to the application anyway, right?

If you want to "get rid of broadcasting", I would suggest to make a configuration option and make the default "off". I don't think it's useless.

The thing with fuse is probably a separate topic and may deserve a bug of its own. Hipd usually drops root privileges to "nobody" (check this with "ps"). Does nobody has access privileges to the hosts file?

[Andrius Bentkus (toxedvirus)]

The problem is that the fall back mechanism doesn't work ether in my case, though I have 2 network interfaces, eth0 is the virtual bride to the default network and eth1 is the internal network.
The problem is that the first side (while pinging the second side) does send successfully the I1 packet and the second side recieves it and responds to it with an R1 but the first side never recieves that.

[Christof Mroz (christof-mroz)]

Did you get it to work at all so far?
Now the question is if the receiver inferred a wrong IP address from the Broadcast packet (received via UDP on Port 10500 as far as I know), thus sending the R1 to nirvana. In any other case, i.e. if tcpdump shows the R1 packet on the sender, the problem's probably not associated with the fall-back. Log files would be quite helpful in any case.

[Miika Komu (miika-iki)]

Broadcast receiving may have been broken by the merge from tinyhip branch. It wasn't tested after the merge.

[Miika Komu (miika-iki)]

http://bazaar.launchpad.net/~hipl-core/hipl/trunk/revision/4609#hipd/input.c

(search for "broadcast" in the code)

[René Hummen (rene-hummen)]

I would remove broadcasting for 3 reasons:

1) Broadcasting I1s is not specified and thus only supported by HIPL. As such there's no interoperability with other implementations, which could lead to situations where HIPL <-> HIPL just works, whereas combinations don't. This can also be confusing for beginners in a heterogenous scenario.

2) Broadcasts are not routed beyond local network boundaries. Hence, it only provide a failover mechanism for very simple network topologies.

3) Instead of trying to broadcast, we should give an explanation why the connection could not be established. A reference to a FAQ (that we would need in such a case) would be a nice starting point for beginners to track down their problem. I.e. "No HIT to IP mapping found for the requested connection. Please refer to FAQ #12345 for further information."

[Miika Komu (miika-iki)]

You can remove if it is what you want. We can reintroduce the feature if standardized or there is need for it.

If you want to achieve the third point in the implementation, some experimentation with TCP, UDP and ICMPv6 based applications is needed. Maybe the error event can be catched in hipd/netdev.c:hip_netdev_event() ?

[Christof Mroz (christof-mroz)]

René: How is point 1) going to cause problems at the I1 receiver? Of course, the I1 sender must be prepared of getting multiple replies etc. but it's running HIPL anyway.
Concerning Point 2) you're right. IMHO it should be retained as an (default-off) option for small-scale testing purposes.

[René Hummen (rene-hummen)]

@Christof:
(1) HIPL is just one of a few HIP implementations. Therefore, the receiver does not necessarily run HIPL.
(2) A default off option will exactly _not_ help beginners, who just install HIPL and want it to work out of the box.

@Miika:
I can't really follow why we need experimentation. hipd recognizes that it does not find a HIT->IP mapping and logs the error I mentioned above.

[René Hummen (rene-hummen)]

How is broadcasting related to the "shotgun" extension? I thought we had removed shotgun from the codebase, but I still found trace of it. Is it still functional?

[Miika Komu (miika-iki)]

I thought you were referring to application level error notifications on BEX failures. You were just referring to system-level notifications, so forget my about my comment.

[Miika Komu (miika-iki)]

Regarding to the shotgun, it is functional but not complete. Do you really want to remove it, I though Tobias was really interested of this feature?

[Tobias Heer (heer)]

I think reducing the failure probability during BEX is a really good thing but I also think that we should have some specification and documentation in the form of a draft to back up the experimentation.
I also think shotgun goes into an interesting direction but I think there is some more research and coding necessary before it evolves into a feature with long-term support.

I did express my interest and doubts about shotgun on the hipsec list before. You may want to read the conversation for more reasoning.

http://www.ietf.org/mail-archive/web/hipsec/current/msg02837.html
http://www.ietf.org/mail-archive/web/hipsec/current/msg02839.html

I am not sure if I should lean towards keeping or abandoning it, though. I guess my recommendation strongly depends on whether it is still actively developed and maintained or if it is "just there". If it is maintained and will serve as input for the IETF or IRTF I think it should definitely stay (but possibly in a feature branch until the experimentation is completed). If it is not maintained anymore not, we will have some optimization without proper documentation (in the form of specification and research papers) that is only supported by HIPL and it will decay because of lacking support.

Some general questions that may make the decision to keep or discard it easier:
* Is the concept well understood? Are its side effects (e.g., interactions with middleboxes) understood and documented?
* Is it well tested? Do we know exactly when it works and when it doesn't?
* Does shotgun solve an real problem that people notice?
* Does it solve the problem well?
* Is it complete or does it need more work?
* Who will maintain and develop it?

[Miika Komu (miika-iki)]

Regarding to broadcast, I just recalled a problem related to NATs that it solved for me. I have a server at home behind a NAT box which has a public IP address. I have pinholed the NAT to pass HTTP and SSH to the server. The DNS contains a public address for the machine.

The problem occurs when I am at home with a laptop and contact my server using its public IP address (via DNS). The connection fails because the NAT box does not support so called hairpinning (I believe this is quite common). So I have to use the private address of the server at home and outside home I can use DNS normally (public IP address). This sucks big time!

While split DNS horizon or bying a more expensive NAT box would solve the case, they both have their tradeoffs (configuration complexity or monetary cost). HIP with an option for broadcast would offer a cheap and straightforward solution for the problem. Right?

[Miika Komu (miika-iki)]

Hmm, the broadcast option as I suggested would actually be a further extension to the shotgun.

[Miika Komu (miika-iki)]

On 08/25/2010 06:16 PM, Tobias Heer wrote:

Hi,

(btw, I think the topic of this comment not for the right bug report but answering nevertheless)

> I think reducing the failure probability during BEX is a really good
> thing but I also think that we should have some specification and
> documentation in the form of a draft to back up the experimentation.
> I also think shotgun goes into an interesting direction but I think
> there is some more research and coding necessary before it evolves
> into a feature with long-term support.
>
> I did express my interest and doubts about shotgun on the hipsec
> list before. You may want to read the conversation for more
> reasoning.
>
> http://www.ietf.org/mail-archive/web/hipsec/current/msg02837.html
> http://www.ietf.org/mail-archive/web/hipsec/current/msg02839.html

I thought I already answered your concerns :)

http://www.ietf.org/mail-archive/web/hipsec/current/msg02844.html

If you want to avoid expensive GPRS links, we need a policy manager. Or plain simple, turn the extension off, like it is now as default.

> I am not sure if I should lean towards keeping or abandoning it,
> though. I guess my recommendation strongly depends on whether it is
> still actively developed and maintained or if it is "just there". If
> it is maintained and will serve as input for the IETF or IRTF I think
> it should definitely stay (but possibly in a feature branch until
> the experimentation is completed). If it is not maintained anymore
> not, we will have some optimization without proper documentation (in
> the form of specification and research papers) that is only supported
> by HIPL and it will decay because of lacking support.
>
> Some general questions that may make the decision to keep or discard
> it easier:
> * Is the concept well understood? Are its side effects
> (e.g., interactions with middleboxes) understood and documented?

I believe the concept is well understood. I don't have any bad experiences with non-HIP middleboxes. Since the amount of HIP-aware middleboxes is practically non-existent, we can only speculate about it.

It should be noticed that the extension is designed so that it affects the end-host implementing it (and not its peer). So it's a local optimization.

> * Is it well tested? Do we know exactly when it works and when it
> doesn't?

I have been using it but stopped because Baris has not completed mobility support yet.

> * Does shotgun solve an real problem that people notice?

I have problems with Teredo connectivity at the office. The current address selection is dumb and ends up usually picking up the non-working Teredo address for me. So it solved the problem for me.

> * Does it solve the problem well?

The ICE-based NAT traversal approach offers something similar but is utterly complicated and not supported by the code base anymore. The native NAT traversal approach is far better, but even it does not support fault tolerance for the base exchange (or UPDATE) itself.

So it in the absence of a better solution, I would say yes.

> * Is it complete or does it need more work?

Baris is working on bug ids 592177 and 592125.

> * Who will maintain and develop it?

Baris completes t...

[René Hummen (rene-hummen)]

I guess, it's time to conclude the discussion ;-)

I have posted my thought about the inclusion of extensions at: https://answers.launchpad.net/hipl/+question/123357 Comments are welcome.

Following this line of argumentation, broadcasts and shotgun should at least have an active maintainer and should get a switch to be disabled. As you, Miika, seem to be in support for the extension, please make broadcasting optional and ensure that shotgun can be switched off completely.

Thanks,
René

[René Hummen (rene-hummen)]

As a matter of fact, broadcasting the I1 breaks the connection tracking of hipfw on a forwarding node. It seems that hipfw gets the packets at the input queue and drops it. IP forwarding still delivers the packet to the responder. However, the R1 cannot be mapped to a correct I1 at the forwarding node.

[René Hummen (rene-hummen)]

Miika, can you please implement a switch to turn off broadcasting, delegate this task or mark this bug as "Fix committed" in case you already fixed the issue. Thanks.

[René Hummen (rene-hummen)]

If I don't hear any objections until 28.12.2010 in the morning, I'll remove broadcasting from the codebase in trunk.

[Miika Komu (miika-iki)]

I object and will fix this issue.

[Miika Komu (miika-iki)]

Fixed in revision 5593. Please note that the shotgun related things are reported in another bug.

[René Hummen (rene-hummen)]

Thanks for the fix.