National Data Roaming

[Sturm Flut]

Situation:
German mobile network operator "o2-de" (MCC 262, MNC 07) has merged with another operator, "e-Plus" (MCC 262, MNC 03). Both operators have decided to allow national roaming between their networks for all customers and for all services (phone calls, SMS, data). The "other" network appears to the customer with the name of their original, provider, but an added plus sign, e.g. if you're an "o2-de" customer the e-Plus network will appear as "o2-de+" to you.

Expected behavior:
The bq Aquaris E4.5 roams into the other network if necessary and all services are available. Ideally the device would not even go into "real" roaming mode because national roaming is different from international roaming.

Actual behavior:
The bq Aquaris E4.5 roams into the other network and mobile data is deactivated because the phone does not differentiate between national and international roaming. The "mobile data roaming" switch has to be explicitely enabled, which can cause problems for the user since the phone supports two SIM cards.

Complications:
- Both operators are changing their networks during live operation, other phones have problems too.

- There is a long list of MVNOs and every one of them has to adapt to the changes too. For example I have a SIM card by o2-de MVNO "netzclub" in my bq phone and the network list now also shows the expected "netzclub+" entry, roaming to it works. On the other hand I have a SIM card by e-Plus MVNO "MEDIONmobile" in my Android Nexus 5 and there the network list does not show the expected "MEDIONmobile+" entry, but just an entry for "o2-de", and I cannot roam to "o2-de" even if I force it.

Notes:
II think there should be a similar situation in Sweden, where operators Tele2 and Telenor created a shared 4G network, and there are probably other countries with comparable setups.

Debugging data:
- I added the output of /usr/share/ofono/scripts/list-operators on the bq phone.

- I added two outputs of /usr/share/ofono/scripts/list-modems on the bq phone, once while connected to my original "netzclub" MVNO network and once while roaming in the "netzclub+" network. All personal data has been removed, but I am willing to share it in private.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Alfonso Sanchez-Beato (alfonsosanchezbeato)]

When you are registered in a network with different MCC/MNC codes to the ones in your SIM, you are roaming unless the MCC/MNC of the network you are registered to is in a list that is stored in a file on the SIM.

In this case two operators with different codes have recently merged, so it is doubtful that SIMs that were created before the merge contain the right information on the SIM to remove the roaming status.

New SIMs should provide the right information. We will try to find a workaround, but meanwhile if you do not want to set data enabled when roaming my suggestion is to ask for a new SIM to the operator.

[Michael Mess (michael-michaelmess)]

We have three cases here which should be handled in a different way:

When you are registered in a network with the same MCC and MNC code, this is your network. Your local tarif will apply.

If you are registered in a network with the same MCC and a different MNC code, this is national roaming. This is the case when national roaming between e-plus (sometimes displayed as o2.de+) and o2.de applies. In Germany national roaming does not imply extra costs, but in other countries the agreements between the companies could provide extra charges when national roaming is used.
In Germany o2 and e-plus allow roaming only for the 3G network, not for 2G network. Even when the card thinks it may access the 2G network of the other company, it is refused.

If you are registered in a network with different MCC code, this is international roaming, regardless the MNC code.
Within the European Union the extra costs for roaming are limited by law, but when roaming in a non-EU network with a EU sim card (which can happen near the European border), roaming costs are not limited by law and can be quite expensive.

Thus it would be desirable to have something like a combination of Blacklist and Whitelist and priorities for networks for each sim card.

For example in the tri-border area between Germany, Austria and Switzerland you might have the following scenarios:

With an O2-Card:
1) o2.de is the card's home network
2) e-plus (o2.de+) is the national roaming network which could be preferred like the home network as there are no extra costs and the same functionality.
3) Any network from Austria is an international roaming network, but the EU tariff applies which limits the costs. When 1 or 2 are not available, this is the best choice
4) Any network from Switzerland is an international roaming network, roaming is more expensive than in the EU and should be avoided when possible. Should be chosen only when 1 - 3 are not available. (I admit that this is not correct for O2 anymore as they count Switzerland as if it was a member of the EU, but for other German card providers, Swiss roaming can be really expensive)

Instead of just allowing or blocking roaming, I would suggest a versatile configuration functionality to choose the network in a more configurable way with automatic network search:

A card preference for general roaming setting:
* Allow any network (international Roaming) with that card
* Allow only national roaming with that card
* Limit to home network (Roaming disabled for that card)

A card preference for data usage (With the valid values as above) to specify when data may be used.

A card preference rule setting for a specific network could specify
* HOME: Prefer like card's home network - No attempt is made to search for the home network, as long as this network is available. When both, this network and the home network are available, the stronger one is chosen.
* (+n) Preferred when home network is not available. Chosen when the home network or an equal network is not available. When multiple networks with this priority are available, the stronger one is used.
* (0) Default (not set): When no preferred network is available, this is chosen.
* (-n) Av...

[Johannes (hummlbach)]

I think I have the same effect with my "lidl mobile" SIM in some places. I'm wondering that this does not affect more people. But okay if a new SIM should solve the problem, it's a good question if it should really be fixed/workarounded in "the wrong place"...

[Sturm Flut (sturmflut)]

My thoughts on this:

- Network mergers have already happened and seem to become more common, especially in joint markets like the EU. So this is not going to go away.

- The providers have explicitly decided to go for national roaming instead of handing out new SIM cards or creating a new, larger, joint network. In many cases existing customers can only get a replacement SIM for a fee.

- How is this intended to work correctly, and how do other platforms handle it? People are telling me that they can use national roaming between o2 and e-Plus here in Germany with their Android devices without even having to enable roaming at all. They just had to restart their phone once and after that if automagically worked. Apparently older Android version had a "Enable National Roaming" switch, but it was removed, so was the process automated, and if yes, how?

[Tony Espy (awe)]

@Simon

Thanks so much for the detailed bug report, and suggested fix(es).

I've changed the Importance of this bug to Wishlist, as it's a feature request that will need to be reviewed by product management before we can move on it. That said, patches are always welcome...

Also, the original bug was "National Data Roaming on BQ Acqarius E4.5". I would argue that your suggested implementation goes much further than this, and would be much more complicated to implement. Some of the features you suggest also aren't really within our control. Much of the network registration logic ( ie. determining which operator to choose based upon signal strength ) is handled within rild, and isn't something that we can have too much influence on.

For the sake of this bug, I think we should limit it to:

A card preference for general roaming setting:
* Allow any network (international Roaming) with that card
* Allow only national roaming with that card
* Limit to home network (Roaming disabled for that card)

A card preference for data usage (With the valid values as above) to specify when data may be used.

These two preferences provide a basic implementation of national roaming. I would suggest breaking out the remainder of comment #7 as a new bug.

[António Miranda (abfm90)]

Does roaming even work?

My friend with his BQ (MEO) went from Portugal to Spain and activated roaming... nothing. no calls, no text, no nothing and the other friend with MEO on Android worked normally...

[Sturm Flut (sturmflut)]

This now affects the MX4 (arale) and Aquaris E5 (vegetahd) as well.

I would like to make a case to rise the importance of this bug to something higher than "Wishlist". Ubuntu currently has no way to generally disable Roaming, my phones are constantly roaming to the "other" national network when a tower with better service is in range, and the data connection drops. So I am forced to enable Data Roaming to keep the data connection alive, which I then have to remember to turn off every time I come near a border or cross a border. I travel a lot and live near the french border :/

The solution for this is in the Android codebase. If I put the same SIM in my Nexus 5 with Android 4.4.4, everything works automagically. I don't even have to enable Roaming at all, the phone switches between both networks as if they were a single one.

[Tony Espy (awe)]

@Simon

Thanks for your comments on bug #1462208.

I will make sure this bug gets reviewed by the product team.

That said, have you ( or anyone else ) tried to get a new SIM from the operator in question and verified whether or not it makes the problem go away ( per Alfonso's suggestion in comment #6 )? I realize that this may be a cost to the end-user, but there's also a significant cost in engineering and QA in order to implement this feature. As such, this decision isn't something I can make on my own. Have you contacted BQ support?

[MaryMo (marymo)]

I have the same roaming bug with my MX4 Ubuntu touch with O2-de and O2-de+.
If i change manually to O2-de i will get mobile internet for a short while.

I guess it will be a temporary solution to change it permanent to O2-de, but how to do / save this options?

[Michael Mess (michael-michaelmess)]

Current behaviour with OTA-12 is that with o2.de+ the internet is not working even when roaming is turned on. A workaround is to turn data roaming off and then on again to let the mobile phone use it.

Another issue I have experienced in Luxembourg is that in the network pulldown, where all cards are listed, the card was listed (I think provider was POST) with internet ("H" was displayed), but the internet was not usable and the card was not listed with "H" in the status line above.
The workaround mentioned above didn't help at all, even when used multiple times. Even a reboot didn't let me get a usable internet connection.
Another provider (I think it was Tango) at another place worked well as expected.

Not being able to use the services I have paid for is a serious issue for me, thus I expect Canonical to fix this with the highest possible priority.
This bug is more than one year old with no real improvements.
How long do I still have to wait for an improvement?
Note: I do not expect a full implementation as mentioned in my earlier comments immediately, but I want to be able to use the internet hassle-free with any network - as I am paying for it - in a short term.

If Canonical still does not do its job to complete a working base environment soon, I have to think about buying a new mobile phone with another OS that does its job: :-(

The main choices would then be:
 - Windows Phone: I don't like the user interface (maybe that could change, if I get used to it?), but people report that it works and there seem to be even more useful apps available than for Ubuntu Phone. Updates are available, but market share is dropping and thus it will probably discontinued soon, either by developers or by the manufacturer.
 - Apple: Expensive, but good hardware, but no specials like Dual-SIM, very restrictive about installing software (You can install only from Apple Store, if you don't want to Jail-Break the device), high security (encrypted file system). Updates are available.
 - Android: Most developer friendly OS, biggest amount of Apps available, but there is a big security issue due to missing updates, because even when a bug has been fixed at google phones do not draw the fix directly from there, instead most phone companies have to build and roll out an update release for each device due to some garbage closed source drivers used there. This additional effort leads to phones with serious securities issues not getting fixed.

I know there would be even more choices, but most apps are available only for these three choices.
It seems that I have the choice between getting security updates (Apple + MS) or having a great choice of apps (Android).

And last not least leaving Ubuntu Phone would't be without any pain: I like it that I can use rsync with backuppc via ssh to automatically backup the data and photos on the device when it is available in the local wlan network, without the need for an external cloud service or a special backup-app installed.

[lgd (lgd)]

Thanks for your hint - my National Roaming changed often and at eplus my network was still dead - it was really frustrating to go 30s into air mode on/off procedure for 4 times in 15 min of walking here, every day. Now I checked the roaming option - it seems the last update had changed it to off. Now all works for me again! I didn't thought that it would be so simple in this case.

Offtopic: I wanted to email you but there is no public email only your gpg without email inside: I'm one of the main authors of the German touch wiki so I know and like UT but in your list of systems you should know that Sailfish OS is a great LINUX (ssh...) handy system too and on some official devices there is an emulator for Android apps too. More on this not on this report here, please. I have an email address on my profile if needed.

Greetings, Benno-007