MASTER Network Manager should probe for GSM and CDMA command sets. It doesnt detect 3g modems with serial capability only in hal

[Alexander Sack]

Binary package hint: network-manager

problem is that almost all modems are detected as "serial" v.250 modems, but that isnt enough for NM to decide whether its a GSM or CDMA modem.

Without that info network manager cannot properly detect which commands to use and ends up not supporting them.

Adding all the pci ids to the modem.fdi isnt the way to go. instead we need to use AT commands to probe whether its a CDMA or GSM modem.

This probing could be done as a hal hook or in NM directly. The right approach has yet to be decided.

Test packages: http://launchpad.net/~network-manager/+archive

[Craig Ringer (ringerc)]

Relaying my comment from bug 262490:

Some devices, like the Dell internal USB HSDPA modems, lack CDC ACM descriptors. There is no way to recognize these devices as GSM/CDMA modems without a kernel driver being explicitly told about it by USB vendor/device ID, or by force-loading the generic usb-serial driver (using vendor and product ID params) then tweaking HAL to tell NM that the serial device is a modem.

To ensure that NM can then identify these devices, any found to be using the `option', `airprime', etc drivers should really be given a modem.command_sets attribute including V.250 , so NM's future GSM/CDMA probing support can identify them. Additionally, the GSM/CDMA drivers need support for user-specified vendor and product IDs because of the extensive re-branding that goes on in this area. I'll submit a patch for that shortly.

I've attached a new HAL fdi file that matches the option and airprime drivers (for a start; there are surely other GSM/CDMA drivers out there) and adds the V.250 capability to them. This should be dropped into /usr/share/hal/fdi/information/20thirdparty/ by the network-manager package. Once NM knows how to probe these devices that'll get them working as soon as the kernel drivers know about them - without having to exhaustively list vendor and product IDs in HAL's config.

[Gonzhauser (gonzhauser)]

Are you sure you attached the right file? It seems pretty small. :(

[Alexander Sack (asac)]

On Wed, Sep 10, 2008 at 02:07:25AM -0000, Craig Ringer wrote:
>
> I've attached a new HAL fdi file that matches the option and airprime
> drivers (for a start; there are surely other GSM/CDMA drivers out there)
> and adds the V.250 capability to them. This should be dropped into
> /usr/share/hal/fdi/information/20thirdparty/ by the network-manager
> package. Once NM knows how to probe these devices that'll get them
> working as soon as the kernel drivers know about them - without having
> to exhaustively list vendor and product IDs in HAL's config.
>
> ** Attachment added: "option.fdi"
> http://launchpadlibrarian.net/17484232/option.fdi
>

Thanks for this. I will try that for the next upload. You also want to
look into probing for GSM/CDMA command set? AFAICT; this could also be
done by a hal probe binary. Let me know.

 - Alexander

[Jason Costomiris (jcostom)]

Alexander,

Thanks much for the fix option, but that file doesn't seem to be working out, at least for me..

I dropped in /usr/share/hal/fdi/information/20thirdparty/ as 10-usb3g.fdi, mimicked the appropriate permissions, and rebooted the system to be sure it would take.

Plug in the 3g device, no go.. So, what should I see in NM to indicate success anyway? In configuring mobile broadband, I see no place to specify which device to use, or is that considered something that should happen automagically?

[Jason Costomiris (jcostom)]

Err, I meant to thank Craig. My bad, sorry.

[Alexander Sack (asac)]

On Fri, Sep 12, 2008 at 12:15:31PM -0000, Jason Costomiris wrote:
> Alexander,
>
> Thanks much for the fix option, but that file doesn't seem to be working
> out, at least for me..
>
> I dropped in /usr/share/hal/fdi/information/20thirdparty/ as
> 10-usb3g.fdi, mimicked the appropriate permissions, and rebooted the
> system to be sure it would take.
>
> Plug in the 3g device, no go.. So, what should I see in NM to indicate
> success anyway? In configuring mobile broadband, I see no place to
> specify which device to use, or is that considered something that should
> happen automagically?
>

This .fdi file will only take care that your device has the "modem"
capability with the v.250 command set. we cannot know more about the
modem and there is still code missing to go on from there to decide
whether its GSM or CDMA. so when you add the .fdi file what you can do
to verify is that you look at lshal output and seeif your modem has a)
'modem' in the capabilities field next to 'serial' and b) that it has
the v.250 command set (though not sure if you see that in lshal).

could you attach your lshal after using his .fdi?

 - Alexander

[Jason Costomiris (jcostom)]

Ah, apologies for my ignorance on this particular process.. I'll spare you all the old gummer "in my day..." rhetoric at this point. :)

Attached is the output of lshal after adding the new fdi.

[Alexander Sack (asac)]

Jason, ok. for you the option.fdi appears to have done its job. Thanks for testing.

[Alexander Sack (asac)]

Jason, one more thing. without any fdi changes, do you have the v.250 command set in lshal?

[Jason Costomiris (jcostom)]

The two appear to be pretty much the same, regardless of whether the new fdi is in place or not.. The original output was attached to another bug I'd opened, so here it is over here..

[Antti Kaijanmäki (kaijanmaki)]

modem.fdi already has a match for Pantech modems.

from:
http://gitweb.freedesktop.org/?p=hal-info.git;a=blob;h=a8cdc12a5a7fd668b6b022597aabb094538f09f7;hb=ee829a040a61df4ece4fcf0ae18b1f43f242b635;f=fdi/information/10freedesktop/10-modem.fdi

 182 <!-- Pantech -->
 183 <match <email address hidden>:usb.vendor_id" int="0x106c">
 184 <!-- PC5740;PC5750;UM150 EVDO rev A card -->
 185 <match <email address hidden>:usb.product_id" int_outof="0x3701;0x3702;0x3711">
 186 <match <email address hidden>:usb.interface.number" int="0">
 187 <append key="info.capabilities" type="strlist">modem</append>
 188 <append key="modem.command_sets" type="strlist">IS-707-A</append>
 189 </match>
 190 </match>
 191 </match>

just add the usb product id to the list and you should be ready to go.

[Alexander Sack (asac)]

ok. after discussing with Antti, the option.fdi appears to be not really an option. For some modems it would detect the right thing, but for others it might detect the wrong device and which even might cause issues.

So the way to go here is to add the probing code for GSM and CDMA either to hal ... or to NetworkManager.

That alone should add support for quite a few modems that are currently only known as v.250. Everything left should be individually configured. Of course, we have to see how many cases are left. Cross your fingers that those are just a few.

[Alexander Sack (asac)]

What we should do for now is to go through the bugs filed against NM about not detected 3g devices and see if they would get fixed by this. The idea is to look for modem and v.250 in the lshal output (Antti correct me if that is inaccurate).

Thanks!

[Antti Kaijanmäki (kaijanmaki)]

Yes, every modem that is marked as v.250 _should_ be detected correctly when probing is in place.

[Jason Costomiris (jcostom)]

Antti nailed it.

I updated the 10-modem.fdi file to include support for the UM150 and UM175. Attached is a patch against the current file from hal-info, which I believe I'm now receiving from the nm PPA..

Works like a treat now. Very slick, nice work guys!

[Antti Kaijanmäki (kaijanmaki)]

great!

Now if someone has time the patch should be sent to <email address hidden> for upstream inclusion. I would do it my self, but I'm really busy AMT :-/

[Antti Kaijanmäki (kaijanmaki)]

I've created a new branch where I pushed a fix to this bug:
https://code.launchpad.net/~network-manager/network-manager/ubuntu.0.7.probev250

I used the same code during the summer when I developed libmbca (Summercode Finland) project and I know it works at least for several Nokia mobile phones and it should work for other devices, too.

[Alexander Sack (asac)]

ok. we have a proposed fix in the branch listed in the "realted branches" section of this bug. Please test.
Simply checkout the branch and run dpkg-buildpackage -rfakeroot -b to build it.

Would appreciate to get some feedback before uploading it to PPA.

Thanks!

[Craig Ringer (ringerc)]

I'm a little puzzled about why HAL needs to be told about every modem the kernel's GSM/CDMA-specific modem drivers, like option and airprime, supports individually. Given the extensive rebranding that goes on surely that'll ensure that Ubuntu will always lag significantly behind on support for new 3G hardware?

Do the option, airprime, etc drivers ever attach to and expose a device that cannot support V.250 and it is inappropriate to probe to type it as GSM or CDMA?

[Antti Kaijanmäki (kaijanmaki)]

OK, so I will try to shed some light to this. Please, correct me if I'm wrong :-)

It has been suggested that every serial device exposed by a certain driver would have "modem" capability set. This can't be done because every exposed serial device can't be used to establish an Internet connection and "modem" is indeed supposed to tell NM and other programs which one of multiple available serial devices can be used to establish a connection. The other serial devices might be used for SMS or some proprietary functions. That's why the modem capability is set individually.

However Ubuntu will not lag behind because hal-info is an information package and thus it can be updated safely, easily and fast even on distributions under feature freeze. And multiple devices from different vendors ship with common modem components and thus the number of different modem vendor_id/product_id out in the wild is relatively small. Just like with ethernet cards; you have dozens or even hundreds of different retailers, vendors and brands, but still they all use the same chips from realtek :-)

[Ruben Romero (huayra)]

The patch is working perfectly with the Nokia E71, before it got recognized but no connection was possible.

Although the mbca did not pop up now to set up the connection automatically, I had to fill the APN manually, but that is just a detail really... Isn't it? The connection is working now anyway (and I am writing from it, my LAN cable is disconnected :)

I will test with the Nokia N95 and Option USB card and give you feedback in this bug and its respective bugs as well.

Followed your build instructions form bug 262392 (N95)

[Ruben Romero (huayra)]

N95 is recognized by nm-tool. No internet connection possible yet (most likely some PPP issues).

So this is fixing bug #262392 as well (marked that as "Fix Committed" as well).

[Craig Ringer (ringerc)]

Antti Kaijanmäki: Your comments do make sense. I'd been thinking in terms of distinguishing GSM/CDMA devices from generic serial devices, rather than considering that some GSM/CDMA devices, while supported by option/airprime/etc, might not be capable of GPRS/EDGE/HSDPA Internet access.

In that context it does seem right to identify specific devices in HAL. There will need to be a howto for users, though, probably in the form of a URL to the community wiki in the help pages.

I do think there will be issues with some laptop vendors (Dell, for example) that rebrand all their hardware. Still, with some decent user documentation so users know how to find out their device IDs and submit them it should be alright.

By the way, I had a go at modifying option.c to accept a user-supplied USB vendor/product ID. It's not at all difficult; just leave an extra empty entry in the module table before the terminator, check the module arguments, and populate it if they're set, just like the generic serial driver does. However, it requires the option driver to ask to see every attached USB device rather than letting the kernel select automatically based on the exported module table. I doubt the kernel folks would be too enthusiastic about every USB serial module having to be poked for every USB device insert ... just on the off chance it's an as-yet-unknown device the user has told one of the modules about.

So, rather than adding vendor= and product= parameters to every driver, I now think it's better to make sure that NetworkManager works properly with the generic USB serial device. That way the user can just drop a HAL .fdi file (edited from a template on, say, the community wiki) into /etc/hal/fdi to tell HAL their device is a modem. To get the kernel to recognise it they'd add something to /etc/modprobe.d like:

install 3g modprobe usbserial vendor=0x413c product=0x8138

Slightly more cumbersome than just loading option.c with vendor and product args, but that'd only really be easier if NM could assume that all devices using the option driver were modems. It can't, so we still need the FDI file, and we might as well use the generic USB serial driver until the next kernel update is released with the new device IDs.

Sound sensible?

BTW - testing probe branch now.

[Craig Ringer (ringerc)]

The built-in GSM modem was detected after I installed the packages built from the network-manager branch referenced above and (for completeness's sake) rebooted the laptop. I've removed all HAL FDI customisations except the one required to tell HAL that my modem is in fact a modem (attached). Note that it does not specify any GSM capabilities/command sets, only the modem capability and V.250 command set.

Something - either hal, or network-manager communicating with hal - has added the GSM command sets to the device's HAL listing.

[Craig Ringer (ringerc)]

lshal when testing latest branch.

Full build procedure, for anyone else wanting to test this branch before it hits the PPA, was:

sudo apt-get install bzr
mkdir ~/tmp
cd ~/tmp
sudo apt-get build-dep network-manager
bzr branch lp:~network-manager/network-manager/ubuntu.0.7
cd ubuntu.0.7
dpkg-buildpackage -rfakeroot -b
sudo dpkg -i ../*.deb

[Alexander Sack (asac)]

i uploaded that branch to intrepid and hardy network-manager PPA. please test from there.

[Alexander Sack (asac)]

Craig, could you please file a bug against hal-info for your dell5520.fdi (and if you have other devices, file individual bugs for them too).

Thanks!

[Launchpad Janitor (janitor)]

This bug was fixed in the package network-manager - 0.7~~svn20080908t183521+eni0-0ubuntu3

---------------
network-manager (0.7~~svn20080908t183521+eni0-0ubuntu3) intrepid; urgency=low

  [ Antti "Wellark" Kaijanmäki ]
  * Fix LP: #268095 - MASTER Network Manager should probe for GSM and CDMA command
    sets; we probe all modems tagged as v.250 in hal that otherwise dont specify a
    GSM or CDMA command set.
    - added debian/patches/add_probe_for_v250_modems.patch
    - modified debian/patches/series

  [ Alexander Sack <email address hidden> ]
  * point Vcs-Bzr to right source tree
    - update debian/control
  * Fix LP: #269755 - "network manager always crashes on resume"; these crashes
    are a regression introduced by fix for LP: #259503 which was first
    released in 0.7~~svn20080908t183521+eni0-0ubuntu1. To fix we redo the patch
    in a way that will eliminate any idle callbacks as soon as they
    become irrelevant.
    - update debian/patches/80_lp259503_access_to_freed_device_struct.patch

  [ Steve Langasek <email address hidden> ]
  * system-settings/plugins/keyfile/plugin.c: it might be better if we were
    casting the data from the hash to a gobject, instead of the key which is
    a filename... fixes a crash whenever trying to handle system-level secrets.
    LP: #269010.
    - add debian/patches/lp269010_keyfile_secrets_crash.patch
    - update debian/patches/series

 -- Alexander Sack <email address hidden> Wed, 17 Sep 2008 02:54:56 +0200

[Gonzhauser (gonzhauser)]

Sony Ericsson K600i (with USB) is now recognized, thank you very much.
After editing the wireless options I can connect to my provider.
But after the connection is successfully established I can't connect to
any website. Even a ping to the nameservers provided by NetworkManager
in /etc/resolv.conf do not answer.

Any help?

[Alexander Sack (asac)]

On Wed, Sep 24, 2008 at 10:46:20AM -0000, Gonzhauser wrote:
> Sony Ericsson K600i (with USB) is now recognized, thank you very much.
> After editing the wireless options I can connect to my provider.
> But after the connection is successfully established I can't connect to
> any website. Even a ping to the nameservers provided by NetworkManager
> in /etc/resolv.conf do not answer.
>

what IPs are listed in your resolv.conf?

 - Alexander

[Gonzhauser (gonzhauser)]

I upgraded with "update-manager -d" today and GSM works.

I plugged my mobile (Sony Ericsson K600i) with a USB cable (it was detected as a K608i), chose my provider (ALDI/Medion), and it worked without further intervention.

Thank you very much.

g

[Alexander Sack (asac)]

On Tue, Oct 28, 2008 at 11:32:53PM -0000, Gonzhauser wrote:
> I upgraded with "update-manager -d" today and GSM works.
>
> I plugged my mobile (Sony Ericsson K600i) with a USB cable (it was
> detected as a K608i), chose my provider (ALDI/Medion), and it worked
> without further intervention.
>

Thanks for verifying. Much appreciated.

 - Alexander