@Netmatters I'm having the exact same problem on a HP 360 G5 with Ubuntu Server 64 11.10. 

Everything was working fine until Ubuntu Desktop 64 11.04.

During the installation (not an upgrade) of Ubuntu Server 64 11.10, I manually set up a static IP address, gateway, netmask and DNS server for eth0  but in the end no network connection was available as expected.

These are two screenshots taken during boot with a camera:

http://imageshack.us/photo/my-images/443/img20111215115606a.jpg/

http://imageshack.us/photo/my-images/163/img20111215115633.jpg/

ifconfig:
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

70-persistent-net.rules:
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.

# PCI device 0x14e4:0x164c (bnx2)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:1a:4b:ce:3e:16", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x14e4:0x164c (bnx2)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:1a:4b:ce:3e:b4", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

sudo lshw -class network
  *-network UNCLAIMED
       description: Ethernet controller
       product: NetXtreme II BCM5708 Gigabit Ethernet
       vendor: Broadcom Corporation
       physical id: 0
       bus info: pci@0000:03:00.0
       version: 12
       width: 64 bits
       clock: 66MHz
       capabilities: pcix pm vpd msi cap_list
       configuration: latency=64 mingnt=64
       resources: memory:f8000000-f9ffffff memory:80100000-801007ff
  *-network DISABLED
       description: Ethernet interface
       product: NetXtreme II BCM5708 Gigabit Ethernet
       vendor: Broadcom Corporation
       physical id: 0
       bus info: pci@0000:05:00.0
       logical name: eth1
       version: 12
       serial: 00:1a:4b:ce:3e:b4
       capacity: 1Gbit/s
       width: 64 bits
       clock: 66MHz
       capabilities: pcix pm vpd msi bus_master cap_list rom ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=bnx2 driverversion=2.1.6 duplex=half firmware=bc 1.9.6 latency=64 link=no mingnt=64 multicast=yes port=twisted pair
       resources: irq:19 memory:fa000000-fbffffff memory:80200000-802007ff

lsmod:
Module                  Size  Used by
psmouse                73882  0 
serio_raw              13166  0 
hpilo                  17399  0 
usbhid                 47198  0 
ipmi_si                53548  0 
hid                    95463  1 usbhid
radeon               1015949  1 
ipmi_msghandler        45838  1 ipmi_si
ttm                    76805  1 radeon
drm_kms_helper         42558  1 radeon
i5000_edac             17675  0 
drm                   236330  3 radeon,ttm,drm_kms_helper
i2c_algo_bit           13423  1 radeon
edac_core              53746  3 i5000_edac
i5k_amb                13310  0 
shpchp                 37345  0 
lp                     17799  0 
parport                46562  1 lp
hpsa                   55440  0 
bnx2                   82704  0 
cciss                 110812  3 

dmesg | grep bnx:
[    1.017141] bnx2: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.1.6 (Mar 7, 2011)
[    1.017183] bnx2 0000:03:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[   61.920050] bnx2: Can't load firmware file "bnx2/bnx2-mips-06-6.2.1.fw"
[   61.920089] bnx2 0000:03:00.0: PCI INT A disabled
[   61.920105] bnx2: probe of 0000:03:00.0 failed with error -2
[   61.920143] bnx2 0000:05:00.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[   65.129655] bnx2 0000:05:00.0: eth0: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem fa000000, IRQ 19, node addr 00:1a:4b:ce:3e:b4[/CODE]

dmesg | grep udev:
[    0.979923] udevd[92]: starting version 173
[   64.929336] udevd[302]: starting version 173
[   65.204423] udevd[312]: renamed network interface eth0 to eth1

I will try with the fix from comment #46 and report back.