Ubuntu

no way to log network connections and traffic of applications (processes)

Reported by nick rundy on 2011-08-04
54
This bug affects 13 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Undecided
Unassigned
rsyslog (Ubuntu)
Undecided
Unassigned

Bug Description

The kernel / netfilter is lacking the the ability to identify the local origin of network conection. (see comments)

This lack was first reported against loging facilities, but tracked down to kernel disability.

Please foreward / reassign to handle this as appropriate.

original post:
---
The gnome "Log File Viewer" does not log the Process Name (or Application Name) that generated the log item. For example, if an outbound internet connection is blocked and this event is logged, only the "ID" (i.e., PID) is shown in the report. But the PID is useless because it is ephemeral and does not live past the session. Users are left with no way to learn what Application or Process was responsible for generating the log item.

The "Process Name" should be listed in log items instead of the PID.

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: gnome-utils 2.30.0-0ubuntu1
ProcVersionSignature: Ubuntu 2.6.32-33.71-generic 2.6.32.41+drm33.18
Uname: Linux 2.6.32-33-generic i686
Architecture: i386
Date: Thu Aug 4 08:05:47 2011
ExecutablePath: /usr/bin/gnome-system-log
InstallationMedia: Ubuntu 10.04.1 LTS "Lucid Lynx" - Release i386 (20100816.1)
ProcEnviron:
 LC_TIME=en_GB.UTF-8
 LANG=en_US.utf8
 SHELL=/bin/bash
SourcePackage: gnome-utils
XsessionErrors: (polkit-gnome-authentication-agent-1:1444): GLib-CRITICAL **: g_once_init_leave: assertion `initialization_value != 0' failed

nick rundy (nrundy) wrote :
Changed in gnome-utils (Ubuntu):
status: New → Confirmed
affects: gnome-utils (Ubuntu) → rsyslog (Ubuntu)
Sebastien Bacher (seb128) wrote :

the log viewer only displays system logs, it's not the software recording those

Jamie Strandboge (jdstrand) wrote :

It isn't the syslogd's responsibility to do a lookup of the pid, it is the application and the application is the one specifying the pid anyway.

Based on the example given, it sounds like what is desired is for the firewall to log the pid of the application that generated the request. This is the kernel's responsibility via netfilter, and netfilter is controlled via iptables (and up above maybe another tool like ufw). However, iptables does not support this (see 'man iptables' for more information). It used to have the '--cmd-owner' option, but this was removed long ago because it was deemed unfixably broken.

Changed in rsyslog (Ubuntu):
status: Confirmed → Won't Fix
nick rundy (nrundy) wrote :

@Jamie Strandboge, is there anything that can be done to get this capability in Ubuntu? Perhaps some other solution can be created or put on the planning board? A lot of people want to have a log of the outgoing internet connections of applications. Unless users are willing to sit and stare at the monitor while connections occur every second they are on their computer, users have no way of learning what apps are making outgoing connections on their computers. There's got to be some way give Ubuntu users this capability.

The capability to log "process names" has been requested by numerous users over the years, here's some links:

Ubuntu Brainstorm: http://brainstorm.ubuntu.com/idea/28288/

Superuser: http://superuser.com/questions/34782/with-linux-iptables-is-it-possible-to-log-the-process-command-name-that-initiat

Robbie Williamson (robbiew) wrote :

nethogs and netstat can connect pids to program names, so in theory someone could add this functionality to iptables. You could also setup a cron job to regularly log netstat output for all network connections to a file. For example, the following command ran as root will timestamp and log all network connections every 5 seconds to log.txt:
   # while [ 1 == 1 ]; do date >> log.txt; netstat -pn -A inet --wide >> log.txt; sleep 5 ; done
You can run without root privledge, however process you don't own won't be included. Hope this helps a little.

Jamie Strandboge (jdstrand) wrote :

To be clear, the kernel is doing the logging, not iptables. Iptables configures netfilter, the part of the kernel that does all this.

That said, the pid is not logged by the kernel so there is nothing to cross-reference in /proc (which is all netstat is doing). Eg:
Jan 26 15:36:57 localhost kernel: [21281.600175] [UFW BLOCK] IN= OUT=wlan0 SRC=10.0.0.2 DST=91.189.90.41 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=37870 DF PROTO=TCP SPT=38053 DPT=23 WINDOW=14600 RES=0x00 SYN URGP=0

What Robbie suggests could kinda work, but because it is polling a snapshot in time, it is not really a satisfactory solution for people wanting to continually map outgoing connections to a program name. The superuser.com site has tips on how to write a program that could poll various things in /proc, but this is not this bug. This bug is asking for logging the process name for network packets (the PID could in theory satisfy this, but there is still the problem of the polling interval).

Iptables does have a --log-uid option, but that doesn't get us all the way there:
Jan 26 15:45:54 localhost kernel: [21818.931215] [TEST] IN= OUT=wlan0 SRC=10.0.0.2 DST=91.189.89.88 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=24075 DF PROTO=TCP SPT=52517 DPT=23 WINDOW=14600 RES=0x00 SYN URGP=0 UID=1000 GID=1000

Most of the desire surrounding this sort of logging has to do with application firewalls where you have specific firewall rules based on the application producing the network traffic. There used to be a --cmd-owner option for iptables that would configure the firewall to more of less do what you wanted but the kernel as of 2.6.14 stopped supporting this. The Debian bug report referring to the removal of --cmd-owner is: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=492284

and the kernel commit is: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=34b4a4a624bafe089107966a6c56d2a1aca026d4

The kernel commit states:
"[NETFILTER]: Remove tasklist_lock abuse in ipt{,6}owner

Rip out cmd/sid/pid matching since its unfixable broken and stands in the
way of locking changes to tasklist_lock."

LSMs such as AppArmor could be used to help with application firewalls, but at this time AppArmor network mediation is very coarse-grained. Support is planned for better network mediation-- the first cut will allow specifying network rules by port. After that we would tie in with secmark which will allow us to filter based on the contents of the secmark. Both of these would improve the situation for application firewalls to varying degrees, and a creative complain-mode global AppArmor policy could in theory be used to show which applications are making the outgoing connections.

Larry Wilson (larry2311) wrote :

Must say I agree with Nick Rundy's comments on 2012-01-25 (above).

We do need a way to track what application is initiating network traffic, that can (or is) logged to the syslog.

Can we get this progressed to the kernel and iptables teams as a feature request.

ceg (ceg) on 2012-04-20
summary: - Log File Viewer does not log "Process Name"
+ no way to log network connections and traffic of applications
ceg (ceg) on 2012-04-20
description: updated
description: updated
summary: no way to log network connections and traffic of applications
+ (processes)

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 820895

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Changed in linux (Ubuntu):
status: New → Incomplete
no longer affects: iptables (Ubuntu)
ceg (ceg) wrote :

This report does not need any logs attatched to be complete and confirmed.

Changed in linux (Ubuntu):
status: Incomplete → Confirmed
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.