I have been seeing this. 
TL;DR: workaround: delete /var/lib/apt/extended_states 

INVESTIGATION

I tried reinstalling the update-manager

$ sudo apt install --reinstall update-manager

but that didn't help.

SO I moved the apt directory and created a blank one

$ sudo mv /var/lib/apt /var/lib/apt.orig
$ sudo mkdir /var/lib/apt

then I updated apt

$ sudo apt update
Get:1 http://us.archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:3 http://archive.canonical.com/ubuntu xenial InRelease [11.5 kB]           
...
Get:119 http://us.archive.ubuntu.com/ubuntu xenial-security/multiverse amd64 DEP-11 Metadata [212 B]
Get:120 http://us.archive.ubuntu.com/ubuntu xenial-security/multiverse DEP-11 64x64 Icons [29 B]
Fetched 55.2 MB in 1min 13s (748 kB/s)                                         
AppStream cache update completed, but some metadata was ignored due to errors.
Reading package lists... Done
Building dependency tree... Done
20 packages can be upgraded. Run 'apt list --upgradable' to see them.
$ 


Now despite the complaint above ("AppStream cache update completed, but some metadata was ignored due to errors.") software-updater works; it lists the upgradeable packages.


The only subdirectory that got recreated under /var/lib/apt is /var/lib/apt/lists (and its subdirectory)


SO I blew away the directory and copied everything except the lists directory and updated again

$ sudo rm -R /var/lib/apt
$ sudo cp -R /var/lib/apt.orig /var/lib/apt
$ sudo rm -R /var/lib/apt/lists/
$ sudo apt update
Get:1 http://us.archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [107 kB]                                 
...
Get:120 http://us.archive.ubuntu.com/ubuntu xenial-security/multiverse DEP-11 64x64 Icons [29 B]               
Fetched 55.2 MB in 1min 13s (747 kB/s)                                                                         
AppStream cache update completed, but some metadata was ignored due to errors.
Reading package lists... Done
Building dependency tree       
Reading state information... Done
20 packages can be upgraded. Run 'apt list --upgradable' to see them.
$ 

I got the same error, HOWEVER Software Updater is blank again.

I tried deleting the /var/lib/apt/periodic directory and updating

$ sudo rm -R /var/lib/apt/periodic/
$ sudo mkdir /var/lib/apt/periodic
$ sudo apt update
Hit:1 http://archive.canonical.com/ubuntu xenial InRelease
Hit:2 http://us.archive.ubuntu.com/ubuntu xenial InRelease                                            
Hit:3 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease                                    
Hit:4 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease                                         
Hit:6 http://us.archive.ubuntu.com/ubuntu xenial-security InRelease                                            
Hit:7 http://security.ubuntu.com/ubuntu xenial-security InRelease                                              
Hit:5 http://screenshots.getdeb.net xenial-getdeb InRelease                                                    
Reading package lists... Done                     
Building dependency tree       
Reading state information... Done
20 packages can be upgraded. Run 'apt list --upgradable' to see them.
$ ls /var/lib/apt/periodic
update-success-stamp
$ 

Now I no longer see a metadata error from apt, but software-updater is still blank.

SO I delete the extended_states file

$ sudo rm /var/lib/apt/extended_states 
$ sudo apt update
Hit:1 http://archive.canonical.com/ubuntu xenial InRelease
Hit:2 http://us.archive.ubuntu.com/ubuntu xenial InRelease                                       
Hit:3 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease                                             
Hit:4 http://security.ubuntu.com/ubuntu xenial-security InRelease                                              
Hit:5 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease                           
Hit:6 http://us.archive.ubuntu.com/ubuntu xenial-security InRelease      
Hit:7 http://screenshots.getdeb.net xenial-getdeb InRelease              
Reading package lists... Done
Building dependency tree... Done
20 packages can be upgraded. Run 'apt list --upgradable' to see them.
$ 

NOW software-updater is showing the upgradeable files


SO just to be sure, I copied the original stuff back and deleted JUST the extended_states file

$ sudo rm -R /var/lib/apt/
$ sudo cp -R /var/lib/apt.orig/ /var/lib/apt/
$ sudo rm /var/lib/apt/extended_states 
$ 


AND software_updater is now showing the upgradeable packages. There's the workaround. Are there consequences of having no extended_states file?