Statistics of IO using is incorrect

[Sergey Borisov]

Each one second, my program reads /sys/block/$DEVICE/stat.

It substracts previously saved values, then saves current values, so I know the hdd activity.
But sometimes (once in 7 days) I've got too big values for sda (between last and current ones) (for example 5GB/second, 19GB/second, etc...).

sudo lshw -class disk
 *-disk
       description: ATA Disk
       product: SPCC Solid State
       physical id: 0.0.0
       bus info: scsi@2:0.0.0
       logical name: /dev/sda
       version: 08.2
       serial: EB84075517B200427436
       size: 111GiB (120GB)
       capabilities: partitioned partitioned:dos
       configuration: ansiversion=5 logicalsectorsize=512 sectorsize=512 signature=d01f3b8e

Also I've written the script for monitoring the statistics

#!/bin/bash

while (true) do
 str=`date`" $(cat /sys/block/sda/stat)"
 echo $str >> log.txt
 sleep 10;
done

And got this values

Fr Jul 28 07:39:38 MSK 2017 2368466 475339 105223134 1362848 5753154 7692148 661424840 31319540 0 4790052 32687100
Fr Jul 28 07:39:48 MSK 2017 2368466 475339 105223134 1362848 5753170 7692160 661425104 31319548 0 4790060 32687108
Fr Jul 28 07:39:58 MSK 2017 2368466 475339 105223134 1362848 5753174 7692162 661425152 31319556 0 4790068 32687116
Fr Jul 28 07:40:08 MSK 2017 2369133 475339 105228750 1363380 5771079 7694395 706209560 31324696 0 4795572 32692780
Fr Jul 28 07:40:18 MSK 2017 2369133 475339 105228750 1363380 5771087 7694431 706209960 31324708 0 4795584 32692792
Fr Jul 28 07:40:28 MSK 2017 2369133 475339 105228750 1363380 5771091 7694433 706210008 31324716 0 4795592 32692800
Fr Jul 28 07:40:38 MSK 2017 2369133 475339 105228750 1363380 5771244 7694714 706213528 31325052 0 4795604 32693136

You can see that the difference (3,4 lines) is very big between 706209560 and 661425152, my ssd can read 250 Mb/sec, but not 5 Gb/sec.

Also you can use this program for monitoring (g++ -std=c++14 name.cpp)

#include <iostream>
#include <fstream>
#include <regex>
#include <tuple>
#include <chrono>
#include <thread>
#include <cstdint>

const int HDD_READ_POS = 2;
const int HDD_WRITE_POS = 6;
const int UNIX_SECTOR_SIZE = 512;
uint64_t prevRead = static_cast<uint64_t>(0);
uint64_t prevWrite = static_cast<uint64_t>(0);

std::tuple<uint64_t, uint64_t> hddStatus(const std::string &name="sda")
{
    std::ifstream in("/sys/block/"+name+"/stat");

    auto readVal_ = static_cast<uint64_t>(0);
    auto writeVal_= static_cast<uint64_t>(0);

    if ( ! in.is_open() ) {
        return std::tuple<uint64_t, uint64_t> (readVal_, writeVal_);
    }

    std::string line;
    std::regex rgx ( "\\d+" );
    std::regex_token_iterator<std::string::iterator> end;

    while (std::getline(in, line) ){

        std::regex_token_iterator<std::string::iterator> iter( line.begin(), line.end(), rgx, 0 );
        int pos_ = 0 ;

        while ( iter != end ) {

            if ( pos_ == HDD_READ_POS){
                readVal_ = std::stoul( *iter ) ;
            }

            if ( pos_ == HDD_WRITE_POS){
                writeVal_ = std::stoul( *iter ) ;
            }

            ++iter;
            ++pos_;
        }
    }

    return std::tuple<uint64_t, uint64_t> (readVal_, writeVal_);

}

void init()
{

        auto values = hddStatus();
        prevRead = std::get<0>( values ) * UNIX_SECTOR_SIZE;
        prevWrite = std::get<1>( values ) * UNIX_SECTOR_SIZE;

}

int main(int argc, char const *argv[])
{
    init();

        while(true){

            std::ofstream stat("statistics.txt", std::fstream::out | std::fstream::app);
            if ( stat.is_open() ){

            auto values = hddStatus();
            auto read = std::get<0>( values ) * UNIX_SECTOR_SIZE;
            auto write = std::get<1>( values ) * UNIX_SECTOR_SIZE;

             // stat<<"Current Read: "<< read<<" Write: "<<write<<'\n';
            if (read > prevRead){
                stat<<"Diff Read: "<< read - prevRead <<'\n';
                std::cout<<"Diff Read: "<< read - prevRead <<'\n';
            }

            if ( write > prevWrite){
                stat<<"Diff Write: "<<write - prevWrite <<'\n';
                std::cout<<"Diff Write: "<<write - prevWrite <<'\n';
            }

            prevRead = read;
            prevWrite = write;

            std::this_thread::sleep_for(std::chrono::seconds(1));

        }
    }

    return 0;

}

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-4.10.0-32-generic 4.10.0-32.36~16.04.1
ProcVersionSignature: Ubuntu 4.10.0-32.36~16.04.1-generic 4.10.17
Uname: Linux 4.10.0-32-generic x86_64
ApportVersion: 2.20.1-0ubuntu2.10
Architecture: amd64
CurrentDesktop: Unity
Date: Fri Aug 25 13:43:59 2017
InstallationDate: Installed on 2017-04-14 (132 days ago)
InstallationMedia: Ubuntu 16.04.1 LTS "Xenial Xerus" - Release amd64 (20160719)
SourcePackage: linux-hwe
UpgradeStatus: No upgrade log present (probably fresh install)
---
ApportVersion: 2.20.9-0ubuntu7
Architecture: amd64
AudioDevicesInUse:
 USER PID ACCESS COMMAND
 /dev/snd/controlC1: sborisov 1839 F.... pulseaudio
 /dev/snd/controlC0: sborisov 1839 F.... pulseaudio
CurrentDesktop: GNOME
DistroRelease: Ubuntu 18.04
EcryptfsInUse: Yes
HibernationDevice: RESUME=UUID=9a00c526-bbf4-4cf2-ba8c-b71de979be1c
InstallationDate: Installed on 2018-03-27 (48 days ago)
InstallationMedia: Ubuntu 17.10 "Artful Aardvark" - Release amd64 (20180105.1)
IwConfig:
 enp5s0 no wireless extensions.

 lo no wireless extensions.
MachineType: Gigabyte Technology Co., Ltd. P61-DS3-B3
NonfreeKernelModules: nvidia
Package: linux (not installed)
ProcFB: 0 VESA VGA
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-4.15.0-20-generic root=UUID=40bed35c-59aa-4efd-80d9-f0cde48413f7 ro quiet splash vt.handoff=1
ProcVersionSignature: Ubuntu 4.15.0-20.21-generic 4.15.17
RelatedPackageVersions:
 linux-restricted-modules-4.15.0-20-generic N/A
 linux-backports-modules-4.15.0-20-generic N/A
 linux-firmware 1.173
RfKill:

Tags: bionic
Uname: Linux 4.15.0-20-generic x86_64
UpgradeStatus: Upgraded to bionic on 2018-04-27 (17 days ago)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo vboxusers wireshark
_MarkForUpload: True
dmi.bios.date: 05/15/2011
dmi.bios.vendor: Award Software International, Inc.
dmi.bios.version: F2
dmi.board.name: P61-DS3-B3
dmi.board.vendor: Gigabyte Technology Co., Ltd.
dmi.board.version: x.x
dmi.chassis.type: 3
dmi.chassis.vendor: Gigabyte Technology Co., Ltd.
dmi.modalias: dmi:bvnAwardSoftwareInternational,Inc.:bvrF2:bd05/15/2011:svnGigabyteTechnologyCo.,Ltd.:pnP61-DS3-B3:pvr:rvnGigabyteTechnologyCo.,Ltd.:rnP61-DS3-B3:rvrx.x:cvnGigabyteTechnologyCo.,Ltd.:ct3:cvr:
dmi.product.name: P61-DS3-B3
dmi.sys.vendor: Gigabyte Technology Co., Ltd.

[Sergey Borisov (borisovs)]

I've used script
#!/bin/bash

while (true) do
        str=`date`" $(cat /sys/block/sda/stat)"
        echo $str >> log.txt
        sleep 1;
done

Reproduced in:
Linux vm145 4.10.0-35-generic #39~16.04.1-Ubuntu SMP Wed Sep 13 09:02:42 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenia

The difference of numbers of sectors written between line 30 and 29 in log.txt is very big and in gigabytes equal 153Gb/sec

(876330784-554468984)*512/(1024*1024*1024)

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1713029

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Sergey Borisov (borisovs)]

apport information

[Sergey Borisov (borisovs)]

apport information

[Sergey Borisov (borisovs)]

apport information

[Sergey Borisov (borisovs)]

apport information

[Sergey Borisov (borisovs)]

apport information

[Sergey Borisov (borisovs)]

apport information

[Sergey Borisov (borisovs)]

apport information

[Sergey Borisov (borisovs)]

apport information

[Sergey Borisov (borisovs)]

apport information

[Sergey Borisov (borisovs)]

apport information

[Sergey Borisov (borisovs)]

apport information

[Sergey Borisov (borisovs)]

apport information

[Sergey Borisov (borisovs)]

apport information