when multiple images are requested for download all the temporal activities fail due to heartbeat timeout

Bug #2057459 reported by Jacopo Rota
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Fix Committed
High
Jacopo Rota
3.5
Fix Committed
High
Jacopo Rota

Bug Description

In 3.5.0 when you donwload multiple images at the same time the network is saturated and the activities are killed due to the heartbeat timeout set to 10 seconds. In particular

```
REPORT_INTERVAL = timedelta(seconds=10)
HEARTBEAT_TIMEOUT = timedelta(seconds=10)
DISK_TIMEOUT = timedelta(minutes=15)
DOWNLOAD_TIMEOUT = timedelta(hours=2)
MAX_SOURCES = 5
CHUNK_SIZE = 5 * (2**20) # 5 MB

            async with self.session.get(
                url, verify_ssl=False, chunked=True
            ) as response, lfile.astore(autocommit=False) as store:
                response.raise_for_status()
                last_update = datetime.now()
                async for data in response.content.iter_chunked(CHUNK_SIZE):
                    activity.heartbeat()
```

we assume to be able to download 5MB of data every 10 seconds, which is not correct in environments with slow network connectivity or when multiple images are downloaded at the same time and the network is saturated.

As a result, the images are never downloaded

Related branches

Jacopo Rota (r00ta)
Changed in maas:
assignee: nobody → Jacopo Rota (r00ta)
Changed in maas:
status: Triaged → Fix Committed
Changed in maas:
milestone: 3.5.0 → 3.6.0
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.