Bug #1822455 “Allow installation of offline maps from app” : Bugs : uNav

Revision history for this message

costales (costales) wrote on 2019-03-31:

#1

Thanks for the feedback.
I would like to implement it, but this need vector maps and local routing library in specific language, then it's not easy :(
I'll keep the bug as reminder.
A hug!

Changed in unav:
status:	New → Triaged

Revision history for this message

Marco Valenti (marco-valenti71) wrote on 2019-09-17:

#2

I tried to install offline maps but without success. It's not 'user friendly' for an unskilled like me. So the chance to select the map directly from the App would be really appreciated. I'm quite happy about the uNAV and aware that is difficult to improve it.

p.s.: I'd like to know, if it's not possibile to use uNAV offline because of a technical issue/limit or because is too difficult/long to develop this option?

Revision history for this message

costales (costales) wrote on 2019-09-17: Re: [Bug 1822455] Re: Allow installation of offline maps from app

#3

Hi Marco,

Yes, I'm sorry, but I can't implement an option for downloading from the
app, it would crazy for the tile servers as OSM.

The unique way would be to implement to read vector maps (.pbf) and load
them.

I have to think on that and test a few things.

A hug,
--
Costales.

Revision history for this message

Michael Rupp (wnymathguy-1) wrote on 2019-10-28:

#4

Ha, like the way the multi-trillion-dollar Google company lets users download offline maps "for free".

I was immediately interested in the option of offline maps.

I tried the method outlined in https://unav.me/offline/ and I thought it was going well, but I seemed to get a full set of empty map tile images. It could have been a PEBKAC type error. I had https://www.openstreetmap.org/export#map=13/42.9726/-78.7479 as my starting point with the box of {(43.0267, -78.8284), (42.9183, -78.6674)}. Maybe that was too big to begin with but to make it worse I put all zoom levels in the comma-separated list. After getting it all on the phone I just had blank map tiles. Instead of Mapnik, I used OpenStreetBrowser (Europe) because it was the first one that seemed to deliver something.

I'm about to try the method shown in this how-to https://openmaptiles.org/docs/ and I'll comment again if that works well.

Revision history for this message

Michael Rupp (wnymathguy-1) wrote on 2019-10-29:

#5

Download full text (3.4 KiB)

Just an update to my #4 comment:

I wish I had some idea what I was doing, but instead, I can only follow recipes and guess at things I don't know yet. I did successfully launch that Docker type map server from OpenMapTiles on my local machine and it was almost as easy as they showed in the videos found through https://openmaptiles.com/server/
https://www.youtube.com/watch?v=fJnZn1IpQHE
My only snag was not putting `sudo` in front of the `docker run --rm -it -v $(pwd):/data -p 8080:80 klokantech/openmaptiles-server` command I copied and pasted from the https://openmaptiles.org/docs/ guide.

I tried again using the JTileDownloader app with the"Output Zoom Level:" selection of 12, the "Output Zoom Levels (ex. 12,13,14):" textbox set to "8-18", option Mapnik selected and in the "Alt. Tileserver:" textbox I tried "http://localhost:8080/" which didn't work, then I tried "http://localhost:8080/styles/osm-bright/?raster#12/" which seemed to work. I transferred over the files to the phone as shown in https://unav.me/offline/ but still the offline maps just shows that pattern of an unplugged electric powercord.

I took a look inside one of the PNG files that came out of the "Download Tiles" operation using the Bluefish Editor. The contents of ~/maps/18/73677/96305.png is:

```
<!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>OSM Bright - TileServer GL</title>
  <link rel="stylesheet" type="text/css" href="/mapbox-gl.css" />
  <script src="/mapbox-gl.js"></script>
  <link rel="stylesheet" type="text/css" href="/mapbox.css" />
  <script src="/mapbox.js"></script>
  <script src="/leaflet-hash.js"></script>
  <style>
    body { margin:0; padding:0; }
    #map { position:absolute; top:0; bottom:0; width:100%; }
  </style>
</head>
<body>
  <h1 style="display:none;">OSM Bright</h1>
  <div id='map'></div>
  <script>
    var q = (location.search || '').substr(1).split('&');
    var preference =
      q.indexOf('vector') >= 0 ? 'vector' :
        (q.indexOf('raster') >= 0 ? 'raster' :
          (mapboxgl.supported() ? 'vector' : 'raster'));
    if (preference == 'vector') {
      mapboxgl.setRTLTextPlugin('/mapbox-gl-rtl-text.js');
      var map = new mapboxgl.Map({
        container: 'map',
        style: '/styles/osm-bright/style.json',
        hash: true
      });
      map.addControl(new mapboxgl.NavigationControl());
    } else {
      var map = L.mapbox.map('map', '/styles/osm-bright.json', { zoomControl: false });
      new L.Control.Zoom({ position: 'topright' }).addTo(map);
      setTimeout(function() {
        new L.Hash(map);
      }, 0);
    }
  </script>
</body>
</html>
```

Is that what a map tile is supposed to look like? I thought tiles were a set of graphic image files in raster format that the software loaded and stitched together as the birds-eye-window slid across the plane. I clicked the "Export TileList" button on JTileDownloader and see "http://localhost:8080/styles/osm-bright/?raster#12/18/73677/96305.png" in the list. When I browse to it I get forwarded to "http://localhost:8080/styles/osm-bright/?raster#7/42.535/-75.194" and it's showing the whole...

Just an update to my #4 comment:

I wish I had some idea what I was doing, but instead, I can only follow recipes and guess at things I don't know yet. I did successfully launch that Docker type map server from OpenMapTiles on my local machine and it was almost as easy as they showed in the videos found through https://openmaptiles.com/server/ 
 https://www.youtube.com/watch?v=fJnZn1IpQHE 
My only snag was not putting `sudo` in front of the `docker run --rm -it -v $(pwd):/data -p 8080:80 klokantech/openmaptiles-server` command I copied and pasted from the https://openmaptiles.org/docs/ guide.

I tried again using the JTileDownloader app with the"Output Zoom Level:" selection of 12, the "Output Zoom Levels (ex. 12,13,14):" textbox set to "8-18", option Mapnik selected and in the "Alt. Tileserver:" textbox I tried "http://localhost:8080/" which didn't work, then I tried "http://localhost:8080/styles/osm-bright/?raster#12/" which seemed to work. I transferred over the files to the phone as shown in https://unav.me/offline/ but still the offline maps just shows that pattern of an unplugged electric powercord.

I took a look inside one of the PNG files that came out of the "Download Tiles" operation using the Bluefish Editor. The contents of ~/maps/18/73677/96305.png is:

```
<!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>OSM Bright - TileServer GL</title>
  <link rel="stylesheet" type="text/css" href="/mapbox-gl.css" />
  <script src="/mapbox-gl.js"></script>
  <link rel="stylesheet" type="text/css" href="/mapbox.css" />
  <script src="/mapbox.js"></script>
  <script src="/leaflet-hash.js"></script>
  <style>
    body { margin:0; padding:0; }
    #map { position:absolute; top:0; bottom:0; width:100%; }
  </style>
</head>
<body>
  <h1 style="display:none;">OSM Bright</h1>
  <div id='map'></div>
  <script>
    var q = (location.search || '').substr(1).split('&');
    var preference =
      q.indexOf('vector') >= 0 ? 'vector' :
        (q.indexOf('raster') >= 0 ? 'raster' :
          (mapboxgl.supported() ? 'vector' : 'raster'));
    if (preference == 'vector') {
      mapboxgl.setRTLTextPlugin('/mapbox-gl-rtl-text.js');
      var map = new mapboxgl.Map({
        container: 'map',
        style: '/styles/osm-bright/style.json',
        hash: true
      });
      map.addControl(new mapboxgl.NavigationControl());
    } else {
      var map = L.mapbox.map('map', '/styles/osm-bright.json', { zoomControl: false });
      new L.Control.Zoom({ position: 'topright' }).addTo(map);
      setTimeout(function() {
        new L.Hash(map);
      }, 0);
    }
  </script>
</body>
</html>
```

Is that what a map tile is supposed to look like? I thought tiles were a set of graphic image files in raster format that the software loaded and stitched together as the birds-eye-window slid across the plane. I clicked the "Export TileList" button on JTileDownloader and see "http://localhost:8080/styles/osm-bright/?raster#12/18/73677/96305.png" in the list. When I browse to it I get forwarded to "http://localhost:8080/styles/osm-bright/?raster#7/42.535/-75.194" and it's showing the whole New York, USA image tile I selected during the initial set-up phase on localhost:8080.

I think I'm going to try it one more time before giving up but I'll use "http://localhost:8080/styles/osm-bright/" where I had "http://localhost:8080/styles/osm-bright/?raster#12/" before.

Revision history for this message

costales (costales) wrote on 2019-10-29:

#6

Hi,
I'm afraid that all servers listed in jTileDownloader are banned,
because this method is so aggressive for the tiles map server.
You should enter a custom URL with another free OSM server, which
allow you to download so much tiles.
As I saw you control Docker, you could install a local OSM server:
https://github.com/Overv/openstreetmap-tile-server
The final tiles are image files in .png format.
A hug
--
Costales.

Revision history for this message

Michael Rupp (wnymathguy-1) wrote on 2019-10-30:

#7

It worked! Using the instructions on https://openmaptiles.com/server/ and using the local URL of http://localhost:8080/styles/osm-bright/ in the JTileDownloader apps "Alt. Tileserver:" textbox worked. I guess I could have done it for osm-bright/ positron/ klokantech-basic/ or dark-matter/ since all four map styles come with that docker thing.

The preview in JTileDownloader was showing map images for the first time while it was making the folders and PNG's. Because it was working so good, I tried a few things you advised against on the https://unav.me/offline/ page. I tried to box in a very large part of my home state. I saw it was going to take 17 days or more to finish. So glad you have an abort button on that program. ;)

I experimented with settings and wound up with Output Zoom Levels = 8-17, which with my bounding coordinates was making 432,580 tiles. That ran about 1.3 days to complete. I'm now waiting about 4 hours for them to go onto the Nexus 7 hoping I didn't try to push an elephant into a shoebox. :)

I hope it works, I cant wait to try offline maps while driving!!!

Revision history for this message

costales (costales) wrote on 2019-10-30:

#8

Hi,
I'm glad it worked for you :))
Remember that you'll only need a few zooms:
zoom 16,17 for cities and a zoom 12,13 for not cities
Best regards.

Revision history for this message

Michael Rupp (wnymathguy-1) wrote on 2019-10-30:

#9

Download full text (3.6 KiB)

@costales I'm writing from a position of ignorance with what I say below, so I won't have my feelings hurt if this is a terrible idea, but...

Is there a way to make a UBports specific script/program that took no user input but ran that Docker-based personal server shown at https://github.com/Overv/openstreetmap-tile-server on the mobile device just for long enough to make tiles go into the ~/.local/share/navigator.costales folder?
https://github.com/Overv/openstreetmap-tile-server#preserving-rendered-tiles

And maybe did cron jobs to update the tiles occasionally?

The imagined UBports specific script/program above could use the current GPS to select what region to bring into the device as well as set a maximum reasonable boundary around that center for the tiles being output to uNav's offline maps folder.
e.g.: adjust boundaries in order to get a set that has a total number of tiles smaller than x-many. (mostly because users like me will ask for unreasonable quantities if it's free to do so)

http://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox=-6.266747,54.447649,-5.599081,54.771571

Changing topics a bit here:
My most vital use case of offline-maps with Google Maps is for an extended trip where I fear having no signal during rural or farmland sections of the drive. I will use Google Maps to pull in overlapping segments of the drive at the largest zoom out preview that their app lets me do it for along what I know to be the intended cross-country trip. This is excessively more data than I need. It's so wasteful but I don't have a better way to ensure the drive will be successful.

My secondary use case is my hometown map (about a 10-mile radius around a home address), just to keep the total data consumption to a minimum while on a cellular network connection because it costs money per byte transferred. Most of the time I don't need the map program, but I let it help me to remember/confirm the best route under confusion or to re-route when road problems along my intended path arise.

Just "thinking out loud" here. It would probably be impossible to make a program that could convert a driving route into a minimum set of necessary offline tiles that would accommodate wrong turns, detours, and finding food/fuel/lodging/repairs along the way. That would be fantastic though!

In my youth, long before personal computers, we would go to the AAA office (https://www.aaa.com/) and get a "TripTik" flipbook made by a clerk while we waited. It was only a 15-minute ordeal to have a book made which would take you across the country.
https://pearlsoftravelwisdom.boardingarea.com/2014/01/remember-triptix/
http://wisechapter3.com/2015/04/20/marketing-triptik-why-every-company-needs-one/
Somehow they would be able to quickly go from two endpoints to a selection of ordered sheets to put into the flip-page binding. The narrow books had all pages with North at the top. They did it with paper so maybe that human algorithm and system of identifying pre-printed sheets would solve such a problem of building a minimum set of tiles for an extended length offline trip.

Oh, and the AAA clerk would also hand you the regional paper maps that your TripTik w...

@costales I'm writing from a position of ignorance with what I say below, so I won't have my feelings hurt if this is a terrible idea, but...

Is there a way to make a UBports specific script/program that took no user input but ran that Docker-based personal server shown at https://github.com/Overv/openstreetmap-tile-server on the mobile device just for long enough to make tiles go into the ~/.local/share/navigator.costales folder? 
https://github.com/Overv/openstreetmap-tile-server#preserving-rendered-tiles

And maybe did cron jobs to update the tiles occasionally?

The imagined UBports specific script/program above could use the current GPS to select what region to bring into the device as well as set a maximum reasonable boundary around that center for the tiles being output to uNav's offline maps folder. 
e.g.: adjust boundaries in order to get a set that has a total number of tiles smaller than x-many. (mostly because users like me will ask for unreasonable quantities if it's free to do so)

http://tools.geofabrik.de/calc/#type=geofabrik_standard&bbox=-6.266747,54.447649,-5.599081,54.771571

Changing topics a bit here:
My most vital use case of offline-maps with Google Maps is for an extended trip where I fear having no signal during rural or farmland sections of the drive. I will use Google Maps to pull in overlapping segments of the drive at the largest zoom out preview that their app lets me do it for along what I know to be the intended cross-country trip. This is excessively more data than I need. It's so wasteful but I don't have a better way to ensure the drive will be successful.

My secondary use case is my hometown map (about a 10-mile radius around a home address), just to keep the total data consumption to a minimum while on a cellular network connection because it costs money per byte transferred. Most of the time I don't need the map program, but I let it help me to remember/confirm the best route under confusion or to re-route when road problems along my intended path arise.

Just "thinking out loud" here. It would probably be impossible to make a program that could convert a driving route into a minimum set of necessary offline tiles that would accommodate wrong turns, detours, and finding food/fuel/lodging/repairs along the way. That would be fantastic though!

In my youth, long before personal computers, we would go to the AAA office (https://www.aaa.com/) and get a "TripTik" flipbook made by a clerk while we waited. It was only a 15-minute ordeal to have a book made which would take you across the country. 
https://pearlsoftravelwisdom.boardingarea.com/2014/01/remember-triptix/
http://wisechapter3.com/2015/04/20/marketing-triptik-why-every-company-needs-one/
Somehow they would be able to quickly go from two endpoints to a selection of ordered sheets to put into the flip-page binding. The narrow books had all pages with North at the top. They did it with paper so maybe that human algorithm and system of identifying pre-printed sheets would solve such a problem of building a minimum set of tiles for an extended length offline trip.

Oh, and the AAA clerk would also hand you the regional paper maps that your TripTik went across. So an ideal offline map system would have the zoom level 1-9 tiles of the trip's overview (mostly for when things went wrong), and the zoom level 14-18 tiles containing the roads which the route touches as well as the adjacent 4 tiles on either side of the one the route is on. Or maybe just every zoom level tile that has the route on it, plus the adjacent tiles pertinent to wrong turns and finding food/fuel/lodging/repairs along the way.

Revision history for this message

costales (costales) wrote on 2019-10-31:

#10

Hi,
No, run a OSM server in a phone would be so so so heavy.
Even if the map doesn't load because of your connectivity, you'll see the
blue line and you just have to follow it and the instructions. IMHO is
enough.
Remember that uNav is caching the maps in the current runtime, then you
could check the path and they will load without connectivity.
A hug.
--
Costales.

Revision history for this message

Michael Rupp (wnymathguy-1) wrote on 2019-11-04:

#11

Quick update here. Referring to the #7 reply above, it took until now to finish moving the 434,095 tiles onto the phone via the USB wire. Yes, that was 101 hours of non-stop file transfer. Files were going in fairly quick at the start, but it was slowing at a constant rate. Towards the end, it was going at about a rate of 2 seconds per image file.

It's really nice to see the offline maps in the app. I'll be making it my primary driving assist app from now on.

costales (costales) on 2020-08-04

Changed in unav:
status:	Triaged → In Progress

costales (costales) on 2020-09-12

Changed in unav:
status:	In Progress → Fix Released

uNav

Allow installation of offline maps from app

Bug Description

Other bug subscribers

Remote bug watches