https://github.com/uktrade/mbtiles-s3-server

Skip to content
 
Sign up

  * Product
      + Features
      + Mobile
      + Actions
      + Codespaces
      + Copilot
      + Packages
      + Security
      + Code review
      + Issues
      + Integrations
      + GitHub Sponsors
      + Customer stories
  * Team
  * Enterprise
  * Explore
      + Explore GitHub
      + Learn and contribute
      + Topics
      + Collections
      + Trending
      + Skills
      + GitHub Sponsors
      + Open source guides
      + Connect with others
      + The ReadME Project
      + Events
      + Community forum
      + GitHub Education
      + GitHub Stars program
  * Marketplace
  * Pricing
      + Plans
      + Compare plans
      + Contact Sales
      + Education

[                    ] 

  *  
    #
    In this repository All GitHub |
    Jump to |

  * No suggested jump to results

  *  
    #
    In this repository All GitHub |
    Jump to |
  *  
    #
    In this organization All GitHub |
    Jump to |
  *  
    #
    In this repository All GitHub |
    Jump to |

Sign in
Sign up
{{ message }}
uktrade / mbtiles-s3-server Public

  * Notifications
  * Fork 0
  * Star 85

Python server to on-the-fly extract and serve vector tiles from an
mbtiles file on S3

License

MIT license
85 stars 0 forks
Star
Notifications

  * Code
  * Issues 0
  * Pull requests 0
  * Actions
  * Projects 0
  * Wiki
  * Security
  * Insights

More

  * Code
  * Issues
  * Pull requests
  * Actions
  * Projects
  * Wiki
  * Security
  * Insights

uktrade/mbtiles-s3-server

This commit does not belong to any branch on this repository, and may
belong to a fork outside of the repository.
main
Switch branches/tags
[                    ]
Branches Tags
Could not load branches
Nothing to show
{{ refName }} default View all branches
Could not load tags
Nothing to show
{{ refName }} default
View all tags
1 branch 11 tags
Code

Latest commit

@michalc
michalc build(release): v0.0.11
...
8d70b52 Jul 10, 2022
build(release): v0.0.11
8d70b52

Git stats

  * 123 commits

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
.circleci
tests: refactor and put test-related files in separate directory
Jul 9, 2022
mbtiles_s3_server
perf: use single sqlite-s3-query context for server
Jul 10, 2022
test
feat: respect x-forwarded-proto
Jul 9, 2022
.coveragerc
build(ci): exclude setup.py in code coverage
Jul 7, 2022
.gitattributes
Initial commit
Jul 6, 2022
.gitignore
Initial commit
Jul 6, 2022
.pre-commit-config.yaml
feat: include styles
Jul 7, 2022
LICENSE
Initial commit
Jul 6, 2022
README.md
docs: fix typo
Jul 10, 2022
example.html
docs: ensure to have maxzoom in example
Jul 10, 2022
requirements.in
feat: combine fonts
Jul 9, 2022
requirements.txt
perf: use single sqlite-s3-query context for server
Jul 10, 2022
requirements_test.in
tests: save tiles file as part of tests
Jul 6, 2022
requirements_test.txt
perf: use single sqlite-s3-query context for server
Jul 10, 2022
setup.py
build(release): v0.0.11
Jul 10, 2022
View code
mbtiles-s3-server Installation Example usage For the curious,
advanced, or developers of this server itself Licenses

README.md

 mbtiles-s3-server CircleCI Test Coverage

Python server to on-the-fly extract and serve vector tiles from
mbtiles files on S3. Javascript, maps styles, fonts, and sprites are
included so you can get setup quickly, especially with OpenMapTiles
mbtiles files, but these are not required to be used. Versioning must
be enabled on the underlying S3 bucket.

mbtiles-s3-server-example.mov

Video showing expected performance. The server hosting the HTML is
running locally, but mbtiles-s3-server is running on AWS,
communicating with a real S3 bucket.

    Note: sprites (icons and textures on maps) are not yet served

 Installation

pip install mbtiles-s3-server

The libsqlite3 binary library is also required, but this is typically
already installed on most systems. The earliest version of libsqlite3
known to work is 2012-12-12 (3.7.15).

 Example usage

 1. Create or obtain an mbtiles file, for example from https://
    openmaptiles.org/.

 2. (Optional) Create another file with a page size of 65536 bytes
    (64KiB) using VACUUM or VACUUM INTO

    sqlite3 my-map.mbtiles "PRAGMA page_size=65536; VACUUM INTO 'mytiles-65536.mbtiles';"

    While this step is optional, performance with default mbtiles
    files that have smaller page sizes can be horrible to the point
    of being unusable - loading of a single tile can take many
    seconds. Performance of this server is limited by the latency of
    calls to S3, and this step effectively reduces the number of
    calls to S3 per map tile.

    Note both VACUUM and VACUUM INTO need disk space since they
    create another database which is approximately the size of the
    original file.

 3. Upload this file to S3, for example to https://
    my-bucket.s3.eu-west-2.amazonaws.com/mytiles-65536.mbtiles

 4. Ensure to have a IAM user that has s3:GetObject and
    s3:GetObjectVersion permissions on this S3 object, for example

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "",
                "Effect": "Allow",
                "Action": [
                    "s3:GetObject",
                    "s3:GetObjectVersion"
                ],
                "Resource": [
                    "arn:aws:s3:::my-bucket/mytiles-65536.mbtiles"
                ]
            }
        ]
    }

 5. Start this server, configured with the location of this object
    and credentials for this user - it's configured using environment
    variables. You can assign the tiles file any version you like, in
    this case, 1.0.0.

    PORT=8080 \
    MBTILES__1__URL=https://my-bucket.s3.eu-west-2.amazonaws.com/mytiles-65536.mbtiles \
    MBTILES__1__IDENTIFIER=mytiles \
    MBTILES__1__VERSION=1.0.0 \
    AWS_REGION=eu-west-2 \
    AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \
    AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY \
    HTTP_ACCESS_CONTROL_ALLOW_ORIGIN="*" \
        python -m mbtiles_s3_server

    The above is only to show what environment variables are needed -
    usually credentials should be avoided in shell commands. When
    developing locally, instead of specifying the AWS_* variables,
    consider using aws-vault, which populates them for you.

    For example if you have a local AWS profile called tiles, you can
    run the below.

    PORT=8080 \
    MBTILES__1__URL=https://my-bucket.s3.eu-west-2.amazonaws.com/mytiles-65536.mbtiles \
    MBTILES__1__IDENTIFIER=mytiles \
    MBTILES__1__VERSION=1.0.0 \
    HTTP_ACCESS_CONTROL_ALLOW_ORIGIN="*" \
        aws-vault exec tiles -- python -m mbtiles_s3_server

 6. On your user-facing site, include HTML that loads these tiles
    from this server, for example to load maps from a server started
    as above running locally serving OpenMapTiles

     <!DOCTYPE html>
     <html>
       <head>
         <meta charset="utf-8">
         <title>Example map</title>
         <meta name="viewport" content="initial-scale=1,maximum-scale=1,user-scalable=no">
         <script src="http://localhost:8080/v1/static/maplibre-gl@2.1.9/maplibre-gl.js"></script>
         <link href="http://localhost:8080/v1/static/maplibre-gl@2.1.9/maplibre-gl.css" rel="stylesheet">
         <style>
           body, html, #map {margin: 0; padding: 0; height: 100%; width: 100%}
         </style>
       </head>
       <body>
         <div id="map"></div>
         <script>
         var map = new maplibregl.Map({
             container: 'map',
             style: 'http://localhost:8080/v1/styles/osm-bright-gl-style@1.0.0/style.json?fonts=fonts-gl@1.0.0&tiles=mytiles@1.0.0',
             center: [0, 0],
             zoom: 1,
             maxZoom: 14,
             attributionControl: false
         });
         map.addControl(new maplibregl.AttributionControl({
             customAttribution: '<a href="https://openmaptiles.org/">(c) OpenMapTiles</a> <a href="https://www.openstreetmap.org/copyright">(c) OpenStreetMap contributors</a>'
         }), 'bottom-right');
         </script>
       </body>
     </html>

    This HTML is included in this repository in example.html. A
    simple server can be started to view it by

    python -m http.server 8081 --bind 127.0.0.1

    and going to http://localhost:8081/example.html

 For the curious, advanced, or developers of this server itself

Hosting your own vector map tiles to show them in a browser requires
quite a few components:

 1. JavaScript and CSS

    A Javascript and CSS library, such as MapLibre GL, and your own
    code to run this library, pointing it to a style file

 2. Style file

    A JSON file that defines how the library should style the map
    tiles, and where it should find the map tiles, glyphs (fonts),
    and the sprite

 3. Glyphs (fonts)

    Sets of fonts; different fonts can be used for different labels
    and zoom levels, as defined in the Style file

 4. Sprite

    A single JSON index file and a single PNG file; the JSON file
    contains the offsets and sizes of images within the single PNG
    file

 5. Vector map tiles

    A set of often hundreds of thousands of tiles each covering a
    different location and different zoom level. These can be
    distributed as a single mbtiles file, but this is not the format
    that the Javascript library accepts. This on-the-fly conversion
    from the mbtiles file to tiles is the main feature of this
    server.

 Licenses

The code of the server itself is released under the MIT license.
However, several components included in mbtiles_s3_server/vendor/ are
released under different licenses.

About

Python server to on-the-fly extract and serve vector tiles from an
mbtiles file on S3

Resources

Readme

License

MIT license

Stars

85 stars

Watchers

4 watching

Forks

0 forks

Releases

11 tags

Packages 0

No packages published

Languages

  * Python 93.2%
  * HTML 4.0%
  * Shell 2.8%

Footer

 (c) 2022 GitHub, Inc.

Footer navigation

  * Terms
  * Privacy
  * Security
  * Status
  * Docs
  * Contact GitHub
  * Pricing
  * API
  * Training
  * Blog
  * About

You can't perform that action at this time.
You signed in with another tab or window. Reload to refresh your
session. You signed out in another tab or window. Reload to refresh
your session.