Outlier Data Bites
with The Outlier
• Getting started with spreadsheets
• Learn pivot tables
• Data visualisation tips
• Map-making with data.
Working with Open Street Map vector data in QGIS
When we teach data journalism most people are eager to make a map but they’re usually unsure of where to get data for a map. And while there are many good sources for map data, if you’re looking for something, like a particular town or neighbourhood, that’s not always so easy.
One solution is to use QGIS and Open Street Map to create your own map data.
We recently did this for a project looking at the spread of cholera near Pretoria in South Africa.
There are lots of maps of South Africa available but most are at a national or provincial level and don’t really show the details of an area. For our purposes, we wanted a map with more detail and which showed the location of major settlements north of Pretoria.
An excellent source of free highly-detailed map data is the Open Street Map (OSM) project but exporting data can be tricky and confusing. An easier way to download OSM data is to use QGIS with a couple of plugins:
Install the QuickOSM plugin
Open QGIS and select the “Plugins” menu item and then “Manage and Install Plugins”.
On the screen that appears click the “Not Installed” option and search for “QuickOSM”. Click on that and then on “Install Plugin”.
Install the QuickMapServices plugin
While you’re in the plugins menu search for the “QuickMapServices” plugin and install that as well.
Add a basemap
There are various ways to get OSM data into QGIS but the easiest is to add a map to the main editing window and then select the area you want data for.
Click Project → New in the main menu to start a new project. Then select Web in the menu followed by QuickMapServices → OSM → OSM Standard. This will add the standard OSM map to the main editing window.
Find your area of interest
Zoom in on the area you’re interested in. To start with zoom in pretty close. The level of detail in OSM is impressive but can also lock up your computer if you try to get too much information at once. Try with a suburb you’re familiar with at first to get a feel for the information available.
Here we zoomed in on an area between Pretoria and Hammanskraal.
Select the data you want
Once you’ve framed the area you’re interested in in the main editing window, you can get the vector data.
Click on Vector in the main menu and then on QuickOSM→QuickOSM.
This will bring up the QuickOSM interface. Depending on the version of QGIS and QuickOSM you have the interface may look slightly different.
The main things you need to look out for are the key, value and are values.
OSM labels all features with a top-level “‘key” value like highway (for all road features) or “waterway” for water features. You need to select a key in all cases.
There is a secondary level called “values” which lists sub-features, for example residential roads or paths. You don’t need to select “values” if you want all features in a specific key.
Then there is the area you want. You either need to provide a specific area using coordinates or, in this case, just supply the map area we already framed.
Here we select “highway” as the key and “residential” as the value, for all the residential roads in the given area.
For the area, we select “canvas extent” which is the are we have already framed in the main editing window:
With these selected, we can run the query. Depending on how much detail you have selected this can take a while to complete. When it is ready you can close the QuickOSM window. You should have something like this. The brown areas in the map are the vector details from OSM. You can tick and untick the layers in the left-hand panel to switch between layers.
Untick the OSM Standard layer to show just the OSM data.
Tip: Change the vector colours
QGIS automatically chooses a colour for the lines in layers. Sometimes this makes it difficult to distinguish between items. For example, if I add a layer for rivers, it would help if they were blue. Once you’ve added a layer with the rivers (using the QuickOSM technique above) you can right-click on the layer with the rivers and change the colour in the styles menu:
Exporting the data
When you’re done you probably want to export the map data, as a shapefile or as GeoJSON.
You can export the individual layers of your map by right-clicking on a layer and selecting Export→Save Features As and then saving as GeoJSON or ESRI Shapefile (the most likely options).
Export multiple layers
I you created multiple layers that you now want to export as one file you’ll need to merge the layers.
To do this, select Processing→Toolbox in the main menu. In the toolbox that appears to the right, search for “merge” and then select “Merge Vector Layers”. You’ll need to select the layers you want to merge and run.
This will create a new layer in your layers panel. You can export this like any other layer by right-clicking on it and selecting “Export→Save Features”.
How to extract multiple features from a shapefile in QGIS
This is a simple(ish) guide to finding and extracting features from a larger map into a smaller one. In this example selected rivers in South Africa are copied into a new GeoJSON format file.
Start with a shapefile and QGIS. This shapefile is of rivers in South Africa and is a good example of selecting multiple portions of a file:
Download this or use your own shapefile.
1. Open the file in QGIS
- Open QGIS and click Project→New to create a new project
- Drag the zip file into the main QGIS window
- A popup will appear. You only need the “wriall500.shp” layer in this case, so click that to select it and then OK to open.
The map file will open in the main window and look something like this:
2. Search for the rivers
Right click on the “wriall500” layer in the bottom left pane and then select “Filter”.
On the next screen select the columns to search and the values.
Double click on the buttons, columns and values to add them to the query box at the bottom.
This can take a bit of experimentation but as a starting point, it’s possible to find a number of rivers by using this query:
“NAME” IN (‘Klip’, ‘Vaal’, ‘Sandspruit’, ‘Harringtonspruit’, ‘Natalspruit’)
Add additional river names in single quote marks and separated by commas.
Use the “Test” button to check the results for errors.
Click OK to select these features. This will close the filter window and get you back to the main QGIS window. You may or may not see the features you selected in the main window at this point. Either way, you can export the selected features:
3. Export the river features
Right-click on wriall500 layer and select Export→Save Features as
On the next pop-up screen:
- Select GeoJSON for the export
- Add a filename
- In the “CRS” section you may see “Invalid Projection”. Change this to something like “WGS84”
- Click OK to export the file.
4. Test in Mapshaper
The easiest way to check the export is to open Mapshaper and drag the newly exported geoJSON file into the main window.
This should show just the rivers selected:
Add a basemap layer to check the river positioning:
5. Export to SVG
In Mapshaper it’s easy to export to SVG which can then be used in Figma or Illustrator to touch up or add details.
6. Final files
The process above produces the following files:
10 steps to create a custom Svelte component library
Although creating complex sites with multiple components in SvelteKit is pretty simple already, creating a library for your most regularly used components adds even more power to your projects.
With a set of common components set up as an NPM package you can easily import them into your projects, and when you change one component in the library it can easily be updated through all projects that use that particular component.
Things like buttons, modals, subscription forms, header elements or SEO tags are prime candidates to be turned into a library for regular re-use.
Setting up a Svelte-based library is not hard but the process has changed as Svelte, and particularly SvelteKit, has developed.
The steps below currently work with SvelteKit 1.x.
Overview
To create a Svelte library we need to:
- Create a new SvelteKit library project
- Create our components in the “lib” directory
- Build the component package in the “dist” directory
- Publish to NPM
Steps
1 – Create a new Svelte Library project:
npm create svelte@latest <app name>
Follow the prompts and create a “Library” project.
Then install the dependencies:
cd <app name>
npm install
2 – Create a component in the src/lib directory. Everything in the “lib” directory will make up your final library release.
/lib/src/Button.svelte
<script>
console.log('Button Component');
// You would import props here
</script>
<button>My Button</button>
<style>
button {
background: #000;
color: #fff;
padding: 10px;
border: none;
border-radius: 5px;
}
</style>
3 – You can create a file in the “routes” directory to test your component and run it like a usual SvelteKit project.
/routes/+page.svelte
<script>
import Button from '../lib/Button.svelte';
</script>
<h1>MHC Components</h1>
A collection of visual and web-design components.
<br/>
<br/>
<Button />
4 – Edit the index.js file in src/lib to import your component and then export it as a named export:
import Button from './Button.svelte';
export { Button };
// If you multiple components, you can re-export them here:
// import Card from './Card.svelte';
// export { Button, Card }
5 – Add package.json and README.md files to the src/lib directory.
Copy the package.json file to the lib directory.
6 – You may need to Install “svelte2tsx” but with a default SvelteKit library project it is likely to be included.
npm install svelte2tsx
7 – Add a jsconfig.json (or tsconfig.json if you’re using Typescript) file to the root of your project. The file should be at the same level as the svelte.config.js file. A jsconfig.json file needs to look like this as a minimum:
{
"extends": "./.svelte-kit/tsconfig.json"
}
8 – Set up a GitHub project and push your first commit to the new respository.
9 – At this point you should be able to build your library. If you set up a SvelteKit Library project you should be able to run either the “build” script or the “package” script:
npm run build
or
npm run package
The output should look something like this:
10 – Publish to NPM. You’ll need an NPMJS.com account for this.
Log into NPM at the command line:
npm login
Then publish your package:
npm publish --access public
If you want to publish the package as part of an NPM organisation you need to include the NPM organisation name in the package.json file:
{
"name": "@mediahackza/graphic-components",
"version": "0.0.1",
...
}
If you get this far you hopefully now have an NPM package set up. You can use the package in any of your projects like a normal package:
Import the package:
npm install @mediahackza/graphic-components
Then import and use the components you need:
<script>
import { Button } from @mediahackza/graphic-components
</script>
<Button />
Final notes
The process for building a Svelte-based package has changed a lot recently. It’s worth keeping an eye on the official packaging section in the Svelte docs so more recent updates.