Making a Simple Interactive Map Prototype with D3…For Total Beginners Who are Totally Impatient

by michael_simeone

Screen Shot 2014-01-07 at 9.56.08 AM

The goal of this piece will be to get you up and running with a simple interactive map accessible by web browser.  Rather than make sure you know everything ranging from computing basics to fundamentals of programming to web development to geospatial graphics to data visualization, we’re going to get to a prototype as quickly as possible.  For those who are daunted by the former, the latter approach can produce tangible results which are good for morale, and many people learn better this way.

We’ll be using D3.js, a javascript library (or collection of tools that can be called upon within a script) that helps map information from input files and join that information to Scalable Vector Graphics (SVG) objects.  Your web browser can draw these with the proper instructions by default, but D3 provides the tools to bind data to them.  SVG is different from bitmap or .jpg or other static image formats, in that they are a series of parameters through which to make an image, rather than a pixel by pixel specification of what something should look like.

Some caveats: never draw too many conclusions from a prototype, and I hope this tutorial results in an occasion to make new friends and collaborators, and since you are not an expert in any of the things listed above, you might need some.  This is not a replacement for collaborating with some of the good and wonderful people out there who do this kind of work.

The map we’re wanting to make will plot data from a comma-separated values file (we can make this in Excel) onto what’s known as an svg image (we’ll talk about how to get one of these soon).  This map will be accessible to anyone with an updated web browser and an internet connection.  The result we will aim for is something like what’s positioned above: a map of a place divided into subsections that are color-coded by value.  We’ll add mouseovers too so that we can display labels and other information as people explore.

I want to make a distinction among the kinds of instructional articles for this kind of material, and where this one fits in.  The most frequent (the kinds that are most helpful to those already with some web and computer background)  put all of their helpful materials into Github and invite you to clone their repository or even compile their code on your own.

There are many, many students of web development who are in the debt of posts like this. But they may not work for you.  You do not know what Github is, or even if you do, you are frustrated that you cannot download anything from this website.  You don’t want to learn everything about javascript, and eventually you plan on getting a really thorough foundation on the models and principles underlying visualization libraries and tools, but that day is not today.  What you want to do now is make a damn map so you can go back and do what you were doing before you had this idea to try out mapping.

So here’s the obligatory mention of wonderful introductions to Github, D3.js, and javascript.  You really should read them sometime.  And more. But not now.

[update December 1, 2015] For more information about data visualization, see a related project, Stories from Data. 

What you will need

Firstly, you’ll need a few pieces of software/web services that you will hold on to after this project.

-The first is a text editor.  Sublime Text is brilliant. Notepad++ and Textwrangler are also very good.

-If you don’t have a dropbox account, sign up for one here.

-Microsoft Excel or OpenOffice

If we were in an ideal world, we’d have our own webservers and sftp our content to our host machines, and we wouldn’t use a spreadsheet program to manipulate data (We’d use R).  But that’s a lot of work to do, R is notoriously tricky for beginners, and your website is hosted by WordPress or Blogger. You really want this map to materialize. So for now, these are the tools we will use.

At this point you either have some data you want to spatially visualize, or you don’t.  It’s important to note that for this purpose, that data will have to be text or numbers.  If you want to make a complex overlay of images, text, video, and draw animated lines connecting places to show complex relationships over space, you certainly could do that using the tools we have here, but it is beyond the scope of this piece.  DH press might be a good start if that’s what you need.

Here, we are going to make what’s known as a Choropleth, the kind of map where we divide space up into regions and color-code the regions by value.  Again, there are so many kinds of maps, but our goal is to get you up and running with a prototype that you can learn from because it works, not attempt to transform you into a programmer or web developer.

The data to use

So back to the issue of data.  You’ll need 3 main pieces:

-a special geospatial file called a Topojson.  If you want to make your own (eventually, you might have to), this tutorial  and this tutorial are very helpful. Otherwise there will be some posted at the bottom for download (besides the us.json we use here) as time goes by. We’re going to use a file that is comprised of all US counties.  If you go to this link and copy the contents into a Sublime Text file and save it as “us.json” (or anything .json), then you’ll have this step wrapped up.

-Since we have a county map of the USA, we’ll need some data that is broken down by county.

Screen Shot 2014-01-07 at 10.45.45 AM

A good start for more data like this is here.  To decode what all of the labels in the column headers are, try this file. 

Just remember that the us.json file we’re using only has counties drawn. When you’ve selected a column that has data you find interesting, delete entries for states for any data you choose and copy the information into a file that looks like this one. The states are listed in all caps and have no values associated with them, so they’re easy to pick out and delete en masse.  Keep in mind you could keep a whole lot of county data (multiple columns) and read selectively from them when you get to making your visualization.  For this tutorial I’ve pared down the .csv to include the number of family and independently owned farms, by county, from the 1992 US Census.

Important: Because we’ve worked with US Census data, NIST has already ensured that two very important things are true:

1) Our topojson has regions designated with Federal Information Processing Standards (FIPS) numbers.  Which,

2) Coincide with the Census data pulled from each county.

Whatever .csv data you have, make sure that there is a column that associates the information to a naming or id standard that is also present in your map/topojson. Working with federal atlases and US Census ensures that this is the case, but wandering outside of the government’s standards will require that you make sure these numbers match up.

So, you’ll want to place both of these files into a folder that you’ve made.  The next thing you’ll do is open a blank text file in Sublime or whichever program you’re using, and paste in the following code included at the bottom of this post.

WARNING: before you paste this code, be advised that your columns must be titled identically to the example files above, and your csv file that contains those columns is named identically too.  Your .json file should be named “us.json” as well.   You could name your variables (column headers, file names, variables within the code itself) anything you like once you become more comfortable.

A tour of the code

First we’ll begin with the header, which will announce that this is an html and not a plain text document, and supply some information about the language (english) and character set (utf-8) contained herein.


<div></div>

<meta charset="utf-8" />

Independent Farms by County - Choropleth
<div></div>
<div><style><!--

Note that in the final section of the header we’re pointing the web browser to some javascript files that will do important work for this web page.  The first will let us map data into svg graphics (d3.js), the second lets us work with multiple input files for a javascript script (queue), and the third helps us interpret the instructions in us.json as an image in the browser (topojson).

The next section defines CSS for objects on the page.  You can read more about CSS here, but basically what we’re doing here is defining style rules for objects that have given labels:


path {
stroke:white;
stroke-width: 1px;
}

body {
font-family: Arial, sans-serif;
}

.legend {
font-size: 12px;
}

div.tooltip {
position: absolute;
text-align: center;
width: 150px;
height: 25px;
padding: 2px;
font-size: 10px;
background: #FFFFE0;
border: 1px;
border-radius: 8px;
pointer-events: none;
}
--></style></div>
<div></div>

“Path” refers to lines drawn as instructed by our topojson file (us.json).

“Body” helps define text included in the area of the html page defined as “body”

Notice that .legend and .tooltip refer to objects we’ll designate with our javascript, but we can still set what they’ll look like here in the CSS.

The next section begins the body of our page, in which we’ll embed javascript.  You’ll see the title of the page, followed by the designation that what follows is a script written in javascript to be executed when the page loads.

You’ll see a lot of “var=”, which is setting up our variables for the code.  Note that the first of the variables affect what values map to what colors.  See that changing up these variables is an easy way to change the appearance of this map (as well as the CSS).  Colors are coded by RGB HEX value (make your own gradients here).  There are multiple ways to scale colors, but this is the one we’ll go with here.

<h1>Independent Farms in the USA</h1>
<pre>
var width = 960,
height = 500;
var color_domain = [500, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, 5000, 5500, 6000]
var ext_color_domain = [0, 500, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, 5000, 5500, 6000]
var legend_labels = ["< 500", "500+", "1000+", "1500+", "2000+", "2500+", "3000+", "3500+", "4000+", "4500+", "5000+", "5500+", "6000+"]
var color = d3.scale.threshold()
.domain(color_domain)
.range(["#dcdcdc", "#d0d6cd", "#bdc9be", "#aabdaf", "#97b0a0", "#84a491", "719782", "#5e8b73", "#4b7e64", "#387255", "#256546", "#125937", "#004d28"]);var div = d3.select("body").append("div")
.attr("class", "tooltip")
.style("opacity", 0);

var svg = d3.select("body").append("svg")
.attr("width", width)
.attr("height", height)
.style("margin", "10px auto");
var path = d3.geo.path()

The “svg” variable is crucial here: it’s a designation for  joining of a to-be-specified svg graphic with the body of the html page. D3 will let us map data from our files onto this svg designation.  Also note that the “path” variable is calling on a capability of D3 to draw lines based on geospatial information fed to it by our topojson.  If we were to change our .path CSS information, it would change how these lines were drawn.

The next section prepares our files to be read by D3 and plotted onto our SVG “canvas.”


queue()
.defer(d3.json, "us.json")
.defer(d3.csv, "data.csv")
.await(ready);

And the section after that will do some very important work: set up two blank containers, and fill them with an array of pairs.  Each pair will be by “id” (the same as the name of our column headers from our .csv).  The result is a list of value by id number that we can call on later.  A .csv file alone does not accomplish this.  This step maps the .csv file so that it is legible as associated values.


function ready(error, us, data) {
var pairRateWithId = {};
var pairNameWithId = {};

data.forEach(function(d) {
pairRateWithId[d.id] = +d.rate;
pairNameWithId[d.id] = d.name;
});

d.rate and d.name refer to the column headers of our .csv.  There’s a “d” before them because it’s a default way of referencing data you’ve mapped into D3 with javascript.  See how we read in “us” and “data”? “d” refers to that data parameter, which in this case is our .csv that (while not necessary) is also named “data.”

And now we’ll select the svg objects we’ve created but not specified, and map our data onto them:


svg.append("g")
.attr("class", "region")
.selectAll("path")
.data(topojson.feature(us, us.objects.counties).features)
.enter().append("path")
.attr("d", path)
.style ( "fill" , function (d) {
return color (pairRateWithId[d.id]);
})
.style("opacity", 0.8)

This will draw each county as an object, each with its own values.  Notice that we’ve named this class of object “county.” If we wanted to change the style of the counties in CSS up at the top, we could just refer to .county and make changes.  Also, the “.data” line associates information from our us.json file with the county objects (the stuff in parentheses refers to the way the topojson hierarchizes information and points the script to the right container in the hierarchy).

Also important is that “color” refers to the function set above in the code (up in the section with all the “var= ” business).  “Color” expects a number as input, but instead of a specific number, we’re going to give it our container filled with pairs of ID numbers and rate values (in this case, it’s family and individual farm counts for each county), and use [d.id] to make sure that we read in a value for each id number.

The rest is what happens when the mouse glances over the county:


.on("mouseover", function(d) {
d3.select(this).transition().duration(300).style("opacity", 1);
div.transition().duration(300)
.style("opacity", 1)
div.text(pairNameWithId[d.id] + " : " + pairRateWithId[d.id])
.style("left", (d3.event.pageX) + "px")
.style("top", (d3.event.pageY -30) + "px");
})
.on("mouseout", function() {
d3.select(this)
.transition().duration(300)
.style("opacity", 0.8);
div.transition().duration(300)
.style("opacity", 0);
})

Notice how we’re calling the county names and farm counts with a similar technique as before.  The “div.text” will behave according to our “div.tooltip” CSS style that was established at the top.  The duration of the transition (which in this case transitions from less to more opacity, creating a highlight effect) is listed in milliseconds.

And now, to draw the key that explains what each color means.  If you want to change what each label is, make sure to adjust the variable “legend_labels.”


var legend = svg.selectAll("g.legend")
.data(ext_color_domain)
.enter().append("g")
.attr("class", "legend");

var ls_w = 20, ls_h = 20;

legend.append("rect")
.attr("x", 20)
.attr("y", function(d, i){ return height - (i*ls_h) - 2*ls_h;})
.attr("width", ls_w)
.attr("height", ls_h)
.style("fill", function(d, i) { return color(d); })
.style("opacity", 0.8);

legend.append("text")
.attr("x", 50)
.attr("y", function(d, i){ return height - (i*ls_h) - ls_h - 4;})
.text(function(d, i){ return legend_labels[i]; });

</script>
</body>
</html>

With this we designate an unspecified group of svg objects as “legend”, associate this group with data from our variables, then attach rectangles and text that are bound to that data. This selection of objects and binding of data to them is what makes D3 so exciting, among other things.

Wrapping up and posting to the web

When you have your html file saved, give it a name and place it in your folder that contains us.json and your data.csv.  Follow these instructions and place the contents of your folder into the “public” folder of your Dropbox to begin hosting the webpage containing your map.

And there you have it.  Swap out files, tweak variables, edit the style: get this to work and then work on changing and breaking it.  Include hyperlinks or interesting text in your mouseovers. Represent more than one value. And so on. After all, sometimes it’s more fun to read about new things in the context of “what can I get my project to do now” rather than “time to learn everything.”

The full code:

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Independent Farms by County - Choropleth</title>
<script type="text/javascript" src="http://d3js.org/d3.v3.min.js"></script>
<script type="text/javascript" src="http://d3js.org/queue.v1.min.js"></script>
<script type="text/javascript" src="http://d3js.org/topojson.v1.min.js"></script>

</head>
<style>

path {
stroke:white;
stroke-width: 1px;
}

body {
font-family: Arial, sans-serif;
}

.city {
font: 10px sans-serif;
font-weight: bold;
}

.legend {
font-size: 12px;
}

div.tooltip {
position: absolute;
text-align: center;
width: 150px;
height: 25px;
padding: 2px;
font-size: 10px;
background: #FFFFE0;
border: 1px;
border-radius: 8px;
pointer-events: none;
}
</style>
<body>
<h1>Independent Farms in the USA</h1>
<script type="text/javascript">
var width = 960,
height = 500;
var color_domain = [500, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, 5000, 5500, 6000]
var ext_color_domain = [0, 500, 1000, 1500, 2000, 2500, 3000, 3500, 4000, 4500, 5000, 5500, 6000]
var legend_labels = ["< 500", "500+", "1000+", "1500+", "2000+", "2500+", "3000+", "3500+", "4000+", "4500+", "5000+", "5500+", "6000+"]
var color = d3.scale.threshold()
.domain(color_domain)
.range(["#dcdcdc", "#d0d6cd", "#bdc9be", "#aabdaf", "#97b0a0", "#84a491", "#719782", "#5e8b73", "#4b7e64", "#387255", "#256546", "#125937", "#004d28"]);

var div = d3.select("body").append("div")
.attr("class", "tooltip")
.style("opacity", 0);

var svg = d3.select("body").append("svg")
.attr("width", width)
.attr("height", height)
.style("margin", "10px auto");
var path = d3.geo.path()

queue()
.defer(d3.json, "us.json")
.defer(d3.csv, "data.csv")
.await(ready);

function ready(error, us, data) {
var pairRateWithId = {};
var pairNameWithId = {};

data.forEach(function(d) {
pairRateWithId[d.id] = +d.rate;
pairNameWithId[d.id] = d.name;
});
svg.append("g")
.attr("class", "county")
.selectAll("path")
.data(topojson.feature(us, us.objects.counties).features)
.enter().append("path")
.attr("d", path)
.style ( "fill" , function (d) {
return color (pairRateWithId[d.id]);
})
.style("opacity", 0.8)
.on("mouseover", function(d) {
d3.select(this).transition().duration(300).style("opacity", 1);
div.transition().duration(300)
.style("opacity", 1)
div.text(pairNameWithId[d.id] + " : " + pairRateWithId[d.id])
.style("left", (d3.event.pageX) + "px")
.style("top", (d3.event.pageY -30) + "px");
})
.on("mouseout", function() {
d3.select(this)
.transition().duration(300)
.style("opacity", 0.8);
div.transition().duration(300)
.style("opacity", 0);
})

};

var legend = svg.selectAll("g.legend")
.data(ext_color_domain)
.enter().append("g")
.attr("class", "legend");

var ls_w = 20, ls_h = 20;

legend.append("rect")
.attr("x", 20)
.attr("y", function(d, i){ return height - (i*ls_h) - 2*ls_h;})
.attr("width", ls_w)
.attr("height", ls_h)
.style("fill", function(d, i) { return color(d); })
.style("opacity", 0.8);

legend.append("text")
.attr("x", 50)
.attr("y", function(d, i){ return height - (i*ls_h) - ls_h - 4;})
.text(function(d, i){ return legend_labels[i]; });

</script>
</body>
</html>

Advertisements