Data Loading in D3

In the previous sections, we have worked with data stored in local variables. In this chapter, we will learn to load data from different types of files and bind it to DOM elements.

D3 can handle different types of data defined either locally in variables or from external files.

D3 provides the following methods to load different types of data from external files.

d3.csv()

We can load a csv file or csv data using the d3.csv() method.

d3.csv(url[, row, callback]);

The first parameter is the url of .csv file, or webapi, or webservice which will return csv data. The second optional parameter is a conversion function which allows us to change the representation. The third optional parameter is a callback function which will be executed once the .csv file/data is loaded. It passes parsed data object as a parameter to callback function.

Lets take a look at how to load the following CSV data stored in a file named employees.csv.

Name, Age
John, 30
Jane, 32

Copy the employees.csv file into the data folder of your project's root folder and write the following code in the <script> tag.

<script>
d3.csv("/data/employees.csv", function(data) {
    for (var i = 0; i < data.length; i++) {
        console.log(data[i].Name);
        console.log(data[i].Age);
    }
});
</script>

Run the above example in a browser and open the developer tools, and click on Console tab and you will see the following result.

As you can see in the above example code, the D3 function d3.csv() takes a file name as an input, processes the file and loads the data into an array of objects. Note that the first row of the csv file does not get printed. This is because the first row is treated as the column name and rest of the rows are considered as the data. The data object loaded by d3 uses these column names as the object's properties and hence are converted to object keys.

Replace the for loop with a simple console.log() statement, printing the data object.

d3.csv("/data/employees.csv", function(data) {
    console.log(data);
});

Now check the console for the printed object. What do you see?

d3.csv() returns the data as an object. This object is an array of objects loaded from your csv file where each object represents one row of your csv file.

The above d3.csv("/data/employees.csv", function(data) { } is equivalent to the following.

d3.csv("/data/employees.csv")
  .get(function(data) {
        console.log(data);
  });

You can use d3.request() method instead of d3.csv, as shown below.

d3.request("/data/employees.csv")
  .mimeType("text/csv")
  .response(function (xhr) { return d3.csvParse(xhr.responseText); })
  .get(function(data) {
      console.log(data);
  });

Use row parameter to convert representation of the data. For example, the following code changes names to upper case.

d3.csv("/data/employees.csv")
  .row(function(d) {
        return {
            age: d.age,
            name: d.name.toUpperCase() // converting name to upper case 
        }; 
   })
  .get(function(data) {
      console.log(data);
  });

d3.json

JSON data can be a single object or an array of JSON objects.

var nameObj = {
    "name": "John",
    "age": 30,
    "city": "New York"
};

var nameArray = [{
    "name": "John",
    "age": 30,
    "city": "New York"
},
{
    "name": "Jane",
    "age": 20,
    "city": "San Francisco"
}];

JSON works in a similar way to CSV. The d3.json() method takes a JSON file as input and converts it into an array of objects.

d3.json(url, callback);

The first parameter is the url of .json file and second parameter is a callback function which will be executed once .json file is loaded. It passes parsed data object as a parameter to callback function.

Let's create a sample file users.json in the data folder of your project's root folder and paste the following JSON in it.

[{
    "name": "John",
    "age": 30,
    "city": "New York"
},
{
    "name": "Jane",
    "age": 20,
    "city": "San Francisco"
}];

Now, load the above JSON file using d3.json() as shown below.

d3.json("/data/users.json", function(data) {
    console.log(data);
});

You should see the following output in the developer console:

As you can see in the above result, D3 created an array of objects with name, city and age properties. This makes it easy for us to work with the data.

d3.tsv

d3.tsv(url, callback);

D3's tsv() method takes a .tsv file as an input url and returns a data object after parsing the TSV file.

TSV data stored in employees.tsv:

Name    Age
John    30
Jane    32

Now, load the above TSV file and parse the data using d3.tsv() as shown below.

d3.tsv("/data/employees.tsv", function(data) {
    for (var i = 0; i < data.length; i++) {
        console.log(data[i].Name);
        console.log(data[i].Age);
    }
});

The above example will display the following output in the developer console.

d3.xml

d3.xml(url, callback);

The d3.xml() method takes a url of xml file and returns an xml object.

The following is a sample XML in employees.xml file.

<?xml version="1.0" encoding="UTF-8"?>
<root>
<row>
    <Name>John</Name>
    <Age>30</Age>
</row>
<row>
    <Name>Jane</Name>
    <Age>32</Age>
</row>
</root>

d3.xml("/data/employees.xml", function(data) {
        console.log(data);
});

You can parse and traverse the above XML as shown below.

d3.xml("\data\employees.xml", function(data) {
        console.log(xml.documentElement.getElementsByTagName("Name", "));
});

This will give you all the tags with the tag name "Name".

Bind Loaded Data

Once we have loaded our data, we have the data object available to work with. For this tutorial, we will work with JSON data stored in a file named users.json.

[{
    "name": "Jon",
    "age": 30,
    "location": "The Wall"
},
{
    "name": "Arya",
    "age": 12,
    "location": "Braavos"
},
{
    "name": "Cersei",
    "age": 42,
    "location": "Kings Landing"
},
{
    "name": "Tyrion",
    "age": 40,
    "location": "Kings Landing "
}]

It is an array of person objects, with each object containing a name, age and location. Let's load the above json data using d3.json() method and bind it with DOM elements.

d3.json("/data/users.json", function(error, data) {
    
    d3.select("body")
        .selectAll("p")
        .data(data)
        .enter()
        .append("p")
        .text(function(d) {
            return d.name + ", " + d.location;
        });

});

You will see the following result in the browser when you run the above code.

Let's run through this code.

d3.json("/data/users.json", function(error, data) {
Our project has a file named "users.json". We loaded this file using d3.json(). The d3.json() method returned a formatted data object. It also returned an argument "error". We will have a look at that shortly.

d3.select("body") Once we have our data object, we want to output the content to our page. Where do we want to add it? That's right - to the body element. So, we select the body element. D3 returns the selection and we can pass this on to the next method using method chaining.

.selectAll("p")
We choose to output our data as paragraphs. You can use divs, spans, lists, whatever you fancy. We want four <p> elements because that is the size of our data. Now, D3 will look for <p> elements on the page. It looks up all <p> elements and sends the references to the next method in the chain. But whoa! We don't have any, do we? Don't panic just yet. D3 is very smart. Read on.

.data(data)
Now we need to bind our data. So we provide our data object to D3's data() function. That data() function passes on the data values from our dataset to the next method in the chain.

.enter()
The enter() function receives the values from data(). In our case, there are four values. But since we don't already have the references to our <p> elements corresponding to these data values, enter() returns empty placeholder references to the new elements.

.append("p")
We now have the references to our elements. The append() method adds these elements to the DOM.

.text(function(d) { return d.name + ", " + d.location; });
And finally the text() method. Most d3 functions accept functions as parameters. In our case, we have passed an anonymous function to the text method that returns a concatenation of name and location from our data object. text() is called and bound to each of our page elements with the corresponding data value.

Error Handling

While loading data from an external source, D3 returns an argument called "error". You can use this argument to check whether your data got loaded successfully.

d3.json("/data/users.json", function(error, data) {
    
    if (error) {
        return console.warn(error);
    }

    d3.select("body")
            .selectAll("p")
            .data(data)
            .enter()
            .append("p")
            .text(function(d) {
                return d.name + ", " + d.location;
            });
    });

If there is some error while loading your data, say your data is malformed; D3 will return an error object. You can check for errors and take a decision based on this.

Thus, you can load data from various sources using the available D3 methods.