Using D3.js with HTML Canvas provides a great improvement upon performance compared to using Scalable Vector Graphics (SVG) elements. Implementing interactive functionality on the canvas element can often be difficult and seldom well-documented. After working with D3.js and the HTML Canvas element for a while, I've learned how to include both the performance gains of rendering on canvas and the ease of interacting with the graph that SVG elements can provide “out of box”.
D3 offers remarkable extensibility and versatility. Empowering you to incorporate and tailor functionalities to a significant degree. However, when it comes to specific interactions like dragging individual nodes or displaying properties upon click, the canvas implementation of D3, while offering enhanced performance compared to SVG elements, poses greater challenges.
Upon completion of this post, you will have achieved two key outcomes. A fully operational front-end interface enabling you to write and submit Cypher queries to Neo4j. As well as an immersive graph visualization seamlessly integrated into the page.
We will demonstrate how to connect to an instance of Neo4j Desktop and visualize graph data from a user-generated query as a force layout graph using D3.js and HTML Canvas. All source code will be linked here for reference.
Before we begin, this blog assumes that you have already done the following steps:
Installed Neo4j Desktop
Loaded data into a blank database from your desired source
Installed an IDE or preferred text editor capable of editing and creating HTML and JavaScript files
If you haven't done so already, you can download Neo4j Desktop by visiting this link. Once installed and running, open the Neo4j Browser from Neo4j Desktop, or navigate to port 7474 on the localhost in a web browser where Neo4j Desktop is running. From there, ensure that your database is blank before running :play movies in the browser’s shell. This dataset is small enough to easily visualize its entirety while providing a good look at how graph databases are structured as nodes and relationships. This is the dataset I will be using.
In the Neo4j Browser, after loading the movie’s dataset, you should then see it available in our database.
Create our HTML File
In your IDE or text editor, create the index.html and index.js files.
The code block below outlines contents of index.html which I am using to provide a canvas element to render our graph and an input field with which to enter the query whose results we want to visualize, and a button to send the query to Neo4j. We’ll define responsiveCanvasSizer() and submitQuery() later.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Neo4j D3 Canvas Visualization</title>
<!-- The latest version of D3.js -->
<script src="https://d3js.org/d3.v7.min.js"></script>
<!-- Bootstrap for simple styling -->
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0-alpha3/dist/css/bootstrap.min.css" rel="stylesheet"
integrity="sha384-KK94CHFLLe+nY2dmCWGMq91rCGa5gtU4mk92HdvYe+M/SXH301p5ILy+dN9+nJOZ" crossorigin="anonymous">
<!-- Our D3.js code -->
<script type="text/javascript" src="index.js"></script>
</head>
<body class="align-center bg-light" onload="responsiveCanvasSizer()">
<div class="container-fluid m-4 shadow p-4 w-75">
<h5 class="text-muted mb-4">Enter Your Query</h5>
<input class="form-control mb-3" placeholder="Cypher query" id="queryContainer">
<button class="btn btn-secondary" onclick="submitQuery()">Run</button>
</div>
<div class="container-fluid m-0">
<div class="row">
<div class="col border border-3">
<!-- the canvas element where the graph is drawn -->
<canvas style="width: 100% object-fit: contain"></canvas>
</div>
</div>
</div>
</body>
</html>The result should appear like so in your browser when opening the file:
Data Fetching
Now that our front end is constructed, we can now move on to the foundation of our application which is submitting a query to Neo4j and returning its results in a D3-friendly format.
At the top of our JavaScript file, we’ll define some constants:
// Neo4j HTTP endpoint for Cypher transaction API
const neo4j_http_url = "http://localhost:7474/db/neo4j/tx"
const neo4jUsername = "neo4j"
const neo4jPassword = "neo4j123"
// used for drawing nodes and arrows later
const circleSize = 30
const arrowHeight = 5
const arrowWidth = 5We’re making connections using the Neo4j Cypher transaction API, which you can learn more about here. Fetching data using this API returns records in a closer format to what D3 is looking for rather than making a Bolt connection, like the official Neo4j JavaScript driver would use.
Next, we’ll define a submitQuery() function which takes a user-generated Cypher string and makes a POST request to our endpoint. Here, we’re creating empty objects to store our properly formatted nodes and links, as we extract and format each node and link, we want to add them there. We then begin writing a fetch statement to send a POST request to the endpoint we defined with a basic HTTP authentication header. This is done by using the credentials we defined above.
Finally, per the Cypher transaction API, we need to properly format our Cypher query as correct JSON before including it within the body of the request. The body contains the statement (the query we are submitting) and the format in which we want the data returned to us (graph, with each result stored on a separate row/result).
const submitQuery = () => {
// Create new, empty objects to hold the nodes and relationships returned by the query results
let nodeItemMap = {}
let linkItemMap = {}
// contents of the query text field
const cypherString = document.querySelector('#queryContainer').value
// make POST request with auth headers
let response = fetch(neo4j_http_url, {
method: 'POST',
// authentication using the username and password of the user in Neo4j
headers: {
"Authorization": "Basic " + btoa(`${neo4jUsername}:${neo4jPassword}`),
"Content-Type": "application/json",
"Accept": "application/jsoncharset=UTF-8",
},
// Formatted request for Neo4j's Cypher Transaction API with generated query included
// https://neo4j.com/docs/http-api/current/actions/query-format/
// generated query is formatted to be valid JSON for insertion into request body
body: '{"statements":[{"statement":"' + cypherString.replace(/(\r\n|\n|\r)/gm, "\\n").replace(/"/g, '\\"') + '", "resultDataContents":["graph", "row"]}]}'
})
.then(res => res.json())
.then(data => {
console.log(data)
})
}Let’s look at the results by opening the console in the browser. To do so, right-click on the page and select “Inspect” or “Inspect Element”. The exact wording of the menu item will change based on the browser you’re using. The console should now list the results of our query like so:
I ran the Cypher query match (n) return n limit 10 which gives me back 10 of any nodes in my database.
Let’s analyze this query. What we’re asking Neo4j for in this query is to match on any node n, whose type is arbitrary, and return that node. Even using the letter ‘n’ is arbitrary. We could use any letter or word if we’re being consistent in what we are referencing. The limit statement instructs Neo4j to return up to 10 results. If we wanted Neo4j to give us any ten person or movie nodes, we would define the type of node we want Neo4j to return like this:
Return up to ten movies: match (n:Movie) return n limit 10
Return up to ten people: match (n:Person) return n limit 10
And so on...
The query from our endpoint returns an array of dictionary results. Within them is a graph object which stores node and relationship information. D3 requires that all nodes and links in our dataset need to be defined as separate arrays of nodes and links. Each being a separate object, with their ID numbers which are later matched to link nodes together.
Here, we’ll unpack these values and return all nodes and links for use in data binding. As we can see above, all the query results are stored in a ‘results’ object. This is all the usable data we need within its ‘data’ object. Due to the extensive nesting of results and unnecessary metadata at each level, extracting the desired information from the code becomes a complex and intricate process.
To ensure a structured organization, each node and link is stored as an object using their unique ID as the key, while the actual node or link object serves as the corresponding value. This approach maintains the sequential order of nodes and links, even if we eventually remove the keys when presenting them for visualization.
…
.then(data => {
// if errors present in the response from Neo4j, propagate alert() dialog box with the error
if (data.errors != null && data.errors.length > 0) {
alert(`Error:${data.errors[0].message}(${data.errors[0].code})`)
}
// if results within valid data are not null or empty, extract the returned nodes/relationships into nodeItemMap and linkItemMap respectively
if (data.results != null && data.results.length > 0 && data.results[0].data != null && data.results[0].data.length > 0) {
let neo4jDataItmArray = data.results[0].data
neo4jDataItmArray.forEach(function (dataItem) { // iterate through all items in the embedded 'results' element returned from Neo4j, https://neo4j.com/docs/http-api/current/actions/result-format/
//Node
if (dataItem.graph.nodes != null && dataItem.graph.nodes.length > 0) {
let neo4jNodeItmArray = dataItem.graph.nodes // all nodes present in the results item
neo4jNodeItmArray.forEach(function (nodeItm) {
if (!(nodeItm.id in nodeItemMap)) // if node is not yet present, create new entry in nodeItemMap whose key is the node ID and value is the node itself
nodeItemMap[nodeItm.id] = nodeItm
})
}
//Link, interchangeably called a relationship
if (dataItem.graph.relationships != null && dataItem.graph.relationships.length > 0) {
let neo4jLinkItmArray = dataItem.graph.relationships // all relationships present in the results item
neo4jLinkItmArray.forEach(function (linkItm) {
if (!(linkItm.id in linkItemMap)) { // if link is not yet present, create new entry in linkItemMap whose key is the link ID and value is the link itself
// D3 force layout graph uses 'startNode' and 'endNode' to determine link start/end points, these are called 'source' and 'target' in JSON results from Neo4j
linkItm.source = linkItm.startNode
linkItm.target = linkItm.endNode
linkItemMap[linkItm.id] = linkItm
}
})
}
})
}
// update the D3 force layout graph with the properly formatted lists of nodes and links from Neo4j
updateGraph(Object.values(nodeItemMap), Object.values(linkItemMap))
})We now have our results in proper format for D3 in nodeItemMap and linkItemMap and passed to updateGraph(). These will handle both the D3 force simulation and data binding query results to it. We use Object.values() to return an array of values from the nodes and links objects which contain keys we no longer need.
Data Binding
To visualize our nodes and links, allowing them to move and be laid out with dynamic physical properties, we load our nodes and links into a new D3 force simulation. This will update node coordinates on each tick of its internal timer and place repulsive force between each one. This is so they don’t overlap or appear too close together.
// create a new D3 force simulation with the nodes and links returned from a query to Neo4j for display on the canvas element
const updateGraph = (nodes, links) => {
const canvas = document.querySelector('canvas')
const width = canvas.width
const height = canvas.height
let transform = d3.zoomIdentity
// This object sets the force between links and instructs the below simulation to use the links provided from query results, https://github.com/d3/d3-force#links
const d3LinkForce = d3.forceLink()
.distance(50)
.strength(0.1)
.links(links)
.id((d) => {
return d.id
})
/*
This defines a new D3 Force Simulation which controls the physical behavior of how nodes and links interact.
https://github.com/d3/d3-force#simulation
*/
let simulation = new d3.forceSimulation()
.force('chargeForce', d3.forceManyBody().strength())
.force('collideForce', d3.forceCollide(circleSize * 3))
// Here, the simulation is instructed to use the nodes returned from the query results and to render links using the force defined above
simulation
.nodes(nodes)
.force("linkForce", d3LinkForce)
.on("tick",simulationUpdate) // on each tick of the simulation's internal timer, call simulationUpdate()
.restart()
d3.select(canvas)
.call(d3.zoom() // https://github.com/d3/d3-zoom
.scaleExtent([0.05, 10]) // zoom out by 20x to zooming in 10x
.on('zoom', zoomed)) // on zoom, call the zoomed function below
function zoomed(e) {
transform = e.transform
simulationUpdate() /* we’ll define this later, this function handles drawing our nodes/links to the canvas*/
}
}
…We also define interactivity such as panning the graph and zooming in and out. Allowing you to view larger graphs that may occupy much of the screen at any given time. The canvas in D3 is equipped with a built-in zoom and pan functionality, accompanied by event listeners. These features are accessed through a transform object that retains the user's canvas manipulation state.
Drawing Links and Nodes
We’ve bound our data to the force layout simulation and can now iterate over nodes and links to draw them on the canvas in a simulationUpdate() function. This will be nested inside of updateGraph() so we can reference nodes and links easily. Unlike SVG elements that exist as discrete and tangible components on the page, canvas elements are more "abstract". When working with canvas, we observe and interact with the rendered output rather than individual elements themselves.
This is where using canvas really shines in performance gains over using SVG elements. Each node, link, and piece of text drawn on the page is simply a visual representation of the data instead of an element present in the Document Object Model (DOM) (i.e., an element you can view in the page source). The DOM represents elements in a logical tree and allows us to script functionality that interacts with these elements.
The canvas just draws the nodes and links while remaining only one DOM element: the canvas. Although executing extensive queries that yield thousands of nodes and links may initially cause a slight slowdown due to the data fetching and force simulation updates, the browser swiftly adapts once all the data is loaded. As a result, the browser effortlessly handles the resulting graph without any issues.
The simulationUpdate() function below is meant to run on each tick of the D3 force simulation’s internal timer which ticks approximately 60 times each second. On each tick, the canvas is instructed to iterate over the nodes and links returned from submitQuery() and draw them using their current coordinates. The force simulation tracks the physical movement of each node whose coordinates are stored as data properties. Simultaneously, on each tick any zoom or pan actions are applied to the canvas using translate and scale.
The canvas uses its “context” to perform these actions. We instruct the context to draw, translate, and scale rather than the actual canvas element. The save and restore functions prevent unnecessary re-drawing by saving state at the beginning of each tick, and then restoring state at the end. Only re-drawing when changes are made to node or link position.
…
//The canvas is cleared and then instructed to draw each node and link with updated locations per the physical force simulation.
function simulationUpdate() {
let context = canvas.getContext(‘2d’)
context.save() // save canvas state, only rerender what’s needed
context.clearRect(0, 0, width, height)
context.translate(transform.x, transform.y)
context.scale(transform.k, transform.k)
// Draw links
links.forEach(function(d) {
context.beginPath()
// use math to determine where the paths should be drawn
const deltaX = d.target.x – d.source.x
const deltaY = d.target.y – d.source.y
const dist = Math.sqrt(deltaX * deltaX + deltaY * deltaY)
const cosTheta = deltaX / dist
const sinTheta = deltaY / dist
const arget = d.source.x + (circleSize * cosTheta)
const arget = d.source.y + (circleSize * sinTheta)
const arget = d.target.x – (circleSize * cosTheta)
const arget = d.target.y – (circleSize * sinTheta)
const arrowLeftX = arget – (arrowHeight * sinTheta) – (arrowWidth * cosTheta)
const arrowLeftY = arget + (arrowHeight * cosTheta) – (arrowWidth * sinTheta)
const arrowRightX = arget + (arrowHeight * sinTheta) – (arrowWidth * cosTheta)
const arrowRightY = arget – (arrowHeight * cosTheta) – (arrowWidth * sinTheta)
// Each link is drawn using SVG-format data to easily draw the dynamically generated arc
let path = new Path2D(`M${ arget},${ arget} ${ arget},${ arget} M${ arget},${ arget} L${arrowLeftX},${arrowLeftY} L${arrowRightX},${arrowRightY} Z`)
context.closePath()
context.stroke(path)
})
// Draw nodes
nodes.forEach(function(d) {
context.beginPath()
context.arc(d.x, d.y, circleSize, 0, 2 * Math.PI)
// fill color
context.fillStyle = ‘#6df1a9’
context.fill()
context.textAlign = “center”
context.textBaseline = “middle”
// Draws the appropriate text on the node
// We either use the name or title property on the node, for person or movie respectively
context.strokeText(d.properties.name || d.properties.title, d.x, d.y)
context.closePath()
context.stroke()
})
context.restore()
}
}Responsive Canvas Sizing
Before we visualize our graph, we need to define responsiveCanvasSizer() defined in our front end. By automatically adjusting the canvas size based on the screen width, we ensure that our canvas is rendered in the optimal resolution. Otherwise, the nodes and text drawn on the canvas may be blurry and difficult to read. Especially with zooming in and out.
function responsiveCanvasSizer() {
const canvas = document.querySelector(‘canvas’)
const rect = canvas.getBoundingClientRect()
// ratio of the resolution in physical pixels to the resolution in CSS pixels
const dpr = window.devicePixelRatio
// Set the “actual” size of the canvas
canvas.width = rect.width * dpr
canvas.height = rect.height * dpr
// Set the “drawn” size of the canvas
canvas.style.width = `${rect.width}px`
canvas.style.height = `${rect.height}px`
}Navigate back to your web browser and enter a query that will display relationships between nodes. My query, for example, will return up to 20 results of people who acted in movies.
The query I’m using here basically asks Neo4j to give us any two nodes p and m in which p acted in m. While we aren’t explicitly defining node labels here, this relationship will only exist between a person node and a movie node, so we don’t need to define those.
You should now be able to zoom and pan around the graph to view all the nodes and relationships. Try using different queries to see how your graph data can be visualized!
Summary
Having successfully retrieved data from our Neo4j database through the front-end interface and established a D3 force simulation, we have effectively bound the queried data to our canvas, creating a visually captivating force layout graph.
Now that you have accomplished this implementation, why not take it a step further? Challenge yourself by incorporating the functionality to drag individual nodes within the graph, allowing for enhanced interactivity and customization. Explore the possibilities and elevate your graph visualization to new heights! Below are a few pointers to get you started:
Using D3.js drag, locate a node using the subject event listener from mouse coordinates on the canvas.
Apply the transformation done on that event to the node in question.
Define, using D3.js drag functionality, what should occur on drag start, duration of the drag event, and the conclusion of the drag event.
Thank you for taking the time to review this article and feel free to contact us if your project needs more advanced capabilities.
תגובות