Performance Optimization tips for Node.js Applications
Performance is an important aspect when building webpages and applications. You need to understand how long your users interact with your app, how often they leave, and the response time as well. <!--more-->
The first 10 seconds are vital to determine if a user will leave or continue interacting with your web pages. You must clearly speed up the page, bring value within these 10 seconds, and get the user's attention to spend more time on your page.
This is where optimizing your page and speeding up the response time becomes important to maintain a good user experience. Node.js is known to produce super-fast performing and scalable web applications. Node.js uses event-driven architecture and non-blocking (asynchronous) tasks that run on a single thread.
As a developer, you need your application to be fast and fully optimized. One could say that Node.js is the fire and performance optimization is the gasoline. Imagine how blazing fast your app could be. This article will cover techniques, tools, and tips to optimize Node.js apps to achieve higher-performance.
Asynchronous Coding
Node.js is designed with single thread concurrency architectures. Asynchronous coding is heavily used in Node.js to ensure a non-blocking operational flow. Asynchronous I/O permits other processing to continue even before the first transmission has finished. Synchronous coding can potentially lock your web page. It uses blocking operations that might block your main thread and this will drastically reduce your web page performance.
Asynchronous coding will allow you to use queues to monitor your workflow, allowing you to append additional tasks and add extra callbacks without blocking your main thread. Even though you may try to use Asynchronous methods, in some instances, it is possible to find your web page making some blocking calls. This is common when using third-party modules. You should keep an eye on the external libraries you are using and take precautions to avoid them dominating synchronous calls.
Take a look at the example below: We used the file read operation in both models using the Node.js file system
Synchronous
const fs = require('fs');
const content = fs.readFileSync("app.txt", "utf8");
console.log(content);
console.log('waiting for the file to read.........');
Asynchronous
//inlude file system module
const fs = require('fs');
//readFile() reads the file
fs.readFile("app.txt", "utf8", function (err, content) {
if (err) {
return console.log(err);
}
//read the file
console.log(content);
});
console.log('waiting for the file to read.....');
With the synchronous snippet, all processes are paused until the file reading is over. The readFileSync() will first read the file and store its data in the memory before printing the data and message in the console.
Asynchronous will print the console message as the system performs other tasks. The "waiting for the file to read" is printed before the file content even though it was the last command. A callback function will be called when the file reading is done. The asynchronous model will have faster execution of our program and avoid the so-called "Call-back Hell".
Query Optimization
Just imagine having more than one million rows of data to query from your database table. The process generated by this database to get the endpoint of this data will greatly affect your application performance. Using bad queries will make your application slower leaving your users dissatisfied. The solution to this is to perform query optimization.
Basic tips to improve your database performance/optimization overview
- Avoid SELECT
This would be the most obvious thing you would use to select your columns, right? Yet, you should not overlook how cumbersome this query can be. To query columns in a table, go for a more distinct approach. Use the SELECT statement to query only the data you need and avoid extra fetching loads to your database. For example, assume you have a table named Customers with the fields FirstName, LastName City, State, Zip, PhoneNumber, and Notes. To SELECT fields FirstName, LastName, City, State, and Zip, the following two queries might be used.
query1
SELECT *
FROM Customers
query2
SELECT FirstName, LastName, Address, City, State, Zip
FROM Customers
The first query will pull all the fields in this table even though we don't need them all. The second query will pull the only the required fields hence the second query will be the more efficient way to perform the SELECT statement. The same idea should apply when using the INSERT and UPDATE statements.
- Use the clause WHERE
The goal of the query is to pull the required records from your database. The WHERE clause helps to filter the records and limits the number of records to be pulled based on conditions. WHERE replaces the HAVING clause that selects records before filtering the dataset. In SQL operations, WHERE statements are faster as it reduces the data being processed by the database engine.
- Use LIMIT to sample the query results
LIMIT will return only the specified number of records. While using LIMIT, ensure that the results are desirable and meaningful when a limit is imposed on the dataset. For example, if our table named Customer has 500 records and we only need the first 100 records, LIMIT will be an efficient way to sample out the desirable results by avoiding the selection of the extra 400 records. Here is an example:
SELECT FirstName, LastName, Address, City, State, Zip
FROM Customers LIMIT 100
- Avoid SELECT DISTINCT
SELECT DISTINCT removes duplicate records by GROUPing them to create distinct results. In our table example of Customers
SELECT DISTINCT FirstName, LastName, State, Zip FROM Customers
may tend to group together some common first and last names such as David Smith. This will cause an inaccurate number of records. This query can be slow to execute when a table has a large number of customers with the name 'David Smith'. Go for a more accurate and efficient query such as:
SELECT FirstName, LastName, State, Zip FROM Customers
and the number of records will be accurate.
- Use wildcard (%) character appropriately
If we want to SELECT customers whose first names start with 'Avi',
query1
SELECT FirstName from Customers where FirstName like ‘%avi%’
query2
SELECT FirstName from Customers where FirstName like ‘avi%’
The first query will pull the FirstNames such as Avishek, Avinash, or Avik. This method is inefficient as it may pull unexpected results where the FirstNames has 'Avi' such as David, Xavier, or Davin. The second query would be more efficient to perform this wildcard.
- Running queries during off-peak hours
In the production database, analytical and database management queries should be executed when the concurrent users are at their lowest peak. Typically at night around 3 to 5 am.
Check out this MySQL query performance optimization article. It has useful Mysql tips that will improve your query writing techniques.
Caching
A cache is a memory buffer where frequently accessed data is temporarily stored to be accessed quicker. Cached data is then retrieved without having to access the origin. Caching will improve your app response time and even reduce some costs such as bandwidth and data volumes.
Caching is a great practice to improve your app performance. If you have a low number of users, your app performance may not be greatly affected. However, performance problems may arise when the traffic grows and you need to maintain the load balance. When this happens, caching your app regularly will be a great technique to achieve higher-performance. Caching can be a little bit hard thus you need tools to cache your app efficiently such as:
- Redis cache is entirely asynchronous with optimal performance to handle cached data requests in a single thread. Consider checking it out. It is a smooth API that will help you manage your client-side and server-side cache.
- Memcached stores data across different nodes. It uses a hashing schema that provides a hash table functionality. These ensure that adding, or removing a server node does not significantly change the mapping of the keys to server nodes.
- Node-cache works almost like Memcached with the set, get, and delete methods. It has a timeout that deletes data from the cache when the timeout expires.
- Nginx will help maintain load balance. Nginx will help cache static files, that will drastically offload the work of the application server. It offers low memory usage and high concurrency.
Go Session Free
Session data is stored in memory. As traffic in your app grows, more sessions will be generated and this might add significant overhead to your server. You need to find the means to store session data or minimize the amount of data stored in a session. Modules such as Express.js can help you create stateless server protocols. Stateless protocols do not save, or store any information from previous visits.
Switch to an external session store such as Redis, Nginx, or MongoDB. Whenever possible, the best option would be to store no session state on your server-side for better performance.
Script Tracing and Logging
Logging helps track your application activities and traffic. When an app is running, it is possible to get fatal errors (even though your app was running properly after production testing). You need to get this feedback on time, find out what code has the error(s), and fix them before your user notices something is faulty in your system.
Commonly used methods for logging in Node.js are console.log() which logs Standard outputs (stdout), and console.error(). They will log standard errors. However, there are more efficient, and scalable libraries/3rd party APIs that will assist you in logging Node.js scripts.
They include:
1. Winston
Winston is a multi-transport async logger, which is simple, universal, and extremely versatile. It makes logging more flexible and extensible. It is the most popular logger based on NPM stats. Winston has different transports with different default levels that indicate message priority. These levels include
- error
- warn
- info
- http
- verbose
- debug
- silly
Example of a simple/custom Winston logger
const winston = require("winston");
const logger = winston.createLogger({
transports: [new winston.transports.Console()],
});
logger.info("Information message");
logger.warn("Warning message");
logger.error("Error message");
Winston logger with file transport
const winston = require("winston");
// Logger configuration
const logConfiguration = {
transports: [
new winston.transports.Console({
level: "verbose",
}),
new winston.transports.File({
level: "error",
level: "debug",
level: "warn",
level: "verbose",
level: "info",
level: "silly",
// create a directory 'logs' that will have the file name 'example.log' with the logs
filename: "./logs/example.log",
}),
],
};
//Create logging configurations
const logger = winston.createLogger(logConfiguration);
// Log some messages
logger.silly("Trace message, Winston!");
logger.debug("Debug message, Winston!");
logger.verbose("A bit more info, Winston!");
logger.info("Hello, Winston!");
logger.warn("Heads up, Winston!");
logger.error("Danger, Winston!");
2. Morgan
Morgan is an HTTP request logger middleware for Node.js applications. Morgan gives insight on how your app is being used and alerts you on potential errors and issues that could be threats to your application. Morgan is considered the most reliable HTTP logger by Node.js developers. Morgan is designed to log errors the way servers like Apache and Nginx carry out to the access-log or error-log.
Below we have an example app that will write one log file per day in the log/directory using the rotating-file-stream module.
const express = require("express");
const morgan = require("morgan");
const path = require("path");
const rfs = require("rotating-file-stream");
const app = express();
// create a rotating write stream
const accessLogStream = rfs.createStream("access.log", {
interval: "1d", // rotate daily
path: path.join(__dirname, "log"),
});
// setup the logger
app.use(morgan("combined", { stream: accessLogStream }));
app.get("/", function (req, res) {
res.send("hello, world!");
});
3. Bunyan
Bunyan is a lightweight logger that creates log records as JSON.
const bunyan = require("bunyan");
const log = bunyan.createLogger({ name: "myapp" });
log.info("hi");
log.warn({ lang: "fr" }, "au revoir");
Click this link for more details to get started with Bunyan.
Logging best practices
- Logging should be meaningful and have a purpose
- Adopt logging at the early stage of app development
- Divide logs into several log files in case you have an application with huge traffic
- Logging should be structured and done in levels
Run Parallel
Ensure parallel execution flow when requesting remote services, database calls, and file system access. Parallelizing tasks will greatly reduce latency and minimize any blocking operations. Parallel operation means running multiple things at the same time. With parallel, you do not have control of what finishes before the other, as your code will be optimized to run tasks at the same time. Generally, Node.js does not technically execute these multiple tasks at the same time. What happens is that each task is pushed to an asynchronous event loop with no control of which task will finish before the other. If your execution needs to complete one or more tasks before the other please consider going asynchronous.
For example:
Async.js is used to run parallel functions.
Simple example using async.parallel with an array
Syntax: async.parallel(tasks, callback).
Parameters
- Tasks: tasks to be executed such as arrays, objects, etc.
- Callback: pass all tasks results and execute when all tasks compilation is completed.
const async = require("async");
async.parallel(
[
function (callback) {
setTimeout(function () {
console.log("Task One");
callback(null, 1);
}, 200);
},
function (callback) {
setTimeout(function () {
console.log("Task Two");
callback(null, 2);
}, 100);
},
],
function (err, results) {
if (err) {
return console.error(err);
}
console.log(results);
// tasks execution order(Task Two then Task One). The result array will be [ 1, 2 ] even though the second function had a shorter timeout.
}
);
Using an object instead of an array
const async = require("async");
// an example using an object instead of an array
async.parallel({
task1: function(callback) {
setTimeout(function() {
console.log('Task One');
callback(null, 1);
}, 200);
},
task2: function(callback) {
setTimeout(function() {
console.log('Task Two');
callback(null, 2);
}, 100);
}
}, function(err, results) {
console.log(results);
// results now equals to: {task2: 2, task1: 1}
});
Client-side Rendering
Powerful MVC/MVVM has contributed to the growth of single-page apps. Frameworks such as AngularJS, Ember, Meteor, and BackboneJS are examples of current powerful MVC technology for client-side rendering.
These frameworks return dynamic data as JSON and display them on a webpage UI rendered on the client-side. What this means is that no markup layout will be sent with each request. Plain JSON will be sent and then rendered on the client-side making the page static on page reload. This saves on bandwidth, which translates to a higher-speed and performance.
Gzip Compression
Gzip compresses HTTP requests and responses. Gzip compresses responses before sending them to the browser, thus, the browser takes a shorter time to fetch them. Gzip also compresses the request to the remote server, which significantly increases web performance.
When using Express.js, choose to use compression. It is a Node.js compression middleware that supports deflate, Gzip, and serve static content. The compression library will compress every request that passes through the middleware.
Example
const compression = require("compression");
const express = require("express");
const app = express();
// compress all responses
app.use(compression());
// add all routes
Here are more details on how to connect express with compression.
Avoid Memory Leaks
A memory leak occurs when a computer program incorrectly manages memory allocations in a way that memory that is no longer needed is not released. With memory leaks, a loaded page will reserve more and more memory. These will slowly occupy your available space to a point where CPU usage is overstretched. Such scenarios will severely affect your application performance.
You may choose to restart your application and these issues will magically disappear but this will not be a reliable solution. The problem will repeat itself and the memory will pile up periodically. Memory leaks can come from a DOM that has been removed from a page but some variables still have a reference to them. If you happen to remove DOM values, make sure to set them to null. When null, the garbage collector will eliminate them and help avoid memory leaks.
Use Chrome Dev Tools to analyze if your web page has any memory leaks. Chrome Dev Tools will help you catch and debug memory leaks. This article will help you understand more about how to detect and solve memory leaks using Chrome Dev Tools.
Real-time Monitoring
Monitoring helps get insight into your production application to ensure a stable and reliable system. Insight is critical in helping detect performance problems within your system.
As a developer, you need to know when your system is down before your customers start complaining about your faulty system. That is why you need real-time alerts so you can be notified immediately.
Proper monitoring also helps get insight into the features of your application's behavior. You need to know how much time your app takes to run each function in the production environment. Also, if you are using microservices, you need to monitor network connections and lower delays in the communication between the two services. A few commonly used third-party tools to monitor Node.js apps includes
You may consider using Google Analytics to get insight such as user visits, traffic, traffic sources, bounce rate, user retention rate, sessions, session durations as well as page views.
Node.js monitoring frameworks include Hapi, Restify, Derby, Express, Koa, and Mocha.
Keeping your Code Light and Compact
When developing mobile and web apps, make sure you apply the concept of making your codebase small and compact for both the client code and server code. This will reduce latency and load times.
When it comes to making your code light, a single page web app is a great choice to consider. For example, let's assume your web app has a page with six JavaScript files. When this page is accessed in the browser, that browser will make six HTTP requests to fetch all your files. This will create a block and wait scenario. This is a good example of when you could minimize and link your multiple files into one to avoid scenarios like this.
Node.js has a lot of open source libraries and modules. During the development stage, you need to ask yourself why are you using this framework over another. You need to find out if a framework is worth using or if there are other simpler ways to write your code.
The point is when you consider using a framework it should be well worth it. This does not mean choosing frameworks is bad. Frameworks are great. They are scalable and have undisputed benefits. Consider using a framework only if it will simplify your code.
For example:
When dealing with date objects, it would be advisable to use a library like Moment.js instead of using the native JavaScript date object. Not because JavaScript date objects are bad in any way, but you will need to add a lot of code setups just to set up and display a simple date format.
For example:
- Using native JavaScript date object
const NowDate = new Date();
const months = ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December'];
const day = NowDate.getDate();
const month = NowDate.getMonth() + 1;
const year = NowDate.getFullYear();
console.log(year + '-' + month + '-' + day);
- Using Moment.js
const moment = require('moment')
const today = moment();
console.log(today.format('YYYY-M-D'));
const format = require("date-fns/format");
const today = format(new Date(), "yyyy-M-dd");
console.log(today);
As a developer, which of the example methods above would you consider?
Remember all these code blocks outputs today's date. When working with frameworks. Consider looking at alternative modules that are relatable to your code context.
For example, date-fns is an alternative date object library to Moment.js. Date-fns is considered more light-weight compared to Moment.js yet, they do the same job.
As a developer, you need to understand that dilemma and wisely choose which library to use during the development stage. In such a scenario, date-fns will improve your bundle size since it is smaller and help avoid performance overhead.
Conclusion
Node.js performance optimization is a broad topic with much ground to cover. This article only covered the tip of the iceberg about some of the common practices you can adopt to get higher-performance results.
Other common practices that you should also consider include:
- Using the latest stable Node.js updates
- Load balancing
- Memory optimization
- CPU profiling
- Node.js timers to schedule tasks
- Prioritize access to local variables
- Avoid using too much memory
- Eliminate unused components of .js libraries
- Removing unused lines of codes
- Having a well-defined execution context
- Avoiding global variables
Consider doing some work on your own and find out which segment of your app needs to be optimized. However, as you practice these optimization tips, do not forget security practices such as SSL/TLS and HTTP/2.
Peer Review Contributions by: Mike White
