Skip to main content

Image processing with HTML canvas

· 11 min read
Simon Porritt

jsPlumb and HTML canvas go back a long way, right to the start, in fact. We got our initial inspiration from Yahoo Pipes, making use of the fact that you can draw bezier curves into canvas elements. Over time the canvas element has become more and more powerful, and it can now do some pretty fancy image processing tasks. For our image processor starter app we wrote a bunch of methods to perform various filtering and transformation operations, collating them from various corners of the internet, and so we thought maybe it would be useful for others if we documented them.

We've released these functions on NPM as an MIT licensed library - take a look at the project page on Github for installation instructions.

Super fast canvas primer

If you're completely new to HTML canvas then there are few things you need to know:

  • a canvas has a width and height and it's definitely best to set these as attributes on the element rather than relying on CSS.
  • all operations on a canvas are not executed on the canvas itself but rather on a context, of which there are a few types, but the one you'll use for basic drawing is '2d'.

You can get a canvas context like this:

const canvas = document.getElementById("myCanvas")
const ctx = canvas.getContext("2d")

You'll see this call throughout the code for our image processor. There's a good primer on the canvas element on MDN.

Image Processing

Two main types of operations when processing images are:

  • Filtering A filter takes an input image and some optional parameters and returns an output image - things such as converting an image to grayscale, inverting the pixel values, adjusting contrast/brightness, etc.

  • Transformation Transformations take one or more images and some parameters and return an output image - things such as blending images, overlaying one image onto another, cropping, resizing, etc.

There are plenty of other parts to an image processor, of course - getting images in and out being not the least of them, but in this article we're going to focus on filters and transformations, and how to do these things with an HTML canvas.

Converting to and from canvas

You'll see references in the code snippets below to a method called canvasToImage. It's responsible for converting the contents of some canvas into an Image.

export async function canvasToImage(canvas:HTMLCanvasElement, type?:string, quality?:number):Promise<HTMLImageElement> {

return new Promise(function(resolve, reject) {

const d = canvas.toDataURL(type, quality)
const oo = new Image()
oo.onload = function () {
resolve(oo)
}

oo.src = d
})
}

type is a string such as "image/png" or "image/jpeg". Theoretically the supported values for this argument are browser specific but nowadays all browsers support PNG and JPEG, and PNG is always the default (ie. if you do not provide a value). quality is used when you're exporting a JPG.

Filtering with an HTML canvas

The code snippets presented here come from the open source canvas image processing library we recently released. The basic approach when filtering with a canvas is:


// 1. get the context from the canvas
const ctx = canvas.getContext("2d")
// 2. apply a filter
ctx.filter = "invert(50%)"
// 3. draw into it
ctx.drawImage(someImage, 0, 0)

Grayscale

A grayscale filter desaturates an image.


async function filterGrayscale(image:HTMLImageElement, amount?:number):Promise<HTMLImageElement> {

const c1 = document.createElement("canvas")
c1.width = image.naturalWidth
c1.height = image.naturalHeight

const ctx1 = c1.getContext("2d")
ctx1.filter = `grayscale(${amount}%)`
ctx1.drawImage(image, 0, 0, c1.width, c1.height)

return canvasToImage(c1)
}

Sepia

Apply a sepia effect to an image. amount determines the depth of the effect.


async function filterSepia(image:HTMLImageElement, amount:number):Promise<HTMLImageElement> {

const c1 = document.createElement("canvas")
c1.width = image.naturalWidth
c1.height = image.naturalHeight

const ctx1 = c1.getContext("2d")
ctx1.filter = `sepia(${amount}%)`
ctx1.drawImage(image, 0, 0, c1.width, c1.height)

return canvasToImage(c1)
}

Invert


async function filterInvert(image:HTMLImageElement, amount:number):Promise<HTMLImageElement> {

const c1 = document.createElement("canvas")
c1.width = image.naturalWidth
c1.height = image.naturalHeight

const ctx1 = c1.getContext("2d")
ctx1.filter = `invert(${amount}%)`
ctx1.drawImage(image, 0, 0, c1.width, c1.height)

return canvasToImage(c1)
}

Saturate

Takes a value with a minimum of 0 and no specific maximum, and adjusts the saturation of the image. Between 0 and 100, the saturate filter works as an inverse of the grayscale filter. Values of greater than 100 will oversaturate the image.


async function filterSaturate(image:HTMLImageElement, amount:number):Promise<HTMLImageElement> {

const c1 = document.createElement("canvas")
c1.width = image.naturalWidth
c1.height = image.naturalHeight

const ctx1 = c1.getContext("2d")
ctx1.filter = `saturate(${amount}%)`
ctx1.drawImage(image, 0, 0, c1.width, c1.height)

return canvasToImage(c1)
}

This cow on the right is 250% saturated.

Opacity

Adjusts the opacity of an image. Valid values for amount are 0-100, inclusive.


async function filterOpacity(image:HTMLImageElement, amount:number):Promise<HTMLImageElement> {

const c1 = document.createElement("canvas")
c1.width = image.naturalWidth
c1.height = image.naturalHeight

const ctx1 = c1.getContext("2d")
ctx1.filter = `opacity(${amount}%)`
ctx1.drawImage(image, 0, 0, c1.width, c1.height)

return canvasToImage(c1)
}

Blur

Blurs an image, using a radius of some number of pixels.


async function filterBlur(image:HTMLImageElement, radius:number):Promise<HTMLImageElement> {

const c1 = document.createElement("canvas")
c1.width = image.naturalWidth
c1.height = image.naturalHeight

const ctx1 = c1.getContext("2d")
ctx1.filter = `blur(${radius}px)`
ctx1.drawImage(image, 0, 0, c1.width, c1.height)

return canvasToImage(c1)
}

This cow on the right has been blurred with a radius of 10 pixels (the default in the canvas image processing library):

Blur gets out of hand pretty quickly. Here's a 75 pixel blurred cow:

Of course the effect of the blur depends on the size of the image.

Brightness

Adjusts the brightness of an image. Valid values for amount are anything from 0 up. 100 means standard brightness. 200 - what you see below - is pretty bright.


async function filterBrightness(image:HTMLImageElement, amount:number):Promise<HTMLImageElement> {

const c1 = document.createElement("canvas")
c1.width = image.naturalWidth
c1.height = image.naturalHeight

const ctx1 = c1.getContext("2d")
ctx1.filter = `brightness(${amount}%)`
ctx1.drawImage(image, 0, 0, c1.width, c1.height)

return canvasToImage(c1)
}

Contrast

Adjusts the contrast of an image. Valid values for amount are anything from 0 up. A value of 0 will give you a completely black image; 100 is normal. The default in our canvas processing library for this operation is 200...but that's really just to have a default.


async function filterContrast(image:HTMLImageElement, amount:number):Promise<HTMLImageElement> {

const c1 = document.createElement("canvas")
c1.width = image.naturalWidth
c1.height = image.naturalHeight

const ctx1 = c1.getContext("2d")
ctx1.filter = `constrast(${amount}%)`
ctx1.drawImage(image, 0, 0, c1.width, c1.height)

return canvasToImage(c1)
}

A cow with 200 contrast:


Transforming images with an HTML canvas

Transformations are a little more complex than filters, and may involve one or more images, as well as other arbitrary input values. The power of HTML canvas, though, makes many of these operations very straightforward.

Mirroring an image

Also known as 'flip', if you're a GIMP user like me. Once again you can use a canvas for this:

async function mirrorImage(img:HTMLImageElement, axis:'x' | 'y' | 'x_y'):Promise<HTMLImageElement> {

// get a canvas, set its dimensions to match the image, and get the 2d context
const outputImage = document.createElement("canvas")
outputImage.width = img.naturalWidth
outputImage.height = img.naturalHeight
const ctx = outputImage.getContext("2d");

// setup transformation values for x and y. This method supports flipping in x, y, or both x _and_ y. We use
// the `scale` property of the canvas context to set this up: if it should be flipped in a specific axis, we use a value of
// -1 for that axis, otherwise we use a value of 1.
const dx = axis === 'y' ? 1 : -1,
x = axis === 'y' ? 0 : -outputImage.width

const dy = axis === 'x' ? 1 : -1,
y = axis === 'x' ? 0 : -outputImage.height

// apply the scale
ctx.scale(dx, dy);

// Draw the image on the canvas
ctx.drawImage(img, x, y);

return canvasToImage(outputImage)
}

This cow looks pretty much the same flipped horizontally:

Maybe that's why it's so appealing, that whole thing about symmetry. Just to show you the mirror image is actually working let's flip the cow in both axes:

Applying a threshold

Out of all the operations discussed in this article, this is the one thing we couldn't do just with some canvas tricks. To apply a threshold to an image we have to get in and mess about with its data:

async function imageThreshold(image:HTMLImageElement, threshold:number, value:number = 255):Promise<HTMLImageElement> {

// get a canvas, set its dimensions, and draw the image in.
const outputImage = document.createElement("canvas");
const ctx1 = outputImage.getContext("2d")
outputImage.width = image.naturalWidth;
outputImage.height = image.naturalHeight;
ctx1.drawImage(image, 0, 0)

// extract the underlying image data
const data1 = ctx1.getImageData(0,0, image.width, image.height)

// each pixel in the image data in a canvas consists of 4 bytes: red, green blue and alpha values.
// we loop through and check each value. Where the pixel is above the threshold we clamp it the value we want. otherwise
// we set it to zero.
for(let i = 0; i < data1.data.length; i += 4) {

const red = data1.data[i]
const green = data1.data[i + 1]
const blue = data1.data[i + 2]

data1.data[i] = red >= threshold ? value : 0
data1.data[i+1] = green >= threshold ? value : 0
data1.data[i+2] = blue>= threshold ? value : 0

}
ctx1.putImageData(data1, 0, 0)
return canvasToImage(outputImage)
}

This is a cow with a threshold of 120 applied to it (and the default of 255 for value):

This is the same threshold but with a value of 120:

Cropping an image

Canvas makes it easy to crop an image:

async function cropImage(image:HTMLImageElement, x:number, y:number, w:number, h:number):Promise<HTMLImageElement> {

// get a canvas and set its dimensions
const c1 = document.createElement("canvas")
c1.width = w
c1.height = h
const ctx1 = c1.getContext("2d")

// draw the image in to the canvas, starting at origin {x,y} on the image, and using the given width and height
ctx1.drawImage(image, x, y, w, h, 0, 0, w, h)
return canvasToImage(c1)
}

Overlaying an image

To overlay an image you just draw both images, one after the other:

async function overlayImage(image1:HTMLImageElement, image2:HTMLImageElement, x?:number, y?:number):Promise<HTMLImageElement> {

const c1 = document.createElement("canvas")
c1.width = image1.naturalWidth
c1.height = image1.naturalHeight
const ctx1 = c1.getContext("2d")

ctx1.drawImage(image1, 0, 0, c1.width, c1.height)
ctx1.drawImage(image2, x || 0, y || 0, image2.naturalWidth, image2.naturalHeight)

return canvasToImage(c1)
}

Clipping an image

Clipping an image is kind of like overlaying an image - you draw both images into the canvas, but with 2 differences:

  • you draw the mask first
  • you set the globalCompositeOperation to source-in before you draw the image you wish to clip
async function clipImage(image:HTMLImageElement, mask:HTMLImageElement):Promise<HTMLImageElement> {

const c1 = document.createElement("canvas")
const ctx1 = c1.getContext("2d")
c1.width = image.naturalWidth
c1.height = image.naturalHeight

ctx1.drawImage(mask, 0, 0)

ctx1.globalCompositeOperation = "source-in"
ctx1.drawImage(image, 0, 0)

return canvasToImage(c1)
}

Blending images

This is also like overlaying images and clipping images, and it all comes down to the globalCompositeOperation. A good resource for finding out what you can do with the globalCompositeOperation is the MDN documentation.

async function blendImages(image1:HTMLImageElement, image2:HTMLImageElement, blendMode?:GlobalCompositeOperation):Promise<HTMLImageElement> {

const c1 = document.createElement(TAG_CANVAS)
c1.width = image1.naturalWidth
c1.height = image1.naturalHeight

const ctx1 = c1.getContext("2d")
ctx1.drawImage(image1, 0, 0, c1.width, c1.height)

ctx1.globalCompositeOperation = blendMode || GLOBAL_COMPOSITE_MULTIPLY
ctx1.drawImage(image2, 0, 0, c1.width, c1.height)

return canvasToImage(c1)
}

Here, instead of overlaying or clipping our cow and its sunglasses, we'll blend them, using a multiply operation (since that's the default in the jsPlumb canvas image processor that were using):

Blend modes are hours of fun. Here's screen:

This is hard-light:


Further Reading

The key to many of the transformation and filtering operations with an HTML canvas is the globalCompositeOperation, which sets the type of compositing operation that a canvas will use when new content is drawn into it. It's an extremely powerful and versatile concept and if you're planning on seriously getting into image processing in webapps it's well worth your time looking into what it can be used for. Another good article on MDN that talks about this concept is this one about clipping - which we'll discuss a little later in this article.

To mess around in a live setting with these filters and transforms, try our Image Processor starter app

You can download/browse the source of the image processing helpers we created on Github.


Start a free trial

Sending your evaluation request...

Interested in the concepts discussed in this article? Start a free trial and see how jsPlumb can help bring your ideas to market in record time.


Get in touch!

If you'd like to discuss any of the ideas/concepts in this article we'd love to hear from you - drop us a line at hello@jsplumbtoolkit.com.