How to find number of pixels in an image Python

I wanted to extract each pixel values so that i can use them for locating simple objects in an image. Every image is made up of pixels and when these values are extracted using python, four values are obtained for each pixel (R,G,B,A). This is called the RGBA color space having the Red, Green, Blue colors and Alpha value respectively.

In python we use a library called PIL (python imaging Library). The modules in this library is used for image processing and has support for many file formats like png, jpg, bmp, gif etc. It comes with large number of functions that can be used to open, extract data, change properties, create new images and much more…

PIL comes pre installed with python2.7 in Ubuntu but for windows it has to be installed manually. But for either operating systems having python2.7 or more can be downloaded from here. and for python3 it can be downloaded from here.

each pixel value can be extracted and stored in a list.Though IDLE shell can be used for it, it can take a long time to extract the values and hence its recommended that it is done using command line interface. The procedure for extraction is :

Recognise morphometric problems (those dealing with the number, size, or shape of the objects in an image).

As computer systems have become faster and more powerful, and cameras and other imaging systems have become commonplace in many other areas of life, the need has grown for researchers to be able to process and analyse image data. Considering the large volumes of data that can be involved - high-resolution images that take up a lot of disk space/virtual memory, and/or collections of many images that must be processed together - and the time-consuming and error-prone nature of manual processing, it can be advantageous or even necessary for this processing and analysis to be automated as a computer program.

This lesson introduces an open source toolkit for processing image data: the Python programming language and the scikit-image (

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

Uses of Image Processing in Research

Automated processing can be used to analyse many different properties of an image, including the distribution and change in colours in the image, the number, size, position, orientation, and shape of objects in the image, and even - when combined with machine learning techniques for object recognition - the type of objects in the image.

Some examples of image processing methods applied in research include:

• imaging a Black Hole
• estimating the population of Emperor Penguins
• the global-scale analysis of marine plankton diversity
• segmentation of liver and vessels from CT images

With this lesson, we aim to provide a thorough grounding in the fundamental concepts and skills of working with image data in Python. Most of the examples used in this lesson focus on one particular class of image processing technique, morphometrics, but what you will learn can be used to solve a much wider range of problems.

Morphometrics

Morphometrics involves counting the number of objects in an image, analyzing the size of the objects, or analyzing the shape of the objects. For example, we might be interested in automatically counting the number of bacterial colonies growing in a Petri dish, as shown in this image:

We could use image processing to find the colonies, count them, and then highlight their locations on the original image, resulting in an image like this:

Why write a program to do that?

Note that you can easily manually count the number of bacteria colonies shown in the morphometric example above. Why should we learn how to write a Python program to do a task we could easily perform with our own eyes? There are at least two reasons to learn how to perform tasks like these with Python and skimage:

1. What if there are many more bacteria colonies in the Petri dish? For example, suppose the image looked like this:

   

   Manually counting the colonies in that image would present more of a challenge. A Python program using skimage could count the number of colonies more accurately, and much more quickly, than a human could.

2. What if you have hundreds, or thousands, of images to consider? Imagine having to manually count colonies on several thousand images like those above. A Python program using skimage could move through all of the images in seconds; how long would a graduate student require to do the task? Which process would be more accurate and repeatable?

As you can see, the simple image processing / computer vision techniques you will learn during this workshop can be very valuable tools for scientific research.

As we move through this workshop, we will learn image analysis methods useful for many different scientific problems. These will be linked together and applied to a real problem in the final end-of-workshop capstone challenge.

Let’s get started, by learning some basics about how images are represented and stored digitally.

Key Points

• Simple Python and skimage (scikit-image) techniques can be used to solve genuine image analysis problems.

• Morphometric problems involve the number, shape, and / or size of the objects in an image.

Overview

Teaching: 20 min
Exercises: 5 min

Questions

• How are images represented in digital format?

Objectives

• Define the terms bit, byte, kilobyte, megabyte, etc.

• Explain how a digital image is composed of pixels.

• Recommend using imageio (resp. skimage) for I/O (resp. image processing) tasks.

• Explain how images are stored in NumPy arrays.

• Explain the left-hand coordinate system used in digital images.

• Explain the RGB additive colour model used in digital images.

• Explain the order of the three colour values in skimage images.

• Explain the characteristics of the BMP, JPEG, and TIFF image formats.

• Explain the difference between lossy and lossless compression.

• Explain the advantages and disadvantages of compressed image formats.

• Explain what information could be contained in image metadata.

The images we see on hard copy, view with our electronic devices, or process with our programs are represented and stored in the computer as numeric abstractions, approximations of what we see with our eyes in the real world. Before we begin to learn how to process images with Python programs, we need to spend some time understanding how these abstractions work.

Pixels

It is important to realise that images are stored as rectangular arrays of hundreds, thousands, or millions of discrete “picture elements,” otherwise known as pixels. Each pixel can be thought of as a single square point of coloured light.

For example, consider this image of a maize seedling, with a square area designated by a red box:

Now, if we zoomed in close enough to see the pixels in the red box, we would see something like this:

Note that each square in the enlarged image area - each pixel - is all one colour, but that each pixel can have a different colour from its neighbors. Viewed from a distance, these pixels seem to blend together to form the image we see.

Working with Pixels

As noted, in practice, real world images will typically be made up of a vast number of pixels, and each of these pixels will be one of potentially millions of colours. While we will deal with pictures of such complexity shortly, let’s start our exploration with 15 pixels in a 5 X 3 matrix with 2 colours and work our way up to that complexity.

Matrices, arrays, images and pixels

The matrix is mathematical concept - numbers evenly arranged in a rectangle. This can be a two dimensional rectangle, like the shape of the screen you’re looking at now. Or it could be a three dimensional equivalent, a cuboid, or have even more dimensions, but always keeping the evenly spaced arrangement of numbers. In computing, array refers to a structure in the computer’s memory where data is stored in evenly-spaced elements. This is strongly analogous to a matrix. A

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

33 array is a type of variable (a simpler example of a type is an integer). For our purposes, the distinction between matrices and arrays is not important, we don’t really care how the computer arranges our data in its memory. The important thing is that the computer stores values describing the pixels in images, as arrays. And the terms matrix and array can be used interchangeably.

First, the necessary imports:

"""
 * Python libraries for learning and performing image processing.
 *
"""
import numpy as np
import matplotlib.pyplot as plt
import ipympl
import imageio.v3 as iio
import skimage

Import Statements in Python

In Python, the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

34 statement is used to load additional functionality into a program. This is necessary when we want our code to do something more specialised, which cannot easily be achieved with the limited set of basic tools and data structures available in the default Python environment.

Additional functionality can be loaded as a single function or object, a module defining several of these, or a library containing many modules. You will encounter several different forms of

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

34 statement.

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

Further Explanation

In the example above, form 1 loads the entire

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

32 library into the program as an object. Individual modules of the library are then available within that object, e.g., to access the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

37 function used in the drawing episode, you would write

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

38.

Form 2 loads only the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

39 module of

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

32 into the program. When we run the code, the program will take less time and use less memory because we will not load the whole

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

32 library. The syntax needed to use the module remains unchanged: to access the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

37 function, we would use the same function call as given for form 1.

To further reduce the time and memory requirements for your program, form 3 can be used to import only a specific function/class from a library/module. Unlike the other forms, when this approach is used, the imported function or class can be called by its name only, without prefixing it with the name of the module/library from which it was loaded, i.e.,

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

43 instead of

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

38 using the example above. One hazard of this form is that importing like this will overwrite any object with the same name that was defined/imported earlier in the program, i.e., the example above would replace any existing object called

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

37 with the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

37 function from

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

47.

Finally, the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

48 keyword can be used when importing, to define a name to be used as shorthand for the library/module being imported. This name is referred to as an alias. Typically, using an alias (such as

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

49 for the NumPy library) saves us a little typing. You may see

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

48 combined with any of the other first three forms of

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

34 statement.

Which form is used often depends on the size and number of additional tools being loaded into the program.

Now that we have our libraries loaded, we will run a Jupyter Magic Command that will ensure our images display in our Jupyter document with pixel information that will help us more efficiently run commands later in the session.

%matplotlib widget

With that taken care of, let’s load our image data from disk using the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

Why not use import skimage # form 1, load whole skimage library import skimage.draw # form 2, load skimage.draw module only from skimage.draw import disk # form 3, load only the disk function import numpy as np # form 4, load all of numpy into an object called np 59

The

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

32 library has its own function to read an image, so you might be asking why we don’t use it here. Actually,

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

59 uses

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

62 internally when loading an image into Python. It is certainly something you may use as you see fit in your own code. In this lesson, we use the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

56 library to read or write (save) images, while

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

32 is dedicated to performing operations on the images. Using

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

56 gives us more flexibility, especially when it comes to handling metadata.

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

You might be thinking, “That does look vaguely like an eight, and I see two colours but how can that be only 15 pixels”. The display of the eight you see does use a lot more screen pixels to display our eight so large, but that does not mean there is information for all those screen pixels in the file. All those extra pixels are a consequence of our viewer creating additional pixels through interpolation. It could have just displayed it as a tiny image using only 15 screen pixels if the viewer was designed differently.

While many image file formats contain descriptive metadata that can be essential, the bulk of a picture file is just arrays of numeric information that, when interpreted according to a certain rule set, become recognizable as an image to us. Our image of an eight is no exception, and

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

print(image.shape)
print(image)

(5, 3)
[[0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

Thus if we have tools that will allow us to manipulate these arrays of numbers, we can manipulate the image. The

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

To make it look like a zero, we need to change the number underlying the centremost pixel to be 1. With the help of those row and column headers, at this small scale we can determine the centre pixel is in row labeled 2 and column labeled 1. Using array slicing, we can then address and assign a new value to that position.

zero = iio.imread(uri="data/eight.tif")
zero[2,1]= 1.0
"""
The follwing line of code creates a new figure for imshow to use in displaying our output. Without it, plt.imshow() would overwrite our previous image in the cell above
"""
fig, ax = plt.subplots()
plt.imshow(zero)
print(zero)

[[0. 0. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

Coordinate system

When we process images, we can access, examine, and / or change the colour of any pixel we wish. To do this, we need some convention on how to access pixels individually; a way to give each one a name, or an address of a sort.

The most common manner to do this, and the one we will use in our programs, is to assign a modified Cartesian coordinate system to the image. The coordinate system we usually see in mathematics has a horizontal x-axis and a vertical y-axis, like this:

The modified coordinate system used for our images will have only positive coordinates, the origin will be in the upper left corner instead of the centre, and y coordinate values will get larger as they go down instead of up, like this:

This is called a left-hand coordinate system. If you hold your left hand in front of your face and point your thumb at the floor, your extended index finger will correspond to the x-axis while your thumb represents the y-axis.

Until you have worked with images for a while, the most common mistake that you will make with coordinates is to forget that y coordinates get larger as they go down instead of up as in a normal Cartesian coordinate system. Consequently, it may be helpful to think in terms of counting down rows (r) for the y-axis and across columns (c) for the x-axis. This can be especially helpful in cases where you need to transpose image viewer data provided in x,y format to y,x format. Thus, we will use cx and ry where appropriate to help bridge these two approaches.

Changing Pixel Values (5 min)

Load another copy of eight named five, and then change the value of pixels so you have what looks like a 5 instead of an 8. Display the image and print out the matrix as well.

Solution

There are many possible solutions, but one method would be . . .

five = iio.imread(uri="data/eight.tif")
five[1,2]= 1.0
five[3,0]= 1.0
fig, ax = plt.subplots()
plt.imshow(five)
print(five)

[[0. 0. 0.]
 [0. 1. 1.]
 [0. 0. 0.]
 [1. 1. 0.]
 [0. 0. 0.]]

More colours

Up to now, we only had a 2 colour matrix, but we can have more if we use other numbers or fractions. One common way is to use the numbers between 0 and 255 to allow for 256 different colours or 256 different levels of grey. Let’s try that out.

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

We now have 3 colours, but are they the three colours you expected? They all appear to be on a continuum of dark purple on the low end and yellow on the high end. This is a consequence of the default colour map (cmap) in this library. You can think of a colour map as an association or mapping of numbers to a specific colour. However, the goal here is not to have one number for every possible colour, but rather to have a continuum of colours that demonstrate relative intensity. In our specific case here for example, 255 or the highest intensity is mapped to yellow, and 0 or the lowest intensity is mapped to a dark purple. The best colour map for your data will vary and there are many options built in, but this default selection was not arbitrary. A lot of science went into making this the default due to its robustness when it comes to how the human mind interprets relative colour values, grey-scale printability, and colour-blind friendliness (You can read more about this default colour map in a Matplotlib tutorial and an explanatory article by the authors). Thus it is a good place to start, and you should change it only with purpose and forethought. For now, let’s see how you can do that using an alternative map you have likely seen before where it will be even easier to see it as a mapped continuum of intensities: greyscale.

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

Above we have exactly the same underying data matrix, but in greyscale. Zero maps to black, 255 maps to white, and 128 maps to medium grey. Here we only have a single channel in the data and utilize a grayscale color map to represent the luminance, or intensity of the data and correspondingly this channel is referred to as the luminance channel.

Even More Colours

This is all well and good at this scale, but what happens when we instead have a picture of a natural landscape that contains millions of colours. Having a one to one mapping of number to colour like this would be inefficient and make adjustments and building tools to do so very difficult. Rather than larger numbers, the solution is to have more numbers in more dimensions. Storing the numbers in a multi-dimensional matrix where each colour or property like transparency is associated with its own dimension allows for individual contributions to a pixel to be adjusted independently. This ability to manipulate properties of groups of pixels separately will be key to certain techniques explored in later chapters of this lesson. To get started let’s see an example of how different dimensions of information combine to produce a set of pixels using a 4 X 4 matrix with 3 dimensions for the colours red, green, and blue. Rather than loading it from a file, we will generate this example using numpy.

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

Previously we had one number being mapped to one colour or intensity. Now we are combining the effect of 3 numbers to arrive at a single colour value. Let’s see an example of that using the blue square at the end of the second row, which has the index [1, 3].

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

This outputs: array([ 7, 1, 110]) The integers in order represent Red, Green, and Blue. Looking at the 3 values and knowing how they map, can help us understand why it is blue. If we divide each value by 255, which is the maximum, we can determine how much it is contributing relative to its maximum potential. Effectively, the red is at 7/255 or 2.8 percent of its potential, the green is at 1/255 or 0.4 percent, and blue is 110/255 or 43.1 percent of its potential. So when you mix those three intensities of colour, blue is winning by a wide margin, but the red and green still contribute to make it a slightly different shade of blue than 0,0,110 would be on its own.

These colours mapped to dimensions of the matrix may be referred to as channels. It may be helpful to display each of these channels independently, to help us understand what is happening. We can do that by multiplying our image array representation with a 1d matrix that has a one for the channel we want to keep and zeros for the rest.

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

If we look at the upper [1, 3] square in all three figures, we can see each of those colour contributions in action. Notice that there are several squares in the blue figure that look even more intensely blue than square [1, 3]. When all three channels are combined though, the blue light of those squares is being diluted by the relative strength of red and green being mixed in with them.

24-bit RGB Colour

This last colour model we used, known as the RGB (Red, Green, Blue) model, is the most common.

As we saw, the RGB model is an additive colour model, which means that the primary colours are mixed together to form other colours. Most frequently, the amount of the primary colour added is represented as an integer in the closed range [0, 255] as seen in the example. Therefore, there are 256 discrete amounts of each primary colour that can be added to produce another colour. The number of discrete amounts of each colour, 256, corresponds to the number of bits used to hold the colour channel value, which is eight (28=256). Since we have three channels with 8 bits for each (8+8+8=24), this is called 24-bit colour depth.

Any particular colour in the RGB model can be expressed by a triplet of integers in [0, 255], representing the red, green, and blue channels, respectively. A larger number in a channel means that more of that primary colour is present.

Thinking about RGB colours (5 min)

Suppose that we represent colours as triples (r, g, b), where each of r, g, and b is an integer in [0, 255]. What colours are represented by each of these triples? (Try to answer these questions without reading further.)

1. (255, 0, 0)
2. (0, 255, 0)
3. (0, 0, 255)
4. (255, 255, 255)
5. (0, 0, 0)
6. (128, 128, 128)

Solution

1. (255, 0, 0) represents red, because the red channel is maximised, while the other two channels have the minimum values.
2. (0, 255, 0) represents green.
3. (0, 0, 255) represents blue.
4. (255, 255, 255) is a little harder. When we mix the maximum value of all three colour channels, we see the colour white.
5. (0, 0, 0) represents the absence of all colour, or black.
6. (128, 128, 128) represents a medium shade of gray. Note that the 24-bit RGB colour model provides at least 254 shades of gray, rather than only fifty.

Note that the RGB colour model may run contrary to your experience, especially if you have mixed primary colours of paint to create new colours. In the RGB model, the lack of any colour is black, while the maximum amount of each of the primary colours is white. With physical paint, we might start with a white base, and then add differing amounts of other paints to produce a darker shade.

After completing the previous challenge, we can look at some further examples of 24-bit RGB colours, in a visual way. The image in the next challenge shows some colour names, their 24-bit RGB triplet values, and the colour itself.

RGB colour table (optional, not included in timing)

We cannot really provide a complete table. To see why, answer this question: How many possible colours can be represented with the 24-bit RGB model?

Solution

There are 24 total bits in an RGB colour of this type, and each bit can be on or off, and so there are 224 = 16,777,216 possible colours with our additive, 24-bit RGB colour model.

Although 24-bit colour depth is common, there are other options. We might have 8-bit colour (3 bits for red and green, but only 2 for blue, providing 8 × 8 × 4 = 256 colours) or 16-bit colour (4 bits for red, green, and blue, plus 4 more for transparency, providing 16 × 16 × 16 = 4096 colours), for example. There are colour depths with more than eight bits per channel, but as the human eye can only discern approximately 10 million different colours, these are not often used.

If you are using an older or inexpensive laptop screen or LCD monitor to view images, it may only support 18-bit colour, capable of displaying 64 × 64 × 64 = 262,144 colours. 24-bit colour images will be converted in some manner to 18-bit, and thus the colour quality you see will not match what is actually in the image.

We can combine our coordinate system with the 24-bit RGB colour model to gain a conceptual understanding of the images we will be working with. An image is a rectangular array of pixels, each with its own coordinate. Each pixel in the image is a square point of coloured light, where the colour is specified by a 24-bit RGB triplet. Such an image is an example of raster graphics.

Image formats

Although the images we will manipulate in our programs are conceptualised as rectangular arrays of RGB triplets, they are not necessarily created, stored, or transmitted in that format. There are several image formats we might encounter, and we should know the basics of at least of few of them. Some formats we might encounter, and their file extensions, are shown in this table:

BMP

The file format that comes closest to our preceding conceptualisation of images is the Device-Independent Bitmap, or BMP, file format. BMP files store raster graphics images as long sequences of binary-encoded numbers that specify the colour of each pixel in the image. Since computer files are one-dimensional structures, the pixel colours are stored one row at a time. That is, the first row of pixels (those with y-coordinate 0) are stored first, followed by the second row (those with y-coordinate 1), and so on. Depending on how it was created, a BMP image might have 8-bit, 16-bit, or 24-bit colour depth.

24-bit BMP images have a relatively simple file format, can be viewed and loaded across a wide variety of operating systems, and have high quality. However, BMP images are not compressed, resulting in very large file sizes for any useful image resolutions.

The idea of image compression is important to us for two reasons: first, compressed images have smaller file sizes, and are therefore easier to store and transmit; and second, compressed images may not have as much detail as their uncompressed counterparts, and so our programs may not be able to detect some important aspect if we are working with compressed images. Since compression is important to us, we should take a brief detour and discuss the concept.

Image compression

Before discussing additional formats, familiarity with image compression will be helpful. Let’s delve into that subject with a challenge. For this challenge, you will need to know about bits / bytes and how those are used to express computer storage capacities. If you already know, you can skip to the challenge below.

Bits and bytes

Before we talk specifically about images, we first need to understand how numbers are stored in a modern digital computer. When we think of a number, we do so using a decimal, or base-10 place-value number system. For example, a number like 659 is 6 × 102 + 5 × 101 + 9 × 100. Each digit in the number is multiplied by a power of 10, based on where it occurs, and there are 10 digits that can occur in each position (0, 1, 2, 3, 4, 5, 6, 7, 8, 9).

In principle, computers could be constructed to represent numbers in exactly the same way. But, the electronic circuits inside a computer are much easier to construct if we restrict the numeric base to only two, instead of 10. (It is easier for circuitry to tell the difference between two voltage levels than it is to differentiate among 10 levels.) So, values in a computer are stored using a binary, or base-2 place-value number system.

In this system, each symbol in a number is called a bit instead of a digit, and there are only two values for each bit (0 and 1). We might imagine a four-bit binary number, 1101. Using the same kind of place-value expansion as we did above for 659, we see that 1101 = 1 × 23 + 1 × 22 + 0 × 21 + 1 × 20, which if we do the math is 8 + 4 + 0 + 1, or 13 in decimal.

Internally, computers have a minimum number of bits that they work with at a given time: eight. A group of eight bits is called a byte. The amount of memory (RAM) and drive space our computers have is quantified by terms like Megabytes (MB), Gigabytes (GB), and Terabytes (TB). The following table provides more formal definitions for these terms.

UnitAbbreviationSizeKilobyteKB1024 bytesMegabyteMB1024 KBGigabyteGB1024 MBTerabyteTB1024 GB

BMP image size (optional, not included in timing)

Imagine that we have a fairly large, but very boring image: a 5,000 × 5,000 pixel image composed of nothing but white pixels. If we used an uncompressed image format such as BMP, with the 24-bit RGB colour model, how much storage would be required for the file?

Solution

In such an image, there are 5,000 × 5,000 = 25,000,000 pixels, and 24 bits for each pixel, leading to 25,000,000 × 24 = 600,000,000 bits, or 75,000,000 bytes (71.5MB). That is quite a lot of space for a very uninteresting image!

Since image files can be very large, various compression schemes exist for saving (approximately) the same information while using less space. These compression techniques can be categorised as lossless or lossy.

Lossless compression

In lossless image compression, we apply some algorithm (i.e., a computerised procedure) to the image, resulting in a file that is significantly smaller than the uncompressed BMP file equivalent would be. Then, when we wish to load and view or process the image, our program reads the compressed file, and reverses the compression process, resulting in an image that is identical to the original. Nothing is lost in the process – hence the term “lossless.”

The general idea of lossless compression is to somehow detect long patterns of bytes in a file that are repeated over and over, and then assign a smaller bit pattern to represent the longer sample. Then, the compressed file is made up of the smaller patterns, rather than the larger ones, thus reducing the number of bytes required to save the file. The compressed file also contains a table of the substituted patterns and the originals, so when the file is decompressed it can be made identical to the original before compression.

To provide you with a concrete example, consider the 71.5 MB white BMP image discussed above. When put through the zip compression utility on Microsoft Windows, the resulting .zip file is only 72 KB in size! That is, the .zip version of the image is three orders of magnitude smaller than the original, and it can be decompressed into a file that is byte-for-byte the same as the original. Since the original is so repetitious - simply the same colour triplet repeated 25,000,000 times - the compression algorithm can dramatically reduce the size of the file.

If you work with .zip or .gz archives, you are dealing with lossless compression.

Lossy compression

Lossy compression takes the original image and discards some of the detail in it, resulting in a smaller file format. The goal is to only throw away detail that someone viewing the image would not notice. Many lossy compression schemes have adjustable levels of compression, so that the image creator can choose the amount of detail that is lost. The more detail that is sacrificed, the smaller the image files will be - but of course, the detail and richness of the image will be lower as well.

This is probably fine for images that are shown on Web pages or printed off on 4 × 6 photo paper, but may or may not be fine for scientific work. You will have to decide whether the loss of image quality and detail are important to your work, versus the space savings afforded by a lossy compression format.

It is important to understand that once an image is saved in a lossy compression format, the lost detail is just that - lost. I.e., unlike lossless formats, given an image saved in a lossy format, there is no way to reconstruct the original image in a byte-by-byte manner.

JPEG

JPEG images are perhaps the most commonly encountered digital images today. JPEG uses lossy compression, and the degree of compression can be tuned to your liking. It supports 24-bit colour depth, and since the format is so widely used, JPEG images can be viewed and manipulated easily on all computing platforms.

Examining actual image sizes (optional, not included in timing)

Let us see the effects of image compression on image size with actual images. The following script creates a square white image 5000 X 5000 pixels, and then saves it as a BMP and as a JPEG image.

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

8

Examine the file sizes of the two output files,

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

70 and

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

71. Does the BMP image size match our previous prediction? How about the JPEG?

Solution

The BMP file,

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

70, is 75,000,054 bytes, which matches our prediction very nicely. The JPEG file,

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

71, is 392,503 bytes, two orders of magnitude smaller than the bitmap version.

Comparing lossless versus lossy compression (optional, not included in timing)

Let us see a hands-on example of lossless versus lossy compression. Once again, open a terminal and navigate to the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

74 directory. The two output images,

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

70 and

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

71, should still be in the directory, along with another image,

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

77.

We can apply lossless compression to any file by using the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

78 command. Recall that the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

70 file contains 75,000,054 bytes. Apply lossless compression to this image by executing the following command:

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

80. This command tells the computer to create a new compressed file,

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

81, from the original bitmap image. Execute a similar command on the tree JPEG file:

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

82.

Having created the compressed file, use the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

83 command to display the contents of the directory. How big are the compressed files? How do those compare to the size of

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

70 and

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

77? What can you conclude from the relative sizes?

Solution

Here is a partial directory listing, showing the sizes of the relevant files there:

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

9

We can see that the regularity of the bitmap image (remember, it is a 5,000 x 5,000 pixel image containing only white pixels) allows the lossless compression scheme to compress the file quite effectively. On the other hand, compressing

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

77 does not create a much smaller file; this is because the JPEG image was already in a compressed format.

Here is an example showing how JPEG compression might impact image quality. Consider this image of several maize seedlings (scaled down here from 11,339 × 11,336 pixels in order to fit the display).

Now, let us zoom in and look at a small section of the label in the original, first in the uncompressed format:

Here is the same area of the image, but in JPEG format. We used a fairly aggressive compression parameter to make the JPEG, in order to illustrate the problems you might encounter with the format.

The JPEG image is of clearly inferior quality. It has less colour variation and noticeable pixelation. Quality differences become even more marked when one examines the colour histograms for each image. A histogram shows how often each colour value appears in an image. The histograms for the uncompressed (left) and compressed (right) images are shown below:

We learn how to make histograms such as these later on in the workshop. The differences in the colour histograms are even more apparent than in the images themselves; clearly the colours in the JPEG image are different from the uncompressed version.

If the quality settings for your JPEG images are high (and the compression rate therefore relatively low), the images may be of sufficient quality for your work. It all depends on how much quality you need, and what restrictions you have on image storage space. Another consideration may be where the images are stored. For example,if your images are stored in the cloud and therefore must be downloaded to your system before you use them, you may wish to use a compressed image format to speed up file transfer time.

PNG

PNG images are well suited for storing diagrams. It uses a lossless compression and is hence often used in web applications for non-photographic images. The format is able to store RGB and plain luminance (single channel, without an associated color) data, among others. Image data is stored row-wise and then, per row, a simple filter, like taking the difference of adjacent pixels, can be applied to increase the compressability of the data. The filtered data is then compressed in the next step and written out to the disk.

TIFF

TIFF images are popular with publishers, graphics designers, and photographers. TIFF images can be uncompressed, or compressed using either lossless or lossy compression schemes, depending on the settings used, and so TIFF images seem to have the benefits of both the BMP and JPEG formats. The main disadvantage of TIFF images (other than the size of images in the uncompressed version of the format) is that they are not universally readable by image viewing and manipulation software.

Metadata

JPEG and TIFF images support the inclusion of metadata in images. Metadata is textual information that is contained within an image file. Metadata holds information about the image itself, such as when the image was captured, where it was captured, what type of camera was used and with what settings, etc. We normally don’t see this metadata when we view an image, but we can view it independently if we wish to (see , below). The important thing to be aware of at this stage is that you cannot rely on the metadata of an image being fully preserved when you use software to process that image. The image reader/writer library that we use throughout this lesson,

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

Accessing Metadata

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

53 provides a way to display or explore the metadata associated with an image. Metadata is served independently from pixel data:

%matplotlib widget

0

%matplotlib widget

1

Other software exists that can help you handle metadata, e.g., Fiji and ImageMagick. You may want to explore these options if you need to work with the metadata of your images.

Summary of image formats used in this lesson

The following table summarises the characteristics of the BMP, JPEG, and TIFF image formats:

Key Points

• Digital images are represented as rectangular arrays of square pixels.

• Digital images use a left-hand coordinate system, with the origin in the upper left corner, the x-axis running to the right, and the y-axis running down. Some learners may prefer to think in terms of counting down rows for the y-axis and across columns for the x-axis. Thus, we will make an effort to allow for both approaches in our lesson presentation.

• Most frequently, digital images use an additive RGB model, with eight bits for the red, green, and blue channels.

• skimage images are stored as multi-dimensional NumPy arrays.

• In skimage images, the red channel is specified first, then the green, then the blue, i.e., RGB.

• Lossless compression retains all the details in an image, but lossy compression results in loss of some of the original image detail.

• BMP images are uncompressed, meaning they have high quality but also that their file sizes are large.

• JPEG images use lossy compression, meaning that their file sizes are smaller, but image quality may suffer.

• TIFF images can be uncompressed or compressed with lossy or lossless compression.

• Depending on the camera or sensor, various useful pieces of information may be stored in an image file, in the image metadata.

Overview

Teaching: 70 min
Exercises: 50 min

Questions

• How can the skimage Python computer vision library be used to work with images?

Objectives

• Read and save images with imageio.

• Display images with matplotlib.

• Resize images with skimage.

• Perform simple image thresholding with NumPy array operations.

• Extract sub-images using array slicing.

We have covered much of how images are represented in computer software. In this episode we will learn some more methods for accessing and changing digital images.

Reading, displaying, and saving images

Imageio provides intuitive functions for reading and writing (saving) images. All of the popular image formats, such as BMP, PNG, JPEG, and TIFF are supported, along with several more esoteric formats. Check the Supported Formats docs for a list of all formats. Matplotlib provides a large collection of plotting utilities.

Let us examine a simple Python program to load, display, and save an image to a different format. Here are the first few lines:

%matplotlib widget

First, we import the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

Next, we will do something with the image:

%matplotlib widget

Once we have the image in the program, we first call

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

Now, we will save the image in another format:

%matplotlib widget

The final statement in the program,

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

%matplotlib widget

Metadata, revisited

Remember, as mentioned in the previous section, images saved with

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

99 will not retain all metadata associated with the original image that was loaded into Python! If the image metadata is important to you, be sure to always keep an unchanged copy of the original image!

Extensions do not always dictate file type

The

%matplotlib widget

02 function automatically uses the file type we specify in the file name parameter’s extension. Note that this is not always the case. For example, if we are editing a document in Microsoft Word, and we save the document as

%matplotlib widget

03 instead of

%matplotlib widget

04, the file is not saved as a PDF document.

Named versus positional arguments

When we call functions in Python, there are two ways we can specify the necessary arguments. We can specify the arguments positionally, i.e., in the order the parameters appear in the function definition, or we can use named arguments.

For example, the

%matplotlib widget

02 function definition specifies two parameters, the resource to save the image to (e.g., a file name, an http address) and the image to write to disk. So, we could save the chair image in the sample code above using positional arguments like this:

%matplotlib widget

06

Since the function expects the first argument to be the file name, there is no confusion about what

%matplotlib widget

07 means. The same goes for the second argument.

The style we will use in this workshop is to name each argument, like this:

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

96

This style will make it easier for you to learn how to use the variety of functions we will cover in this workshop.

Resizing an image (10 min)

Add

%matplotlib widget

09 and

%matplotlib widget

10 to your list of imports. Using the

%matplotlib widget

11 image located in the data folder, write a Python script to read your image into a variable named

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

93. Then, resize the image to 10 percent of its current size using these lines of code:

%matplotlib widget

5

As it is used here, the parameters to the

%matplotlib widget

13 function are the image to transform,

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

93, the dimensions we want the new image to have,

%matplotlib widget

15.

Note that the pixel values in the new image are an approximation of the original values and should not be confused with actual, observed data. This is because

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

32 interpolates the pixel values when reducing or increasing the size of an image.

%matplotlib widget

17 has a number of optional parameters that allow the user to control this interpolation. You can find more details in the .

Image files on disk are normally stored as whole numbers for space efficiency, but transformations and other math operations often result in conversion to floating point numbers. Using the

%matplotlib widget

18 method converts it back to whole numbers before we save it back to disk. If we don’t convert it before saving,

%matplotlib widget

02 may not recognise it as image data.

Next, write the resized image out to a new file named

%matplotlib widget

20 in your data directory. Finally, use

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

95 with each of your image variables to display both images in your notebook. Don’t forget to use

%matplotlib widget

22 so you don’t overwrite the first image with the second. Images may appear the same size in jupyter, but you can see the size difference by comparing the scales for each. You can also see the differnce in file storage size on disk by hovering your mouse cursor over the original and the new file in the jupyter file browser, using

%matplotlib widget

23 in your shell, or the OS file browser if it is configured to show file sizes.

Solution

Here is what your Python script might look like.

%matplotlib widget

6

The script resizes the

%matplotlib widget

24 image by a factor of 10 in both dimensions, saves the result to the

%matplotlib widget

25 file, and displays original and resized for comparision.

Manipulating pixels

In the Image Basics episode, we individually manipulated the colours of pixels by changing the numbers stored in the image’s NumPy array. Let’s apply the principles learned there along with some new principles to a real world example.

Suppose we are interested in this maize root cluster image. We want to be able to focus our program’s attention on the roots themselves, while ignoring the black background.

Since the image is stored as an array of numbers, we can simply look through the array for pixel colour values that are less than some threshold value. This process is called thresholding, and we will see more powerful methods to perform the thresholding task in the Thresholding episode. Here, though, we will look at a simple and elegant NumPy method for thresholding. Let us develop a program that keeps only the pixel colour values in an image that have value greater than or equal to 128. This will keep the pixels that are brighter than half of “full brightness”, i.e., pixels that do not belong to the black background. We will start by reading the image and displaying it.

%matplotlib widget

Now we can threshold the image and display the result.

%matplotlib widget

The NumPy command to ignore all low-intensity pixels is

%matplotlib widget

Converting colour images to grayscale

It is often easier to work with grayscale images, which have a single channel, instead of colour images, which have three channels. Skimage offers the function

%matplotlib widget

%matplotlib widget

Colour and %matplotlib widget 29

The Carpentries generally prefers UK English spelling, which is why we use “colour” in the explanatory text of this lesson. However,

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

32 contains many modules and functions that include the US English spelling,

%matplotlib widget

29. The exact spelling matters here, e.g. you will encounter an error if you try to run

%matplotlib widget

32. To account for this, we will use the US English spelling,

%matplotlib widget

29, in example Python code throughout the lesson. You will encounter a similar approach with “centre” and

%matplotlib widget

34.

%matplotlib widget

We can also load colour images as grayscale directly by passing the argument

%matplotlib widget

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

Keeping only low intensity pixels (10 min)

A little earlier, we showed how we could use Python and skimage to turn on only the high intensity pixels from an image, while turning all the low intensity pixels off. Now, you can practice doing the opposite - keeping all the low intensity pixels while changing the high intensity ones.

The file

%matplotlib widget

37 is an RGB image of a sudoku puzzle:

Your task is to turn all of the white pixels in the image to a light gray colour, say with the intensity of each formerly white pixel set to 0.75. The results should look like this:

Hint: this is an instance where it is helpful to convert the image from RGB to grayscale.

Solution

First, load the image file in and convert it to grayscale:

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

1

Then, change all high intensity pixel values (> 0.75) to 0.75:

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

2

Finally, display modified image:

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

3

Plotting single channel images (cmap, vmin, vmax)

Compared to a colour image, a grayscale image contains only a single intensity value per pixel. When we plot such an image with

%matplotlib widget

38, matplotlib uses a colour map, to assign each intensity value a colour. The default colour map is called “viridis” and maps low values to purple and high values to yellow. We can instruct matplotlib to map low values to black and high values to white instead, by calling

%matplotlib widget

38 with

%matplotlib widget

40. The documentation contains an overview of pre-defined colour maps.

Furthermore, matplotlib determines the minimum and maximum values of the colour map dynamically from the image, by default. That means, that in an image, where the minimum is 0.25 and the maximum is 0.75, those values will be mapped to black and white respectively (and not dark gray and light gray as you might expect). If there are defined minimum and maximum vales, you can specify them via

%matplotlib widget

41 and

%matplotlib widget

42 to get the desired output.

If you forget about this, it can lead to unexpected results. Try removing the

%matplotlib widget

42 parameter from the sudoku challenge solution and see what happens.

Access via slicing

As noted in the previous lesson skimage images are stored as NumPy arrays, so we can use array slicing to select rectangular areas of an image. Then, we can save the selection as a new image, change the pixels in the image, and so on. It is important to remember that coordinates are specified in (ry, cx) order and that colour values are specified in (r, g, b) order when doing these manipulations.

Consider this image of a whiteboard, and suppose that we want to create a sub-image with just the portion that says “odd + even = odd,” along with the red box that is drawn around the words.

Using the same display technique we have used throughout this course, we can determine the coordinates of the corners of the area we wish to extract by hovering the mouse near the points of interest and noting the coordinates. If we do that, we might settle on a rectangular area with an upper-left coordinate of (135, 60) and a lower-right coordinate of (480, 150), as shown in this version of the whiteboard picture:

Note that the coordinates in the preceding image are specified in (cx, ry) order. Now if our entire whiteboard image is stored as an skimage image named

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

%matplotlib widget

Our array slicing specifies the range of y-coordinates or rows first,

%matplotlib widget

%matplotlib widget

%matplotlib widget

A script to create the subimage would start by loading the image:

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

Then we use array slicing to create a new image with our selected area and then display the new image.

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

We can also change the values in an image, as shown next.

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

First, we sample a single pixel’s colour at a particular location of the image, saving it in a variable named

%matplotlib widget

%matplotlib widget

%matplotlib widget

Practicing with slices (10 min - optional, not included in timing)

Using the techniques you just learned, write a script that creates, displays, and saves a sub-image containing only the plant and its roots from “data/maize-root-cluster.jpg”

Solution

Here is the completed Python program to select only the plant and roots in the image.

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

7

Key Points

• Images are read from disk with the

  import skimage                 # form 1, load whole skimage library
  import skimage.draw            # form 2, load skimage.draw module only
  from skimage.draw import disk  # form 3, load only the disk function
  import numpy as np             # form 4, load all of numpy into an object called np
  

  62 function.

• We create a window that automatically scales the displayed image with matplotlib and calling

  %matplotlib widget
  

  53 on the global figure object.

• Colour images can be transformed to grayscale using

  %matplotlib widget
  

  27 or, in many cases, be read as grayscale directly by passing the argument

  %matplotlib widget
  

  35 to

  import skimage                 # form 1, load whole skimage library
  import skimage.draw            # form 2, load skimage.draw module only
  from skimage.draw import disk  # form 3, load only the disk function
  import numpy as np             # form 4, load all of numpy into an object called np
  

  62.

• We can resize images with the

  %matplotlib widget
  

  13 function.

• NumPy array commands, such as

  %matplotlib widget
  

  26, can be used to manipulate the pixels of an image.

• Array slicing can be used to extract sub-images or modify areas of images, e.g.,

  %matplotlib widget
  

  59.

• Metadata is not retained when images are loaded as skimage images.

Overview

Teaching: 45 min
Exercises: 45 min

Questions

• How can we draw on skimage images and use bitwise operations and masks to select certain parts of an image?

Objectives

• Create a blank, black skimage image.

• Draw rectangles and other shapes on skimage images.

• Explain how a white shape on a black background can be used as a mask to select specific parts of an image.

• Use bitwise operations to apply a mask to an image.

The next series of episodes covers a basic toolkit of skimage operators. With these tools, we will be able to create programs to perform simple analyses of images based on changes in colour or shape.

Drawing on images

Often we wish to select only a portion of an image to analyze, and ignore the rest. Creating a rectangular sub-image with slicing, as we did in the Image Representation in skimage episode is one option for simple cases. Another option is to create another special image, of the same size as the original, with white pixels indicating the region to save and black pixels everywhere else. Such an image is called a mask. In preparing a mask, we sometimes need to be able to draw a shape - a circle or a rectangle, say - on a black image. skimage provides tools to do that.

Consider this image of maize seedlings:

Now, suppose we want to analyze only the area of the image containing the roots themselves; we do not care to look at the kernels, or anything else about the plants. Further, we wish to exclude the frame of the container holding the seedlings as well. Hovering over the image with our mouse, could tell us that the upper-left coordinate of the sub-area we are interested in is (44, 357), while the lower-right coordinate is (720, 740). These coordinates are shown in (x, y) order.

A Python program to create a mask to select only that area of the image would start with a now-familiar section of code to open and display the original image:

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

As before, we first import the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

NumPy allows indexing of images/arrays with “boolean” arrays of the same size. Indexing with a boolean array is also called mask indexing. The “pixels” in such a mask array can only take two values:

%matplotlib widget

%matplotlib widget

%matplotlib widget

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

The first argument to the

%matplotlib widget

%matplotlib widget

%matplotlib widget

%matplotlib widget

%matplotlib widget

%matplotlib widget

%matplotlib widget

%matplotlib widget

Next, we draw a filled, rectangle on the mask:

print(image.shape)
print(image)

Here is what our constructed mask looks like:

The parameters of the

%matplotlib widget

%matplotlib widget

%matplotlib widget

%matplotlib widget

%matplotlib widget

%matplotlib widget

%matplotlib widget

Check the documentation!

When using an skimage function for the first time - or the fifth time - it is wise to check how the function is used, via the skimage documentation or other usage examples on programming-related sites such as Stack Overflow. Basic information about skimage functions can be found interactively in Python, via commands like

%matplotlib widget

83 or

%matplotlib widget

84. Take notes in your lab notebook. And, it is always wise to run some test code to verify that the functions your program uses are behaving in the manner you intend.

Variable naming conventions!

You may have wondered why we called the return values of the rectangle function

%matplotlib widget

81 and

%matplotlib widget

82?! You may have guessed that

%matplotlib widget

87 is short for

%matplotlib widget

88 and

%matplotlib widget

89 is short for

%matplotlib widget

90. However, the rectangle function returns mutiple rows and columns; thus we used a convention of doubling the letter

%matplotlib widget

87 to

%matplotlib widget

81 (and

%matplotlib widget

89 to

%matplotlib widget

82) to indicate that those are multiple values. In fact it may have even been clearer to name those variables

%matplotlib widget

95 and

%matplotlib widget

96; however this would have been also much longer. Whatever you decide to do, try to stick to some already existing conventions, such that it is easier for other people to understand your code.

Other drawing operations (15 min)

There are other functions for drawing on images, in addition to the

%matplotlib widget

97 function. We can draw circles, lines, text, and other shapes as well. These drawing functions may be useful later on, to help annotate images that our programs produce. Practice some of these functions here.

Circles can be drawn with the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

38 function, which takes two parameters: the (ry, cx) point of the centre of the circle, and the radius of the circle. There is an optional

%matplotlib widget

99 parameter that can be supplied to this function. It will limit the output coordinates for cases where the circle dimensions exceed the ones of the image.

Lines can be drawn with the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

00 function, which takes four parameters: the (ry, cx) coordinate of one end of the line, and the (ry, cx) coordinate of the other end of the line.

Other drawing functions supported by skimage can be found in .

First let’s make an empty, black image with a size of 800x600 pixels:

print(image.shape)
print(image)

1

Now your task is to draw some other coloured shapes and lines on the image, perhaps something like this:

Solution

Drawing a circle:

print(image.shape)
print(image)

2

Drawing a line:

print(image.shape)
print(image)

3

print(image.shape)
print(image)

4

We could expand this solution, if we wanted, to draw rectangles, circles and lines at random positions within our black canvas. To do this, we could use the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

01 python module, and the function

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

02, which can produce random numbers within a certain range.

Let’s draw 15 randomly placed circles:

print(image.shape)
print(image)

5

We could expand this even further to also randomly choose whether to plot a rectangle, a circle, or a square. Again, we do this with the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

01 module, now using the function

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

04 that returns a random number between 0.0 and 1.0.

print(image.shape)
print(image)

6

Image modification

All that remains is the task of modifying the image using our mask in such a way that the areas with

%matplotlib widget

How does a mask work? (optional, not included in timing)

Now, consider the mask image we created above. The values of the mask that corresponds to the portion of the image we are interested in are all

%matplotlib widget

66, while the values of the mask that corresponds to the portion of the image we want to remove are all

%matplotlib widget

65.

How do we change the original image using the mask?

Solution

When indexing the image using the mask, we access only those pixels at positions where the mask is

%matplotlib widget

65. So, when indexing with the mask, one can set those values to 0, and effectively remove them from the image.

Now we can write a Python program to use a mask to retain only the portions of our maize roots image that actually contains the seedling roots. We load the original image and create the mask in the same way as before:

print(image.shape)
print(image)

Then, we use numpy indexing to remove the portions of the image, where the mask is

%matplotlib widget

print(image.shape)
print(image)

Then, we display the masked image.

print(image.shape)
print(image)

The resulting masked image should look like this:

Masking an image of your own (optional, not included in timing)

Now, it is your turn to practice. Using your mobile phone, tablet, webcam, or digital camera, take an image of an object with a simple overall geometric shape (think rectangular or circular). Copy that image to your computer, write some code to make a mask, and apply it to select the part of the image containing your object. For example, here is an image of a remote control:

And, here is the end result of a program masking out everything but the remote:

Solution

Here is a Python program to produce the cropped remote control image shown above. Of course, your program should be tailored to your image.

(5, 3)
[[0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

0

Masking a 96-well plate image (30 min)

Consider this image of a 96-well plate that has been scanned on a flatbed scanner.

(5, 3)
[[0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

1

Suppose that we are interested in the colours of the solutions in each of the wells. We do not care about the colour of the rest of the image, i.e., the plastic that makes up the well plate itself.

Your task is to write some code that will produce a mask that will mask out everything except for the wells. To help with this, you should use the text file

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

10 that contains the (cx, ry) coordinates of the centre of each of the 96 wells in this image. You may assume that each of the wells has a radius of 16 pixels.

Your program should produce output that looks like this:

Solution

(5, 3)
[[0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

2

Masking a 96-well plate image, take two (optional, not included in timing)

If you spent some time looking at the contents of the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

10 file from the previous challenge, you may have noticed that the centres of each well in the image are very regular. Assuming that the images are scanned in such a way that the wells are always in the same place, and that the image is perfectly oriented (i.e., it does not slant one way or another), we could produce our well plate mask without having to read in the coordinates of the centres of each well. Assume that the centre of the upper left well in the image is at location cx = 91 and ry = 108, and that there are 70 pixels between each centre in the cx dimension and 72 pixels between each centre in the ry dimension. Each well still has a radius of 16 pixels. Write a Python program that produces the same output image as in the previous challenge, but without having to read in the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

12 file. Hint: use nested for loops.

Solution

Here is a Python program that is able to create the masked image without having to read in the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

12 file.

(5, 3)
[[0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

3

Key Points

• We can use the NumPy

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  14 function to create a blank, black image.

• We can draw on skimage images with functions such as

  %matplotlib widget
  

  97,

  import skimage                 # form 1, load whole skimage library
  import skimage.draw            # form 2, load skimage.draw module only
  from skimage.draw import disk  # form 3, load only the disk function
  import numpy as np             # form 4, load all of numpy into an object called np
  

  38,

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  00, and more.

• The drawing functions return indices to pixels that can be set directly.

Overview

Teaching: 40 min
Exercises: 40 min

Questions

• How can we create grayscale and colour histograms to understand the distribution of colour values in an image?

Objectives

• Explain what a histogram is.

• Load an image in grayscale format.

• Create and display grayscale and colour histograms for entire images.

• Create and display grayscale and colour histograms for certain areas of images, via masks.

In this episode, we will learn how to use skimage functions to create and display histograms for images.

Introduction to Histograms

As it pertains to images, a histogram is a graphical representation showing how frequently various colour values occur in the image. We saw in the Image Basics episode that we could use a histogram to visualise the differences in uncompressed and compressed image formats. If your project involves detecting colour changes between images, histograms will prove to be very useful, and histograms are also quite handy as a preparatory step before performing thresholding.

Grayscale Histograms

We will start with grayscale images, and then move on to colour images. We will use this image of a plant seedling as an example:

Here we load the image in grayscale instead of full colour, and display it:

(5, 3)
[[0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

Again, we use the

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

%matplotlib widget

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

Then, we convert the grayscale image of integer dtype, with 0-255 range, into a floating-point one with 0-1 range, by calling the function

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

We now use the function

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

(5, 3)
[[0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

The parameter

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

The parameter

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

The first output of the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

Next, we turn our attention to displaying the histogram, by taking advantage of the plotting facilities of the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

(5, 3)
[[0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

We create the plot with

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

Variable-length argument lists

Note that we cannot used named parameters for the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

40 or

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

41 functions. This is because these functions are defined to take an arbitrary number of unnamed arguments. The designers wrote the functions this way because they are very versatile, and creating named parameters for all of the possible ways to use them would be complicated.

Finally, we create the histogram plot itself with

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

Histograms in matplotlib

Matplotlib provides a dedicated function to compute and display histograms:

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

44. We will not use it in this lesson in order to understand how to calculate histograms in more detail. In practice, it is a good idea to use this function, because it visualises histograms more appropriately than

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

41. Here, you could use it by calling

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

46 instead of

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

47 and

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

41 (

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

49 is a numpy function that converts our two-dimensional image into a one-dimensional array).

Using a mask for a histogram (15 min)

Looking at the histogram above, you will notice that there is a large number of very dark pixels, as indicated in the chart by the spike around the grayscale value 0.12. That is not so surprising, since the original image is mostly black background. What if we want to focus more closely on the leaf of the seedling? That is where a mask enters the picture!

First, hover over the plant seedling image with your mouse to determine the (x, y) coordinates of a bounding box around the leaf of the seedling. Then, using techniques from the Drawing and Bitwise Operations episode, create a mask with a white rectangle covering that bounding box.

After you have created the mask, apply it to the input image before passing it to the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

28 function.

Solution

(5, 3)
[[0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

7

Your histogram of the masked area should look something like this:

Colour Histograms

We can also create histograms for full colour images, in addition to grayscale histograms. We have seen colour histograms before, in the Image Basics episode. A program to create colour histograms starts in a familiar way:

(5, 3)
[[0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

We read the original image, now in full colour, and display it.

Next, we create the histogram, by calling the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

(5, 3)
[[0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

We will draw the histogram line for each channel in a different colour, and so we create a tuple of the colours to use for the three lines with the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

line of code. Then, we limit the range of the x-axis with the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

Next, we use the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

The Python built-in

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

Iterators, tuples, and image = iio.imread(uri="data/eight.tif") plt.imshow(image) 57

In Python, an iterator, or an iterable object, is something that can be iterated over with the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

55 control structure. A tuple is a sequence of objects, just like a list. However, a tuple cannot be changed, and a tuple is indicated by parentheses instead of square brackets. The

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

57 function takes an iterable object, and returns an iterator of tuples consisting of the 0-based index and the corresponding object.

For example, consider this small Python program:

zero = iio.imread(uri="data/eight.tif")
zero[2,1]= 1.0
"""
The follwing line of code creates a new figure for imshow to use in displaying our output. Without it, plt.imshow() would overwrite our previous image in the cell above
"""
fig, ax = plt.subplots()
plt.imshow(zero)
print(zero)

0

Executing this program would produce the following output:

zero = iio.imread(uri="data/eight.tif")
zero[2,1]= 1.0
"""
The follwing line of code creates a new figure for imshow to use in displaying our output. Without it, plt.imshow() would overwrite our previous image in the cell above
"""
fig, ax = plt.subplots()
plt.imshow(zero)
print(zero)

1

In our colour histogram program, we are using a tuple,

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

%matplotlib widget

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

%matplotlib widget

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

Inside the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

function call, and then add a histogram line of the correct colour to the plot with the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

function call. Note the use of our loop variables,

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

%matplotlib widget

Finally we label our axes and display the histogram, shown here:

Colour histogram with a mask (25 min)

We can also apply a mask to the images we apply the colour histogram process to, in the same way we did for grayscale histograms. Consider this image of a well plate, where various chemical sensors have been applied to water and various concentrations of hydrochloric acid and sodium hydroxide:

zero = iio.imread(uri="data/eight.tif")
zero[2,1]= 1.0
"""
The follwing line of code creates a new figure for imshow to use in displaying our output. Without it, plt.imshow() would overwrite our previous image in the cell above
"""
fig, ax = plt.subplots()
plt.imshow(zero)
print(zero)

2

Suppose we are interested in the colour histogram of one of the sensors in the well plate image, specifically, the seventh well from the left in the topmost row, which shows Erythrosin B reacting with water.

Hover over the image with your mouse to find the centre of that well and the radius (in pixels) of the well. Then create a circular mask to select only the desired well. Then, use that mask to apply the colour histogram operation to that well.

Your masked image should look like this:

And, the program should produce a colour histogram that looks like this:

Solution

zero = iio.imread(uri="data/eight.tif")
zero[2,1]= 1.0
"""
The follwing line of code creates a new figure for imshow to use in displaying our output. Without it, plt.imshow() would overwrite our previous image in the cell above
"""
fig, ax = plt.subplots()
plt.imshow(zero)
print(zero)

3

Key Points

• In many cases, we can load images in grayscale by passing the

  %matplotlib widget
  

  35 argument to the

  import skimage                 # form 1, load whole skimage library
  import skimage.draw            # form 2, load skimage.draw module only
  from skimage.draw import disk  # form 3, load only the disk function
  import numpy as np             # form 4, load all of numpy into an object called np
  

  62 function.

• We can create histograms of images with the

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  28 function.

• We can separate the RGB channels of an image using slicing operations.

• We can display histograms using the

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  79

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  80,

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  81,

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  82,

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  83,

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  84,

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  85, and

  %matplotlib widget
  

  53 functions.

Overview

Teaching: 35 min
Exercises: 25 min

Questions

• How can we apply a low-pass blurring filter to an image?

Objectives

• Explain why applying a low-pass blurring filter to an image is beneficial.

• Apply a Gaussian blur filter to an image using skimage.

In this episode, we will learn how to use skimage functions to blur images.

When processing an image, we are often interested in identifying objects represented within it so that we can perform some further analysis of these objects e.g. by counting them, measuring their sizes, etc. An important concept associated with the identification of objects in an image is that of edges: the lines that represent a transition from one group of similar pixels in the image to another different group. One example of an edge is the pixels that represent the boundaries of an object in an image, where the background of the image ends and the object begins.

When we blur an image, we make the colour transition from one side of an edge in the image to another smooth rather than sudden. The effect is to average out rapid changes in pixel intensity. A blur is a very common operation we need to perform before other tasks such as thresholding. There are several different blurring functions in the

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

Filters

In the day-to-day, macroscopic world, we have physical filters which separate out objects by size. A filter with small holes allows only small objects through, leaving larger objects behind. This is a good analogy for image filters. A high-pass filter will retain the smaller details in an image, filtering out the larger ones. A low-pass filter retains the larger features, analogous to what’s left behind by a physical filter mesh. High- and low-pass, here, refer to high and low spatial frequencies in the image. Details associated with high spatial frequencies are small, a lot of these features would fit across an image. Features associated with low spatial frequencies are large - maybe a couple of big features per image.

Blurring

Blurring is to make something less clear or distinct. This could be interpreted quite broadly in the context of image analysis - anything that reduces or distorts the detail of an image might apply. Applying a low pass filter, which removes detail occurring at high spatial frequencies, is perceived as a blurring effect. A Gaussian blur is a filter that makes use of a Gaussian kernel.

Kernels

A kernel can be used to implement a filter on an image. A kernel, in this context, is a small matrix which is combined with the image using a mathematical technique: convolution. Different sizes, shapes and contents of kernel produce different effects. The kernel can be thought of as a little image in itself, and will favour features of a similar size and shape in the main image. On convolution with an image, a big, blobby kernel will retain big, blobby, low spatial frequency features.

Gaussian blur

Consider this image of a cat, in particular the area of the image outlined by the white square.

Now, zoom in on the area of the cat’s eye, as shown in the left-hand image below. When we apply a filter, we consider each pixel in the image, one at a time. In this example, the pixel we are currently working on is highlighted in red, as shown in the right-hand image.

When we apply a filter, we consider rectangular groups of pixels surrounding each pixel in the image, in turn. The kernel is another group of pixels (a separate matrix / small image), of the same dimensions as the rectangular group of pixels in the image, that moves along with the pixel being worked on by the filter. The width and height of the kernel must be an odd number, so that the pixel being worked on is always in its centre. In the example shown above, the kernel is square, with a dimension of seven pixels.

To apply the kernel to the current pixel, an average of the the colour values of the pixels surrounding it is calculated, weighted by the values in the kernel. In a Gaussian blur, the pixels nearest the centre of the kernel are given more weight than those far away from the centre. The rate at which this weight diminishes is determined by a Gaussian function, hence the name Gaussian blur.

A Gaussian function maps random variables into a normal distribution or “Bell Curve”.

The shape of the function is described by a mean value μ, and a variance value σ². The mean determines the central point of the bell curve on the x axis, and the variance describes the spread of the curve.

In fact, when using Gaussian functions in Gaussian blurring, we use a 2D Gaussian function to account for X and Y dimensions, but the same rules apply. The mean μ is always 0, and represents the middle of the 2D kernel. Increasing values of σ² in either dimension increases the amount of blurring in that dimension.

The averaging is done on a channel-by-channel basis, and the average channel values become the new value for the pixel in the filtered image. Larger kernels have more values factored into the average, and this implies that a larger kernel will blur the image more than a smaller kernel.

To get an idea of how this works, consider this plot of the two-dimensional Gaussian function:

Imagine that plot laid over the kernel for the Gaussian blur filter. The height of the plot corresponds to the weight given to the underlying pixel in the kernel. I.e., the pixels close to the centre become more important to the filtered pixel colour than the pixels close to the outer limits of the kernel. The shape of the Gaussian function is controlled via its standard deviation, or sigma. A large sigma value results in a flatter shape, while a smaller sigma value results in a more pronounced peak. The mathematics involved in the Gaussian blur filter are not quite that simple, but this explanation gives you the basic idea.

To illustrate the blur process, consider the blue channel colour values from the seven-by-seven region of the cat image above:

The filter is going to determine the new blue channel value for the centre pixel – the one that currently has the value 86. The filter calculates a weighted average of all the blue channel values in the kernel giving higher weight to the pixels near the centre of the kernel.

This weighted average, the sum of the multiplications, becomes the new value for the centre pixel (3, 3). The same process would be used to determine the green and red channel values, and then the kernel would be moved over to apply the filter to the next pixel in the image.

Image edges

Something different needs to happen for pixels near the outer limits of the image, since the kernel for the filter may be partially off the image. For example, what happens when the filter is applied to the upper-left pixel of the image? Here are the blue channel pixel values for the upper-left pixel of the cat image, again assuming a seven-by-seven kernel:

zero = iio.imread(uri="data/eight.tif")
zero[2,1]= 1.0
"""
The follwing line of code creates a new figure for imshow to use in displaying our output. Without it, plt.imshow() would overwrite our previous image in the cell above
"""
fig, ax = plt.subplots()
plt.imshow(zero)
print(zero)

4

The upper-left pixel is the one with value 4. Since the pixel is at the upper-left corner, there are no pixels underneath much of the kernel; here, this is represented by x’s. So, what does the filter do in that situation?

The default mode is to fill in the nearest pixel value from the image. For each of the missing x’s the image value closest to the x is used. If we fill in a few of the missing pixels, you will see how this works:

zero = iio.imread(uri="data/eight.tif")
zero[2,1]= 1.0
"""
The follwing line of code creates a new figure for imshow to use in displaying our output. Without it, plt.imshow() would overwrite our previous image in the cell above
"""
fig, ax = plt.subplots()
plt.imshow(zero)
print(zero)

5

Another strategy to fill those missing values is to reflect the pixels that are in the image to fill in for the pixels that are missing from the kernel.

zero = iio.imread(uri="data/eight.tif")
zero[2,1]= 1.0
"""
The follwing line of code creates a new figure for imshow to use in displaying our output. Without it, plt.imshow() would overwrite our previous image in the cell above
"""
fig, ax = plt.subplots()
plt.imshow(zero)
print(zero)

6

A similar process would be used to fill in all of the other missing pixels from the kernel. Other border modes are available; you can learn more about them in the skimage documentation.

This animation shows how the blur kernel moves along in the original image in order to calculate the colour channel values for the blurred image.

skimage has built-in functions to perform blurring for us, so we do not have to perform all of these mathematical operations ourselves. Let’s work through an example of blurring an image with the skimage Gaussian blur function.

First, we load the image, and display it:

zero = iio.imread(uri="data/eight.tif")
zero[2,1]= 1.0
"""
The follwing line of code creates a new figure for imshow to use in displaying our output. Without it, plt.imshow() would overwrite our previous image in the cell above
"""
fig, ax = plt.subplots()
plt.imshow(zero)
print(zero)

Next, we apply the gaussian blur:

zero = iio.imread(uri="data/eight.tif")
zero[2,1]= 1.0
"""
The follwing line of code creates a new figure for imshow to use in displaying our output. Without it, plt.imshow() would overwrite our previous image in the cell above
"""
fig, ax = plt.subplots()
plt.imshow(zero)
print(zero)

The first two parameters to

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

The last parameter to

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

Finally, we display the blurred image:

zero = iio.imread(uri="data/eight.tif")
zero[2,1]= 1.0
"""
The follwing line of code creates a new figure for imshow to use in displaying our output. Without it, plt.imshow() would overwrite our previous image in the cell above
"""
fig, ax = plt.subplots()
plt.imshow(zero)
print(zero)

Experimenting with sigma values (10 min)

The size and shape of the kernel used to blur an image can have a significant effect on the result of the blurring and any downstream analysis carried out on the blurred image. The next two exercises ask you to experiment with the sigma values of the kernel, which is a good way to develop your understanding of how the choice of kernel can influence the result of blurring.

First, try running the code above with a range of smaller and larger sigma values. Generally speaking, what effect does the sigma value have on the blurred image?

Solution

Generally speaking, the larger the sigma value, the more blurry the result. A larger sigma will tend to get rid of more noise in the image, which will help for other operations we will cover soon, such as thresholding. However, a larger sigma also tends to eliminate some of the detail from the image. So, we must strike a balance with the sigma value used for blur filters.

Experimenting with kernel shape (10 min - optional, not included in timing)

Now, what is the effect of applying an asymmetric kernel to blurring an image? Try running the code above with different sigmas in the ry and cx direction. For example, a sigma of 1.0 in the ry direction, and 6.0 in the cx direction.

Solution

[[0. 0. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

0

These unequal sigma values produce a kernel that is rectangular instead of square. The result is an image that is much more blurred in the x direction than the y direction. For most use cases, a uniform blurring effect is desirable and this kind of asymmetric blurring should be avoided. However, it can be helpful in specific circumstances e.g. when noise is present in your image in a particular pattern or orientation, such as vertical lines, or when you want to remove uniform noise without blurring edges present in the image in a particular orientation.

Other methods of blurring

The Gaussian blur is a way to apply a low-pass filter in skimage. It is often used to remove Gaussian (i. e., random) noise from the image. For other kinds of noise, e.g. “salt and pepper”, a median filter is typically used. See for a list of available filters.

Key Points

• Applying a low-pass blurring filter smooths edges and removes noise from an image.

• Blurring is often used as a first step before we perform thresholding or edge detection.

• The Gaussian blur can be applied to an image with the

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  88 function.

• Larger sigma values may remove more noise, but they will also remove detail from an image.

Overview

Teaching: 60 min
Exercises: 50 min

Questions

• How can we use thresholding to produce a binary image?

Objectives

• Explain what thresholding is and how it can be used.

• Use histograms to determine appropriate threshold values to use for the thresholding process.

• Apply simple, fixed-level binary thresholding to an image.

• Explain the difference between using the operator

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  99 or the operator

  print(image.shape)
  print(image)
  

  00 to threshold an image represented by a numpy array.

• Describe the shape of a binary image produced by thresholding via

  image = iio.imread(uri="data/eight.tif")
  plt.imshow(image)
  

  99 or

  print(image.shape)
  print(image)
  

  00.

• Explain when Otsu’s method for automatic thresholding is appropriate.

• Apply automatic thresholding to an image using Otsu’s method.

• Use the

  print(image.shape)
  print(image)
  

  03 function to count the number of non-zero pixels in an image.

In this episode, we will learn how to use skimage functions to apply thresholding to an image. Thresholding is a type of image segmentation, where we change the pixels of an image to make the image easier to analyze. In thresholding, we convert an image from colour or grayscale into a binary image, i.e., one that is simply black and white. Most frequently, we use thresholding as a way to select areas of interest of an image, while ignoring the parts we are not concerned with. We have already done some simple thresholding, in the “Manipulating pixels” section of the Image Representation in skimage episode. In that case, we used a simple NumPy array manipulation to separate the pixels belonging to the root system of a plant from the black background. In this episode, we will learn how to use skimage functions to perform thresholding. Then, we will use the masks returned by these functions to select the parts of an image we are interested in.

Simple thresholding

Consider the image

print(image.shape)
print(image)

[[0. 0. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

Now suppose we want to select only the shapes from the image. In other words, we want to leave the pixels belonging to the shapes “on,” while turning the rest of the pixels “off,” by setting their colour channel values to zeros. The skimage library has several different methods of thresholding. We will start with the simplest version, which involves an important step of human input. Specifically, in this simple, fixed-level thresholding, we have to provide a threshold value

print(image.shape)
print(image)

The process works like this. First, we will load the original image, convert it to grayscale, and de-noise it as in the Blurring Images episode.

[[0. 0. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

Next, we would like to apply the threshold

print(image.shape)
print(image)

print(image.shape)
print(image)

print(image.shape)
print(image)

print(image.shape)
print(image)

The histogram for the shapes image shown above can be produced as in the Creating Histograms episode.

[[0. 0. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

Since the image has a white background, most of the pixels in the image are white. This corresponds nicely to what we see in the histogram: there is a peak near the value of 1.0. If we want to select the shapes and not the background, we want to turn off the white background pixels, while leaving the pixels for the shapes turned on. So, we should choose a value of

print(image.shape)
print(image)

print(image.shape)
print(image)

To apply the threshold

print(image.shape)
print(image)

print(image.shape)
print(image)

print(image.shape)
print(image)

print(image.shape)
print(image)

print(image.shape)
print(image)

%matplotlib widget

%matplotlib widget

%matplotlib widget

[[0. 0. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

You can see that the areas where the shapes were in the original area are now white, while the rest of the mask image is black.

What makes a good threshold?

As is often the case, the answer to this question is “it depends”. In the example above, we could have just switched off all the white background pixels by choosing

print(image.shape)
print(image)

20, but this would leave us with some background noise in the mask image. On the other hand, if we choose too low a value for the threshold, we could lose some of the shapes that are too bright. You can experiment with the threshold by re-running the above code lines with different values for

print(image.shape)
print(image)

05. In practice, it is a matter of domain knowledge and experience to interpret the peaks in the histogram so to determine an appropriate threshold. The process often involves trial and error, which is a drawback of the simple thresholding method. Below we will introduce automatic thresholding, which uses a quantitative, mathematical definition for a good threshold that allows us to determine the value of

print(image.shape)
print(image)

05 automatically. It is worth noting that the principle for simple and automatic thresholding can also be used for images with pixel ranges other than [0.0, 1.0]. For example, we could perform thresholding on pixel intensity values in the range [0, 255] as we have already seen in the Image Representation in skimage episode.

We can now apply the

print(image.shape)
print(image)

[[0. 0. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

More practice with simple thresholding (15 min)

Now, it is your turn to practice. Suppose we want to use simple thresholding to select only the coloured shapes (in this particular case we consider grayish to be a colour, too) from the image

print(image.shape)
print(image)

24:

First, plot the grayscale histogram as in the Creating Histogram episode and examine the distribution of grayscale values in the image. What do you think would be a good value for the threshold

print(image.shape)
print(image)

05?

Solution

The histogram for the

print(image.shape)
print(image)

24 image can be shown with

[[0. 0. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

6

We can see a large spike around 0.3, and a smaller spike around 0.7. The spike near 0.3 represents the darker background, so it seems like a value close to

print(image.shape)
print(image)

27 would be a good choice.

Next, create a mask to turn the pixels above the threshold

print(image.shape)
print(image)

05 on and pixels below the threshold

print(image.shape)
print(image)

05 off. Note that unlike the image with a white background we used above, here the peak for the background colour is at a lower gray level than the shapes. Therefore, change the comparison operator less

print(image.shape)
print(image)

00 to greater

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

99 to create the appropriate mask. Then apply the mask to the image and view the thresholded image. If everything works as it should, your output should show only the coloured shapes on a black background.

Solution

Here are the commands to create and view the binary mask

[[0. 0. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

7

And here are the commands to apply the mask and view the thresholded image

[[0. 0. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

8

Automatic thresholding

The downside of the simple thresholding technique is that we have to make an educated guess about the threshold

print(image.shape)
print(image)

Denoising an image before thresholding

In practice, it is often necessary to denoise the image before thresholding, which can be done with one of the methods from the Blurring Images episode.

Consider the image

print(image.shape)
print(image)

[[0. 0. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 1. 0.]
 [0. 0. 0.]]

We use Gaussian blur with a sigma of 1.0 to denoise the root image. Let us look at the grayscale histogram of the denoised image.

five = iio.imread(uri="data/eight.tif")
five[1,2]= 1.0
five[3,0]= 1.0
fig, ax = plt.subplots()
plt.imshow(five)
print(five)

The histogram has a significant peak around 0.2, and a second, smaller peak very near 1.0. Thus, this image is a good candidate for thresholding with Otsu’s method. The mathematical details of how this works are complicated (see if you are interested), but the outcome is that Otsu’s method finds a threshold value between the two peaks of a grayscale histogram.

The

print(image.shape)
print(image)

print(image.shape)
print(image)

five = iio.imread(uri="data/eight.tif")
five[1,2]= 1.0
five[3,0]= 1.0
fig, ax = plt.subplots()
plt.imshow(five)
print(five)

five = iio.imread(uri="data/eight.tif")
five[1,2]= 1.0
five[3,0]= 1.0
fig, ax = plt.subplots()
plt.imshow(five)
print(five)

For this root image and a Gaussian blur with the chosen sigma of 1.0, the computed threshold value is 0.42. No we can create a binary mask with the comparison operator

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

five = iio.imread(uri="data/eight.tif")
five[1,2]= 1.0
five[3,0]= 1.0
fig, ax = plt.subplots()
plt.imshow(five)
print(five)

Finally, we use the mask to select the foreground:

five = iio.imread(uri="data/eight.tif")
five[1,2]= 1.0
five[3,0]= 1.0
fig, ax = plt.subplots()
plt.imshow(five)
print(five)

Application: measuring root mass

Let us now turn to an application where we can apply thresholding and other techniques we have learned to this point. Consider these four maize root system images, which you can find in the files

print(image.shape)
print(image)

print(image.shape)
print(image)

print(image.shape)
print(image)

print(image.shape)
print(image)

Suppose we are interested in the amount of plant material in each image, and in particular how that amount changes from image to image. Perhaps the images represent the growth of the plant over time, or perhaps the images show four different maize varieties at the same phase of their growth. The question we would like to answer is, “how much root mass is in each image?”

We will first construct a Python program to measure this value for a single image. Our strategy will be this:

1. Read the image, converting it to grayscale as it is read. For this application we do not need the colour image.
2. Blur the image.
3. Use Otsu’s method of thresholding to create a binary image, where the pixels that were part of the maize plant are white, and everything else is black.
4. Save the binary image so it can be examined later.
5. Count the white pixels in the binary image, and divide by the number of pixels in the image. This ratio will be a measure of the root mass of the plant in the image.
6. Output the name of the image processed and the root mass ratio.

Our intent is to perform these steps and produce the numeric result - a measure of the root mass in the image - without human intervention. Implementing the steps within a Python function will enable us to call this function for different images.

Here is a Python function that implements this root-mass-measuring strategy. Since the function is intended to produce numeric output without human interaction, it does not display any of the images. Almost all of the commands should be familiar, and in fact, it may seem simpler than the code we have worked on thus far, because we are not displaying any of the images.

five = iio.imread(uri="data/eight.tif")
five[1,2]= 1.0
five[3,0]= 1.0
fig, ax = plt.subplots()
plt.imshow(five)
print(five)

The function begins with reading the original image from the file

print(image.shape)
print(image)

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

%matplotlib widget

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

print(image.shape)
print(image)

The final part of the function determines the root mass ratio in the image. Recall that in the

print(image.shape)
print(image)

print(image.shape)
print(image)

print(image.shape)
print(image)

print(image.shape)
print(image)

We can call this function with any filename and provide a sigma value for the blurring. If no sigma value is provided, the default value 1.0 will be used. For example, for the file

print(image.shape)
print(image)

five = iio.imread(uri="data/eight.tif")
five[1,2]= 1.0
five[3,0]= 1.0
fig, ax = plt.subplots()
plt.imshow(five)
print(five)

five = iio.imread(uri="data/eight.tif")
five[1,2]= 1.0
five[3,0]= 1.0
fig, ax = plt.subplots()
plt.imshow(five)
print(five)

Now we can use the function to process the series of four images shown above. In a real-world scientific situation, there might be dozens, hundreds, or even thousands of images to process. To save us the tedium of calling the function for each image by hand, we can write a loop that processes all files automatically. The following code block assumes that the files are located in the same directory and the filenames all start with the trial- prefix and end with the .jpg suffix.

five = iio.imread(uri="data/eight.tif")
five[1,2]= 1.0
five[3,0]= 1.0
fig, ax = plt.subplots()
plt.imshow(five)
print(five)

five = iio.imread(uri="data/eight.tif")
five[1,2]= 1.0
five[3,0]= 1.0
fig, ax = plt.subplots()
plt.imshow(five)
print(five)

Ignoring more of the images – brainstorming (10 min)

Let us take a closer look at the binary masks produced by the

print(image.shape)
print(image)

51 function.

You may have noticed in the section on automatic thresholding that the thresholded image does include regions of the image aside of the plant root: the numbered labels and the white circles in each image are preserved during the thresholding, because their grayscale values are above the threshold. Therefore, our calculated root mass ratios include the white pixels of the label and white circle that are not part of the plant root. Those extra pixels affect how accurate the root mass calculation is!

How might we remove the labels and circles before calculating the ratio, so that our results are more accurate? Think about some options given what we have learned so far.

Solution

One approach we might take is to try to completely mask out a region from each image, particularly, the area containing the white circle and the numbered label. If we had coordinates for a rectangular area on the image that contained the circle and the label, we could mask the area out easily by using techniques we learned in the Drawing and Bitwise Operations episode.

However, a closer inspection of the binary images raises some issues with that approach. Since the roots are not always constrained to a certain area in the image, and since the circles and labels are in different locations each time, we would have difficulties coming up with a single rectangle that would work for every image. We could create a different masking rectangle for each image, but that is not a practicable approach if we have hundreds or thousands of images to process.

Another approach we could take is to apply two thresholding steps to the image. Look at the graylevel histogram of the file

print(image.shape)
print(image)

37 shown above again: Notice the peak near 1.0? Recall that a grayscale value of 1.0 corresponds to white pixels: the peak corresponds to the white label and circle. So, we could use simple binary thresholding to mask the white circle and label from the image, and then we could use Otsu’s method to select the pixels in the plant portion of the image.

Note that most of this extra work in processing the image could have been avoided during the experimental design stage, with some careful consideration of how the resulting images would be used. For example, all of the following measures could have made the images easier to process, by helping us predict and/or detect where the label is in the image and subsequently mask it from further processing:

• Using labels with a consistent size and shape
• Placing all the labels in the same position, relative to the sample
• Using a non-white label, with non-black writing

Ignoring more of the images – implementation (30 min - optional, not included in timing)

Implement an enhanced version of the function

print(image.shape)
print(image)

51 that applies simple binary thresholding to remove the white circle and label from the image before applying Otsu’s method.

Solution

We can apply a simple binary thresholding with a threshold

print(image.shape)
print(image)

54 to remove the label and circle from the image. We use the binary mask to set the pixels in the blurred image to zero (black).

[[0. 0. 0.]
 [0. 1. 1.]
 [0. 0. 0.]
 [1. 1. 0.]
 [0. 0. 0.]]

0

The output of the improved program does illustrate that the white circles and labels were skewing our root mass ratios:

[[0. 0. 0.]
 [0. 1. 1.]
 [0. 0. 0.]
 [1. 1. 0.]
 [0. 0. 0.]]

1

Here are the binary images produced by the additional thresholding. Note that we have not completely removed the offending white pixels. Outlines still remain. However, we have reduced the number of extraneous pixels, which should make the output more accurate.

Thresholding a bacteria colony image (15 min)

In the images directory

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

74, you will find an image named

print(image.shape)
print(image)

56.

This is one of the images you will be working with in the morphometric challenge at the end of the workshop.

1. Plot and inspect the grayscale histogram of the image to determine a good threshold value for the image.
2. Create a binary mask that leaves the pixels in the bacteria colonies “on” while turning the rest of the pixels in the image “off”.

Solution

Here is the code to create the grayscale histogram:

[[0. 0. 0.]
 [0. 1. 1.]
 [0. 0. 0.]
 [1. 1. 0.]
 [0. 0. 0.]]

2

The peak near one corresponds to the white image background, and the broader peak around 0.5 corresponds to the yellow/brown culture medium in the dish. The small peak near zero is what we are after: the dark bacteria colonies. A reasonable choice thus might be to leave pixels below

print(image.shape)
print(image)

57 on.

Here is the code to create and show the binarized image using the

print(image.shape)
print(image)

00 operator with a threshold

print(image.shape)
print(image)

57:

[[0. 0. 0.]
 [0. 1. 1.]
 [0. 0. 0.]
 [1. 1. 0.]
 [0. 0. 0.]]

3

When you experiment with the threshold a bit, you can see that in particular the size of the bacteria colony near the edge of the dish in the top right is affected by the choice of the threshold.

Key Points

• Thresholding produces a binary image, where all pixels with intensities above (or below) a threshold value are turned on, while all other pixels are turned off.

• The binary images produced by thresholding are held in two-dimensional NumPy arrays, since they have only one colour value channel. They are boolean, hence they contain the values 0 (off) and 1 (on).

• Thresholding can be used to create masks that select only the interesting parts of an image, or as the first step before edge detection or finding contours.

Overview

Teaching: 70 min
Exercises: 55 min

Questions

• How to extract separate objects from an image and describe these objects quantitatively.

Objectives

• Understand the term object in the context of images.

• Learn about pixel connectivity.

• Learn how Connected Component Analysis (CCA) works.

• Use CCA to produce an image that highlights every object in a different colour.

• Characterise each object with numbers that describe its appearance.

Objects

In the Thresholding episode we have covered dividing an image into foreground and background pixels. In the shapes example image, we considered the coloured shapes as foreground objects on a white background.

In thresholding we went from the original image to this version:

Here, we created a mask that only highlights the parts of the image that we find interesting, the objects. All objects have pixel value of

%matplotlib widget

%matplotlib widget

By looking at the mask image, one can count the objects that are present in the image (7). But how did we actually do that, how did we decide which lump of pixels constitutes a single object?

Pixel Neighborhoods

In order to decide which pixels belong to the same object, one can exploit their neighborhood: pixels that are directly next to each other and belong to the foreground class can be considered to belong to the same object.

Let’s discuss the concept of pixel neighborhoods in more detail. Consider the following mask “image” with 8 rows, and 8 columns. For the purpose of illustration, the digit

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

print(image.shape)
print(image)

[[0. 0. 0.]
 [0. 1. 1.]
 [0. 0. 0.]
 [1. 1. 0.]
 [0. 0. 0.]]

The pixels are organised in a rectangular grid. In order to understand pixel neighborhoods we will introduce the concept of “jumps” between pixels. The jumps follow two rules: First rule is that one jump is only allowed along the column, or the row. Diagonal jumps are not allowed. So, from a centre pixel, denoted with

print(image.shape)
print(image)

%matplotlib widget

[[0. 0. 0.]
 [0. 1. 1.]
 [0. 0. 0.]
 [1. 1. 0.]
 [0. 0. 0.]]

The pixels on the diagonal (from

print(image.shape)
print(image)

print(image.shape)
print(image)

The second rule states that in a sequence of jumps, one may only jump in row and column direction once -> they have to be orthogonal. An example of a sequence of orthogonal jumps is shown below. Starting from

print(image.shape)
print(image)

[[0. 0. 0.]
 [0. 1. 1.]
 [0. 0. 0.]
 [1. 1. 0.]
 [0. 0. 0.]]

All pixels reachable with one, or two jumps form the 2-jump neighborhood. The grid below illustrates the pixels reachable from the centre pixel

print(image.shape)
print(image)

%matplotlib widget

image = iio.imread(uri="data/eight.tif")
plt.imshow(image)

[[0. 0. 0.]
 [0. 1. 1.]
 [0. 0. 0.]
 [1. 1. 0.]
 [0. 0. 0.]]

We want to revisit our example image mask from above and apply the two different neighborhood rules. With a single jump connectivity for each pixel, we get two resulting objects, highlighted in the image with

print(image.shape)
print(image)

print(image.shape)
print(image)

[[0. 0. 0.]
 [0. 1. 1.]
 [0. 0. 0.]
 [1. 1. 0.]
 [0. 0. 0.]]

In the 1-jump version, only pixels that have direct neighbors along rows or columns are considered connected. Diagonal connections are not included in the 1-jump neighborhood. With two jumps, however, we only get a single object

print(image.shape)
print(image)

[[0. 0. 0.]
 [0. 1. 1.]
 [0. 0. 0.]
 [1. 1. 0.]
 [0. 0. 0.]]

Object counting (optional, not included in timing)

How many objects with 1 orthogonal jump, how many with 2 orthogonal jumps?

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

00

1 jump

a) 1 b) 5 c) 2

Solution

b) 5

2 jumps

a) 2 b) 3 c) 5

Solution

a) 2

Jumps and neighborhoods

We have just introduced how you can reach different neighboring pixels by performing one or more orthogonal jumps. We have used the terms 1-jump and 2-jump neighborhood. There is also a different way of referring to these neighborhoods: the 4- and 8-neighborhood. With a single jump you can reach four pixels from a given starting pixel. Hence, the 1-jump neighborhood corresponds to the 4-neighborhood. When two orthogonal jumps are allowed, eight pixels can be reached, so the 2-jump neighborhood corresponds to the 8-neighborhood.

Connected Component Analysis

In order to find the objects in an image, we want to employ an operation that is called Connected Component Analysis (CCA). This operation takes a binary image as an input. Usually, the

%matplotlib widget

%matplotlib widget

print(image.shape)
print(image)

print(image.shape)
print(image)

import skimage                 # form 1, load whole skimage library
import skimage.draw            # form 2, load skimage.draw module only
from skimage.draw import disk  # form 3, load only the disk function
import numpy as np             # form 4, load all of numpy into an object called np

Note the new import of

print(image.shape)
print(image)

print(image.shape)
print(image)