from PIL import Image
import matplotlib.pyplot as plt
import numpy as np
import scipy.linalg
=3)
np.set_printoptions(precision
'figure.figsize'] = [15, 15] plt.rcParams[
= Image.open('data/2020-04-10-Finding-Vanishing-Points/csgo-dust2-mid.jpeg')
img
plt.imshow(img) plt.show()
= {'VP1': [1371.892, 630.421],
vanishing_points 'VP2': [-10651.54 , 536.681],
'VP3': [1272.225, 7683.02 ]}
plt.imshow(img)for point_name in ['VP1','VP2','VP3']:
= vanishing_points[point_name]
vp 0],vp[1],label=point_name)
plt.scatter(vp[
plt.legend() plt.show()
We now have three different vanishing points. Using these let’s try and use them to gain some insights into the cameras’ relationship with the scene.
Hartley & Zisserman (H&Z) put it best, “Vanishing points are images of points at infinity, and provide orientation (attitude) information in a similar manner to that provided by the fixed stars.”
H&Z also provide an algorithm (Example 8.27, page 226) to extract the camera calibration matrix K from 3 mutually orthogonal vanishing points.
Let’s go and implement it in practice.
Some algebra
H&Z propose a matrix
Where:
And:
If we can find this matrix
- Zero Skew
- Square Pixels
From H&Z, we have: “K is obtained from
For good practice, we can also normalize the matrix so that the lower right value
= np.linalg.inv(np.linalg.cholesky(omega)).T
K /=K[2,2] K
Working backwards, we are faced with the task of finding
Multiplying through, we find that:
Factorising:
We can now find all the coefficients we need for our matrix from each pair of vanishing points.
We have three pairs of vanishing points:
- 1 & 2
- 2 & 3
- 3 & 1
From each pair, we can find a new set of values for
Stacking them all on top of each other, we end up with the matrix
In Python:
def generate_A(vanishing_points):
= []
A for (point_name_1, point_name_2) in [('VP1','VP2'),('VP2','VP3'),('VP3','VP1')]:
= vanishing_points[point_name_1]
vp1 = vanishing_points[point_name_2]
vp2
= vp1
x1,y1 = vp2
x2,y2
= x2*x1 + y2*y1
w1 = x2 + x1
w2 = y2 + y1
w3 = 1
w4
A.append([w1,w2,w3,w4])
= np.array(A)
A return(A)
def generate_A(vanishing_points):
= []
A for (point_name_1, point_name_2) in [('VP1','VP2'),('VP2','VP3'),('VP3','VP1')]:
= vanishing_points[point_name_1]
vp1 = vanishing_points[point_name_2]
vp2
= vp1
x1,y1 = vp2
x2,y2
= x2*x1 + y2*y1
w1 = x2 + x1
w2 = y2 + y1
w3 = 1
w4
A.append([w1,w2,w3,w4])
= np.array(A)
A return(A)
def compute_K(A):
= scipy.linalg.null_space(A).ravel()
w
= w[0]
w1 = w[1]
w2 = w[2]
w3 = w[3]
w4
= np.array([[w1,0,w2],
omega 0,w1,w3],
[
[w2,w3,w4]])= np.linalg.inv(np.linalg.cholesky(omega)).T
K /=K[2,2]
Kreturn(K)
= generate_A(vanishing_points)
A = compute_K(A) K
print(K)
[[7.276e+02 0.000e+00 1.327e+03]
[6.236e-14 7.276e+02 7.060e+02]
[1.218e-16 0.000e+00 1.000e+00]]
The Calibration Matrix
So now we have the calibration matrix
- The Focal length in pixels:
or (728) - The x coordinate of the camera optical centre:
(1327) - The y coordinate of the camera optical centre
(706)
2560/2.0,1600/2.0,color='G')
plt.scatter(0,2],K[1,2],color='R')
plt.scatter(K[
plt.imshow(img) plt.show()