question about camera intrinsic

[meidachen]

Hi,

Thank you so much for sharing this amazing work!! I have a quick question about the camera intrinsics that are being used when rendering. Do you assume the c_x and c_y are always at the center of the image (width / 2, height / 2)?

Thank you!

Meida Chen

[kwea123]

yeah currently they use projection matrix which assumes it's always the center, but in fact you can get rid of that and instead use $K$ which allows any cx cy. But you have to derive the gradient by yourself (not difficult at all).

[vairleon]

yeah currently they use projection matrix which assumes it's always the center, but in fact you can get rid of that and instead use K which allows any cx cy. But you have to derive the gradient by yourself (not difficult at all).

I would like to ask if I want to add cx and cy, which derivatives of the variables need to be modified? I can see from the code that the derivation process does not seem to require any special modifications. Is there anything I haven't noticed?

[Octweiyi]

yeah currently they use projection matrix which assumes it's always the center, but in fact you can get rid of that and instead use K which allows any cx cy. But you have to derive the gradient by yourself (not difficult at all).

I would like to ask if I want to add cx and cy, which derivatives of the variables need to be modified? I can see from the code that the derivation process does not seem to require any special modifications. Is there anything I haven't noticed?

yes, there is no need to modify any derivations. just modify the getProjectionMatrix by using the intrinsics K. For the details refer this link https://stackoverflow.com/questions/22064084/how-to-create-perspective-projection-matrix-given-focal-points-and-camera-princ. I hope this can help.

[kwea123]

@Octweiyi yes, you are right, I checked it and found that you can replace the projection matrix with

P[0, 0] = 2 * fx / W
P[1, 1] = 2 * fy / H
P[0, 2] = 2 * (cx / W) - 0.5
P[1, 2] = 2 * (cy / H) - 0.5
P[2, 2] = -(zfar + znear) / (zfar - znear)
P[3, 2] = 1.0
P[2, 3] = -(2 * zfar * znear) / (zfar - znear)

also zfar and znear are really irrelevant, you can set them to 0 and it doesn't affect the result

[limacv]

It feels like there is a small mistake in your equation @kwea123:

P[0, 2] = 2 * (cx / W) - 1
P[1, 2] = 2 * (cy / H) - 1

When cx = W/2, P[0, 2] should be 0 instead of 0.5 in your equation.

[cs-mshah]

I wanted to ask one more thing: Won't the camera intrinsics be scaled when the resolution is scaled? I didn't find any code which recomputes the FovX and FovY when the resolution is not 1.
Ref: https://stackoverflow.com/questions/74749690/how-will-the-camera-intrinsics-change-if-an-image-is-cropped-resized

[yifanlu0227]

@cs-mshah FovX and FovY do not need rescale because they are always there. But when you use the intrinsics K, you need to rescale it according to your new resolution.

[whu-lyh]

Hi @Octweiyi and @kwea123 why is there no need to modify the derivatives? Currently, the J is connected with the $f_x$ and $f_y$ only, but what if the $c_x$ and $c_y$ are not at the center of the image?

[Octave1990]

Hi @Octweiyi and @kwea123 why is there no need to modify the derivatives? Currently, the J is connected with the f x and f y only, but what if the c x and c y are not at the center of the image?

Image projection function in camera coordinates : u = fx * x + cx, the derivative du/dx = fx is independent with cx. For cy, the same logic applies.