XRay 2D-3D Registration¶
X-ray projections with their own geometries are registered to 3D CT volume. X-ray projections, i.e. cone beam projections, should be acquired from the same CT volume.
Further information can be found in our demo video “Enhance your 2D-3D Registration from CT to X-ray/Fluoro images with ImFusion Suite”.
Input¶
- One 3D CT volume.
- Two or more X-ray projections.
Output¶
Transformation of the X-ray geometry with respect to the world coordinate system of the input CT volume, i.e. the patient coordinate system.
Description¶
When you open the X-ray 2D-3D registration, these views are displayed:
- On the top-left view, 2D X-ray projections, i.e. shots, are displayed.
- 3D CT volume is displayed in slice views as usual.
- On the bottom-center view, the registration of X-ray shots to the 3D CT volume is visualized.
- On the bottom-right view, DRR (digitally reconstructed radiograph) of the first shot is displayed.
The goal is to optimize the transformation of CT volume and the iso-center of the X-ray geometries such that the difference between the DRRs and the X-ray projections will be minimized.
In order to obtain a valid registration result, it is important that the X-ray shots are calibrated correctly. SDK users can use the ConeBeamCalibration algorithm to calibrate their X-ray shots. It is also possible to provide individual projection matrices describing the X-ray geometry from Cone-Beam Geometry. Projection matrices should be in OpenCV convention and from world to pixel coordinates.
When the X-ray projections are loaded, they need to have the relative transformation between each other. This means that the system needs to be tracked with the X-ray geometries while the X-ray projections are acquired.
There are three main tabs for X-ray 2D-3D registration: Initialization, Edit X-Ray Geometry, and History.
Initialization¶
To compute the registration, the user has to initialize it by applying transformation on the X-ray geometries. You can determine this transformation manually, by editting the X-ray geometries, or by using point-based annotations in both X-ray shots and CT volume to define corresponding target points.
There are four different initialization modes:
- Manual registration initialization
- Point + direction initialization
- Key points initialization
- Custom initialization
Epipolar lines
are created with point + direction initialization and key points initialization modes.
They are drawn as blue lines on the other X-ray shots just after setting the shot annotation on each X-ray shot.
It can be used as a reference by the user while setting the shot annotations,
since it is known that the corresponding shot annotation should be set somewhere on the corresponding epipolar line.
(Re-) Initialize Registration initializes the registration algorithm before Compute. Hide/Show annotations are toggling the visibility of shot/volume annotations, epipolar lines, and target point rays.
Manual Registration Initialization¶
Manual registration mode initializes the registration by applying rigid transformation on the X-ray geometries. You can use Translate and Rotate to apply the transformation via translation and rotation widgets on the registration view. You can check the relative transformation of the CT volume by using the DRR views per shot. You can reset the transformation by using Reset.
Point + Direction Initialization¶
Point + direction initialization mode is based on user-set annotations given by two points on each shot and two points on the CT volume. Here, the shot/volume annotations are arrows. The initial point is the first shot/volume target point and the end point is the second shot/volume target point.
The initialization result is the relative transformation for the pose of CT volume and the rotation center. The rotation center becomes the first volume target point. With the relative transformation, CT volume is rotated so that the direction between the volume target points matches the direction between the shot target points on the 3D geometry. In order to find the shot target points on the 3D geometry, the shot target points on the 2D X-ray shots are backprojected to rays. Then, the shot target points on the 3D geometry are found by computing the intersection or the closest point between these target point rays.
Masks are set on the X-ray shots when the registration is initialized for the first time, and they can be updated by the user later. Masks are drawn such that the shot annotation will be inside the mask, which is also the case for the key points initialization mode.
After setting the volume annotation, you can rotate the CT volume around the arrow annotation with the rotation slider. You can set the optional volume direction annotation to specify the direction of the first shot.
Target point rays
are backprojected target points from detectors to sources, which are shown with yellow lines on the registration view.
The thicker one is for the first shot target point and the thinner one is for the second.
Epipolar lines
are created from the first shot target point of shot annotations.
Key Points Initialization¶
You can add key points by the suggested default names, which are numbers in order, or you can set custom key point names by editting the placeholder text. Key points for each X-ray shot and CT volume comes as automatically selected after addition. The Shot and Volume buttons are checkable, so you can uncheck the key points that you do not wish to use. Or, you can also skip them by right clicking. You can Reset and Remove key points.
As it was in the point + direction initialization mode, epipolar lines
are created for each shot target point.
The key point name that the epipolar lines are created from is displayed on the center of the epipolar lines.
Additionally, shot error lines
are displayed with red lines on the X-ray shots. These lines show the difference between the shot target points
and the corresponding volume target point which is forward-projected on these X-ray shots.
Target point rays
for each target point are displayed on the registration view as before.
Custom Initialization¶
For SDK users, it is possible to customize X-Ray 2D-3D registration further including the initialization. For further information, consult our public demos on Github.
Edit X-Ray Geometry¶
Optionally, a user can use this advanced section to manually manipulate the X-ray geometries to be registered.
You can edit Translation
and Rotation
parameters directly, so that you can apply rigid transformations on the X-ray geometries.
These parameters are the same parameters as the volume iso-matrix parameters under Cone-beam Geometry.
History¶
Under History, the previous registrations are saved with their ID
, Computation Phase
, and Comment
.
Computation phases are Original, Initialization, Optimized, and Unknown.
Parameters:¶
Anatomy name
: Anatomy name can be editted here.Num resolution levels
: X-ray 2D-3D registation algorithm supports for multiple resolution levels with the approach of image pyramids.Current resolution level
: The current resolution level to be shown on the DRR views.- You can select only the views inside the masks by clicking on Toggle show masks.
- You can see all of the DRRs by clicking on One DRR per shot.
- Finally, you can compute your registration by Compute after initializing the registration.