Volume rendering in unity and CustusX overview
VOLUME RENDERING IN UNITY AND CUSTUSX OVERVIEW
SVTH: Văn Đức Khá nh, Lê Thị Phương Thảo
GVHD: ThS Nguyễn Huy Anh
Tó m tắt - Mục đích của bài bá o này là cung cấp thô ng tin và tổng quan ban đầu về kết xuất khối
lượng trong sự thống nhất và trên phần mềm CustusX.
Abstract - The purpose of this paper is to provide information and an initial overview of volume
rendering in unity and on CustusX software.
I.
INTRODUCTION
1.1 What is volume rendering?
Volume rendering represents a collection of methods used in computer graphics and
scientific visualization to create a 2D projection from a discretely sampled 3D data set.
An example of a 3D data set is a collection of MRI, CT, or MicroCT scanner 2D slice
images. For instance, a series of 2D slice images of a human brain can be assembled to render
3D volume rendered images using a volume rendering algorithm.
Volume rendering and the data visualization it enables allows experts to understand
medical data from CAT or MRI scanners, complex fluid dynamics (example: flow and
pressure of blood), data from seismic events, and other volumetric data for which geometric
surfaces are unavailable—or just too difficult or cost-prohibitive to generate. Volume
visualization provides a way to parse that complex data, and reveal intricate 3D relationships.
Volume rendering is mostly used in medical applications but it is also used in car part
manufacturing and scientific rendering: chemistry, entomology, etc.
1.2 Volume rendering technique
There is more than one volume rendering technique, and the correct one for your
application depends on several factors. 3D volume rendering methods can be grouped into
four categories: “ray casting or raymarching”, “resampling or shear-warp”, “texture
slicing”, and “splatting”. However, I will focus on raycasting and texture slicing because
these two techniques are common and relate to our following research.
Direct volume rendering ray casting is typically achieved as follows. Starting with the
camera, the user moves into and through the volume, gathering data on color, density, light,
and gradient at each point. The end result is a slightly cloudy image that communicates details
about the surveyed portion of the field. This direct volume rendering is generally better for
visualizing softer data sets that represent density variations, flows fields, and related things.
318
Picture 1: Ray-casting example
For more distinct boundaries and structures, such as anything from neighborhoods to
bones, it may be more useful to examine volume data using other volume rendering
techniques such as 3D slicer volume rendering. In this technique, the user slices through the
volume to acquire the right information at each point.
Picture 2: example of slice of skull on 3D Slicer
II.
Method of approach Volume Rendering
Volume Rendering in UNITY
2.1
Volume rendering can operate on unity thanks to the working of Matis Lavik, reference
Lavik has implemented a project on volume rendering on unity with basic and useful features.
This is an interesting project, aimed at research and learning materials for us.
319
The project has the following functions:
Data import to Unity: raw data, Dicom, image sequence
You can freely move, rotate objects to observe
Select the object you will have 3 render modes, this is how the object will be visual,
Direct Volume Rendering (using transfer functions)
Picture 3: Direct Volume Rendering
Direct volume rendering is the most standard rendering mode. It sends rays through the
dataset, and uses "transfer functions" (1D or 2D) to determine the color and opacity.
Picture 4: 1D transfer functions
320
Picture 5: 2D transfer functions
Maximum Intensity Projection (shows the maximum density)
The object will be displayed as black and white
Picture 6: Maximum Intensity Projection
Isosurface Rendering: a closed surface separates “outside” from “inside”. The boundary
between “outside” and “inside” is the isosurface
321
Picture 7: Isosurface rendering
In 3 render modes, you can move the slider to increase or decrease visible value
Slice rendering: using slicing plane, when you move plane up/down, right/left, it wil
visualize the right information at each point.
322
2.2 Volume rendering on CustusX
1. CustusX overview
CustusX is not certified for normal clinical use.
CustusX is a product developed for the purpose of image-guided therapy for pre-surgery
and ultrasound. The software's volume rendering technique is slice rendering, giving the
doctor an overview of the patient's part that needs to be treated. The software is equipped with
many functions for projecting images so that the most visually easy to see, can be seen as
hight light by the color of the parts by transfer function, the software uses 3 slicer (x y z) at
once The doctor observes more aspects of the patient than the part of the patient. and many
other visibility possibilities.
Image tracking registration function: this is a function that allows users to merge 2
volumes together and view them in relationship. one volume will be fixed and 1 volume will
move. After registering the landmark in both volumes, an algorithm will merge the 2 volumes
to match the registered landmarks. This help user see things that previously impossible to see.
Before
After
Tracking function: custus introduces the use of a physical tracking device to enter the
patient's body to locate lesions such as cancerous tumors, which can also be seen in pre-
operative CT images is displayed on the screen.
323
CustusX can display video from an external source using a frame grabber.
CustusX can be used to record images from an ultrasound probe, and create 3D volumes
based on this recording.
III. Implementation
3.1. User Interface and Function of Volume Rendering in Unity
In this project, Volume Rendering in Unity used to show the abdomen of human. It
should be implemented in reality such as hopital; however, we are student, so this application
only executed in normal environment. The same types of this project are applied for surgery
in order that the doctor can observe the organs of object. This thing will assist the doctor in
diagnosing problems of patient.
The following picture presents the user interface of application.
324
The screen display eight buttons with different functions and the model.
Threshold button using for control the magnitude or intensity that must be exceeded for
a certain reaction, phenomenon, result, or condition to occur or be manifested.
Intensity button using for control the measurable amount of a property namely
brightness.
X: Min using for slice model from right to left, contrary to X: Max.
Y: Min using for slice model from back to front, contrary to Y: Max.
Z: Min using for slice model from top to bottom, contrary to Z: Max.
Volume Rendering in Unity allow user to rotate, zoom and turn the model flexibly.
The below video shows the model when it sliced from front:
The below video shows the model when it sliced from one side:
325
The below video shows the model when it sliced from bottom:
The below video shows the model when it sliced from top:
326
3.2 User Interface and Function of CustusX
This is main window of CustusX application:
The main window presents the main features of CustusX.
The View Area display visualizations of volumes and models. It consists of several Views,
each of which can visualize either 2D, 3D or video data.
327
The 3D views show data in Sagittal, Axial, Coronal or in a plane defined by the active
tool. Sagittal relating to or denoting the suture on top of the skull which runs between the
parietal bones in a front to back direction. Axial forming or relating to an axis. Coronal
relating to the crown of the head.
Each View belong to a View Group and all Views in a View Group display the same
data.
Widgets containing the bulk of functionality in CustusX.
The Workflow can be broken down into a series of steps:
• Preoperative data acquisition
• Planning
• Registration
• Navigation + Intraoperative data acquisition
• Postoperative analysis
Each step contains its own widget and layout configuration.
This video implemented on laptop, displaying CustusX application after imported
DICOM.
The CustusX allows the user to rotate model in all side by moving cursor.
Using ACS, the user can slice the model in order to view structure of it.
328
IV. Comparison between Volume Rendering in Unity and CustusX
Volume Rendering in Unity
CustusX
Using a main window user interface
Simple interface, model and the based on the Qt QMainWindow class,
functional buttons are which provides menus, toolbars,
UI
represented on the unity screen configurable widgets, and a central view
area.
The speed to load data of 2 application is not significant, however, CustusX
seem faster a bit
Performance
329
• Create vertex data
(array of vertices)
• Create vertex buffer
(send vertices to GPU)
• Bind vertex buffer and
index buffer, and draw
Using a layered architecture. The external
libraries contribute to much of the
functionality,with CustusX being an
integrating platform.
Render
Platform
Unity
CTK/OSGi framework
• Create a complete navigation
system.
• Containing a toolbox of
navigation features, image-
processing algorithms,
An Introduction of volume
rendering technique, set of
datas
Functions
connections to external hardware
and other functionality useful for
working with IGT.
More complexity with more functions,
beside using slice rendering to visualize
Do not have more functions
like CustusX, the project
provide techniques and data
type are used in volume
rendering, that brings an
overview to someone new to
this field of study
model,
functions such as: image landmark, video
streaming, ultrasound acquisition,
CustusX
provide
another
Complexity
tracking, and import DICOM and many
other small functions that help users to
see more clearly
VR in Unity is for research and CustusX is an application for research in
study, It can be implemented to image-guided therapy with a focus on
view object in real world that Intraoperative use and ultrasound
Applicability
bring realistic study to user
imaging.
• Interval between renderings
• Max Render Size: Volumes are
downsampled to this size before rendered
in 3D (2D still shows slices through
original data).
• Smart Render: Render only once per
second if there is no changes in the scene.
• 2D Overlay: Enable multiple volumes
in 2D (GPU based multi slicer, can only
handle up to 4 images).
• Lighting
• Gradient
• Colour
Parameters
affecting the
performance
• Opacity
• Optimized views: Speedup by
merging all 2D views into a single
vtkRenderWindow
• Still Update Rate: Property in
330
vtkRenderWindow. Increasing this value
may improve rendering speed at the cost
of quality.
• 3D Renderer: Select method for
rendering images in 3D
• Direct volume rendering
• Maximum intensity
project
Mode
• Iosurface rendering
We have presented about the definition, feature, function and applicability of Volume
Rendering in Unity and CustusX. In general, using CustusX to realized Volume Rendering
will provide the user with many respects as regards this field. Therefore, the user will
understand obviously essence of Volume Rendering. Nevertheless, by combinating Unity
with Volume Rendering, the users able to apply it to real environment so that they can
practice, study and assist doctor in examining.
REFERENCES
1. Christian Askeland·Ole Vegard Solberg·Janne Beate Lervik Bakeng·Ingerid
Reinertsen·Geir Arne Tangen·Erlend Fagertun Hofstad·Daniel Høyer Iversen·Cecilie
Våpenstad·Tormod Selbekk·Thomas Langø·Toril A. Nagelhus Hernes·Håkon Olav
Leira·Geirmund
Unsgård·Frank
Lindseth
[accessed Apr 25 2021].CustusX: an open-source research platform for image-guidedtherapy
2. Matias Lavik (2020) Volume Rendering in Unity. Available:
April, 2021)
4.
‘Image
registration’
(2021).
Wikipedia.
Available
at:
331
14 trang
27000
Free
Bạn đang xem tài liệu "Volume rendering in unity and CustusX overview", để tải tài liệu gốc về máy hãy click vào nút Download ở trên
File đính kèm:
- volume_rendering_in_unity_and_custusx_overview.pdf
