We will first investigate texture warping further with emphasis on automation and on techniques suitable for subsequent texture fusion. One solution to create a photorealistic view is to project the view pixel by pixel on the object and then trace the projected area on the images in order to get the gray level for the view pixel. If there is some knowledge about the camera calibration, the distortions can be compensated in the tracing process. For the tracing purpose, we need to know the orientations of the images relative to the object. The orientations can be determined using the traditional photogrammetric techniques or using an interactive orientation process, where the model is projected on the image and then moved and rotated so that it ``fits'' to the image. There are several things that should be taken into account when determining the gray level of the pixel. The traced area has different size and shape on different images, different images have different orientations relative to the light sources, the traced area may be in a dead angle on some images, and so on. The result will depend on the accuracy of the orientations and the rules used for determining the ``real'' gray level.
More and more detailed radiometric information should be attached onto the object surface in proportion as the geometry of the model becomes more detailed. This information is obtained from the video images by segmenting an appropriate area and we would thus need a fast, reliable, and automatic segmentation algorithm to do it. Our idea is that since we usually know the relative orientations of the images with respect to the object model, these orientations could be directly used to determine the area to be segmented in the image space. The automatization of this algorithm may be difficult, however.