3DTV service compatible frame packing format from Sisvel

Here detailed informations about the Sisvel Technology‘s system to embed 3D and 2D images in the same channel.
Frame packing systems are the key technology that will be adopted in the first generation of 3DTV broadcasting services.
Squeezing the Left and Right images in a single High Oefinition frame allows the service provider to reuse part of the existing production infrastructure and the whole of the existing distribution infrastructure. In most cases, the set top boxes already deployed at the user’s premises for HO services can also be reused.

Unfortunately, the frame packing requires subsampling/decimation of the source images to make them squeezable in half the size of a HO frame and, depending on the chosen technique, this might cause two different adverse consequences:

• in the case of vertical or horizontal subsampling, the image resolution in the two directions will be different, causing a degradation of the perceived image quality

• if the applied subsampling technique is staggered subsampling or quincunx, the balance between horizontal and vertical resolution is preserved, at the price of a loss of diagonal resolution. However, in order to produce the composite image, the staggered pixels need to be “aligned” giving rise to a loss of correlation (the edges look somewhat jagged), and consequently the following encoding stage may generate compression artefacts and/or a higher bit rate is required in order to obtain a satisfactory quality.

Moreover, due to the geometrical layout of the composite frame, that contains both the left and the right views, the resulting video will not be service (backwards) compatible: legacy HO decoders will be able to decompress the video, but 2D TV-sets will display the composite picture and not, as it would be desirable, one of the two composing images (L or R).

A new frame packing system

To overcome the above problems, we have devised a new solution, allowing the storage of two 720p frames in a single 1080p frame: in this case, if the L and R pictures are originated in the 720p format, no decimation is needed and the reconstructed L and R pictures will preserve their original resolution and will not suffer from the unbalance of the V and H resolution.

Moreover, since one of the two images remains unaltered, the resulting format can also be made service compatible (see below).

As illustrated in the figure, the first 720p frame (L, in the example) can be inserted unaltered in the 1080p container.

The second (R) will have to be cut in slices, so as to fit the space left free in the container frame


The three slices Rl, R2 and R3 can be inserted in the container frame as illustrated in the following figure


Experimental results prove that, by choosing properly the way to cut the second frame in slices, the following phase of compression will not cause any relevant artefacts.
Service compatibiIity
A proper management of H.264 encoding parameters enables the deployment of a service compatible frame packing format.

One of the metadata encoded in the H.264 bitstream is the so called “cropping rectangle”, used to signal to the decoder the part of the decoded frame that has to be output to the display. This parameter was introduced because the encoder is able to process images having a vertical and horizontal size multiple of 16 pixels (the size of a macroblock), while 1080 cannot be divided bV 16: thus the cropping rectangle has been defined to cut out a 1920 x 1080 image from the 1920 x 1088 used bV the encoder.

Making the above described frame packing format is straightforward: as illustrated in the picture, a 1280 x 720 cropping rectangle (highlighted in light red) is applied on the unaltered frame.

A legacy decoder, while receiving an H.264 encoded 30 signal would display only the area of the frame enclosed by the cropping rectangle

In other words, existing H.264 decoders, without any change of the firmware, can provide a 2D compatible picture, while they need a very simple firmware upgrade (through downloading) in order to be able to feed a 30 display.

ConclusionsThe usage of the above described format, along with in-band 3D-Switch signalling or H.264 SEI messages provides the following advantages:

• if the original L and R pictures are of the 720p format, there is no loss of resolution both for the 3D picture and for the compatible 2D picture

• the unbalancing between V and H resolution caused by the known frame packing formats is avoided

• service compatibility is assured, namely an H.264 HO legacy decoder, without any software change, can feed a 2D display

• a very simple software patch needs to be downloaded into the existing H.264 HD legacy decoders in order to feed a 3D displav with the proper pictures.