The structure of thermal convection in horizontal plane Couette flow is investigated. Numerical experiments show that transformation of the structure takes place in the linear stage of perturbation growth. In the non-rotating or slowly rotating case, the transformation from cellular to parallel roll convection occurs. In the rapidly rotating case, on the other hand, the transformation from cellular to parallel roll convection occurs via transverse roll convection. As a result, transverse roll convection of finite amplitude can be formed in the rapidly rotating frame of reference. The formation mechanism of the transverse roll is investigated by linear analysis. In both the non-rotating and rotating cases, the horizontal velocity shear temporarily accelerates the growth rate of the transverse mode for a relatively short period. In the non-rotating or slowly rotating case, however, the net effect of this horizontal velocity shear is small. In the rapidly rotating case, on the other hand, the shearing of the perturbation by the background flow results in greatly enhancing potential energy conversion of the transverse mode. As a result, the net effect becomes large enough to make the transverse mode dominate over the parallel mode. Although the transverse roll convection of finite amplitude appears for a relatively short period of time just after its onset, the present result might explain some aspects of observed transverse rolls, such as in the Jovian atmosphere.