When you start the new report, choose Mailing Label Report instead of blank report. After adding your tables, the first option is which mailing label template (row/columns). You have the option there of customizing the width and height of the label, so try something like 4 width and 2.25 height. If you are not getting 2 x 4 labels, go into the section expert, highlight the higher order details section (above detail_a)->layout tab and adjust the height and width there.
-LB