No, you don't really need to use the brass tube.
However, you may find that the combination of the brass tube and the glue that's holding it in will offer more flexibility than the aluminium.
The brass tube will stretch to accept the pressed in parts and still shrink back enough to grip them afterwards, but the aluminium may just stretch to accept them and not offer much grip later. You therefore may have to glue the parts into the aluminium to keep them there.