Preheat oven to 350°F. Grease loaf pan and set aside.
In large bowl, stir together butter, brown sugar, cane sugar and vanilla sugar. Stir in eggs one at a time. In another bowl, mash bananas and stir in milk; stir into sugar mixture.
In large bowl, whisk together all-purpose flour, whole-wheat flour and baking powder. Fold into sugar mixture until combined; fold in chocolate and almonds. Scrape into prepared pan.
Cut rhubarb stalks to fit pan lengthwise, then press half of them into dough; place remaining stalks on top.
Bake until cake tester inserted into centre comes out clean, about 50 minutes.