Credit cards, deposits, loans, pension funds, mutual funds–which of these products are relevant to a bank's clients, and at what time in their banking journey? We propose a modeling framework for item recommendation using a multi-head self-attentive encoder and a novel sequential input data representation accounting for the temporal context of both item ownership and user metadata. We evaluate our model on a large public dataset from Santander, and achieve a top-1 and top-5 precision of 98.9% and 40.2%, respectively, thereby improving upon a number of state-of-the-art models. Further, we consider serendipity, novelty and coverage to exhibit a trade-off with recommendation relevance. The continuous user representation learned by our model may inform decisions far more impactful than the recommendations themselves.