In recent years, tagging system has become a building block o summarize the content of items for further functions like retrieval or personalized recommendation in various web applications. One nontrivial requirement is to precisely deliver a list of suitable items when users interact with the systems via inputing a specific tag (i.e. a query term). Different from traditional recommender systems, we need deal with a collaborative retrieval (CR) problem, where both characteristics of retrieval and recommendation should be considered to model a ternary relationship involved with query× user× item. Recently, several works are proposed to study CR task from users’ perspective. However, they miss a significant challenge raising from the sparse content of items. In this work, we argue that items will suffer from the sparsity problem more severely than users, since items are usually observed with fewer features to support a feature-based or content-based algorithm. To tackle this problem, we aim to sufficiently explore the sophisticated relationship of each query× user× item triple from items’ perspective. By integrating item-based collaborative information for this joint task, we present an alternative factorized model that could better evaluate the ranks of those items with sparse information for the given query-user pair. In addition, we suggest to employ a recently proposed bayesian personalized ranking (BPR) algorithm to optimize latent collaborative retrieval problem from pairwise learning perspective. The experimental results on two real-world datasets, (i.e. Last.fm, Yelp), verified the efficiency and effectiveness of our proposed approach at top-k ranking metric.