Sharing the spectrum with in-band full-duplex (FD) primary users (PUs) is a challenging and interesting problem in the underlay cognitive radio (CR) systems. The self-interference introduced at the primary network may dramatically impede the secondary user (SU) opportunity to access the spectrum. To tackle this problem, we use the so-called improper Gaussian signaling. Particularly, we assume the downlink transmission of a SU that uses improper Gaussian signaling while the FD PU pair implements the regular proper Gaussian signaling. First, we derive a closed form expression and an upper bound for the SU and PUs outage probabilities, respectively. Second, we optimize the SU signal parameters to minimize its outage probability while maintaining the required PUs quality-of-service based on the average channel state information (CSI). Moreover, we provide the conditions to reap merits from employing improper Gaussian signaling at the SU. Third, we design the SU signal parameters based on perfect knowledge of its direct link instantaneous CSI and investigate all benefits that can be achieved at both the SU and PUs. Finally, we provide some numerical results that demonstrate the advantages of using improper Gaussian signaling to access the spectrum of the FD PUs.