IoT devices are permeating every corner of our lives today paving the road for more substantial smart systems. Despite their ability to collect and analyze a significant amount of sensory data, traditional IoT typically depends on fixed policies and schedules to enhance user experience. However, fixed policies that do not account for variations in human mood, reactions, and expectations, fail to achieve the promised user experience. In this paper, we propose an architecture for personalized and autonomous IoT systems that weaves personalization and context-awareness into the very fabric of smart systems. By building upon ideas from reinforcement learning, we show—using an example of smart and personalized home services—how the proposed architecture can adapt to human behaviors that are varying between individuals and vary, for the same individual, across time while addressing some of the security and privacy challenges.